Author name cluster

Junfei Xiao

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models

Tiezheng Zhang
Yitong Li
Yu-Cheng Chou
Jieneng Chen
Alan Yuille
Chen Wei
Junfei Xiao

Building state-of-the-art Vision-Language Models (VLMs) with strong captioning capabilities typically necessitates training on billions of high-quality image-text pairs, requiring millions of GPU hours. This paper introduces the Vision-Language-Vision (VLV) auto-encoder framework, which strategically leverages key pretrained components: a vision encoder, the decoder of a Text-to-Image (T2I) diffusion model, and subsequently, a Large Language Model (LLM). Specifically, we establish an information bottleneck by regularizing the language representation space, achieved through freezing the pretrained T2I diffusion decoder. Our VLV pipeline effectively distills knowledge from the text-conditioned diffusion model using continuous embeddings, demonstrating comprehensive semantic understanding via high-quality reconstructions. Furthermore, by fine-tuning a pretrained LLM to decode the intermediate language representations into detailed descriptions, we construct a state-of-the-art (SoTA) captioner comparable to leading models like GPT-4o and Gemini 2. 0 Flash. Our method demonstrates exceptional cost-efficiency and significantly reduces data requirements; by primarily utilizing single-modal images for training and maximizing the utility of existing pretrained models (image encoder, T2I diffusion model, and LLM), it circumvents the need for massive paired image-text datasets, keeping the total training expenditure under $1, 000 USD.

PDF Details

ICML Conference 2025 Conference Paper

What If We Recaption Billions of Web Images with LLaMA-3?

Xianhang Li
Haoqin Tu
Mude Hui
Zeyu Wang 0008
Bingchen Zhao
Junfei Xiao
Sucheng Ren
Jieru Mei

Web-crawled image-text pairs are inherently noisy. Prior studies demonstrate that semantically aligning and enriching textual descriptions of these pairs can significantly enhance model training across various vision-language tasks, particularly text-to-image generation. However, large-scale investigations in this area remain predominantly closed-source. Our paper aims to bridge this community effort, leveraging the powerful and $\textit{open-sourced}$ LLaMA-3, a GPT-4 level LLM. Our recaptioning pipeline is simple: first, we fine-tune a LLaMA-3-8B powered LLaVA-1. 5 and then employ it to recaption 1. 3 billion images from the DataComp-1B dataset. Our empirical results confirm that this enhanced dataset, Recap-DataComp-1B, offers substantial benefits in training advanced vision-language models. For discriminative models like CLIP, we observe an average of 3. 1% enhanced zero-shot performance cross four cross-modal retrieval tasks using a mixed set of the original and our captions. For generative models like text-to-image Diffusion Transformers, the generated images exhibit a significant improvement in alignment with users’ text instructions, especially in following complex queries. Our project page is https: //www. haqtu. me/Recap-Datacomp-1B/.

Details

ICML Conference 2024 Conference Paper

Rejuvenating image-GPT as Strong Visual Representation Learners

Sucheng Ren
Zeyu Wang 0008
Hongru Zhu
Junfei Xiao
Alan L. Yuille
Cihang Xie

This paper enhances image-GPT (iGPT), one of the pioneering works that introduce autoregressive pretraining to predict the next pixels for visual representation learning. Two simple yet essential changes are made. First, we shift the prediction target from raw pixels to semantic tokens, enabling a higher-level understanding of visual content. Second, we supplement the autoregressive modeling by instructing the model to predict not only the next tokens but also the visible tokens. This pipeline is particularly effective when semantic tokens are encoded by discriminatively trained models, such as CLIP. We introduce this novel approach as D-iGPT. Extensive experiments showcase that D-iGPT excels as a strong learner of visual representations: A notable achievement is its compelling performance on the ImageNet-1K dataset — by training on publicly available datasets, D-iGPT unprecedentedly achieves 90. 0% top-1 accuracy with a vanilla ViT-H. Additionally, D-iGPT shows strong generalization on the downstream task. Code is available at https: //github. com/OliverRensu/D-iGPT.

Details

ICRA Conference 2019 Conference Paper

A bio-robotic remora disc with attachment and detachment capabilities for reversible underwater hitchhiking

Siqi Wang
Lei Li
Yufeng Chen 0003
Yueping Wang
Wenguang Sun
Junfei Xiao
Dylan K. Wainwright
Tianmiao Wang

Remoras employ their adhesive discs to rapidly attach to and detach from a wide range of marine surfaces. By analyzing high-speed images of remoras' (Echeneis naucrates) hitchhiking behavior, we describe the fish's detachment mechanism as a lip curling up to break the seal between the disc and substrate. By mimicking the kinematic and morphological properties of the biological disc, we fabricated a multi-material biomimetic disc (whose stiffness spans four orders of magnitude) that is capable of both attachment and detachment. Detachment is realized by a flexible cable-driven mechanism that curls the anterior region of the silicone soft lip, allows leakage under the disc, and equalizes the internal pressure to the external pressure. The disc lamellae with attached carbon fiber spinules can be rotated by hydraulic soft actuators whose internal pressure is precisely tuned to the ambient underwater pressure. During attachment, increasing the rotational angle of the lamellae and the preload of the disc significantly enhanced the adhesive forces. We found that curling up the soft lip and folding down the lamellae rapidly reduced the pulling force of the disc by a factor of 254 compared to that under the attached state, which lead to detachment. Based on these mechanisms, underwater maneuvers involving repeated attachment and detachment were demonstrated with an integrated ROV unit that had a self-contained actuation and control system for the disc. This study lays a foundation for the development of fully untethered robotic systems for underwater hitchhiking in real-world marine environments.

Details