Author name cluster

Jiawei Zhou

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

JBHI Journal 2026 Journal Article

Multi-Channel Temporal Interference Retinal Stimulation Based on Reinforcement Learning

Xiayu Chen
Wennan Chan
Yingqiang Meng
Runze Liu
Yueyi Yu
Sheng Hu
Jijun Han
Xiaoxiao Wang

Retinal degenerative diseases such as age-related macular degeneration and retinitis pigmentosa cause severe vision impairment, while current electrical stimulation therapies are limited by poor spatial targeting precision. As a promising non-invasive alternative, the efficacy of temporal interference stimulation (TIS) for retinal targeting depends on optimized multi-electrode parameters. This study reconstructed a whole-head finite element model with detailed ocular structures and applied reinforcement learning (RL)-based multi-channel electrode parameter optimization to retinal stimulation. Systematic evaluation demonstrated that the focal precision of TIS improves with increasing channel numbers (consistent across all subject head models), with RL significantly outperforming conventional genetic algorithms (GA) and unsupervised neural networks (USNN) in focusing capability. Furthermore, by implementing the computationally intensive envelope calculation using the JAX framework, we achieved a nearly order-of-magnitude reduction in optimization time (to approx. 2 minutes per run on an RTX 4090D), significantly enhancing the practical feasibility of the proposed RL framework. This work provides a novel and computationally efficient methodology for precise non-invasive neuromodulation parameter optimization, applicable not only to retinal diseases but potentially to broader neurological conditions.

Details DOI

NeurIPS Conference 2025 Conference Paper

Beyond Expectations: Quantile-Guided Alignment for Risk-Calibrated Language Models

Xinran Wang
Jin Du
Azal Khan
qi le
Enmao Diao
Jiawei Zhou
Jie Ding
Ali Anwar

Large language models can generate rare but catastrophic outputs, such as harmful conversations or insecure code. Existing Reinforcement Learning from Human Feedback (RLHF) typically maximizes average reward, leaving high-risk tail events insufficiently controlled. We introduce Quantile‑Guided Alignment (QA), a framework that allows users to specify desired improvements at any quantile—individually or across multiple reward dimensions—thus shifting the distribution of outputs with finer control toward safer, more desirable outcomes. The method extends standard RLHF via an augmented reward formulation that enforces quantile constraints. Experiments on conversation and code‐generation tasks show that quantile alignment significantly enhances quality at targeted tails while maintaining overall performance. The results position QA as a principled route to risk‑calibrated language models with tail‑focused alignment.

PDF Details

ICLR Conference 2025 Conference Paper

Chunk-Distilled Language Modeling

Yanhong Li
Karen Livescu
Jiawei Zhou

We introduce Chunk-Distilled Language Modeling (CD-LM), an approach to text generation that addresses two challenges in current large language models (LLMs): the inefficiency of token-level generation, and the difficulty of adapting to new data and knowledge. Our method combines deep network-based LLMs with a straightforward retrieval module, which allows the generation of multi-token text chunks at a single decoding step. Our retrieval framework enables flexible construction of model- or domain-specific datastores, either leveraging the internal knowledge of existing models, or incorporating expert insights from human-annotated corpora. This adaptability allows for enhanced control over the language model's distribution without necessitating additional training. We present the CD-LM formulation along with performance metrics demonstrating its ability to improve language model performance and efficiency across a diverse set of downstream applications. Code and data will be made publicly available.

Details

NeurIPS Conference 2025 Conference Paper

Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding

Yixiong Fang
Ziran Yang
Zhaorun Chen
Zhuokai Zhao
Jiawei Zhou

Large vision-language models (LVLMs) excel at multimodal tasks but are prone to misinterpreting visual inputs, often resulting in hallucinations and unreliable outputs. We present Dropout Decoding, a novel inference-time approach that quantifies the uncertainty of visual tokens and selectively masks uncertain tokens to improve decoding. Our method measures the uncertainty of each visual token by projecting it onto the text space and decomposing it into aleatoric and epistemic components. Specifically, we focus on epistemic uncertainty, which captures perception-related errors more effectively. Inspired by dropout regularization, we introduce uncertainty-guided token dropout, which applies the dropout principle to input visual tokens instead of model parameters, and during inference rather than training. By aggregating predictions from an ensemble of masked decoding contexts, we can robustly mitigate errors arising from visual token misinterpretations. Evaluations on benchmarks including CHAIR, THRONE, and MMBench demonstrate that Dropout Decoding significantly reduces object hallucinations (OH) and enhances both reliability and quality of LVLM outputs across diverse visual contexts.

PDF Details

NeurIPS Conference 2025 Conference Paper

MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?

Zhaorun Chen
Zichen Wen
Yichao Du
Yiyang Zhou
Chenhang Cui
Siwei Han
Jen Weng
Chaoqi Wang

While text-to-image models like GPT-4o-Image and FLUX are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, current multimodal judges frequently undergo inadequate evaluation of their capabilities and limitations, potentially leading to misalignment and unsafe fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel benchmark which incorporates a comprehensive preference dataset to evaluate multimodal judges in providing feedback for image generation models across six key perspectives: alignment, safety, image quality, bias, composition, and visualization. Specifically, we evaluate a large variety of multimodal judges including smaller-sized CLIP-based scoring models, open-source VLMs, and close-source VLMs on each decomposed subcategory of our preference dataset. Experiments reveal that close-source VLMs generally provide better feedback, with GPT-4o outperforming other judges in average. Compared with open-source VLMs, smaller-sized scoring models can provide better feedback regarding text-image alignment and image quality, while VLMs provide more accurate feedback regarding safety and generation bias due to their stronger reasoning capabilities. Further studies in feedback scale reveal that VLM judges can generally provide more accurate and stable feedback in natural language than numerical scales. Notably, human evaluations on end-to-end and fine-tuned models using separate feedback from these multimodal judges provide similar conclusions, further confirming the effectiveness of MJ-Bench.

PDF Details

NeurIPS Conference 2025 Conference Paper

Optimizing Retrieval for RAG via Reinforcement Learning

Jiawei Zhou
Lei Chen

As retrieval-augmented generation (RAG) becomes more widespread, the role of retrieval is shifting from retrieving information for human browsing to retrieving context for AI reasoning. This shift creates more complex search environments, where relevance is difficult to pre-define. Existing retrievers rely on supervised fine-tuning (SFT) with human labels or synthetic data, resulting in static relevance that struggles to adapt to diverse RAG environments. To address this challenge, we propose R3, a Retrieval framework optimized for RAG through Reinforcement learning (RL). Specifically, we adopt an RL training paradigm that enables the retriever to explore and self-improve within given RAG environments, automating the learning process with minimal manual experimentation or tuning effort. Extensive experiments across diverse tasks demonstrate that \ours improves RAG performance by 5. 2% over the original retriever and surpasses state-of-the-art retrievers by 4. 9%, while achieving comparable results to LLM-augmented retrieval and RAG systems built on post-trained or instruction-tuned LLMs. It is both efficient and practical, requiring only 4 GPUs and completing training within a single day.

PDF Details

NeurIPS Conference 2024 Conference Paper

CNCA: Toward Customizable and Natural Generation of Adversarial Camouflage for Vehicle Detectors

Linye Lyu
Jiawei Zhou
Daojing He
Yu Li

Prior works on physical adversarial camouflage against vehicle detectors mainly focus on the effectiveness and robustness of the attack. The current most successful methods optimize 3D vehicle texture at a pixel level. However, this results in conspicuous and attention-grabbing patterns in the generated camouflage, which humans can easily identify. To address this issue, we propose a Customizable and Natural Camouflage Attack (CNCA) method by leveraging an off-the-shelf pre-trained diffusion model. By sampling the optimal texture image from the diffusion model with a user-specific text prompt, our method can generate natural and customizable adversarial camouflage while maintaining high attack performance. With extensive experiments on the digital and physical worlds and user studies, the results demonstrate that our proposed method can generate significantly more natural-looking camouflage than the state-of-the-art baselines while achieving competitive attack performance.

PDF Details DOI

ICML Conference 2024 Conference Paper

HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding

Zhaorun Chen
Zhuokai Zhao
Hongyin Luo
Huaxiu Yao
Bo Li 0026
Jiawei Zhou

While large vision-language models (LVLMs) have demonstrated impressive capabilities in interpreting multi-modal contexts, they invariably suffer from object hallucinations (OH). We introduce HALC, a novel decoding algorithm designed to mitigate OH in LVLMs. HALC leverages distinct fine-grained optimal visual information in vision-language tasks and operates on both local and global contexts simultaneously. Specifically, HALC integrates a robust auto-focal grounding mechanism (locally) to correct hallucinated tokens on the fly, and a specialized beam search algorithm (globally) to significantly reduce OH while preserving text generation quality. Additionally, HALC can be integrated into any LVLMs as a plug-and-play module without extra training. Extensive experimental studies demonstrate HALC’s effectiveness in reducing OH, outperforming state-of-the-arts across four benchmarks. Code is released at https: //github. com/BillChan226/HALC.

Details

ICML Conference 2024 Conference Paper

RAUCA: A Novel Physical Adversarial Attack on Vehicle Detectors via Robust and Accurate Camouflage Generation

Jiawei Zhou
Linye Lyu
Daojing He
Yu Li 0007

Adversarial camouflage is a widely used physical attack against vehicle detectors for its superiority in multi-view attack performance. One promising approach involves using differentiable neural renderers to facilitate adversarial camouflage optimization through gradient back-propagation. However, existing methods often struggle to capture environmental characteristics during the rendering process or produce adversarial textures that can precisely map to the target vehicle, resulting in suboptimal attack performance. Moreover, these approaches neglect diverse weather conditions, reducing the efficacy of generated camouflage across varying weather scenarios. To tackle these challenges, we propose a robust and accurate camouflage generation method, namely RAUCA. The core of RAUCA is a novel neural rendering component, Neural Renderer Plus (NRP), which can accurately project vehicle textures and render images with environmental characteristics such as lighting and weather. In addition, we integrate a multi-weather dataset for camouflage generation, leveraging the NRP to enhance the attack robustness. Experimental results on six popular object detectors show that RAUCA consistently outperforms existing methods in both simulation and real-world settings.

Details

NeurIPS Conference 2024 Conference Paper

SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization

Wanhua Li
Zibin Meng
Jiawei Zhou
Donglai Wei
Chuang Gan
Hanspeter Pfister

Social relation reasoning aims to identify relation categories such as friends, spouses, and colleagues from images. While current methods adopt the paradigm of training a dedicated network end-to-end using labeled image data, they are limited in terms of generalizability and interpretability. To address these issues, we first present a simple yet well-crafted framework named SocialGPT, which combines the perception capability of Vision Foundation Models (VFMs) and the reasoning capability of Large Language Models (LLMs) within a modular framework, providing a strong baseline for social relation recognition. Specifically, we instruct VFMs to translate image content into a textual social story, and then utilize LLMs for text-based reasoning. SocialGPT introduces systematic design principles to adapt VFMs and LLMs separately and bridge their gaps. Without additional model training, it achieves competitive zero-shot results on two databases while offering interpretable answers, as LLMs can generate language-based explanations for the decisions. The manual prompt design process for LLMs at the reasoning phase is tedious and an automated prompt optimization method is desired. As we essentially convert a visual classification task into a generative task of LLMs, automatic prompt optimization encounters a unique long prompt optimization issue. To address this issue, we further propose the Greedy Segment Prompt Optimization (GSPO), which performs a greedy search by utilizing gradient information at the segment level. Experimental results show that GSPO significantly improves performance, and our method also generalizes to different image styles. The code is available at https: //github. com/Mengzibin/SocialGPT.

PDF Details DOI

YNIMG Journal 2019 Journal Article

Two cortical deficits underlie amblyopia: A multifocal fMRI analysis

Reza Farivar
Jiawei Zhou
Yufeng Huang
Lixia Feng
Yifeng Zhou
Robert F. Hess

Amblyopia is a relatively common (incidence 3%) developmental disorder in which there is loss of vision as a consequence of a disruption to normal visual development. Although the deficit is monocular and known to be of cortical origin, the nature of the processing deficit is controversial. Human behavioral studies have identified two main deficits — a loss of contrast sensitivity and perceived spatial distortions. Here we use a multifocal fMRI approach to ascertain, in a group of anisometropic amblyopes, whether these two deficits have a single common cause or whether they are the result of two underlying independent cortical disorders. We found that fMRI magnitudes were attenuated in amblyopic eye stimulation, and that there was poor fidelity for co-localization of the activity clusters between the amblyopic and fellow-fixing eye stimulation. These effects varied across eccentricities and correlate with the degree of amblyopia but not with one another, suggesting two independent cortical deficits: a reduced responsiveness as well as reduced fidelity of spatial representation. These deficits are independent of eccentricity within the central field and consistent across early cortical visual areas.

Details DOI