Author name cluster

Yadong Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

AAMAS Conference 2026 Conference Paper

Plan-and-Execute: LLM-Guided Reinforcement Learning with Cross-Modal Fusion for Long-Sequence Decision Making

Yadong Li
Tong Zhang
Zhen Cui

We introduce Plan-and-Execute (PLEX), a novel framework that synergies the abstract reasoning capabilities of large language models (LLMs) with the grounded reinforcement learning (RL). In this architecture, an LLM serves as a dynamic planner, iteratively decomposing complex language instructions into structured subgoal sequences. A dedicated RL agent, leveraging cross-modal attention mechanism, then executes these subgoals by fusing language instructions with high-dimensional visual observations to learn and optimize its decision-making policies. This hierarchical coordination enables our agent to master long-horizon tasks through a tight loop of reasoning and grounded interaction. Comprehensive evaluation in the MiniGrid and MiniHack environment confirms that PLEX achieves a significant performance improvement over existing methods across diverse scenarios. The PLEX exhibits superior sample efficiency, particularly in complex, long-horizon tasks that require sustained reasoning and action sequences.

PDF

AAAI Conference 2025 Conference Paper

MHBench: Demystifying Motion Hallucination in VideoLLMs

Ming Kong
Xianzhou Zeng
Luyuan Chen
Yadong Li
Bo Yan
Qiang Zhu

Similar to Language or Image LLMs, VideoLLMs are also plagued by hallucination issues. Hallucinations in videos not only manifest in the spatial dimension regarding the perception of the existence of visual objects (static) but also the temporal dimension influencing the perception of actions and events (dynamic). This paper introduces the concept of Motion Hallucination for the first time, exploring the hallucination phenomena caused by insufficient motion perception capabilities in VideoLMMs, as well as how to detect, evaluate, and mitigate the hallucination. To this end, we propose the first benchmark for assessing motion hallucination MHBench, which consists of 1,200 videos of 20 different action categories. By constructing a collection of adversarial triplet types of videos (original/antonym/incomplete), we achieve a comprehensive evaluation of motion hallucination. Furthermore, we present a Motion Contrastive Decoding (MotionCD) method, which employs bidirectional motion elimination between the original video and its reverse playback to construct an amateur model that removes the influence of motion while preserving visual information, thereby effectively suppressing motion hallucination. Extensive experiments on MHBench reveal that current state-of-the-art VideoLLMs significantly suffer from motion hallucination, while the introduction of MotionCD can effectively mitigate this issue, achieving up to a 15.1% performance improvement. We hope this work will guide future efforts in avoiding and mitigating hallucinations in VideoLLMs.

PDF Details DOI

AAAI Conference 2025 Conference Paper

MoLE:Decoding by Mixture of Layer Experts Alleviates Hallucination in Large Vision-Language Models

Tian Liang
Yuetian Du
Jing Huang
Ming Kong
Luyuan Chen
Yadong Li
Siye Chen
Qiang Zhu

Recent advancements in Large Vision-Language Models (LVLMs) highlight their ability to integrate and process multi-modal information. However, hallucinations—where generated content is inconsistent with input vision and instructions—remain a challenge. In this paper, we analyze LVLMs' layer-wise decoding and identify that hallucinations can arise during the reasoning and factual information injection process. Additionally, as the number of generated tokens increases, the forgetting of the original prompt may also lead to hallucinations.To address this, we propose a training-free decoding method called Mixture of Layer Experts (MoLE). MoLE leverages a heuristic gating mechanism to dynamically select multiple layers of LVLMs as expert layers: the Final Expert, the Second Opinion expert, and the Prompt Retention Expert. By the cooperation of each expert, MoLE enhances the robustness and faithfulness of the generation process. Our extensive experiments demonstrate that MoLE significantly reduces hallucinations, outperforming the current state-of-the-art decoding techniques across three mainstream LVLMs and two established hallucination benchmarks. Moreover, our method reveals the potential of LVLMs to independently produce more reliable and accurate outputs.

PDF Details DOI

ICLR Conference 2025 Conference Paper

RFMamba: Frequency-Aware State Space Model for RF-Based Human-Centric Perception

Rui Zhang 0120
Ruixu Geng
Yadong Li
Ruiyuan Song
Hanqin Gong
Dongheng Zhang
Yang Hu 0006
Yan Chen 0007

Human-centric perception with radio frequency (RF) signals has recently entered a new era of end-to-end processing with Transformers. Considering the long-sequence nature of RF signals, the State Space Model (SSM) has emerged as a superior alternative due to its effective long-sequence modeling and linear complexity. However, integrating SSM into RF-based sensing presents unique challenges including the fundamentally different signal representation, distinct frequency responses in different scenarios, and incomplete capture caused by specular reflection. To address this, we carefully devise a dual-branch SSM block that is characterized by adaptively grasping the most informative frequency cues and the assistant spatial information to fully explore the human representations from radar echoes. Based on these two branchs, we further introduce an SSM-based network for handling various downstream human perception tasks, named RFMamba. Extensive experimental results demonstrate the superior performance of our proposed RFMamba across all three downstream tasks. To the best of our knowledge, RFMamba is the first attempt to introduce SSM into RF-based human-centric perception.

Details

IJCAI Conference 2024 Conference Paper

Learning-Based Tracking-before-Detect for RF-Based Unconstrained Indoor Human Tracking

Zhi Wu
Dongheng Zhang
Zixin Shang
Yuqin Yuan
Hanqin Gong
Binquan Wang
Zhi Lu
Yadong Li

Existing efforts on human tracking using wireless signal are primarily focused on constrained scenarios with only a few individuals in empty spaces. However, in practical unconstrained scenarios with severe interference and attenuation, accurate multi-person tracking has been intractable. In this paper, we propose NeuralTBD, utilizing the capability of deep models and advancement of Tracking-Before-Detect (TBD) methodology to achieve accurate human tracking. TBD is a classical tracking methodology from signal processing accumulating measurement in time domain to distinguish target traces from interference, which however relies on handcrafted shape/motion models, impeding efficacy in complex indoor scenarios. To tackle this challenge, we build an end-to-end learning-based TBD framework leverages the advanced modeling capabilities of deep models to significantly enhance the performance of TBD. To evaluate NeuralTBD, we collect an RF-based tracking dataset in unconstrained scenarios, which encompasses 4 million annotated radar frames with up to 19 individuals acting in 6 different scenarios. NeuralTBD realizes a 70% improvement in performance compared to conventional TBD methods. To our knowledge, this is the first attempt dealing with RF-based unconstrained human tracking. The code and dataset will be released.

PDF Details DOI