Author name cluster

Zhenyu Hou

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ICML Conference 2025 Conference Paper

Controlling Large Language Model with Latent Action

Chengxing Jia
Ziniu Li
Pengyuan Wang
Yi-Chen Li 0001
Zhenyu Hou
Yuxiao Dong
Yang Yu 0001

Adapting Large Language Models (LLMs) to downstream tasks using Reinforcement Learning (RL) has proven to be an effective approach. However, LLMs do not inherently define the structure of an agent for RL training, particularly in terms of specifying the action space. This paper studies learning a compact latent action space to enhance the controllability and exploration of RL for LLMs. Inspired by reinforcement learning from observations, we propose Co ntrolling Large Language Models with L atent A ctions CoLA, a framework that integrates a latent action space into pre-trained LLMs. CoLA employs an inverse dynamics model to extract latent actions conditioned on future tokens, ensuring that the next token prediction is partially influenced by these actions. Simultaneously, CoLA fine-tunes the pre-trained LLM to function as a language world model, capable of incorporating latent actions as inputs. Additionally, CoLA trains a policy model to generate actions within this language world model. The policy model can be trained via behavior cloning to mimic a standard language model or through RL to maximize task-specific rewards. In this work, we apply CoLA to the Llama-3. 1-8B model. Our experiments demonstrate that, compared to RL with token-level actions, CoLA ’s latent actions enable greater semantic diversity. For enhancing downstream tasks, we show that CoLA with RL achieves a score of 42. 4 on the math500 benchmark, surpassing the baseline score of 38. 2, and reaches 68. 2 when augmented with a Monte Carlo Tree Search variant. Furthermore, CoLA with RL consistently improves performance on agent-based tasks without degrading the pre-trained LLM’s capabilities, unlike the baseline. Finally, CoLA reduces computation time by half in tasks involving enhanced thinking prompts for LLMs via RL. These results highlight CoLA ’s potential to advance RL-based adaptation of LLMs for downstream applications. The CoLA model is available at https: //huggingface. co/LAMDA-RL/Llama-3. 1-CoLA-10B.

Details

JBHI Journal 2025 Journal Article

Enhancing Automated Seizure Detection via Self-Calibrating Spatial-Temporal EEG Features with SC-LSTM

Wenhao Li
Qiran Chen
Zhenyu Hou
Shi Chang
Zhenhong Ye
Jiangping Chen
Guan Ning Lin

Epilepsy, a highly individualized neurological disorder, affects millions globally. Electroencephalography (EEG) remains the cornerstone for seizure diagnosis, yet manual interpretation is labor-intensive and often unreliable due to the complexity of multi-channel, high-dimensional data. Traditional machine learning models often struggle with overfitting and fail in fully capturing the highdimensional, temporal dynamics of EEG signals, restricting their clinical utility. In this study, we propose SC-LSTM, a novel hybrid deep learning architecture that integrates dynamic spatial and temporal feature extraction to enhance automated seizure detection. SC-LSTM comprises a SelfCalibrated Reconstruction Module (SCConvNet) for adaptive spatial feature representation and a Bidirectional Long Short-Term Memory (Bi-LSTM) network for modeling temporal dependency. This parallel processing framework captures patient-specific EEG variability more effectively than traditional sequential models, promoting robust and discriminative feature learning. Comprehensive evaluations on two real-world neonatal EEG datasets, using K-fold crossvalidation and simulated single-channel signal loss, demonstrate that SC-LSTM achieved an accuracy of 97% and an area under the curve of 0. 99, significantly surpassing the performance of CNN and CNN-LSTM models. Importantly, SC-LSTM maintained high diagnostic performance even under conditions of partial data loss from critical brain regions, underscoring its resilience to clinical variability and signal artifacts. By improving accuracy, stability, and adaptability in seizure detection, SC-LSTM exemplifies the application of artificial intelligence to support individualized diagnostics and embodies the core principles of precision medicine. The open-source availability of SC-LSTM further facilitates reproducibility, clinical translation, and future extensions across broader neurological disorder monitoring applications.

Details DOI

IROS Conference 2025 Conference Paper

Real-time Spatial-temporal Traversability Assessment via Feature-based Sparse Gaussian Process

Senming Tan
Zhenyu Hou
Zhihao Zhang
Long Xu 0002
Mengke Zhang
Zhaoqi He
Chao Xu 0001
Fei Gao 0011

Terrain analysis is critical for the practical application of ground mobile robots in real-world tasks, especially in outdoor unstructured environments. In this paper, we propose a novel spatial-temporal traversability assessment method, which aims to enable autonomous robots to effectively navigate through complex terrains. Our approach utilizes sparse Gaussian processes (SGP) to extract geometric features (curvature, gradient, elevation, etc.) directly from point cloud scans. These features are then used to construct a high-resolution local traversability map. Then, we design a spatial-temporal Bayesian Gaussian kernel (BGK) inference method to dynamically evaluate traversability scores, integrating historical and real-time data while considering factors such as slope, flatness, gradient, and uncertainty metrics. GPU acceleration is applied in the feature extraction step, and the system achieves real-time performance. Extensive simulation experiments across diverse terrain scenarios demonstrate that our method outperforms SOTA approaches in both accuracy and computational efficiency. Additionally, we develop an autonomous navigation framework integrated with the traversability map and validate it with a differential driven vehicle in complex outdoor environments. Our code will be open-source for further research and development by the community, https://github.com/ZJU-FAST-Lab/FSGP_BGK.

Details

ICML Conference 2025 Conference Paper

T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling

Zhenyu Hou
Xin Lv
Rui Lu 0002
Jiajie Zhang
Yujiang Li
Zijun Yao 0002
Juanzi Li
Jie Tang 0001

Large language models (LLMs) have demonstrated remarkable capabilities in complex reasoning tasks. However, existing approaches mainly rely on imitation learning and struggle to achieve effective test-time scaling. While reinforcement learning (RL) holds promise for enabling self-exploration, recent attempts yield modest improvements in complex reasoning. In this paper, we present T1 to scale RL by encouraging exploration and understand inference scaling. We first initialize the LLM using synthesized chain-of-thought data that integrates trial-and-error and self-verification. To scale RL training, we promote increased sampling diversity through over-sampling. We demonstrate that T1 with open LLMs as its base exhibits inference scaling behavior and achieves superior performance on challenging math reasoning benchmarks. More importantly, we present a simple strategy to examine inference scaling, where increased inference budgets directly lead to T1’s better performance without any additional verification. The model weights and training data are publicly available at https: //github. com/THUDM/T1.

Details

AAAI Conference 2023 Conference Paper

MTDiag: An Effective Multi-Task Framework for Automatic Diagnosis

Zhenyu Hou
Yukuo Cen
Ziding Liu
Dongxue Wu
Baoyan Wang
Xuanhe Li
Lei Hong
Jie Tang

Automatic diagnosis systems aim to probe for symptoms (i.e., symptom checking) and diagnose disease through multi-turn conversations with patients. Most previous works formulate it as a sequential decision process and use reinforcement learning (RL) to decide whether to inquire about symptoms or make a diagnosis. However, these RL-based methods heavily rely on the elaborate reward function and usually suffer from an unstable training process and low data efficiency. In this work, we propose an effective multi-task framework for automatic diagnosis called MTDiag. We first reformulate symptom checking as a multi-label classification task by direct supervision. Each medical dialogue is equivalently converted into multiple samples for classification, which can also help alleviate the data scarcity problem. Furthermore, we design a multi-task learning strategy to guide the symptom checking procedure with disease information and further utilize contrastive learning to better distinguish symptoms between diseases. Extensive experimental results show that our method achieves state-of-the-art performance on four public datasets with 1.7%~3.1% improvement in disease diagnosis, demonstrating the superiority of the proposed method. Additionally, our model is now deployed in an online medical consultant system as an assistant tool for real-life doctors.

PDF Details DOI