Author name cluster

Tianyu Chen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

19 papers

2 author rows

AAAI Conference 2026 Conference Paper

Fine-Tuned LLMs Know They Don’t Know: A Parameter-Efficient Approach to Recovering Honesty

Zeyu Shi
Ziming Wang
Tianyu Chen
Shiqi Gao
Haoyi Zhou
Qingyun Sun
Jianxin Li

The honesty of Large Language Models (LLMs) is increasingly important for safe deployment in high-stakes domains. However, this crucial trait is severely undermined by supervised fine-tuning (SFT), a common technique for model specialization. Existing recovery methods rely on data-intensive global parameter adjustments, implicitly assuming that SFT deeply corrupts the models' ability to recognize their knowledge boundaries. However, we observe that fine‑tuned LLMs still preserve this ability; what is damaged is their capacity to faithfully express that awareness. Building on this, we propose Honesty-Critical Neurons Restoration (HCNR) to surgically repair this suppressed capacity. HCNR identifies and restores key expression-governing neurons to their pre-trained state while harmonizing them with task-oriented neurons via Hessian-guided compensation. Experiments on four QA tasks and five LLM families demonstrate that HCNR effectively recovers 33.25% of the compromised honesty while achieving at least 2.23x speedup with over 10x less data compared to baseline methods, offering a practical solution for trustworthy LLM deployment.

PDF Details DOI

AAAI Conference 2026 Conference Paper

FRoD: Full-Rank Efficient Fine-Tuning with Rotational Degrees for Fast Convergence

Guoan Wan
Tianyu Chen
Fangzheng Feng
Haoyi Zhou
Runhua Xu

Parameter-efficient fine-tuning (PEFT) methods have emerged as a practical solution for adapting large foundation models to downstream tasks, reducing computational and memory costs by updating only a small subset of parameters. Among them, approaches like LoRA aim to strike a balance between efficiency and expressiveness, but often suffer from slow convergence and limited adaptation capacity due to their inherent low-rank constraints. This trade-off hampers the ability of PEFT methods to capture complex patterns needed for diverse tasks. To address these challenges, we propose FRoD, a novel fine-tuning method that combines hierarchical joint decomposition with rotational degrees of freedom. By extracting a globally shared basis across layers and injecting sparse, learnable perturbations into scaling factors for flexible full-rank updates, FRoD enhances expressiveness and efficiency, leading to faster and more robust convergence. On 20 benchmarks spanning vision, reasoning, and language understanding, FRoD matches full model fine-tuning in accuracy, while using only 1.72% of trainable parameters under identical training budgets.

PDF Details DOI

AAAI Conference 2026 Conference Paper

GeoNum: Bridging Numerical Continuity and Language Semantics via Geometric Embedding

Shengkai Jin
Tianyu Chen
Chonghan Gao
Jun Han

Large language models excel at semantic reasoning yet struggle with numerical tasks because tokenization disrupts geometric continuity. Traditional methods fragment numerically close values into inconsistent token sequences, severing the correspondence between numerical proximity and representational similarity, which is essential for numerical cognition. We introduce GeoNum, a geometrically coherent numerical embedding based on polar coordinate decomposition. By encoding integer magnitudes through classification and fractional components via trigonometric regression, GeoNum constructs a continuous manifold where numerical distance is preserved geometrically. A three-stage framework progressively integrates GeoNum into pretrained language models via self-supervised pretraining, projection alignment, and efficient adaptation. Experimental results across diverse arithmetic benchmarks demonstrate consistent gains in high-precision accuracy and improved interpolation and extrapolation, underscoring the promising benefits of geometric continuity for numerical modeling in large language models.

PDF Details DOI

JBHI Journal 2026 Journal Article

MVTT-GMamba: A Multimodal Graph Reasoning Framework for Anti-VEGF Efficacy Prediction in Diabetic Macular Edema

Shijing Wu
Yifan Zheng
Tianyu Chen
Gang Wu
Qinkun Zhi
Ailing Sui
Haixia Bai
Junhui Shen

Accurately predicting individual responses to Anti-Vascular Endothelial Growth Factor (Anti-VEGF) efficacy in diabetic macular edema (DME) remains a critical challenge in personalized ophthalmic care. Existing methods often rely on unimodal data or suffer from ineffective multimodal feature extraction and fusion, leading to modality redundancy and performance degradation. To address these limitations, we propose MVTT-GMamba, a novel multimodal learning framework that integrates optical coherence tomography (OCT) images and structured clinical indicators for early and accurate Anti-VEGF efficacy prediction. At the core of MVTT-GMamba is a feature-wise heterogeneous graph reasoning paradigm that explicitly models inter-patient and inter-feature relations, together with an adaptive, graph-guided prediction head that progressively anneals structural priors into the classifier. Building on this core, we adopt domain-tailored MambaVision and TabTransformer encoders and an early cross-attention fusion module to realize fine-grained multimodal representation learning. Extensive experiments on both a private clinical dataset (DMETHERA-ECSAHZU) and the public APTOS2021 benchmark demonstrate that MVTT-GMamba consistently outperforms state-of-the-art methods across all evaluation metrics. In addition, Grad-CAM visualizations reveal that the model attends to clinically relevant retinal regions, providing enhanced interpretability. Code is available at: https://github.com/DME666/DME.

Details DOI

AAAI Conference 2026 Conference Paper

Towards Long-window Anchoring in Vision-Language Model Distillation

Haoyi Zhou
Shuo Li
Tianyu Chen
Qi Song
Chonghan Gao
Jianxin Li

While large vision-language models (VLMs) demonstrate impressive long-context understanding, their prevalent small branches fails on linguistics-photography alignment for limited window size. We discover that knowledge distillation improve students capability as compelementary to Rotary Position Embeddings (RoPE) on certain windows size (anchored from large models). Building on this insight, we propose LAid, which explicitly targets the transfer of long-range attention mechanisms through two complementary components: (1) a progressive distance-weighted attention matching that dynamically emphasizes longer position differences during training, and (2) a learnable RoPE response gain modulation that selectively amplifies position sensitivity where needed. Extensive experiments across multiple model families demonstrate that LAid-distilled models achieve up to 3.2× longer effective context windows compared to baseline small models, while maintaining or improving performance on standard VL benchmarks. Spectral analysis also suggests that LAid successfully preserves crucial low-frequency attention components that conventional methods fail to transfer. Our work not only provides practical techniques for building more efficient long-context VLMs but also offers theoretical insights into how positional understanding emerges and transfers during distillation.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Automated Proof Generation for Rust Code via Self-Evolution

Tianyu Chen
Shuai Lu
Shan Lu 0001
Yeyun Gong
Chenyuan Yang
Xuheng Li
Md Rakib Hossain Misu
Hao Yu 0016

Ensuring correctness is crucial for code generation. Formal verification offers a definitive assurance of correctness, but demands substantial human effort in proof construction and hence raises a pressing need for automation. The primary obsta- cle lies in the severe lack of data—there is much fewer proofs than code snippets for Large Language Models (LLMs) to train upon. In this paper, we introduce SAFE, a framework that overcomes the lack of human-written proofs to enable automated proof generation of Rust code. SAFE establishes a self-evolving cycle where data synthesis and fine-tuning collaborate to enhance the model capability, leveraging the definitive power of a symbolic verifier in telling correct proofs from incorrect ones. SAFE also re-purposes the large number of synthesized incorrect proofs to train the self-debugging capability of the fine-tuned models, empowering them to fix incorrect proofs based on the verifier’s feedback. SAFE demonstrates superior efficiency and precision compared to GPT-4o. Through tens of thousands of synthesized proofs and the self-debugging mechanism, we improve the capa- bility of open-source models, initially unacquainted with formal verification, to automatically write proofs for Rust code. This advancement leads to a signifi- cant improvement in performance, achieving a 52.52% accuracy rate in a bench- mark crafted by human experts, a significant leap over GPT-4o’s performance of 14.39%.

Details

NeurIPS Conference 2025 Conference Paper

CoLT: The conditional localization test for assessing the accuracy of neural posterior estimates

Tianyu Chen
Vansh Bansal
James Scott

We consider the problem of validating whether a neural posterior estimate $q(\theta \mid x)$ is an accurate approximation to the true, unknown true posterior $p(\theta \mid x)$. Existing methods for evaluating the quality of an NPE estimate are largely derived from classifier-based tests or divergence measures, but these suffer from several practical drawbacks. As an alternative, we introduce the *Conditional Localization Test* (**CoLT**), a principled method designed to detect discrepancies between $p(\theta \mid x)$ and $q(\theta \mid x)$ across the full range of conditioning inputs. Rather than relying on exhaustive comparisons or density estimation at every $x$, CoLT learns a localization function that adaptively selects points $\theta_l(x)$ where the neural posterior $q$ deviates most strongly from the true posterior $p$ for that $x$. This approach is particularly advantageous in typical simulation-based inference settings, where only a single draw $\theta \sim p(\theta \mid x)$ from the true posterior is observed for each conditioning input, but where the neural posterior $q(\theta \mid x)$ can be sampled an arbitrary number of times. Our theoretical results establish necessary and sufficient conditions for assessing distributional equality across all $x$, offering both rigorous guarantees and practical scalability. Empirically, we demonstrate that CoLT not only performs better than existing methods at comparing $p$ and $q$, but also pinpoints regions of significant divergence, providing actionable insights for model refinement. These properties position CoLT as a state-of-the-art solution for validating neural posterior estimates.

PDF Details

IJCAI Conference 2025 Conference Paper

FreqMoE: Dynamic Frequency Enhancement for Neural PDE Solvers

Tianyu Chen
Haoyi Zhou
Ying Li
Hao Wang
Zhenzhe Zhang
Tianchen Zhu
Shanghang Zhang
Jianxin Li

Fourier Neural Operators (FNO) have emerged as promising solutions for efficiently solving partial differential equations (PDEs) by learning infinite-dimensional function mappings through frequency domain transformations. However, the sparsity of high-frequency signals limits computational efficiency for high-dimensional inputs, and fixed-pattern truncation often causes high-frequency signal loss, reducing performance in scenarios such as high-resolution inputs or long-term predictions. To address these challenges, we propose FreqMoE, an efficient and progressive training framework that exploits the dependency of high-frequency signals on low-frequency components. The model first learns low-frequency weights and then applies a sparse upward-cycling strategy to construct a mixture of experts (MoE) in the frequency domain, effectively extending the learned weights to high-frequency regions. Experiments on both regular and irregular grid PDEs demonstrate that FreqMoE achieves up to 16. 6 percent accuracy improvement while using merely 2. 1 percent parameters (47. 32x reduction) compared to dense FNO. Furthermore, the approach demonstrates remarkable stability in long-term predictions and generalizes seamlessly to various FNO variants and grid structures, establishing a new Low frequency Pretraining, High frequency Fine-tuning'' paradigm for solving PDEs.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay

Yifan Sun
Jingyan Shen
Yibin Wang
Tianyu Chen
Zhendong Wang
Mingyuan Zhou
Huan ZHang

Reinforcement learning (RL) has become an effective approach for fine-tuning large language models (LLMs), particularly to enhance their reasoning capabilities. However, RL fine-tuning remains highly resource-intensive, and existing work has largely overlooked the problem of data efficiency. In this paper, we propose two techniques to improve data efficiency in LLM RL fine-tuning: difficulty-targeted online data selection and rollout replay. We introduce the notion of adaptive difficulty to guide online data selection, prioritizing questions of moderate difficulty that are more likely to yield informative learning signals. To estimate adaptive difficulty efficiently, we develop an attention-based framework that requires rollouts for only a small reference set of questions. The adaptive difficulty of the remaining questions is then estimated based on their similarity to this set. To further reduce rollout cost, we introduce a rollout replay mechanism inspired by experience replay in traditional RL. This technique reuses recent rollouts, lowering per-step computation while maintaining stable updates. Experiments across 6 LLM-dataset combinations show that our method reduces RL fine-tuning time by 23% to 62% while reaching the same level of performance as the original GRPO algorithm. Our code repository is available at https: //github. com/ASTRAL-Group/data-efficient-llm-rl/.

PDF Details

TIST Journal 2025 Journal Article

Leveraging LLMs for Semantic Correlation Enhancement in Spatial-temporal Imputation

Xin Xue
Haoyi Zhou
Lanhao Li
Yihan Lin
Tianyu Chen
Jianxin Li

Spatial-temporal imputation remains a challenging problem in transportation, environment and healthcare, where the missing value is filled based on spatial, temporal, and cross correlations. Previous research mainly focused on feature-level correlation integration and comprehension with the hand-crafted enhancement strategy. Meanwhile, the recently prevalent large language models (LLMs) provide token-level understanding for language linguistics, and whether they could be applied for spatial-temporal correlation enhancement is under exploration. To this end, we proposed an LLM-native framework STOMA to fully utilize the intrinsic relevance. We designed semantic enhancing methods by converting the complex correlations, e.g. spatial correlation in network, temporal correlation with periodicity and cross correlation from human behavior, into the embedded tokens. Specifically, we reform dynamic time warping as an asymmetric correlation constructor for complex dynamics. We adapt the proposed backbone along with the spatial-temporal fine-tuning technique, and the empirical results demonstrate the effectiveness of our methods over recent LLM-inspired methods evaluating on real-world datasets.

Details DOI

EAAI Journal 2025 Journal Article

Multisource-domain regression transfer learning framework for predicting student academic performance considering balanced similarity

Li Wang
Lucong Zhang
Haotian Wu
Teng Zhang
Ke Qiu
Tianyu Chen
Hongwu Qin

The increasing integration of information technology and artificial intelligence has extensively implemented computer-aided intelligent education systems in higher education. A critical task within these systems is student performance prediction, which forecasts future academic outcomes by analyzing data such as historical grades, learning behaviors, and classroom participation. This enables early intervention and personalized teaching based on scientific evidence. However, most existing methods rely on traditional machine learning techniques, which can hardly address issues such as domain distribution discrepancies and data imbalance effectively. To overcome these challenges, we propose a multisource-domain transfer learning regression framework that integrates domain selection, hybrid feature extraction, and dynamic joint distribution adaptation techniques. Specifically, the framework first selects appropriate source domains on the basis of preset thresholds via cross-validation. Thereafter, a hybrid feature extractor is used to derive (i) common features from the target and selected source domains and (ii) domain-specific features from the target domain. Finally, a dynamic adaptive factor is introduced to balance differences between the marginal and conditional distributions. Experimental results indicate that the proposed framework significantly reduces the root mean square error with an average prediction improvement of 21. 05 %, compared with baseline methods and other advanced approaches.

Details DOI

ICML Conference 2025 Conference Paper

OmniArch: Building Foundation Model for Scientific Computing

Tianyu Chen
Haoyi Zhou
Ying Li 0128
Hao Wang 0073
Chonghan Gao
Rongye Shi
Shanghang Zhang
Jianxin Li 0002

Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery.

Details

ICML Conference 2025 Conference Paper

PINNsAgent: Automated PDE Surrogation with Large Language Models

Qingpo Wuwu
Chonghan Gao
Tianyu Chen
Yihang Huang
Yuekai Zhang
Jianing Wang
Jianxin Li 0002
Haoyi Zhou

Solving partial differential equations (PDEs) using neural methods has been a long-standing scientific and engineering research pursuit. Physics-Informed Neural Networks (PINNs) have emerged as a promising alternative to traditional numerical methods for solving PDEs. However, the gap between domain-specific knowledge and deep learning expertise often limits the practical application of PINNs. Previous works typically involve manually conducting extensive PINNs experiments and summarizing heuristic rules for hyperparameter tuning. In this work, we introduce PINNsAgent, a novel surrogation framework that leverages large language models (LLMs) to bridge the gap between domain-specific knowledge and deep learning. PINNsAgent integrates Physics-Guided Knowledge Replay (PGKR) for efficient knowledge transfer from solved PDEs to similar problems, and Memory Tree Reasoning for exploring the search space of optimal PINNs architectures. We evaluate PINNsAgent on 14 benchmark PDEs, demonstrating its effectiveness in automating the surrogation process and significantly improving the accuracy of PINNs-based solutions.

Details

NeurIPS Conference 2024 Conference Paper

Diffusion Policies Creating a Trust Region for Offline Reinforcement Learning

Tianyu Chen
Zhendong Wang
Mingyuan Zhou

Offline reinforcement learning (RL) leverages pre-collected datasets to train optimal policies. Diffusion Q-Learning (DQL), introducing diffusion models as a powerful and expressive policy class, significantly boosts the performance of offline RL. However, its reliance on iterative denoising sampling to generate actions slows down both training and inference. While several recent attempts have tried to accelerate diffusion-QL, the improvement in training and/or inference speed often results in degraded performance. In this paper, we introduce a dual policy approach, Diffusion Trusted Q-Learning (DTQL), which comprises a diffusion policy for pure behavior cloning and a practical one-step policy. We bridge the two polices by a newly introduced diffusion trust region loss. The diffusion policy maintains expressiveness, while the trust region loss directs the one-step policy to explore freely and seek modes within the region defined by the diffusion policy. DTQL eliminates the need for iterative denoising sampling during both training and inference, making it remarkably computationally efficient. We evaluate its effectiveness and algorithmic characteristics against popular Kullback-Leibler (KL) based distillation methods in 2D bandit scenarios and gym tasks. We then show that DTQL could not only outperform other methods on the majority of the D4RL benchmark tasks but also demonstrate efficiency in training and inference speeds. The PyTorch implementation is available at https: //github. com/TianyuCodings/Diffusion Trusted Q_Learning.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Identifying General Mechanism Shifts in Linear Causal Representations

Tianyu Chen
Kevin Bello
Francesco Locatello
Bryon Aragam
Pradeep Ravikumar

We consider the linear causal representation learning setting where we observe a linear mixing of $d$ unknown latent factors, which follow a linear structural causal model. Recent work has shown that it is possible to recover the latent factors as well as the underlying structural causal model over them, up to permutation and scaling, provided that we have at least $d$ environments, each of which corresponds to perfect interventions on a single latent node (factor). After this powerful result, a key open problem faced by the community has been to relax these conditions: allow for coarser than perfect single-node interventions, and allow for fewer than $d$ of them, since the number of latent factors $d$ could be very large. In this work, we consider precisely such a setting, where we allow a smaller than $d$ number of environments, and also allow for very coarse interventions that can very coarsely \textit{change the entire causal graph over the latent factors}. On the flip side, we relax what we wish to extract to simply the \textit{list of nodes that have shifted between one or more environments}. We provide a surprising identifiability result that it is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes. Our identifiability proof moreover is a constructive one: we explicitly provide necessary and sufficient conditions for a node to be a shifted node, and show that we can check these conditions given observed data. Our algorithm lends itself very naturally to the sample setting where instead of just interventional distributions, we are provided datasets of samples from each of these distributions. We corroborate our results on both synthetic experiments as well as an interesting psychometric dataset. The code can be found at https: //github. com/TianyuCodings/iLCS.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Towards Editing Time Series

Baoyu Jing
Shuqi Gu
Tianyu Chen
Zhiyu Yang
Dongsheng Li
Jingrui He
Kan Ren

Synthesizing time series data is pivotal in modern society, aiding effective decision making and ensuring privacy preservation in various scenarios. Time series are associated with various attributes, including trends, seasonality, and external information such as location. Recent research has predominantly focused on random unconditional synthesis or conditional synthesis. Nonetheless, these paradigms generate time series from scratch and are incapable of manipulating existing time series samples. This paper introduces a novel task, called Time Series Editing (TSE), to synthesize time series by manipulating existing time series. The objective is to modify the given time series according to the specified attributes while preserving other properties unchanged. This task is not trivial due to the inadequacy of data coverage and the intricate relationships between time series and their attributes. To address these issues, we introduce a novel diffusion model, called TEdit. The proposed TEdit is trained using a novel bootstrap learning algorithm that effectively enhances the coverage of the original data. It is also equipped with an innovative multi-resolution modeling and generation paradigm to capture the complex relationships between time series and their attributes. Experimental results demonstrate the efficacy of TEdit for editing specified attributes upon the existing time series data. The project page is at https: //seqml. github. io/tse.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models

Tianyu Chen
Kevin Bello
Bryon Aragam
Pradeep Ravikumar

Structural causal models (SCMs) are widely used in various disciplines to represent causal relationships among variables in complex systems. Unfortunately, the underlying causal structure is often unknown, and estimating it from data remains a challenging task. In many situations, however, the end goal is to localize the changes (shifts) in the causal mechanisms between related datasets instead of learning the full causal structure of the individual datasets. Some applications include root cause analysis, analyzing gene regulatory network structure changes between healthy and cancerous individuals, or explaining distribution shifts. This paper focuses on identifying the causal mechanism shifts in two or more related datasets over the same set of variables--- without estimating the entire DAG structure of each SCM. Prior work under this setting assumed linear models with Gaussian noises; instead, in this work we assume that each SCM belongs to the more general class of nonlinear additive noise models (ANMs). A key technical contribution of this work is to show that the Jacobian of the score function for the mixture distribution allows for the identification of shifts under general non-parametric functional mechanisms. Once the shifted variables are identified, we leverage recent work to estimate the structural differences, if any, for the shifted variables. Experiments on synthetic and real-world data are provided to showcase the applicability of this approach. Code implementing the proposed method is open-source and publicly available at https: //github. com/kevinsbello/iSCAN.

PDF Details

AAAI Conference 2023 Conference Paper

MoEC: Mixture of Expert Clusters

Yuan Xie
Shaohan Huang
Tianyu Chen
Furu Wei

Sparsely Mixture of Experts (MoE) has received great interest due to its promising scaling capability with affordable computational overhead. MoE models convert dense layers into sparse experts, and utilize a gated routing network to make experts conditionally activated. However, as the number of experts grows, MoE with outrageous parameters suffers from overfitting and sparse data allocation. Such problems are especially severe on tasks with limited data, thus hindering the progress towards improving performance by scaling up. We verify that there exists a performance upper bound of scaling up sparse MoE. In this work, we propose Mixture of Expert Clusters — a general approach to enable expert layers to learn more diverse and appropriate knowledge by imposing variance-based constraints on the routing stage. Given this, we could further propose a cluster-level expert dropout strategy specifically designed for the expert cluster structure. Our experiments reveal that MoEC could improve performance on machine translation and natural language understanding tasks. MoEC plays a positive role in mitigating overfitting and sparse data allocation problems, thus fully releasing the potential of large-scale sparse models.

PDF Details DOI

IROS Conference 2015 Conference Paper

Humanoid full-body manipulation planning with multiple initial guesses and key postures

Bowei Tang
Tianyu Chen
Christopher G. Atkeson

We present an optimization method to solve coupled redundant inverse kinematics problems and generate trajectories for humanoid robot full-body manipulation. The basic idea of our algorithm is to divide a manipulation task into a series of key postures, generate multiple diverse initial guesses for each key posture, and use optimization to find inverse kinematics solutions based on these initial guesses. We then find an optimal series of key postures and form a continuous trajectory. Our approach is implemented in a Gazebo simulation using the Atlas humanoid robot from Boston Dynamics.

Details