Author name cluster

Ziqian Lin

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

AAAI Conference 2026 Conference Paper

DS-ProGen: A Dual-Structure Deep Language Model for Functional Protein Design

Yanting Li
Zikang Wang
Jiyue Jiang
Ziqian Lin
Dongchen He
Yuheng Shan
Yanruisheng Shao
Jiayi Li

Inverse Protein Folding (IPF) is a critical subtask in the field of protein design, aiming to engineer amino acid sequences capable of folding correctly into a specified three-dimensional (3D) conformation. Although substantial progress has been achieved in recent years, existing methods generally rely on either backbone coordinates or molecular surface features alone, which restricts their ability to fully capture the complex chemical and geometric constraints necessary for precise sequence prediction. To address this limitation, we present DS-ProGen, a dual-structure deep language model for functional protein design, which integrates both backbone geometry and surface-level representations. By incorporating backbone coordinates as well as surface chemical and geometric descriptors into a next-amino-acid prediction paradigm, DS-ProGen is able to generate functionally relevant and structurally stable sequences while satisfying both global and local conformational constraints. On the PRIDE dataset, DS-ProGen attains the current state-of-the-art recovery rate of 61.47%, demonstrating the synergistic advantage of multi-modal structural encoding in protein design. Furthermore, DS-ProGen excels in predicting interactions with a variety of biological partners, including ligands, ions, and RNA, confirming its robust functional retention capabilities.

PDF Details DOI

EAAI Journal 2026 Journal Article

Modeling carbon price dynamics driven by external factors: A hybrid framework combining Mamba state-space model and transformer

Zhenkai Qin
Baozhong Wei
Qining Luo
Dongze Wu
Weiqi Qin
Xiaochuan Yu
Ziqian Lin

Accurate multi-step forecasting of carbon prices plays a pivotal role in optimizing the allocation of carbon resources, mitigating emissions, and promoting sustainable development. Moreover, it underpins the formulation of stable and effective carbon pricing mechanisms, offering critical insights for business strategies and investment planning. In this study, we propose BMambaXer, a novel hybrid forecasting framework that synergistically integrates a Bidirectional Mamba architecture with Transformer components to jointly model endogenous temporal dependencies and exogenous driving forces. To overcome the inherent limitation of the standard Mamba module—which can only capture unidirectional dependencies—we incorporate an additional reverse-time Mamba stream to achieve comprehensive bidirectional sequence modeling, thereby enhancing the model’s ability to capture complex temporal patterns in carbon price evolution. Furthermore, by embedding exogenous variables through a variable-wise integration mechanism, our model improves causal interpretability and enriches contextual awareness, leading to more robust and accurate predictions. We conduct extensive experiments across three real-world carbon trading markets and multiple forecasting horizons. Results consistently show that BMambaXer outperforms all baseline models in both prediction accuracy and stability. This framework presents a scalable and interpretable solution for carbon price forecasting, offering actionable insights for carbon market regulation, energy policy formulation, and enterprise-level carbon risk management in dynamic, uncertainty-prone environments. The implementation is publicly available at: https: //github. com/weibaozhong/BMambaXer.

Details DOI

AAAI Conference 2026 Conference Paper

RMSAGen: Integrating Multiple Sequence Alignment for Function RNA Design

Jiyue Jiang
Yanyu Chen
Qingchuan Zhang
Jiayi Li
Xiangyu Shi
Chang Zhou
Ziqian Lin
Jiuming Wang

Biological sequences, including RNAs and proteins, share similarities with natural languages, enabling the application of advanced language models to various biological tasks. However, due to its flexibility and lack of experimental data, RNA is a particularly challenging biological ``language'' compared to other biological sequences like proteins. RNA multiple sequence alignments (MSAs), which align evolutionarily related RNA sequences, can greatly enhance RNA biology modeling, as evidenced by their significant roles in structure prediction and function annotation. This raises the question of whether RNA MSAs can also benefit RNA design, which remains unexplored. This paper introduces RMSAGen, a model comprising RMSA-Encoder and RMSA-Decoder, that leverages MSAs to design functional RNA sequences. RMSA-Encoder effectively extracts MSA features, enhancing performance in functional prediction and solvent accessibility prediction tasks and supporting RMSA-Decoder in accurate RNA generation. RMSAGen can design RNA sequences that effectively bind to target RNA-binding proteins, and the design performance improves with an increasing number of sequences. In addition, the ribozymes designed with structural features by RMSAGen show strong computational metrics and exhibit biological activity during gel electrophoresis. These results highlight the effectiveness of RMSAGen, establishing it as a powerful tool and a new direction for RNA design.

PDF Details DOI

TMLR Journal 2026 Journal Article

Transformers in the Dark: Navigating Unknown Search Spaces via Bandit Feedback

Jungtaek Kim
Thomas Zeng
Ziqian Lin
Minjae Lee
Chungpa Lee
Jy-yong Sohn
Hyung Il Koo
Kangwook Lee

Effective problem solving with Large Language Models (LLMs) can be enhanced when they are paired with external search algorithms. By viewing the space of diverse ideas and their follow-up possibilities as a tree structure, the search algorithm can navigate such a search space and guide the LLM toward better solutions more efficiently. While the search algorithm enables an effective balance between exploitation and exploration of a tree-structured space, the need for an external component can complicate the overall problem-solving process. We therefore pose the following question: Can LLMs or their underlying Transformer architectures approximate a search algorithm? To answer this question, we first introduce a simplified framework in which tree extensions and feedback signals are externally specified, allowing for controlled evaluation of search capabilities. We call this setting unknown tree search with bandit feedback. Within this setting, we show that Transformers are theoretically expressive enough to implement distinct search strategies and can be trained from scratch to approximate those strategies. Our Transformer models exhibit the possibility of generalizing to unseen conditions such as longer horizons or deeper trees. Furthermore, we demonstrate that continued task-focused training unlocks the complete capabilities of a pretrained LLM, by fine-tuning the LLM on search trajectories.

PDF Details

ICML Conference 2025 Conference Paper

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition

Zheyang Xiong
Ziyang Cai
John Cooper
Albert Ge
Vasilis Papageorgiou
Zack Sifakis
Angeliki Giannou
Ziqian Lin

Large Language Models (LLMs) have demonstrated remarkable in-context learning (ICL) capabilities. In this study, we explore a surprising phenomenon related to ICL: LLMs can perform multiple, computationally distinct ICL tasks simultaneously, during a single inference call, a capability we term task superposition". We provide empirical evidence of this phenomenon across various LLM families and scales and show that this phenomenon emerges even if we train the model to in-context learn one task at a time. We offer theoretical explanations that this capability is well within the expressive power of transformers. We also explore how LLMs internally compose task vectors during superposition. Furthermore, we show that larger models can solve more ICL tasks in parallel, and better calibrate their output distribution. Our findings offer insights into the latent capabilities of LLMs, further substantiate the perspective of "LLMs as superposition of simulators", and raise questions about the mechanisms enabling simultaneous task execution.

Details

ICML Conference 2024 Conference Paper

Dual Operating Modes of In-Context Learning

Ziqian Lin
Kangwook Lee 0001

In-context learning (ICL) exhibits dual operating modes: task learning, i. e. , acquiring a new skill from in-context samples, and task retrieval, i. e. , locating and activating a relevant pretrained skill. Recent theoretical work proposes various mathematical models to analyze ICL, but they cannot fully explain the duality. In this work, we analyze a generalized probabilistic model for pretraining data, obtaining a quantitative understanding of the two operating modes of ICL. Leveraging our analysis, we provide the first explanation of an unexplained phenomenon observed with real-world large language models (LLMs). Under some settings, the ICL risk initially increases and then decreases with more in-context examples. Our analysis offers a plausible explanation for this "early ascent" phenomenon: a limited number of in-context samples may lead to the retrieval of an incorrect skill, thereby increasing the risk, which will eventually diminish as task learning takes effect with more in-context samples. We also analyze ICL with biased labels, e. g. , zero-shot ICL, where in-context examples are assigned random labels, and predict the bounded efficacy of such approaches. We corroborate our analysis and predictions with extensive experiments with Transformers and LLMs.

Details

NeurIPS Conference 2022 Conference Paper

LIFT: Language-Interfaced Fine-Tuning for Non-language Machine Learning Tasks

Tuan Dinh
Yuchen Zeng
Ruisu Zhang
Ziqian Lin
Michael Gira
Shashank Rajput
Jy-yong Sohn
Dimitris Papailiopoulos

Fine-tuning pretrained language models (LMs) without making any architectural changes has become a norm for learning various language downstream tasks. However, for non-language downstream tasks, a common practice is to employ task-specific designs for input, output layers, and loss functions. For instance, it is possible to fine-tune an LM into an MNIST classifier by replacing the word embedding layer with an image patch embedding layer, the word token output layer with a 10-way output layer, and the word prediction loss with a 10-way classification loss, respectively. A natural question arises: Can LM fine-tuning solve non-language downstream tasks without changing the model architecture or loss function? To answer this, we propose Language-Interfaced Fine-Tuning (LIFT) and study its efficacy and limitations by conducting an extensive empirical study on a suite of non-language classification and regression tasks. LIFT does not make any changes to the model architecture or loss function, and it solely relies on the natural language interface, enabling "no-code machine learning with LMs. " We find that LIFT performs comparably well across a wide range of low-dimensional classification and regression tasks, matching the performances of the best baselines in many cases, especially for the classification tasks. We also report experimental results on the fundamental properties of LIFT, including inductive bias, robustness, and sample complexity. We also analyze the effect of pretraining on LIFT and a few properties/techniques specific to LIFT, e. g. , context-aware learning via appropriate prompting, calibrated predictions, data generation, and two-stage fine-tuning. Our code is available at https: //github. com/UW-Madison-Lee-Lab/LanguageInterfacedFineTuning.

PDF Details

IJCAI Conference 2020 Conference Paper

A Sequential Convolution Network for Population Flow Prediction with Explicitly Correlation Modelling

Jie Feng
Ziqian Lin
Tong Xia
Funing Sun
Diansheng Guo
Yong Li

Population flow prediction is one of the most fundamental components in many applications from urban management to transportation schedule. It is challenging due to the complicated spatial-temporal correlation. While many studies have been done in recent years, they fail to simultaneously and effectively model the spatial correlation and temporal variations among population flows. In this paper, we propose Convolution based Sequential and Cross Network (CSCNet) to solve them. On the one hand, we design a CNN based sequential structure with progressively merging the flow features from different time in different CNN layers to model the spatial-temporal information simultaneously. On the other hand, we make use of the transition flow as the proxy to efficiently and explicitly capture the dynamic correlation between different types of population flows. Extensive experiments on 4 datasets demonstrate that CSCNet outperforms the state-of-the-art baselines by reducing the prediction error around 7. 7%∼10. 4%.

PDF Details DOI

AAAI Conference 2019 Conference Paper

DeepSTN+: Context-Aware Spatial-Temporal Neural Network for Crowd Flow Prediction in Metropolis

Ziqian Lin
Jie Feng
Ziyang Lu
Yong Li
Depeng Jin

Crowd flow prediction is of great importance in a wide range of applications from urban planning, traffic control to public safety. It aims to predict the inflow (the traffic of crowds entering a region in a given time interval) and outflow (the traffic of crowds leaving a region for other places) of each region in the city with knowing the historical flow data. In this paper, we propose DeepSTN+, a deep learning-based convolutional model, to predict crowd flows in the metropolis. First, Deep- STN+ employs the ConvPlus structure to model the longrange spatial dependence among crowd flows in different regions. Further, PoI distributions and time factor are combined to express the effect of location attributes to introduce prior knowledge of the crowd movements. Finally, we propose an effective fusion mechanism to stabilize the training process, which further improves the performance. Extensive experimental results based on two real-life datasets demonstrate the superiority of our model, i. e. , DeepSTN+ reduces the error of the crowd flow prediction by approximately 8%∼13% compared with the state-of-the-art baselines.

PDF Details