Tian Zhou

AAAI Conference 2026 Conference Paper

SimDiff: Simpler Yet Better Diffusion Model for Time Series Point Forecasting

• Hang Ding
• Xue Wang
• Tian Zhou
• Tao Yao

Diffusion models have recently shown promise in time series forecasting, particularly for probabilistic predictions. However, they often fail to achieve state-of-the-art point estimation performance compared to regression-based methods. This limitation stems from difficulties in providing sufficient contextual bias to track distribution shifts and in balancing output diversity with the stability and precision required for point forecasts. Existing diffusion-based approaches mainly focus on full-distribution modeling under probabilistic frameworks, often with likelihood maximization objectives, while paying little attention to dedicated strategies for high-accuracy point estimation. Moreover, other existing point prediction diffusion methods frequently rely on pre-trained or jointly trained mature models for contextual bias, sacrificing the generative flexibility of diffusion models. To address these challenges, we propose SimDiff, a single-stage, end-to-end framework. SimDiff employs a single unified Transformer network carefully tailored to serve as both denoiser and predictor, eliminating the need for external pre-trained or jointly trained regressors. It achieves state-of-the-art point estimation performance by leveraging intrinsic output diversity and improving mean squared error accuracy through multiple inference ensembling. Key innovations, including normalization independence and the median-of-means estimator, further enhance adaptability and stability. Extensive experiments demonstrate that SimDiff significantly outperforms existing methods in time series point forecasting.

AAAI Conference 2025 Conference Paper

Unlocking the Power of LSTM for Long Term Time Series Forecasting

• Yaxuan Kong
• Zepu Wang
• Yuqi Nie
• Tian Zhou
• Stefan Zohren
• Yuxuan Liang
• Peng Sun
• Qingsong Wen

Traditional recurrent neural network architectures, such as long short-term memory neural networks (LSTM), have historically held a prominent role in time series forecasting (TSF) tasks. While the recently introduced sLSTM for Natural Language Processing (NLP) introduces exponential gating and memory mixing that are beneficial for long term sequential learning, its potential short memory issue is a barrier to applying sLSTM directly in TSF. To address this, we propose a simple yet efficient algorithm named P-sLSTM, which is built upon sLSTM by incorporating patching and channel independence. These modifications substantially enhance sLSTM's performance in TSF, achieving state-of-the-art results. Furthermore, we provide theoretical justifications for our design, and conduct extensive comparative and analytical experiments to fully validate the efficiency and superior performance of our model.

JBHI Journal 2024 Journal Article

S2VQ-VAE: Semi-Supervised Vector Quantised-Variational AutoEncoder for Automatic Evaluation of Trail Making Test

• Zeshen Tang
• Shiyu Tang
• Haoran Wang
• Renren Li
• Xiaochen Zhang
• Wei Zhang
• Xiao Yuan
• Yaning Zang

Background: Computer-aided detection of cognitive impairment garnered increasing attention, offering older adults in the community access to more objective, ecologically valid, and convenient cognitive assessments using multimodal sensing technology on digital devices. Methodology: In this study, we aimed to develop an automated method for screening cognitive impairment, building on paper- and electronic TMTs. We proposed a novel deep representation learning approach named Semi-Supervised Vector Quantised-Variational AutoEncoder (S2VQ-VAE). Within S2VQ-VAE, we incorporated intra- and inter-class correlation losses to disentangle class-related factors. These factors were then combined with various real-time obtainable features (including demographic, time-related, pressure-related, and jerk-related features) to create a robust feature engineering block. Finally, we identified the light gradient boosting machine as the optimal classifier. The experiments were conducted on a dataset collected from older adults in the community. Results: The experimental results showed that the proposed multi-type feature fusion method outperformed the conventional method used in paper-based TMTs and the existing VAE-based feature extraction in terms of screening performance. Conclusions: In conclusion, the proposed deep representation learning method significantly enhances the cognitive diagnosis capabilities of behavior-based TMTs and streamlines large-scale community-based cognitive impairment screening while reducing the workload of professional healthcare staff.

NeurIPS Conference 2023 Conference Paper

One Fits All: Power General Time Series Analysis by Pretrained LM

• Tian Zhou
• Peisong Niu
• Xue Wang
• Liang Sun
• Rong Jin

Although we have witnessed great success of pre-trained models in natural language processing (NLP) and computer vision (CV), limited progress has been made for general time series analysis. Unlike NLP and CV where a unified model can be used to perform different tasks, specially designed approach still dominates in each time series analysis task such as classification, anomaly detection, forecasting, and few-shot learning. The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model. This model, known as the Frozen Pretrained Transformer (FPT), is evaluated through fine-tuning on all major types of tasks involving time series. Our results demonstrate that pre-trained models on natural language or images can lead to a comparable or state-of-the-art performance in all main time series analysis tasks, as illustrated in Figure1. We also found both theoretically and empirically that the self-attention module behaviors similarly to principle component analysis (PCA), an observation that helps explains how transformer bridges the domain gap and a crucial step towards understanding the universality of a pre-trained transformer. The code is publicly available at https: //anonymous. 4open. science/r/Pretrained-LM-for-TSForcasting-C561.

IJCAI Conference 2023 Conference Paper

Transformers in Time Series: A Survey

• Qingsong Wen
• Tian Zhou
• Chaoli Zhang
• Weiqi Chen
• Ziqing Ma
• Junchi Yan
• Liang Sun

Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance.

NeurIPS Conference 2022 Conference Paper

FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting

• Tian Zhou
• Ziqing Ma
• Xue Wang
• Qingsong Wen
• Liang Sun
• Tao Yao
• Wotao Yin
• Rong Jin

Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise present in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a Frequency improved Legendre Memory model, or FiLM: it applies Legendre polynomial projections to approximate historical information, uses Fourier projection to remove noise, and adds a low-rank approximation to speed up computation. Our empirical studies show that the proposed FiLM significantly improves the accuracy of state-of-the-art models in multivariate and univariate long-term forecasting by (19. 2%, 22. 6%), respectively. We also demonstrate that the representation module developed in this work can be used as a general plugin to improve the long-term prediction performance of other deep learning modules. Code is available at https: //github. com/tianzhou2011/FiLM/.

AAAI Conference 2015 Conference Paper

Touchless Telerobotic Surgery — Is It Possible at All?

• Tian Zhou
• Maria Cabrera
• Juan Wachs