Author name cluster

Zitao Liu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

21 papers

1 author row

AAAI Conference 2026 Conference Paper

A Compress-Expand Framework for Automatic Lesson Plan Generation

Shuyan Huang
Ying Zheng
Xiaoli Zeng
Zitao Liu

Creating a well-structured lesson plan is essential for improving classroom efficiency, yet it is often a labor-intensive process. Recently, many studies have leveraged large language models (LLMs) to generate lesson plans automatically. However, existing methods heavily rely on LLMs that are pre-trained on large-scale universal corpora, which often lack critical educational theory and textbook-specific information. This can lead to inconsistencies and misalignments with textbook content. To address these challenges, we propose CE-LessonPlan, a novel compress-expand framework to generate lesson plans by effectively combining external lesson plan references and textbook information. The framework consists of two key components: a compressor, which synthesizes multiple retrieved references into a cohesive document, and an expander, which integrates textbook-specific information with the parametric knowledge of LLMs to produce another enriched lesson plan. The outputs of the compressor and expander are then seamlessly integrated to create a comprehensive golden context, further enhancing the lesson plan generation process with LLMs. We conduct extensive experiments to demonstrate that CE-LessonPlan outperforms existing methods for generating lesson plans.

PDF Details DOI

TIST Journal 2026 Journal Article

Improving Knowledge Tracing through Multi-Source Scaling with Decoder-Only Transformers

Teng Guo
Bojun Zhan
Shuyan Huang
Jiahao Chen
Xiangyu Zhao
Mingliang Hou
Zitao Liu

Knowledge tracing (KT) is a problem of modeling students’ knowledge states to predict their future performance by observing their historical learning interactions. The collection of educational data presents significant challenges, as students’ limited learning engagement restricts the generation of large-scale interaction data, while stringent privacy regulations further limit the availability of student learning sequences from online platforms. Hence, it is crucial to enhance the capabilities of deep learning-based KT (DLKT) models by constructing large-scale datasets through the integration of student interaction data across multiple subjects and sources. The success of ChatGPT demonstrates that the decoder-only Transformer architecture is highly effective in capturing complex information from large-scale sequential data. Against this background, we propose a novel decoder-only Transformer architecture-based model, named Unified DLKT ( UniKT ), to learn coherent and unified representations across a wide range of data sources. Specifically, we combine student learning sequences from six educational scenarios and utilize a multi-source encoding to learn unified representations of interactions from mixed data. UniKT is a stack of Transformer decoder layers for handling long-term dependencies among students’ historical interactions and future performance. We evaluate UniKT on six publicly available real-world educational datasets, and experimental results demonstrate that our method outperforms the majority of existing DLKT models in terms of AUC and accuracy. Furthermore, the empirical analysis shows the strong transferability and adaptability of UniKT in learning from multiple sources. To encourage reproducible research, we make our data and code publicly available at https://pykt.org/.

Details DOI

AAAI Conference 2026 Conference Paper

Learning from Scoring Disagreements: Contrastive Error Mining for Efficient and Robust LLM-based Assessment

Lei Chen
Tengteng Cheng
BoYu Gao
Zitao Liu
Weiqi Luo

Automated grading of student responses still faces numerous challenges, particularly when dealing with complex and ambiguous answers. In particular, large models are prone to scoring bias when handling uncertain responses, and few-shot reasoning methods often lack stability, which limits their applicability in real educational scenarios. To tackle these challenges, we propose the Contrastive Error Mining and FineTuning (CEM-FT) framework, which automatically identifies high-value hard samples by analyzing scoring disagreements between a full fine-tuned model and a few-shot model. A lightweight LoRA adapter is then trained on these samples to refine model performance with minimal computational overhead. Experiments on the SciEntsbank, Beetle, and Mohler datasets show that CEM-FT can improve QWK by up to 3.9% compared to the fine-tuned Qwen model on SciEntsbank datasets, which is a significant improvement over the few-shot baseline. The proposed framework substantially enhances both scoring accuracy and consistency, providing a practical, robust solution for reliable automated assessment with large language models.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Cognitive Fluctuations Enhanced Attention Network for Knowledge Tracing

Mingliang Hou
Xueyi Li
Teng Guo
Zitao Liu
Mi Tian
Renqiang Luo
Weiqi Luo

Knowledge tracing (KT) involves using the historical records of student-learning interactions to anticipate their performance on forthcoming questions. Central to this process is the modeling of human cognition to gain deeper insights into how knowledge is acquired and retained. Human cognition is characterized by two key features: long-term cognitive trends, reflecting the gradual accumulation and stabilization of knowledge over time, and short-term cognitive fluctuations, which arise from transient factors such as forgetting or momentary lapses in attention. Although existing attention-based KT models effectively capture long-term cognitive trends, they often fail to adequately address short-term cognitive fluctuations. These limitations lead to overly smoothed cognitive features and reduced model performance, especially when the test data length exceeds the training data length. To address these problems, we propose FlucKT, a novel short-term cognitive fluctuations enhanced attention network for KT tasks. FlucKT improves the attention mechanism in two ways: First, by using a decomposition-based layer with causal convolution to separate and dynamically reweight long-term and short-term cognitive features. Second, by introducing a kernelized bias attention score penalty to enhance focus on short-term fluctuations, improving length generalization capabilities. Our contributions are validated through extensive experiments on three real-world datasets, demonstrating significant improvements in length generalization and prediction performance.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning

Xiao Han
ZIMO ZHAO
Wanyu Wang
Maolin Wang
Zitao Liu
Yi Chang
Xiangyu Zhao

Recent advancements in Large Language Models (LLMs) have emphasized the critical role of fine-tuning (FT) techniques in adapting LLMs to specific tasks, especially when retraining from scratch is computationally infeasible. Fine-tuning enables LLMs to leverage task- or domain-specific data, producing models that more effectively meet the requirements of targeted applications. However, conventional FT approaches often suffer from catastrophic forgetting and suboptimal data efficiency, limiting their real-world applicability. To address these challenges, this paper proposes DEAL, a novel framework that integrates Low-Rank Adaptation (LoRA) with a continuous fine-tuning strategy. By incorporating knowledge retention and adaptive parameter update modules, the framework mitigates the limitations of existing FT methods while maintaining efficiency. Experiments on 15 diverse datasets show that DEAL consistently outperforms baseline methods, yielding substantial gains in task accuracy and resource efficiency. These findings demonstrate the potential of our approach to advance continual adaptation in LLMs by enhancing task performance while improving resource efficiency. The source code is publicly available at https: //github. com/Applied-Machine-Learning-Lab/DEAL.

PDF Details

IJCAI Conference 2025 Conference Paper

Denoised Attention and Question-Augmented Representations for Knowledge Tracing

Jiwei Deng
Youheng Bai
Mingliang Hou
Teng Guo
Zitao Liu
Weiqi Luo

Knowledge tracing (KT) is an essential task in online education systems. It aims to predict the future performance of students based on their historical learning interaction data. Despite significant advancements in attention-based KT models, they still face some limitations: inaccurate input representation and excessive student forgetting modeling. These limitations often lead to the attention noise problem: the model assigns non-negligible attention weight to some information that is cognitively irrelevant in nature, thereby generating interference signals. To address this problem, we propose a novel KT model, i. e. , DenoiseKT. DenoiseKT effectively models the difficulty of the questions and utilizes graph neural network to capture the complex relationship between questions, thereby refining the representations of input features. Additionally, the denoised attention mechanism introduces a weight factor to reduce the model's attention weight distribution on irrelevant information. We extensively compare DenoiseKT with 22 state-of-the-art KT models on 4 widely-used public datasets. Experimental results show that DenoiseKT can effectively solve the attention noise problem and outperform other models. The source code of DenoiseKT is available at https: //pykt. org.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Rethinking and Improving Student Learning and Forgetting Processes for Attention based Knowledge Tracing Models

Youheng Bai
Xueyi Li
Zitao Liu
Yaying Huang
Mi Tian
Weiqi Luo

Knowledge tracing (KT) models students' knowledge states and predicts their future performance based on their historical interaction data. However, attention based KT models struggle to accurately capture diverse forgetting behaviors in ever-growing interaction sequences. First, existing models use uniform time decay matrices, conflating forgetting representations with problem relevance. Second, the fixed-length window prediction paradigm fails to model continuous forgetting processes in expanding sequences. To address these challenges, this paper introduces LefoKT, a unified architecture that enhances attention based KT models by incorporating proposed relative forgetting attention. LefoKT improves forgetting modeling through relative forgetting attention to decouple forgetting patterns from problem relevance. It also enhances attention based KT models' length extrapolation capability for capturing continuous forgetting processes in ever-growing interaction sequences. Extensive experimental results on three datasets validate the effectiveness of LefoKT.

PDF Details DOI

AAAI Conference 2025 Conference Paper

SIGMA: Selective Gated Mamba for Sequential Recommendation

Ziwei Liu
Qidong Liu
Yejing Wang
Wanyu Wang
Pengyue Jia
Maolin Wang
Zitao Liu
Yi Chang

Sequential Recommender Systems (SRS) has stood out as a highly promising technique in numerous domains due to its impressive capability of capturing complex user preferences. Current SRS have employed transformer-based models to give the next-item prediction. Nevertheless, its quadratic computational complexity has often resulted in notable inefficiencies, posing a significant obstacle to real-time recommendation processes. Recently, Mamba has demonstrated its exceptional effectiveness in time series prediction, delivering substantial improvements in both efficiency and effectiveness. However, directly applying Mamba to SRS poses certain challenges. Its unidirectional structure may impede the ability to capture contextual information in user-item interactions, while its instability in state estimation may hinder the ability to capture short-term patterns in interaction sequences. To address these issues, we propose a novel framework called Selective Gated Mamba for Sequential Recommendation (SIGMA). By introducing the Partially Flipped Mamba (PF-Mamba), we construct a special bi-directional structure to address the context modeling challenge. Then, to consolidate PF-Mamba's performance, we employed an input-dependent Dense Selective Gate (DS Gate) to allocate the weights of the two directions and further filter the sequential information. Moreover, for short sequence modeling, we devise a Feature Extract GRU (FE-GRU) to capture the short-term dependencies. Experimental results demonstrate that SIGMA significantly outperforms existing baselines across five real-world datasets. Our implementation code is available in Supplementary Material to ease reproducibility.

PDF Details DOI

AAAI Conference 2025 Conference Paper

What Are Step-Level Reward Models Rewarding? Counterintuitive Findings from MCTS-Boosted Mathematical Reasoning

Yiran Ma
Zui Chen
Tianqiao Liu
Mi Tian
Zhuo Liu
Zitao Liu
Weiqi Luo

Step-level reward models (SRMs) can significantly enhance mathematical reasoning performance through process supervision or step-level preference alignment based on reinforcement learning. The performance of SRMs is pivotal, as they serve as critical guidelines, ensuring that each step in the reasoning process is aligned with desired outcomes. Recently, AlphaZero-like methods, where Monte Carlo Tree Search (MCTS) is employed for automatic step-level preference annotation, have proven particularly effective. However, the precise mechanisms behind the success of SRMs remain largely unexplored. To address this gap, this study delves into the counterintuitive aspects of SRMs, particularly focusing on MCTS-based approaches. Our findings reveal that the removal of natural language descriptions of thought processes has minimal impact on the efficacy of SRMs. Furthermore, we demonstrate that SRMs are adept at assessing the complex logical coherence present in mathematical language while having difficulty in natural language. These insights provide a nuanced understanding of the core elements that drive effective step-level reward modeling in mathematical reasoning. By shedding light on these mechanisms, this study offers valuable guidance for developing more efficient and streamlined SRMs, which can be achieved by focusing on the crucial parts of mathematical reasoning.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Enhancing Length Generalization for Attention Based Knowledge Tracing Models with Linear Biases

Xueyi Li
Youheng Bai
Teng Guo
Zitao Liu
Yaying Huang
Xiangyu Zhao
Feng Xia
Weiqi Luo

Knowledge tracing (KT) is the task of predicting students' future performance based on their historical learning interaction data. With the rapid advancement of attention mechanisms, many attention based KT models are developed. However, existing attention based KT models exhibit performance drops as the number of student interactions increases beyond the number of interactions on which the KT models are trained. We refer to this as the length generalization of KT model. In this paper, we propose stableKT to enhance length generalization that is able to learn from short sequences and maintain high prediction performance when generalizing on long sequences. Furthermore, we design a multi-head aggregation module to capture the complex relationships between questions and the corresponding knowledge components (KCs) by combining dot-product attention and hyperbolic attention. Experimental results on three public educational datasets show that our model exhibits robust capability of length generalization and outperforms all baseline models in terms of AUC. To encourage reproducible research, we make our data and code publicly available at https: //pykt. org.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Improving Interpretability of Deep Sequential Knowledge Tracing Models with Question-centric Cognitive Representations

Jiahao Chen
Zitao Liu
Shuyan Huang
Qiongqiong Liu
Weiqi Luo

Knowledge tracing (KT) is a crucial technique to predict students’ future performance by observing their historical learning processes. Due to the powerful representation ability of deep neural networks, remarkable progress has been made by using deep learning techniques to solve the KT problem. The majority of existing approaches rely on the homogeneous question assumption that questions have equivalent contributions if they share the same set of knowledge components. Unfortunately, this assumption is inaccurate in real-world educational scenarios. Furthermore, it is very challenging to interpret the prediction results from the existing deep learning based KT models. Therefore, in this paper, we present QIKT, a question-centric interpretable KT model to address the above challenges. The proposed QIKT approach explicitly models students’ knowledge state variations at a ﬁne-grained level with question-sensitive cognitive representations that are jointly learned from a question-centric knowledge acquisition module and a question-centric problem solving module. Meanwhile, the QIKT utilizes an item response theory based prediction layer to generate interpretable prediction results. The proposed QIKT model is evaluated on three public real-world educational datasets. The results demonstrate that our approach is superior on the KT prediction task, and it outperforms a wide range of deep learning based KT models in terms of prediction accuracy with better model interpretability. To encourage reproducible results, we have provided all the datasets and code at https://pykt.org/.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

XES3G5M: A Knowledge Tracing Benchmark Dataset with Auxiliary Information

Zitao Liu
Qiongqiong Liu
Teng Guo
Jiahao Chen
Shuyan Huang
Xiangyu Zhao
Jiliang Tang
Weiqi Luo

Knowledge tracing (KT) is a task that predicts students' future performance based on their historical learning interactions. With the rapid development of deep learning techniques, existing KT approaches follow a data-driven paradigm that uses massive problem-solving records to model students' learning processes. However, although the educational contexts contain various factors that may have an influence on student learning outcomes, existing public KT datasets mainly consist of anonymized ID-like features, which may hinder the research advances towards this field. Therefore, in this work, we present, \emph{XES3G5M}, a large-scale dataset with rich auxiliary information about questions and their associated knowledge components (KCs)\footnote{\label{ft: kc}A KC is a generalization of everyday terms like concept, principle, fact, or skill. }. The XES3G5M dataset is collected from a real-world online math learning platform, which contains 7, 652 questions, and 865 KCs with 5, 549, 635 interactions from 18, 066 students. To the best of our knowledge, the XES3G5M dataset not only has the largest number of KCs in math domain but contains the richest contextual information including tree structured KC relations, question types, textual contents and analysis and student response timestamps. Furthermore, we build a comprehensive benchmark on 19 state-of-the-art deep learning based knowledge tracing (DLKT) models. Extensive experiments demonstrate the effectiveness of leveraging the auxiliary information in our XES3G5M with DLKT models. We hope the proposed dataset can effectively facilitate the KT research work.

PDF Details

NeurIPS Conference 2022 Conference Paper

pyKT: A Python Library to Benchmark Deep Learning based Knowledge Tracing Models

Zitao Liu
Qiongqiong Liu
Jiahao Chen
Shuyan Huang
Jiliang Tang
Weiqi Luo

Knowledge tracing (KT) is the task of using students' historical learning interaction data to model their knowledge mastery over time so as to make predictions on their future interaction performance. Recently, remarkable progress has been made of using various deep learning techniques to solve the KT problem. However, the success behind deep learning based knowledge tracing (DLKT) approaches is still left somewhat unknown and proper measurement and analysis of these DLKT approaches remain a challenge. First, data preprocessing procedures in existing works are often private and custom, which limits experimental standardization. Furthermore, existing DLKT studies often differ in terms of the evaluation protocol and are far away real-world educational contexts. To address these problems, we introduce a comprehensive python based benchmark platform, \textsc{pyKT}, to guarantee valid comparisons across DLKT methods via thorough evaluations. The \textsc{pyKT} library consists of a standardized set of integrated data preprocessing procedures on 7 popular datasets across different domains, and 10 frequently compared DLKT model implementations for transparent experiments. Results from our fine-grained and rigorous empirical KT studies yield a set of observations and suggestions for effective DLKT, e. g. , wrong evaluation setting may cause label leakage that generally leads to performance inflation; and the improvement of many DLKT approaches is minimal compared to the very first DLKT model proposed by Piech et al. \cite{piech2015deep}. We have open sourced \textsc{pyKT} and our experimental results at \url{https: //pykt. org/}. We welcome contributions from other research groups and practitioners.

PDF Details

AAAI Conference 2021 Conference Paper

Contrastive Clustering

Yunfan Li
Peng Hu
Zitao Liu
Dezhong Peng
Joey Tianyi Zhou
Xi Peng

In this paper, we propose an online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature matrix could be regarded as soft labels of instances, and accordingly the columns could be further regarded as cluster representations. By simultaneously optimizing the instance- and cluster-level contrastive loss, the model jointly learns representations and cluster assignments in an end-to-end manner. Besides, the proposed method could timely compute the cluster assignment for each individual, even when the data is presented in streams. Extensive experimental results show that CC remarkably outperforms 17 competitive clustering methods on six challenging image benchmarks. In particular, CC achieves an NMI of 0. 705 (0. 431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline. The code is available at https: //github. com/XLearning-SCU/2021-AAAI-CC.

PDF Details

NeurIPS Conference 2021 Conference Paper

Graph Neural Networks with Adaptive Residual

Xiaorui Liu
Jiayuan Ding
Wei Jin
Han Xu
Yao Ma
Zitao Liu
Jiliang Tang

Graph neural networks (GNNs) have shown the power in graph representation learning for numerous tasks. In this work, we discover an interesting phenomenon that although residual connections in the message passing of GNNs help improve the performance, they immensely amplify GNNs' vulnerability against abnormal node features. This is undesirable because in real-world applications, node features in graphs could often be abnormal such as being naturally noisy or adversarially manipulated. We analyze possible reasons to understand this phenomenon and aim to design GNNs with stronger resilience to abnormal features. Our understandings motivate us to propose and derive a simple, efficient, interpretable, and adaptive message passing scheme, leading to a novel GNN with Adaptive Residual, AirGNN. Extensive experiments under various abnormal feature scenarios demonstrate the effectiveness of the proposed algorithm.

PDF Details

NeurIPS Conference 2020 Conference Paper

CLEARER: Multi-Scale Neural Architecture Search for Image Restoration

Yuanbiao Gou
Boyun Li
Zitao Liu
Songfan Yang
Xi Peng

Multi-scale neural networks have shown effectiveness in image restoration tasks, which are usually designed and integrated in a handcrafted manner. Different from the existing labor-intensive handcrafted architecture design paradigms, we present a novel method, termed as multi-sCaLe nEural ARchitecture sEarch for image Restoration (CLEARER), which is a speciﬁcally designed neural architecture search (NAS) for image restoration. Our contributions are twofold. On one hand, we design a multi-scale search space that consists of three task-ﬂexible modules. Namely, 1) Parallel module that connects multi-resolution neural blocks in parallel, while preserving the channels and spatial-resolution in each neural block, 2) Transition module remains the existing multi-resolution features while extending them to a lower resolution, 3) Fusion module integrates multi-resolution features by passing the features of the parallel neural blocks to the current neural blocks. On the other hand, we present novel losses which could 1) balance the tradeoff between the model complexity and performance, which is highly expected to image restoration; and 2) relax the discrete architecture parameters into a continuous distribution which approximates to either 0 or 1. As a result, a differentiable strategy could be employed to search when to fuse or extract multi-resolution features, while the discretization issue faced by the gradient-based NAS could be alleviated. The proposed CLEARER could search a promising architecture in two GPU hours. Extensive experiments show the promising performance of our method comparing with nine image denoising methods and eight image deraining approaches in quantitative and qualitative evaluations. The codes are available at https: //github. com/limit-scu.

PDF Details

AAAI Conference 2020 Conference Paper

Graduate Employment Prediction with Bias

Teng Guo
Feng Xia
Shihao Zhen
Xiaomei Bai
Dongyu Zhang
Zitao Liu
Jiliang Tang

The failure of landing a job for college students could cause serious social consequences such as drunkenness and suicide. In addition to academic performance, unconscious biases can become one key obstacle for hunting jobs for graduating students. Thus, it is necessary to understand these unconscious biases so that we can help these students at an early stage with more personalized intervention. In this paper, we develop a framework, i. e. , MAYA (Multi-mAjor emploYment stAtus) to predict students’ employment status while considering biases. The framework consists of four major components. Firstly, we solve the heterogeneity of student courses by embedding academic performance into a uniﬁed space. Then, we apply a generative adversarial network (GAN) to overcome the class imbalance problem. Thirdly, we adopt Long Short-Term Memory (LSTM) with a novel dropout mechanism to comprehensively capture sequential information among semesters. Finally, we design a bias-based regularization to capture the job market biases. We conduct extensive experiments on a large-scale educational dataset and the results demonstrate the effectiveness of our prediction framework.

PDF Details

AAAI Conference 2020 Conference Paper

Learning Multi-Level Dependencies for Robust Word Recognition

Zhiwei Wang
Hui Liu
Jiliang Tang
Songfan Yang
Gale Yan Huang
Zitao Liu

Robust language processing systems are becoming increasingly important given the recent awareness of dangerous situations where brittle machine learning models can be easily broken with the presence of noises. In this paper, we introduce a robust word recognition framework that captures multi-level sequential dependencies in noised sentences. The proposed framework employs a sequence-to-sequence model over characters of each word, whose output is given to a word-level bi-directional recurrent neural network. We conduct extensive experiments to verify the effectiveness of the framework. The results show that the proposed framework outperforms state-of-the-art methods by a large margin and they also suggest that character-level dependencies can play an important role in word recognition. The code of the proposed framework and the major experiments are publicly available1.

PDF Details

AAAI Conference 2016 Conference Paper

Learning Adaptive Forecasting Models from Irregularly Sampled Multivariate Clinical Data

Zitao Liu
Milos Hauskrecht

Building accurate predictive models of clinical multivariate time series is crucial for understanding of the patient condition, the dynamics of a disease, and clinical decision making. A challenging aspect of this process is that the model should be ﬂexible and adaptive to reﬂect well patient-speciﬁc temporal behaviors and this also in the case when the available patient-speciﬁc data are sparse and short span. To address this problem we propose and develop an adaptive two-stage forecasting approach for modeling multivariate, irregularly sampled clinical time series of varying lengths. The proposed model (1) learns the population trend from a collection of time series for past patients; (2) captures individual-speciﬁc short-term multivariate variability; and (3) adapts by automatically adjusting its predictions based on new observations. The proposed forecasting model is evaluated on a real-world clinical time series dataset. The results demonstrate that our approach is superior on the prediction tasks for multivariate, irregularly sampled clinical time series, and it outperforms both the population based and patient-speciﬁc time series prediction models in terms of prediction accuracy.

PDF Details

AAAI Conference 2015 Conference Paper

A Regularized Linear Dynamical System Framework for Multivariate Time Series Analysis

Zitao Liu
Milos Hauskrecht

Linear Dynamical System (LDS) is an elegant mathematical framework for modeling and learning Multivariate Time Series (MTS). However, in general, it is difficult to set the dimension of an LDS’s hidden state space. A small number of hidden states may not be able to model the complexities of a MTS, while a large number of hidden states can lead to overfitting. In this paper, we study learning methods that impose various regularization penalties on the transition matrix of the LDS model and propose a regularized LDS learning framework (rLDS) which aims to (1) automatically shut down LDSs’ spurious and unnecessary dimensions, and consequently, address the problem of choosing the optimal number of hidden states; (2) prevent the overfitting problem given a small amount of MTS data; and (3) support accurate MTS forecasting. To learn the regularized LDS from data we incorporate a second order cone program and a generalized gradient descent method into the Maximum a Posteriori framework and use Expectation Maximization to obtain a low-rank transition matrix of the LDS model. We propose two priors for modeling the matrix which lead to two instances of our rLDS. We show that our rLDS is able to recover well the intrinsic dimensionality of the time series dynamics and it improves the predictive performance when compared to baselines on both synthetic and real-world MTS datasets.

PDF Details

AIIM Journal 2015 Journal Article

Clinical time series prediction: Toward a hierarchical dynamical system framework

Zitao Liu
Milos Hauskrecht

Objective Developing machine learning and data mining algorithms for building temporal models of clinical time series is important for understanding of the patient condition, the dynamics of a disease, effect of various patient management interventions and clinical decision making. In this work, we propose and develop a novel hierarchical framework for modeling clinical time series data of varied length and with irregularly sampled observations. Materials and methods Our hierarchical dynamical system framework for modeling clinical time series combines advantages of the two temporal modeling approaches: the linear dynamical system and the Gaussian process. We model the irregularly sampled clinical time series by using multiple Gaussian process sequences in the lower level of our hierarchical framework and capture the transitions between Gaussian processes by utilizing the linear dynamical system. The experiments are conducted on the complete blood count (CBC) panel data of 1000 post-surgical cardiac patients during their hospitalization. Our framework is evaluated and compared to multiple baseline approaches in terms of the mean absolute prediction error and the absolute percentage error. Results We tested our framework by first learning the time series model from data for the patients in the training set, and then using it to predict future time series values for the patients in the test set. We show that our model outperforms multiple existing models in terms of its predictive accuracy. Our method achieved a 3. 13% average prediction accuracy improvement on ten CBC lab time series when it was compared against the best performing baseline. A 5. 25% average accuracy improvement was observed when only short-term predictions were considered. Conclusion A new hierarchical dynamical system framework that lets us model irregularly sampled time series data is a promising new direction for modeling clinical time series and for improving their predictive performance.

Details DOI