Author name cluster

Seulki Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

8 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

AliO: Output Alignment Matters in Long-Term Time Series Forecasting

Kwangryeol Park
Jaeho Kim
Seulki Lee

Long-term Time Series Forecasting (LTSF) tasks, which leverage the current data sequence as input to predict the future sequence, have become increasingly crucial in real-world applications such as weather forecasting and planning of electricity consumption. However, state-of-the-art LTSF models often fail to achieve prediction output alignment for the same timestamps across lagged input sequences. Instead, these models exhibit low output alignment, resulting in fluctuation in prediction outputs for the same timestamps, undermining the model's reliability. To address this, we propose AliO (Align Outputs), a novel approach designed to improve the output alignment of LTSF models by reducing the discrepancies between prediction outputs for the same timestamps in both the time and frequency domains. To measure output alignment, we introduce a new metric, TAM (Time Alignment Metric), which quantifies the alignment between prediction outputs, whereas existing metrics such as MSE only capture the distance between prediction outputs and ground truths. Experimental results show that AliO effectively improves the output alignment, i. e. , up to 58. 2\% in TAM, while maintaining or enhancing the forecasting performance (up to 27. 5\%). This improved output alignment increases the reliability of the LTSF models, making them more applicable in real-world scenarios. The code implementation is on an anonymous GitHub repository.

PDF Details

ICLR Conference 2025 Conference Paper

PPT: Patch Order Do Matters In Time Series Pretext Task

Jaeho Kim
Kwangryeol Park
Sukmin Yun
Seulki Lee

Recently, patch-based models have been widely discussed in time series analysis. However, existing pretext tasks for patch-based learning, such as masking, may not capture essential time and channel-wise patch interdependencies in time series data, presumed to result in subpar model performance. In this work, we introduce *Patch order-aware Pretext Task (PPT)*, a new self-supervised patch order learning pretext task for time series classification. PPT exploits the intrinsic sequential order information among patches across time and channel dimensions of time series data, where model training is aided by channel-wise patch permutations. The permutation disrupts patch order consistency across time and channel dimensions with controlled intensity to provide supervisory signals for learning time series order characteristics. To this end, we propose two patch order-aware learning methods: patch order consistency learning, which quantifies patch order correctness, and contrastive learning, which distinguishes weakly permuted patch sequences from strongly permuted ones. With patch order learning, we observe enhanced model performance, e.g., improving up to 7% accuracy for the supervised cardiogram task and outperforming mask-based learning by 5% in the self-supervised human activity recognition task. We also propose ACF-CoS, an evaluation metric that measures the *importance of orderness* for time series datasets, which enables pre-examination of the efficacy of PPT in model training.

Details

AAAI Conference 2025 Conference Paper

SMMF: Square-Matricized Momentum Factorization for Memory-Efficient Optimization

Kwangryeol Park
Seulki Lee

We propose SMMF (Square-Matricized Momentum Factorization), a memory-efficient optimizer that reduces the memory requirement of the widely used adaptive learning rate optimizers, such as Adam, by up to 96%. SMMF enables flexible and efficient factorization of an arbitrary rank (shape) of the first and second momentum tensors during optimization, based on the proposed square-matricization and one-time single matrix factorization. From this, it becomes effectively applicable to any rank (shape) of momentum tensors, i.e., bias, matrix, and any rank-d tensors, prevalent in various deep model architectures, such as CNNs (high rank) and Transformers (low rank), in contrast to existing memory-efficient optimizers that applies only to a particular (rank-2) momentum tensor, e.g., linear layers. We conduct a regret bound analysis of SMMF, which shows that it converges similarly to non-memory-efficient adaptive learning rate optimizers, such as AdamNC, providing a theoretical basis for its competitive optimization capability. In our experiment, SMMF takes up to 96% less memory compared to state-of-the-art memoryefficient optimizers, e.g., Adafactor, CAME, and SM3, while achieving comparable model performance on various CNN and Transformer tasks.

PDF Details DOI

ICML Conference 2025 Conference Paper

TransPL: VQ-Code Transition Matrices for Pseudo-Labeling of Time Series Unsupervised Domain Adaptation

Jaeho Kim
Seulki Lee

Unsupervised domain adaptation (UDA) for time series data remains a critical challenge in deep learning, with traditional pseudo-labeling strategies failing to capture temporal patterns and channel-wise shifts between domains, producing sub-optimal pseudo labels. As such, we introduce TransPL, a novel approach that addresses these limitations by modeling the joint distribution $P(X, y)$ of the source domain through code transition matrices, where the codes are derived from vector quantization (VQ) of time series patches. Our method constructs class- and channel-wise code transition matrices from the source domain and employs Bayes’ rule for target domain adaptation, generating pseudo-labels based on channel-wise weighted class-conditional likelihoods. TransPL offers three key advantages: explicit modeling of temporal transitions and channel-wise shifts between different domains, versatility towards different UDA scenarios (e. g. , weakly-supervised UDA), and explainable pseudo-label generation. We validate TransPL’s effectiveness through extensive analysis on four time series UDA benchmarks and confirm that it consistently outperforms state-of-the-art pseudo-labeling methods by a strong margin (6. 1% accuracy improvement, 4. 9% F1 improvement), while providing interpretable insights into the domain adaptation process through its learned code transition matrices.

Details

NeurIPS Conference 2023 Conference Paper

Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks

Changhyeon Lee
Seulki Lee

In this paper, we propose to approximate the softmax output, which is the key product of the attention mechanism, to reduce its activation memory usage when training attention-based networks (aka Transformers). During the forward pass of the network, the proposed softmax output approximation method stores only a small fraction of the entire softmax output required for back-propagation and evicts the rest of the softmax output from memory. Then, during the backward pass, the evicted softmax activation output is approximated to compose the gradient to perform back-propagation for model training. Considering most attention-based models heavily rely on the softmax-based attention module that usually takes one of the biggest portions of the network, approximating the softmax activation output can be a simple yet effective way to decrease the training memory requirement of many attention-based networks. The experiment with various attention-based models and relevant tasks, i. e. , machine translation, text classification, and sentiment analysis, shows that it curtails the activation memory usage of the softmax-based attention module by up to 84% (6. 2× less memory) in model training while achieving comparable or better performance, e. g. , up to 5. 4% higher classification accuracy.

PDF Details

JBHI Journal 2020 Journal Article

Portable Detection of Apnea and Hypopnea Events Using Bio-Impedance of the Chest and Deep Learning

Tom Van Steenkiste
Willemijn Groenendaal
Pauline Dreesen
Seulki Lee
Susie Klerkx
Ruben de Francisco
Dirk Deschrijver
Tom Dhaene

Sleep apnea is one of the most common sleep-related breathing disorders. It is diagnosed through an overnight sleep study in a specialized sleep clinic. This setup is expensive and the number of beds and staff are limited, leading to a long waiting time. To enable more patients to be tested, and repeated monitoring for diagnosed patients, portable sleep monitoring devices are being developed. These devices automatically detect sleep apnea events in one or more respiration-related signals. There are multiple methods to measure respiration, with varying levels of signal quality and comfort for the patient. In this study, the potential of using the bio-impedance (bioZ) of the chest as a respiratory surrogate is analyzed. A novel portable device is presented, combined with a two-phase Long Short-Term Memory (LSTM) deep learning algorithm for automated event detection. The setup is benchmarked using simultaneous recordings of the device and the traditional polysomnography in 25 patients. The results demonstrate that using only the bioZ, an area under the precision-recall curve of 46. 9% can be achieved, which is on par with automatic scoring using a polysomnography respiration channel. The sensitivity, specificity and accuracy are 58. 4%, 76. 2% and 72. 8% respectively. This confirms the potential of using the bioZ device and deep learning algorithm for automatically detecting sleep respiration events during the night, in a portable and comfortable setup.

Details DOI

EAAI Journal 2019 Journal Article

Process monitoring using variational autoencoder for high-dimensional nonlinear processes

Seulki Lee
Mingu Kwak
Kwok-Leung Tsui
Seoung Bum Kim

In many industries, statistical process monitoring techniques play a key role in improving processes through variation reduction and defect prevention. Modern large-scale industrial processes require appropriate monitoring techniques that can efficiently address high-dimensional nonlinear processes. Such processes have been successfully monitored with several latent variable-based methods. However, because these monitoring methods use Hotelling’s T 2 statistics in the reduced space, a normality assumption underlies the construction of these tools. This assumption has limited the use of latent variable-based monitoring charts in both nonlinear and nonnormal situations. In this study, we propose a variational autoencoder (VAE) as a monitoring method that can address both nonlinear and nonnormal situations in high-dimensional processes. VAE is appropriate for T 2 charts because it causes the reduced space to follow a multivariate normal distribution. The effectiveness and applicability of the proposed VAE-based chart were demonstrated through experiments on simulated data and real data from a thin-film-transistor liquid-crystal display process.

Details DOI

EAAI Journal 2018 Journal Article

Time-adaptive support vector data description for nonstationary process monitoring

Seulki Lee
Seoung Bum Kim

Statistical process control techniques are widely used for quality control to monitor the stability of a process over time. In modern manufacturing systems with complex and variable processes, appropriate control chart techniques that can efficiently address nonnormal processes are required. Furthermore, in real manufacturing environments, process changes occur frequently because of various factors such as product and setpoint changes, catalyst degradation, seasonal variations, and sensor drift. However, conventional control chart schemes cannot necessarily accommodate all possible future conditions of a process because they are formulated based on information recorded in the early stages of the process. Several attempts have been made to accommodate process changes over time. In the present paper, we propose a time-adaptive support vector data description-based control chart that can address not only nonnormal in-control observations, but also time-varying processes. The effectiveness and applicability of the proposed chart was demonstrated through experiments with simulated data and real data from the metal frame process in mobile device manufacturing.

Details DOI