Author name cluster

Dongha Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

12 papers

1 author row

NeurIPS Conference 2025 Conference Paper

Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning

Yeongbin Seo
Dongha Lee
Jaehyung Kim
Jinyoung Yeo

Autoregressive (AR) language models generate text one token at a time, which limits their inference speed. Diffusion-based language models offer a promising alternative, as they can decode multiple tokens in parallel. However, we identify a key bottleneck in current diffusion LMs: the \textbf{long decoding-window problem}, where tokens generated far from the input context often become irrelevant or repetitive. Previous solutions like semi-autoregressive address this issue by splitting windows into blocks (sacrificing bidirectionality), but we find that this also leads to \textbf{time-interval expansion problem}, sacrificing the speed. Therefore, semi-AR eliminates the main advantages of diffusion models. To overcome this, we propose Convolutional decoding (\textit{Conv}), a normalization-based method that narrows the decoding window without hard segmentation, leading to better fluency and flexibility. Additionally, we introduce Rejecting Rule-based Fine-Tuning (R2FT), a post-hoc training scheme that better aligns tokens at positions far from context. Our methods achieve state-of-the-art results on open-ended generation benchmarks (e. g. , AlpacaEval) among diffusion LM baselines, with significantly lower step size than previous works, demonstrating both speed and quality improvements. The code is available online (\url{https: //github. com/ybseo-ac/Conv}).

PDF Details

AAAI Conference 2024 Conference Paper

Large Language Models Are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales

Taeyoon Kwon
Kai Tzu-iunn Ong
Dongjin Kang
Seungjun Moon
Jeong Ryong Lee
Dosik Hwang
Beomseok Sohn
Yongsik Sim

Machine reasoning has made great progress in recent years owing to large language models (LLMs). In the clinical domain, however, most NLP-driven projects mainly focus on clinical classification or reading comprehension, and under-explore clinical reasoning for disease diagnosis due to the expensive rationale annotation with clinicians. In this work, we present a "reasoning-aware" diagnosis framework that rationalizes the diagnostic process via prompt-based learning in a time- and labor-efficient manner, and learns to reason over the prompt-generated rationales. Specifically, we address the clinical reasoning for disease diagnosis, where the LLM generates diagnostic rationales providing its insight on presented patient data and the reasoning path towards the diagnosis, namely Clinical Chain-of-Thought (Clinical CoT). We empirically demonstrate LLMs/LMs' ability of clinical reasoning via extensive experiments and analyses on both rationale generation and disease diagnosis in various settings. We further propose a novel set of criteria for evaluating machine-generated rationales' potential for real-world clinical settings, facilitating and benefiting future research in this area.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Multi-Domain Recommendation to Attract Users via Domain Preference Modeling

Hyunjun Ju
SeongKu Kang
Dongha Lee
Junyoung Hwang
Sanghwan Jang
Hwanjo Yu

Recently, web platforms are operating various service domains simultaneously. Targeting a platform that operates multiple service domains, we introduce a new task, Multi-Domain Recommendation to Attract Users (MDRAU), which recommends items from multiple ``unseen'' domains with which each user has not interacted yet, by using knowledge from the user's ``seen'' domains. In this paper, we point out two challenges of MDRAU task. First, there are numerous possible combinations of mappings from seen to unseen domains because users have usually interacted with a different subset of service domains. Second, a user might have different preference for each of the target unseen domains, which requires recommendations to reflect users' preference on domains as well as items. To tackle these challenges, we propose DRIP framework that models users' preference at two levels (i.e., domain and item) and learns various seen-unseen domain mappings in a unified way with masked domain modeling. Our extensive experiments demonstrate the effectiveness of DRIP in MDRAU task and its ability to capture users' domain-level preferences.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning

Yeongbin Seo
Dongha Lee
Jinyoung Yeo

Previous studies on continual knowledge learning (CKL) in large language models (LLMs) have predominantly focused on approaches such as regularization, architectural modifications, and rehearsal techniques to mitigate catastrophic forgetting. However, these methods naively inherit the inefficiencies of standard training procedures, indiscriminately applying uniform weight across all tokens, which can lead to unnecessary parameter updates and increased forgetting. To address these shortcomings, we propose a novel CKL approach termed Train-Attention-Augmented Language Model (TAALM), which enhances learning efficiency by dynamically predicting and applying weights to tokens based on their usefulness. This method employs a meta-learning framework that optimizes token importance predictions, facilitating targeted knowledge updates and minimizing forgetting. Also, we observe that existing benchmarks do not clearly exhibit the trade-off between learning and retaining, therefore we propose a new benchmark, LAMA-ckl, to address this issue. Through experiments conducted on both newly introduced and established CKL benchmarks, TAALM proves the state-of-the-art performance upon the baselines, and also shows synergistic compatibility when integrated with previous CKL approaches. The code and the dataset are available online.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Learning Topology-Specific Experts for Molecular Property Prediction

Suyeon Kim
Dongha Lee
SeongKu Kang
Seonghyeon Lee
Hwanjo Yu

Recently, graph neural networks (GNNs) have been successfully applied to predicting molecular properties, which is one of the most classical cheminformatics tasks with various applications. Despite their effectiveness, we empirically observe that training a single GNN model for diverse molecules with distinct structural patterns limits its prediction performance. In this paper, motivated by this observation, we propose TopExpert to leverage topology-specific prediction models (referred to as experts), each of which is responsible for each molecular group sharing similar topological semantics. That is, each expert learns topology-specific discriminative features while being trained with its corresponding topological group. To tackle the key challenge of grouping molecules by their topological patterns, we introduce a clustering-based gating module that assigns an input molecule into one of the clusters and further optimizes the gating module with two different types of self-supervision: topological semantics induced by GNNs and molecular scaffolds, respectively. Extensive experiments demonstrate that TopExpert has boosted the performance for molecular property prediction and also achieved better generalization for new molecules with unseen scaffolds than baselines. The code is available at https://github.com/kimsu55/ToxExpert.

PDF Details DOI

YNIMG Journal 2022 Journal Article

A populational connection distribution map for the whole brain white matter reveals ordered cortical wiring in the space of white matter

Dongha Lee
Hae-Jeong Park

The white matter in the brain is composed of neural fibers that wire the cortical and subcortical brain systems. Considering the complexity in terms of interconnections of many neural populations within the narrow space surrounded by the folding walls of the gray matter, the brain requires a certain way of structured wiring. To explore the three-dimensional organization of wiring in an accessible manner, we focused on voxel-level wiring patterns in the white matter according to cortical distributions in which each voxel mediates the wiring. We constructed a voxel-wise connection distribution map from the whole white matter voxels to 360 cortical regions using probabilistic tractography of the 100 diffusion imaging data in the Human Connectome Project. We then explored the spatial organization of the fiber bundles at the white matter voxels in terms of the maximal and second maximal cortical connection labels and the local gradient and entropy of cortical connection density using the population connection distribution map. We presented dominant cortical connection labels, local gradient, and connection entropy for the most representative white matter regions, including the internal capsule, external capsule, corpus callosum, cingulum bundle, and uncinate fascicles, most of which were introduced in the current study. Those major tracts showed a gradient organization of connection distributions for individual voxels. This organized pattern, as reflected in the whole brain connection distribution map, could be established to optimize wiring in the entire brain within the physical space of the white matter.

Details DOI

AAAI Conference 2021 Conference Paper

Learnable Dynamic Temporal Pooling for Time Series Classification

Dongha Lee
Seonghyeon Lee
Hwanjo Yu

With the increase of available time series data, predicting their class labels has been one of the most important challenges in a wide range of disciplines. Recent studies on time series classification show that convolutional neural networks (CNN) achieved the state-of-the-art performance as a single classifier. In this work, pointing out that the global pooling layer that is usually adopted by existing CNN classifiers discards the temporal information of high-level features, we present a dynamic temporal pooling (DTP) technique that reduces the temporal size of hidden representations by aggregating the features at the segment-level. For the partition of a whole series into multiple segments, we utilize dynamic time warping (DTW) to align each time point in a temporal order with the prototypical features of the segments, which can be optimized simultaneously with the network parameters of CNN classifiers. The DTP layer combined with a fullyconnected layer helps to extract further discriminative features considering their temporal position within an input time series. Extensive experiments on both univariate and multivariate time series datasets show that our proposed pooling significantly improves the classification performance.

PDF Details

IJCAI Conference 2020 Conference Paper

Convolutional Neural Networks with Compression Complexity Pooling for Out-of-Distribution Image Detection

Sehun Yu
Dongha Lee
Hwanjo Yu

To reliably detect out-of-distribution images based on already deployed convolutional neural networks, several recent studies on the out-of-distribution detection have tried to define effective confidence scores without retraining the model. Although they have shown promising results, most of them need to find the optimal hyperparameter values by using a few out-of-distribution images, which eventually assumes a specific test distribution and makes it less practical for real-world applications. In this work, we propose a novel out-of-distribution detection method termed as MALCOM, which neither uses any out-of-distribution sample nor retrains the model. Inspired by an observation that the global average pooling cannot capture spatial information of feature maps in convolutional neural networks, our method aims to extract informative sequential patterns from the feature maps. To this end, we introduce a similarity metric that focuses on shared patterns between two sequences based on the normalized compression distance. In short, MALCOM uses both the global average and the spatial patterns of feature maps to identify out-of-distribution images accurately.

PDF Details DOI

IJCAI Conference 2019 Conference Paper

Action Space Learning for Heterogeneous User Behavior Prediction

Dongha Lee
Chanyoung Park
Hyunjun Ju
Junyoung Hwang
Hwanjo Yu

Users' behaviors observed in many web-based applications are usually heterogeneous, so modeling their behaviors considering the interplay among multiple types of actions is important. However, recent collaborative filtering (CF) methods based on a metric learning approach cannot learn multiple types of user actions, because they are developed for only a single type of user actions. This paper proposes a novel metric learning method, called METAS, to jointly model heterogeneous user behaviors. Specifically, it learns two distinct spaces: 1) action space which captures the relations among all observed and unobserved actions, and 2) entity space which captures high-level similarities among users and among items. Each action vector in the action space is computed using a non-linear function and its corresponding entity vectors in the entity space. In addition, METAS adopts an efficient triplet mining algorithm to effectively speed up the convergence of metric learning. Experimental results show that METAS outperforms the state-of-the-art methods in predicting users' heterogeneous actions, and its entity space represents the user-user and item-item similarities more clearly than the space trained by the other methods.

PDF Details

YNIMG Journal 2019 Journal Article

Neurofeedback learning for mental practice rather than repetitive practice improves neural pattern consistency and functional network efficiency in the subsequent mental motor execution

Dongha Lee
Changwon Jang
Hae-Jeong Park

During brain modulation, repeated mental practice may not always result in efficient learning. Particularly, the effectiveness of mental motor practice depends on how well one induces neural activity in a desired state consistently across mental trials, which calls for feedbacks to adjust one's performance. We hypothesized that even a brief experience of neurofeedback learning enhances trial-by-trial neural pattern consistency during subsequent mental motor execution and that this experience would change recruitment of functional connectivity in the motor imagery and default mode networks. To test this hypothesis, we conducted an experiment with two sessions of mental motor practice before and after a neurofeedback training session, in which participants conducted four types of first-person mental motor execution tasks (walking forward, turning left, turning right, and touching a tree). During the neurofeedback training session, in which participants conducted a virtual navigation game, 10 experimental participants received real-time fMRI neuro-feedbacks, while 10 control participants simply repeated the same mental task according to given cues without feedbacks. The experimental group showed significantly higher effects of neuro-feedback training on trial-by-trial consistencies and classification accuracies of activated neural patterns than the control group. Task-performing global node strength and network efficiency were increased in the motor imagery network but decreased in the default mode network only in the experimental group. These results demonstrate that even a brief experience of feedback learning is more effective than simple practice repetitions without evaluation, which was reflected in increased neural pattern consistency and task-dependent functional connectivity during a mental motor execution task.

Details DOI

YNIMG Journal 2015 Journal Article

Multivariate detrending of fMRI signal drifts for real-time multiclass pattern classification

Dongha Lee
Changwon Jang
Hae-Jeong Park

Signal drift in functional magnetic resonance imaging (fMRI) is an unavoidable artifact that limits classification performance in multi-voxel pattern analysis of fMRI. As conventional methods to reduce signal drift, global demeaning or proportional scaling disregards regional variations of drift, whereas voxel-wise univariate detrending is too sensitive to noisy fluctuations. To overcome these drawbacks, we propose a multivariate real-time detrending method for multiclass classification that involves spatial demeaning at each scan and the recursive detrending of drifts in the classifier outputs driven by a multiclass linear support vector machine. Experiments using binary and multiclass data showed that the linear trend estimation of the classifier output drift for each class (a weighted sum of drifts in the class-specific voxels) was more robust against voxel-wise artifacts that lead to inconsistent spatial patterns and the effect of online processing than voxel-wise detrending. The classification performance of the proposed method was significantly better, especially for multiclass data, than that of voxel-wise linear detrending, global demeaning, and classifier output detrending without demeaning. We concluded that the multivariate approach using classifier output detrending of fMRI signals with spatial demeaning preserves spatial patterns, is less sensitive than conventional methods to sample size, and increases classification performance, which is a useful feature for real-time fMRI classification.

Details DOI

YNIMG Journal 2012 Journal Article

Are brain networks stable during a 24-hour period?

Bumhee Park
Joong Il Kim
Dongha Lee
Seok-Oh Jeong
Jong Doo Lee
Hae-Jeong Park

Despite the widespread view of the brain as a large complex network, the dynamicity of the brain network over the course of a day has yet to be explored. To investigate whether the spontaneous human brain network maintains long-term stability throughout a day, we evaluated the intra-class correlation coefficient (ICC) of results from an independent component analysis (ICA), seed correlation analysis, and graph-theoretical analysis of resting state functional MRI, acquired from 12 young adults at three-hour intervals over 24 consecutive hours. According to the ICC of the usage strength of the independent network component defined by the root mean square of the temporal weights of the network components, the default mode network centered at the posterior cingulate cortex and precuneus, the superior parietal, and secondary motor networks showed a high temporal stability throughout the day (ICC>0. 5). However, high intra-individual dynamicity was observed in the default mode network, including the anterior cingulate cortex and medial prefrontal cortex or posterior–anterior cingulate cortex, the hippocampal network, and the parietal and temporal networks. Seed correlation analysis showed a highly stable (ICC>0. 5) extent of functionally connected regions from the posterior cingulate cortex, but poor stability from the hippocampus throughout the day. Graph-theoretical analysis using local and global network efficiency suggested that local brain networks are temporally stable but that long-range integration behaves dynamically in the course of a day. These results imply that dynamic network properties are a nature of the resting state brain network, which remains to be further researched.

Details DOI