Arrow Research search

Author name cluster

Noseong Park

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

40 papers
2 author rows

Possible papers

40

AAAI Conference 2026 Conference Paper

Are Graph Transformers Necessary? Efficient Long-Range Message Passing with Fractal Nodes in MPNNs

  • Jeongwhan Choi
  • Seungjun Park
  • Sumin Park
  • Sung-Bae Cho
  • Noseong Park

Graph Neural Networks (GNNs) have emerged as powerful tools for learning on graph-structured data, but often struggle to balance local and global information. While graph Transformers aim to address this by enabling long-range interactions, they often overlook the inherent locality and efficiency of Message Passing Neural Networks (MPNNs). We propose a new concept called 'fractal nodes', inspired by the fractal structure observed in real-world networks. Our approach is based on the intuition that graph partitioning naturally induces fractal structure, where subgraphs often reflect the connectivity patterns of the full graph. Fractal nodes are designed to coexist with the original nodes and adaptively aggregate subgraph-level feature representations, thereby enforcing feature similarity within each subgraph. We show that fractal nodes alleviate the over-squashing problem by providing direct shortcut connections that enable long-range propagation of subgraph-level representations. Experiment results show that our method improves the expressive power of MPNNs and achieves comparable or better performance to graph Transformers while maintaining the computational efficiency of MPNN by improving the long-range dependencies of MPNN.

PDF Details DOI

AAAI Conference 2026 Conference Paper

How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts

  • Sumin Park
  • Noseong Park

Finding the optimal configuration of Sparse Mixture-of- Experts (SMoE) that maximizes semantic differentiation among experts is essential for exploiting the full potential of MoE architectures. However, existing SMoE frameworks either heavily rely on hyperparameter tuning or overlook the importance of diversifying semantic roles across experts when adapting the expert pool size. We propose Mixture-of-Experts for Adaptive Semantic Specialization (MASS), a semantic-aware MoE framework for adaptive expert expansion and dynamic routing. MASS introduces two key advancements: (i) a gradient-based semantic drift detector that prompts targeted expert expansion when the existing expert pool lacks capacity to capture the full semantic diversity of the data, and (ii) an integration of adaptive routing strategy that dynamically adjusts expert usage based on token-level routing confidence mass. We first demonstrate that MASS reliably converges to the point of optimal balance between cost-performance trade-off with notably improved sematic specialization in a highly controlled synthetic setup. Further empirical results on real-world datasets across language and vision domains show that MASS consistently outperforms a range of strong MoE baselines, demonstrating its domain robustness and enhanced expert specialization.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Efficiently Parameterized Neural Metriplectic Systems

  • Anthony Gruber
  • Kookjin Lee
  • Haksoo Lim
  • Noseong Park
  • Nathaniel Trask

Metriplectic systems are learned from data in a way that scales quadratically in both the size of the state and the rank of the metriplectic operators. In addition to being provably energy-conserving and entropy-stable, the proposed neural metriplectic systems (NMS) approach includes approximation results that demonstrate its ability to accurately learn metriplectic dynamics from data, along with an error estimate that indicates its potential for generalization to unseen timescales when the approximation error is low. Examples are provided to illustrate performance both with full state information available and when entropic variables are unknown, confirming that the NMS approach exhibits superior accuracy and scalability without compromising on model expressivity.

Details

IJCAI Conference 2025 Conference Paper

Learning Advanced Self-Attention for Linear Transformers in the Singular Value Domain

  • Hyowon Wi
  • Jeongwhan Choi
  • Noseong Park

Transformers have demonstrated remarkable performance across diverse domains. The key component of Transformers is self-attention, which learns the relationship between any two tokens in the input sequence. Recent studies have revealed that the self-attention can be understood as a normalized adjacency matrix of a graph. Notably, from the perspective of graph signal processing (GSP), the self-attention can be equivalently defined as a simple graph filter, applying GSP using the value vector as the signal. However, the self-attention is a graph filter defined with only the first order of the polynomial matrix, and acts as a low-pass filter preventing the effective leverage of various frequency information. Consequently, existing self-attention mechanisms are designed in a rather simplified manner. Therefore, we propose a novel method, called Attentive Graph Filter (AGF), interpreting the self-attention as learning the graph filter in the singular value domain from the perspective of graph signal processing for directed graphs with the linear complexity w. r. t. the input length. In our experiments, we demonstrate that AGF achieves state-of-the-art performance on various tasks, including Long Range Arena benchmark and time series classification. Code is available at https: //github. com/hyowonwi/agf.

PDF Details DOI

AAAI Conference 2025 Conference Paper

MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity

  • Kanghyun Choi
  • Hyeyoon Lee
  • Dain Kwon
  • SunJong Park
  • Kyuyeun Kim
  • Noseong Park
  • Jonghyun Choi
  • Jinho Lee

Data-free quantization (DFQ) is a technique that creates a lightweight network from its full-precision counterpart without the original training data, often through a synthetic dataset. Although several DFQ methods have been proposed for vision transformer (ViT) architectures, they fail to achieve efficacy in low-bit settings. Examining the existing methods, we observe that their synthetic data produce misaligned attention maps, while those of the real samples are highly aligned. From this observation, we find that aligning attention maps of synthetic data helps improve the overall performance of quantized ViTs. Motivated by this finding, we devise MimiQ, a novel DFQ method designed for ViTs that enhances inter-head attention similarity. First, we generate synthetic data by aligning head-wise attention outputs from each spatial query patch. Then, we align the attention maps of the quantized network to those of the full-precision teacher by applying head-wise structural attention distillation. The experimental results show that the proposed method significantly outperforms baselines, setting a new state-of-the-art for ViT-DFQ.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Neural Functions for Learning Periodic Signal

  • Woojin Cho
  • Minju Jo
  • Kookjin Lee
  • Noseong Park

As function approximators, deep neural networks have served as an effective tool to represent various signal types. Recent approaches utilize multi-layer perceptrons (MLPs) to learn a nonlinear mapping from a coordinate to its corresponding signal, facilitating the learning of continuous neural representations from discrete data points. Despite notable successes in learning diverse signal types, coordinate-based MLPs often face issues of overfitting and limited generalizability beyond the training region, resulting in subpar extrapolation performance. This study addresses scenarios where the underlying true signals exhibit periodic properties, either spatially or temporally. We propose a novel network architecture, which extracts periodic patterns from measurements and leverages this information to represent the signal, thereby enhancing generalization and improving extrapolation performance. We demonstrate the efficacy of the proposed method through comprehensive experiments, including the learning of the periodic solutions for differential equations, and time series imputation (interpolation) and forecasting (extrapolation) on real-world datasets.

Details

NeurIPS Conference 2025 Conference Paper

PDEfuncta: Spectrally-Aware Neural Representation for PDE Solution Modeling

  • Minju Jo
  • Woojin Cho
  • Uvini Balasuriya Mudiyanselage
  • Seungjun Lee
  • Noseong Park
  • Kookjin Lee

Scientific machine learning often involves representing complex solution fields that exhibit high-frequency features such as sharp transitions, fine-scale oscillations, and localized structures. While implicit neural representations (INRs) have shown promise for continuous function modeling, capturing such high-frequency behavior remains a challenge—especially when modeling multiple solution fields with a shared network. Prior work addressing spectral bias in INRs has primarily focused on single-instance settings, limiting scalability and generalization. In this work, we propose Global Fourier Modulation (GFM), a novel modulation technique that injects high-frequency information at each layer of the INR through Fourier-based reparameterization. This enables compact and accurate representation of multiple solution fields using low-dimensional latent vectors. Building upon GFM, we introduce PDEfuncta, a meta-learning framework designed to learn multi-modal solution fields and support generalization to new tasks. Through empirical studies on diverse scientific problems, we demonstrate that our method not only improves representational quality but also shows potential for forward and inverse inference tasks without the need for retraining.

PDF Details

ICLR Conference 2025 Conference Paper

PIORF: Physics-Informed Ollivier-Ricci Flow for Long-Range Interactions in Mesh Graph Neural Networks

  • Youn-Yeol Yu
  • Jeongwhan Choi 0002
  • Jaehyeon Park
  • Kookjin Lee
  • Noseong Park

Recently, data-driven simulators based on graph neural networks have gained attention in modeling physical systems on unstructured meshes. However, they struggle with long-range dependencies in fluid flows, particularly in refined mesh regions. This challenge, known as the 'over-squashing' problem, hinders information propagation. While existing graph rewiring methods address this issue to some extent, they only consider graph topology, overlooking the underlying physical phenomena. We propose Physics-Informed Ollivier--Ricci Flow (PIORF), a novel rewiring method that combines physical correlations with graph topology. PIORF uses Ollivier--Ricci curvature (ORC) to identify bottleneck regions and connects these areas with nodes in high-velocity gradient nodes, enabling long-range interactions and mitigating over-squashing. Our approach is computationally efficient in rewiring edges and can scale to larger simulations. Experimental results on 3 fluid dynamics benchmark datasets show that PIORF consistently outperforms baseline models and existing rewiring methods, achieving up to 26.2\% improvement.

Details

AIIM Journal 2025 Journal Article

Predicting drug-drug interactions: A deep learning approach with GCN-based collaborative filtering

  • Yeon Uk Jeong
  • Jeongwhan Choi
  • Noseong Park
  • Jae Yong Ryu
  • Yi Rang Kim

The use of combination drugs among patients is increasing due to effectiveness compared to monotherapies. However, healthcare providers should continue to be concerned about the potential risks associated with patient safety arising from drug-drug interactions (DDIs) when they use combination drugs. Whereas direct physicochemical interactions contribute to certain cases of DDIs, the majority of DDIs occur because one drug modulates enzymes, such as cytochrome P450, responsible for metabolizing another drug. Therefore, drugs that interact with the same family drugs are more likely to interact with each other by mediating specific enzymes. Adapted from techniques used to recommend users with similar interests, we introduce an AI recommendation model with graph convolutional network (GCN) and collaborative filtering that analyzes the connectivity of interacting drugs rather than their chemical structures. This approach deviates from typical classification models by not requiring sampling of undefined interactions as negative samples, allowing the prediction of potential interactions for all unknown drug pairs, circumventing the challenges associated with selecting negative interactions and data imbalance. Our methodology used the DrugBank database (version 5. 1. 9 released on January 3, 2022), encompassing 4, 072 drugs and 1, 391, 790 drug pairs with interactions. Furthermore, the robustness of the model was verified through a 5-fold validation and external data validation using TWOSIDES data. Notably, our model’s efficacy is established solely through the exploitation of DDI reports, offering a versatile framework capable of accurately predicting interactions among diverse drug types. The source code for this project is distributed on GitHub (https: //github. com/yeonuk-Jeong/DDI-OCF).

Details DOI

NeurIPS Conference 2025 Conference Paper

TV-Rec: Time-Variant Convolutional Filter for Sequential Recommendation

  • Yehjin Shin
  • Jeongwhan Choi
  • Seojin Kim
  • Noseong Park

Recently, convolutional filters have been increasingly adopted in sequential recommendation for their ability to capture local sequential patterns. However, most of these models complement convolutional filters with self-attention. This is because convolutional filters alone, generally fixed filters, struggle to capture global interactions necessary for accurate recommendation. We propose \textbf{T}ime-\textbf{V}ariant Convolutional Filters for Sequential \textbf{Rec}ommendation (TV-Rec), a model inspired by graph signal processing, where time-variant graph filters capture position-dependent temporal variations in user sequences. By replacing both fixed kernels and self-attention with time-variant filters, TV-Rec achieves higher expressive power and better captures complex interaction patterns in user behavior. This design not only eliminates the need for self-attention but also reduces computation while accelerating inference. Extensive experiments on six public benchmarks show that TV-Rec outperforms state-of-the-art baselines by an average of 7. 49\%.

PDF Details

AAAI Conference 2024 Conference Paper

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

  • Yehjin Shin
  • Jeongwhan Choi
  • Hyowon Wi
  • Noseong Park

Sequential recommendation (SR) models based on Transformers have achieved remarkable successes. The self-attention mechanism of Transformers for computer vision and natural language processing suffers from the oversmoothing problem, i.e., hidden representations becoming similar to tokens. In the SR domain, we, for the first time, show that the same problem occurs. We present pioneering investigations that reveal the low-pass filtering nature of self-attention in the SR, which causes oversmoothing. To this end, we propose a novel method called Beyond Self-Attention for Sequential Recommendation (BSARec), which leverages the Fourier transform to i) inject an inductive bias by considering fine-grained sequential patterns and ii) integrate low and high-frequency information to mitigate oversmoothing. Our discovery shows significant advancements in the SR domain and is expected to bridge the gap for existing Transformer-based SR models. We test our proposed approach through extensive experiments on 6 benchmark datasets. The experimental results demonstrate that our model outperforms 7 baseline methods in terms of recommendation performance. Our code is available at https://github.com/yehjin-shin/BSARec.

PDF Details DOI

ICML Conference 2024 Conference Paper

DataFreeShield: Defending Adversarial Attacks without Training Data

  • Hyeyoon Lee
  • Kanghyun Choi
  • Dain Kwon
  • Sunjong Park
  • Mayoore Selvarasa Jaiswal
  • Noseong Park
  • Jonghyun Choi
  • Jinho Lee 0001

Recent advances in adversarial robustness rely on an abundant set of training data, where using external or additional datasets has become a common setting. However, in real life, the training data is often kept private for security and privacy issues, while only the pretrained weight is available to the public. In such scenarios, existing methods that assume accessibility to the original data become inapplicable. Thus we investigate the pivotal problem of data-free adversarial robustness, where we try to achieve adversarial robustness without accessing any real data. Through a preliminary study, we highlight the severity of the problem by showing that robustness without the original dataset is difficult to achieve, even with similar domain datasets. To address this issue, we propose DataFreeShield, which tackles the problem from two perspectives: surrogate dataset generation and adversarial training using the generated data. Through extensive validation, we show that DataFreeShield outperforms baselines, demonstrating that the proposed method sets the first entirely data-free solution for the adversarial robustness problem.

Details

NeurIPS Conference 2024 Conference Paper

Graph Convolutions Enrich the Self-Attention in Transformers!

  • Jeongwhan Choi
  • Hyowon Wi
  • Jayoung Kim
  • Yehjin Shin
  • Kookjin Lee
  • Nathaniel Trask
  • Noseong Park

Transformers, renowned for their self-attention mechanism, have achieved state-of-the-art performance across various tasks in natural language processing, computer vision, time-series modeling, etc. However, one of the challenges with deep Transformer models is the oversmoothing problem, where representations across layers converge to indistinguishable values, leading to significant performance degradation. We interpret the original self-attention as a simple graph filter and redesign it from a graph signal processing (GSP) perspective. We propose a graph-filter-based self-attention (GFSA) to learn a general yet effective one, whose complexity, however, is slightly larger than that of the original self-attention mechanism. We demonstrate that GFSA improves the performance of Transformers in various fields, including computer vision, natural language processing, graph-level tasks, speech recognition, and code classification.

PDF Details DOI

ICLR Conference 2024 Conference Paper

Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer

  • Youn-Yeol Yu
  • Jeongwhan Choi 0002
  • Woojin Cho
  • Kookjin Lee
  • Nayong Kim
  • Kiseok Chang
  • ChangSeung Woo
  • Ilho Kim

Recently, many mesh-based graph neural network (GNN) models have been proposed for modeling complex high-dimensional physical systems. Remarkable achievements have been made in significantly reducing the solving time compared to traditional numerical solvers. These methods are typically designed to i) reduce the computational cost in solving physical dynamics and/or ii) propose techniques to enhance the solution accuracy in fluid and rigid body dynamics. However, it remains under-explored whether they are effective in addressing the challenges of flexible body dynamics, where instantaneous collisions occur within a very short timeframe. In this paper, we present Hierarchical Contact Mesh Transformer (HCMT), which uses hierarchical mesh structures and can learn long-range dependencies (occurred by collisions) among spatially distant positions of a body --- two close positions in a higher-level mesh correspond to two distant positions in a lower-level mesh. HCMT enables long-range interactions, and the hierarchical mesh structure quickly propagates collision effects to faraway positions. To this end, it consists of a contact mesh Transformer and a hierarchical mesh Transformer (CMT and HMT, respectively). Lastly, we propose a flexible body dynamics dataset, consisting of trajectories that reflect experimental settings frequently used in the display industry for product designs. We also compare the performance of several baselines using well-known benchmark datasets. Our results show that HCMT provides significant performance improvements over existing methods. Our code is available at https://github.com/yuyudeep/hcmt.

Details

AAAI Conference 2024 Conference Paper

Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations

  • Woojin Cho
  • Seunghyeon Cho
  • Hyundong Jin
  • Jinsung Jeon
  • Kookjin Lee
  • Sanghyun Hong
  • Dongeun Lee
  • Jonghyun Choi

Neural ordinary differential equations (NODEs), one of the most influential works of the differential equation-based deep learning, are to continuously generalize residual networks and opened a new field. They are currently utilized for various downstream tasks, e.g., image classification, time series classification, image generation, etc. Its key part is how to model the time-derivative of the hidden state, denoted dh(t)/dt. People have habitually used conventional neural network architectures, e.g., fully-connected layers followed by non-linear activations. In this paper, however, we present a neural operator-based method to define the time-derivative term. Neural operators were initially proposed to model the differential operator of partial differential equations (PDEs). Since the time-derivative of NODEs can be understood as a special type of the differential operator, our proposed method, called branched Fourier neural operator (BFNO), makes sense. In our experiments with general downstream tasks, our method significantly outperforms existing methods.

PDF Details DOI

ICLR Conference 2024 Conference Paper

PAC-FNO: Parallel-Structured All-Component Fourier Neural Operators for Recognizing Low-Quality Images

  • Jinsung Jeon
  • Hyundong Jin
  • Jonghyun Choi
  • Sanghyun Hong 0001
  • Dongeun Lee 0001
  • Kookjin Lee
  • Noseong Park

A standard practice in developing image recognition models is to train a model on a specific image resolution and then deploy it. However, in real-world inference, models often encounter images different from the training sets in resolution and/or subject to natural variations such as weather changes, noise types and compression artifacts. While traditional solutions involve training multiple models for different resolutions or input variations, these methods are computationally expensive and thus do not scale in practice. To this end, we propose a novel neural network model, parallel-structured and all-component Fourier neural operator (PAC-FNO), that addresses the problem. Unlike conventional feed-forward neural networks, PAC-FNO operates in the frequency domain, allowing it to handle images of varying resolutions within a single model. We also propose a two-stage algorithm for training PAC-FNO with a minimal modification to the original, downstream model. Moreover, the proposed PAC-FNO is ready to work with existing image recognition models. Extensively evaluating methods with seven image recognition benchmarks, we show that the proposed PAC-FNO improves the performance of existing baseline models on images with various resolutions by up to 77.1% and various types of natural variations in the images at inference.

Details

ICML Conference 2024 Conference Paper

PANDA: Expanded Width-Aware Message Passing Beyond Rewiring

  • Jeongwhan Choi 0002
  • Sumin Park
  • Hyowon Wi
  • Sung-Bae Cho
  • Noseong Park

Recent research in the field of graph neural network (GNN) has identified a critical issue known as "over-squashing, " resulting from the bottleneck phenomenon in graph structures, which impedes the propagation of long-range information. Prior works have proposed a variety of graph rewiring concepts that aim at optimizing the spatial or spectral properties of graphs to promote the signal propagation. However, such approaches inevitably deteriorate the original graph topology, which may lead to a distortion of information flow. To address this, we introduce an ex pand ed width- a ware ( PANDA ) message passing, a new message passing paradigm where nodes with high centrality, a potential source of over-squashing, are selectively expanded in width to encapsulate the growing influx of signals from distant nodes. Experimental results show that our method outperforms existing rewiring methods, suggesting that selectively expanding the hidden state of nodes can be a compelling alternative to graph rewiring for addressing the over-squashing.

Details

ICML Conference 2024 Conference Paper

Parameterized Physics-informed Neural Networks for Parameterized PDEs

  • Woojin Cho
  • Minju Jo
  • Haksoo Lim
  • Kookjin Lee
  • Dongeun Lee 0001
  • Sanghyun Hong 0001
  • Noseong Park

Complex physical systems are often described by partial differential equations (PDEs) that depend on parameters such as the Raynolds number in fluid mechanics. In applications such as design optimization or uncertainty quantification, solutions of those PDEs need to be evaluated at numerous points in the parameter space. While physics-informed neural networks (PINNs) have emerged as a new strong competitor as a surrogate, their usage in this scenario remains underexplored due to the inherent need for repetitive and time-consuming training. In this paper, we address this problem by proposing a novel extension, parameterized physics-informed neural networks (P$^2$INNs). P$^2$INNs enable modeling the solutions of parameterized PDEs via explicitly encoding a latent representation of PDE parameters. With the extensive empirical evaluation, we demonstrate that P$^2$INNs outperform the baselines both in accuracy and parameter efficiency on benchmark 1D and 2D parameterized PDEs and are also effective in overcoming the known “failure modes”.

Details

ICML Conference 2024 Conference Paper

Polynomial-based Self-Attention for Table Representation Learning

  • Jayoung Kim 0002
  • Yehjin Shin
  • Jeongwhan Choi 0002
  • Hyowon Wi
  • Noseong Park

Structured data, which constitutes a significant portion of existing data types, has been a long-standing research topic in the field of machine learning. Various representation learning methods for tabular data have been proposed, ranging from encoder-decoder structures to Transformers. Among these, Transformer-based methods have achieved state-of-the-art performance not only in tabular data but also in various other fields, including computer vision and natural language processing. However, recent studies have revealed that self-attention, a key component of Transformers, can lead to an oversmoothing issue. We show that Transformers for tabular data also face this problem. To tackle the problem, we suggest a novel self-attention layer for tabular data, leveraging matrix polynomials. This proposed layer serves as a replacement for the original self-attention layer, contributing to the improvement of model scalability. In our experiments with three representative table learning models equipped with our proposed layer, we illustrate that the layer effectively mitigates the oversmoothing problem and enhances the representation performance of the existing methods, outperforming the state-of-the-art table representation methods.

Details

IJCAI Conference 2024 Conference Paper

SVD-AE: Simple Autoencoders for Collaborative Filtering

  • Seoyoung Hong
  • Jeongwhan Choi
  • Yeon-Chang Lee
  • Srijan Kumar
  • Noseong Park

Collaborative filtering (CF) methods for recommendation systems have been extensively researched, ranging from matrix factorization and autoencoder-based to graph filtering-based methods. Recently, lightweight methods that require almost no training have been recently proposed to reduce overall computation. However, existing methods still have room to improve the trade-offs among accuracy, efficiency, and robustness. In particular, there are no well-designed closed-form studies for balanced CF in terms of the aforementioned trade-offs. In this paper, we design SVD-AE, a simple yet effective singular vector decomposition (SVD)-based linear autoencoder, whose closed-form solution can be defined based on SVD for CF. SVD-AE does not require iterative training processes as its closed-form solution can be calculated at once. Furthermore, given the noisy nature of the rating matrix, we explore the robustness against such noisy interactions of existing CF methods and our SVD-AE. As a result, we demonstrate that our simple design choice based on truncated SVD can be used to strengthen the noise robustness of the recommendation while improving efficiency. Code is available at https: //github. com/seoyoungh/svd-ae.

PDF Details DOI

ICML Conference 2023 Conference Paper

CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesis

  • Chaejeong Lee
  • Jayoung Kim 0002
  • Noseong Park

With growing attention to tabular data these days, the attempt to apply a synthetic table to various tasks has been expanded toward various scenarios. Owing to the recent advances in generative modeling, fake data generated by tabular data synthesis models become sophisticated and realistic. However, there still exists a difficulty in modeling discrete variables (columns) of tabular data. In this work, we propose to process continuous and discrete variables separately (but being conditioned on each other) by two diffusion models. The two diffusion models are co-evolved during training by reading conditions from each other. In order to further bind the diffusion models, moreover, we introduce a contrastive learning method with a negative sampling method. In our experiments with 11 real-world tabular datasets and 8 baseline methods, we prove the efficacy of the proposed method, called $\texttt{CoDi}$. Our code is available at https: //github. com/ChaejeongLee/CoDi.

Details

TIST Journal 2023 Journal Article

Graph Neural Rough Differential Equations for Traffic Forecasting

  • Jeongwhan Choi
  • Noseong Park

Traffic forecasting is one of the most popular spatio-temporal tasks in the field of machine learning. A prevalent approach in the field is to combine graph convolutional networks and recurrent neural networks for the spatio-temporal processing. There has been fierce competition and many novel methods have been proposed. In this article, we present the method of spatio-temporal graph neural rough differential equation (STG-NRDE). Neural rough differential equations (NRDEs) are a breakthrough concept for processing time-series data. Their main concept is to use the log-signature transform to convert a time-series sample into a relatively shorter series of feature vectors. We extend the concept and design two NRDEs: one for the temporal processing and the other for the spatial processing. After that, we combine them into a single framework. We conduct experiments with 6 benchmark datasets and 27 baselines. STG-NRDE shows the best accuracy in all cases, outperforming all those 27 baselines by non-trivial margins.

Details DOI

ICML Conference 2023 Conference Paper

GREAD: Graph Neural Reaction-Diffusion Networks

  • Jeongwhan Choi 0002
  • Seoyoung Hong 0001
  • Noseong Park
  • Sung-Bae Cho

Graph neural networks (GNNs) are one of the most popular research topics for deep learning. GNN methods typically have been designed on top of the graph signal processing theory. In particular, diffusion equations have been widely used for designing the core processing layer of GNNs, and therefore they are inevitably vulnerable to the notorious oversmoothing problem. Recently, a couple of papers paid attention to reaction equations in conjunctions with diffusion equations. However, they all consider limited forms of reaction equations. To this end, we present a reaction-diffusion equation-based GNN method that considers all popular types of reaction equations in addition to one special reaction equation designed by us. To our knowledge, our paper is one of the most comprehensive studies on reaction-diffusion equation-based GNNs. In our experiments with 9 datasets and 28 baselines, our method, called GREAD, outperforms them in a majority of cases. Further synthetic data experiments show that it mitigates the oversmoothing problem and works well for various homophily rates.

Details

IJCAI Conference 2023 Conference Paper

Hawkes Process Based on Controlled Differential Equations

  • Minju Jo
  • Seungji Kook
  • Noseong Park

Hawkes processes are a popular framework to model the occurrence of sequential events, i. e. , occurrence dynamics, in several fields such as social diffusion. In real-world scenarios, the inter-arrival time among events is irregular. However, existing neural network-based Hawkes process models not only i) fail to capture such complicated irregular dynamics, but also ii) resort to heuristics to calculate the log-likelihood of events since they are mostly based on neural networks designed for regular discrete inputs. To this end, we present the concept of Hawkes process based on controlled differential equations (HP-CDE), by adopting the neural controlled differential equation (neural CDE) technology which is an analogue to continuous RNNs. Since HP-CDE continuously reads data, i) irregular time-series datasets can be properly treated preserving their uneven temporal spaces, and ii) the log-likelihood can be exactly computed. Moreover, as both Hawkes processes and neural CDEs are first developed to model complicated human behavioral dynamics, neural CDE-based Hawkes processes are successful in modeling such occurrence dynamics. In our experiments with 4 real-world datasets, our method outperforms existing methods by non-trivial margins.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks

  • Woojin Cho
  • Kookjin Lee
  • Donsub Rim
  • Noseong Park

In various engineering and applied science applications, repetitive numerical simulations of partial differential equations (PDEs) for varying input parameters are often required (e. g. , aircraft shape optimization over many design parameters) and solvers are required to perform rapid execution. In this study, we suggest a path that potentially opens up a possibility for physics-informed neural networks (PINNs), emerging deep-learning-based solvers, to be considered as one such solver. Although PINNs have pioneered a proper integration of deep-learning and scientific computing, they require repetitive time-consuming training of neural networks, which is not suitable for many-query scenarios. To address this issue, we propose a lightweight low-rank PINNs containing only hundreds of model parameters and an associated hypernetwork-based meta-learning algorithm, which allows efficient approximation of solutions of PDEs for varying ranges of PDE input parameters. Moreover, we show that the proposed method is effective in overcoming a challenging issue, known as "failure modes" of PINNs.

PDF Details

AAAI Conference 2023 Conference Paper

Learnable Path in Neural Controlled Differential Equations

  • Sheo Yon Jhin
  • Minju Jo
  • Seungji Kook
  • Noseong Park

Neural controlled differential equations (NCDEs), which are continuous analogues to recurrent neural networks (RNNs), are a specialized model in (irregular) time-series processing. In comparison with similar models, e.g., neural ordinary differential equations (NODEs), the key distinctive characteristics of NCDEs are i) the adoption of the continuous path created by an interpolation algorithm from each raw discrete time-series sample and ii) the adoption of the Riemann--Stieltjes integral. It is the continuous path which makes NCDEs be analogues to continuous RNNs. However, NCDEs use existing interpolation algorithms to create the path, which is unclear whether they can create an optimal path. To this end, we present a method to generate another latent path (rather than relying on existing interpolation algorithms), which is identical to learning an appropriate interpolation method. We design an encoder-decoder module based on NCDEs and NODEs, and a special training method for it. Our method shows the best performance in both time-series classification and forecasting.

PDF Details DOI

AIIM Journal 2023 Journal Article

Self-Attention LSTM-FCN model for arrhythmia classification and uncertainty assessment

  • JaeYeon Park
  • Kichang Lee
  • Noseong Park
  • Seng Chan You
  • JeongGil Ko

This paper presents ArrhyMon, a self-attention-based LSTM-FCN model for arrhythmia classification from ECG signal inputs. ArrhyMon targets to detect and classify six different types of arrhythmia apart from normal ECG patterns. To the best of our knowledge, ArrhyMon is the first end-to-end classification model that successfully targets the classification of six detailed arrhythmia types and compared to previous work does not require additional preprocessing and/or feature extraction operations separate from the classification model. ArrhyMon’s deep learning model is designed to capture and exploit both global and local features embedded in ECG sequences by integrating fully convolutional network (FCN) layers and a self-attention-based long and short-term memory (LSTM) architecture. Moreover, to enhance its practicality, ArrhyMon incorporates a deep ensemble-based uncertainty model that generates a confidence-level measure for each classification result. We evaluate ArrhyMon’s effectiveness using three publicly available arrhythmia datasets (i. e. , MIT-BIH, Physionet Cardiology Challenge 2017 and 2020/2021) to show that ArrhyMon achieves state-of-the-art classification performance (average accuracy 99. 63%), and that confidence measures show close correlation with subjective diagnosis made from practitioners.

Details DOI

ICLR Conference 2023 Conference Paper

STaSy: Score-based Tabular data Synthesis

  • Jayoung Kim 0002
  • Chaejeong Lee
  • Noseong Park

Tabular data synthesis is a long-standing research topic in machine learning. Many different methods have been proposed over the past decades, ranging from statistical methods to deep generative methods. However, it has not always been successful due to the complicated nature of real-world tabular data. In this paper, we present a new model named $\textbf{S}$core-based $\textbf{Ta}$bular data $\textbf{Sy}$nthesis ($\texttt{STaSy}$) and its training strategy based on the paradigm of score-based generative modeling. Despite the fact that score-based generative models have resolved many issues in generative models, there still exists room for improvement in tabular data synthesis. Our proposed training strategy includes a self-paced learning technique and a fine-tuning strategy, which further increases the sampling quality and diversity by stabilizing the denoising score matching training. Furthermore, we also conduct rigorous experimental studies in terms of the generative task trilemma: sampling quality, diversity, and time. In our experiments with 15 benchmark tabular datasets and 7 baselines, our method outperforms existing methods in terms of task-dependant evaluations and diversity.

Details

AAAI Conference 2022 Conference Paper

Graph Neural Controlled Differential Equations for Traffic Forecasting

  • Jeongwhan Choi
  • Hwangyong Choi
  • Jeehyun Hwang
  • Noseong Park

Traffic forecasting is one of the most popular spatio-temporal tasks in the field of machine learning. A prevalent approach in the field is to combine graph convolutional networks and recurrent neural networks for the spatio-temporal processing. There has been fierce competition and many novel methods have been proposed. In this paper, we present the method of spatio-temporal graph neural controlled differential equation (STG-NCDE). Neural controlled differential equations (NCDEs) are a breakthrough concept for processing sequential data. We extend the concept and design two NCDEs: one for the temporal processing and the other for the spatial processing. After that, we combine them into a single framework. We conduct experiments with 6 benchmark datasets and 20 baselines. STG-NCDE shows the best accuracy in all cases, outperforming all those 20 baselines by non-trivial margins.

PDF Details

NeurIPS Conference 2022 Conference Paper

GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks

  • Jinsung Jeon
  • JEONGHAK KIM
  • Haryong Song
  • Seunghyeon Cho
  • Noseong Park

Time series synthesis is an important research topic in the field of deep learning, which can be used for data augmentation. Time series data types can be broadly classified into regular or irregular. However, there are no existing generative models that show good performance for both types without any model changes. Therefore, we present a general purpose model capable of synthesizing regular and irregular time series data. To our knowledge, we are the first designing a general purpose time series synthesis model, which is one of the most challenging settings for time series synthesis. To this end, we design a generative adversarial network-based method, where many related techniques are carefully integrated into a single framework, ranging from neural ordinary/controlled differential equations to continuous time-flow processes. Our method outperforms all existing methods.

PDF Details

ICLR Conference 2022 Conference Paper

LORD: Lower-Dimensional Embedding of Log-Signature in Neural Rough Differential Equations

  • Jaehoon Lee 0002
  • Jinsung Jeon
  • Sheo Yon Jhin
  • Jihyeon Hyeong
  • Jayoung Kim 0002
  • Minju Jo
  • Seungji Kook
  • Noseong Park

The problem of processing very long time-series data (e.g., a length of more than 10,000) is a long-standing research problem in machine learning. Recently, one breakthrough, called neural rough differential equations (NRDEs), has been proposed and has shown that it is able to process such data. Their main concept is to use the log-signature transform, which is known to be more efficient than the Fourier transform for irregular long time-series, to convert a very long time-series sample into a relatively shorter series of feature vectors. However, the log-signature transform causes non-trivial spatial overheads. To this end, we present the method of LOweR-Dimensional embedding of log-signature (LORD), where we define an NRDE-based autoencoder to implant the higher-depth log-signature knowledge into the lower-depth log-signature. We show that the encoder successfully combines the higher-depth and the lower-depth log-signature knowledge, which greatly stabilizes the training process and increases the model accuracy. In our experiments with benchmark datasets, the improvement ratio by our method is up to 75\% in terms of various classification and forecasting evaluation metrics.

Details

ICML Conference 2021 Conference Paper

A Novel Method to Solve Neural Knapsack Problems

  • Duanshun Li
  • Jing Liu 0024
  • Dongeun Lee 0001
  • Ali Seyedmazloom
  • Giridhar Kaushik
  • Kookjin Lee
  • Noseong Park

0-1 knapsack is of fundamental importance across many fields. In this paper, we present a game-theoretic method to solve 0-1 knapsack problems (KPs) where the number of items (products) is large and the values of items are not predetermined but decided by an external value assignment function (e. g. , a neural network in our case) during the optimization process. While existing papers are interested in predicting solutions with neural networks for classical KPs whose objective functions are mostly linear functions, we are interested in solving KPs whose objective functions are neural networks. In other words, we choose a subset of items that maximize the sum of the values predicted by neural networks. Its key challenge is how to optimize the neural network-based non-linear KP objective with a budget constraint. Our solution is inspired by game-theoretic approaches in deep learning, e. g. , generative adversarial networks. After formally defining our two-player game, we develop an adaptive gradient ascent method to solve it. In our experiments, our method successfully solves two neural network-based non-linear KPs and conventional linear KPs with 1 million items.

Details

AAAI Conference 2021 Conference Paper

DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation

  • Jungeun Kim
  • Kookjin Lee
  • Dongeun Lee
  • Sheo Yon Jhin
  • Noseong Park

We present a method for learning dynamics of complex physical processes described by time-dependent nonlinear partial differential equations (PDEs). Our particular interest lies in extrapolating solutions in time beyond the range of temporal domain used in training. Our choice for a baseline method is physics-informed neural network (PINN) because the method parameterizes not only the solutions, but also the equations that describe the dynamics of physical processes. We demonstrate that PINN performs poorly on extrapolation tasks in many benchmark problems. To address this, we propose a novel method for better training PINN and demonstrate that our newly enhanced PINNs can accurately extrapolate solutions in time. Our method shows up to 72% smaller errors than existing methods in terms of the standard L2-norm metric.

PDF Details

NeurIPS Conference 2021 Conference Paper

Invertible Tabular GANs: Killing Two Birds with One Stone for Tabular Data Synthesis

  • Jaehoon Lee
  • Jihyeon Hyeong
  • Jinsung Jeon
  • Noseong Park
  • Jihoon Cho

Tabular data synthesis has received wide attention in the literature. This is because available data is often limited, incomplete, or cannot be obtained easily, and data privacy is becoming increasingly important. In this work, we present a generalized GAN framework for tabular synthesis, which combines the adversarial training of GANs and the negative log-density regularization of invertible neural networks. The proposed framework can be used for two distinctive objectives. First, we can further improve the synthesis quality, by decreasing the negative log-density of real records in the process of adversarial training. On the other hand, by increasing the negative log-density of real records, realistic fake records can be synthesized in a way that they are not too much close to real records and reduce the chance of potential information leakage. We conduct experiments with real-world datasets for classification, regression, and privacy attacks. In general, the proposed method demonstrates the best synthesis quality (in terms of task-oriented evaluation metrics, e. g. , F1) when decreasing the negative log-density during the adversarial training. If increasing the negative log-density, our experimental results show that the distance between real and fake records increases, enhancing robustness against privacy attacks.

PDF Details

NeurIPS Conference 2021 Conference Paper

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

  • Kanghyun Choi
  • Deokki Hong
  • Noseong Park
  • Youngsok Kim
  • Jinho Lee

Model quantization is known as a promising method to compress deep neural networks, especially for inferences on lightweight mobile or edge devices. However, model quantization usually requires access to the original training data to maintain the accuracy of the full-precision models, which is often infeasible in real-world scenarios for security and privacy issues. A popular approach to perform quantization without access to the original data is to use synthetically generated samples, based on batch-normalization statistics or adversarial learning. However, the drawback of such approaches is that they primarily rely on random noise input to the generator to attain diversity of the synthetic samples. We find that this is often insufficient to capture the distribution of the original data, especially around the decision boundaries. To this end, we propose Qimera, a method that uses superposed latent embeddings to generate synthetic boundary supporting samples. For the superposed embeddings to better reflect the original distribution, we also propose using an additional disentanglement mapping layer and extracting information from the full-precision model. The experimental results show that Qimera achieves state-of-the-art performances for various settings on data-free quantization. Code is available at https: //github. com/iamkanghyunchoi/qimera.

PDF Details

IJCAI Conference 2019 Conference Paper

FakeTables: Using GANs to Generate Functional Dependency Preserving Tables with Bounded Real Data

  • Haipeng Chen
  • Sushil Jajodia
  • Jing Liu
  • Noseong Park
  • Vadim Sokolov
  • V. S. Subrahmanian

In many cases, an organization wishes to release some data, but is restricted in the amount of data to be released due to legal, privacy and other concerns. For instance, the US Census Bureau releases only 1% of its table of records every year, along with statistics about the entire table. However, the machine learning (ML) models trained on the released sub-table are usually sub-optimal. In this paper, our goal is to find a way to augment the sub-table by generating a synthetic table from the released sub-table, under the constraints that the generated synthetic table (i) has similar statistics as the entire table, and (ii) preserves the functional dependencies of the released sub-table. We propose a novel generative adversarial network framework called ITS-GAN, where both the generator and the discriminator are specifically designed to satisfy these two constraints. By evaluating the augmentation performance of ITS-GAN on two representative datasets, the US Census Bureau data and US Bureau of Transportation Statistics (BTS) data, we show that ITS-GAN yields high quality classification results, and significantly outperforms various state-of-the-art data augmentation approaches.

PDF Details

IJCAI Conference 2019 Conference Paper

VEST: A System for Vulnerability Exploit Scoring & Timing

  • Haipeng Chen
  • Jing Liu
  • Rui Liu
  • Noseong Park
  • V. S. Subrahmanian

Knowing if/when a cyber-vulnerability will be exploited and how severe the vulnerability is can help enterprise security officers (ESOs) come up with appropriate patching schedules. Today, this ability is severely compromised: our study of data from Mitre and NIST shows that on average there is a 132 day gap between the announcement of a vulnerability by Mitre and the time NIST provides an analysis with severity score estimates and 8 important severity attributes. Many attacks happen during this very 132-day window. We present Vulnerability Exploit Scoring \& Timing (VEST), a system for (early) prediction and visualization of if/when a vulnerability will be exploited, and its estimated severity attributes and score.

PDF Details

IJCAI Conference 2018 Conference Paper

MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

  • David Keetae Park
  • Seungjoo Yoo
  • Hyojin Bahng
  • Jaegul Choo
  • Noseong Park

Recently, generative adversarial networks (GANs) have shown promising performance in generating realistic images. However, they often struggle in learning complex underlying modalities in a given dataset, resulting in poor-quality generated images. To mitigate this problem, we present a novel approach called mixture of experts GAN (MEGAN), an ensemble approach of multiple generator networks. Each generator network in MEGAN specializes in generating images with a particular subset of modalities, e. g. , an image class. Instead of incorporating a separate step of handcrafted clustering of multiple modalities, our proposed model is trained through an end-to-end learning of multiple generators via gating networks, which is responsible for choosing the appropriate generator network for a given condition. We adopt the categorical reparameterization trick for a categorical decision to be made in selecting a generator while maintaining the flow of the gradients. We demonstrate that individual generators learn different and salient subparts of the data and achieve a multiscale structural similarity (MS-SSIM) score of 0. 2470 for CelebA and a competitive unsupervised inception score of 8. 33 in CIFAR-10.

PDF Details

TIST Journal 2017 Journal Article

Data-Driven Frequency-Based Airline Profit Maximization

  • Bo An
  • Haipeng Chen
  • Noseong Park
  • V. S. Subrahmanian

Although numerous traditional models predict market share and demand along airline routes, the prediction of existing models is not precise enough, and to the best of our knowledge, there is no use of data mining--based forecasting techniques for improving airline profitability. We propose the maximizing airline profits (MAP) architecture designed to help airlines and make two key contributions in airline market share and route demand prediction and prediction-based airline profit optimization. Compared to past methods used to forecast market share and demand along airline routes, we introduce a novel ensemble forecasting (MAP-EF) approach considering two new classes of features: (i) features derived from clusters of similar routes and (ii) features based on equilibrium pricing. We show that MAP-EF achieves much better Pearson correlation coefficients (greater than 0.95 vs. 0.82 for market share, 0.98 vs. 0.77 for demand) and R 2 -values compared to three state-of-the-art works for forecasting market share and demand while showing much lower variance. Using the results of MAP-EF, we develop MAP--bilevel branch and bound (MAP-BBB) and MAP-greedy (MAP-G) algorithms to optimally allocate flight frequencies over multiple routes to maximize an airline’s profit. We also study two extensions of the profit maximization problem considering frequency constraints and long-term profits. Furthermore, we develop algorithms for computing Nash equilibrium frequencies when there are multiple strategic airlines. Experimental results show that airlines can increase profits by a significant margin. All experiments were conducted with data aggregated from four sources: the U.S. Bureau of Transportation Statistics (BTS), the U.S. Bureau of Economic Analysis (BEA), the National Transportation Safety Board (NTSB), and the U.S. Census Bureau (CB).

Details DOI

IS Journal 2015 Journal Article

Saving rhinos with predictive analytics

  • Noseong Park
  • Edoardo Serra
  • V.S. Subrahmanian

This article, the first entry in the new Predictive Analytics column, looks at the problem of animal poaching. The authors describe their Anti-Poaching Engine system, which builds on behavior models of both rhinos and poachers to protect as many animals as possible.

Details DOI
v2026.09.13