Author name cluster

Dongho Kim

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

YNIMG Journal 2025 Journal Article

7T Spin-echo BOLD fMRI enhances spatial specificity in the human motor cortex during finger movement tasks

Sohyun Han
Dongho Kim
Seulgi Eun
HyungJoon Cho
Seong-Gi Kim

The human primary motor cortex (M1) follows a well-established somatotopic organization, yet finer-scale representations, such as mirrored finger maps, have remained difficult to resolve non-invasively. To investigate movement representations in an action-based framework rather than a strictly somatotopic layout, we conducted both conventional gradient-echo (GE) and highly specific spin-echo (SE) BOLD fMRI at 7 T with 1 mm isotropic resolution. Subjects performed 1-Hz visually-instructed thumb-index finger or thumb-ring finger opposition tasks, and their finger movements were recorded using an MR-compatible data glove to verify proper task performance. In each subject, the activated M1 region spanning multiple slices was subdivided into ten columns along a medial-to-lateral axis. Finger dominance (index vs. ring) was determined within each column. In GE-BOLD fMRI, two distinct tasks exhibited similar activation patterns across columns, reflecting its limited ability to resolve columnar activation differences due to contamination from draining vein effects. In contrast, SE-BOLD fMRI revealed alternating task dominance across columns, demonstrating higher spatial specificity compared to GE-BOLD. By integrating SE-BOLD fMRI, but not GE-BOLD, with behavioral data, we present a more accurate mesoscopic mapping of motor activity in individual subjects. These findings provide non-invasive evidence of fine-grained motor organization, demonstrating the utility of SE-BOLD contrast for mapping mesoscopic representations.

Details DOI

ICML Conference 2023 Conference Paper

Harmonic Neural Networks

Atiyo Ghosh
Antonio Andrea Gentile
Mario Dagrada
Chul Lee
Seong-Hyok Sean Kim
Hyukgeun Cha
Yunjun Choi
Dongho Kim

Harmonic functions are abundant in nature, appearing in limiting cases of Maxwell’s, Navier-Stokes equations, the heat and the wave equation. Consequently, there are many applications of harmonic functions from industrial process optimisation to robotic path planning and the calculation of first exit times of random walks. Despite their ubiquity and relevance, there have been few attempts to incorporate inductive biases towards harmonic functions in machine learning contexts. In this work, we demonstrate effective means of representing harmonic functions in neural networks and extend such results also to quantum neural networks to demonstrate the generality of our approach. We benchmark our approaches against (quantum) physics-informed neural networks, where we show favourable performance.

Details

ICML Conference 2020 Conference Paper

Batch Reinforcement Learning with Hyperparameter Gradients

Byung-Jun Lee 0001
Jongmin Lee 0004
Peter Vrancx
Dongho Kim
Kee-Eung Kim

We consider the batch reinforcement learning problem where the agent needs to learn only from a fixed batch of data, without further interaction with the environment. In such a scenario, we want to prevent the optimized policy from deviating too much from the data collection policy since the estimation becomes highly unstable otherwise due to the off-policy nature of the problem. However, imposing this requirement too strongly will result in a policy that merely follows the data collection policy. Unlike prior work where this trade-off is controlled by hand-tuned hyperparameters, we propose a novel batch reinforcement learning approach, batch optimization of policy and hyperparameter (BOPAH), that uses a gradient-based optimization of the hyperparameter using held-out data. We show that BOPAH outperforms other batch reinforcement learning algorithms in tabular and continuous control tasks, by finding a good balance to the trade-off between adhering to the data collection policy and pursuing the possible policy improvement.

Details

NeurIPS Conference 2018 Conference Paper

Distributed Multitask Reinforcement Learning with Quadratic Convergence

Rasul Tutunov
Dongho Kim
Haitham Bou Ammar

Multitask reinforcement learning (MTRL) suffers from scalability issues when the number of tasks or trajectories grows large. The main reason behind this drawback is the reliance on centeralised solutions. Recent methods exploited the connection between MTRL and general consensus to propose scalable solutions. These methods, however, suffer from two drawbacks. First, they rely on predefined objectives, and, second, exhibit linear convergence guarantees. In this paper, we improve over state-of-the-art by deriving multitask reinforcement learning from a variational inference perspective. We then propose a novel distributed solver for MTRL with quadratic convergence guarantees.

PDF Details

NeurIPS Conference 2012 Conference Paper

Cost-Sensitive Exploration in Bayesian Reinforcement Learning

Dongho Kim
Kee-Eung Kim
Pascal Poupart

In this paper, we consider Bayesian reinforcement learning (BRL) where actions incur costs in addition to rewards, and thus exploration has to be constrained in terms of the expected total cost while learning to maximize the expected long-term total reward. In order to formalize cost-sensitive exploration, we use the constrained Markov decision process (CMDP) as the model of the environment, in which we can naturally encode exploration requirements using the cost function. We extend BEETLE, a model-based BRL method, for learning in the environment with cost constraints. We demonstrate the cost-sensitive exploration behaviour in a number of simulated problems.

PDF Details

ICAPS Conference 2011 Conference Paper

Closing the Gap: Improved Bounds on Optimal POMDP Solutions

Pascal Poupart
Kee-Eung Kim
Dongho Kim

POMDP algorithms have made significant progress in recent years by allowing practitioners to find good solutions to increasingly large problems. Most approaches (including point-based and policy iteration techniques) operate by refining a lower bound of the optimal value function. Several approaches (e. g. , HSVI2, SARSOP, grid-based approaches and online forward search) also refine an upper bound. However, approximating the optimal value function by an upper bound is computationally expensive and therefore tightness is often sacrificed to improve efficiency (e. g. , sawtooth approximation). In this paper, we describe a new approach to efficiently compute tighter bounds by i) conducting a prioritized breadth first search over the reachable beliefs, ii) propagating upper bound improvements with an augmented POMDP and iii) using exact linear programming (instead of the sawtooth approximation) for upper bound interpolation. As a result, we can represent the bounds more compactly and significantly reduce the gap between upper and lower bounds on several benchmark problems.

Details

IJCAI Conference 2011 Conference Paper

Point-Based Value Iteration for Constrained POMDPs

Dongho Kim
Jaesong Lee
Kee-Eung Kim
Pascal Poupart

Constrained partially observable Markov decision processes (CPOMDPs) extend the standard POMDPs by allowing the specification of constraints on some aspects of the policy in addition to the optimality objective for the value function. CPOMDPs have many practical advantages over standard POMDPs since they naturally model problems involving limited resource or multiple objectives. In this paper, we show that the optimal policies in CPOMDPs can be randomized, and present exact and approximate dynamic programming methods for computing randomized optimal policies. While the exact method requires solving a minimax quadratically constrained program (QCP) in each dynamic programming update, the approximate method utilizes the point-based value update with a linear program (LP). We show that the randomized policies are significantly better than the deterministic ones. We also demonstrate that the approximate point-based method is scalable to solve large problems.

PDF Details DOI