Chi-Hoon Lee

ICAPS Conference 2024 Conference Paper

Control in Stochastic Environment with Delays: A Model-based Reinforcement Learning Approach

• Zhiyuan Yao 0001
• Ionut Florescu
• Chi-Hoon Lee

In this paper we are introducing a new reinforcement learning method for control problems in environments with delayed feedback. Specifically, our method employs stochastic planning, versus previous methods that used deterministic planning. This allows us to embed risk preference in the policy optimization problem. We show that this formulation can recover the optimal policy for problems with deterministic transitions. We contrast our policy with two prior methods from literature. We apply the methodology to simple tasks to understand its features. Then, we compare the performance of the methods in controlling multiple Atari games.

AAAI Conference 2008 Short Paper

Constrained Classification on Structured Data

• Chi-Hoon Lee
• Russell Greiner

Most standard learning algorithms, such as Logistic Regression (LR) and the Support Vector Machine (SVM), are designed to deal with i. i. d. (independent and identically distributed) data. They therefore do not work effectively for tasks that involve non-i. i. d. data, such as “region segmentation”. (Eg, the “tumor vs non-tumor” labels in a medical image are correlated, in that adjacent pixels typically have the same label.) This has motivated the work in random fields, which has produced classifiers for such non-i. i. d. data that are significantly better than standard i. i. d. -based classifiers. However, these random field methods are often too slow to be trained for the tasks they were designed to solve. This paper presents a novel variant, Pseudo Conditional Random Fields (PCRFs), that is also based on i. i. d. learners, to allow efficient training but also incorporates correlations, like random fields. We demonstrate that this system is as accurate as other random fields variants, but significantly faster to train.

NeurIPS Conference 2006 Conference Paper

Learning to Model Spatial Dependency: Semi-Supervised Discriminative Random Fields

• Chi-Hoon Lee
• Shaojun Wang
• Feng Jiao
• Dale Schuurmans
• Russell Greiner

We present a novel, semi-supervised approach to training discriminative random fields (DRFs) that efficiently exploits labeled and unlabeled training data to achieve improved accuracy in a variety of image processing tasks. We formulate DRF training as a form of MAP estimation that combines conditional loglikelihood on labeled data, given a data-dependent prior, with a conditional entropy regularizer defined on unlabeled data. Although the training objective is no longer concave, we develop an efficient local optimization procedure that produces classifiers that are more accurate than ones based on standard supervised DRF training. We then apply our semi-supervised approach to train DRFs to segment both synthetic and real data sets, and demonstrate significant improvements over supervised DRFs in each case.

ICML Conference 2006 Conference Paper

Using query-specific variance estimates to combine Bayesian classifiers

• Chi-Hoon Lee
• Russell Greiner
• Shaojun Wang