Author name cluster

Jaedeok Kim

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

AAAI Conference 2020 Conference Paper

Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks

Jaedeok Kim
Chiyoun Park
Hyun-Joo Jung
Yoonsuck Choe

Architecture optimization, which is a technique for ﬁnding an efﬁcient neural network that meets certain requirements, generally reduces to a set of multiple-choice selection problems among alternative sub-structures or parameters. The discrete nature of the selection problem, however, makes this optimization difﬁcult. To tackle this problem we introduce a novel concept of a trainable gate function. The trainable gate function, which confers a differentiable property to discretevalued variables, allows us to directly optimize loss functions that include non-differentiable discrete values such as 0-1 selection. The proposed trainable gate can be applied to pruning. Pruning can be carried out simply by appending the proposed trainable gate functions to each intermediate output tensor followed by ﬁne-tuning the overall model, using any gradient-based training methods. So the proposed method can jointly optimize the selection of the pruned channels while ﬁne-tuning the weights of the pruned model at the same time. Our experimental results demonstrate that the proposed method efﬁciently optimizes arbitrary neural networks in various tasks such as image classiﬁcation, style transfer, optical ﬂow estimation, and neural machine translation.

PDF Details

AAAI Conference 2019 Short Paper

Comparing Sample-Wise Learnability across Deep Neural Network Models

Seung-Geon Lee
Jaedeok Kim
Hyun-Joo Jung
Yoonsuck Choe

Estimating the relative importance of each sample in a training set has important practical and theoretical value, such as in importance sampling or curriculum learning. This kind of focus on individual samples invokes the concept of samplewise learnability: How easy is it to correctly learn each sample (cf. PAC learnability)? In this paper, we approach the sample-wise learnability problem within a deep learning context. We propose a measure of the learnability of a sample with a given deep neural network (DNN) model. The basic idea is to train the given model on the training set, and for each sample, aggregate the hits and misses over the entire training epochs. Our experiments show that the samplewise learnability measure collected this way is highly linearly correlated across different DNN models (ResNet-20, VGG-16, and MobileNet), suggesting that such a measure can provide deep general insights on the data’s properties. We expect our method to help develop better curricula for training, and help us better understand the data itself.

PDF Details