Author name cluster

Minsik Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

EAAI Journal 2025 Journal Article

Self-supervised learning with ensemble representations

Kyoungmin Han
Minsik Lee

Many computer vision applications, such as medical image processing, have struggled with a lack of labeled data. Recently, contrastive self-supervised learning has made remarkable progress in unsupervised representation learning and become a promising alternative in situations with limited labeled data. The simple Siamese network (SimSiam) is a well-known example in this area, known for its simplicity despite its powerful performance. However, it is known to be sensitive to changes in training configurations, such as hyperparameters and augmentation settings, due to its structural characteristics. To address this issue, we focus on the structural similarity between contrastive learning and the teacher–student framework in knowledge distillation. Inspired by the ensemble-based knowledge distillation approach, we propose a new self-supervised learning method, the ensemble representation model for simple Siamese networks (EnSiam), by introducing an ensemble representation in pseudo labels. This can reduce the variance in the contrastive learning procedure, providing better performance. Experiments demonstrate that EnSiam performs better than existing methods for popular datasets, showing that EnSiam is capable of learning high-quality representations. Furthermore, we extend our approach to various other self-supervised learning methods and empirically confirm that ensemble representations can consistently improve performance in general self-supervised learning.

Details DOI

ICLR Conference 2025 Conference Paper

SplineGS: Learning Smooth Trajectories in Gaussian Splatting for Dynamic Scene Reconstruction

Jihwan Yoon
Sangbeom Han
Jaeseok Oh
Minsik Lee

Reconstructing complex scenes with deforming objects for novel view synthesis is a challenging task. Recent works have addressed this with 3D Gaussian Splatting, which effectively reconstructs static scenes with high quality in short training time, by adding specialized modules for the deformations of Gaussian blobs. However, designing an effective deformation module that incorporates appropriate spatiotemporal inductive biases still remains unresolved. To address this issue, we propose SplineGS in this paper, which utilizes non-uniform rational B-splines (NURBS), an extension of B-spline, to represent temporally smooth deformation. A set of representative trajectories are learned based on NURBS, and the individual trajectories of Gaussian blobs are represented as linear combinations of these trajectories for spatial smoothness. The weights of the combinations are trained based on a multi-resolution hash table and an MLP, with the positions of the Gaussian blobs as the keys. Thanks to this design, the proposed method does not need any regularizers for trajectories, which enables efficient training. Experiments demonstrate that the proposed method provides competitive performance over the existing methods with much shorter training time.

Details

AAAI Conference 2019 Conference Paper

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary

Byeongho Heo
Minsik Lee
Sangdoo Yun
Jin Young Choi

Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its decision boundary, so a good classifier bears a good decision boundary. Therefore, transferring information closely related to the decision boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a decision boundary. Based on this idea, to transfer more accurate information about the decision boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the decision boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the stateof-the-arts performance.

PDF Details

AAAI Conference 2019 Conference Paper

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons

Byeongho Heo
Minsik Lee
Sangdoo Yun
Jin Young Choi

An activation boundary for a neuron refers to a separating hyperplane that determines whether the neuron is activated or deactivated. It has been long considered in neural networks that the activations of neurons, rather than their exact output values, play the most important role in forming classificationfriendly partitions of the hidden feature space. However, as far as we know, this aspect of neural networks has not been considered in the literature of knowledge transfer. In this paper, we propose a knowledge transfer method via distillation of activation boundaries formed by hidden neurons. For the distillation, we propose an activation transfer loss that has the minimum value when the boundaries generated by the student coincide with those by the teacher. Since the activation transfer loss is not differentiable, we design a piecewise differentiable loss approximating the activation transfer loss. By the proposed method, the student learns a separating boundary between activation region and deactivation region formed by each neuron in the teacher. Through the experiments in various aspects of knowledge transfer, it is verified that the proposed method outperforms the current state-of-the-art.

PDF Details