Author name cluster

Seongmin Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

AAAI Conference 2025 System Paper

LLM Attributor: Interactive Visual Attribution for LLM Generation

Seongmin Lee
Zijie J. Wang
Aishwarya Chakravarthy
Alec Helbling
ShengYun Peng
Mansi Phute
Duen Horng (Polo) Chau
Minsuk Kahng

While large language models (LLMs) have shown remarkable capability to generate convincing text across diverse domains, concerns around its potential risks have highlighted the importance of understanding the rationale behind text generation. We present LLM ATTRIBUTOR, a Python library that provides interactive visualizations for training data attribution of an LLM’s text generation. Our library offers a new way to quickly attribute an LLM’s text generation to training data points to inspect model behaviors, enhance its trustworthiness, and compare model-generated text with user-provided text. Thanks to LLM ATTRIBUTOR’s broad support for computational notebooks, users can easily integrate it into their workflow to interactively visualize attributions of their models.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Shape it Up! Restoring LLM Safety during Finetuning

ShengYun Peng
Pin-Yu Chen
Jianfeng Chi
Seongmin Lee
Duen Horng Chau

Finetuning large language models (LLMs) enables user-specific customization but introduces important safety risks: even a few harmful examples can compromise safety alignment. A common mitigation strategy is to update the model more strongly on examples deemed safe, while downweighting or excluding those flagged as unsafe. However, because safety context can shift within a single example, updating the model equally on both harmful and harmless parts of a response is suboptimal — an atomic treatment we term static safety shaping. In contrast, we propose dynamic safety shaping (DSS), a dynamic shaping framework that uses fine-grained safety signals to reinforce learning from safe segments of a response while suppressing unsafe content. To enable such fine-grained control during finetuning, we introduce a key insight: guardrail models, traditionally used for filtering, can be repurposed to evaluate partial responses, tracking how safety risk evolves throughout the response, segment by segment. This leads to the Safety Trajectory Assessment of Response (STAR), a token-level signal that enables shaping to operate dynamically over the training sequence. Building on this, we present ★DSS, a DSS method guided by STAR scores that robustly mitigates finetuning risks and delivers substantial safety improvements across diverse threats, datasets, and model families, all without compromising capability on intended tasks. We encourage future safety research to build on dynamic shaping principles for stronger mitigation against evolving finetuning risks. Our code is publicly available at https: //github. com/poloclub/star-dss

PDF Details

AAAI Conference 2025 System Paper

TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models

Aeree Cho
Grace C. Kim
Alexander Karpekov
Alec Helbling
Zijie J. Wang
Seongmin Lee
Benjamin Hoover
Duen Horng (Polo) Chau

Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present TRANSFORMER EXPLAINER, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and smooth transitions across abstraction levels of math operations and model structures. It runs a live GPT-2 model locally in the user’s browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. 125,000 users have used our open-source tool at https://poloclub.github.io/ transformer-explainer/.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

AVIN-Chat: An Audio-Visual Interactive Chatbot System with Emotional State Tuning

Chanhyuk Park
Jungbin Cho
Junwan Kim
Seongmin Lee
Jungsu Kim
Sanghoon Lee

This work presents an audio-visual interactive chatbot (AVIN-Chat) system that allows users to have face-to-face conversations with 3D avatars in real-time. Compared to the previous chatbot services, which provide text-only or speech-only communications, the proposed AVIN-Chat can offer audio-visual communications providing users with a superior experience quality. In addition, the proposed AVIN-Chat emotionally speaks and expresses according to the user's emotional state. Thus, it enables users to establish a strong bond with the chatbot system, increasing the user's immersion. Through user subjective tests, it is demonstrated that the proposed system provides users with a higher sense of immersion than previous chatbot systems. The demonstration video is available at https: //www. youtube. com/watch? v=Z74uIV9k7_k.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

Interactive Visual Learning for Stable Diffusion

Seongmin Lee
Benjamin Hoover
Hendrik Strobelt
Zijie J. Wang
ShengYun Peng
Austin Wright
Kevin Li
Haekyu Park

Diffusion-based generative models’ impressive ability to create convincing images has garnered global attention. However, their complex internal structures and operations often pose challenges for non-experts to grasp. We introduce Diffusion Explainer, the first interactive visualization tool designed to elucidate how Stable Diffusion transforms text prompts into images. It tightly integrates a visual overview of Stable Diffusion’s complex components with detailed explanations of their underlying operations. This integration enables users to fluidly transition between multiple levels of abstraction through animations and interactive elements. Offering real-time hands-on experience, Diffusion Explainer allows users to adjust Stable Diffusion’s hyperparameters and prompts without the need for installation or specialized hardware. Accessible via users’ web browsers, Diffusion Explainer is making significant strides in democratizing AI education, fostering broader public access. More than 7, 200 users spanning 113 countries have used our open-sourced tool at https: //poloclub. github. io/diffusion-explainer/. A video demo is available at https: //youtu. be/MbkIADZjPnA.

PDF Details DOI

IJCAI Conference 2024 Conference Paper

InViTe: Individual Virtual Transfer for Personalized 3D Face Generation System

Mingyu Jang
Kyungjune Lee
Seongmin Lee
Hoseok Tong
Juwan Chung
Yusung Ro
Sanghoon Lee

With the expansion of the virtual communication industry using VR/AR, it has attracted increasing attention to enable users to represent their personalities in a 3D avatar. As the face of 3D avatars plays a crucial role in conveying human personality, a system that generates and manipulates 3D faces is desired. However, establishing the system is challenging due to the need for human effort and specialized knowledge. To fill this void, we present the Individual Virtual Transfer (InViTe), which enables the creation and customization of a 3D face according to the user's preference. Our proposed system is featured for 1) 3D face reconstruction with high fidelity texture map, 2) 3D face personalization, 3) realistic rendering results, and 4) real-time mobile virtual applications. We conduct an experiment to demonstrate that the proposed system can achieve sufficient individual personalization of 3D faces. Furthermore, we evaluate the system's data transmission protocol and demonstrate its efficiency. The demonstration video is available at https: //www. youtube. com/watch? v=D 4pXZvGUWU.

PDF Details DOI