Author name cluster

Yen-Ling Kuo

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

13 papers

2 author rows

AAMAS Conference 2026 Conference Paper

MENSA: Leveraging Mental Simulation for In-Context Policy Improvement in LLM Agents

Chung-Che Chang
Erick Chandra
Jane Yung-jen Hsu
Yen-Ling Kuo

Large Language Model (LLM) powered agents have shown promise in sequential decision-making tasks in interactive environments. However, prior agent frameworks usually rely on advanced LLM capabilities such as planning or instruction following to carry out tasks successfully. Effectively improving the performance of an LLM agent without assuming these capabilities remains challenging. To address this issue, we propose MENtal Simulation Agent (MENSA), anovelmodel-basedapproachthatenhancesLLMagents without fine-tuning. MENSA leverages the fundamental ability of any LLMs, text completion, to generate forecasts of action-state pairs(i. e. ,transitions)forfuturetimesteps. Theseforecastsareused toconstructasetofrelevantpastexperiences, whichareprovidedto the LLM agent in context to improve its decision-making behavior. We evaluate MENSA in two challenging interactive environments, ScienceWorld and NetHack, and show that MENSA improves performance across various sizes of LLMs. Using large models (e. g. , GPT-4o-mini), MENSA outperforms previous state-of-the-art methodsby+15. 8pointsinScienceWorldandby+40. 0pointsinNetHack. Even with smaller models like Phi-3-mini, MENSA achieves a gain of +11. 9 points in ScienceWorld. Our results further suggest that MENSA is less affected by an LLM’s limitations in instructionfollowing and planning compared to baselines. Project page and code are available at: https: //roger0426. github. io/MENSA.

PDF

ICRA Conference 2025 Conference Paper

Diff-Dagger: Uncertainty Estimation With Diffusion Policy for Robotic Manipulation

Sung-Wook Lee
Xuhui Kang
Yen-Ling Kuo

Recently, diffusion policy has shown impressive results in handling multi-modal tasks in robotic manipulation. However, it has fundamental limitations in out-of-distribution failures that persist due to compounding errors and its limited capability to extrapolate. One way to address these limitations is robot-gated DAgger, an interactive imitation learning with a robot query system to actively seek expert help during policy rollout. While robot-gated DAgger has high potential for learning at scale, existing methods like Ensemble-DAgger struggle with highly expressive policies: They often misinterpret policy disagreements as uncertainty at multi-modal decision points. To address this problem, we introduce Diff-DAgger, an efficient robot-gated DAgger algorithm that leverages the training objective of diffusion policy. We evaluate Diff-DAgger across different robot tasks including stacking, pushing, and plugging, and show that Diff-DAgger improves the task failure prediction by 39. 0 %, the task completion rate by 20. 6 %, and reduces the wall-clock time by a factor of 7. 8. We hope that this work opens up a path for efficiently incorporating expressive yet data-hungry policies into interactive robot learning settings. The project website is available at: https://diffdagger.github.io.

Details

AAAI Conference 2025 Conference Paper

MuMA-ToM: Multi-modal Multi-Agent Theory of Mind

Haojun Shi
Suyu Ye
Xinyu Fang
Chuanyang Jin
Leyla Isik
Yen-Ling Kuo
Tianmin Shu

Understanding people's social interactions in complex real-world scenarios often relies on intricate mental reasoning. To truly understand how and why people interact with one another, we must infer the underlying mental states that give rise to the social interactions, i.e., Theory of Mind reasoning in multi-agent interactions. Additionally, social interactions are often multi-modal -- we can watch people's actions, hear their conversations, and/or read about their past behaviors. For AI systems to successfully and safely interact with people in real-world environments, they also need to understand people's mental states as well as their inferences about each other's mental states based on multi-modal information about their interactions. For this, we introduce MuMA-ToM, a Multi-modal Multi-Agent Theory of Mind benchmark. MuMA-ToM is the first multi-modal Theory of Mind benchmark that evaluates mental reasoning in embodied multi-agent interactions. In MuMA-ToM, we provide video and text descriptions of people's multi-modal behavior in realistic household environments. Based on the context, we then ask questions about people's goals, beliefs, and beliefs about others' goals. We validated MuMA-ToM in a human experiment and provided a human baseline. We also proposed a novel multi-modal, multi-agent ToM model, LIMP (Language model-based Inverse Multi-agent Planning). Our experimental results show that LIMP significantly outperforms state-of-the-art methods, including large multi-modal models (e.g., GPT-4o, Gemini-1.5 Pro) and a recent multi-modal ToM model, BIP-ALM.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Learning Representations for Robust Human-Robot Interaction

Yen-Ling Kuo

For robots to robustly and flexibly interact with humans, they need to acquire skills to use across scenarios. One way to enable the generalization of skills is to learn representations that are useful for downstream tasks. Learning a representation for interactions requires an understanding of what (e.g., objects) as well as how (e.g., actions, controls, and manners) to interact with. However, most existing language or visual representations mainly focus on objects. To enable robust human-robot interactions, we need a representation that is not just grounded at the object level but to reason at the action level. The ability to reason about an agent’s own actions and other’s actions will be crucial for long-tail interactions. My research focuses on leveraging the compositional nature of language and reward functions to learn representations that generalize to novel scenarios. Together with the information from multiple modalities, the learned representation can reason about task progress, future behaviors, and the goals/beliefs of an agent. The above ideas have been demonstrated in my research on building robots to understand language and engage in social interactions.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Neural Amortized Inference for Nested Multi-Agent Reasoning

Kunal Jha
Tuan Anh Le
Chuanyang Jin
Yen-Ling Kuo
Joshua B. Tenenbaum
Tianmin Shu

Multi-agent interactions, such as communication, teaching, and bluffing, often rely on higher-order social inference, i.e., understanding how others infer oneself. Such intricate reasoning can be effectively modeled through nested multi-agent reasoning. Nonetheless, the computational complexity escalates exponentially with each level of reasoning, posing a significant challenge. However, humans effortlessly perform complex social inferences as part of their daily lives. To bridge the gap between human-like inference capabilities and computational limitations, we propose a novel approach: leveraging neural networks to amortize high-order social inference, thereby expediting nested multi-agent reasoning. We evaluate our method in two challenging multi-agent interaction domains. The experimental results demonstrate that our method is computationally efficient while exhibiting minimal degradation in accuracy.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Zero-Shot Linear Combinations of Grounded Social Interactions with Linear Social MDPs

Ravi Tejwani
Yen-Ling Kuo
Tianmin Shu
Bennett Stankovits
Dan Gutfreund
Joshua B. Tenenbaum
Boris Katz
Andrei Barbu

Humans and animals engage in rich social interactions. It is often theorized that a relatively small number of basic social interactions give rise to the full range of behavior observed. But no computational theory explaining how social interactions combine together has been proposed before. We do so here. We take a model, the Social MDP, which is able to express a range of social interactions, and extend it to represent linear combinations of social interactions. Practically for robotics applications, such models are now able to not just express that an agent should help another agent, but to express goal-centric social interactions. Perhaps an agent is helping someone get dressed, but preventing them from falling, and is happy to exchange stories in the meantime. How an agent responds socially, should depend on what it thinks the other agent is doing at that point in time. To encode this notion, we take linear combinations of social interactions as defined in Social MDPs, and compute the weights on those combinations on the fly depending on the estimated goals of other agents. This new model, the Linear Social MDP, enables zero-shot reasoning about complex social interactions, provides a mathematical basis for the long-standing intuition that social interactions should compose, and leads to interesting new behaviors that we validate using human observers. Complex social interactions are part of the future of intelligent agents, and having principled mathematical models built on a foundation like MDPs will make it possible to bring social interactions to every robotic application.

PDF Details DOI

ICRA Conference 2022 Conference Paper

Incorporating Rich Social Interactions Into MDPs

Ravi Tejwani
Yen-Ling Kuo
Tianmin Shu
Bennett Stankovits
Dan Gutfreund
Joshua B. Tenenbaum
Boris Katz
Andrei Barbu

Much of what we do as humans is engage socially with other agents, a skill that robots must also eventually possess. We demonstrate that a rich theory of social interactions originating from microsociology can be formalized by extending a nested MDP where agents reason about arbitrary functions of each other's rewards. This extended Social MDP allows us to encode the five basic interactions that underlie microsociology: cooperation, conflict, coercion, competition, and exchange. The result is a robotic agent capable of executing social interactions in new environments with no interaction-specific training; like humans it can engage socially in novel ways even without a single example of that social interaction. Moreover, the estimations of these Social MDPs align closely with the judge-ments of humans when considering which social interaction is taking place in an environment. This method both sheds light on the nature of social interactions, by providing concrete mathematical definitions, and brings rich social interactions into a mathematical framework that has proven to be natural for robotics.

Details

ICRA Conference 2022 Conference Paper

Trajectory Prediction with Linguistic Representations

Yen-Ling Kuo
Xin Huang 0018
Andrei Barbu
Stephen G. McGill
Boris Katz
John J. Leonard
Guy Rosman

Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories, and is trained using trajectory samples with partially-annotated captions. The model learns the meaning of each of the words without direct per-word supervision. At inference time, it generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval. This generated description is used to refine predictions of the trajectories of multiple agents. We train and validate our model on the Argoverse dataset, and demonstrate improved accuracy results in trajectory prediction. In addition, our model is more interpretable: it presents part of its reasoning in plain language as captions, which can aid model development and can aid in building confidence in the model before deploying it.

Details

ICRA Conference 2020 Conference Paper

Deep compositional robotic planners that follow natural language commands

Yen-Ling Kuo
Boris Katz
Andrei Barbu

We demonstrate how a sampling-based robotic planner can be augmented to learn to understand a sequence of natural language commands in a continuous configuration space to move and manipulate objects. Our approach combines a deep network structured according to the parse of a complex command that includes objects, verbs, spatial relations, and attributes, with a sampling-based planner, RRT. A recurrent hierarchical deep network controls how the planner explores the environment, determines when a planned path is likely to achieve a goal, and estimates the confidence of each move to trade off exploitation and exploration between the network and the planner. Planners are designed to have near-optimal behavior when information about the task is missing, while networks learn to exploit observations which are available from the environment, making the two naturally complementary. Combining the two enables generalization to new maps, new kinds of obstacles, and more complex sentences that do not occur in the training set. Little data is required to train the model despite it jointly acquiring a CNN that extracts features from the environment as it learns the meanings of words. The model provides a level of interpretability through the use of attention maps allowing users to see its reasoning steps despite being an end-to-end model. This end-to-end model allows robots to learn to follow natural language commands in challenging continuous environments.

Details

IROS Conference 2020 Conference Paper

Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas

Yen-Ling Kuo
Boris Katz
Andrei Barbu

We demonstrate a reinforcement learning agent which uses a compositional recurrent neural network that takes as input an LTL formula and determines satisfying actions. The input LTL formulas have never been seen before, yet the network performs zero-shot generalization to satisfy them. This is a novel form of multi-task learning for RL agents where agents learn from one diverse set of tasks and generalize to a new set of diverse tasks. The formulation of the network enables this capacity to generalize. We demonstrate this ability in two domains. In a symbolic domain, the agent finds a sequence of letters that is accepted. In a Minecraft-like environment, the agent finds a sequence of actions that conform to the formula. While prior work could learn to execute one formula reliably given examples of that formula, we demonstrate how to encode all formulas reliably. This could form the basis of new multitask agents that discover sub-tasks and execute them without any additional training, as well as the agents which follow more complex linguistic commands. The structures required for this generalization are specific to LTL formulas, which opens up an interesting theoretical question: what structures are required in neural networks for zero-shot generalization to different logics?

Details

IROS Conference 2018 Conference Paper

Deep Sequential Models for Sampling-Based Planning

Yen-Ling Kuo
Andrei Barbu
Boris Katz

We demonstrate how a sequence model and a sampling-based planner can influence each other to produce efficient plans and how such a model can automatically learn to take advantage of observations of the environment. Sampling-based planners such as RRT generally know nothing of their environments even if they have traversed similar spaces many times. A sequence model, such as an HMM or LSTM, guides the search for good paths. The resulting model, called DeRRT*, observes the state of the planner and the local environment to bias the next move and next planner state. The neural-network-based models avoid manual feature engineering by co-training a convolutional network which processes map features and observations from sensors. We incorporate this sequence model in a manner that combines its likelihood with the existing bias for searching large unexplored Voronoi regions. This leads to more efficient trajectories with fewer rejected samples even in difficult domains such as when escaping bug traps. This model can also be used for dimensionality reduction in multi-agent environments with dynamic obstacles. Instead of planning in a high-dimensional space that includes the configurations of the other agents, we plan in a low-dimensional subspace relying on the sequence model to bias samples using the observed behavior of the other agents. The techniques presented here are general, include both graphical models and deep learning approaches, and can be adapted to a range of planners.

Details

AAMAS Conference 2011 Conference Paper

Multiagent Environment Design in Human Computation

Chien-Ju Ho
Yen-Ling Kuo
Jane Yung-jen Hsu

This research explores human computation systems as a multi-agent platform for coordinating human brain power. We are interested in the problem of designing human computation systems. In particular, we focus on the situation where developers cannot re-design the whole platform but can only make limited changes to the environments. We extend the framework of environment design to multiple agent in the context of human computation. To incorporate the collective information from multiple agents, we propose two approaches, agent type elicitation and collaborative filtering, under different assumptions. The formulation and algorithms provide solutions for developers in human computation systems to find the environment settings maximizing their goal functions.

PDF

IJCAI Conference 2011 Conference Paper

Resource-Bounded Crowd-Sourcing of Commonsense Knowledge

Yen-Ling Kuo
Jane Yung-jen Hsu

Knowledge acquisition is the essential process of extracting and encoding knowledge, both domainspecific and commonsense, to be used in intelligent systems. While many large knowledge bases have been constructed, none is close to complete. This paper presents an approach to improving a knowledge base efficiently under resource constraints. Using a guiding knowledge base, questions are generated from a weak form of similarity-based inference given the glossary mapping between two knowledge bases. The candidate questions are prioritized in terms of the concept coverage of the target knowledge. Experiments were conducted to find questions to grow the Chinese ConceptNet using the English ConceptNet as a guide. The results were evaluated by online users to verify that 94. 17% of the questions and 85. 77% of the answersare good. In addition, the answers collected in a six-week period showed consistent improvement to a 36. 33% increase in concept coverage of the Chinese commonsense knowledge base against the English ConceptNet.

PDF Details DOI