Author name cluster

Hang Yin

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

2 author rows

AAMAS Conference 2025 Conference Paper

Context Adaptive Memory-Efficient LLM Inference for Edge Multi-Agent Systems

Hamza Mohammed
Hang Yin
Sai Chand Boyapati

Large Language Models (LLMs) excel at multi-document QA, summarization, code generation, and other language-intensive tasks, yet they demand substantial memory resources for storing keyvalue (KV) caches and processing attention in long-context scenarios. These requirements often prohibit on-device or edge deployments in multi-agent systems (MAS), where multiple agents share or update contextual information and need efficient inference pipelines. We present CASK (Context-Adaptive Sparse Key-value), an inference-time strategy that reduces memory usage while preserving strong performance on extended contexts. CASK addresses this challenge with two complementary mechanisms: a dynamic sparse attention module—a lightweight, meta-learned component—that identifies the most relevant context tokens, and an adaptive KV-cache compression technique that dynamically quantizes and prunes less critical key-value pairs based on usage frequency and recency. These innovations enable near-lossless performance on long-context tasks while cutting memory usage by up to 40% and boosting inference speed by as much as 20%. Evaluations on LongBench [2] and multi-agent benchmarks show that CASK maintains over 95% of baseline accuracy while allowing more agents or extended histories under tight GPU budgets. Integration into a vision-language agent for collaborative, multimodal contexts underscores its practicality for resource-constrained LLM deployments in MAS.

PDF

IROS Conference 2025 Conference Paper

Embodied Instruction Following in Unknown Environments

Zhenyu Wu
Ziwei Wang 0001
Xiuwei Xu
Hang Yin
Yinan Liang
Angyuan Ma
Jiwen Lu
Haibin Yan

Enabling embodied agents to complete complex human instructions from natural language is crucial to autonomous systems in household services. Conventional methods can only accomplish human instructions in the known environment where all interactive objects are provided to the embodied agent, and directly deploying the existing approaches for the unknown environment usually generates infeasible plans that manipulate non-existing objects. On the contrary, we propose an embodied instruction following (EIF) method for complex tasks in the unknown environment, where the agent efficiently explores the unknown environment to generate feasible plans with existing objects to accomplish abstract instructions. Specifically, we build a hierarchical embodied instruction following framework including the high-level task planner and the low-level exploration controller with multimodal large language models. We then construct a semantic representation map of the scene with dynamic region attention to demonstrate the known visual clues, where the goal of task planning and scene exploration is aligned for human instruction. For the task planner, we generate the feasible step-by-step plans for human goal accomplishment according to the task completion process and the known visual clues. For the exploration controller, the optimal navigation or object interaction policy is predicted based on the generated step-wise plans and the known visual clues. The experimental results demonstrate that our method can achieve 45. 09% success rate in 204 complex human instructions such as making breakfast and tidying rooms in large house-level scenes. Code and supplementary are available at https://gary3410.github.io/eif_unknown/.

Details

IROS Conference 2025 Conference Paper

Real-time Iteration Scheme for Diffusion Policy

Yufei Duan
Hang Yin
Danica Kragic

Diffusion Policies have demonstrated impressive performance in robotic manipulation tasks. However, their long inference time, resulting from an extensive iterative denoising process, and the need to execute an action chunk before the next prediction to maintain consistent actions limit their applicability to latency-critical tasks or simple tasks with a short cycle time. While recent methods explored distillation or alternative policy structures to accelerate inference, these often demand additional training, which can be resource-intensive for large robotic models. In this paper, we introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme, a method from optimal control that accelerates optimization by leveraging solutions from previous time steps as initial guesses for subsequent iterations. We explore the application of this scheme in diffusion inference and propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation. The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign. This enables a seamless integration into many pre-trained diffusion-based models, in particular, to resource-demanding large models. We also provide theoretical conditions for the contractivity which could be useful for estimating the initial denoising step. Quantitative results from extensive simulation experiments show a substantial reduction in inference time, with comparable overall performance compared with Diffusion Policy using full-step denoising. Our project page with additional resources is available at: https://rti-dp.github.io/

Details

NeurIPS Conference 2024 Conference Paper

Lambda: Learning Matchable Prior For Entity Alignment with Unlabeled Dangling Cases

Hang Yin
Liyao Xiang
Dong Ding
Yuheng He
Yihan Wu
Pengzhi Chu
Xinbing Wang
Chenghu Zhou

We investigate the entity alignment (EA) problem with unlabeled dangling cases, meaning that partial entities have no counterparts in the other knowledge graph (KG), yet these entities are unlabeled. The problem arises when the source and target graphs are of different scales, and it is much cheaper to label the matchable pairs than the dangling entities. To address this challenge, we propose the framework \textit{Lambda} for dangling detection and entity alignment. Lambda features a GNN-based encoder called KEESA with a spectral contrastive learning loss for EA and a positive-unlabeled learning algorithm called iPULE for dangling detection. Our dangling detection module offers theoretical guarantees of unbiasedness, uniform deviation bounds, and convergence. Experimental results demonstrate that each component contributes to overall performances that are superior to baselines, even when baselines additionally exploit 30\% of dangling entities labeled for training.

PDF Details DOI

TMLR Journal 2024 Journal Article

Reducing Variance in Meta-Learning via Laplace Approximation for Regression Tasks

Alfredo Reichlin
Gustaf Tegnér
Miguel Vasco
Hang Yin
Mårten Björkman
Danica Kragic

Given a finite set of sample points, meta-learning algorithms aim to learn an optimal adaptation strategy for new, unseen tasks. Often, this data can be ambiguous as it might belong to different tasks concurrently. This is particularly the case in meta-regression tasks. In such cases, the estimated adaptation strategy is subject to high variance due to the limited amount of support data for each task, which often leads to sub-optimal generalization performance. In this work, we address the problem of variance reduction in gradient-based meta-learning and formalize the class of problems prone to this, a condition we refer to as \emph{task overlap}. Specifically, we propose a novel approach that reduces the variance of the gradient estimate by weighing each support point individually by the variance of its posterior over the parameters. To estimate the posterior, we utilize the Laplace approximation, which allows us to express the variance in terms of the curvature of the loss landscape of our meta-learner. Experimental results demonstrate the effectiveness of the proposed method and highlight the importance of variance reduction in meta-learning.

PDF Details

AAAI Conference 2024 Conference Paper

Scalable Motion Style Transfer with Constrained Diffusion Generation

Wenjie Yin
Yi Yu
Hang Yin
Danica Kragic
Mårten Björkman

Current training of motion style transfer systems relies on consistency losses across style domains to preserve contents, hindering its scalable application to a large number of domains and private data. Recent image transfer works show the potential of independent training on each domain by leveraging implicit bridging between diffusion models, with the content preservation, however, limited to simple data patterns. We address this by imposing biased sampling in backward diffusion while maintaining the domain independence in the training stage. We construct the bias from the source domain keyframes and apply them as the gradient of content constraints, yielding a framework with keyframe manifold constraint gradients (KMCGs). Our validation demonstrates the success of training separate models to transfer between as many as ten dance motion styles. Comprehensive experiments find a significant improvement in preserving motion contents in comparison to baseline and ablative diffusion-based style transfer models. In addition, we perform a human study for a subjective assessment of the quality of generated dance motions. The results validate the competitiveness of KMCGs.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation

Hang Yin
Xiuwei Xu
Zhenyu Wu
Jie Zhou
Jiwen Lu

In this paper, we propose a new framework for zero-shot object navigation. Existing zero-shot object navigation methods prompt LLM with the text of spatially closed objects, which lacks enough scene context for in-depth reasoning. To better preserve the information of environment and fully exploit the reasoning ability of LLM, we propose to represent the observed scene with 3D scene graph. The scene graph encodes the relationships between objects, groups and rooms with a LLM-friendly structure, for which we design a hierarchical chain-of-thought prompt to help LLM reason the goal location according to scene context by traversing the nodes and edges. Moreover, benefit from the scene graph representation, we further design a re-perception mechanism to empower the object navigation framework with the ability to correct perception error. We conduct extensive experiments on MP3D, HM3D and RoboTHOR environments, where SG-Nav surpasses previous state-of-the-art zero-shot methods by more than \textbf{10\%} SR on all benchmarks, while the decision process is explainable. To the best of our knowledge, SG-Nav is the first zero-shot method that achieves even higher performance than supervised object navigation methods on the challenging MP3D benchmark. Code of this project will be released in the final version.

PDF Details DOI

AAMAS Conference 2022 Conference Paper

How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents

Miguel Vasco
Hang Yin
Francisco S. Melo
Ana Paiva

This work addresses the problem of sensing the world: how to learn a multimodal representation of a reinforcement learning agent’s environment that allows the execution of tasks under incomplete perceptual conditions. To address such problem, we argue for hierarchy in the design of representation models and contribute with a novel multimodal representation model, MUSE. The proposed model learns a hierarchy of representations: low-level modalityspecific representations, encoded from raw observation data, and a high-level multimodal representation, encoding joint-modality information to allow robust state estimation. We employ MUSE as the perceptual model of deep reinforcement learning agents provided with multimodal observations in Atari games. We perform a comparative study over different designs of reinforcement learning agents, showing that MUSE allows agents to perform tasks under incomplete perceptual experience with minimal performance loss. Finally, we also evaluate the generative performance of MUSE in literature-standard multimodal scenarios with higher number and more complex modalities, showing that it outperforms state-of-the-art multimodal variational autoencoders in single and cross-modality generation.

PDF

AAAI Conference 2022 Conference Paper

On the Impact of Spurious Correlation for Out-of-Distribution Detection

Yifei Ming
Hang Yin
Yixuan Li

Modern neural networks can assign high confidence to inputs drawn from outside the training distribution, posing threats to models in real-world deployments. While much research attention has been placed on designing new out-of-distribution (OOD) detection methods, the precise definition of OOD is often left in vagueness and falls short of the desired notion of OOD in reality. In this paper, we present a new formalization and model the data shifts by taking into account both the invariant and environmental (spurious) features. Under such formalization, we systematically investigate how spurious correlation in the training set impacts OOD detection. Our results suggest that the detection performance is severely worsened when the correlation between spurious features and labels is increased in the training set. We further show insights on detection methods that are more effective in reducing the impact of spurious correlation, and provide theoretical analysis on why reliance on environmental features leads to high OOD detection error. Our work aims to facilitate better understandings of OOD samples and their formalization, as well as the exploration of methods that enhance OOD detection. Code is available at https: //github. com/deeplearning-wisc/Spurious_OOD.

PDF Details

NeurIPS Conference 2021 Conference Paper

Benchmarking the Combinatorial Generalizability of Complex Query Answering on Knowledge Graphs

Zihao Wang
Hang Yin
Yangqiu Song

Complex Query Answering (CQA) is an important reasoning task on knowledge graphs. Current CQA learning models have been shown to be able to generalize from atomic operators to more complex formulas, which can be regarded as the combinatorial generalizability. In this paper, we present EFO-1-QA, a new dataset to benchmark the combinatorial generalizability of CQA models by including 301 different queries types, which is 20 times larger than existing datasets. Besides, our benchmark, for the first time, provide a benchmark to evaluate and analyze the impact of different operators and normal forms by using (a) 7 choices of the operator systems and (b) 9 forms of complex queries. Specifically, we provide the detailed study of the combinatorial generalizability of two commonly used operators, i. e. , projection and intersection, and justify the impact of the forms of queries given the canonical choice of operators. Our code and data can provide an effective pipeline to benchmark CQA models.

PDF Details

NeurIPS Conference 2021 Conference Paper

Dynamic Environments with Deformable Objects

Rika Antonova
peiyang shi
Hang Yin
Zehang Weng
Danica Kragic

We propose a set of environments with dynamic tasks that involve highly deformable topologically non-trivial objects. These environments facilitate easy experimentation: offer fast runtime, support large-scale parallel data generation, are easy to connect to reinforcement learning frameworks with OpenAI Gym API. We offer several types of benchmark tasks with varying levels of complexity, provide variants with procedurally generated cloth objects and randomized material textures. Moreover, we allow users to customize the tasks: import custom objects and textures, adjust size and material properties of deformable objects. We prioritize dynamic aspects of the tasks: forgoing 2D tabletop manipulation in favor of 3D tasks, with gravity and inertia playing a non-negligible role. Such advanced challenges require insights from multiple fields: machine learning and computer vision to process high-dimensional inputs, methods from computer graphics and topology to inspire structured and interpretable representations, insights from robotics to learn advanced control policies. We aim to help researches from these fields contribute their insights and simplify establishing interdisciplinary collaborations.

PDF Details

AAMAS Conference 2017 Conference Paper

Affect of Robot's Competencies on Children's Perception

Shruti Chandra
Raul Paradeda
Hang Yin
Pierre Dillenbourg
Rui Prada
Ana Paiva

The focus of the research described in this paper is to explore children’s perception of a social robot’s learning abilities and behavior in an educational context. With this purpose, we conducted a long-term study with children in a school by adopting the learning-by-teaching learning method. The scenario involves a ”learner-agent” (a robot) which seeks help from a child (a teacher) in correcting the shapes of a few letters it writes. Two versions of the robot were built: one where it learns and another where it does not improve over time. The results of the study suggest that children’s social relationship with the robot was not affected by the learning abilities of the agent.

PDF

AAAI Conference 2017 Conference Paper

Associate Latent Encodings in Learning from Demonstrations

Hang Yin
Francisco Melo
Aude Billard
Ana Paiva

We contribute a learning from demonstration approach for robots to acquire skills from multi-modal high-dimensional data. Both latent representations and associations of different modalities are proposed to be jointly learned through an adapted variational auto-encoder. The implementation and results are demonstrated in a robotic handwriting scenario, where the visual sensory input and the arm joint writing motion are learned and coupled. We show the latent representations successfully construct a task manifold for the observed sensor modalities. Moreover, the learned associations can be exploited to directly synthesize arm joint handwriting motion from an image input in an end-to-end manner. The advantages of learning associative latent encodings are further highlighted with the examples of inferring upon incomplete input images. A comparison with alternative methods demonstrates the superiority of the present approach in these challenging tasks.

PDF Details

IJCAI Conference 2016 Conference Paper

Synthesizing Robotic Handwriting Motion by Learning from Human Demonstrations

Hang Yin
Patr
iacute; cia Alves-Oliveira
Francisco S. Melo
Aude Billard
Ana Paiva

This paper contributes a novel framework that enables a robotic agent to efficiently learn and synthesize believable handwriting motion. We situate the framework as a foundation with the goal of allowing children to observe, correct and engage with the robot to learn themselves the handwriting skill. The framework adapts the principle behind ensemble methods - where improved performance is obtained by combining the output of multiple simple algorithms - in an inverse optimal control problem. This integration addresses the challenges of rapid extraction and representation of multiple-mode motion trajectories, with the cost forms which are transferable and interpretable in the development of the robot compliance control. It also introduces the incorporation of a human movement inspired feature, which provides intuitive motion modulation to generalize the synthesis with poor robotic written samples for children to identify and correct. We present the results on the success of synthesizing a variety of natural-looking motion samples based upon the learned cost functions. The framework is validated by a user study, where the synthesized dynamical motion is shown to be hard to distinguish from the real human handwriting.

PDF Details