Author name cluster

He Kong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

10 papers

2 author rows

AAAI Conference 2026 Conference Paper

Intention-Aware Diffusion Model for Pedestrian Trajectory Prediction

Yu Liu
Zhijie Liu
Xiao Ren
Youfu Li
He Kong

Predicting pedestrian motion trajectories is critical for the path planning and motion control of autonomous vehicles. Recent diffusion-based models have shown promising results in capturing the inherent stochasticity of pedestrian behavior for trajectory prediction. However, the absence of explicit semantic modelling of pedestrian intent in many diffusion-based methods may result in misinterpreted behaviors and reduced prediction accuracy. To address the above challenges, we propose a diffusion-based pedestrian trajectory prediction framework that incorporates both short-term and long-term motion intentions. Short-term intent is modelled using a residual polar representation, which decouples direction and magnitude to capture fine-grained local motion patterns. Long-term intent is estimated through a learnable, token-based endpoint predictor that generates multiple candidate goals with associated probabilities, enabling multimodal and context-aware intention modelling. Furthermore, we enhance the diffusion process by incorporating adaptive guidance and a residual noise predictor that dynamically refines denoising accuracy. The proposed framework is evaluated on the widely used ETH, UCY, NBA, and SDD benchmarks, demonstrating competitive results against state-of-the-art methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Orion: Steering Personalized Web Agents via Global-Micro Profiling and Adaptive Intent Tracking

Die Hu
Jingguo Ge
Weitao Tang
He Kong
Liangxiong Li
Bingzhen Wu

Recently, Large Language Models (LLMs) based Web Agents have shown significant potential in web understanding and interaction tasks. However, their personalization ability and user experience remain limited by the ambiguity and dynamic nature of user intent, struggling to model diverse user interests and track intent changes over time. To address these challenges, this paper proposes Orion, a novel personalized Web Agent. Orion adopts a global-micro profiling mechanism to balance users' long-term stable preferences and scenario-based needs, and introduces context-aware interest retrieval to enhance personalization. Additionally, we design adaptive profile tracking and proactive disambiguation mechanisms to effectively address the continuous evolution of user intent in multi-turn interactions. Orion is optimized through end-to-end online reinforcement learning, improving personalized reasoning and decision-making ability in real interactive scenarios. Experiments demonstrate that Orion significantly outperforms state-of-the-art baselines in personalized understanding and task efficiency.

PDF Details DOI

IROS Conference 2025 Conference Paper

SAGENet: Binaural Echo-Based 3D Depth Estimation with Sparse Angular Queries and Refined Geometric Cues

Guangyao Liu
Weimeng Cui
Yuzhang Xi
Liu Yang 0021
Peixuan Hu
He Kong
Zhi Wang 0003

In this paper, we propose SAGENet that utilizes only binaural echoes (i. e. , for scenarios when vision perception seriously degrades) for scene depth estimation. Unlike previous methods that implicitly learn spatial features from echoes, which may cause shape and scale drift, SAGENet explicitly extracts spatial cues, effectively enhancing depth estimation accuracy. First, we leverage signal processing to generate coarse 2D geometric cues, which contain scene scale and shape information, as additional input for the 3D depth estimation network. This approach aids the network in better reconstructing depth information from the scene. Given the substantial noise in the 2D geometric cues, we design a geometric cue consistency denoising loss function to help the network accurately interpret the scale and shape information embedded in the features. Second, we initialize learnable queries with angular spectrum peaks and fuse them with audio features via self-attention to guide the network to focus on the first few reflections echo dominant feature, while effectively suppressing interference from reverberation. Finally, Our experimental results on the Replica and real-world BatVision datasets show that the proposed method outperforms the existing binaural echo-based methods (including BatVision) by more than 5% and 10% in absolute relative error, respectively. To benefit the community, we open-source the code at https://github.com/zjuersdsd/SAGENet.git.

Details

IROS Conference 2024 Conference Paper

Asynchronous Microphone Array Calibration using Hybrid TDOA Information

Chengjie Zhang
Jiang Wang
He Kong

Asynchronous microphone array calibration is a prerequisite for many audition robot applications. A popular solution to the above calibration problem is the batch form of Simultaneous Localisation and Mapping (SLAM), using the time difference of arrival measurements between two microphones (TDOA-M), and the robot (which serves as a moving sound source during calibration) odometry information. In this paper, we introduce a new form of measurement for microphone array calibration, i. e. the time difference of arrival between adjacent sound events (TDOA-S) with respect to the microphone channels. We propose to use TDOA-S and TDOA-M, called hybrid TDOA, together with odometry measurements for bath SLAM-based calibration of asynchronous microphone arrays. Extensive simulation and real-world experiments show that our method is more independent of microphone number, less sensitive to initial values (when using off-the-shelf algorithms such as Gauss-Newton iterations), and has better calibration accuracy and robustness under various TDOA noises. Simulation results also demonstrate that our method has a lower Cramér-Rao lower bound (CRLB) for microphone parameters. To benefit the community, we open-source our code and data at https://github.com/AISLAB-sustech/Hybrid-TDOA-Calib.

Details

IROS Conference 2024 Conference Paper

I-ASM: Iterative Acoustic Scene Mapping for Enhanced Robot Auditory Perception in Complex Indoor Environments

Linya Fu
Yuanzheng He
Jiang Wang
Xu Qiao
He Kong

This paper addresses the challenge of acoustic scene mapping (ASM) in complex indoor environments with multiple sound sources. Unlike existing methods that rely on prior data association or SLAM frameworks, we propose a novel particle filter-based iterative framework, termed I-ASM, for ASM using a mobile robot equipped with a microphone array and LiDAR. I-ASM harnesses an innovative "implicit association" to align sound sources with Direction of Arrival (DoA) observations without requiring explicit pairing, thereby streamlining the mapping process. Given inputs including an occupancy map, DoA estimates from various robot positions, and corresponding robot pose data, I-ASM performs multi-source mapping through an iterative cycle of "Filtering-Clustering-Implicit Associating". The proposed framework has been tested in real-world scenarios with up to 10 concurrent sound sources, demonstrating its robustness against missing and false DoA estimates while achieving high-quality ASM results. To benefit the community, we open-source all the codes and data at https://github.com/AISLAB-sustech/Acoustic-Scene-Mapping

Details

IJCAI Conference 2024 Conference Paper

ScreenAgent: A Vision Language Model-driven Computer Control Agent

Runliang Niu
Jindong Li
Shiqi Wang
Yali Fu
Xiyu Hu
Xueyuan Leng
He Kong
Yi Chang

Large Language Models (LLM) can invoke a variety of tools and APIs to complete complex tasks. The computer, as the most powerful and universal tool, could potentially be controlled by a trained LLM agent. Powered by the computer, we can hopefully build a more generalized agent to assist humans in various daily digital works. In this paper, we construct an environment for a Vision Language Model (VLM) agent to interact with a real computer screen. Within this environment, the agent can observe screenshots and manipulate the Graphical User Interface (GUI) by outputting mouse and keyboard actions. We also design an automated control pipeline that includes planning, acting, and reflecting phases, guiding the agent to continuously interact with the environment and complete multi-step tasks. Additionally, we construct the ScreenAgent Dataset, which collects screenshots and action sequences when completing daily computer tasks. Finally, we train a model, ScreenAgent, which achieves comparable computer control capabilities to GPT-4V and demonstrated more precise UI positioning capabilities. Our attempts could inspire further research on building a generalist LLM agent. The code and more detailed information are at https: //github. com/niuzaisheng/ScreenAgent.

PDF Details DOI

ICRA Conference 2021 Conference Paper

Active Information Acquisition under Arbitrary Unknown Disturbances

Jennifer Wakulicz
He Kong
Salah Sukkarieh

Trajectory optimization of sensing robots to actively gather information of targets has received much attention in the past. It is well-known that under the assumption of linear Gaussian target dynamics and sensor models the stochastic Active Information Acquisition problem is equivalent to a deterministic optimal control problem. However, the above-mentioned assumptions regarding the target dynamic model are limiting. In real-world scenarios, the target may be subject to disturbances whose models or statistical properties are hard or impossible to obtain. Typical scenarios include abrupt maneuvers, jumping disturbances due to interactions with the environment, anomalous misbehaviors due to system faults/attacks, etc. Motivated by the above considerations, in this paper we consider targets whose dynamic models are subject to arbitrary unknown inputs whose models or statistical properties are not assumed to be available. In particular, with the aid of an unknown input decoupled filter, we formulate the sensor trajectory planning problem to track evolution of the target state and analyse the resulting performance for both the state and unknown input evolution tracking. Inspired by concepts of Reduced Value Iteration, a suboptimal solution that expands a search tree via Forward Value Iteration with informativeness-based pruning is proposed. Concrete suboptimality performance guarantees for tracking both the state and the unknown input are established. Numerical simulations of a target tracking example are presented to compare the proposed solution with a greedy policy.

Details

ICRA Conference 2020 Conference Paper

Path Planning in Dynamic Environments using Generative RNNs and Monte Carlo Tree Search

Stuart Eiffert
He Kong
Navid Pirmarzdashti
Salah Sukkarieh

State of the art methods for robotic path planning in dynamic environments, such as crowds or traffic, rely on hand crafted motion models for agents. These models often do not reflect interactions of agents in real world scenarios. To overcome this limitation, this paper proposes an integrated path planning framework using generative Recurrent Neural Networks within a Monte Carlo Tree Search (MCTS). This approach uses a learnt model of social response to predict crowd dynamics during planning across the action space. This extends our recent work using generative RNNs to learn the relationship between planned robotic actions and the likely response of a crowd. We show that the proposed framework can considerably improve motion prediction accuracy during interactions, allowing more effective path planning. The performance of our method is compared in simulation with existing methods for collision avoidance in a crowd of pedestrians, demonstrating the ability to control future states of nearby individuals. We also conduct preliminary real world tests to validate the effectiveness of our method.

Details

IROS Conference 2019 Conference Paper

Modelling of Uniaxial EGaIn-Based Strain Sensors for Proprioceptive Sensing of Soft Robots

Abdullah Al-Azzawi
A. Mounir Boudali
He Kong
Ali Haydar Göktogan
Salah Sukkarieh

Soft strain resistive sensors based on eutectic gallium-indium liquid metal can play an important role in proprioceptive sensing for soft robots. However, there are no available mathematical models to accurately estimate the strain as a function of the measured resistance. Furthermore, non-uniform strain in the microchannels has not been analysed yet. In this paper, we introduce a new model to estimate the strain or elongation in sub-millimetre scale, and analyse its accuracy through a customised testing set-up and procedure. The effect of strain rate on the measurement accuracy is also studied. We compare existing theoretical models with our experimental results, and discuss the differences between them. Moreover, we analyse the effect of strain rate on hysteresis caused by the viscoelastic behaviour and introduce a new model for it to be potentially used for future work. This paper demonstrates, among other things, that rational models could provide high accuracy in strain estimation, and might help to enhance proprioceptive sensing and state control of soft robots.

Details

ICRA Conference 2019 Conference Paper

Receding horizon estimation and control with structured noise blocking for mobile robot slip compensation

Nathan D. Wallace
He Kong
Andrew John Hill
Salah Sukkarieh

The control of field robots in varying and uncertain terrain conditions presents a challenge for autonomous navigation. Online estimation of the wheel-terrain slip characteristics is essential for generating the accurate control predictions necessary for tracking trajectories in off-road environments. Receding horizon estimation (RHE) provides a powerful framework for constrained estimation, and when combined with receding horizon control (RHC), yields an adaptive optimisation-based control method. Presently, such methods assume slip to be constant over the estimation horizon, while our proposed structured blocking approach relaxes this assumption, resulting in improved state and parameter estimation. We demonstrate and compare the performance of this method in simulation, and propose an overlapping-block strategy to ameliorate some of the limitations encountered in applying noise-blocking in a receding horizon estimation and control (RHEC) context.

Details