Author name cluster

Daeun Song

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

10 papers

1 author row

IROS Conference 2025 Conference Paper

AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learning

Yangzhe Kong
Daeun Song
Jing Liang 0006
Dinesh Manocha
Ziyu Yao 0002
Xuesu Xiao

We present a novel method, AutoSpatial, an efficient approach with structured spatial grounding to enhance VLMs’ spatial reasoning. By combining minimal manual supervision with large-scale Visual Question-Answering (VQA) pairs auto-labeling, our approach tackles the challenge of VLMs’ limited spatial understanding in social navigation tasks. By applying a hierarchical two-round VQA strategy during training, AutoSpatial achieves both global and detailed understanding of scenarios, demonstrating more accurate spatial perception, movement prediction, Chain of Thought (CoT) reasoning, final action, and explanation compared to other SOTA approaches. These five components are essential for comprehensive social navigation reasoning. Our approach was evaluated using both expert systems (GPT-4o, Gemini 2. 0 Flash, and Claude 3. 5 Sonnet) that provided cross-validation scores and human evaluators who assigned relative rankings to compare model performances across four key aspects. Augmented by the enhanced spatial reasoning capabilities, AutoSpatial demonstrates substantial improvements by averaged cross-validation score from expert systems in: perception & prediction (up to 10. 71%), reasoning (up to 16. 26%), action (up to 20. 50%), and explanation (up to 18. 73%) compared to baseline models trained only on manually annotated data.

Details

ICRA Conference 2025 Conference Paper

Gnd: Global Navigation Dataset With Multi-Modal Perception and Multi-Category Traversability in Outdoor Campus Environments

Jing Liang 0006
Dibyendu Das
Daeun Song
Md Nahid Hasan Shuvo
Mohammad Durrani
Karthik Taranath
Ivan Penskiy
Dinesh Manocha

Navigating large-scale outdoor environments requires complex reasoning in terms of geometric structures, environmental semantics, and terrain characteristics, which are typically captured by onboard sensors such as LiDAR and cameras. While current mobile robots can navigate such environments using pre-defined, high-precision maps based on hand-crafted rules catered for the specific environment, they lack commonsense reasoning capabilities, especially the traversability analysis, that most humans possess when navigating unknown outdoor spaces. To address this gap, we introduce the Global Navigation Dataset (GND), a large-scale dataset that integrates multi-modal sensory data, including 3D LiDAR point clouds and RGB and 360° images, as well as multi-category traversability maps (pedestrian walkways, vehicle roadways, stairs, off-road terrain, and obstacles) from ten university campuses. These environments encompass a variety of parks, urban settings, elevation changes, and campus layouts of different scales. The dataset covers approximately $2. 7 ~\text{km}^{2}$ and includes at least 350 buildings in total. We also present a set of novel applications of GND to showcase its utility to enable global robot navigation, such as map-based global navigation, mapless navigation, and global place recognition. GND's website can be found at https://cs.gmu.edu/xiao/Research/GND/.

Details

IROS Conference 2025 Conference Paper

Social-LLaVA: Enhancing Social Robot Navigation through Human-Language Reasoning

Amirreza Payandeh
Daeun Song
Mohammad Nazeri
Jing Liang 0006
Praneel Mukherjee
Amir Hossain Raj
Yangzhe Kong
Dinesh Manocha

As mobile robots become increasingly common in human-centric environments, social navigation—adhering to unwritten social norms rather than merely avoiding pedestrians—has drawn growing attention. Existing methods, from hand-crafted techniques to learning-based approaches, often overlook the nuanced context and scene understanding that humans naturally exhibit. Inspired by studies indicating the critical role of language in cognition and reasoning, we propose a new approach to bridge robot perception and socially aware actions through human-like language reasoning. We introduce Social robot Navigation via Explainable Interactions (SNEI), a human-annotated vision-language dataset comprising over 40K Visual Question Answering (VQA) pairs across 2K unique social scenarios, drawn from diverse, unstructured public spaces. SNEI contains perception, prediction, chain-of-thought reasoning, action, and explanation, thereby allowing robots to interpret social contexts in human language. We fine-tune a Vision-Language Model, Social-LLaVA, on SNEI to demonstrate the potential of language-guided reasoning for high-level navigation tasks. Experimental evaluations—both quantitative and qualitative—demonstrate that Social-LLaVA can outperform state-of-the-art models. †.

Details

IROS Conference 2024 Conference Paper

AGL-Net: Aerial-Ground Cross-Modal Global Localization with Varying Scales

Tianrui Guan
Ruiqi Xian
Xijun Wang 0002
Xiyang Wu
Mohamed Elnoor
Daeun Song
Dinesh Manocha

We present AGL-NET, a novel learning-based method for global localization using LiDAR point clouds and satellite maps. AGL-Net tackles two critical challenges: bridging the representation gap between image and points modalities for robust feature matching, and handling inherent scale discrepancies between global view and local view. To address these challenges, AGL-Net leverages a unified network architecture with a novel two-stage matching design. The first stage extracts informative neural features directly from raw sensor data and performs initial feature matching. The second stage refines this matching process by extracting informative skeleton features and incorporating a novel scale alignment step to rectify scale variations between LiDAR and map data. Furthermore, a novel scale and skeleton loss function guides the network toward learning scale-invariant feature representations, eliminating the need for pre-processing satellite maps. This significantly improves real-world applicability in scenarios with unknown map scales. To facilitate rigorous performance evaluation, we introduce a meticulously designed dataset within the CARLA simulator specifically tailored for metric localization training and assessment.

Details

IROS Conference 2024 Conference Paper

DTG: Diffusion-based Trajectory Generation for Mapless Global Navigation

Jing Liang 0006
Amirreza Payandeh
Daeun Song
Xuesu Xiao
Dinesh Manocha

We present a novel end-to-end diffusion-based trajectory generation method, DTG, for mapless global navigation in challenging outdoor scenarios with occlusions and unstructured off-road features like grass, buildings, bushes, etc. Given a distant goal, our approach computes a trajectory that satisfies the following goals: (1) minimize the travel distance to the goal; (2) maximize the traversability by choosing paths that do not lie in undesirable areas. Specifically, we present a novel Conditional RNN(CRNN) for diffusion models to efficiently generate trajectories. Furthermore, we propose an adaptive training method that ensures that the diffusion model generates more traversable trajectories. We evaluate our methods in various outdoor scenes and compare the performance with other global navigation algorithms on a Husky robot. In practice, we observe at least a 15% improvement in traveling distance and around a 7% improvement in traversability. Video and Code: https://github.com/jingGM/DTG.git.

Details

IROS Conference 2023 Conference Paper

Stroke-Based Rendering and Planning for Robotic Performance of Artistic Drawing

Ivaylo Ilinkin
Daeun Song
Young J. Kim

We present a new robotic drawing system based on stroke-based rendering (SBR). Our motivation is the artistic quality of the whole performance. Not only should the generated strokes in the final drawing resemble the input image, but the stroke sequence should also exhibit a human artist's planning process. Thus, when a robot executes the drawing task, both the drawing results and the way the robot executes would look artistic. Our SBR system is based on image segmentation and depth estimation. It generates the drawing strokes in an order that allows for the intended shape to be perceived quickly and for its detailed features to be filled in and emerge gradually when observed by the human. This ordering represents a stroke plan that the drawing robot should follow to create an artistic rendering of images. We experimentally demonstrate that our SBR-based drawing makes visually pleasing artistic images, and our robotic system can replicate the result with proper sequences of stroke drawing.

Details

ICRA Conference 2021 Conference Paper

Learning to steer a locomotion contact planner

Jason Chemin
Pierre Fernbach
Daeun Song
Guilhem Saurel
Nicolas Mansard
Steve Tonneau

The combinatorics inherent to the issue of planning legged locomotion can be addressed by decomposing the problem: first, select a guide path abstracting the contacts with a heuristic model; then compute the contact sequence to balance the robot gait along the guide path. While several models have been proposed to compute such a path, none have yet managed to efficiently capture the complexity of legged locomotion on arbitrary terrain. In this paper, we present a novel method to automatically build a local controller, or steering method, to generate a guide path along which a feasible contact sequence can be built. Our reinforcement learning approach is coupled with a geometric condition for feasibility during the training, which improves the convergence rate without incurring a loss in generality. We have designed a dedicated environment along with an associated reward function to run a classical reinforcement learning algorithm that computes the steering method. The policy takes a target velocity and a local heightmap of the terrain around the robot as inputs, and steers the path where new contacts should be created. It is then coupled with a contact generator that creates the contacts to support the robot movement. We show that the trained policy has an improved generalization and higher success rate at generating feasible contact plans than previous approaches. As a result, this policy can be used with a path planning algorithm to navigate complex environments.

Details

ICRA Conference 2020 Conference Paper

SL1M: Sparse L1-norm Minimization for contact planning on uneven terrain

Steve Tonneau
Daeun Song
Pierre Fernbach
Nicolas Mansard
Michel Taïx
Andrea Del Prete

One of the main challenges of planning legged locomotion in complex environments is the combinatorial contact selection problem. Recent contributions propose to use integer variables to represent which contact surface is selected, and then to rely on modern mixed-integer (MI) optimization solvers to handle this combinatorial issue. To reduce the computational cost of MI, we exploit the sparsity properties of L1 norm minimization techniques to relax the contact planning problem into a feasibility linear program. Our approach accounts for kinematic reachability of the center of mass (COM) and of the contact effectors. We ensure the existence of a quasi-static COM trajectory by restricting our plan to quasi-flat contacts. For planning 10 steps with less than 10 potential contact surfaces for each phase, our approach is 50 to 100 times faster that its MI counterpart, which suggests potential applications for online contact re-planning. The method is demonstrated in simulation with the humanoid robots HRP-2 and Talos over various scenarios.

Details

ICRA Conference 2019 Conference Paper

Distortion-free Robotic Surface-drawing using Conformal Mapping

Daeun Song
Young J. Kim

We present a robotic pen-drawing system that is capable of faithfully reproducing pen art on an unknown surface. Our robotic system relies on an industrial, seven-degree-of freedom manipulator that can be both position- and impedance-controlled. In order to estimate a rough geometry of the target, continuous surface, we first generate a point cloud of the surface using an RGB-D camera, which is filtered to remove outliers and calibrated to the physical canvas surface. Then, our control algorithm physically reproduces digital drawing on the surface by impedance-controlling the manipulator. Our impedance-controlled drawing algorithm compensates for the uncertainty and incompleteness inherent to a point-cloud estimation of the drawing surface. Moreover, since drawing 2D vector pen art on a 3D surface requires surface parameterization that does not destroy the original 2D drawing, we rely on the least squares conformal mapping. Specifically, the conformal map reduces angle distortion during surface parameterization. As a result, our system can create distortion-free and complicated pen drawings on general surfaces with many unpredictable bumps robustly and faithfully.

Details

ICRA Conference 2018 Conference Paper

Artistic Pen Drawing on an Arbitrary Surface Using an Impedance-Controlled Robot

Daeun Song
Taekhee Lee
Young J. Kim

We present a semi-autonomous robotic pen-drawing system that is capable of creating pen art on an arbitrary surface with varying thickness of pen strokes but without reconstructing the surface explicitly. Our robotic system relies on an industrial, seven-degree-of-freedom (7DoF) manipulator that can be both position- and impedance-controlled. We use a vector-graphics engine to take an artist's pen drawing as input and generate Bézier spline curves with varying offsets. In order to estimate geometric details of the target, unknown surface, during drawing, we rely on incremental and adaptive sampling on the surface using a combination of position and impedance control. Then, our control algorithm physically replicates this drawing on any arbitrary, continuous surface by impedance-controlling the manipulator. We demonstrate that our system can create visually-pleasing and complicated artistic pen drawings on general surfaces without explicit surface-reconstruction nor visual feedback.

Details