Trey Smith

ICRA Conference 2025 Conference Paper

AstroLoc2: Fast Sequential Depth-Enhanced Localization for Free-Flying Robots

• Ryan Soussan
• Marina Moreira 0001
• Brian Coltin
• Trey Smith

We present AstroLoc2, a monocular and time-offlight (ToF) visual-inertial graph-based localizer used by the Astrobee free-flying robots on the International Space Station (ISS). AstroLoc2 sequentially performs odometry and absolute localization in a single process to decouple map noise from velocity and IMU bias estimation and run efficiently on resource constrained platforms. It improves monocular visual-inertial odometry robustness by adding ToF correspondence factors and uses adaptive map-matching to increase image registration reliability in dynamic environments while preserving fast matching in static ones. We evaluate the performance of AstroLoc2 on a public dataset of 10 ISS activities and show that it improves localization accuracy by 16 % and success rates by 5. 5 % while maintaining a faster runtime than leading methods. AstroLoc2 has enabled the Astrobee robots to perform higher precision maneuvers in changing environments on the ISS. It can be configured for other limited computation platforms and we release the source code to the public.

ICRA Conference 2024 Conference Paper

An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

• Eric T. Chang
• Runsheng Wang
• Peter Ballentine
• Jingxi Xu 0002
• Trey Smith
• Brian Coltin
• Ioannis Kymissis
• Matei Ciocarlie

In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled quickly, are affordable, and occupy a very small footprint. Our prototype sensor uses only a sparse array (8-9 mm spacing) of distributed MEMS microphones (<$1, 3. 76×2. 95×1. 10 mm) embedded under an elastomer. We use transformer-based architectures for data analysis, taking advantage of the microphones’ high sampling rate to run our models on time-series data as opposed to individual snapshots. This approach allows us to obtain 77. 3% average accuracy on 4-class texture classification (84. 2% when excluding the slowest drag velocity), 1. 8 mm mean error on contact localization, and 5. 6 mm/s mean error on contact velocity. We show that the learned texture and localization models are robust to varying velocity and generalize to unseen velocities. We also report that our sensor provides fast contact detection, an important advantage of fast transducers. This investigation illustrates the capabilities one can achieve with a MEMS microphone array alone, leaving valuable sensor real estate available for integration with complementary tactile sensing modalities.

ICRA Conference 2022 Conference Paper

AstroLoc: An Efficient and Robust Localizer for a Free-flying Robot

• Ryan Soussan
• Varsha Kumar
• Brian Coltin
• Trey Smith

We present AstroLoc, an efficient and robust monocular visual-inertial graph-based localization system used by the Astrobee free-flying robots onboard the International Space Station (ISS). We provide a novel localization system that limits the traditionally higher computation times for graph-based localization systems and enables the resource constrained Astrobee robots to benefit from their increased accuracy. We also introduce methods for handling cheirality issues for visual odometry and localization factors that further increase localization robustness. We evaluate the performance of AstroLoc on a dataset of ISS activities and show that it greatly improves pose, velocity, and IMU bias estimation accuracy while efficiently running in a limited computation environment. AstroLoc has improved the localization accuracy for the Astrobee robots on the ISS and has led to more successful and longer duration activities. While the AstroLoc system is tuned for the Astrobee robots, it can be configured for any resource constrained platform. The source code for AstroLoc is released to the public.

ICRA Conference 2022 Conference Paper

Robust Semantic Mapping and Localization on a Free-Flying Robot in Microgravity

• Ian D. Miller
• Ryan Soussan
• Brian Coltin
• Trey Smith
• Vijay Kumar 0001

We propose a system that uses semantic object detections to localize a microgravity free-flyer. Many applications require absolute localization in a known reference frame, such as the execution of waypoint trajectories defined by human operators. Classical geometric methods build a map of point features, which may not be able to be associated after lighting or environmental changes. By contrast, semantics remain invariant to changes up to the robustness of the detection algorithm and motion of the semantic objects. In this work, we describe our approaches for both offline semantic map generation as well as online localization against a semantic map, intended to run in real-time on the robot. We additionally demonstrate how our semantic localizer outperforms image-feature matching in some cases, and show the robustness of the algorithm to environmental changes. Crucially, we show in our experiments that when semantics are used to supplement point features, localization is always improved. To our knowledge, these experiments demonstrate the first use of learned semantics for localization on a free-flying robot in microgravity.

IROS Conference 2021 Conference Paper

A Multi-Axis FBG-Based Tactile Sensor for Gripping in Space

• Samuel Frishman
• Julia Di
• Zulekha Karachiwalla
• Richard J. Black
• Kian Moslehi
• Trey Smith
• Brian Coltin
• Bijan Moslehi

Tactile sensing can improve end-effector control and grasp quality, especially for free-flying robots where target approach and alignment present particular challenges. However, many current tactile sensing technologies are not suitable for the harsh environment of space. We present a tactile sensor that measures normal and biaxial shear strains in the pads of a gripper using a single optical fiber with Bragg grating (FBG) sensors. Compared to conventional wired solutions, the encapsulated optical fibers are immune to electromagnetic interference — critical in the harsh environment of space. Sampling is possible at over 1 kHz to detect dynamic events. We mount sensor pads on a custom two-fingered gripper with independent control of the distal and proximal phalanges, allowing for grip readjustment based on sensing data. Calibrated sensor data for forces match those from a commercial multiaxial load cell with an average 96. 2% RMS for all taxels. We demonstrate the gripper on tasks motivated by the Astrobee free-flying robots in the International Space Station (ISS): gripping corners, detecting misaligned grasps, and improving load sharing over the contact areas in pinch grasps.

IROS Conference 2018 Conference Paper

HTC Vive: Analysis and Accuracy Improvement

• Miguel Borges
• Andrew Symington
• Brian Coltin
• Trey Smith
• Rodrigo M. M. Ventura

HTC Vive has been gaining attention as a cost-effective, off-the-shelf tracking system for collecting ground truth pose data. We assess this system's pose estimation through a series of controlled experiments where we show its precision to be in the millimeter magnitude and accuracy to range from millimeter to meter. We also show that Vive gives greater weight to inertial measurements in order to produce a smooth trajectory for virtual reality applications. Hence, the Vive's off the shelf algorithm is poorly suited for robotics applications such as measuring ground truth poses, where accuracy and repeatability are key. Therefore we introduce a new open-source tracking algorithm and calibration procedure for Vive which address these problems. We also show that our approach improves the pose estimation repeatability and accuracy by up to two orders of magnitude.

IROS Conference 2016 Conference Paper

Smooth trajectory generation on SE(3) for a free flying space robot

• Michael Watterson
• Trey Smith
• Vijay Kumar 0001

We propose a new optimal trajectory generation technique on SE(3) which avoids known obstacles. We leverage techniques from differential geometry and Lie algebra to formulate a cost functional which is intrinsic to the geometric structure of this space and makes physical sense. We propose an approximation technique to generate trajectories on the subgroup SO(3) and use Semidefinite Programming (SDP) to approximate an NP-Hard problem with one which is tractable to compute. From this trajectory on the subgroup, the trajectory generation on the other dimensions of the group becomes a Quadratic Program (QP). For obstacle avoidance, we use a computational geometric technique to decompose the environment into overlapping convex regions to confine the trajectory. We show how this motion planning technique can be used to generate feasible trajectories for a space robot in SE(3) and describe controllers that enable the execution of the generated trajectory. We compare our method to other geometric techniques for calculating trajectories on SO(3) and SE(3), but in an obstacle-free environment.

ICRA Conference 2008 Conference Paper

Information-optimal selective data return for autonomous rover traverse science and survey

• David R. Thompson 0001
• Trey Smith
• David Wettergreen

Selective data return leverages onboard data analysis to allocate limited bandwidth resources during remote exploration. Here we present an adaptive method to subsample image sequences for downlink. We treat selective data return as a compression problem in which the explorer agent transmits the subset of measurements that are most informative with respect to the complete dataset. Experiments demonstrate selective downlink of navigation imagery by a rover during autonomous geologic investigations in the Atacama desert of Chile. Here automatic analysis identifies informative images using classifications based on natural image statistics. Image texture analysis, together with a context-sensitive Hidden Markov Model representation, permits adaptive downlink in response to geologic unit boundaries. Selective data return improves the science content of returned data for this geologic mapping task.

ICAPS Conference 2007 Conference Paper

Generating Exponentially Smaller POMDP Models Using Conditionally Irrelevant Variable Abstraction

• Trey Smith
• David R. Thompson 0001
• David Wettergreen

The state of a POMDP can often be factored into a tuple of n state variables. The corresponding flat model, with size exponential in n, may be intractably large. We present a novel method called conditionally irrelevant variable abstraction (CIVA) for losslessly compressing the factored model, which is then expanded into an exponentially smaller flat model in a representation compatible with many existing POMDP solvers. We applied CIVA to previously intractable problems from a robotic exploration domain. We were able to abstract, expand, and approximately solve POMDPs that had up to 1024 states in the uncompressed flat representation.

AAAI Conference 2006 Conference Paper

Focused Real-Time Dynamic Programming for MDPs: Squeezing More Out of a Heuristic

• Trey Smith

Real-time dynamic programming (RTDP) is a heuristic search algorithm for solving MDPs. We present a modified algorithm called Focused RTDP with several improvements. While RTDP maintains only an upper bound on the long-term reward function, FRTDP maintains two-sided bounds and bases the output policy on the lower bound. FRTDP guides search with a new rule for outcome selection, focusing on parts of the search graph that contribute most to uncertainty about the values of good policies. FRTDP has modified trial termination criteria that should allow it to solve some problems (within ) that RTDP cannot. Experiments show that for all the problems we studied, FRTDP significantly outperforms RTDP and LRTDP, and converges with up to six times fewer backups than the state-of-the-art HDP algorithm.

UAI Conference 2005 Conference Paper

Point-Based POMDP Algorithms: Improved Analysis and Implementation

• Trey Smith
• Reid G. Simmons

AAAI Conference 2005 Short Paper

Rover Science Autonomy: Probabilistic Planning for Science-Aware Exploration

• Trey Smith

Future Mars rovers will have the ability to autonomously navigate for distances of kilometers. In one sol a traverse may take a rover into unexplored areas beyond its local horizon. The rover can explore these areas more effectively if it is able to detect and react to science opportunities on its own, what we call science autonomy. We are studying science autonomy in two ways: first, by implementing a simple science autonomy system on a rover in the field, and second, by developing probabilistic planning technology that can enable more principled autonomous decision-making in future systems.

UAI Conference 2004 Conference Paper

Heuristic Search Value Iteration for POMDPs

• Trey Smith
• Reid G. Simmons

We present a novel POMDP planning algorithm called heuristic search value iteration (HSVI).HSVI is an anytime algorithm that returns a policy and a provable bound on its regret with respect to the optimal policy. HSVI gets its power by combining two well-known techniques: attention-focusing search heuristics and piecewise linear convex representations of the value function. HSVI's soundness and convergence have been proven. On some benchmark problems from the literature, HSVI displays speedups of greater than 100 with respect to other state-of-the-art POMDP value iteration algorithms. We also apply HSVI to a new rover exploration problem 10 times larger than most POMDP problems in the literature.

ICRA Conference 2000 Conference Paper

Recent Progress in Local and Global Traversability for Planetary Rovers

• Sanjiv Singh
• Reid G. Simmons
• Trey Smith
• Anthony Stentz
• Vandi Verma
• Alex Yahja
• Kurt Schwehr

Autonomous planetary rovers operating in vast unknown environments must operate efficiently because of size, power and computing limitations. Recently, we have developed a rover capable of efficient obstacle avoidance and path planning. The rover uses binocular stereo vision to sense potentially cluttered outdoor environments. Navigation is performed by a combination of several modules that each "vote" for the next best action for the robot to execute. The key distinction of our system is that it produces globally intelligent behavior with a small computational resource - all processing and decision making are done on a single processor. These algorithms have been tested on our outdoor prototype rover, Bullwinkle, and have recently driven the rover 100 m at a speed of 15 cm/sec. In this paper we report on the extension on the systems that we have previously developed that were necessary to achieve autonomous navigation in this domain.