Author name cluster

Stephen T. Nuske

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

1 author row

IROS Conference 2016 Conference Paper

Long distance visual ground-based signaling for unmanned aerial vehicles

Volker Grabe
Stephen T. Nuske

We present a long-range visual signal detection system that is suitable for an unmanned aerial vehicle to find an optical signal released at a desired landing site for the purposes of cargo delivery or rescue situations where radio signals or other communication systems are not available or the wind conditions at the landing site need to be signaled. The challenge here is to have a signal and detection system that works from long range (> 1000m) amongst ground clutter during various seasonal conditions on passive imagery. We use a smoke-grenade as a ground signal, which has the advantageous properties of being easy to carry by ground crews because of its light weight and small size, but when released has a long visual signaling range. We employ a camera system on the UAV with a visual texture feature extraction approach in a machine learning framework to classify image patches as `signal' or `background'. We study conventional approaches and develop a visual feature descriptor that can better differentiate the appearance of the visual signal under varying conditions and, when used to train a random-forest classifier, outperforms commonly used feature descriptors. The system was rigorously and quantitatively evaluated on data collected from a camera mounted on a helicopter and flown towards a plume of signal smoke over a variety of seasons, ground conditions, weather conditions, and environments. Our system was capable of detecting the smoke cloud with both precision and recall rates greater than 0. 95 from ranges between 1000m and 1500m. Further, we develop a method to estimate wind orientation and approximate wind strength by assessing the shape of the smoke signal. We present a preliminary evaluation of the wind estimation in conditions with different wind intensities and orientations relative to the approach direction.

Details

ICRA Conference 2016 Conference Paper

Texture-based fruit detection via images using the smooth patterns on the fruit

Zania S. Pothen
Stephen T. Nuske

This paper describes a keypoint detection algorithm to accurately detect round fruits in high resolution imagery. The significant challenge associated with round fruits such as grapes and apples is that the surface is smooth and lacks definition and contrasting features, the contours of the fruit may be partially occluded, and the color of the fruit often blends with background foliage. We propose a fruit detection algorithm that utilizes the gradual variation of intensity and gradient orientation on the surface of the fruit. Candidate fruit locations, or “seed points” are tested for both monotonically decreasing intensity and gradient orientation profiles. Candidate fruit locations that pass the initial filter are classified using modified histogram of oriented gradients combined with a pairwise intensity comparison texture descriptor and random forest classifier. We analyse the performance of the fruit detection algorithm on image datasets of grapes and apples using human labeled images as ground truth. Our method to detect candidate fruit locations is scale invariant, robust to partial occlusions and more accurate than existing methods. We achieve overall F1 accuracy score of 0. 82 for grapes and 0. 80 for apples. We demonstrate our method is more accurate than existing methods.

Details

ICRA Conference 2013 Conference Paper

Infrastructure-free shipdeck tracking for autonomous landing

Sankalp Arora
Sezal Jain
Sebastian A. Scherer
Stephen T. Nuske
Lyle Chamberlain
Sanjiv Singh

Shipdeck landing is one of the most challenging tasks for a rotorcraft. Current autonomous rotorcraft use shipdeck mounted transponders to measure the relative pose of the vehicle to the landing pad. This tracking system is not only expensive but renders an unequipped ship unlandable. We address the challenge of tracking a shipdeck without additional infrastructure on the deck. We present two methods based on video and lidar that are able to track the shipdeck starting at a considerable distance from the ship. This redundant sensor design enables us to have two independent tracking systems. We show the results of the tracking algorithms in three different environments - field testing results on actual helicopter flights, in simulation with a moving shipdeck for lidar based tracking and in laboratory using an occluded, and, moving scaled model of a landing deck for camera based tracking. The complimentary modalities allow shipdeck tracking under varying conditions.

Details

ICRA Conference 2012 Conference Paper

Global pose estimation with limited GPS and long range visual odometry

Joern Rehder
Kamal Gupta
Stephen T. Nuske
Sanjiv Singh

Here we present an approach to estimate the global pose of a vehicle in the face of two distinct problems; first, when using stereo visual odometry for relative motion estimation, a lack of features at close range causes a bias in the motion estimate. The other challenge is localizing in the global coordinate frame using very infrequent GPS measurements. Solving these problems we demonstrate a method to estimate and correct for the bias in visual odometry and a sensor fusion algorithm capable of exploiting sparse global measurements. Our graph-based state estimation framework is capable of inferring global orientation using a unified representation of local and global measurements and recovers from inaccurate initial estimates of the state, as intermittently available GPS information may delay the observability of the entire state. We also demonstrate a reduction of the complexity of the problem to achieve real-time throughput. In our experiments, we show in an outdoor dataset with distant features where our bias corrected visual odometry solution makes a fivefold improvement in the accuracy of the estimated translation compared to a standard approach. For a traverse of 2km we demonstrate the capabilities of our graph-based state estimation approach to successfully infer global orientation with as few as 6 GPS measurements and with two-fold improvement in mean position error using the corrected visual odometry.

Details

IROS Conference 2011 Conference Paper

Efficient target geolocation by highly uncertain small air vehicles

Ben Grocholsky
Michael Dille
Stephen T. Nuske

Geolocation of a ground object or target of interest from live video is a common task required of small and micro unmanned aerial vehicles (SUAVs and MAVs) in surveillance and rescue applications. However, such vehicles commonly carry low-cost and light-weight sensors providing poor bandwidth and accuracy whose contribution to observations is nonlinear, resulting in poor geolocation performance by standard techniques. This paper proposes the application of an efficient over-parameterized state representation to the problem of geolocation that is able to handle large, time-varying, and non-Gaussian sensor error to produce better geolocation estimates than typical approaches and which provides computing and communication benefits in applications such as predictive control and distributed collaboration. We evaluate our filter on real flight data, demonstrating its ability to efficiently produce a solution with tight confidence bounds given highly uncertain data.

Details

IROS Conference 2011 Conference Paper

Perception for a river mapping robot

Andrew Chambers
Supreeth Achar
Stephen T. Nuske
Joern Rehder
Bernd Kitt
Lyle Chamberlain
Justin Haines
Sebastian A. Scherer

We consider the perceptual challenges inherent in the robotic manipulation of previously unseen socks, with the end goal of manipulation by a household robot for laundry. The task poses challenging problems in modeling the appearance, shape and configuration of these textile items that tend to exhibit high variability in texture, design, and style while being highly articulated objects.

Details

ICRA Conference 2011 Conference Paper

Self-supervised segmentation of river scenes

Supreeth Achar
Bharath Sankaran
Stephen T. Nuske
Sebastian A. Scherer
Sanjiv Singh

Here we consider the problem of automatically segmenting images taken from a boat or low-flying aircraft. Such a capability is important for autonomous river following and mapping. The need for accurate segmentation in a wide variety of riverine environments challenges the state of the art vision-based methods that have been used in more structured environments such as roads and highways. Apart from the lack of structure, the principal difficulty is the large spatial and temporal variations in the appearance of water in the presence of nearby vegetation and with reflections from the sky. We propose a self-supervised method to segment images into ‘sky’, ‘river’ and ‘shore’ (vegetation + structures) regions. Our approach uses assumptions about river scene structure to learn appearance models based on features like color, texture and image location which are used to segment the image. We validated our algorithm by testing on four datasets captured under varying conditions on different rivers. Our self-supervised algorithm had higher accuracy rates than a supervised alternative, often significantly more accurate, and does not need to be retrained to work under different conditions.

Details

IROS Conference 2011 Conference Paper

Yield estimation in vineyards by visual grape detection

Stephen T. Nuske
Supreeth Achar
Terry Bates
Srinivasa G. Narasimhan
Sanjiv Singh

The harvest yield in vineyards can vary significantly from year to year and also spatially within plots due to variations in climate, soil conditions and pests. Fine grained knowledge of crop yields can allow viticulturists to better manage their vineyards. The current industry practice for yield prediction is destructive, expensive and spatially sparse - during the growing season sparse samples are taken and extrapolated to determine overall yield. We present an automated method that uses computer vision to detect and count grape berries. The method could potentially be deployed across large vineyards taking measurements at every vine in a non-destructive manner. Our berry detection uses both shape and visual texture and we can demonstrate detection of green berries against a green leaf background. Berry detections are counted and the eventual harvest yield is predicted. Results are presented for 224 vines (over 450 meters) of two different grape varieties and compared against the actual harvest yield as groundtruth. We calibrate our berry count to yield and find that we can predict yield of individual vineyard rows to within 9. 8% of actual crop weight.

Details

ICRA Conference 2010 Conference Paper

Vision-based localization using an edge map extracted from 3D laser range data

Paulo V. K. Borges
Robert Zlot
Michael Bosse
Stephen T. Nuske
Ashley Tews

Reliable real-time localization is a key component of autonomous industrial vehicle systems. We consider the problem of using on-board vision to determine a vehicle's pose in a known, but non-static, environment. While feasible technologies exist for vehicle localization, many are not suited for industrial settings where the vehicle must operate dependably both indoors and outdoors and in a range of lighting conditions. We extend the capabilities of an existing vision-based localization system, in a continued effort to improve the robustness, reliability and utility of an automated industrial vehicle system. The vehicle pose is estimated by comparing an edge-filtered version of a video stream to an available 3D edge map of the site. We enhance the previous system by additionally filtering the camera input for straight lines using a Hough transform, observing that the 3D environment map contains only linear features. In addition, we present an automated approach for generating 3D edge maps from laser point clouds, removing the need for manual map surveying and also reducing the time for map generation down from days to minutes. We present extensive localization results in multiple lighting conditions comparing the system with and without the proposed enhancements.

Details

ICRA Conference 2008 Conference Paper

Visual localisation in outdoor industrial building environments

Stephen T. Nuske
Jonathan Roberts 0001
Gordon F. Wyeth

This paper presents a vision-based method of vehicle localisation that has been developed and tested on a large forklift type robotic vehicle which operates in a mainly outdoor industrial setting. The localiser uses a sparse 3D-edge-map of the environment and a particle filter to estimate the pose of the vehicle. The vehicle operates in dynamic and non-uniform outdoor lighting conditions, an issue that is addressed by using knowledge of the scene to intelligently adjust the camera exposure and hence improve the quality of the information in the image. Results from the industrial vehicle are shown and compared to another laser-based localiser which acts as a ground truth. An improved likelihood metric, using per-edge calculation, is presented and has shown to be 40% more accurate in estimating rotation. Visual localization results from the vehicle driving an arbitrary 1. 5km path during a bright sunny period show an average position error of 0. 44m and rotation error of 0. 62deg.

Details

ICRA Conference 2006 Conference Paper

Extending the Dynamic Range of Robotic Vision

Stephen T. Nuske
Jonathan Roberts 0001
Gordon F. Wyeth

Conventional cameras have limited dynamic range, and as a result vision-based robots cannot effectively view an environment made up of both sunny outdoor areas and darker indoor areas. This paper presents an approach to extend the effective dynamic range of a camera, achieved by changing the exposure level of the camera in real-time to form a sequence of images which collectively cover a wide range of radiance. Individual control algorithms for each image have been developed to maximize the viewable area across the sequence. Spatial discrepancies between images, caused by the moving robot, are improved by a real-time image registration process. The sequence is then combined by merging color and contour information. By integrating these techniques it becomes possible to operate a vision-based robot in wide radiance range scenes

Details