Takeo Kanade

ICRA Conference 2014 Conference Paper

Online approximate model representation of unknown objects

• Kiho Kwak
• Jun-Sik Kim 0001
• Daniel F. Huber
• Takeo Kanade

Object representation is useful for many computer vision tasks, such as object detection, recognition, and tracking. Computer vision tasks must handle situations where unknown objects appear and must detect and track some object which is not in the trained database. In such cases, the system must learn or, otherwise derive, descriptions of new objects. In this paper, we investigate creating a representation of previously unknown objects that newly appear in the scene. The representation creates a viewpoint-invariant and scale-normalized model approximately describing an unknown object with multimodal sensors. Those properties of the representation facilitate 3D tracking of the object using 2D-to-2D image matching. The representation has both benefits of an implicit model (referred to as a view-based model) and an explicit model (referred to as a shape-based model). Experimental results demonstrate the viability of the proposed representation and outperform the existing approaches for 3D-pose estimation.

ICRA Conference 2013 Conference Paper

Maneuver-based autonomous navigation of a small fixed-wing UAV

• Myung Hwangbo
• Takeo Kanade

An urban operation of unmanned aerial vehicles (UAVs) demands a high level of autonomy for tasks presented in a cluttered environment. While fixed-wing UAVs have been well suited for long-endurance missions at a high altitude, their navigation inside an urban area brings more challenges in motion planning and control. The inability to hover and low agility in motion cause more difficulties on planning a feasible path in a compact region, and a limited payload allows only low-grade sensors for state estimation and control.

IROS Conference 2012 Conference Paper

6D pose estimation of textureless shiny objects using random ferns for bin-picking

• José Jeronimo Rodrigues
• Jun-Sik Kim 0001
• Makoto Furukawa
• João M. F. Xavier
• Pedro M. Q. Aguiar
• Takeo Kanade

We address the problem of 6D pose estimation of a textureless and shiny object from single-view 2D images, for a bin-picking task. For a textureless object like a mechanical part, conventional visual feature matching usually fails due to the absence of rich texture features. Hierarchical template matching assumes that few templates can cover all object appearances. However, the appearance of a shiny object largely depends on its pose and illumination. Furthermore, in a bin-picking task, we must cope with partial occlusions, shadows, and inter-reflections.

ICRA Conference 2012 Conference Paper

Real-time topometric localization

• Hernán Badino
• Daniel F. Huber
• Takeo Kanade

Autonomous vehicles must be capable of localizing even in GPS denied situations. In this paper, we propose a real-time method to localize a vehicle along a route using visual imagery or range information. Our approach is an implementation of topometric localization, which combines the robustness of topological localization with the geometric accuracy of metric methods. We construct a map by navigating the route using a GPS-equipped vehicle and building a compact database of simple visual and 3D features. We then localize using a Bayesian filter to match sequences of visual or range measurements to the database. The algorithm is reliable across wide environmental changes, including lighting differences, seasonal variations, and occlusions, achieving an average localization accuracy of 1 m over an 8 km route. The method converges correctly even with wrong initial position estimates solving the kidnapped robot problem.

IROS Conference 2011 Conference Paper

Extrinsic calibration of a single line scanning lidar and a camera

• Kiho Kwak
• Daniel F. Huber
• Hernán Badino
• Takeo Kanade

In robotic hands design tendon driven systems have been considered for years. The main advantage is a small end effector inertia e. g. a light, small hand with high dynamics due to remote actuators. To protect the actuators from impact in unknown environments a compliant mechanism can be used. It absorbs energy during an impact or saves energy to enhance the joint dynamics. In this paper an antagonistic tendon mechanism is presented. It fits 38 times in the DLR Hand Arm System forearm and enables is adapted to the different finger joints and different tendon lengths. A magnetic sensor was developed for the force measurement of the tendons. Finally, the calibration and the robustness are demonstrated through a set of experiments.

ICRA Conference 2011 Conference Paper

Fast and accurate computation of surface normals from range images

• Hernán Badino
• Daniel F. Huber
• Yongwoon Park
• Takeo Kanade

The fast and accurate computation of surface normals from a point cloud is a critical step for many 3D robotics and automotive problems, including terrain estimation, mapping, navigation, object segmentation, and object recognition. To obtain the tangent plane to the surface at a point, the traditional approach applies total least squares to its small neighborhood. However, least squares becomes computationally very expensive when applied to the millions of measurements per second that current range sensors can generate. We reformulate the traditional least squares solution to allow the fast computation of surface normals, and propose a new approach that obtains the normals by calculating the derivatives of the surface from a spherical range image. Furthermore, we show that the traditional least squares problem is very sensitive to range noise and must be normalized to obtain accurate results. Experimental results with synthetic and real data demonstrate that our proposed method is not only more efficient by up to two orders of magnitude, but provides better accuracy than the traditional least squares for practical neighborhood sizes.

ICRA Conference 2011 Conference Paper

Visual-inertial UAV attitude estimation using urban scene regularities

• Myung Hwangbo
• Takeo Kanade

We present a drift-free attitude estimation method that uses image line segments for the correction of accumulated errors in integrated gyro rates when an unmanned aerial vehicle (UAV) operates in urban areas. Since man-made environments generally exhibit strong regularity in structure, a set of line segments that are either parallel or orthogonal to the gravitational direction can provide visual measurements for the absolute attitude from a calibrated camera. Line segments are robustly classified with the assumption that a single vertical vanishing point or multiple horizontal vanishing points exist. In the fusion with gyro angles, we introduce a new Kalman update step that directly uses line segments rather than vanishing points. The simulation and experiment based on urban images at distant views are provided to demonstrate that our method can serve as a robust visual attitude sensor for aerial robot navigation.

IROS Conference 2010 Conference Paper

Analysis of task feasibility for a home robot using prismatic joints

• Tomoaki Mashimo
• Rosen Diankov
• Takateru Urakubo
• Takeo Kanade

This paper evaluates the dynamic and kinematic properties of a prismatic mechanism and shows its capabilities in performing home manipulation tasks when integrated into a robotic arm. Our design is motivated from the observation that human hand motions often follow a linear trajectory when manipulating everyday objects. We present the mechanical design for a light-weight, energy-efficient robot named PRISM that emphasizes translational motion. By simulating the dynamics equations and comparing the structure of commonly used anthropomorphic arms and our proposed arm, we verify that translational motion is more energy efficient with PRISM, and the robot can maneuver itself in narrower places. Through simulation experiments using state of the art manipulation planning algorithms, we analyze the success rates of PRISM and an anthropomorphic robot arm in performing basic tasks. The simulation experiments center on pick-and-place tasks in cluttered kitchen scenes. We show a real-world prototype of PRISM and perform several manipulation experiments with it.

ICRA Conference 2010 Conference Paper

Boundary detection based on supervised learning

• Kiho Kwak
• Daniel F. Huber
• Jeongsook Chae
• Takeo Kanade

Detecting the boundaries of objects is a key step in separating foreground objects from the background, which is useful for robotics and computer vision applications, such as object detection, recognition, and tracking. We propose a new method for detecting object boundaries using planar laser scanners (LIDARs) and, optionally, co-registered imagery. We formulate boundary detection as a classification problem, in which we estimate whether a boundary exists in the gap between two consecutive range measurements. Features derived from the LIDAR and imagery are used to train a support vector machine (SVM) classifier to label pairs of range measurements as boundary or non-boundary. We compare this approach to an existing boundary detection algorithm that uses dynamically adjusted thresholds. Experiments show that the new method performs better even when only LIDAR features are used, and additional improvement occurs when image-based features are included, too. The new algorithm performs better on difficult boundary cases, such as obliquely viewed objects.

IROS Conference 2010 Conference Paper

Efficient pulling motion of a two-link robot arm near singular configuration

• Takateru Urakubo
• Tomoaki Mashimo
• Takeo Kanade

This paper discusses the advantages of singular configurations of a two-link robot arm in achieving tasks of pulling or lifting a heavy object. Optimal base location and arm motion for minimizing the joint torques are examined by numerical simulations, and the base location where the robot arm is near a singular configuration at the start time of task is optimal. It is shown analytically that joint torques can supply energy to the system composed of the robot arm and the object efficiently near singular configurations of the arm. The energy supply rates at two singular configurations are derived based on the equations of motion of the system.

NeurIPS Conference 2010 Conference Paper

Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces

• Abhinav Gupta
• Martial Hebert
• Takeo Kanade
• David Blei

There has been a recent push in extraction of 3D spatial layout of scenes. However, none of these approaches model the 3D interaction between objects and the spatial layout. In this paper, we argue for a parametric representation of objects in 3D, which allows us to incorporate volumetric constraints of the physical world. We show that augmenting current structured prediction techniques with volumetric reasoning significantly improves the performance of the state-of-the-art.

IROS Conference 2010 Conference Paper

Singularity-based mechanism with high responsiveness

• Tomoaki Mashimo
• Takateru Urakubo
• Takeo Kanade

We propose a singularity-based mechanism (SBM) to exploit the singular configuration that improves the angular acceleration instead of constraining the movement. The tradeoff between the responsiveness and the range of motion is achieved by varying a length of linkage in the SBM. In this paper, we clarify the responsiveness of the SBM using the dynamics analysis. For the demonstration, we build an experimental SBM system with the high responsiveness, a practical range of motion, and a size comparable to a human arm. In the experiment, the effectiveness of the SBM is shown in a vertical lifting task. The characteristic of the SBM that generates a large acceleration at start is similar to the human arm moved by a muscle. The similarity between the SBM and the human arm is analyzed in terms of the static torque.

IROS Conference 2009 Conference Paper

Inertial-aided KLT feature tracking for a moving camera

• Myung Hwangbo
• Jun-Sik Kim 0001
• Takeo Kanade

We propose a novel inertial-aided KLT feature tracking method robust to camera ego-motions. The conventional KLT uses images only and its working condition is inherently limited to small appearance change between images. When big optical flows are induced by a camera-ego motion, an inertial sensor attached to the camera can provide a good prediction to preserve the tracking performance. We use a low-grade MEMS-based gyroscope to refine an initial condition of the nonlinear optimization in the KLT. It increases the possibility for warping parameters to be in the convergence region of the KLT. For longer tracking with less drift, we use the affine photometric model and it can effectively deal with camera rolling and outdoor illumination change. Extra computational cost caused by this higher-order motion model is alleviated by restraining the Hessian update and GPU acceleration. Experimental results are provided for both indoor and outdoor scenes and GPU implementation issues are discussed.

IROS Conference 2009 Conference Paper

Optimal placement of a two-link manipulator for door opening

• Takateru Urakubo
• Tomoaki Mashimo
• Takeo Kanade

This paper presents a study on the optimal base location and arm motion of a mobile manipulator for door opening task. Numerical simulation results show that the base location where the manipulability of the two-link arm is almost degenerated at the start and end points of door opening is optimal. We show by analysis that the location has an advantage in supplying kinetic energy to the door by using torques at the joints of arm. In order to represent properly the arm motion near a singular point of manipulability, the rotational motion of the door is parameterized by piecewise fifth order polynomials of time, and the parameters of polynomials are optimized to minimize the joint torques.

ICRA Conference 2008 Conference Paper

Factorization-based calibration method for MEMS inertial measurement unit

• Myung Hwangbo
• Takeo Kanade

We present an easy-to-use calibration method for MEMS inertial sensor units based on the Factorization method which was originally invented for shape-and-motion recovery in computer vision. Our method requires no explicit knowledge of individual motions applied during calibration procedure. Instead a set of motion constraints in the form of an inner-product is used to factorize sensor measurements into a calibration matrix (that represents intrinsic sensor parameters) and a motion matrix (that represents acceleration or angular velocity). These motion constraints can be collected quickly from a low-cost calibration apparatus. Our method is not limited to just triad configurations but also applicable to any coordination of more than three sensor elements. A redundant configuration has the benefit that all the calibration parameters including biases are estimated at once. Simulation and experiments are provided to verify the proposed method.

ICRA Conference 2008 Conference Paper

Motion estimation using multiple non-overlapping cameras for small unmanned aerial vehicles

• Jun-Sik Kim 0001
• Myung Hwangbo
• Takeo Kanade

An imaging sensor made of multiple light-weight non-overlapping cameras is an effective sensor for a small unmanned aerial vehicle that has strong payload limitation. This paper presents a method for motion estimation by assuming that such a multi-camera system is a spherical imaging system (that is, the cameras share a single optical center). We derive analytically and empirically a condition for a multi-camera system to be modeled as a spherical camera. Interestingly, not only does the spherical assumption simplify the algorithms and calibration procedure, but also motion estimation based on that assumption becomes more accurate.

NeurIPS Conference 2008 Conference Paper

Nonrigid Structure from Motion in Trajectory Space

• Ijaz Akhter
• Yaser Sheikh
• Sohaib Khan
• Takeo Kanade

Existing approaches to nonrigid structure from motion assume that the instantaneous 3D shape of a deforming object is a linear combination of basis shapes, which have to be estimated anew for each video sequence. In contrast, we propose that the evolving 3D structure be described by a linear combination of basis trajectories. The principal advantage of this lateral approach is that we do not need to estimate any basis vectors during computation. Instead, we show that generic bases over trajectories, such as the Discrete Cosine Transform (DCT) bases, can be used to effectively describe most real motions. This results in a significant reduction in unknowns, and corresponding stability, in estimation. We report empirical performance, quantitatively using motion capture data and qualitatively on several video sequences exhibiting nonrigid motions including piece-wise rigid motion, articulated motion, partially nonrigid motion (such as a facial expression), and highly nonrigid motion (such as a person dancing).

ICRA Conference 2007 Conference Paper

Efficient Two-phase 3D Motion Planning for Small Fixed-wing UAVs

• Myung Hwangbo
• James J. Kuffner
• Takeo Kanade

We present an efficient two-phase approach to motion planning for small fixed-wing Unmanned Aerial Vehicles (UAVs) navigating in complex 3D air slalom environments. A coarse global motion planner first computes a kinematically feasible obstacle-free path in a discretized 3D workspace which roughly satisfies the kinematic constraints of the UAV. Given a coarse global path, a fine local motion planner is used to compute a more accurate trajectory for the UAV at a higher level of detail. The local planner is iterated as the vehicle traverses and refines the global path as needed up to its planning horizon. We also introduce a new planning heuristic for 3D motions of fixed-wing UAVs based on 2D Dubins curves, along with precomputed sets of motion primitives derived from the vehicle dynamics model in order to achieve high efficiency.

IROS Conference 2007 Conference Paper

GPU-accelerated real-time 3D tracking for humanoid locomotion and stair climbing

• Philipp Michel
• Joel E. Chestnutt
• Satoshi Kagami
• Koichi Nishiwaki
• James J. Kuffner
• Takeo Kanade

For humanoid robots to fully realize their biped potential in a three-dimensional world and step over, around or onto obstacles such as stairs, appropriate and efficient approaches to execution, planning and perception are required. To this end, we have accelerated a robust model-based three-dimensional tracking system by programmable graphics hardware to operate online at frame-rate during locomotion of a humanoid robot. The tracker recovers the full 6 degree-of- freedom pose of viewable objects relative to the robot. Leveraging the computational resources of the GPU for perception has enabled us to increase our tracker's robustness to the significant camera displacement and camera shake typically encountered during humanoid navigation. We have combined our approach with a footstep planner and a controller capable of adaptively adjusting the height of swing leg trajectories. The resulting integrated perception-planning-action system has allowed an HRP-2 humanoid robot to successfully and rapidly localize, approach and climb stairs, as well as to avoid obstacles during walking.

IROS Conference 2007 Conference Paper

Locomotion among dynamic obstacles for the honda ASIMO

• Joel E. Chestnutt
• Philipp Michel
• James J. Kuffner
• Takeo Kanade

We have equipped a Honda ASIMO humanoid with the ability to navigate autonomously in obstacle-filled environments. In addition to finding its way through known, fixed obstacle configurations, the planning system can reason about the future state of the world to locomote through challenging environments when the obstacle motions can be inferred from observation. This video presents work using a vision system to predict the velocities of objects in the scene, allowing ASIMO to safely navigate autonomously through a dynamic environment. Neither obstacle positions nor velocities are known at the start of the trial, but are estimated online as the robot walks. The planner constantly adjusts the footstep path with the latest estimates of ASIMO’s position and the obstacle trajectories, allowing the robot to successfully circumnavigate the moving obstacles.

ICML Conference 2006 Conference Paper

Discriminative cluster analysis

• Fernando De la Torre
• Takeo Kanade

Clustering is one of the most widely used statistical tools for data analysis. Among all existing clustering techniques, k-means is a very popular method because of its ease of programming and because it accomplishes a good trade-off between achieved performance and computational complexity. However, k-means is prone to local minima problems, and it does not scale too well with high dimensional data sets. A common approach to dealing with high dimensional data is to cluster in the space spanned by the principal components (PC). In this paper, we show the benefits of clustering in a low dimensional discriminative space rather than in the PC space (generative). In particular, we propose a new clustering algorithm called Discriminative Cluster Analysis (DCA). DCA jointly performs dimensionality reduction and clustering. Several toy and real examples show the benefits of DCA versus traditional PCA+k-means clustering. Additionally, a new matrix formulation is proposed and connections with related techniques such as spectral graph methods and linear discriminant analysis are provided.

ICRA Conference 2006 Conference Paper

Online Environment Reconstruction for Biped Navigation

• Philipp Michel
• Joel E. Chestnutt
• Satoshi Kagami
• Koichi Nishiwaki
• James J. Kuffner
• Takeo Kanade

As navigation autonomy becomes an increasingly important research topic for biped humanoid robots, efficient approaches to perception and mapping that are suited to the unique characteristics of humanoids and their typical operating environments are required. This paper presents a system for online environment reconstruction that utilizes both external sensors for global localization, and on-body sensors for detailed local mapping. An external optical motion capture system is used to accurately localize on-board sensors that integrate successive 2D views of a calibrated camera and range measurements from a SwissRanger SR-2 time-of-flight sensor to construct global environment maps in real-time. Environment obstacle geometry is encoded in 2D occupancy grids and 2. 5D height maps for navigation planning. We present an on-body implementation for the HRP-2 humanoid robot that, combined with a footstep planner, enables the robot to autonomously traverse dynamic environments containing unpredictably moving obstacles

IROS Conference 2005 Conference Paper

Design of an MR-compatible three-axis force sensor

• Mitsunori Tada
• Takeo Kanade

This paper presents a newly designed and developed MR-compatible three-axis force sensor: its principle, structure and performance. It employs a new MR-compatible optical micrometry based on differential measure of light intensity. This technology enables highly accurate and sensitive two degrees-of-freedom displacement sensing by using optoelectronic devices and pair of fiber optics. In order to realize three-axis force sensitivity, two micrometers are aligned in orthogonal directions. The accuracy of this force sensor is better than 3. 0% and the maximum displacement of the detector is about 40 /spl mu/m under the applied force ranging from 0 to 15 N in vertical, and -8 to 8 N in horizontal directions. The loss of homogeneity of the magnetic field and signal-to-noise ratio of the MR image caused by this sensor are observed to be 0. 49 ppm and 0. 49 to 7. 30% that show the sufficient MR compatibility of this sensor.

ICRA Conference 2005 Conference Paper

Footstep Planning for the Honda ASIMO Humanoid

• Joel E. Chestnutt
• Manfred Lau
• German K. M. Cheung
• James J. Kuffner
• Jessica K. Hodgins
• Takeo Kanade

Despite the recent achievements in stable dynamic walking for many humanoid robots, relatively little navigation autonomy has been achieved. In particular, the ability to autonomously select foot placement positions to avoid obstacles while walking is an important step towards improved navigation autonomy for humanoids. We present a footstep planner for the Honda ASIMO humanoid robot that plans a sequence of footstep positions to navigate toward a goal location while avoiding obstacles. The possible future foot placement positions are dependent on the current state of the robot. Using a finite set of state-dependent actions, we use an A* search to compute optimal sequences of footstep locations up to a time-limited planning horizon. We present experimental results demonstrating the robot navigating through both static and dynamic known environments that include obstacles moving on predictable trajectories.

ICRA Conference 2005 Conference Paper

Learning to Track Multiple People in Omnidirectional Video

• Fernando De la Torre
• Carlos Vallespí
• Paul E. Rybski
• Manuela Veloso
• Takeo Kanade

Meetings are a very important part of everyday life for professionals working in universities, companies or governmental institutions. We have designed a physical awareness system called CAMEO (Camera Assisted Meeting Event Observer), a hardware/software system to record and monitor people's activities in meetings. CAMEO captures a high resolution omnidirectional view of the meeting by stitching images coming from almost concentric cameras. Besides recording capability, CAMEO automatically detects people and learns a person-specific facial appearance model (PS-FAM) for each of the participants. The PSFAMs allow more robust/reliable tracking and identification. In this paper, we describe the video-capturing device, photometric/geometric autocalibration process, and the multiple people tracking system. The effectiveness and robustness of the proposed system is demonstrated over several real-time experiments and a large data set of videos.

ICML Conference 2005 Conference Paper

Multimodal oriented discriminant analysis

• Fernando De la Torre
• Takeo Kanade

IROS Conference 2005 Conference Paper

Online dense local 3D world reconstruction from stereo image sequences

• Satoshi Kagami
• Yutaka Takaoka
• Yusuke Kida
• Koichi Nishiwaki
• Takeo Kanade

This paper describes an online 3D reconstruction system from stereo image sequences to obtain a dense local world model for robot navigation. The proposed method consists of three components: 1) stereo depth map calculation, 2) correspondence calculation in time sequential images by tracking raw image features, 3) 6DOF camera motion estimation by RANSAC and integrate depth map into 3D reconstructed model. We examined and evaluated our method in a motion capture environment for comparison. Finally experimental results of a humanoid robot H7 are denoted.

UAI Conference 2004 Conference Paper

Maximum Entropy for Collaborative Filtering

• C. Lawrence Zitnick
• Takeo Kanade

Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order interactions is generally not present. Second, the variables that we are conditioning upon vary for each query. That is, users label different variables during each query. For this reason, there is no consistent input to output mapping. To address these problems we purpose a maximum entropy approach using a non-standard measure of entropy. This approach can be simplified to solving a set of linear equations that can be efficiently solved.

ICRA Conference 2004 Conference Paper

Microphone Array for 2D Sound Localization and Capture

• Satoshi Kagami
• Hiroshi Mizoguchi
• Yuki Tamai
• Takeo Kanade

This work describes two circular microphone arrays and a square microphone array which can be used for sound localization and sound capture. Sound capture by microphone array is achieved by sum and delay beam former (SDBF). A dedicated PCI 128-channel simultaneous input analog-to-digital (AD) board is developed for a 128 ch microphone array with a maximum sampling rate of 22. 7 /spl mu/s/sample. Simulation of sound pressure distribution of 24 and 128 ch circular microphone array and 128 ch square microphone array are shown. Then a 24 ch circular microphone array and a 128 ch square microphone array have been developed. The 24 ch circular microphone array can capture sound from an arbitrary direction. The 128 ch square microphone array can capture sound from a specific point. Both systems are evaluated by using frequency components of the sound. The circular type system can be used on a mobile robot including humanoid robot and square type can be extended towards room coverage type application.

IROS Conference 2004 Conference Paper

People detection and tracking in high resolution panoramic video mosaic

• Raju Patil
• Paul E. Rybski
• Takeo Kanade
• Manuela Veloso

We have designed a physical awareness system called CAMEO, the camera assisted meeting event observer, which consists of a multi-camera omnidirectional vision system designed to be used in meeting environments. CAMEO is designed to monitor the activities of people in meetings so that it can generate a semantically-indexed summary of what occurred in the meeting. In this paper, we describe CAMEO's fast people detection and tracking module. This module makes use of a combination of frame differencing, face detection, and adaptive color blob tracking based on mean shift analysis to detect and track people in the panoramic image. We describe this algorithm and present experimental results from captured meeting logs.

IROS Conference 2003 Conference Paper

3D ultrasonic tagging system for observing human activity

• Yoshifumi Nishida
• Hiroshi Aizawa
• Toshio Hori
• Nell H. Hoffman
• Takeo Kanade
• Masayoshi Kakikura

This paper describes an ultrasonic tagging system developed for robustly observing human activity in a living area. Using ultrasonic transmitter tags with unique identifiers, the system is shown through experimental application to be able to track the three-dimensional motion of tagged objects in real time with high accuracy, resolution and robustness to occlusion. The use of an ultrasonic system is desirable because of its low cost and use of commercial components, and the proposed system achieves high accuracy and robustness through the use of many redundant sensors. The system employs multilateration to locate tagged objects using one of two estimation algorithms, a least-squares optimization method or a random sample consensus method.

IROS Conference 2003 Conference Paper

An autonomous blimp for a surveillance system

• Takanori Fukao
• Kazushi Fujitani
• Takeo Kanade

An autonomous blimp for a surveillance system, which is circling around a specified target with only one camera, is designed in this paper. For this purpose, an extension of Lucas-Kanade algorithm for detection and tracking of features with rotation and scaling is provided, and a simplified structure-from-motion algorithm is applied to improve the accuracy of state estimation. A tracking controller is designed for the blimp which is an underactuated system. The desired path of the blimp is also generated from image information of a target. The blimp flies around the target automatically, after a commander sets it. Some experiments are performed indoors by using an aerial blimp.

ICRA Conference 2003 Conference Paper

Integrated modeling and robust control for full-envelope flight of robotic helicopters

• Marco La Civita
• George Papageorgiou
• William C. Messner
• Takeo Kanade

To accomplish successfully the complex future mission in civilian and military scenarios, robotic helicopters need to have controllers that exploit their full dynamic capabilities. The absence of high-fidelity simulation models has prevented the use of well established multivariable control techniques for the design of high-bandwidth full-flight-envelope control systems. Existing model-based controllers are of low bandwidth and cover only small portions of the vehicle's flight envelope. In this paper we present the results of the synergistic use of high-fidelity integrated modeling strategies, robust multivariable control techniques, and classical gain scheduling for the rapid and reliable design of high-bandwidth full-flight envelope controllers for robotic helicopters. We implemented and flight tested a gain-scheduled H/sub /spl infin// loop-shaping controller on the Carnegie Mellon University (CMU) Yamaha R-50 robotic helicopter. During the flight tests, the CMU R-50 flew several high-speed maneuvers. We believe that our modeling/control approach quickly delivers controllers that exploit the full dynamic capabilities of the airframe and thus are ready to be used by higher level navigation systems for complex autonomous missions.

IJCAI Conference 1997 Conference Paper

Name-It: Naming and Detecting Faces in Video by the Integration of Image and Natural Language Processing

• Shin'ichi Satoh
• Yuichi Nakamura
• Takeo Kanade

We have been developing Name-It, a system that associates faces and names in news videos. First, as the only knowledge source, the system is given news videos which include image sequences and transcripts obtained from audio tracks or closed caption texts. The system can then either infer the name of a given face and output the name candidates, or can locate the faces in news videos by a name. To accomplish this task, the system extracts faces from image sequences and names from transcripts, both of which might correspond to key persons in news topics. The proposed system takes full advantage of advanced image and natural language processing. The image processing contributes to the extraction of face sequences which provide rich information for face-name association. The processing also helps to select the best frontal view of a face in a face sequence to enhance the face identification which is required for the processing. On the other hand, the natural language processing effectively extracts names by using lexical/grammatical analysis and knowledge of the news video topics structure. The success of our experiments demonstrates the benefits of the advanced image and natural language processing methods and their incorporation.

ICRA Conference 1996 Conference Paper

A sorting image sensor: an example of massively parallel intensity-to-time processing for low-latency computational sensors

• Vladimir Brajovic
• Takeo Kanade

The need for low-latency vision systems is growing: high speed visual servoing and vision-based human computer interface. In this paper we present a new intensity-to-time processing paradigm suitable for low-latency massively parallel global computation over fine-grained data such as images. As an example of a low-latency global computation, we have developed a VLSI sorting computational sensor-a sensor which sorts all pixels of an input image by their intensities, as the image is being sensed. The first sorting sensor prototype is a 21 by 26 array of cells. It detects an image focused thereon and computes the image of indices as well as the image's cumulative histogram, before the intensity data are readout. The image of indices never saturates and has uniform histogram. Under user's control, the chip can perform other operations including simple segmentation and labeling.

IROS Conference 1996 Conference Paper

Vision-based visual/haptic registration for WYSIWYF display

• Yasuyoshi Yokokohji
• Ralph L. Hollis
• Takeo Kanade

We have been working on developing a visual/haptic interface for virtual environments. The authors have previously (1996) proposed a WYSIWYF (What You See Is What You Feel) concept which ensures a correct visual/haptic registration so that what the user can see via a visual interface is consistent with what he/she can feel through a haptic interface. The key components of the WYSIWYF display are (i) vision-based tracking, (ii) video keying, and (iii) physically-based simulation. The first prototype has been built and the proposed concept was demonstrated. It turned out, however, that the original system had a bottleneck in the vision tracking component and the performance was not satisfactory (slow frame rate and large latency). To solve the problem of our first prototype, we have implemented a fast tracker which can track more than 100 markers in video-rate. In this paper, new experimental results are shown followed by the improvements of the vision-based tracking component.

IROS Conference 1995 Conference Paper

Development of a video-rate stereo machine

• Takeo Kanade
• Hiroshi Kano
• Shigeru Kimura
• Atsushi Yoshida
• Kazuo Oda

A video-rate stereo machine has been developed at CMU with the capability of generating a dense range map, aligned with an intensity image, at the video rate. The target performance of the CMU video-rate stereo machine is: 1) multi-image input of 6 cameras; 2) high throughput of 30 million point/spl times/disparity measurement per second; 3) high frame rate of 30 frame/sec; 4) a dense depth map of 256/spl times/240 pixels; 5) disparity search range of up to 60 pixels; 6) high precision of up to 7 bits (with interpolation); and 7) uncertainty estimation available for each pixel.

NeurIPS Conference 1995 Conference Paper

Human Face Detection in Visual Scenes

• Henry Rowley
• Shumeet Baluja
• Takeo Kanade

We present a neural network-based face detection system. A retinally connected neural network examines small windows of an image, and decides whether each window contains a face. The system arbitrates between multiple networks to improve performance over a single network. We use a bootstrap algorithm for training, which adds false detections into the training set as training progresses. This eliminates the difficult task of manually selecting non-face training examples, which must be chosen to span the entire space of non-face images. Comparisons with another state-of-the-art face detection system are presented; our system has better performance in terms of detection and false-positive rates.

IROS Conference 1994 Conference Paper

Fast template matching based on the normalized correlation by using multiresolution eigenimages

• Shinichi Yoshimura
• Takeo Kanade

Presents a fast computation method of the normalized correlation for multiple rotated templates by using multiresolution eigenimages. This method allows the authors to accurately detect both location and orientation of the object in a scene at faster rate than applying conventional template matching to the rotated object. Since the correlation among slightly rotated templates is high, the authors first apply the Karhunen-Loeve expansion to a set of rotated templates and extract "eigenimages" from them. Each template in this set can be approximated by a linear combination of these eigenimages and it substitute for the template in computing the normalized correlation. The number of eigenimages is smaller than that of original templates and computation cost becomes small. Second, the authors employ a multiresolution image structure to reduce the number of rotated templates and location search area. For the lower resolution image, the position and angle are coarsely obtained in a wide region. Then not only searching area for the position but also the range of rotation angle of templates at the next layer can be limited to the neighbor of the prior results. The authors implemented the proposed algorithm on a vision system and realized computation time around 600 msec and achieved sub pixel resolution for translation and 0. 3 degree maximum error for 360 degree rotation on the 512 by 480 gray scale image. Experimental results are shown to demonstrate the accuracy, efficiency and feasibility of the proposed method. >

ICRA Conference 1994 Conference Paper

Real-Thme 3-D Pose Estimation Using a High-Speed Range Sensor

• David A. Simon
• Martial Hebert
• Takeo Kanade

This paper describes a system which can perform full 3-D pose estimation of a single arbitrarily shaped, rigid object at rates up to 10 Hz. A triangular mesh model of the object to be tracked is generated offline using conventional range sensors. Real-time range data of the object is sensed by the CMU high speed VLSI range sensor. Pose estimation is performed by registering the real-time range data to the triangular mesh model using an enhanced implementation of the Iterative Closest Point (ICP) Algorithm introduced by Besl and McKay (1992). The method does not require explicit feature extraction or specification of correspondence. Pose estimation accuracies of the order of 1% of the object size in translation, and 1 degree in rotation have been measured. >

ICRA Conference 1994 Conference Paper

Sensor Placement Desigu for Object Pose Determination with Three Light-Stripe Range Finders

• Keiichi Kemmotsu
• Takeo Kanade

The pose (position and orientation) of a polyhedral object can be determined with range data obtained from simple light-stripe range finders. However, localization results are sensitive to where those range finders are placed in the workspace, that is, sensor placement. It is advantageous for vision tasks in a factory environment to plan optimal sensing positions off-line all at once rather than online sequentially. This paper presents a method for finding an optimal sensor placement off-line to accurately determine the pose of an object when using three light-stripe range finders. We evaluate a sensor placement on the basis of average performance measures such as an error rate of object recognition, recognition speed and pose uncertainty over the state space of object pose by a Monte Carlo method. An optimal sensor placement which is given a maximal score by a scalar function of the performance measures is selected by another Monte Carlo method. We emphasize that the expected performance of our system under an optimal sensor placement can be characterized completely via simulation. >

AIJ Journal 1993 Journal Article

From a real chair to a negative chair

• Takeo Kanade

IROS Conference 1992 Conference Paper

A VISI Smart Sensor For Fast Range Imaging

• Andrew Gruss
• Shigeyuki Tada
• Takeo Kanade

ICRA Conference 1992 Conference Paper

Adaptive control of space robot system with an attitude controlled base

• Yangsheng Xu
• Harry Shum
• Ju-Jang Lee
• Takeo Kanade

The authors discuss adaptive control of a space robot system with an attitude-controlled base on which the robot is attached. An adaptive control scheme in joint space is proposed. Since most tasks are specified in inertia space, instead of joint space, the authors discuss the issues associated to adaptive control in inertia space and identify two potential problems, unavailability of the joint trajectory (since mapping from inertia space trajectory is dynamics-dependent and subject to uncertainty), and nonlinear parameterization in inertia space. For a planar system, the linear parameterization problem is investigated, the design procedure of the controller is illustrated, and the validity and effectiveness of the proposed control scheme are demonstrated. >

ICRA Conference 1992 Conference Paper

Control system of Self-Mobile Space Manipulator

• Yangsheng Xu
• H. Benjamin Brown
• Mark Friedman
• Takeo Kanade

Self-Mobile Space Manipulator (SM/sup 2/) is a simple, 5-DOF (degree-of-freedom), 1/3-scale, laboratory version of a robot designed to walk on the trusswork and other exterior surfaces of Space Station Freedom. It will be capable of routine tasks such as inspection, parts transportation, and simple maintenance procedures. The authors have designed and built the robot and gravity compensation system to permit simulated zero-gravity experiments. They have developed the control system for the SM/sup 2/ including control hardware architecture and operating system, control station with various interfaces, hierarchical control structure, multiphase control strategy for step motion, and various low-level controllers. The system provides operator-friendly real-time monitoring, and robust control for 3D locomotion movements of the flexible robot. >

IROS Conference 1992 Conference Paper

Mobility And Manipulation Of A Light-weight Space Robot

• Yangsheng Xu
• H. Benjamin Brown
• Shigeru Aoki
• Takeo Kanade

ICRA Conference 1991 Conference Paper

A very fast VLSI rangefinder

• Takeo Kanade
• Andrew Gruss
• L. Richard Carley

The authors present a very fast lightstripe rangefinder based on an IC array of photoreceptor and analog signal processor cells which acquires 1000 frames of range image per second-two orders of magnitude faster than currently available rangefinding methods. Unlike a conventional lightstripe range-finder, which obtains a frame of range image by the step-and-repeat process of projecting a stripe and grabbing and analyzing a camera image, the VLSI sensor array of this rangefinder gathers range data in parallel as a scene is swept continuously by a moving stripe. Each cell continuously monitors the output of its photoreceptor, and detects and remembers the time at which it observed the peak incident light intensity during the sweep of the stripe. Prototype rangefinding systems have been built using a 28*32 array of these sensing elements. >

ICRA Conference 1991 Conference Paper

Extracting topographic features for outdoor mobile robots

• In-So Kweon
• Takeo Kanade

Methods are presented for building high-level terrain descriptions, referred as topographic maps, by extracting terrain features like peaks, pits, ridges, and ravines from the contour map. The resulting topographic map contains the location and type of terrain features as well as the ground topography. The authors develop new definitions for those topographic features based on the contour map. They build a contour map from an elevation map and generate the connectivity tree of all regions separated by the contours. The authors use this connectivity tree, called a topographic change tree, to extract the topographic features. Experimental results on a digital elevation model support the definitions for topographic features and the approach. >

ICRA Conference 1991 Conference Paper

Recovering shape in the presence of interreflections

• Shree K. Nayar
• Katsushi Ikeuchi
• Takeo Kanade

An algorithm for recovering the shape and reflectance of Lambertian surfaces in the presence of interreflections is presented. The surfaces may be of arbitrary but continuous shape, and with possibly varying and unknown reflectance. The actual shape and reflectance are recovered from the pseudoshape and pseudoreflectance estimated by a local shape-from-intensity method (e. g. , photometric stereo). Thus, the algorithm enhances the performance and the utility of existing shape-from-intensity methods. From the results reported, two observations can be made that are pertinent to machine vision: interreflections can cause vision algorithms to produce unacceptably erroneous results and hence should not be ignored; and at least some interreflection problems are tractable and solvable. >

ICRA Conference 1991 Conference Paper

Vision and control techniques for robotic visual tracking

• Nikolaos P. Papanikolopoulos
• Pradeep K. Khosla
• Takeo Kanade

Algorithms for robotic real-time visual tracking of arbitrary 3-D objects traveling at unknown velocities in a 2-D space are presented. The problem of visual tracking is formulated as a problem of combining control with computer vision. A mathematical formulation that is general enough to be extended to the problem of tracking 3-D objects in 3-D space is presented. The authors propose the use of sum-of-squared differences optical flow for the computation of the vector of discrete displacements each instant of time. These displacements can be fed either directly to a PI controller, a pole assignment controller, or a discrete steady-state Kalman filter. In the latter case, the Kalman filter calculates the estimated values of the system's states and exogenous disturbances, and a discrete LQG controller computes the desired motion of the robotic system. The outputs of the controllers are sent to a Cartesian robotic controller that drives the robot. >

IROS Conference 1990 Conference Paper

High resolution terrain map from multiple sensor data

• In-So Kweon
• Takeo Kanade

Describes a terrain mapping 3D vision system to build a high resolution terrain map from multiple range images and a digital elevation model (DEM). To build a composite map of the environment from multiple sensor data, the terrain mapping system needs a representation of the terrain that must be appropriate for multiple sensor data. Building a composite terrain map also requires estimating motion between sensor views and merging these views into a composite map. The terrain representation described consists of a grid-based representation, called elevation map. The authors develop the locus method to build elevation maps from range images. The locus method uses a model of the sensor to interpolate at arbitrary resolution without making any assumptions on the terrain shape other than the continuity of the surface. They also present a pixel-based or iconic terrain matching algorithm to estimate the vehicle motion from a sequence of range images. This terrain matching method uses the locus method to solve correspondence and occlusion problems. Comprehensive test results using a long sequence of range images and a DEM for rugged outdoor terrain are given.

ICRA Conference 1989 Conference Paper

CHIMERA: a real-time programming environment for manipulator control

• Donald E. Schmitz
• Pradeep K. Khosla
• Regis Hoffman
• Takeo Kanade

CHIMERA is a real-time computing environment used in the Reconfigurable Modular Manipulator System project. CHIMERA, which is both a hardware and software environment, allows rapid development and implementation of real-time control programs. It provides a C/Unix-flavored concurrent programming environment for a Motorola 68020 multiprocessor hardware configuration connected to a Sun workstation. CHIMERA has been implemented using commercial hardware in conjunction with a sophisticated, locally developed software package, resulting in a reliable, reasonably priced, and easily duplicated system. CHIMERA is currently being ported for real-time control of the CMU Direct Drive Arm II. The authors describe the implementation and capabilities of the CHIMERA environment and illustrate how these features are used in robot control applications. >

IROS Conference 1989 Conference Paper

Methods for Identifying Footfall Positions for a Legged Robot

• Claude Caillas
• Martial Hebert
• Eric Krotkov
• In-So Kweon
• Takeo Kanade

ICRA Conference 1989 Conference Paper

Shape and reflectance from an image sequence generated using extended sources

• Shree K. Nayar
• Katsushi Ikeuchi
• Takeo Kanade

The authors present a method for determining the shapes of surfaces whose reflectance properties may vary from Lambertian to specular, without prior knowledge of the relative strengths of the Lambertian and specular components of reflection. The object surface is illuminated using extended light sources and is viewed from a single direction. Surface illumination using extended sources makes it possible to ensure the detection of both Lambertian and specular reflections. Multiple source directions are used to obtain an image sequence of the object. An extraction algorithm uses the set of image intensity values measured at each surface point to compute orientation as well as relative strengths of the Lambertian and specular reflection components. The proposed method is called photometric sampling, as it uses samples of photometric function that relates image intensity to surface orientation, reflectance, and light source characteristics. Experiments were conducted on Lambertian surfaces, specular surfaces, and hybrid surfaces, whose reflectance models are composed of both Lambertian and specular components. The results show high accuracy in measured orientations and estimated reflectance parameters. >

ICRA Conference 1989 Conference Paper

Terrain mapping for a roving planetary explorer

• Martial Hebert
• Claude Caillas
• Eric Krotkov
• In-So Kweon
• Takeo Kanade

The authors are prototyping a legged vehicle, the Ambler, for an exploratory mission on another planet, conceivably Mars, where it is to traverse uncharted areas and collect material samples. They describe how the rover can construct from range imagery a geometric terrain representation, i. e. , elevation map that includes uncertainty, unknown areas, and local features. First, they present an algorithm for constructing an elevation map from a single range image. By virtue of working in spherical-polar space, the algorithm is independent of the desired map resolution and the orientation of the sensor, unlike algorithms that work in Cartesian space. Secondly, the authors present a two-stage matching technique (feature matching followed by iconic matching) that identifies the transformation T corresponding to the vehicle displacement between two viewing positions. Thirdly, to support legged locomotion over rough terrain, they describe methods for evaluating regions of the constructed elevation maps as footholds. >

ICRA Conference 1988 Conference Paper

Geometric camera calibration using systems of linear equations

• Keith D. Gremban
• Charles E. Thorpe
• Takeo Kanade

Geometric camera calibration is the process of determining a mapping between points in world coordinates and the corresponding image locations of the points. In previous methods, calibration typically involved the iterative solution to a system of nonlinear equations. A method is presented for performing camera calibration that provides a complete, accurate solution, using only linear systems of equations. By using two calibration planes, a line-of-sight vector is defined for each pixel in the image. The effective focal point of a camera can be obtained by solving the system that defines the intersection point of the line-of-sight vectors. Once the focal point has been determined, a complete camera model can be obtained with a straightforward least-squares procedure. This method of geometric camera calibration has the advantages of being accurate, efficient, and practical for a wide variety of applications. >

ICRA Conference 1986 Conference Paper

An algorithm to determine the identifiable parameters in dynamic robot models

• Pradeep K. Khosla
• Takeo Kanade

AIJ Journal 1986 Journal Article

Incremental reconstruction of 3D scenes from multiple, complex images

• Martin Herman
• Takeo Kanade

The 3D Mosaic system is a vision system that incrementally reconstructs complex 3D scenes from a sequence of images obtained from multiple viewpoints. The system encompasses several levels of the vision process, starting with images and ending with symbolic scene descriptions. This paper describes the various components of the system, including stereo analysis, monocular analysis, and constructing and updating the scene model. In addition, the representation of the scene model is described. This model is intended for tasks such as matching, display generation, planning paths through the scene, and making other decisions about the scene environment. Examples showing how the system is used to interpret complex aerial photographs of urban scenes are presented. Each view of the scene, which may be either a single image or a stereo pair, undergoes analysis which results in a 3D wire-frame description that represents portions of edges and vertices of objects. The model is a surface-based description constructed from the wire frames. With each successive view, the model is incrementally updated and gradually becomes more accurate and complete. Task-specific knowledge, involving block-shaped objects in an urban scene, is used to extract the wire frames and construct and update the model. The model is represented as a graph in terms of symbolic primitives such as faces, edges, vertices, and their topology and geometry. This permits the representation of partially complete, planar-faced objects. Because incremental modifications to the model must be easy to perform, the model contains mechanisms to (1) add primitives in a manner such that constraints on geometry imposed by these additions are propagated throughout the model, and (2) modify and delete primitives if discrepancies arise between newly derived and current information. The model also contains mechanisms that permit the generation, addition, and deletion of hypotheses for parts of the scene for which there is little data.

ICRA Conference 1986 Conference Paper

Progress in robot road-following

• Richard S. Wallace 0001
• K. Matsuzaki
• Yoshimasa Goto
• Jill D. Crisman
• Jon A. Webb
• Takeo Kanade

We report progress in visual road following by autonomous robot vehicles. We present results and work in progress in the areas of system architecture, image rectification and camera calibration, oriented edge tracking, color classification and road-region segmentation, extracting geometric structure, and the use of a map. In test runs of an outdoor robot vehicle, the Terregator, under control of the Warp computer, we have demonstrated continuous motion vision-guided road-following at speeds up to 1. 08 km/hour with image processing and steering servo loop times of 3 sec.

ICRA Conference 1986 Conference Paper

Real-time implementation and evaluation of model-based controls on CMU DD Arm II

• Pradeep K. Khosla
• Takeo Kanade

This paper presents the experimental results of the real-time performance of model-based control algorithms. We compare the computed-torque scheme which utilizes the complete dynamics model of the manipulator with the independent joint control scheme which assumes a decoupled and linear model of the manipulator dynamics. The two manipulator control schemes have been implemented on the CMU DD Arm II with a sampling period of 2 ms. Our initial investigation shows that the computed-torque scheme outperforms the independent joint control scheme as long as there is no torque saturation in the actuators.

IJCAI Conference 1985 Conference Paper

First Results in Robot Road-Following

• Richard Wallace
• Anthony Stentz
• Charles Thorpe
• Hans Maravec
• William Whittaker
• Takeo Kanade

IJCAI Conference 1985 Conference Paper

Optical Navigation by the Method of Differences

• Bruce D. Lucas
• Takeo Kanade

IJCAI Conference 1985 Conference Paper

Stereo by Two-Level Dynamic Programming

• Yuichi Ohta
• Takeo Kanade

AIJ Journal 1981 Journal Article

Recovery of the three-dimensional shape of an object from a single view

• Takeo Kanade

AIJ Journal 1980 Journal Article

A theory of Origami world

• Takeo Kanade