Author name cluster

Blake Hannaford

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

62 papers

2 author rows

ICRA Conference 2021 Conference Paper

Learning Surgical Motion Pattern from Small Data in Endoscopic Sinus and Skull Base Surgeries

Yangming Li
Randall A. Bly
Sarah Akkina
Fangbo Qin
Rajeev C. Saxena
Ian Humphreys
Mark Whipple
Kris S. Moe

Existing studies demonstrated that surgical motion patterns are strongly correlated with surgical outcomes. Real surgeries are complicated and it is expensive to harvest surgical data. Consequently, existing researches on surgical motion patterns focus on specific concise surgical tasks or simple surgical procedures. The paper presents a surgical motion pattern modeling technique that uses small data but can be applied to virtually any Endoscopic Sinus and Skull Base Surgeries (ESSBSs). The proposed method decreases the dimensionalities of the feature space through projecting surgical instrument motions into the endoscope coordinate, based on human expert domain knowledge. Furthermore, the method uses kinematic features and learns the motion pattern with Gaussian Process learning techniques. Comparing with existing surgical motion pattern modeling methods, the proposed method: 1, learns the motion model from small data; 2, can be generally applied to ESSBSs because it neither assumes nor depends on specific surgical tasks; 3, provides informative results in a real-time manner for optimizing surgical motions for improving surgical outcomes. The proposed method was verified by predicting surgical skill levels on cadaver surgeries. The results show the real-time prediction precision is higher than 81% and the offline accumulated precision reach 100%.

IJCAI Conference 2020 Conference Paper

IKBT: Solving Symbolic Inverse Kinematics with Behavior Tree (Extended Abstract)

Dianmu Zhang
Blake Hannaford

Inverse kinematics solves the problem of how to control robot arm joints to achieve desired end effector positions, which is critical to any robot arm design and implementations of control algorithms. It is a common misunderstanding that closed-form inverse kinematics analysis is solved. Popular software and algorithms, such as gradient descent or any multi-variant equations solving algorithm, claims solving inverse kinematics but only on the numerical level. While the numerical inverse kinematics solutions are relatively straightforward to obtain, these methods often fail, even when the inverse kinematics solutions exist. Therefore, closed-form inverse kinematics analysis is superior, but there is no generalized automated algorithm. Up till now, the high-level logical reasoning involved in solving closed-form inverse kinematics made it hard to automate, so it's handled by human experts. We developed IKBT, a knowledge-based intelligent system that can mimic human experts' behaviors in solving closed-from inverse kinematics using Behavior Tree. Knowledge and rules used by engineers when solving closed-from inverse kinematics are encoded as actions in Behavior Tree. The order of applying these rules is governed by higher level composite nodes, which resembles the logical reasoning process of engineers. It is also the first time that the dependency of joint variables, an important issue in inverse kinematics analysis, is automatically tracked in graph form. Besides generating closed-form solutions, IKBT also explains its solving strategies in human (engineers) interpretable form. This is a proof-of-concept of using Behavior Trees to solve high-cognitive problems.

PDF Details DOI

IROS Conference 2020 Conference Paper

LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images

Shan Lin
Fangbo Qin
Yangming Li
Randall A. Bly
Kris S. Moe
Blake Hannaford

Intelligent vision is appealing in computer-assisted and robotic surgeries. Vision-based analysis with deep learning usually requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. We investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model live-cadaver GAN (LC-GAN) based on generative adversarial networks (GANs). We consider a situation when a labeled cadaveric surgery dataset is available while the task is instrument segmentation on an unlabeled live surgery dataset. We train LC-GAN to learn the mappings between the cadaveric and live images. For live image segmentation, we first translate the live images to fake-cadaveric images with LC-GAN and then perform segmentation on the fake-cadaveric images with models trained on the real cadaveric dataset. The proposed method fully makes use of the labeled cadaveric dataset for live image segmentation without the need to label the live dataset. LC-GAN has two generators with different architectures that leverage the deep feature representation learned from the cadaveric image based segmentation task. Moreover, we propose the structural similarity loss and segmentation consistency loss to improve the semantic consistency during translation. Our model achieves better image-to-image translation and leads to improved segmentation performance in the proposed cross-domain segmentation task.

ICRA Conference 2020 Conference Paper

RAVEN-S: Design and Simulation of a Robot for Teleoperated Microgravity Rodent Dissection Under Time Delay

Andrew Lewis 0001
David Drajeske
John Raiti
Angelique Berens
Jacob Rosen 0001
Blake Hannaford

The International Space Station (ISS) serves as a research lab for a wide variety of experiments including some that study the biological effects of microgravity and spaceflight using the Rodent Habitat and Microgravity Science Glovebox (MSG). Astronauts train for onboard dissections of rodents following basic training. An alternative approach for conducting these experiments is teleoperation of a robot located on the ISS from earth by a scientist who is proficient in rodent dissection. This pilot study addresses (1) the effects of extreme time delay on skill degradation during Fundamentals of Laparoscopic Surgery (FLS) tasks and rodent dissections using RAVEN II; (2) derivation and testing of rudimentary interaction force estimation; (3) elicitation of design requirements for an onboard dissection robot, RAVEN-S; and (4) simulation of the RAVEN-S prototype design with dissection data. The results indicate that the tasks' completion times increased by a factor of up to 9 for a 3 s time delay while performing manipulation and cutting tasks (FLS model) and by a factor of up to 3 for a 0. 75 s time delay during mouse dissection tasks (animal model). Average robot forces/torques of 14N/0. 1Nm (peak 90N/0. 75Nm) were measured along with average linear/angular velocities of 0. 02m/s/4rad/s (peak 0. 1m/s/40rad/s) during dissection. A triangular configuration of three arms with respect to the operation site showed the best configuration given the MSG geometry and the dissection tasks. In conclusion, the results confirm the feasibility of utilizing a surgically-inspired RAVEN-S robot for teleoperated rodent dissection for successful completion of the predefined tasks in the presence of communications time delay between the ISS and ground control.

ICRA Conference 2020 Conference Paper

Real-time Data Driven Precision Estimator for RAVEN-II Surgical Robot End Effector Position

Haonan Peng 0002
Xingjian Yang
Yun-Hsuan Su
Blake Hannaford

Surgical robots have been introduced to operating rooms over the past few decades due to their high sensitivity, small size, and remote controllability. The cable-driven nature of many surgical robots allows the systems to be dexterous and lightweight, with diameters as low as 5mm. However, due to the slack and stretch of the cables and the backlash of the gears, inevitable uncertainties are brought into the kinematics calcu-lation [1]. Since the reported end effector position of surgical robots like RAVEN-II [2] is directly calculated using the motor encoder measurements and forward kinematics, it may contain relatively large error up to 10mm, whereas semi-autonomous functions being introduced into abdominal surgeries require position inaccuracy of at most 1mm. To resolve the problem, a cost-effective, real-time and data-driven pipeline for robot end effector position precision estimation is proposed and tested on RAVEN-II. Analysis shows an improved end effector position error of around 1mm RMS traversing through the entire robot workspace without high-resolution motion tracker. The open source code, data sets, videos, and user guide can be found at //github.com/HaonanPeng/RAVEN Neural Network Estimator.

JAIR Journal 2019 Journal Article

IKBT: Solving Symbolic Inverse Kinematics with Behavior Tree

Dianmu Zhang
Blake Hannaford

Inverse kinematics solves the problem of how to control robot arm joints to achieve desired end effector positions, which is critical to any robot arm design and implementations of control algorithms. It is a common misunderstanding that closed-form inverse kinematics analysis is solved. Popular software and algorithms, such as gradient descent or any multi-variant equations solving algorithm, claims solving inverse kinematics but only on the numerical level. While the numerical inverse kinematics solutions are relatively straightforward to obtain, these methods often fail, due to dependency on specific numerical values, even when the inverse kinematics solutions exist. Therefore, closed-form inverse kinematics analysis is superior, but there is no generalized automated algorithm. Up till now, the high-level logical reasoning involved in solving closed-form inverse kinematics made it hard to automate, so it's handled by human experts. We developed IKBT, a knowledge-based intelligent system that can mimic human experts' behaviors in solving closed-from inverse kinematics using Behavior Tree. Knowledge and rules used by engineers when solving closed-from inverse kinematics are encoded as actions in Behavior Tree. The order of applying these rules is governed by higher level composite nodes, which resembles the logical reasoning process of engineers. It is also the first time that the dependency of joint variables, an important issue in inverse kinematics analysis, is automatically tracked in graph form. Besides generating closed-form solutions, IKBT also explains its solving strategies in human (engineers) interpretable form. This is a proof-of-concept of using Behavior Trees to solve high-cognitive problems.

PDF Details DOI

IROS Conference 2019 Conference Paper

Multicamera 3D Reconstruction of Dynamic Surgical Cavities: Non-Rigid Registration and Point Classification

Yun-Hsuan Su
Kevin Huang 0001
Blake Hannaford

Deformable objects and surfaces are ubiquitous in the daily lives of humans – from the garments in fashion to soft tissues within the body. Because of this routine interaction with soft materials, humans are adept and trained in manipulation of deformable objects while avoiding irreversible damage. The dexterity and care involved is largely facilitated through a combination of the human haptic sense of touch and visual observations of object deformation [1]. While this scenario presents itself as a trivially intuitive task, it becomes significantly more difficult and complex with the deprivation of both 3D depth perception and haptic senses. This deprived state is not dissimilar to the scenarios encountered in many robot-assisted minimally invasive surgeries. As a result, unintentional tissue damage can occur due to lack of force feedback and fine 3D visibility [2]. One approach to remediate these issues combines real-time dynamic 3D reconstruction and vision-based force estimation for haptic feedback. Toward that end, this work continues research in a series of studies focusing on multicamera 3D reconstruction of dynamic surgical cavities. Previous work introduced a novel approach of camera grouping and pair sequencing [3]. This paper builds upon that work by introducing a method for non-rigid, sparse point cloud registration and subsequent point classification. In particular, to enable deformation and force analyses, surfaces are locally classified into three categories: static, shifting and deforming. The topics addressed in this paper present open challenges and ongoing research directions for researchers to this day [4], and provide a step towards real-time 3D reconstruction and force feedback in robot-assisted surgery.

ICRA Conference 2019 Conference Paper

Surgical Instrument Segmentation for Endoscopic Vision with Data Fusion of rediction and Kinematic Pose

Fangbo Qin
Yangming Li
Yun-Hsuan Su
De Xu
Blake Hannaford

The real-time and robust surgical instrument segmentation is an important issue for endoscopic vision. We propose an instrument segmentation method fusing the convolutional neural networks (CNN) prediction and the kinematic pose information. First, the CNN model ToolNet-C is designed, which cascades a convolutional feature extractor trained over numerous unlabeled images and a pixel-wise segmentor trained on few labeled images. Second, the silhouette projection of the instrument body onto the endoscopic image is implemented based on the measured kinematic pose. Third, the particle filter with the shape matching likelihood and the weight suppression is proposed for data fusion, whose estimate refines the kinematic pose. The refined pose determines an accurate silhouette mask, which is the final segmentation output. The experiments are conducted with a surgical navigation system, several animal-tissue backgrounds, and a debrider instrument.

ICRA Conference 2018 Conference Paper

A Novel Recurrent Neural Network for Improving Redundant Manipulator Motion Planning Completeness

Yangming Li
Shuai Li 0002
Blake Hannaford

Recurrent Neural Networks (RNNs) demonstrated advantages on control precision, system robustness and computational efficiency, and have been widely applied to redundant manipulator control optimization. Existing RNN control schemes locally optimize trajectories and are efficient and reliable on obstacle avoidance. However, for motion planning, they suffer from local minimum and do not have planning completeness. This work explained the cause of the planning incompleteness and addressed the problem with a novel RNN control scheme. The paper presented the proposed method in detail and analyzed the global stability and the planning completeness in theory. The proposed method was compared with other three control schemes on the precision, the robustness and the planning completeness in software simulation and the results shows the proposed method has improved precision and robustness, and planning completeness.

IROS Conference 2018 Conference Paper

Comparison of 3D Surgical Tool Segmentation Procedures with Robot Kinematics Prior

Yun-Hsuan Su
Issac Huang
Kevin Huang 0001
Blake Hannaford

3D reconstruction and surgical tool segmentation are necessary for several advanced tasks in robot-assisted laparoscopic surgery. These tasks include vision-based force estimation, surgical guidance, and medical image registration where pre-operative data (CT or MRI scan image slices) are overlaid on patient anatomy in real-time during surgery [1] to name a few. In this work, two main strategies were considered: (1) initialize with surgical tool segmentation from 2D images, then proceed to local 3D reconstruction near the tool-tissue interaction region by projecting the segmented result into 3D space, and (2) initialize with 3D reconstruction of the entire surgical task space, followed by surgical tool segmentation from within the 3D reconstructed model. Both methods were implemented on the Raven II surgical robot system, and accuracy and time complexity for both methods were comparatively analyzed while considering various task parameters. Finally, based on the results of this work, guidelines for selecting reconstruction and segmentation strategies and procedure for particular situations are outlined in Section V.

IROS Conference 2018 Conference Paper

Learned Hand Gesture Classification Through Synthetically Generated Training Samples

Kyle Lindgren
Niveditha Kalavakonda
David E. Caballero
Kevin Huang 0001
Blake Hannaford

Hand gestures are a natural component of human-human communication. Simple hand gestures are intuitive and can exhibit great lexical variety. It stands to reason that such a user input mechanism can have many benefits, including seamless interaction, intuitive control and robustness to physical constraints and ambient electrical, light and sound interference. However, while semantic and logical information encoded via hand gestures is readily decoded by humans, leveraging this communication channel in human-machine interfaces remains a challenge. Recent data-driven deep learning approaches are promising towards uncovering abstract and complex relationships that manual and direct rule-based classification schemes fail to discover. Such an approach is amenable towards hand gesture recognition, but requires myriad data which can be collected physically via user experiments. This process, however, is onerous and tedious. A streamlined approach with less overhead is sought. To that end, this work presents a novel method of synthetic hand gesture dataset generation that leverages modern gaming engines. Furthermore, preliminary results indicate that the dataset, despite being synthetic and requiring no physical data collection, is both accurate and rich enough to train a real-world hand gesture classifier that operates in real-time.

IROS Conference 2018 Conference Paper

Soft-obstacle Avoidance for Redundant Manipulators with Recurrent Neural Network

Yangming Li
Blake Hannaford

Compressing soft-obstacles secondary to a controlled motion task is common for human beings. While these tasks are nearly trivial for teleoperated robots, they remain a challenging problem in robotic autonomy. Addressing the problem is significant. For example, in Minimally Invasive Surgeries (MISs), safely compressing soft tissues ensures the surgical safety and decreases tissue removal, thus dramatically decreases surgical trauma and operating room time, and leads to improved surgical outcomes. In this work, we define the problem of soft-obstacle avoidance and project the safety motion constraints into the task space and the velocity space. We illustrate the significance of addressing this problem in the robotic surgery scenario. We present a Recurrent Neural Networks (RNNs) based solution, which formulates the problem as an inequality constrained optimization problem and solves it in its dual space. The application of the proposed method was demonstrated in the Raven II surgical robot. Experimental results demonstrated that the proposed method is effective in addressing the soft-obstacle avoidance problem.

IROS Conference 2017 Conference Paper

Improving control precision and motion adaptiveness for surgical robot with recurrent neural network

Yangming Li
Shuai Li 0002
David E. Caballero
Muneaki Miyasaka
Andrew Lewis 0001
Blake Hannaford

Surgical robot research is driven by the desire of improving surgical outcomes. This paper proposed a Recurrent Neural Network based controller to address two problems: 1) improving control precision, 2) increasing adaptiveness for robot motion (explained in Section I). RNN was adopted in this work mainly because 1) the problem formulation naturally matches RNN structure, 2) RNN has advantages as an biologically inspired method. The proposed method was explained in detail and analysis shows that the proposed method is able to dynamically regulate outputs to increase the adaptiveness and the control precision. This paper uses Raven II surgical robot as an example to show the application of the proposed method, and the numeral simulation results from the proposed method and three other controllers show that the proposed method has improved precision, improved high robustness against noise and increased movement smoothness, and it keeps the manipulator links as far away as possible from physical boundaries, which potentially increases surgical safety and leads to improved surgical outcomes.

IROS Conference 2017 Conference Paper

Integrated asymmetric stop operator based model for strain stress hysteresis characteristics of cable driven robots loaded longitudinally

Omar Aljanaideh
Muneaki Miyasaka
Blake Hannaford

Beside the output-input hysteresis, the longitudinally loaded cables of medical robotics such as RAVEN II exhibit asymmetric saturated strain-load hysteresis loops. This study investigates modeling the hysteresis nonlinearities of these cables using a stop-operator based Prandtl-Ishlinskii (SPI) model that is integrated with a memoryless function. The stop-operator based model is employed to account for the hysteresis nonlinearities, while the memoryless function is introduced to characterize saturation and asymmetric effects. A numerical example is presented to compare the properties of the proposed model with the classic SPI model. The response of the suggested model was evaluated on the hysteresis properties of two different cables subjected to triangular harmonic input of 0 to 0. 001 with 6. 25 × 10 -5 strain/s. The characterization error of the thick cable was found as 1. 55 %, while the error was calculated as 1. 25 % for the thin cable. The relative significance of the proposed model was further examined by comparing the measured data with the classic SPI model. The results showed that the classic model yields substantial characterization errors when the asymmetry and saturation effects of the strain-load hysteresis loops are ignored.

ICRA Conference 2017 Conference Paper

Roboscope: A flexible and bendable surgical robot for single portal Minimally Invasive Surgery

Jacob Rosen 0001
Laligam N. Sekhar
Daniel Glozman
Muneaki Miyasaka
Jesse Dosher
Brian Dellon
Kris S. Moe
Aylin Kim

Minimally Invasive Surgery (MIS) can reduce iatrogenic injury and decrease the possibility of surgical complications. This paper presents a novel flexible and bendable endoscopic device, “Roboscope”, which delivers two instruments, two miniature scanning fiber endoscopes, and a suction/irrigation port to the operation site through a single portal. Compared with existing bendable and steerable robotic surgical systems, Roboscope provides two bending degrees of freedom for its outer sheath and two insertion degrees of freedom, while simultaneously delivering two instruments and two endoscopes to the surgical site. Each bending axis and insertion freedom of Roboscope is independently controllable via an external actuation pack. Surgical tools can be changed without retracting the robot arm. This paper presents the design of the Roboscope mechanical system, electrical system, and control and software systems, design requirements and prototyping validation as well as analysis of Roboscope workspece.

ICRA Conference 2016 Conference Paper

Dynamic modeling of cable driven elongated surgical instruments for sensorless grip force estimation

Yangming Li
Muneaki Miyasaka
Mohammad Haghighipanah
Lei Cheng
Blake Hannaford

Haptic feedback plays a key role in surgeries, but it is still a missing component in robotic Minimally Invasive Surgeries. This paper proposes a dynamic model-based sensorless grip force estimation method to address the haptic perception problem for commonly used elongated cable-driven surgical instruments. Cable and cable-pulley properties are studied for dynamic modeling; grip forces, along with driven motor and gripper jaw positions and velocities are jointly estimated with Unscented Kalman Filter and only motor encoder readings and motor output torques are assumed to be known. A bounding filter is used to compensate for model inaccuracy and to improve method robustness. The proposed method was validated on a 10mm gripper which is driven by a Raven-II surgical robot. The gripper was equipped with 1-dimensional force sensors which served as ground truth data. The experimental results showed that the proposed method provides sufficiently good grip force estimation, while only motor encoder and the motor torques are used as observations.

ICRA Conference 2016 Conference Paper

Hysteresis model of longitudinally loaded cable for cable driven robots and identification of the parameters

Muneaki Miyasaka
Mohammad Haghighipanah
Yangming Li
Blake Hannaford

In this paper, we propose model of longitudinally loaded cable based on the Bouc-Wen hysteresis model and within the framework of the Duhem operator. By optimizing the 9 hysteresis model parameters with a genetic algorithm, the proposed model is shown to be capable of representing quasi-static response of two different diameter cables, 0. 61 mm (thin) and 1. 19 mm (thick), used for the RAVEN II surgical robotic surgery platform. The construction of the cable is 7 strands with 19 individual wires per strand. Furthermore, it is shown that the dynamic response of the cables are captured by adding a linear damping term. The hysteresis model and linear damper with the optimized parameters accurately models a longitudinal vibration test result in terms of frequency, steady state stretch, and logarithmic decrement. Energy dissipation due solely to the hysteresis term is approximately calculated to be 57 and 71% of the total energy loss for the thin and thick cables respectively. The proposed model may be used for cables with different contraction and diameter and can be applied for control of cable driven robots in which cables are stretched longitudinally without large excitation of other modes.

ICRA Conference 2016 Conference Paper

Unscented Kalman Filter and 3D vision to improve cable driven surgical robot joint angle estimation

Mohammad Haghighipanah
Muneaki Miyasaka
Yangming Li
Blake Hannaford

Cable driven manipulators are popular in surgical robots due to compact design, low inertia, and remote actuation. In these manipulators, encoders are usually mounted on the motor, and joint angles are estimated based on transmission kinematics. However, due to non-linear properties of cables such as cable stretch, lower stiffness, and uncertainties in kinematic model parameters, the precision of joint angle estimation is limited with transmission kinematics approach. To improve the positioning of these manipulators, we use a pair of low cost stereo camera as the observation for joint angles and we input these noisy measurements into an Unscented Kalman Filter (UKF) for state estimation. We use the dual UKF to estimate cable parameters and states offline. We evaluated the effectiveness of the proposed method on a Raven-II experimental surgical research platform. Additional encoders at the joint output were employed as a reference system. From the experiments, the UKF improved the accuracy of joint angle estimation by 33– 72%. Also, we tested the reliability of state estimation under camera occlusion. We found that when the system dynamics is tuned with offline UKF parameter estimation, the camera occlusion has no effect on the online state estimation.

IROS Conference 2015 Conference Paper

Improving position precision of a servo-controlled elastic cable driven surgical robot using Unscented Kalman Filter

Mohammad Haghighipanah
Yangming Li
Muneaki Miyasaka
Blake Hannaford

Cable driven power transmission is popular in many manipulator applications including medical arms. In spite of advantages obtained by removing motors from the mechanism, cable transmission introduces higher non-linearity and more uncertainties such as cable stretch and cable coupling. In order to improve the control precision and robustness of the Raven-II surgical robot, particularly for automation applications, the Unscented Kalman Filter (UKF) was adopted for state estimation. The UKF estimated state variables of the Raven-II dynamic model from sensor data. The dual UKF was used offline to estimate cable coupling parameters. The experimental results showed that the proposed method improved joint position estimation precision and the estimation consistency, especially on the more elastic links. The improvements for links 2 and 3 of the Raven were 36. 76%, and 62. 99%, respectively. For link 1 the improvement was 1. 43% because the transmission is very stiff.

IROS Conference 2015 Conference Paper

Measurement of the cable-pulley Coulomb and viscous friction for a cable-driven surgical robotic system

Muneaki Miyasaka
Joseph Matheson
Andrew Lewis 0001
Blake Hannaford

In this paper we present experimentally obtained cable-pulley Coulomb and viscous friction for cable-driven surgical robotic systems including the RAVEN II surgical robotic research platform. In the study of controlling cable-driven systems a simple mathematical model which does not capture physical behavior well is often employed. Even though control of such systems is achievable without an accurate model, fully understanding the behavior of the system will potentially realize more robust control. A surgical robot is one of the systems that often relies on cables as an actuation method as well as pulleys to guide them. Systems with such structure encounter frictional force related to conditions of cable and pulley such as cable velocity, tension, type and number of pulley, and angle of cable wrapping around pulley. Using a couple of test platforms that incorporate cable, pulleys, and other experimental conditions corresponding to the RAVEN II system, it is shown that cable-pulley friction is function of tension, wrap angle, and number of pulleys and not of magnitude of cable velocity.

IROS Conference 2015 Conference Paper

Path planning for semi-automated simulated robotic neurosurgery

Danying Hu
Yuanzheng Gong
Blake Hannaford
Eric J. Seibel

This paper considers the semi-automated robotic surgical procedure for removing the brain tumor margins, where the manual operation is a tedious and time-consuming task for surgeons. We present robust path planning methods for robotic ablation of tumor residues in various shapes, which are represented in point-clouds instead of analytical geometry. Along with the path plans, corresponding metrics are also delivered to the surgeon for selecting the optimal candidate in the automated robotic ablation. The selected path plan is then executed and tested on RAVEN™ II surgical robot platform as part of the semi-automated robotic brain tumor ablation surgery in a simulated tissue phantom.

ICRA Conference 2015 Conference Paper

Semi-autonomous simulated brain tumor ablation with RAVENII Surgical Robot using behavior tree

Danying Hu
Yuanzheng Gong
Blake Hannaford
Eric J. Seibel

Medical robots have been widely used to assist surgeons to carry out dexterous surgical tasks via various ways. Most of the tasks require surgeon's operation directly or indirectly. Certain level of autonomy in robotic surgery could not only free the surgeon from some tedious repetitive tasks, but also utilize the advantages of robot: high dexterity and accuracy. This paper presents a semi-autonomous neurosurgical procedure of brain tumor ablation using RAVEN Surgical Robot and stereo visual feedback. By integrating with the behavior tree framework, the whole surgical task is modeled flexibly and intelligently as nodes and leaves of a behavior tree. This paper provides three contributions mainly: (1) describing the brain tumor ablation as an ideal candidate for autonomous robotic surgery, (2) modeling and implementing the semi-autonomous surgical task using behavior tree framework, and (3) designing an experimental simulated ablation task for feasibility study and robot performance analysis.

ICRA Conference 2014 Conference Paper

Dynamically evaluated gravity compensation for the RAVEN surgical robot

Andrew Lewis 0001
Blake Hannaford

Using an accelerometer on the base of a robot, it is possible to calculate the torque required from each actuator in order to maintain a known pose regardless of base orientation with respect to the direction or magnitude of gravity. A simple and novel method has been developed and implemented for overcoming gravity induced torques on the RAVEN TM surgical research robot. This innovation will allow for accurate control of serial robot manipulators with re-orientable bases or for those operating in non-stationary environments such as boats, space stations, or moving vehicles.

IROS Conference 2014 Conference Paper

Experimental evaluation of guidance and forbidden region virtual fixtures for object telemanipulation

Hawkeye H. I. King
Blake Hannaford

Telerobotic task performance cannot compare to direct object manipulation with the hands. However, the computer in-the-loop offers the potential to give assistance to a human operator. The present work studies a class of computer assistance functions known as haptic virtual fixtures (VF). The objective is to use practical human trials to discover the ways VFs impact task execution.

ICRA Conference 2012 Conference Paper

Application of Unscented Kalman Filter to a cable driven surgical robot: A simulation study

Srikrishnan Ramadurai
Sina Nia Kosari
Hawkeye H. I. King
Howard Jay Chizeck
Blake Hannaford

Cable driven power transmissions are used in applications such as haptic devices, surgical robots etc. The use of flexible cable based power transmission often causes relative motion between the motor actuator and mechanism joint during operation due to the elasticity of the cable. State-space control methods can be used to improve performance, but may require state estimates. For nonlinear systems, the Unscented Kalman Filter (UKF) provides a computationally efficient way to obtain state estimates. The UKF is applied here to a simulation of a minimially invasive surgical robot, to study the state estimation for a cable driven system with nonlinear dynamics. State estimates from the UKF are compared with the known states available from the simulation. These state estimates are also utilized by two different controllers interacting with the simulation to test the UKF performance under closed loop control. We tested the UKF performance with error perturbations in the system model's cable stiffness parameter.

ICRA Conference 2012 Conference Paper

Robotic compression of soft tissue

Sina Nia Kosari
Srikrishnan Ramadurai
Howard Jay Chizeck
Blake Hannaford

This paper investigates automation of soft tissue compression for robot-assisted surgery. This is a fundamental task in surgery and includes interaction with a variety of tissues with unknown properties. In addition, due to sterilization and size constraints the use of contact force and position sensors are often avoided in surgical applications. We propose an Adaptive Model Predictive Control approach for execution of given tool trajectories in contact with unknown tissues in the absence of contact measurements. The Unscented Kalman Filter is employed in advance of system operation to identify the dynamics of a cable driven manipulator. These dynamics are then used to estimate contact force and position in free motion and in contact with tissue. An optimal control problem for automating tissue compression is formulated and is solved in real-time using Differential Dynamic Programming with Automatic Differentiation. The proposed methods are evaluated in experiments on an artificial tissue sample with unknown properties.

ICRA Conference 2010 Conference Paper

Plugfest 2009: Global interoperability in Telerobotics and telemedicine

Hawkeye H. I. King
Blake Hannaford
Ka-Wai Kwok
Guang-Zhong Yang
Paul G. Griffiths
Allison M. Okamura
Ildar Farkhatdinov
Jee-Hwan Ryu

Despite the great diversity of teleoperator designs and applications, their underlying control systems have many similarities. These similarities can be exploited to enable inter-operability between heterogeneous systems. We have developed a network data specification, the Interoperable Telerobotics Protocol, that can be used for Internet based control of a wide range of teleoperators. In this work we test interoperable telerobotics on the global Internet, focusing on the telesurgery application domain. Fourteen globally dispersed telerobotic master and slave systems were connected in thirty trials in one twenty four hour period. Users performed common manipulation tasks to demonstrate effective master-slave operation. With twenty eight (93%) successful, unique connections the results show a high potential for standardizing telerobotic operation. Furthermore, new paradigms for telesurgical operation and training are presented, including a networked surgery trainer and upper-limb exoskeleton control of micro-manipulators.

ICRA Conference 2009 Conference Paper

Bilateral teleoperation with time delay using modified wave variable based controller

Kenji Kawashima
Kotaro Tadano
Cong Wang
Ganesh Sankaranarayanan
Blake Hannaford

Force-reflecting teleoperators in which the remote environment is kinesthetically coupled to the operator can considerably increase task performance. Wave-variable-based controllers can support the stable operation of force-reflecting teleoperators under arbitrary communication delays. Transparency in such systems is compromised in order to maintain stability. We had previously proposed a modified wave variable controller that implemented additional wave impedance in the wave variable transformations in order to focus more closely on force tracking. In this paper, we present a new controller for bilateral teleoperators based on the modified wave variable control method which provides superior position and force tracking performance compared to the traditional wave-variable-based method. Moreover, the method has high stability. Theoretical investigation and experimental results confirm the performance of this new controller.

ICRA Conference 2009 Conference Paper

Effect of time delay on telesurgical performance

Mitchell J. H. Lum
Jacob Rosen 0001
Thomas S. Lendvay
Mika N. Sinanan
Blake Hannaford

In the area of surgical robotics no standard means of performance evaluation has been established. Thousands of surgeons have gone through the SAGES FLS Program, and the psychomotor skill portion of the program is considered the gold standard in laparoscopic skills evaluation. This research describes the use of the FLS block transfer task to evaluate the performance of both surgeons and non-surgeons teleoperating under different time delay conditions on the University of Washington RAVEN Surgical Robot. Time delays of 0 ms, 250 ms, and 500 ms were used and a statistically significant difference in mean block transfer time as well as mean tool tip path length were shown. For this task no significant difference was shown between the surgeon and non-surgeon groups. Clearly surgeon input and feedback is key to surgical robotic system development, but this result implies that non-surgeon subjects can be tested for simple usability evaluations.

ICRA Conference 2009 Conference Paper

Global transparency analysis of the Lawrence teleoperator architecture

Edvard Naerum
Blake Hannaford

Despite the frequent use of the Lawrence architecture since its introduction in the early 90's, its global transparency characteristics have not yet been fully analyzed. That is the goal of this paper. We state and prove necessary and sufficient conditions for transparency, with special attention to the information sent across the communication layer. In particular, it is shown that transparency can be preserved even though one, and even two, communication channels are set to zero. The results may serve as a guideline for transparent teleoperator design.

IROS Conference 2008 Conference Paper

Bilateral teleoperation with time delay using modified wave variables

Kenji Kawashima
Kotaro Tadano
Ganesh Sankaranarayanan
Blake Hannaford

Force-reflecting teleoperators in which the remote environment is kinesthetically coupled to the operator can considerably increase task performance. However, wave-variable-based controllers can support the stable operation of force-reflecting teleoperators under arbitrary communication delays. Transparency in such systems is compromised in order to maintain stability. In this paper, we present a new controller for bilateral teleoperators based on the wave variable control method which provides superior force tracking performance compared to the traditional wave-variable-based method. Additionally, this method also improves the phase delay induced by the latency in the communication network. Both simulation and experimental results confirm the performance of this new controller.

ICRA Conference 2008 Conference Paper

Experimental comparison of internet haptic collaboration with time-delay compensation techniques

Ganesh Sankaranarayanan
Blake Hannaford

In this paper we analyzed the performance of a peer-to-peer haptic collaboration system with two users jointly manipulating an object with mass and damping properties. We used objective measures to compare tuned PD, wave variables and time domain passivity controllers subject to real time delays from the Internet through similar experimental parameters. We set up a packet reflector network at our collaborators' servers in order to able to perform the experiment with subjects located in the same laboratory. Subjects were blinded to which controller was used and received them in a randomized sequence. UDP data packets were used for haptic data communication and the packet transmission rate was maintained at 1000 Hz. Our experimental results show that the tuned PD controller gave the best performance in terms of position error and wave variables in terms of force.

IROS Conference 2008 Conference Paper

Model-based passivity control for bilateral teleoperation of a surgical robot with time delay

Kenji Kawashima
Kotaro Tadano
Ganesh Sankaranarayanan
Blake Hannaford

In minimally invasive telesurgical systems, displaying the forces measured at the slave side is an important issue. Latency in communication lines limits the transmission of vivid tactile sensations and drives the system unstable. In this paper, we propose a new model-based approach for the bilateral control of a telesurgical robot using time-domain passivity control. This method consists of a virtual slave model implemented at the master side to estimate the force on the slave side. During the operation, the estimated force from the virtual slave is added to the actual measured force transmitted from the slave in order to maintain the passivity of the system. Both simulation and experimental results confirm the performance of this new controller.

ICRA Conference 2007 Conference Paper

Automated Tool Handling for the Trauma Pod Surgical Robot

Diana C. W. Friedman
Jesse Dosher
Tim Kowalewski
Jacob Rosen 0001
Blake Hannaford

In order to enable robotic surgery without human assistance, a means must be developed to change tools. As part of the larger Trauma Pod Project, we developed the Tool Rack Subsystem - an automated tool rack capable of holding, accepting, and dispensing up to 14 tools for the da Vinci™ surgical robot. Borrowing some techniques from industrial automation, we developed a robust system capable of presenting any stored tool in 700ms or less. Tools are positively retained in a sterilizable carousel in a compliant manner designed to accomodate misalignment during tool exchange. RFID equipment is integrated into the system and the tools so that tools can be inventoried and presented by function or serial number instead of rack position. The resulting device has completed testing and integration into the Trauma Pod system and met all its design requirements.

IROS Conference 2007 Conference Paper

Comparison of transient performance in the control of soft tissue grasping

Xiaolong Yu
Howard Jay Chizeck
Blake Hannaford

In robot-assisted surgery, surgical tools interact with tissues that have nonlinear mechanical properties. For situations where a pre-specified trajectory of tool positions (or applied forces) is desired, there are many controller designs that might be used. Four candidates are comparatively evaluated here, via computer simulation involving a nonlinear model of soft tissue behavior during grasping actions. The parameters for this model were obtained experimentally (in earlier work). The four candidate controllers are: (1) a well- tuned PID controller; (2) feedback linearization in combination with deadbeat control; (3) an optimal open-loop control law obtained via minimization of a quadratic cost function; and (4) a model predictive controller. Simulation trials are used to compare the transient performance of these candidate controllers under different assumptions regarding input and output noises. The conditions where each of the candidates is best are characterized.

ICRA Conference 2005 Conference Paper

A Simulation/Experimental Study of the Noisy Behavior of the Time Domain Passivity Controller for Haptic Interfaces

Jee-Hwan Ryu
Jong-Hwan Kim
Dong-Soo Kwon
Blake Hannaford

A noisy behavior of the time domain passivity controller during the period of low velocity is analyzed. Main reasons of the noisy behavior are investigated through a simulation with a one-DOF haptic interface model. It is shown that the PO/PC is ineffective in dissipating the produced energy when the sign of the velocity, which is numerically calculated from the measured position, is suddenly changed, and when this velocity is zero. These cases happen during the period of low velocity due to the limited resolution of the position sensor. New methods, ignoring the produced energy from the velocity sign change, and holding the control force while the velocity is zero, are proposed for removing the noisy behavior. The feasibility of the developed methods is proved with both a simulation and a real experiment.

ICRA Conference 2005 Conference Paper

Hill-Based Model as a Myoprocessor for a Neural Controlled Powered Exoskeleton Arm - Parameters Optimization

Ettore Cavallaro
Jacob Rosen 0001
Joel C. Perry
Stephen Burns
Blake Hannaford

The exoskeleton robot, serving as an assistive device worn by the human (orthotic), functions as a human-amplifier. Setting the human machine interface (HMI) at the neuro-muscular level may lead to seamless integration and an intuitive control of the exoskeleton arm as a natural extension of the human body. At the core of the exoskeleton HMI there is a myoprocessor. It is a model of the human muscle, running in real-time and in parallel to the physiological muscle, that predicts joint torque as a function of the joint kinematics and neural activation levels. The study is focused on developing a myoprocessor based on the Hill phenomenological muscle model. Genetic algorithms were used to optimize model internal parameters using an experimental database that provides inputs to the model and allows for performance assessment. The results indicate high correlation between joint moment predictions of the model and the measured data. Consequently, the myoprocessor seems an adequate model, sufficiently robust for further integration into the exoskeleton control system.

ICRA Conference 2004 Conference Paper

Kinematic Optimization of a Spherical Mechanism for a Minimally Invasive Surgical Robot

Mitchell J. H. Lum
Jacob Rosen 0001
Mika N. Sinanan
Blake Hannaford

Advances in surgical technology allow physicians to more effectively provide care to their patients. Minimally invasive surgery (MIS) has revolutionized the way a significant number of procedures are performed. Advances in technology have led to the fusion of MIS techniques and robotic devices; however, such systems are currently large and cumbersome. By optimizing a spherical mechanism based on in-vivo data collected during MIS procedures, this paper focuses on a bottom-up approach in developing a new class of surgical robot arms. The spherical mechanism is a rotational manipulator with all axes intersecting at the center of the sphere. Locating the rotational center of the mechanism at the MIS port makes this class of mechanism a suitable candidate for the first two links of a surgical robot for both minimally invasive and open surgery. For optimizing the mechanism structure, the forward and inverse kinematics, as well as the Jacobian matrix, were derived. Using the Jacobian, mechanism isotropy was considered as the performance metric. The dexterous workspace (DWS) is defined as a high dexterity region defined by a right circular cone with a vertex angel of 60/spl deg/ in which 95% of the tool motions are contained based on in-vivo measurements. The extended dexterous workspace (EDWS) is defined as the workspace required to reach the entire abdominal cavity with MIS instruments and defined by a cone with an elliptical cross section created by two orthogonal vertex angels of 60/spl deg/ and 90/spl deg/. Optimization across both the DWS and a superset of the EDWS led to a mechanism configuration with link length angles of 74/spl deg/ and 60/spl deg/ that maximizes kinematic performance and compactness. The workspace of this design covers the entire EDWS and is the optimal design for the next generation of surgical manipulator. By directly applying in-vivo experimental data from MIS in order to optimize the spherical manipulator a design that maximizes performance and minimizes size has been developed. A pair of prototype manipulators is developed based on these results.

IROS Conference 2003 Conference Paper

Anisotropies of touch in haptic icon exploration

Gregory S. Lee
Blake Hannaford

Handheld devices are enhancing many aspects of our lives. As increasingly complex devices appear with decreasing form factors, haptics may become an essential tool for interacting with them. In this regime of operation, issues of power, weight and volume are of significant importance. The haptic thresholds of the index finger for active exploration of a two dimensional virtual environment for two icon alignments and two finger motions were measured. Using all possible combinations of two finger motions, flexion/extension and finger abduction/adduction, and two icon alignments, vertical and horizontal, were measured separately. Haptic thresholds ranged from 15 to 24 milliNewtons. Thresholds were affected by finger motion, but not by icon alignment.

ICRA Conference 2003 Conference Paper

Sampled and continuous time passivity and stability of virtual environments

Jee-Hwan Ryu
Yoon Sang Kim
Blake Hannaford

We propose new time domain passivity observer (PO) and passivity controller (PC) which removes the constant velocity assumption during one sample time, which was used in our previous PO/PC approach. A new sampled time definition of passivity is introduced, and this new definition is compared with the previous sampled time definition of passivity. Through this comparison, we propose the more accurate PO/PC approach. The proposed new PO/PC approach is applied to "Excalibur" haptic interface system with very high stiffness (K = 120 KN/m) virtual environment (VE), and stable contact is demonstrated.

IROS Conference 2003 Conference Paper

Time domain passivity control for 6 degrees of freedom haptic displays

Carsten Preusche
Gerhard Hirzinger
Jee-Hwan Ryu
Blake Hannaford

In this paper a modification of the time domain passivity controller is presented to improve its performance and transparency in case of multi degrees of freedom (dof) haptic interaction. In multi-dof application the concept needs to be extended by additional conditions to distribute the adaptive damping appropriately among the degrees of freedom. This can be solved by using the geometrical information coded in the output signals of the system. Experiments show the validity of this concept.

IROS Conference 2003 Conference Paper

Time domain passivity control with reference energy behavior

Jee-Hwan Ryu
Blake Hannaford
Carsten Preusche
Gerhard Hirzinger

A recently proposed method for stabilizing haptic interfaces and teleoperation systems was tested with a "PHANToM" commercial haptic device. The "passivity observer" (PO) and "passivity controller" (PC) stabilization method was formed to stabilize the system but also excite high frequency mode in the device. To solve this problem, we propose a method to use a time-varying desired energy threshold instead of fixed zero energy threshold for the PO, and make the actual energy input follow the time-varying energy threshold. With the time-varying energy threshold, we make the PC control action smooth without sudden impulsive behavior by distributing the dissipation. The proposed new PO/PC approach is applied to PHANToM with high stiffness (K=500 N/m), and stable and smooth contact is guarantee. Resetting and active environment display problems can also be solved with the reference energy following idea.

IROS Conference 2002 Conference Paper

Stability guaranteed control: Time domain passivity approach

Jee-Hwan Ryu
Dong-Soo Kwon
Blake Hannaford

A new, energy-based method is proposed for guaranteeing the stability of large classes of control systems with minimum performance losses. Based on a network presentation, the large classes of control systems are analyzed in a unified framework. In this unified network model, the concept of passivity is used to study the stability of large classes of control systems. For guaranteeing the stability condition, the time-domain passivity controller is extended to a 2-port network to make the controller 2-port passive. The developed method is tested with numerical simulation in the regulation of a single link flexible manipulator. Totally stable control is achieved under a wide variety of operating conditions and uncertainties without any model information.

ICRA Conference 2002 Conference Paper

Stable Teleoperation with Time Domain Passivity Control

Jee-Hwan Ryu
Dong-Soo Kwon
Blake Hannaford

A new bilateral control scheme is proposed to ensure stable teleoperation under a wide variety of operating conditions. To guarantee the stability of a teleoperation system, a previously proposed energy-based method is extended to a two-port network. The issues in implementing the "passivity observer" and "passivity controller" to teleoperation systems are studied. The method is tested with our two-DOF master/slave teleoperation system. Totally stable teleoperation is achieved under conditions such as hard wall contact (stiffness>150 kN/m) and hard surface following.

ICRA Conference 2002 Conference Paper

The BlueDRAGON - A System for Measuring the Kinematics and the Dynamics of Minimally Invasive Surgical Tools In-Vivo

Jacob Rosen 0001
Jeffrey D. Brown
Lily Chang
Marco Barreca
Mika N. Sinanan
Blake Hannaford

Minimally invasive surgery involves a multidimensional series of tasks requiring a synthesis between visual information and the kinematics and dynamics of the surgical tools. Analysis of these sources of information is a key step in mastering MIS, but may also be used to define objective criteria for characterizing surgical performance. The BlueDRAGON is a new system for acquiring the kinematics and dynamics of two endoscopic tools synchronized with the visual view of the surgical scene. It includes two four-bar passive mechanisms equipped with position and force torque sensors for measuring the positions and orientations of the two endoscopic tools along with the forces and torques (F/T) applied by the surgeon's hands. The methodology of decomposing the surgical task is based on a fully connected, 28 finite-states Markov model where each states corresponded to a fundamental tool/tissue interaction based on the tool kinematics and associated with unique F/T signatures. The experimental protocol includes seven MIS tasks performed on an animal model by 30 surgeons at different levels of their residency training including expert surgeons. From the preliminary analysis of these data, the major differences between residents at different skill levels are discussed. Systems like surgical robots or virtual reality simulators that inherently measure the kinematics and dynamics of the surgical tool may benefit from inclusion of the proposed methodology for the analysis of efficacy and objective evaluation of surgical skills during training.

IROS Conference 2001 Conference Paper

Some practical issues in time domain passivity control of haptic interfaces

Yoon Sang Kim
Blake Hannaford

In this paper, two major practical issues are studied to improve the performance of a new energy based method of achieving stable, high performance haptic interface control. The first issue is related to resetting the amount of energy accumulated in the passivity observer for faster operation. A heuristic method is derived and experimentally tested for the resetting and it is shown to help the PC to operate sooner when the system gets active. The second one was noise in velocity measurements being magnified into audible force signals by the controller. This issue was addressed by the introduction of a velocity threshold and it is verified that the velocity threshold makes the PC more free from the noise effect at low velocity. Experimental results are presented for the "Excalibur" haptic device.

ICRA Conference 2001 Conference Paper

Time Domain Passivity Control of Haptic Interface

Blake Hannaford
Jee-Hwan Ryu

An energy-based method is presented for controlling a haptic interface system to ensure stable contact under a wide variety of operating conditions. System stability is analyzed in terms of the time-domain definition of passivity. We define a "passivity observer" (PO) which measures energy flow in and out of one or more subsystems in real-time software. Active behavior is indicated by a negative value of the PO at any time. We also define the "passivity controller" (PC), an adaptive dissipative element which, at each time sample, absorbs exactly the net energy output (if any) measured by the PO. The method is tested with simulation and implementation in the "Excalibur" haptic interface system. Totally stable operation was achieved under conditions such as stiffness >100 N/mm or time delays of 15 ms. The PO/PC method requires very little additional computation and does not require a dynamical model to be identified.

IROS Conference 2000 Conference Paper

Development of a biomimetic position sensor for robotic kinaesthesia

Kristen N. Jaax
Pierre-Henry Marbot
Blake Hannaford

This paper presents a biomimetic sensor for transducing displacements. Our sensor is a robotic analog of the biological muscle spindle, an actuated position sensor which transduces muscle displacement for kinaesthetic awareness. The mechanical filter exhibits the desired step response. The transducer possesses the desired linear response with a sensitivity of 34nm/Hz. Finally, the encoder circuitry successfully maps the millivolt output to a pulse frequency range of 1150 Hz to 12. 5 kHz. Results from an integrated system test show that the sensor can successfully detect errors in trajectory tracking introduced by both phase lag and perturbations. By physically realizing the hypothesized core features of a biological muscle spindle in engineering hardware, we evoked the type of actuated sensor output seen in the biological muscle spindle, a widely utilized tool of biological motor control.

ICRA Conference 2000 Conference Paper

Stable Haptic Interaction Using the Excalibur Force Display

Richard J. Adams
Daniel Klowden
Blake Hannaford

Creating a compelling haptic sense of immersion in a virtual environment is a challenging task for the control engineer. A haptic display must render both low impedance free-space motion and high impedance rigid constraints while ensuring stable interaction. This paper outlines a control design approach for the most common haptic display implementation, the impedance display. Two-port absolute stability criteria are used to develop explicit design bounds for virtual coupling networks which guarantee system stability for a broad class of human operators and virtual environments. The technique is applied to the Excalibur three-axis force display. The resulting absolutely stable haptic interface is the centerpiece of a virtual building block simulation which emulates the behavior of LEGO/sup TM/ bricks in a virtual environment.

ICRA Conference 1999 Conference Paper

4-Axis Electromagnetic Microgripper

Arianna Menciassi
Blake Hannaford
Maria Chiara Carrozza
Paolo Dario

This paper describes a novel 4-axis microgripping system consisting of two fingers, each driven by a 2-axis moving coil actuator taken from a CD-lens assembly. These electromagnetic actuators are small, very linear, virtually frictionless and low cost. We measured the electrical actuator parameters and characterized the actuator performance in terms of displacement vs. current, force vs. current and resonant frequency. Experimental results indicate that the proposed microgripping system can be an attractive solution to the problem of micromanipulating small objects for precision manufacturing and biotechnology with high accuracy in a relatively large workspace.

ICRA Conference 1998 Conference Paper

A Practical Measure of Dynamic Response of Haptic Devices

Manuel Moreyra
Blake Hannaford

A method is described to characterize and experimentally measure the dynamic performance of haptic display devices. The method characterizes the response to impulse inputs of various frequencies characteristic of simulating hard contacts in virtual environments. By comparing the experimentally measured velocity just after the impulse with the actual velocity, a dimensionless measure of structural distortion is derived. The method is easy to apply because no additional sensors or test fixtures are required. This paper presents a derivation of the structural deformation ratio for the single degree of freedom case, generalization to N-DOF spatial devices, and experimental results for a single axis of a rugged haptic device in our laboratory.

IROS Conference 1998 Conference Paper

A two-port framework for the design of unconditionally stable haptic interfaces

Richard J. Adams
Blake Hannaford

A haptic interface is a kinesthetic link between a human operator and a virtual environment. This paper addresses stability and performance issues associated with haptic interaction. It generalizes and extends the concept of a virtual coupling network, an artificial connection between a haptic display and a virtual world, to include both the impedance and admittance models of haptic interaction. A benchmark example exposes an important duality between these two cases. Linear circuit theory is used to develop necessary and sufficient conditions for the stability of a haptic simulation, assuming the human operator and virtual environment are passive. These equations lead to an explicit design procedure for virtual coupling networks which give maximum performance while guaranteeing stability. By decoupling the haptic display control problem from the design of virtual environments, the use of a virtual coupling network frees the developer of haptic-enabled virtual reality models from issues of mechanical stability.

IROS Conference 1998 Conference Paper

Fatigue characteristics of McKibben artificial muscle actuators

Glenn K. Klute
Blake Hannaford

The McKibben artificial muscle is a pneumatic actuator whose properties include a very high force to weight ratio. This characteristic makes it very attractive for a wide range of applications such as mobile robots and prosthetic appliances for the disabled. Typical applications often require a significant number of repeated contractions and extensions or cycles of the actuator. This repeated action leads to fatigue and failure of the actuator, yielding a life span that is often shorter than its more common robotic counterparts such as electric motors or pneumatic cylinders. In this paper, we develop a model that predicts the maximum number of life cycles of the actuator based on available uniaxial tensile properties of the actuator's inner bladder. Experimental results, which validate the model, reveal McKibben actuators fabricated with natural latex rubber bladders have a fatigue limit 24 times greater than actuators fabricated with synthetic silicone rubber at large contraction ratios.

IROS Conference 1998 Conference Paper

Haptic feedback of kinematic conditioning for telerobotic applications

Thavida Maneewarn
Blake Hannaford

Kinematic conditioning of robot manipulators is the problem where small motions in Cartesian space cause excessive joint velocities. This problem is significant in teleoperation. Haptic feedback provides the bi-directional flow of information which allows the operator to control the telerobot interactively. Haptic feedback of kinematic conditioning is proposed as a new approach to achieve better performance in telerobotic control near kinematic singularities. Four different singularity force feedback methods are defined and studied. Experimental results with a force feedback master and simulated slave system show that teleoperation performance near singular configurations was affected and improved by using singularity force feedback.

ICRA Conference 1997 Conference Paper

Telerobotic remote handling of protein crystals

Blake Hannaford
James Hewitt
Thavida Maneewarn
Steven C. Venema
Matthew Appleby
Robert Ehresman

A combined university/industry team has developed a prototype system for handling protein crystals aboard the space station. This system uses a miniature direct drive robot, CCD television cameras, and a client-server computing system using Internet protocols to support the capture of protein crystals from aqueous growth solutions. The system was demonstrated between Huntsville AL. and Seattle WA. An operator in Huntsville controlled the mini robot by invoking predefined relative and absolute macro files. The operators observed results using video images sent through the Internet link using Cu-SeeMe video conferencing software. In 3 of 4 trials, the operators successfully captured 0. 5 mm simulated protein crystals into a glass capillary. The system is a promising start for the development of a space-station based remote protein crystal analysis facility.

IROS Conference 1995 Conference Paper

Kalman filter based calibration of precision motion control

Steven C. Venema
Blake Hannaford

A method is described and validated for the automatic calibration of analog sine-wave quadrature sensors, such as optical encoders, embedded in a functioning system. The algorithm uses a Kalman filter to estimate the true position of the direct-drive actuator joint using a model of it's dynamics, an applied actuator command and measurements from the uncalibrated sensor. From the estimated true position, a lookup table is constructed which corrects sensor errors. Our results indicate that this method achieves accuracies typical of interferometric calibration, without requiring an external measurement device. The accuracy is surprisingly robust to modeling errors.

IROS Conference 1994 Conference Paper

A 5-axis mini direct drive robot for time delayed teleoperation

Blake Hannaford
Pierre-Henry Marbot
Manuel Moreyra
Steven C. Venema

A previously developed 3 axis mini direct drive robot has been enhanced with two additional direct drive axes for general positioning and orientation of an axially symmetric tool. The arm has a work volume of about 50 cc and will have 5-10 micron or better resolution and repeatability. The arm forms an initial prototype for the NASA/University of Washington MicroTrex flight telerobotics experiment. The contemplated terrestrial applications include handling sub-microliter liquid samples for electrophoresis, and micro-manipulation with scaled force reflection. >

ICRA Conference 1994 Conference Paper

Static and Dynamic Characteristics of McKibben Pneumatic Artificial Muscles

Ching-Ping Chou
Blake Hannaford

This paper reports mechanical testing and modeling results for the McKibben artificial muscle pneumatic actuator. This device first developed in the 1950's, contains an expanding tube surrounded by braided cords. The authors report static and dynamic length-tension testing results and derive a linearized model of these properties for three different models. The results are briefly compared with human muscle properties to evaluate the suitability of McKibben actuators for human muscle emulation in biologically based robot arms. >

IROS Conference 1991 Conference Paper

Hidden Markov model analysis of manufacturing process information

Blake Hannaford

A method is presented for using hidden Markov models (HMMs) for the analysis of force, torque, and position signals from sensors in manufacturing machines. The HMM can detect the transitions between contact states and compute a measure of the task quality using a model of the task developed by the manufacturing engineer and optimized on training data. The HMM method has been evaluated in extensive experimentation with teleoperation and the results suggest even higher effectiveness in automation and manufacturing applications.

ICRA Conference 1989 Conference Paper

Stability and performance tradeoffs in bi-lateral telemanipulation

Blake Hannaford

Kinesthetic force feedback provides measurable increase in remote manipulation system performance. Intensive computation time requirements or operation under conditions of time delay can cause serious stability problems in control-system design. The author presents a simplified linear analysis of this stability problem for the forward-flow generalized architecture and uses the hybrid two-port representation to express the loop gain of the traditional master-slave architecture, which can be subjected to similar analysis. The hybrid two-port representation is also used to express the effects on the fidelity of manipulation or feel of one design approach used to stabilize the forward-flow architecture. The results suggest that, when local force feedback at the slave side is used to reduce manipulator stability problems, a price is paid in terms of telemanipulation fidelity. >

ICRA Conference 1988 Conference Paper

Experimental and simulation studies of hard contact in force reflecting teleoperation

Blake Hannaford
Robert Anderson

Experiments and simulations of a single-axis force-reflecting teleoperation system have been conducted to investigate the problem of contacting a hard environment and maintaining a controlled force in teleoperation in which position is fed forward from the hand controller (master) to the manipulator (slave), and force is fed back to the human operator through motors in the master. The simulations, using an electrical circuit model, reproduce the behavior of the real system, including effects of human operator biomechanics. It is shown that human operator properties, which vary as a result of different types of grasp of the handle, affect the stability of the system in the hard-contact task. The effect of a heavier grasp on the handle is equivalent to increased hand-controlled velocity damping in terms of the systems stability in the contact task, but control system damping sufficient to guarantee stable contact results in perceptible sluggishness of the control handle's response in free motion. These results suggest that human operator biomechanics must be taken into account to guarantee stable and ergonomic performance of advanced teleoperators. >

ICRA Conference 1987 Conference Paper

Hand trigger system for bi-lateral gripping control in teleoperation

Paolo Fiorini
Blake Hannaford
Bruno Jau
Edwin Kan
Antal K. Bejczy

A new device for human operator control of a robotic gripper has been developed and preliminary evaluation has been performed. The JPL Force Reflecting Hand Trigger system features: an instrumented index finger trigger with load cell detection of finger force. A servo controlled, lead screw driven backdrive capability by which the trigger's position can be made to follow that of the remotely controlled gripper. And a novel feedback mechanism by which clamping force or some other signal can be fed back via a swiveling motion, also servo controlled, of the trigger surface (force reflection). This system has undergone preliminary testing in which the amount of force reflection is varied and dynamic force tracking response is observed.