Gesture-Based Interaction with a Pet Robot

Pet robots are autonomous robots capable of exhibiting animal-like behaviors, including emotional ones, as they interact with people and objects surrounding them. As pet robots become moreintegrated into our lives, a morenatural way of communicatingwith themwill becomenecessary. Similarly, they will need to understandhumangestures in order to perceive our intentions and communicate with us moreeffectively. In this paper, wepresent an extensible, real-time, visionbased communicationsystem that interprets 2D dynamichand gestures in complexenvironments. Ourstrategy for interpreting hand gestures consists of: hand segmentation, feature extraction, and gesture recognition. To segment the hand from the cluttered background, this system uses both motion and color information. Thelocation of the hand is subsequently tracked as the user makesthe gesture and its trajectory information is stored in a feature vector. Finally, the gestureis interpreted using this vector andtranslated into a command that the robot understands. Weimplemented our system on Yuppy, a pet robot prototype. Currently, via an external microcamera, we can navigate Yuppyin unstructured environments using hand gestures.

Authors

• Milyn C. Moy
• MIT Artificial Intelligence Laboratory / Oracle Corporation

Keywords

No keywords are indexed for this paper.