Functionality Discovery and Prediction of Physical Objects

Functionality is a fundamental attribute of an object which indicates the capability to be used to perform specific actions. It is critical to empower robots the functionality knowledge in discovering appropriate objects for a task e. g. cut cake using knife. Existing research works have focused on understanding object functionality through human-object-interaction from extensively annotated image or video data and are hard to scale up. In this paper, we (1) mine object-functionality knowledge through pattern-based and model-based methods from text, (2) introduce a novel task on physical objectfunctionality prediction, which consumes an image and an action query to predict whether the object in the image can perform the action, and (3) propose a method to leverage the mined functionality knowledge for the new task. Our experimental results show the effectiveness of our methods.

Authors

• Lei Ji Microsoft Research Asia
• Botian Shi Shanghai AI Lab
• Xianglin Guo
• Xilin Chen Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences University of the Chinese Academy of Sciences

Keywords

No keywords are indexed for this paper.