TMLR Journal 2022 Journal Article
Benchmarking Progress to Infant-Level Physical Reasoning in AI
- Luca Weihs
- Amanda Yuile
- Renée Baillargeon
- Cynthia Fisher
- Gary Marcus
- Roozbeh Mottaghi
- Aniruddha Kembhavi
To what extent do modern AI systems comprehend the physical world? We introduce the open-access Infant-Level Physical Reasoning Benchmark (InfLevel) to gain insight into this question. We evaluate ten neural-network architectures developed for video understanding on tasks designed to test these models' ability to reason about three essential physical principles which researchers have shown to guide human infants' physical understanding. We explore the sensitivity of each AI system to the continuity of objects as they travel through space and time, to the solidity of objects, and to gravity. We find strikingly consistent results across 60 experiments with multiple systems, training regimes, and evaluation metrics: current popular visual-understanding systems are at or near chance on all three principles of physical reasoning. We close by suggesting some potential ways forward.