Catching a Baseball: A Reinforcement Learning Perspective Using a Neural Network

Moments after a baseball batter has hit a fly ball, an outfielder has to decide whether to run forward or backward to catch the ball. Judging a fly ball is a difficult task, especially when the fielder is in the plane of the ball’ s trajectory. There exists several alternative hypotheses in the literature which identify different perceptual features available to the fielder that may provide useful cues as to the location of the ball’ s landing point. A recent study in experimental psychology suggests that to intercept the ball, the fielder has to run such that the double derivative of tan4 with respect to time is close to zero (i. e. d2(tanq5)/dt2 z 0), where q5is the elevation angle of the ball from the fielder’ s perspective (McLeod & Dlenes 1993). We investigate whether cl2 (tang5)/dt 2 information is a useful cue to learn this task in the Adaptive Heuristic Critic (d7iC) reinforcement learning framework. Our results provide supporting evidence that d2(tanq5)/dt2 information furnishes strong initial cue in determining the landing point of the ball and plays a key role in the learning process. However our simulations show that during later stages of the ball’ s flight, yet another perceptual feature, the perpendicular velocity of the ball (vp) with respect to the fielder, provides stronger cues as to the location of the landing point. The trained network generalized to novel circumstances and also exhibited some of the behaviors recorded by experimental psychologists on human data. We believe that much can be gained by using reinforcement learning approaches to learn common physical tasks, and similarly motivated work could stimulate useful interdisciplinary research on the subject.

Authors

• Rajarshi Das Jeffrey O. Kephart

Keywords

No keywords are indexed for this paper.