Jonathan Rowe

AAAI Conference 2020 Conference Paper

Predictive Student Modeling in Educational Games with Multi-Task Learning

• Michael Geden
• Andrew Emerson
• Jonathan Rowe
• Roger Azevedo
• James Lester

Modeling student knowledge is critical in adaptive learning environments. Predictive student modeling enables formative assessment of student knowledge and skills, and it drives personalized support to create learning experiences that are both effective and engaging. Traditional approaches to predictive student modeling utilize features extracted from VWXGHQWV¶ LQWHUDFWLRQ WUDFH GDWD WR SUHGLFW VWXGHQW WHVW performance, aggregating student test performance as a single output label. We reformulate predictive student modeling as a multi-task learning problem, modeling questions from student WHVW GDWD DV GLVWLQFW ³WDVNV ´: H GHPRQVWUDWH WKH HIIHFWLYHQHVV of this approach by utilizing student data from a series of laboratory-based and classroom-based studies conducted with a game-based learning environment for microbiology education, CRYSTAL ISLAND. Using sequential representations of student gameplay, results show that multi-task stacked LSTMs with residual connections significantly outperform baseline models that do not use the multi-task formulation. Additionally, the accuracy of predictive student models is improved as the number of tasks increases. These findings have significant implications for the design and development of predictive student models in adaptive learning environments.

IJCAI Conference 2018 Conference Paper

High-Fidelity Simulated Players for Interactive Narrative Planning

• Pengcheng Wang
• Jonathan Rowe
• Wookhee Min
• Bradford Mott
• James Lester

Interactive narrative planning offers significant potential for creating adaptive gameplay experiences. While data-driven techniques have been devised that utilize player interaction data to induce policies for interactive narrative planners, they require enormously large gameplay datasets. A promising approach to addressing this challenge is creating simulated players whose behaviors closely approximate those of human players. In this paper, we propose a novel approach to generating high-fidelity simulated players based on deep recurrent highway networks and deep convolutional networks. Empirical results demonstrate that the proposed models significantly outperform the prior state-of-the-art in generating high-fidelity simulated player models that accurately imitate human players’ narrative interactions. Using the high-fidelity simulated player models, we show the advantage of more exploratory reinforcement learning methods for deriving generalizable narrative adaptation policies.

IJCAI Conference 2017 Conference Paper

Interactive Narrative Personalization with Deep Reinforcement Learning

• Pengcheng Wang
• Jonathan Rowe
• Wookhee Min
• Bradford Mott
• James Lester

Data-driven techniques for interactive narrative generation are the subject of growing interest. Reinforcement learning (RL) offers significant potential for devising data-driven interactive narrative generators that tailor players’ story experiences by inducing policies from player interaction logs. A key open question in RL-based interactive narrative generation is how to model complex player interaction patterns to learn effective policies. In this paper we present a deep RL-based interactive narrative generation framework that leverages synthetic data produced by a bipartite simulated player model. Specifically, the framework involves training a set of Q-networks to control adaptable narrative event sequences with long short-term memory network-based simulated players. We investigate the deep RL framework’s performance with an educational interactive narrative, Crystal Island. Results suggest that the deep RL-based narrative generation framework yields effective personalized interactive narratives.

IJCAI Conference 2016 Conference Paper

Player Goal Recognition in Open-World Digital Games with Long Short-Term Memory Networks

• Wookhee Min
• Bradford Mott
• Jonathan Rowe
• Barry Liu
• James Lester

Recent years have seen a growing interest in player modeling for digital games. Goal recognition, which aims to accurately recognize players' goals from observations of low-level player actions, is a key problem in player modeling. However, player goal recognition poses significant challenges because of the inherent complexity and uncertainty pervading gameplay. In this paper, we formulate player goal recognition as a sequence labeling task and introduce a goal recognition framework based on long short-term memory (LSTM) networks. Results show that LSTM-based goal recognition is significantly more accurate than previous state-of-the-art methods, including n-gram encoded feedforward neural networks pre-trained with stacked denoising autoencoders, as well as Markov logic network-based models. Because of increased goal recognition accuracy and the elimination of labor-intensive feature engineering, LSTM-based goal recognition provides an effective solution to a central problem in player modeling for open-world digital games.

TCS Journal 2014 Journal Article

Genetic and Evolutionary Computation

• Tobias Friedrich
• Jonathan Rowe

AAAI Conference 2012 Conference Paper

Goal Recognition with Markov Logic Networks for Player-Adaptive Games

• Eun Ha
• Jonathan Rowe
• Bradford Mott
• James Lester

Goal recognition in digital games involves inferring players’ goals from observed sequences of low level player actions. Goal recognition models support player adaptive digital games, which dynamically augment game events in response to player choices for a range of applications, including entertainment, training, and education. However, digital games pose significant challenges for goal recognition, such as exploratory actions and ill defined goals. This paper presents a goal recognition framework based on Markov logic networks (MLNs). The model’s parameters are directly learned from a corpus that was collected from player interactions with a non linear educational game. An empirical evaluation demonstrates that the MLN goal recognition framework accurately predicts players’ goals in a game environment with exploratory actions and ill defined goals.

TCS Journal 2006 Journal Article

Some results about the Markov chains associated to GPs and general EAs

• Boris Mitavskiy
• Jonathan Rowe

Geiringer's theorem is a statement which tells us something about the limiting frequency of occurrence of a certain individual when a classical genetic algorithm is executed in the absence of selection and mutation. Recently Poli, Stephens, Wright and Rowe extended the original theorem of Geiringer to include the case of variable-length genetic algorithms and linear genetic programming. In the current paper a rather powerful finite population version of Geiringer's theorem which has been established recently by Mitavskiy is used to derive a schema-based version of the theorem for nonlinear genetic programming with homologous crossover. The theorem also applies in the presence of “node mutation”. The corresponding formula in case when “node mutation” is present has been established. The limitation of the finite population Geiringer result is that it applies only in the absence of selection. In the current paper we also observe some general inequalities concerning the stationary distribution of the Markov chain associated to an evolutionary algorithm in which selection is the last (output) stage of a cycle. Moreover we prove an “anti-communism” theorem which applies to a wide class of EAs and says that for small enough mutation rate, the stationary distribution of the Markov chain modelling the EA cannot be uniform.

TCS Journal 2006 Journal Article

Subthreshold-seeking local search

• Darrell Whitley
• Jonathan Rowe

Algorithms for parameter optimization display subthreshold-seeking behavior when the majority of the points that the algorithm samples have an evaluation less than some target threshold. We first analyze a simple “subthreshold-seeker” algorithm. Further theoretical analysis details conditions that allow subthreshold-seeking behavior for local search algorithms using Binary and Gray code representations. The analysis also shows that subthreshold-seeking behavior can be increased by using higher bit precision. However, higher precision also can reduce exploration. A simple modification to a bit-climber is proposed that improves its subthreshold-seeking behavior. Experiments show that this modification results in both improved search efficiency and effectiveness on common benchmark problems.