Jack Mostow

AAAI Conference 2025 Short Paper

Using Next Sentence Prediction to Test ChatGPT’s Text Comprehension (Student Abstract)

• Ojas M Agarwal
• Madelein Villegas
• Jack Mostow

We propose the Next Sentence Prediction (NSP) task as a simple, objective, scalable, automated way to test ChatGPT’s text comprehension. Given a context excerpted from a children’s story, the task is to distinguish the next story sentence from a later sentence in the story. We analyze how ChatGPT’s performance on this task is related to various features of the text, using data from English and Swahili children’s stories.

AAAI Conference 2021 Short Paper

Early Prediction of Children’s Task Completion in a Tablet Tutor using Visual Features (Student Abstract)

• Bikram Boote
• Mansi Agarwal
• Jack Mostow

Intelligent tutoring systems could benefit from human teachers’ ability to monitor students’ affective states by watching them and thereby detecting early warning signs of disengagement in time to prevent it. Toward that goal, this paper describes a method that uses input from a tablet tutor’s userfacing camera to predict whether the student will complete the current activity or disengage from it. Training a disengagement predictor is useful not only in itself but also in identifying visual indicators of negative affective states even when they don’t lead to non-completion of the task. Unlike prior work that relied on tutor-specific features, the method relies solely on visual features and so could potentially apply to other tutors. We present a deep learning method to make such predictions based on a Long Short Term Memory (LSTM) model that uses a target replication loss function. We train and test the model on screen capture videos of children in Tanzania using a tablet tutor to learn basic Swahili literacy and numeracy. We achieve balanced-class-size prediction accuracy of 73. 3% when 40% of the activity is still left.

AAAI Conference 2019 Short Paper

What’s Most Broken? A Tool to Assist Data-Driven Iterative Improvement of an Intelligent Tutoring System

• Mononito Goswami
• Shiven Mian
• Jack Mostow

Intelligent Tutoring Systems (ITS) have great potential to change the educational landscape by bringing scientifically tested one-to-one tutoring to remote and under-served areas. However, effective ITSs are too complex to perfect. Instead, a practical guiding principle for ITS development and improvement is to fix what’s most broken. In this paper we present SPOT (Statistical Probe of Tutoring): a tool that mines data logged by an Intelligent Tutoring System to identify the ‘hot spots’ most detrimental to its efficiency and effectiveness in terms of its software reliability, usability, task difficulty, student engagement, and other criteria. SPOT uses heuristics and machine learning to discover, characterize, and prioritize such hot spots in order to focus ITS refinement on what matters most. We applied SPOT to data logged by RoboTutor, an ITS that teaches children basic reading, writing and arithmetic.

AAAI Conference 2018 Short Paper

Relating Children’s Automatically Detected Facial Expressions to Their Behavior in RoboTutor

• Mayank Saxena
• Rohith Pillai
• Jack Mostow

Can student behavior be anticipated in real-time so that an intelligent tutor system can adapt its content to keep the student engaged? Current methods detect affective states of students during learning session to determine their engagement levels, but apply the learning in next session in the form of intervention policies and tutor responses. However, if students’ imminent behavioral action could be anticipated from their affective states in real-time, this could lead to much more responsive intervention policies by the tutor and assist in keeping the student engaged in an activity, thereby increasing tutor efficacy as well as student engagement levels. In this paper we explore if there exist any links between a student’s affective states and his/her imminent behavior action in RoboTutor, an intelligent tutor system for children to learn math, reading and writing. We then exploit our findings to develop a real-time student behavior prediction module.

AAAI Conference 1999 Conference Paper

Authoring New Material in a Reading Tutor that Listens

• Jack Mostow
• Gregory Aist
• Carnegie Mellon University

Project LISTEN’s Reading Tutor helps children learn to read by providing assisted practice in reading connected text. A key goal is to provide assistance for reading any English text entered by students or adults. This live demonstration shows how the Reading Tutor helps users enter and narrate stories, and then helps children read them. Areas: intelligent interfaces, computer-aided instruction, dialog, speech recognition.

AAAI Conference 1997 Conference Paper

The Sounds of Silence: Towards Automated Evaluation of Student Learning in a Reading Tutor that Listens

• Jack Mostow

We propose a paradigm for ecologically valid, authentic, unobtrusive, automatic, data-rich, fast, robust, and sensitive evaluation of computer-assisted student performance. We instantiate this paradigm in the context of a Reading Tutor that listens to children read aloud, and helps them. We introduce inter-word latency as a simple prosodic measure of assisted reading performance. Finally, to validate the measure and analyze performance improvement, we report initial experimental results from the first extended in-school deployment of the Reading Tutor. Content areas: computer aided education, spoken language understanding, user interfaces, cognitive modeling, multimedia

AAAI Conference 1994 Conference Paper

A Prototype Reading Coach that Listens

• Jack Mostow
• Alexander G. Hauptmann

We report progress on a new approach to combatting illiteracy -- getting computers to listen to children read aloud. We describe a fully automated prototype coach for oral reading. It displays a story on the screen, listens as a child reads it, and decides whether and how to intervene. We report on pilot experiments with low-reading second graders to test whether these interventions are technically feasible to automate and pedagogically effective to perform. By adapting a continuous speech recognizer, we detected 49% of the misread words, with a false alarm rate under 4%. By incorporating the interventions in a simulated coach, we enabled the children to read and comprehend material at a reading level 0.6 years higher than what they could read on their own. We show how the prototype uses the recognizer to trigger these interventions automatically.

AAAI Conference 1994 Conference Paper

A Reading Coach that Listens: (Edited Video Transcript)

• Jack Mostow
• Stevn F. Roth
• Adam Swift

At Carnegie Mellon University, Project LISTEN' is t'aking a novel approach to the problem of illiteracy. We have developed a prototype automated reading coach that listens to a child read aloud, and helps when needed. The coach provides a combination of reading and listening, in which the child reads wherever possible, and the coach helps wherever necessary -- a bit like training wheels on a bicycle.

AIJ Journal 1993 Journal Article

An apprentice-based approach to knowledge acquisition

• Sridhar Mahadevan
• Tom M. Mitchell
• Jack Mostow
• Lou Steinberg
• Prasad V. Tadepalli

We explore here the feasibility of learning apprentice programs: interactive knowledge-based assistants that learn by observing and analyzing the problem-solving steps of their users. In particular, we describe a learning apprentice for digital circuit design, called LEAP. LEAP learns feasible ways of decomposing circuit modules into submodules, as well as the recommended method when there are competing feasible decompositions. VBL is an explanation-based learning technique used in LEAP to infer problem-reduction operators for decomposing circuit modules. PED is a general extension of explanation-based learning to incomplete domain theories containing determinations. PED is used in LEAP to learn control rules for ranking alternative decompositions as well as to extend LEAP's partial theory of circuit cost. An experimental study shows that by using this approach LEAP can learn a significant subset of a manually created knowledge base for boolean circuit design. The experimental study also reveals some limitations of LEAP, and more generally suggests directions for further research in building effective learning apprentice systems.

AAAI Conference 1993 Conference Paper

Towards a Reading Coach that Listens: Automated Detection of Oral Reading Errors

• Jack Mostow
• Lin Lawrence Chase

AIJ Journal 1989 Journal Article

Design by derivational analogy:Issues in the automated replay of design plans

• Jack Mostow

Derivational analogy solves a problem by replaying the plan used to solve a previous problem, modifying it where necessary. We analyze how four published systems use this approach to help design (or redesign) complex artifacts like programs and circuits. We compare how they represent, acquire, and retrieve design plans; how they determine which parts of the old and new designs correspond; how they decide which steps of a design plan are appropriate to replay and adapt them to the new problem; and how they reuse partial plans. We show how each system's approach to these seven issues affects the SCOPE of problems it can solve, its EVOLVABILITY to solve new problems, the QUALITY of its solutions, the EFFICIENCY of its computation, and its AUTONOMY from the user.

IJCAI Conference 1989 Conference Paper

Discovering Admissible Heuristics by Abstracting and Optimizing: A Transformational Approach

• Jack Mostow
• Armand E. Prieditis

IJCAI Conference 1987 Conference Paper

Failsafe—A Floor Planner that Uses EBG to Leam from Its Failures

• Jack Mostow
• Neeraj Bhatnagar

AAAI Conference 1986 Conference Paper

Towards Explicit Integration of Knowledge in Expert Systems: An Analysis of MYClN’s Therapy Selection Algorithm

• Jack Mostow

The knowledge integration problem arises in rule-based expert systems when two or more recommendations made by right-hand sides of rules must be combined. Current expert systems address this problem either by engineering the rule set to avoid it, or by using a single integration technique built into the interpreter, e.g., certainty factor combination. We argue that multiple techniques are needed and that their use -- and underlying assumptions -- should be made explicit. We identify some of the techniques used in MYCIN’s therapy selection algorithm to integrate the diverse goals it attempts to satisfy, and suggest how knowledge of such techniques could be used to support construction, explanation, and maintenance of expert systems.

IJCAI Conference 1985 Conference Paper

Automating Program Speedup by Deciding What to Cache

• Jack Mostow
• Donald Cohen

AAAI Conference 1983 Conference Paper

A Problem-Solver for Making Advice Operational

• Jack Mostow

One problem with taking advice arises when the advice is expressed in terms of data or actions unavailable to the advice- taker. For example. in the card game Hearts, the advice "don’t lead a suit in which some opponent has no cards left" is non- operational because players cannot see their opponents’ cards. Operationalization is the process of converting such advice into an executable (perhaps heuristic) procedure. This paper describes an interactive system, called BAR. that operationalizes advice by applying a series of program transformations. By applying different transformation sequences, BAR can operationalize the same advice in very different ways. BAR uses means-ends analysis and planning in an abstraction space. Rather than using a hand-coded difference table, BAR analyzes the transformations to identify transformation sequences that might help solve a given problem. Thus new transformations can be added without modifying the problem-solver itself. Although user intervention is required to select among alternative plans, BAR reduces the number of alternatives by 10 3 compared to an earlier operationalizer.