David B. Leake

NeSy Conference 2023 Conference Paper

Large Language Models Need Symbolic AI

• Kristian J. Hammond
• David B. Leake

The capability of systems based on large language models (LLMs), such as ChatGPT, to generate humanlike text has captured the attention of the public and the scientific community. It has prompted both predictions that systems such as ChatGPT will transform AI and enumerations of system problems with hopes of solving them by scale and training. This position paper argues that both over-optimistic views and disppointments reflect misconceptions of the fundamental nature of LLMs as language models. As such, they are statistical models of language production and fluency, with associated strengths and limitations; they are not—and should not be expected to be—knowledge models of the world, nor do they reflect the core role of language beyond the statistics: communication. The paper argues that realizing that role will require driving LLMs with symbolic systems based on goals, facts, reasoning, and memory.

ECAI Conference 2016 Conference Paper

A Computational Method for Extracting, Representing, and Predicting Social Closeness

• Katherine Metcalf
• David B. Leake

Identifying the social closeness between two individuals is a key skill in any situation where interpersonal relations play a role. Consequently, such capacity is needed to enable AI systems to understand human interactions and interact naturally in social situations. However, current research has relied on simple proxies for social closeness, and richer models are difficult to achieve. This paper presents an approach to predicting social closeness based on linguistic interactions and demonstrates an ability to predict social closeness with high accuracy.

AAAI Conference 1997 Conference Paper

Case-Based Similarity Assessment: Estimating Adaptability from Experience

• David B. Leake

Case-based problem-solving systems rely on similarity assessment to select stored cases whose solutions are easily adaptable to fit current problems. However, widely-used similarity assessment strategies, such as evaluation of semantic similarity, can be poor predictors of adaptability. As a result, systems may select cases that are difficult or impossible for them to adapt, even when easily adaptable cases are available in memory. This paper presents a new similarity assessment approach which couples similarity judgments directly to a case library containing the systemIs adaptation knowledge. It examines this approach in the context of a case-based planning system that learns both new plans and new adaptations. Empirical tests of alternative similarity assessment strategies show that this approach enables better case selection and increases the benefits accrued from learned adaptations.

IJCAI Conference 1997 Conference Paper

Learning to Integrate Multiple Knowledge Sources for Case-Based Reasoning

• David B. Leake
• Andrew Kinley
• David Wilson

The case* based reasoning process depends on multiple overlapping knowledge sources, each of which provides an opportunity for learning. Exploiting these opportunities requires not only determining the learning mechanisms to use for each individual knowledge source, but also how the different learning mechanisms interact and their combined utility. This paper presents a case study examining the relative contributions and costs involved in learning processes for three different knowledge sources—cases, case adaptation knowledge, and similarity information—in a casebased planner. It demonstrates the importance of interactions between different learning processes and identifies a promising method for integrating multiple learning methods to improve case-based reasoning.

AAAI Conference 1996 Conference Paper

Acquiring Case Adaptation Knowledge: A Hybrid Approach

• David B. Leake

The ability of case-based reasoning (CBR) systems to apply cases to novel situations depends on their case adaptation knowledge. However, endowing CBR systems with adequate adaptation knowledge has proven to be a very difficult task. This paper describes a hybrid method for performing case adaptation, using a combination of rule-based and case-based reasoning. It shows how this approach provides a framework for acquiring flexible adaptation knowledge from experiences with autonomous adaptation and suggests its potential as a basis for acquisition of adaptation knowledge from interactive user guidance. It also presents initial experimental results examining the benefits of the approach and comparing the relative contributions of case learning and adaptation learning to reasoning performance.

IJCAI Conference 1995 Conference Paper

Using Introspective Reasoning to Refine Indexing

• Susan Fox
• David B. Leake

Introspective reasoning about a system's own reasoning processes can form the basis for learning to refine those reasoning processes. The ROBBIE1 system uses introspective reasoning to monitor the retrieval process of a case-based planner to detect retrieval of inappropriate cases. When retrieval problems are detected, the source of the problems is explained and the explanations are used to determine new indices to use during future case retrieval. The goal of ROBBIE's learning is to increase its ability to focus retrieval on relevant cases, with the aim of simultaneously decreasing the number of candidates to consider and increasing the likelihood that the system will be able to successfully adapt the retrieved cases to fit the current situation. We evaluate the benefits of the approach in light of empirical results examining the effects of index learning in the ROBBIE system.

KER Journal 1994 Journal Article

Case-based reasoning

• David B. Leake

Case-based reasoning (CBR) systems reason from experience: they solve new problems by retrieving relevant prior cases and adapting them to fit new situations. In 1988 the first case-based reasoning workshop, sponsored by DARPA, identified theoretical foundations and fundamental issues for case-based reasoning research. Since then, much investigation has examined the CBR process itself, the validity of CBR as a cognitive model, and the application of CBR technology. The results of that work include refinements in theories of the case-based reasoning process, psychological evidence for human case-based reasoning, and the fielding of over 100 CBR applications.

IJCAI Conference 1993 Conference Paper

Focusing Construction and Selection of Abductive Hypotheses

• David B. Leake

Many abductive understanding systems explain novel situations by a chaining process that is neutral to explainer needs beyond generating some plausible explanation for the event being explained. This paper examines the relationship of standard models of abductive understanding to the case-based explanation model. In case-based explanation, construction and selection of abductive hypotheses are focused by specific explanations of prior episodes and by goal-based criteria reflecting current information needs. The case-based method is inspired by observations of human explanation of anomalous events during everyday understanding, and this paper focuses on the method's contributions to the problems of building good explanations in everyday domains. We identify five central issues, compare how those issues are addressed in traditional and case-based explanation models, and discuss motivations for using the case-based approach to facilitate generation of plausible and useful explanations in domains that are complex and imperfectly understood.

AAAI Conference 1991 Conference Paper

An Indexing Vocabulary for Case-Based Explanation

• David B. Leake

The success of case-based reasoning depends on effective retrieval of relevant prior cases. If retrieval is expensive, or if the cases retrieved are inappropriate, retrieval and adaptation costs will nullify many of the advantages of reasoning from prior experience. We propose an indexing vocabulary to facilitate retrieval of explanations in a casebased explanation system. The explanations we consider are explanations of anomalies (conflicts between new situations and prior expectations or beliefs). Our vocabulary groups anomalies according to the type of information used to generate the expectations or beliefs that failed, and according to how the expectations failed. We argue that by using this vocabulary to characterize anomalies, and retrieving explanations that were built to account for similarly-characterized past anomalies, a case-based explanation system can restrict retrieval to explanations likely to be relevant. In addition, the vocabulary can be used to organize general explanation strategies that suggest paths for explanation in novel situations.

AIJ Journal 1989 Journal Article

Creativity and learning in a case-based explainer

• Roger C. Schank
• David B. Leake

Explanation-based learning (EBL) is a very powerful method for category formation. Since EBL algorithms depend on having good explanations, it is crucial to have effective ways to build explanations, especially in complex real-world situations where complete causal information is not available. When people encounter new situations, they often explain them by remembering old explanations, and adapting them to fit. We believe that this case-based approach to explanation holds promise for use in AI systems, both for routine explanation and to creatively explain situations quite unlike what the system has encountered before. Building new explanations from old ones relies on having explanations available in memory. We describe explanation patterns (XPs), knowledge structures that package the reasoning underlying explanations. Using the SWALE system as a base, we discuss the retrieval and modification process, and the criteria used when deciding which explanation to accept. We also discuss issues in learning XPs: what generalization strategies are appropriate for real-world explanations, and which indexing strategies are appropriate for XPs. SWALE's explanations allow it to understand nonstandard stories, and the XPs it learns increase its efficiency in dealing with similar anomalies in the future.

AAAI Conference 1988 Conference Paper

Evaluating Explanations

• David B. Leake