Robert E. Mercer

ECAI Conference 2023 Conference Paper

Investigating the Learning Behaviour of In-Context Learning: A Comparison with Supervised Learning

• Xindi Wang 0001
• Yufei Wang 0003
• Can Xu
• Xiubo Geng
• Bowen Zhang
• Chongyang Tao
• Frank Rudzicz
• Robert E. Mercer

Large language models (LLMs) have shown remarkable capacity for in-context learning (ICL), where learning a new task from just a few training examples is done without being explicitly pre-trained. However, despite the success of LLMs, there has been little understanding of how ICL learns the knowledge from the given prompts. In this paper, to make progress toward understanding the learning behaviour of ICL, we train the same LLMs with the same demonstration examples via ICL and supervised learning (SL), respectively, and investigate their performance under label perturbations (i. e. , noisy labels and label imbalance) on a range of classification tasks. First, via extensive experiments, we find that gold labels have significant impacts on the downstream in-context performance, especially for large language models; however, imbalanced labels matter little to ICL across all model sizes. Second, when comparing with SL, we show empirically that ICL is less sensitive to label perturbations than SL, and ICL gradually attains comparable performance to SL as the model size increases.

AAAI Conference 2020 Conference Paper

Modelling Sentence Pairs via Reinforcement Learning: An Actor-Critic Approach to Learn the Irrelevant Words

• Mahtab Ahmed
• Robert E. Mercer

Learning sentence representation is a fundamental task in Natural Language Processing. Most of the existing sentence pair modelling architectures focus only on extracting and using the rich sentence pair features. The drawback of utilizing all of these features makes the learning process much harder. In this study, we propose a reinforcement learning (RL) method to learn a sentence pair representation when performing tasks like semantic similarity, paraphrase identi- fication, and question-answer pair modelling. We formulate this learning problem as a sequential decision making task where the decision made in the current state will have a strong impact on the following decisions. We address this decision making with a policy gradient RL method which chooses the irrelevant words to delete by looking at the sub-optimal representation of the sentences being compared. With this policy, extensive experiments show that our model achieves on par performance when learning task-specific representations of sentence pairs without needing any further knowledge like parse trees. We suggest that the simplicity of each task inference provided by our RL model makes it easier to explain.

ECAI Conference 2006 Conference Paper

On Probing and Multi-Threading in Platypus

• Jean Gressmann
• Tomi Janhunen
• Robert E. Mercer
• Torsten Schaub
• Sven Thiele
• Richard Tichy

The PLATYPUS approach offers a generic platform for distributed answer set solving, accommodating a variety of different architectures for distributing the search for answer sets across different processes and different search modes for modifying search behaviour. We describe two major extensions of PLATYPUS. First, we present its probing mode which provides a controlled non-linear traversal of the search space. Second, we present its new multi-threading architecture allowing for intra-process distribution. Both contributions are underpinned by experimental results illustrating their computational impact.

NMR Workshop 2002 Conference Paper

Optimality theory through default logic

• Philippe Besnard
• Robert E. Mercer
• Torsten Schaub

Optimality Theory is an approach to linguistic problems which is based on rules with exceptions, resorting to a ranking among rules to resolve conflicts arising from competing rules. In such a way, dealing with linguistic problems amounts to applying rules with exceptions: That is reasoning. A related issue is then about a formalization of the logic at work. An immediate candidate is Default Logic which is dedicated to reasoning from rules with exceptions. Moreover, there are versions of default logic with priorities. We show that Default Logic is well-suited as a specification language capturing Optimality Theory and suggests that implementations of default logic can be applied to run experiments with grammatical interaction in the sense of Optimality Theory and beyond.

AIJ Journal 1989 Journal Article

The importance of open and recursive circumscription

• Philippe Besnard
• Yves Moinard
• Robert E. Mercer

Circumscription is known to result in an inconsistency when applied to certain consistent theories. To counter this problem, closed nonrecursive circumscription, a restricted form of circumscription that has been proved not to affect the consistency of the theory over which circumscription is applied, has been proposed. We show that closed nonrecursive circumscription involves an excessive weakening of standard circumscription by establishing that closed nonrecursive circumscription is incomplete for some crucial theories over which standard circumscription is consistent and complete. First, we prove that closed circumscription cannot yield the desired uniqueness formula for the simplest of existential theories. Second, we prove that nonrecursive circumscription fails to be as strong as predicate completion for Horn clause theories. Third, we prove that the natural way to strengthen circumscription, that is, adding more variable predicates, may weaken nonrecursive circumscription.

NMR Workshop 1984 Conference Paper

On the Adequacy of Predicate Circumscription for Closed-World Reasoning

• David W. Etherington
• Robert E. Mercer
• Raymond Reiter