TERRY R. PAYNE

ECAI Conference 2025 Conference Paper

Task-Based Ontology Evaluation for Question Generation

• Samah AlKhuzaey
• Floriana Grasso
• Terry R. Payne
• Valentina Tamma

Ontology-based question generation is an important application of semantic-aware systems that enables the creation of large question banks for diverse learning environments. The effectiveness of these systems, both in terms of the calibre and cognitive difficulty of the resulting questions, depends heavily on the quality and modelling approach of the underlying ontologies, making it crucial to assess their fitness for this task. To date, there has been no comprehensive investigation into the specific ontology aspects or characteristics that affect the question generation process. Therefore, this paper proposes a set of requirements and task-specific metrics for evaluating the fitness of ontologies for question generation tasks in pedagogical settings. Using the ROMEO methodology (a structured framework used for identifying task-specific metrics), a set of evaluation metrics have been derived from an expert assessment of questions generated by a question generation model. To validate the proposed metrics, we apply them to a set of ontologies previously used in question generation to illustrate how the metric scores align with and complement findings reported in earlier studies. The analysis confirms that ontology characteristics significantly impact the effectiveness of question generation, with different ontologies exhibiting varying performance levels. This highlights the importance of assessing ontology quality with respect to Automatic Question Generation (AQG) tasks.

AAMAS Conference 2024 Conference Paper

Large Language Model Assissted Multi-Agent Dialogue for Ontology Alignment

• Shiyao Zhang
• Yuji Dong
• Yichuan Zhang
• TERRY R. PAYNE
• Jie Zhang

Ontology alignment is critical in cross-domain integration; however, it typically necessitates the involvement of a human domain-expert, which can make the task costly. Although a variety of machinelearning approaches have been proposed that can simplify this task by learning the patterns from experts, such techniques are still susceptible to domain knowledge updates that could potentially change the patterns and lead to extra expert involvement. The use of Large Language Models (LLMs) has demonstrated a general cognitive ability, which has the potential to assist ontology alignment from the cognition level, thus obviating the need for costly expert involvement. However, the process by which the output of LLMs is generated can be opaque and thus the reliability and interpretability of such models is not always predictable. This paper proposes a dialogue model, in which multiple agents negotiate the correspondence between two knowledge sets with the support from an LLM. We demonstrate that this approach not only reduces the need for the involvement of a domain expert for ontology alignment, but that the results are interpretable despite the use of LLMs.

AAMAS Conference 2017 Conference Paper

Mechanism Design for Ontology Alignment

• Piotr Krysta
• Minming Li
• TERRY R. PAYNE
• Nan Zhi

The aim of the ontology alignment problem is to find meaningful correspondences between two ontologies represented as collections of entities. This problem can be modelled as a novel mechanism design problem on an edge-weighted bipartite graph, where each side of the graph holds each agent’s private entities, and the objective is to maximise the agents’ social welfare. Having studied implementation in dominant strategies with and without payments, we report on findings that for truthful mechanisms, these problems need to be solved optimally. We also study greedy allocation rules with a first-price payment rule, and implementation in pure, mixed & Bayesian Nash equilibria, and have found tight bounds on the price of anarchy and stability.

AAMAS Conference 2016 Conference Paper

A Dialogue Protocol to Support Meaning Negotiation (Extended Abstract)

• Gabrielle Santos
• Valentina Tamma
• TERRY R. PAYNE
• Floriana Grasso

Despite numerous efforts, the problem of dynamically reconciling heterogeneity within open distributed multi-agent systems is far from solved. As different systems often use their own vocabularies to express the content of communication messages (ontologies), semantic reconciliation requires some form of agreement over a shared model, obtained through an alignment whereby concepts in the requester’s ontology are mapped (translated) into concepts in the respondent’s one. This paper presents a dialogue that allows agents to reach an agreement over a correspondence between two entities in their respective ontologies in a decentralised fashion, without requiring prior knowledge over their ontological models.

AAMAS Conference 2016 Conference Paper

A Synergy Coalition Group Based Dynamic Programming Algorithm for Coalition Formation

• Luke Riley
• Katie Atkinson
• Paul E. Dunne
• TERRY R. PAYNE

Coalition formation in characteristic function games entails agents partitioning themselves into a coalition structure and assigning the numeric rewards of each coalition via a payoff vector. Various coalition structure generation algorithms have been proposed that guarantee that an optimal coalition structure is found. We present the Synergy Coalition Group-based Dynamic Programming (SCG- DP) algorithm that guarantees that an optimal coalition structure and a least core stable payoff vector is found. This is completed by extending the existing results for the Synergy Coalition Group (SCG) representation to show that only coalitions in the SCG are needed to find a weak-least core stable payoff vector. The SCG- DP algorithm builds on this result by performing only the search operations necessary to guarantee that coalitions in the SCG of the given characteristic function game are found. The number of operations required is significantly less for many coalition-value distributions compared to the original Dynamic Programming (DP) algorithm [34] that finds an optimal coalition structure (e. g. only ∼60% of DP’s coalition lookup operations are performed in SCG- DP for 18 agents using a normal coalition-value distribution). Our experimental results show that a lower bound for these operations in SCG-DP converges onto 50%. This is an increase on the ∼33% bound of the optimal dynamic programming (ODP) algorithm [14], but ODP does not search for a stable solution. General Terms Algorithms, Economics, Theory

EUMAS Conference 2015 Conference Paper

A Dialectical Approach to Enable Decision Making in Online Trading

• Wei Bai 0005
• Emmanuel M. Tadjouddine
• Terry R. Payne

Abstract Software agents, acting on behalf of humans, have been identified as an important solution for future electronic markets. Such agents can make their own decisions given prior preferences and the market environment. These preferences can be described using web ontology languages (OWL), while the market can be represented in a machine-understandable way by utilizing the technique of Semantic Web Services (SWS). Besides, SWS enables agents to automatically discover, select, compose and invoke services. To extend the dependability and interactivity of SWS, we have utilized dialogue games and the proof-carrying code to enable buyers interact with sellers, so that interest properties for an online auction market can be automatically certified. Our decision making framework combines formal proofs with informal evidence collected by web services in a dialogue game between a seller and a buyer. We have implemented our approach and experimental results have demonstrated the feasibility as well as the validity of this framework as an enabler for a buyer agent to enter or not an online auction.

IJCAI Conference 2009 Conference Paper

• Paul Doran
• Valentina Tamma
• TERRY R. PAYNE
• Ignazio Palmisano

Effective communication in open environments relies on the ability of agents to reach a mutual understanding of the exchanged message by reconciling the vocabulary (ontology) used. Various approaches have considered how mutually acceptable mappings between corresponding concepts in the agents’ own ontologies may be determined dynamically through argumentation-based negotiation (such as Meaning-based Argumentation). However, the complexity of this process is high, approaching Π (p) 2 -complete in some cases. As reducing this complexity is non-trivial, we propose the use of ontology modularization as a means of reducing the space over which possible concepts are negotiated. The suitability of different modularization approaches as filtering mechanisms for reducing the negotiation search space is investigated, and a framework that integrates modularization with Meaning-based Argumentation is proposed. We empirically demonstrate that some modularization approaches not only reduce the number of alignments required to reach consensus, but also predict those cases where a service provider is unable to satisfy a request, without the need for negotiation.

AAMAS Conference 2009 Conference Paper

Emergent Service Provisioning and Demand Estimation through Self-Organizing Agent Communities

• Mariusz Jacyno
• Seth Bullock
• Michael Luck
• TERRY R. PAYNE

A major challenge within open markets is the ability to satisfy service demand with an adequate supply of service providers, especially when such demand may be volatile due to changing requirements, or fluctuations in the availability of services. Ideally, this supply and demand should be balanced; however, when consumer demand changes over time, and providers independently choose which services they provide, a coordination problem known as ‘herding’ can arise bringing instability to the market. This behavior can emerge when consumers share similar preferences for the same providers, and thus compete for the same resources. Likewise, providers which share estimates of fluctuating demand may respond in unison, withdrawing some services to introduce others, and thus oscillate the available supply around some ideal equilibrium. One approach to avoid this unstable behavior is to limit the flow of information between agents, such that they possess an incomplete and subjective view of the local service availability. We propose a model of an adaptive service-offering mechanism, in which providers adapt their choice of services offered to consumers, based on perceived demand. By varying the volume of information shared by agents, we demonstrate that a co-adaptive equilibrium can be achieved, thus avoiding the herding problem. As the knowledge that agents possess is limited, they self-organise into community structures that support locally shared information. We demonstrate that such a model is capable of reducing instability in service demand and thus increase utility (based on successful service provision) by up to 59%, when compared to the use of globally available information.

AAMAS Conference 2007 Conference Paper

An Advanced Bidding Agent for Advertisement Selection on Public Displays

• Alex Rogers
• Esther David
• TERRY R. PAYNE
• Nicholas R. Jennings

In this paper we present an advanced bidding agent that participates in first-price sealed bid auctions to allocate advertising space on BluScreen – an experimental public advertisement system that detects users through the presence of their Bluetooth enabled devices. Our bidding agent is able to build probabilistic models of both the behaviour of users who view the adverts, and the auctions that it participates within. It then uses these models to maximise the exposure that its adverts receive. We evaluate the effectiveness of this bidding agent through simulation against a range of alternative selection mechanisms including a simple bidding strategy, random allocation, and a centralised optimal allocation with perfect foresight. Our bidding agent significantly outperforms both the simple bidding strategy and the random allocation, and in a mixed population of agents it is able to expose its adverts to 25% more users than the simple bidding strategy. Moreover, its performance is within 7. 5% of that of the centralised optimal allocation despite the highly uncertain environment in which it must operate.

AAMAS Conference 2007 Conference Paper

Provisioning Heterogeneous and Unreliable Providers for Service Workflows

• Sebastian Stein
• Nicholas R. Jennings
• TERRY R. PAYNE

In this paper, we address the problemof provisioning unreliable and heterogeneous service providers for the constituent tasks of abstract workflows. Specifically, we deal with unreliable providers by provisioning multiple service providers redundantly for specific tasks, and we employ a local search mechanism to choose among many heterogeneous providers that offer the same type of service. We empirically show that our strategy can achieve significant improvements over current approaches, and we demonstrate that it works well over a range of environments.

EUMAS Conference 2006 Conference Paper

Agents Arguing over Ontology Alignments

• Loredana Laera
• Valentina Tamma
• Jérôme Euzenat
• Trevor J. M. Bench-Capon
• Terry R. Payne

ECAI Conference 2006 Conference Paper

Auction Mechanisms for Efficient Advertisement Selection on Public Displays

• Terry R. Payne
• Esther David
• Nicholas R. Jennings
• Matthew Sharifi

Public electronic displays can be used as an advertising medium when space is a scarce resource, and it is desirable to expose many adverts to as wide an audience as possible. Although the efficiency of such advertising systems can be improved if the display is aware of the identity and interests of the audience, this knowledge is difficult to acquire when users are not actively interacting with the display. To this end, we present BluScreen, an intelligent public display, which selects and displays adverts in response to users detected in the audience. Here, users are identified and their advert viewing history tracked, by detecting any Bluetooth-enabled devices they are carrying (e. g. phones, PDAs, etc.). Within BluScreen we have implemented an agent system that utilises an auction-based marketplace to efficiently select adverts for the display, and deployed this within an installation in our Department. We demonstrate, by means of an empirical evaluation, that the performance of this auction-based mechanism when used with our proposed bidding strategy, efficiently selects the best adverts in response to the audience presence. We bench-marked our advertising method with two other commonly applied selection methods for displaying adverts on public displays; specifically the Round-Robin and the Random approaches. The results show that our auction-based approach, that utilised the novel use of Bluetooth detection, outperforms these two methods by up to 64%.

EUMAS Conference 2006 Conference Paper

Auction Mechanisms for Efficient Advertisement Selection on Public Displays

• Terry R. Payne
• Esther David
• Nicholas R. Jennings
• Matthew Sharifi

ECAI Conference 2006 Conference Paper

Flexible Provisioning of Service Workflows

• Sebastian Stein 0001
• Nicholas R. Jennings
• Terry R. Payne

Service-oriented computing is a promising paradigm for highly distributed and complex computer systems. In such systems, services are offered by provider agents over a computer network and automatically discovered and provisioned by consumer agents that need particular resources or behaviours for their workflows. However, in open systems where there are significant degrees of uncertainty and dynamism, and where the agents are self-interested, the provisioning of these services needs to be performed in a more flexible way than has hitherto been considered. To this end, we devise a number of heuristics that vary provisioning according to the predicted performance of provider agents. We then empirically benchmark our algorithms and show that they lead to a 350% improvement in average utility, while successfully completing 5–6 times as many workflows as current approaches.

EUMAS Conference 2006 Conference Paper

Flexible Provisioning of Service Workflows

• Sebastian Stein 0001
• Nicholas R. Jennings
• Terry R. Payne

KER Journal 2004 Journal Article

The synergy of electronic commerce, agents, and semantic Web services

• M. BRIAN BLAKE
• Simon Parsons
• TERRY R. PAYNE

Advancements in software agents and Semantic Web service technologies are generally enhancing the landscape of electronic commerce. Semantic Web service technologies promise the standardisation and discoverability of software capabilities for network-enabled organisations. Moreover, with the addition of the intelligence and autonomy of software agents, transactions may be equally automated for consumer-to-consumer, business-to-consumer, and business-to-business collaborations. The 2003 Workshop on Electronic Commerce, Agents, and Semantic Web Services was held in conjunction with the International Conference on Electronic Commerce (ICEC2003). The purpose of this workshop was to bring together researchers and practitioners in the areas of electronic commerce, agents, and Semantic Web services to discuss the state-of-art in each individual area in addition to the synergies among the areas. This paper contains a summary of the workshop presentations and a discussion of next steps for Semantic Web services created in the working sessions concluding the workshop.

ECAI Conference 1998 Conference Paper

Implicit Feature Selection with the Value Difference Metric

• Terry R. Payne
• Peter Edwards