Marc Cavazza

JAAMAS Journal 2021 Journal Article

Automated narrative planning model extension

• Julie Porteous
• João F. Ferreira
• Marc Cavazza

Abstract Interactive Narrative is an emerging application of automated planning, in which a planning domain is used to generate a consistent chain of narrative actions that constitute a plot structure. The task of creating narrative planning domains has been identified as a bottleneck which is hampering further development of the field. This stems from the difficulties faced by humans authoring such planning domains due to the need to provide the range of alternative content, such as actions, which are required to support the important properties of diversity and robustness. Narrative planning domains must be capable of generating diverse sets of narratives to ensure system replayability, and they must also be able to respond robustly in the face of narrative execution failure due to user interaction. In this paper, we introduce a novel approach to the development of narrative planning domains based on the automatic expansion of a baseline planning domain through application of principled operations applied to both operators and predicates. We overview two such operations in this paper. The first of these, anton for anton ymic operators, is based on the generation of contrary operators that can be invoked in the face of action failure, and whose structure is derived from a model of state transitions triggered by the original operator. Since the intention is for additional operators to be incorporated to the baseline, human-authored, domain model, the generated contents should be human-readable. This is achieved by using combined linguistic resources to access antonyms of predicates occurring inside operators, and parsing them from and into hyphenated units. The second operation, part of the same approach, referred to as thype, generates variants of operators by exploring type hierarchies for the main concepts associated with individual operators; the resulting concepts being fully integrated into a new operator’s structure. Our evaluation procedures are directly derived from the target properties of narrative planning domains, which are diversity and robustness, the former being measured through plot diversity and the latter, plot continuation following planned action execution failure. We used published narrative domains as datasets for these evaluations. Results demonstrated strong generative ability, and even more significant plan completion following action failure. Moreover, our evaluation demonstrates the synergic nature of anton and thype when applied simultaneously. Future work will focus on improving the integration of anton and thype operations through better balance between linguistic and conceptual hierarchies.

NeurIPS Conference 2021 Conference Paper

Generalization Bounds for Graph Embedding Using Negative Sampling: Linear vs Hyperbolic

• Atsushi Suzuki
• Atsushi Nitanda
• Jing Wang
• Linchuan Xu
• Kenji Yamanishi
• Marc Cavazza

Graph embedding, which represents real-world entities in a mathematical space, has enabled numerous applications such as analyzing natural languages, social networks, biochemical networks, and knowledge bases. It has been experimentally shown that graph embedding in hyperbolic space can represent hierarchical tree-like data more effectively than embedding in linear space, owing to hyperbolic space's exponential growth property. However, since the theoretical comparison has been limited to ideal noiseless settings, the potential for the hyperbolic space's property to worsen the generalization error for practical data has not been analyzed. In this paper, we provide a generalization error bound applicable for graph embedding both in linear and hyperbolic spaces under various negative sampling settings that appear in graph embedding. Our bound states that error is polynomial and exponential with respect to the embedding space's radius in linear and hyperbolic spaces, respectively, which implies that hyperbolic space's exponential growth property worsens the error. Using our bound, we clarify the data size condition on which graph embedding in hyperbolic space can represent a tree better than in Euclidean space by discussing the bias-variance trade-off. Our bound also shows that imbalanced data distribution, which often appears in graph embedding, can worsen the error.

ICML Conference 2021 Conference Paper

Generalization Error Bound for Hyperbolic Ordinal Embedding

• Atsushi Suzuki 0002
• Atsushi Nitanda
• Jing Wang 0023
• Linchuan Xu
• Kenji Yamanishi
• Marc Cavazza

Hyperbolic ordinal embedding (HOE) represents entities as points in hyperbolic space so that they agree as well as possible with given constraints in the form of entity $i$ is more similar to entity $j$ than to entity $k$. It has been experimentally shown that HOE can obtain representations of hierarchical data such as a knowledge base and a citation network effectively, owing to hyperbolic space’s exponential growth property. However, its theoretical analysis has been limited to ideal noiseless settings, and its generalization error in compensation for hyperbolic space’s exponential representation ability has not been guaranteed. The difficulty is that existing generalization error bound derivations for ordinal embedding based on the Gramian matrix are not applicable in HOE, since hyperbolic space is not inner-product space. In this paper, through our novel characterization of HOE with decomposed Lorentz Gramian matrices, we provide a generalization error bound of HOE for the first time, which is at most exponential with respect to the embedding space’s radius. Our comparison between the bounds of HOE and Euclidean ordinal embedding shows that HOE’s generalization error comes at a reasonable cost considering its exponential representation ability.

AAAI Conference 2021 Conference Paper

Narrative Plan Generation with Self-Supervised Learning

• Mihai Polceanu
• Julie Porteous
• Alan Lindsay
• Marc Cavazza

Narrative Generation has attracted significant interest as a novel application of Automated Planning techniques. However, the vast amount of narrative material available opens the way to the use of Deep Learning techniques. In this paper, we explore the feasibility of narrative generation through selfsupervised learning, using sequence embedding techniques or auto-encoders to produce narrative sequences. We use datasets of well-formed plots generated by a narrative planning approach, using pre-existing, published, narrative planning domains, to train generative models. Our experiments demonstrate the ability of generative sequence models to produce narrative plots with similar structure to those obtained with planning techniques, but with significant plot novelty in comparison with the training set. Most importantly, generated plots share structural properties associated with narrative quality measures used in Planning-based methods. As planbased structures account for a higher level of causality and narrative consistency, this suggests that our approach is able to extend a set of narratives with novel sequences that display the same high-level narrative properties. Unlike methods developed to extend sets of textual narratives, ours operates at the level of plot structure. Thus, it has the potential to be used across various media for plots of significant complexity, being initially limited to training and generation operating in the same narrative genre.

AAMAS Conference 2019 Conference Paper

Multi-Agent Path Finding for UAV Traffic Management

• Florence Ho
• Ana Salta
• Ruben Geraldes
• Artur Goncalves
• Marc Cavazza
• Helmut Prendinger

Unmanned aerial vehicles (UAVs) are expected to provide a wide range of services, whereby UAV fleets will be managed by several independent service providers in shared low-altitude airspace. One important element, or redundancy, for safe and efficient UAV operation is pre-flight Conflict Detection and Resolution (CDR) methods that generate conflict-free paths for UAVs before the actual flight. Multi-Agent Path Finding (MAPF) has already been successfully applied to comparable problems with ground robots. However, most MAPF methods were tested with simplifying assumptions which do not reflect important characteristics of many real-world domains, such as delivery by UAVs where heterogeneous agents need to be considered, and new requests for flight operations are received continuously. In this paper, we extend CBS and ECBS to efficiently incorporate heterogeneous agents with computational geometry and we reduce the search space with spatio-temporal pruning. Moreover, our work introduces a “batching” method into CBS and ECBS to address increased amounts of requests for delivery operations in an efficient manner. We compare the performance of our “batching” approach in terms of runtime and solution cost to a “first-come first-served” approach. Our scenarios are based on a study on UAV usage predicted for 2030 in a real area in Japan. Our simulations indicate that our proposed ECBS based “batching” approach is more time efficient than incremental planning based on Cooperative A*, and hence can meet the requirements of timely and accurate response on delivery requests to users of such UTM services.

YNIMG Journal 2019 Journal Article

Volitional limbic neuromodulation exerts a beneficial clinical effect on Fibromyalgia

• Noam Goldway
• Jacob Ablin
• Omer Lubin
• Yoav Zamir
• Jackob Nimrod Keynan
• Ayelet Or-Borichev
• Marc Cavazza
• Fred Charles

Volitional neural modulation using neurofeedback has been indicated as a potential treatment for chronic conditions that involve peripheral and central neural dysregulation. Here we utilized neurofeedback in patients suffering from Fibromyalgia - a chronic pain syndrome that involves sleep disturbance and emotion dysregulation. These ancillary symptoms, which have an amplificating effect on pain, are known to be mediated by heightened limbic activity. In order to reliably probe limbic activity in a scalable manner fit for EEG-neurofeedback training, we utilized an Electrical Finger Print (EFP) model of amygdala-BOLD signal (termed Amyg-EFP), that has been successfully validated in our lab in the context of volitional neuromodulation. We anticipated that Amyg-EFP-neurofeedback training aimed at limbic down modulation would improve chronic pain in patients suffering from Fibromyalgia, by reducing sleep disorder improving emotion regulation. We further expected that improved clinical status would correspond with successful training as indicated by improved down modulation of the Amygdala-EFP signal. Thirty-Four Fibromyalgia patients (31F; age 35. 6 ± 11. 82) participated in a randomized placebo-controlled trial with biweekly Amyg-EFP-neurofeedback sessions or sham neurofeedback (n = 9) for a total duration of five consecutive weeks. Following training, participants in the real-neurofeedback group were divided into good (n = 13) or poor (n = 12) modulators according to their success in the neurofeedback training. Before and after treatment, self-reports on pain, depression, anxiety, fatigue and sleep quality were obtained, as well as objective sleep indices. Long-term clinical follow-up was made available, within up to three years of the neurofeedback training completion. REM latency and objective sleep quality index were robustly improved following the treatment course only in the real-neurofeedback group (time × group p < 0. 05) and to a greater extent among good modulators (time × sub-group p < 0. 05). In contrast, self-report measures did not reveal a treatment-specific response at the end of the neurofeedback training. However, the follow-up assessment revealed a delayed improvement in chronic pain and subjective sleep experience, evident only in the real-neurofeedback group (time × group p < 0. 05). Moderation analysis showed that the enduring clinical effects on pain evident in the follow-up assessment were predicted by the immediate improvements following training in objective sleep and subjective affect measures. Our findings suggest that Amyg-EFP-neurofeedback that specifically targets limbic activity down modulation offers a successful principled approach for volitional EEG based neuromodulation treatment in Fibromyalgia patients. Importantly, it seems that via its immediate sleep improving effect, the neurofeedback training induced a delayed reduction in the target subjective symptom of chronic pain, far and beyond the immediate placebo effect. This indirect approach to chronic pain management reflects the substantial link between somatic and affective dysregulation that can be successfully targeted using neurofeedback.

AAMAS Conference 2018 Conference Paper

Simulating Shared Airspace for Service UAVs with Conflict Resolution

• Florence Ho
• Ruben Geraldes
• Artur Gon�alves
• Marc Cavazza
• Helmut Prendinger

In future UAV-based services, UAV fleets will be managed by independent service providers in shared low-altitude airspace. Therefore, Conflict Detection and Resolution (CDR) methods that solve conflicts, i. e. possible collisions, between UAVs of all service providers are a key element of the Unmanned Aircraft System Traffic Management (UTM) system. We present a top-to-bottom algorithmic system with an extension to UAV operations of ORCA, a state-ofthe-art algorithm in robotics. Then, using extreme-conflict situations, we empirically determine optimal parameter values for our adapted ORCA, and we observe a better performance compared to the standard use of ORCA. Finally, using realistic UAV traffic situations for delivery, we perform extensive simulations to study the potential occurrence and distribution of collisions, and to assess safety parameters for CDR.

AAMAS Conference 2017 Conference Paper

An Interactive Narrative Platform for Story Understanding Experiments

• Julie Porteous
• Fred Charles
• Cameron Smith
• Marc Cavazza
• Jolien Mouw
• Paul van den Broek

Interactive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. The focus of previous research has included aspects such as the quality of automatically generated narratives and the way in which audiences respond to them. However in this work we have developed a mechanism for control of interactive narratives that supports their use in experiments to assess story understanding. This is implemented in our demonstration system, which features two parts: an interface that allows high-level specification of criteria for story understanding experiments; and a participant interface in which virtual narratives, conforming to the experimental design, are presented as 3D visualizations. The virtual narrative is based on a pre-existing children’s story, and features a cast of virtual characters.

AAMAS Conference 2017 Conference Paper

Using Virtual Narratives to Explore Children's Story Understanding

• Julie Porteous
• Fred Charles
• Cameron Smith
• Marc Cavazza
• Jolien Mouw
• Paul van den Broek

Interactive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. Previous research in this area has focused on assessment of aspects such as the quality of the automatically generated narratives and their acceptance by the audience. However in our work we deviate from this to explore the use of interactive narratives to support cognitive psychology experiments in story understanding. We hypothesized that the use of virtual narratives would enable narrative comprehension to be studied independently of linguistic phenomena. To assess this we developed a demonstration interactive narrative featuring a virtual environment (Unity3D engine) based on a pre-existing children’s story which allows for the generation of variants of the original story that can be "told" via visualization in the 3D world. In the paper we introduce a narrative generation mechanism that provides control over insertion of cues facilitating story understanding, whilst also ensuring that the plot itself is unaffected. An intuitive user interface allows experimenters to insert and order cues and specific events while the narrative generation techniques ensure these requests are effected in a consistent fashion. We also report the results of a field experiment with children (age 9-10) that demonstrates the potential for the use of virtual narratives in story understanding experiments. Our results demonstrated acceptance of virtual narratives, the usability of the system and the impact of cue insertion on inference and story understanding.

AAMAS Conference 2016 Conference Paper

A Value Equivalence Approach for Solving Interactive Dynamic Influence Diagrams

• Ross Conroy
• Yifeng Zeng
• Marc Cavazza
• Jing Tang
• Yinghui Pan

Interactive dynamic influence diagrams (I-DIDs) are recognized graphical models for sequential multiagent decision making under uncertainty. They represent the problem of how a subject agent acts in a common setting shared with other agents who may act in sophisticated ways. The difficulty in solving I-DIDs is mainly due to an exponentially growing space of candidate models ascribed to other agents over time. in order to minimize the model space, the previous I-DID techniques prune behaviorally equivalent models. In this paper, we challenge the minimal set of models and propose a value equivalence approach to further compress the model space. The new method reduces the space by additionally pruning behaviorally distinct models that result in the same expected value of the subject agent’s optimal policy. To achieve this, we propose to learn the value from available data particularly in practical applications of real-time strategy games. We demonstrate the performance of the new technique in two problem domains.

ECAI Conference 2016 Conference Paper

Plan-Based Narrative Generation with Coordinated Subplots

• Julie Porteous
• Fred Charles
• Marc Cavazza

Despite recent progress in plan-based narrative generation, one major limitation is that systems tend to produce a single plotline whose progression entirely determines the narrative experience. However, for certain narrative genres such as serial dramas and soaps, multiple interleaved subplots are expected by the audience, as this tends to be the norm in real-world, human-authored narratives. Current narrative generation techniques have overlooked this important requirement, something which could improve the perceived quality of generated stories. To this end, we have developed a flexible plan-based approach to multiplot narrative generation, that successfully generates narratives conforming to different subplot profiles, in terms of the number of subplots interleaved and the relative time spent on each presentation. We have identified specific challenges such as: distribution of virtual characters across subplots; length of each subplot presentation; and transitioning between subplots.

IJCAI Conference 2015 Conference Paper

Learning Behaviors in Agents Systems with Interactive Dynamic Influence Diagrams

• Ross Conroy
• Yifeng Zeng
• Marc Cavazza
• Yingke Chen

Interactive dynamic influence diagrams (I-DIDs) are a well recognized decision model that explicitly considers how multiagent interaction affects individual decision making. To predict behavior of other agents, I-DIDs require models of the other agents to be known ahead of time and manually encoded. This becomes a barrier to I-DID applications in a human-agent interaction setting, such as development of intelligent non-player characters (NPCs) in real-time strategy (RTS) games, where models of other agents or human players are often inaccessible to domain experts. In this paper, we use automatic techniques for learning behavior of other agents from replay data in RTS games. We propose a learning algorithm with improvement over existing work by building a full profile of agent behavior. This is the first time that data-driven learning techniques are embedded into the I-DID decision making framework. We evaluate the performance of our approach on two test cases.

IJCAI Conference 2015 Conference Paper

Optimal Route Search with the Coverage of Users' Preferences

• Yifeng Zeng
• Xuefeng Chen
• Xin Cao
• Shengchao Qin
• Marc Cavazza
• Yanping Xiang

The preferences of users are important in route search and planning. For example, when a user plans a trip within a city, their preferences can be expressed as keywords shopping mall, restaurant, and museum, with weights 0. 5, 0. 4, and 0. 1, respectively. The resulting route should best satisfy their weighted preferences. In this paper, we take into account the weighted user preferences in route search, and present a keyword coverage problem, which finds an optimal route from a source location to a target location such that the keyword coverage is optimized and that the budget score satisfies a specified constraint. We prove that this problem is NP-hard. To solve this complex problem, we propose an optimal route search based on an A* variant for which we have defined an admissible heuristic function. The experiments conducted on real-world datasets demonstrate both the efficiency and accuracy of our proposed algorithms.

AAAI Conference 2015 Conference Paper

Using Social Relationships to Control Narrative Generation

• Julie Porteous
• Fred Charles
• Marc Cavazza

Narrative generation represents an application domain for AI planning where plan quality is related to properties such as shape of plan trajectory. In our work we have developed a plan-based approach to narrative generation that uses character relationships as a key determinant in controlling plan shape (relationships are key in genres such as serial dramas and soaps). Our approach is implemented in a demonstration Interactive Narrative, called NETWORKING, set in the medical drama genre. The system features a user-friendly mechanism for specifying relationships between virtual characters, via a social network and real-time visualisation of generated narratives on a 3D stage.

AAMAS Conference 2013 Conference Paper

A Social Network Interface to an Interactive Narrative

• Julie Porteous
• Fred Charles
• Marc Cavazza

Regular viewers of serial dramas tend to construct a model of the social relationships between characters as a main determinant for the narrative events that constitute each episode. We have developed a novel approach to interactive narrative in which this dependency is made explicit and can be used to control the generation of various episodes for a baseline drama. This is implemented in our demonstration system, NetworkING (social Network for Interactive Narrative Generation), which provides an interface that enables users to set the social relationships between virtual characters in order to create an episode of their choosing which they can watch as it is visualised as a 3D animation. The domain for the interactive narrative is a medical drama with a cast of virtual characters, such as doctors, nurses and patients. The use of the social network makes the relationships between characters visible and hence leads to the generation of narratives featuring shenanigans within the context of medical story lines.

AAMAS Conference 2011 Conference Paper

Controlling Narrative Time in Interactive Storytelling

• Julie Porteous
• Jonathan Teutenberg
• Fred Charles
• Marc Cavazza

Narrative time has an important role to play in Interactive Storytelling (IS). The prevailing approach to controlling narrative time has been to use implicit models that allow only limited temporal reasoning about virtual agent behaviour. In contrast, this paper proposes the use of an explicit model of narrative time which provides a control mechanism that enhances narrative generation, orchestration of virtual agents and number of possibilities for the staging of agent actions. This approach can help address a number of problems experienced in IS systems both at the level of execution staging and at the level of narrative generation. Consequently it has a number of advantages: it is more flexible with respect to the staging of virtual agent actions; it reduces the possibility of timing problems in the coordination of virtual agents; and it enables more expressive representation of narrative worlds and narrative generative power. Overall it provides a uniform, consistent, principled and rigorous approach to the problem of time in agent-based storytelling. In the paper we demonstrate how this approach to controlling narrative time can be implemented within an IS system and illustrate this using our fully implemented IS system that features virtual agents inspired by Shakespeare's The Merchant of Venice. The paper presents results of an experimental evaluation with the system that demonstrates the use of this approach to co-ordinate the actions of virtual agents and to increase narrative generative power.

AAMAS Conference 2011 Conference Paper

Interactive Storytelling with Temporal Planning

• Julie Porteous
• Jonathan Teutenberg
• Fred Charles
• Marc Cavazza

Narrative time has an important role to play in Interactive Storytelling (IS) systems. In contrast to prevailing IS approaches which use implicit models of time, in our work we have used an explicit model of narrative time. The goal of the demonstration IS system is to show how this explicit temporal representation and reasoning can help overcome certain problems experienced in IS systems such as the coordination of virtual agents and system inflexibility with respect to the staging of virtual agent actions. The fully implemented system features virtual agents and situations inspired by Shakespeare's play The Merchant of Venice.

ICAPS Conference 2011 Conference Paper

Visual Programming of Plan Dynamics Using Constraints and Landmarks

• Julie Porteous
• Jonathan Teutenberg
• David Pizzi
• Marc Cavazza

In recent years, there has been considerable interest in the use of planning techniques in the area of new media. Many traditional planning notions no longer apply in the context of these applications. In particular, it can be difficult to answer the important question of what constitutes a good plan for the domain, but there is an emerging consensus that plan dynamics play an important role. As a consequence, it is important to support representation of such aspects. Our solution is to introduce a meta-level of representation that is an abstraction of the domain with respect to both time and causality, and to develop a visual representation of this in the form of a narrative arc. This visual representation can then be used in a visual programming approach to the exploration and specification of plan dynamics. In the paper we outline this approach to meta-level representation using constraints along with the visual programming interface we have developed. We illustrate the approach with examples of visual programming in the development of an interactive entertainment system based on Shakespeare's play ``The Merchant of Venice''

TIST Journal 2010 Journal Article

Applying planning to interactive storytelling

• Julie Porteous
• Marc Cavazza
• Fred Charles

We have seen ten years of the application of AI planning to the problem of narrative generation in Interactive Storytelling (IS). In that time planning has emerged as the dominant technology and has featured in a number of prototype systems. Nevertheless key issues remain, such as how best to control the shape of the narrative that is generated (e.g., by using narrative control knowledge, i.e., knowledge about narrative features that enhance user experience) and also how best to provide support for real-time interactive performance in order to scale up to more realistic sized systems. Recent progress in planning technology has opened up new avenues for IS and we have developed a novel approach to narrative generation that builds on this. Our approach is to specify narrative control knowledge for a given story world using state trajectory constraints and then to treat these state constraints as landmarks and to use them to decompose narrative generation in order to address scalability issues and the goal of real-time performance in larger story domains. This approach to narrative generation is fully implemented in an interactive narrative based on the “Merchant of Venice.” The contribution of the work lies both in our novel use of state constraints to specify narrative control knowledge for interactive storytelling and also our development of an approach to narrative generation that exploits such constraints. In the article we show how the use of state constraints can provide a unified perspective on important problems faced in IS.

AAMAS Conference 2010 Conference Paper

How Was Your Day? A Companion ECA

• Marc Cavazza
• Raul Santos de la Camara
• Markku Turunen

We demonstrate a "Companion" ECA, which is able to provideadvice and support to the user, taking into account emotionsexpressed by her through dialogue. The integration of allrequired multimodal I/O components is based on interactionstrategies defining the shape of dialogue, on the ECA's responsetimes, and on the underlying affective strategy. The systemsupports free conversation on an everyday life scenario in whichthe user comments her day at the office.

ECAI Conference 2010 Conference Paper

Linear Logic for Non-Linear Storytelling

• Anne-Gwenn Bosser
• Marc Cavazza
• Ronan Champagnat

Whilst narrative representations have played a prominent role in AI research, there has been a renewed interest in the topic with the development of interactive narratives. A typical approach aims at generating narratives from baseline action representations, most often using planning techniques. However, this research has developed empirically, often as an application of planning. In this paper, we explore a more rigorous formalisation of narrative concepts, both at the action level and at the plot level. Our aim is to investigate how to bridge the gap between action descriptions and narrative concepts, by considering the latter from the perspective of resource consumption and causality. We propose to use Linear Logic, often introduced as a logic of resources, for it provides, through linear implication, a better description of causality than in Classical and Intuitionistic Logic. Besides advances in the fundamental principles of narrative formalisation, this approach can support the formal validation of scenario description as a preliminary step to their implementation via other computational formalisms.

AAMAS Conference 2010 Conference Paper

Multimodal Interaction with a Virtual Character in Interactive Storytelling

• Nikolaus Bee
• Johannes Wagner
• Elisabeth Andre
• Fred Charles
• David Pizzi
• Marc Cavazza

A number of interactive storytelling (IS) systems offer the user the possibility to input natural language input which determines how a story progresses. In this paper, we present an approach where the user's affective and attentive state mainly drives the evolution of the narrative. We introduce a framework for real-time signal processing (SSI) to analyze the users' state which then influences the feelings of the story characters and their actions in the story. SSI is not only used to enable more natural character responses that are sensitive to the user's state at runtime. In addition SSI offers the possibility to collect a large variety of synchronized user data which can be used to analyze the user's experience in offline mode. The underlying narrative in which the approach was tested is based on a classical XIX$^\mathrm{th}$ century psychological novel: Madame Bovary, by Flaubert.

AAMAS Conference 2010 Conference Paper

Narrative Generation through Characters' Point of View

• Julie Porteous
• Marc Cavazza
• Fred Charles

Virtual Actors are at the heart of Interactive Storytellingsystems and in recent years multiple approaches have beendescribed to specify their autonomous behaviour. One wellknown problem is how to achieve a balance between thecharacters' autonomy, defined in terms of their individualroles and motivations, and the global structure of the plot, which tends to emphasise narrative phenomena and the coordination of multiple characters. In this paper we report anew approach to the definition of virtual characters aimed atachieving a balance between character autonomy and globalplot structure. Where previous approaches have tended tofocus on individual actions our objective is to reincorporatehigher-level narrative elements in the behaviour of individual actors and address the relation between character andplot at the level of behaviour representation. To this endwe introduce the notion of a characters' Point of View andshow how it enables a story to be described from the perspective of a number of different characters: it is not merelya presentation effect it is also a different way to tell a story. As an illustration, we have developed an Interactive Narrative based on Shakespeare's Merchant of Venice. The system, which features a novel planning approach to story generation, can generate very different stories depending on thePoint of View adopted and support dynamic modification ofthe story world which results in different story consequences. In the paper, we illustrate this approach using example narratives generated using our fully implemented prototype.

AAMAS Conference 2009 Conference Paper

Emotional Input for Character-based Interactive Storytelling

• Marc Cavazza
• David Pizzi
• Fred Charles
• Thurid Vogt
• Elisabeth André

In most Interactive Storytelling systems, user interaction is based on natural language communication with virtual agents, either through isolated utterances or through dialogue. Natural language communication is also an essential element of interactive narratives in which the user is supposed to impersonate one of the story’s characters. Whilst techniques for narrative generation and agent behaviour have made significant progress in recent years, natural language processing remains a bottleneck hampering the scalability of Interactive Storytelling systems. In this paper, we introduce a novel interaction technique based solely on emotional speech recognition. It allows the user to take part in dialogue with virtual actors without any constraints on style or expressivity, by mapping the recognised emotional categories to narrative situations and virtual characters feelings. Our Interactive Storytelling system uses an emotional planner to drive characters’ behaviours. The main feature of this approach is that characters’ feelings are part of the planning domain and are at the heart of narrative representations. The emotional speech recogniser analyses the speech signal to produce a variety of features which can be used to define ad-hoc categories on which to train the system. The content of our interactive narrative is an adaptation of one chapter of the XIXth century classic novel, Madame Bovary, which is well suited to a formalisation in terms of characters’ feelings. At various stages of the narrative, the user can address the main character or respond to her, impersonating her lover. The emotional category extracted from the user utterance can be analysed in terms of the current narrative context, which includes characters’ beliefs, feelings and expectations, to produce a specific influence on the target character, which will become visible through a change in its behaviour, achieving a high level of realism for the interaction. A limited number of emotional categories is sufficient to drive the narrative across multiple courses of actions, since it comprises over thirty narrative functions. We report results from a fully implemented prototype, both in terms of proof of concept and of usability through a preliminary user study.

AAMAS Conference 2008 Conference Paper

A �Companion' ECA with Planning and Activity Modelling

• Marc Cavazza
• Cameron Smith
• Daniel Charlton
• Li Zhang
• Jaakko Hakulinen
• Markku Turunen

In this paper, we describe the development of an Embodied Conversational Agent (ECA) implementing the concept of a companion, i. e. an agent supporting the persistent representation of user activities and dialogue-based communication with the user. This first experiment implements a Health and Fitness companion aimed at promoting a healthier lifestyle. The system operates by generating an ‘ideal’ plan of daily activities from background knowledge and dialogue interaction with the user. This plan then becomes an activity model, which will later be instantiated by reports from the user and analysed by the agent from the perspective of initial objectives. At various stages of the day, the plan can still be adapted through further dialogue. The agent is embodied using a wireless rabbit (Nabaztag™) device situated in the user’s home. After describing the planning component, based on Hierarchical Task Networks (HTN) and the spoken dialogue system, we present a working example from the system illustrating its behaviour through various phases of user activity generation, updating and re-planning.

AAMAS Conference 2008 Conference Paper

An Embodied Conversational Agent as a Lifestyle Advisor

• Cameron Smith
• Daniel Charlton
• Li Zhang
• Jaakko Hakulinen
• Markku Turunen
• Marc Cavazza

Persistent Embodied Conversational Agents (ECA) can be used to assist users in their daily activities. We introduce the Health and Fitness Companion (HFC), which is a conversational system aimed at promoting a healthier lifestyle. The system is embodied using the NabaztagTM device, a wireless plastic rabbit supporting multimodal input and output. The HFC integrates a spoken dialogue system previously developed by some of the authors, and a cognitive model, based on Hierarchical Task Network (HTN) planning, which enables the ECA to reason on the user activities and generate plans for recommended activities. A typical demonstration scenario consists in two short conversation sessions between the HFC and the user. During the first the HFC helps the user to plan her day ahead, while the second involves reporting the activities actually carried out. The system has undergone early tests with generic users, which have demonstrated its usability and stability (89% successful dialogue completions).

AAMAS Conference 2008 Conference Paper

Emotional Reading of Medical Texts Using Conversational Agents

• Gersende Georg
• Catherine Pelachaud
• Marc Cavazza

In this paper, we present a prototype that helps visualizing the relative importance of sentences extracted from medical texts using Embodied Conversational Agents (ECA). We propose to map rhetorical structures automatically recognized in the documents onto a set of communicative acts controlling the expression of an ECA. As a consequence, the ECA will dramatize a sentence to reflect its perceived importance and rhetorical strength (advice, requirement, open proposal, etc). This prototype is constituted of three sub-systems: i) G-DEE, a text analysis module ii) a mapping module which converts rhetorical structures produced by the text analysis module into communicative functions driving the ECA animation and iii) an ECA system. By bringing the text to life, this system could help their authors (in our application, expert physicians) to reflect on the potential impact of the writing style they have adopted. The use of ECA reintroduces an affective element which cannot easily be captured by other methods for analyzing document’s style.

AAMAS Conference 2007 Conference Paper

Extending Character-based Storytelling with Awareness and Feelings

• David Pizzi
• Marc Cavazza
• Jean-Luc Lugrin

Most Interactive Storytelling systems developed to date have followed a task-based approach to story representation, using planning techniques to drive the story by generating a sequence of actions, which essentially "solve" the task to which the story is equated. One major limitation of this approach has been that it fails to incorporate characters' psychology, and as a consequence important aesthetic aspects of the narrative cannot be easily captured by Interactive Storytelling. In this paper, we introduce a new approach to Interactive Storytelling, which aims at reconciling narrative actions with the characters' attributed psychology as stated in the narrative. Our long-term goal is to be able to explore Interactive Storytelling for those narrative genres which are based on the characters' psychology rather than solely on their actions. We used as a starting point the formalisation by Flaubert himself of his novel Madame Bovary, which includes a detailed account of characters' desires and feelings. We describe a prototype in which characters' behaviour is driven by a real-time search-based planning system applying operators whose content is based on a specific inventory of feelings. Furthermore, the actual pattern of evolution of the character's plan, as measured through the variation of the search heuristic, is used to confer a sense of awareness to the characters, which can be used to generate feelings about its overall situation, from feelings of boredom to hope.

AAMAS Conference 2004 Conference Paper

Exploring the Scalability of Character-based Storytelling

• Fred Charles
• Marc Cavazza

AAMAS Conference 2002 Conference Paper

Interacting with Virtual Characters in Interactive Storytelling

• Marc Cavazza
• Fred Charles
• Steven J. Mead