Riccardo Sartea

EUMAS Conference 2020 Conference Paper

A Game of Double Agents: Repeated Stackelberg Games with Role Switch

• Matteo Murari
• Alessandro Farinelli
• Riccardo Sartea

Abstract We introduce a novel variation of the widely used 2-player Stackelberg game formalism. In our variation, a master player can decide to act as a leader or as a follower across the iterations of the game. This model naturally arises in many real-world applications and particularly in cyber-security scenarios, where an analyzer agent can arbitrarily decide which role to play in each iteration. We propose a first solution approach for this model assuming bounded rationality for the players and adopting a Monte Carlo Tree Search approach to devise the analyzer’s strategy. We empirically show the effectiveness of our method in two experimental domains, i. e. synthetic game instances (using randomly generated games) and malware analysis (using real malware samples).

EAAI Journal 2020 Journal Article

SECUR-AMA: Active Malware Analysis Based on Monte Carlo Tree Search for Android Systems

• Riccardo Sartea
• Alessandro Farinelli
• Matteo Murari

AAMAS Conference 2019 Conference Paper

Agent Behavioral Analysis Based on Absorbing Markov Chains

• Riccardo Sartea
• Alessandro Farinelli
• Matteo Murari

We propose a novel technique to identify known behaviors of intelligent agents acting within uncertain environments. We employ Markov chains to represent the observed behavioral models of the agents and we formulate the problem as a classification task. In particular, we propose to use the long-term transition probability values of moving between states of the Markov chain as features. Additionally, we transform our models into absorbing Markov chains, enabling the use of standard techniques to compute such features. The empirical evaluation considers two scenarios: the identification of given strategies in classical games, and the detection of malicious behaviors in malware analysis. Results show that our approach can provide informative features to successfully identify known behavioral patterns. In more detail, we show that focusing on the long-term transition probability enables to diminish the error introduced by noisy states and transitions that may be present in an observed behavioral model. We pose particular attention to the case of noise that may be intentionally introduced by a target agent to deceive an observer agent.

AAMAS Conference 2019 Conference Paper

eXplainable Modeling (XM): Data Analysis for Intelligent Agents

• Alberto Castellini
• Francesco Masillo
• Riccardo Sartea
• Alessandro Farinelli

Intelligent agents perform key tasks in several application domains by processing sensor data and taking actions that maximize reward functions based on internal models of the environment and the agent itself. In this paper we present eXplainable Modeling (XM), a Python software which supports data analysis for intelligent agents. XM enables to analyze state-models, namely models of the agent states, discovered from sensor traces by data-driven methods, and to interpret them for improved situation awareness. The main features of the tool are described through the analysis of a real case study concerning aquatic drones for water monitoring.

AAMAS Conference 2018 Conference Paper

Detection of Intelligent Agent Behaviors Using Markov Chains

• Riccardo Sartea
• Alessandro Farinelli

We consider the problem of detecting the behavior of intelligent agents operating in stochastic environments. In particular, we focus on a scenario where we are given two models for agent behaviors and we are interested in detecting whether one model appears within the other model. We use Markov chains to represent the behavioral models of the agents and we propose to extract the long-run probabilities as features that can be used to detect if one model is contained in the other. Results show that our approach is capable of detecting known strategies for agents interacting within classical games and to categorize malware behaviors.

IJCAI Conference 2017 Conference Paper

A Monte Carlo Tree Search approach to Active Malware Analysis

• Riccardo Sartea
• Alessandro Farinelli

Active Malware Analysis (AMA) focuses on acquiring knowledge about dangerous software by executing actions that trigger a response in the malware. A key problem for AMA is to design strategies that select most informative actions for the analysis. To devise such actions, we model AMA as a stochastic game between an analyzer agent and a malware sample, and we propose a reinforcement learning algorithm based on Monte Carlo Tree Search. Crucially, our approach does not require a pre-specified malware model but, in contrast to most existing analysis techniques, we generate such model while interacting with the malware. We evaluate our solution using clustering techniques on models generated by analyzing real malware samples. Results show that our approach learns faster than existing techniques even without any prior information on the samples.