AAMAS Conference 2026 Conference Paper
On the Trade-Off Between Transparency and Security in Adversarial Machine Learning
- Lucas Fenaux
- Christopher Srinivasa
- Florian Kerschbaum
Transparency and security are both central to Responsible AI, but they may conflict in adversarial settings. We investigate the strategiceffectoftransparencyforagentsthroughthelensoftransferable adversarial example attacks. In transferable adversarial example attacks, attackers maliciously perturb their inputs using surrogate models to fool a defender’s target model. These models can be defended or undefended, with both players having to decide which to use. Using a large-scale empirical evaluation of nine attacks across 181 models, we find that attackers are more successful when they match the defender’s decision; hence, obscurity could be beneficial to the defender. With game theory, we analyze this trade-off between transparency and security by modeling this problem as both a Nash game and a Stackelberg game, and comparing the expected outcomes. Our analysis confirms that only knowing whether a defender’s model is defended or not can sometimes be enough to damage its security. This result serves as an indicator of the general trade-off between transparency and security, suggesting that transparency in AI systems can be at odds with security. Beyond adversarialmachinelearning, ourworkillustrateshowgame-theoretic reasoning can uncover conflicts between transparency and security.