Author name cluster

Nils Holzenberger

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

AAAI Conference 2026 Conference Paper

Can LLMs Identify Tax Abuse?

Andrew Blair-Stanek
Nils Holzenberger
Benjamin Van Durme

We investigate whether large language models can discover and analyze U.S. tax-minimization strategies. This real-world domain challenges even seasoned human experts, and progress can reduce tax revenue lost from well-advised, wealthy taxpayers. We evaluate the most advanced LLMs on their ability to (1) interpret and verify tax strategies, (2) fill in gaps in partially specified strategies, and (3) generate complete, end-to-end strategies from scratch. This domain should be of particular interest to the LLM reasoning community: unlike synthetic challenge problems or scientific reasoning tasks, U.S. tax law involves navigating hundreds of thousands of pages of statutes, case law, and administrative guidance, all updated regularly. Notably, an LLM identified an apparently novel tax strategy, highlighting these models' potential to revolutionize tax agencies' fight against tax abuse.

PDF Details DOI

AAAI Conference 2026 Conference Paper

Language Models and Logic Programs for Trustworthy Tax Reasoning

William Jurayj
Nils Holzenberger
Benjamin Van Durme

According to the United States Internal Revenue Service, "the average American spends $270 and 13 hours filing their taxes". Even beyond the U.S., tax filing requires complex reasoning, combining application of overlapping rules with numerical calculations. Because errors can incur costly penalties, any automated system must deliver high accuracy and auditability, making modern large language models (LLMs) poorly suited for this task. We propose an approach that integrates LLMs with a symbolic solver to calculate tax obligations. We evaluate variants of this system on the challenging StAtutory Reasoning Assessment (SARA) dataset, and include a novel method for estimating the cost of deploying such a system based on real-world penalties for tax errors. We further show how combining up-front translation of plain-text rules into formal logic programs, combined with intelligently retrieved exemplars for formal case representations, can dramatically improve performance on this task and reduce costs to well below real-world averages. Our results demonstrate the effectiveness of applying semantic parsing methods to statutory reasoning, and show promising economic feasibility of neuro-symbolic architectures for increasing access to reliable tax assistance.

PDF Details DOI

EUMAS Conference 2024 Conference Paper

Rules2Lab: from Prolog Knowledge-Base, to Learning Agents, to Norm Engineering

Peter Fratric
Nils Holzenberger
David Restrepo Amariles

Abstract This paper proposes a methodology, called Rules2Lab, that maps a Prolog knowledge base onto a Gymnasium environment. States, actions, and constraints are defined in Prolog, while temporal, computational, and sub-symbolic operations are delegated to Python. We demonstrate our approach through a case study on privacy vulnerabilities in a data marketplace. In our simulation, a reinforcement learning agent attempts to access sensitive data, with a privacy breach defined by the similarity between inferred and private data. Inductive logic programming is then used to engineer a new norm that prevents such breaches, demonstrating how a new rule can be seamlessly integrated into the knowledge base. Preliminary results highlight how a Gymnasium environment can be effectively combined with logic-based modeling and inference.

Details

NeurIPS Conference 2023 Conference Paper

LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models

Neel Guha
Julian Nyarko
Daniel Ho
Christopher Ré
Adam Chilton
Aditya K
Alex Chohlas-Wood
Austin Peters

The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform? To enable greater study of this question, we present LegalBench: a collaboratively constructed legal reasoning benchmark consisting of 162 tasks covering six different types of legal reasoning. LegalBench was built through an interdisciplinary process, in which we collected tasks designed and hand-crafted by legal professionals. Because these subject matter experts took a leading role in construction, tasks either measure legal reasoning capabilities that are practically useful, or measure reasoning skills that lawyers find interesting. To enable cross-disciplinary conversations about LLMs in the law, we additionally show how popular legal frameworks for describing legal reasoning—which distinguish between its many forms—correspond to LegalBench tasks, thus giving lawyers and LLM developers a common vocabulary. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.

PDF Details