Author name cluster

Nuno Moniz

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks

Anna Sokol
Elizabeth Daly
Michael Hind
David Piorkowski
Xiangliang Zhang
Nuno Moniz
Nitesh Chawla

Large language models (LLMs) are powerful tools capable of handling diverse tasks. Comparing and selecting appropriate LLMs for specific tasks requires systematic evaluation methods, as models exhibit varying capabilities across different domains. However, finding suitable benchmarks is difficult given the many available options. This complexity not only increases the risk of benchmark misuse and misinterpretation but also demands substantial effort from LLM users, seeking the most suitable benchmarks for their specific needs. To address these issues, we introduce BenchmarkCards, an intuitive and validated documentation framework that standardizes critical benchmark attributes such as objectives, methodologies, data sources, and limitations. Through user studies involving benchmark creators and users, we show that BenchmarkCards can simplify benchmark selection and enhance transparency, facilitating informed decision-making in evaluating LLMs. Data & Code: github. com/SokolAnn/BenchmarkCards huggingface. co/datasets/ASokol/BenchmarkCards

PDF Details

IJCAI Conference 2025 Conference Paper

Fast Explanations via Policy Gradient-Optimized Explainer

Deng Pan
Nuno Moniz
Nitesh V. Chawla

The challenge of delivering efficient explanations is a critical barrier that prevents the adoption of model explanations in real-world applications. Existing approaches often depend on extensive model queries for sample-level explanations or rely on expert's knowledge of specific model structures that trade general applicability for efficiency. To address these limitations, this paper introduces a novel framework Fast EXplanation (FEX) that represents attribution-based explanations via probability distributions, which are optimized by leveraging the policy gradient method. The proposed framework offers a robust, scalable solution for real-time, large-scale model explanations, bridging the gap between efficiency and applicability. We validate our framework on image and text classification tasks and the experiments demonstrate that our method reduces inference time by over 97 percent and memory usage by 70 percent compared to traditional model-agnostic approaches while maintaining high-quality explanations and broad applicability.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge

Jiayi Ye
Yanbo Wang 0005
Yue Huang 0001
Dongping Chen
Qihui Zhang
Nuno Moniz
Tian Gao
Werner Geyer

LLM-as-a-Judge has been widely utilized as an evaluation method in various benchmarks and served as supervised rewards in model training. However, despite their excellence in many domains, potential issues are under-explored, undermining their reliability and the scope of their utility. Therefore, we identify 12 key potential biases and propose a new automated bias quantification framework—CALM—which systematically quantifies and analyzes each type of bias in LLM-as-a-Judge by using automated and principle-guided modification. Our experiments cover multiple popular language models, and the results indicate that while advanced models have achieved commendable overall performance, significant biases persist in certain specific tasks. Empirical results suggest that there remains room for improvement in the reliability of LLM-as-a-Judge. Moreover, we also discuss the explicit and implicit influence of these biases and give some suggestions for the reliable application of LLM-as-a-Judge. Our work highlights the need for stakeholders to address these issues and remind users to exercise caution in LLM-as-a-Judge applications.

Details

IJCAI Conference 2025 Conference Paper

Leveraging Artificial Intelligence to Bridge Gaps in Pediatric Oncology Care for Marginalized Spanish-Speaking Communities

Grigorii Khvatskii
Angelica Garcia Martinez
Deng Pan
Matthew Belcher
Gerónimo Medrano Loera
Dayana Pineda Pérez
Juan Emmanuel Ferrari Muñoz-Ledo
Horacio Márquez-González

In low-and middle-income countries (LMICs) pediatric cancer patients and their caregivers often suffer from effects of underfunded, fragmented and outdated healthcare systems. One of these effects is a breakdown of communication between hospital staff and caregivers, which is felt stronger among vulnerable populations. Our proposed solution integrates Large Language Models (LLM) and Automatic Speech Recognition (ASR) technologies to enhance communication between caregivers and healthcare providers while integrating community feedback. We combine cutting-edge technology with existing hospital infrastructure to allow for easy deployment and testing. The system will improve access to health, nutrition, and parental care programs, prioritizing caregiver engagement and real-time interaction. Ultimately, our system will pave the way to more equitable access to medical care, and address structural barriers affecting marginalized communities.

PDF Details DOI

IJCAI Conference 2025 Conference Paper

Towards Fairness with Limited Demographics via Disentangled Learning

Zichong Wang
Anqi Wu
Nuno Moniz
Shu Hu
Bart Knijnenburg
Xingquan Zhu
Wenbin Zhang

Fairness in artificial intelligence has garnered increasing attention due to concerns about discriminatory AI-based decision-making, prompting the development of numerous mitigation approaches. However, most existing methods assume that demographic information is readily available, which may not align with real-world scenarios where such information is often incomplete. To this end, this paper tackles the pervasive yet overlooked challenge of developing fair machine learning algorithms with limited demographics. Specifically, we explore leveraging limited demographic information to accurately infer missing demographics while simultaneously evaluating and optimizing model fairness. We argue that this approach better aligns with common real-world socially sensitive scenarios involving limited demographics. Extensive experiments on three benchmark datasets highlight the effectiveness of the proposed method, surpassing state-of-the-art with significant gains in fairness while maintaining comparable utility.

PDF Details DOI