Moninder Singh

AAAI Conference 2024 Conference Paper

SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models

• Manish Nagireddy
• Lamogha Chiazor
• Moninder Singh
• Ioana Baldini

Current datasets for unwanted social bias auditing are limited to studying protected demographic features such as race and gender. In this work, we introduce a comprehensive benchmark that is meant to capture the amplification of social bias, via stigmas, in generative language models. Taking inspiration from social science research, we start with a documented list of 93 US-centric stigmas and curate a question-answering (QA) dataset which involves simple social situations. Our benchmark, SocialStigmaQA, contains roughly 10K prompts, with a variety of prompt styles, carefully constructed to systematically test for both social bias and model robustness. We present results for SocialStigmaQA with two open source generative language models and we find that the proportion of socially biased output ranges from 45% to 59% across a variety of decoding strategies and prompting styles. We demonstrate that the deliberate design of the templates in our benchmark (e.g., adding biasing text to the prompt or using different verbs that change the answer that indicates bias) impacts the model tendencies to generate socially biased output. Additionally, through manual evaluation, we discover problematic patterns in the generated chain-of-thought output that range from subtle bias to lack of reasoning. Warning: This paper contains examples of text which are toxic, biased, and potentially harmful.

NeurIPS Conference 2022 Conference Paper

On the Safety of Interpretable Machine Learning: A Maximum Deviation Approach

• Dennis Wei
• Rahul Nair
• Amit Dhurandhar
• Kush R. Varshney
• Elizabeth Daly
• Moninder Singh

Interpretable and explainable machine learning has seen a recent surge of interest. We focus on safety as a key motivation behind the surge and make the relationship between interpretability and safety more quantitative. Toward assessing safety, we introduce the concept of maximum deviation via an optimization problem to find the largest deviation of a supervised learning model from a reference model regarded as safe. We then show how interpretability facilitates this safety assessment. For models including decision trees, generalized linear and additive models, the maximum deviation can be computed exactly and efficiently. For tree ensembles, which are not regarded as interpretable, discrete optimization techniques can still provide informative bounds. For a broader class of piecewise Lipschitz functions, we leverage the multi-armed bandit literature to show that interpretability produces tighter (regret) bounds on the maximum deviation. We present case studies, including one on mortgage approval, to illustrate our methods and the insights about models that may be obtained from deviation maximization.

AAAI Conference 2021 Conference Paper

Anomaly Attribution with Likelihood Compensation

• Tsuyoshi Idé
• Amit Dhurandhar
• Jiří Navrátil
• Moninder Singh
• Naoki Abe

This paper addresses the task of explaining anomalous predictions of a black-box regression model. When using a black-box model, such as one to predict building energy consumption from many sensor measurements, we often have a situation where some observed samples may significantly deviate from their prediction. It may be due to a sub-optimal black-box model, or simply because those samples are outliers. In either case, one would ideally want to compute a “responsibility score” indicative of the extent to which an input variable is responsible for the anomalous output. In this work, we formalize this task as a statistical inverse problem: Given model deviation from the expected value, infer the responsibility score of each of the input variables. We propose a new method called likelihood compensation (LC), which is founded on the likelihood principle and computes a correction to each input variable. To the best of our knowledge, this is the first principled framework that computes a responsibility score for real valued anomalous model deviations. We apply our approach to a real-world building energy prediction task and confirm its utility based on expert feedback.

JMLR Journal 2020 Journal Article

AI Explainability 360: An Extensible Toolkit for Understanding Data and Machine Learning Models

• Vijay Arya
• Rachel K. E. Bellamy
• Pin-Yu Chen
• Amit Dhurandhar
• Michael Hind
• Samuel C. Hoffman
• Stephanie Houde
• Q. Vera Liao

As artificial intelligence algorithms make further inroads in high-stakes societal applications, there are increasing calls from multiple stakeholders for these algorithms to explain their outputs. To make matters more challenging, different personas of consumers of explanations have different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360, an open-source Python toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of interpretation and explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. The toolkit is not only the software, but also guidance material, tutorials, and an interactive web demo to introduce AI explainability to different audiences. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2020. ( edit, beta )

IJCAI Conference 2019 Conference Paper

Teaching AI Agents Ethical Values Using Reinforcement Learning and Policy Orchestration

• Ritesh Noothigattu
• Djallel Bouneffouf
• Nicholas Mattei
• Rachita Chandra
• Piyush Madan
• Kush R. Varshney
• Murray Campbell
• Moninder Singh

Autonomous cyber-physical agents play an increasingly large role in our lives. To ensure that they behave in ways aligned with the values of society, we must develop techniques that allow these agents to not only maximize their reward in an environment, but also to learn and follow the implicit constraints of society. We detail a novel approach that uses inverse reinforcement learning to learn a set of unspecified constraints from demonstrations and reinforcement learning to learn to maximize environmental rewards. A contextual bandit-based orchestrator then picks between the two policies: constraint-based and environment reward-based. The contextual bandit orchestrator allows the agent to mix policies in novel ways, taking the best actions from either a reward-maximizing or constrained policy. In addition, the orchestrator is transparent on which policy is being employed at each time step. We test our algorithms using Pac-Man and show that the agent is able to learn to act optimally, act within the demonstrated constraints, and mix these two functions in complex ways.

AAAI Conference 2018 Conference Paper

Assessing National Development Plans for Alignment With Sustainable Development Goals via Semantic Search

• Jonathan Galsurkar
• Moninder Singh
• Lingfei Wu
• Aditya Vempaty
• Mikhail Sushkov
• Devika Iyer
• Serge Kapto
• Kush Varshney

The United Nations Development Programme (UNDP) helps countries implement the United Nations (UN) Sustainable Development Goals (SDGs), an agenda for tackling major societal issues such as poverty, hunger, and environmental degradation by the year 2030. A key service provided by UNDP to countries that seek it is a review of national development plans and sector strategies by policy experts to assess alignment of national targets with one or more of the 169 targets of the 17 SDGs. Known as the Rapid Integrated Assessment (RIA), this process involves manual review of hundreds, if not thousands, of pages of documents and takes weeks to complete. In this work, we develop a natural language processing-based methodology to accelerate the work- flow of policy experts. Specifically we use paragraph embedding techniques to find paragraphs in the documents that match the semantic concepts of each of the SDG targets. One novel technical contribution of our work is in our use of historical RIAs from other countries as a form of neighborhoodbased supervision for matches in the country under study. We have successfully piloted the algorithm to perform the RIA for Papua New Guinea’s national plan, with the UNDP estimating it will help reduce their completion time from an estimated 3-4 weeks to 3 days.

AAAI Conference 1997 Conference Paper

Learning Bayesian Networks from Incomplete Data

• Moninder Singh

Much of the current research in learning Bayesian Networks fails to effectively deal with missing data. Most of the methods assume that the data is complete, or make the data complete using fairly ad-hoc methods; other methods do deal with missing data but learn only the conditional probabilities, assuming that the structure is known. We present a principled approach to learn both the Bayesian network structure as well as the conditional probabilities from incomplete data. The proposed algorithm is an iterative method that uses a combination of Expectation-Maximization (EM) and Imputation techniques. Results are presented on synthetic data sets which show that the performance of the new algorithm is much better than ad-hoc methods for handling missing data.

AAAI Conference 1996 Short Paper

Dynamic Constraint-Based Planning in Trauma Management

• Moninder Singh

This research deals with planning in domains with dynamically changing, multiple, interacting goals. What distinguishes this work from reactive planners (e.g. (Firby 1987)) is the fact that the goals for which planning is done are not known in advance; rather, goals are formed and change rapidly during the planning process itself. Although planners that produce appropriate plans exist for such domains (Rymon et al. 1993), we want a planner that also provides a basis for explaining why some action is chosen over another or why some goal is no longer relevant etc., which is necessary for effective decision support (Gertner 1994).

ICML Conference 1996 Conference Paper

Efficient Learning of Selective Bayesian Network Classifiers

• Moninder Singh
• Gregory M. Provan

AAAI Conference 1996 Short Paper

Induction of Selective Bayesian Networks from Data

• Moninder Singh

Bayesian networks (Pearl 1988), which provide a compact graphical way to express complex probabilistic relationships among several random variables, are rapidly becoming the tool of choice for dealing with uncertainty in knowledge based systems. Amongst the many advantages offered by Bayesian networks over other representations such as decision trees and neural networks are the ease of comprehensibility to humans, effectiveness as complex decision making models and elicitability of informative prior distributions.

ICML Conference 1996 Conference Paper

Learning Goal Oriented Bayesian Networks for Telecommunications Risk Management

• Kazuo J. Ezawa
• Moninder Singh
• Steven W. Norton

ICML Conference 1995 Conference Paper

A Comparison of Induction Algorithms for Selective and non-Selective Bayesian Classifiers

• Moninder Singh
• Gregory M. Provan

UAI Conference 1993 Conference Paper

An Algorithm for the Construction of Bayesian Network Structures from Data

• Moninder Singh
• Marco Valtorta

Previous algorithms for the construction of Bayesian belief network structures from data have been either highly dependent on conditional independence (CI) tests, or have required an ordering on the nodes to be supplied by the user. We present an algorithm that integrates these two approaches - CI tests are used to generate an ordering on the nodes from the database which is then used to recover the underlying Bayesian network structure using a non CI based method. Results of preliminary evaluation of the algorithm on two networks (ALARM and LED) are presented. We also discuss some algorithm performance issues and open problems.