Author name cluster

Nitesh Chawla

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

1 author row

NeurIPS Conference 2025 Conference Paper

BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks

Anna Sokol
Elizabeth Daly
Michael Hind
David Piorkowski
Xiangliang Zhang
Nuno Moniz
Nitesh Chawla

Large language models (LLMs) are powerful tools capable of handling diverse tasks. Comparing and selecting appropriate LLMs for specific tasks requires systematic evaluation methods, as models exhibit varying capabilities across different domains. However, finding suitable benchmarks is difficult given the many available options. This complexity not only increases the risk of benchmark misuse and misinterpretation but also demands substantial effort from LLM users, seeking the most suitable benchmarks for their specific needs. To address these issues, we introduce BenchmarkCards, an intuitive and validated documentation framework that standardizes critical benchmark attributes such as objectives, methodologies, data sources, and limitations. Through user studies involving benchmark creators and users, we show that BenchmarkCards can simplify benchmark selection and enhance transparency, facilitating informed decision-making in evaluating LLMs. Data & Code: github. com/SokolAnn/BenchmarkCards huggingface. co/datasets/ASokol/BenchmarkCards

PDF Details

NeurIPS Conference 2025 Conference Paper

ChemOrch: Empowering LLMs with Chemical Intelligence via Groundbreaking Synthetic Instructions

Yue Huang
Zhengzhe Jiang
Xiaonan Luo
Kehan Guo
Haomin Zhuang
Yujun Zhou
Zhengqing Yuan
Xiaoqi Sun

Empowering large language models (LLMs) with chemical intelligence remains a challenge due to the scarcity of high-quality, domain-specific instruction-response datasets and the misalignment of existing synthetic data generation pipelines with the inherently hierarchical and rule-governed structure of chemical information. To address this, we propose ChemOrch, a framework that synthesizes chemically grounded instruction–response pairs through a two-stage process: task-controlled instruction generation and tool-aware response construction. ChemOrch enables controllable diversity and levels of difficulty for the generated tasks and ensures response precision through tool planning & distillation, and tool-based self-repair mechanisms. The effectiveness of ChemOrch is evaluated based on: 1) the \textbf{high quality} of generated instruction data, demonstrating superior diversity and strong alignment with chemical constraints; 2) the \textbf{dynamic generation of evaluation tasks} that more effectively reveal LLM weaknesses in chemistry; and 3) the significant \textbf{improvement of LLM chemistry capabilities} when the generated instruction data are used for fine-tuning. Our work thus represents a critical step toward scalable and verifiable chemical intelligence in LLMs. The code is available at \url{https: //anonymous. 4open. science/r/ChemOrch-854A}.

PDF Details

NeurIPS Conference 2023 Conference Paper

What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks

Taicheng Guo
Kehan Guo
Bozhao Nan
Zhenwen Liang
Zhichun Guo
Nitesh Chawla
Olaf Wiest
Xiangliang Zhang

Large Language Models (LLMs) with strong abilities in natural language processing tasks have emerged and have been applied in various kinds of areas such as science, finance and software engineering. However, the capability of LLMs to advance the field of chemistry remains unclear. In this paper, rather than pursuing state-of-the-art performance, we aim to evaluate capabilities of LLMs in a wide range of tasks across the chemistry domain. We identify three key chemistry-related capabilities including understanding, reasoning and explaining to explore in LLMs and establish a benchmark containing eight chemistry tasks. Our analysis draws on widely recognized datasets facilitating a broad exploration of the capacities of LLMs within the context of practical chemistry. Five LLMs (GPT-4, GPT-3. 5, Davinci-003, Llama and Galactica) are evaluated for each chemistry task in zero-shot and few-shot in-context learning settings with carefully selected demonstration examples and specially crafted prompts. Our investigation found that GPT-4 outperformed other models and LLMs exhibit different competitive levels in eight chemistry tasks. In addition to the key findings from the comprehensive benchmark analysis, our work provides insights into the limitation of current LLMs and the impact of in-context learning settings on LLMs’ performance across various chemistry tasks. The code and datasets used in this study are available at https: //github. com/ChemFoundationModels/ChemLLMBench.

PDF Details

AAAI Conference 2020 Conference Paper

Graph Few-Shot Learning via Knowledge Transfer

Huaxiu Yao
Chuxu Zhang
Ying Wei
Meng Jiang
Suhang Wang
Junzhou Huang
Nitesh Chawla
Zhenhui Li

Towards the challenging problem of semi-supervised node classiﬁcation, there have been extensive studies. As a frontier, Graph Neural Networks (GNNs) have aroused great interest recently, which update the representation of each node by aggregating information of its neighbors. However, most GNNs have shallow layers with a limited receptive ﬁeld and may not achieve satisfactory performance especially when the number of labeled nodes is quite small. To address this challenge, we innovatively propose a graph few-shot learning (GFL) algorithm that incorporates prior knowledge learned from auxiliary graphs to improve classiﬁcation accuracy on the target graph. Speciﬁcally, a transferable metric space characterized by a node embedding and a graph-speciﬁc prototype embedding function is shared between auxiliary graphs and the target, facilitating the transfer of structural knowledge. Extensive experiments and ablation studies on four real-world graph datasets demonstrate the effectiveness of our proposed model and the contribution of each component.

PDF Details

AAAI Conference 2007 Conference Paper

Actively Exploring Creation of Face Space(s) for Improved Face Recognition

Nitesh Chawla

We propose a learning framework that actively explores creation of face space(s) by selecting images that are complementary to the images already represented in the face space. We also construct ensembles of classiﬁers learned from such actively sampled image sets, which further provides improvement in the recognition rates. We not only signiﬁcantly reduce the number of images required in the training set but also improve the accuracy over learning from all the images. We also show that the single face space or ensemble of face spaces, thus constructed, has a higher generalization performance across different illumination and expression conditions.

PDF Details