Noam Slonim

AAAI Conference 2023 System Paper

nBIIG: A Neural BI Insights Generation System for Table Reporting

• Yotam Perlitz
• Dafna Sheinwald
• Noam Slonim
• Michal Shmueli-Scheuer

We present nBIIG, a neural Business Intelligence (BI) Insights Generation system. Given a table, our system applies various analyses to create corresponding RDF representations, and then uses a neural model to generate fluent textual insights out of these representations. The generated insights can be used by an analyst, via a human-in-the-loop paradigm, to enhance the task of creating compelling table reports. The underlying generative neural model is trained over large and carefully distilled data, curated from multiple BI domains. Thus, the system can generate faithful and fluent insights over open-domain tables, making it practical and useful.

AAAI Conference 2022 Conference Paper

Fortunately, Discourse Markers Can Enhance Language Models for Sentiment Analysis

• Liat Ein-Dor
• Ilya Shnayderman
• Artem Spector
• Lena Dankin
• Ranit Aharonov
• Noam Slonim

In recent years, pretrained language models have revolutionized the NLP world, while achieving state-of-the-art performance in various downstream tasks. However, in many cases, these models do not perform well when labeled data is scarce and the model is expected to perform in the zero or few shot setting. Recently, several works have shown that continual pretraining or performing a second phase of pretraining (inter-training), which is better aligned with the downstream task, can lead to improved results, especially in the scarce data setting. Here, we propose to leverage sentiment-carrying discoursemarkers to generate large-scale weakly-labeled data, which in turn can be used to adapt general-purpose language models to the task of sentiment classification. In addition, we propose a new method for adapting sentiment classification models to new domains. This method is based on automatic identification of domain-specific sentiment-carrying discourse markers. Extensive experimental results show the value of our approach on various benchmark datasets. Code, models and data are available at https: //github. com/ibm/tslm-discourse-markers.

AAAI Conference 2020 Conference Paper

A Large-Scale Dataset for Argument Quality Ranking: Construction and Analysis

• Shai Gretz
• Roni Friedman
• Edo Cohen-Karlik
• Assaf Toledo
• Dan Lahav
• Ranit Aharonov
• Noam Slonim

Identifying the quality of free-text arguments has become an important task in the rapidly expanding field of computational argumentation. In this work, we explore the challenging task of argument quality ranking. To this end, we created a corpus of 30, 497 arguments carefully annotated for point-wise quality, released as part of this work. To the best of our knowledge, this is the largest dataset annotated for point-wise argument quality, larger by a factor of five than previously released datasets. Moreover, we address the core issue of inducing a labeled score from crowd annotations by performing a comprehensive evaluation of different approaches to this problem. In addition, we analyze the quality dimensions that characterize this dataset. Finally, we present a neural method for argument quality ranking, which outperforms several baselines on our own dataset, as well as previous methods published for another dataset.

AAAI Conference 2020 Conference Paper

Corpus Wide Argument Mining—A Working Solution

• Liat Ein-Dor
• Eyal Shnarch
• Lena Dankin
• Alon Halfon
• Benjamin Sznajder
• Ariel Gera
• Carlos Alzate
• Martin Gleize

One of the main tasks in argument mining is the retrieval of argumentative content pertaining to a given topic. Most previous work addressed this task by retrieving a relatively small number of relevant documents as the initial source for such content. This line of research yielded moderate success, which is of limited use in a real-world system. Furthermore, for such a system to yield a comprehensive set of relevant arguments, over a wide range of topics, it requires leveraging a large and diverse corpus in an appropriate manner. Here we present a first end-to-end high-precision, corpus-wide argument mining system. This is made possible by combining sentence-level queries over an appropriate indexing of a very large corpus of newspaper articles, with an iterative annotation scheme. This scheme addresses the inherent label bias in the data and pinpoints the regions of the sample space whose manual labeling is required to obtain high-precision among top-ranked candidates.

IJCAI Conference 2013 Conference Paper

Hartigan's K-Means Versus Lloyd's K-Means — Is It Time for a Change?

• Noam Slonim
• Ehud Aharoni
• Koby Crammer

Hartigan’s method for k-means clustering holds several potential advantages compared to the classical and prevalent optimization heuristic known as Lloyd’s algorithm. E. g. , it was recently shown that the set of local minima of Hartigan’s algorithm is a subset of those of Lloyd’s method. We develop a closed-form expression that allows to establish Hartigan’s method for k-means clustering with any Bregman divergence, and further strengthen the case of preferring Hartigan’s algorithm over Lloyd’s algorithm. Specifically, we characterize a range of problems with various noise levels of the inputs, for which any random partition represents a local minimum for Lloyd’s algorithm, while Hartigan’s algorithm easily converges to the correct solution. Extensive experiments on synthetic and real-world data further support our theoretical analysis.

IJCAI Conference 2011 Conference Paper

Active Online Classification via Information Maximization

• Noam Slonim
• Elad Yom-Tov
• Koby Crammer

We propose an online classification approach for co-occurrence data which is based on a simple information theoretic principle. We further show how to properly estimate the uncertainty associated with each prediction of our scheme and demonstrate how to exploit these uncertainty estimates. First, in order to abstain highly uncertain predictions. And second, within an active learning framework, in order to preserve classification accuracy while substantially reducing training set size. Our method is highly efficient in terms of run-time and memory footprint requirements. Experimental results in the domain of text classification demonstrate that the classification accuracy of our method is superior or comparable to other state-of-the-art online classification algorithms.

NeurIPS Conference 2006 Conference Paper

Information Bottleneck for Non Co-Occurrence Data

• Yevgeny Seldin
• Noam Slonim
• Naftali Tishby

We present a general model-independent approach to the analysis of data in cases when these data do not appear in the form of co-occurrence of two variables X, Y, but rather as a sample of values of an unknown (stochastic) function Z (X, Y ). For example, in gene expression data, the expression level Z is a function of gene X and condition Y; or in movie ratings data the rating Z is a function of viewer X and movie Y. The approach represents a consistent extension of the Information Bottleneck method that has previously relied on the availability of co-occurrence statistics. By altering the relevance variable we eliminate the need in the sample of joint distribution of all input variables. This new formulation also enables simple MDL-like model complexity control and prediction of missing values of Z. The approach is analyzed and shown to be on a par with the best known clustering algorithms for a wide range of domains. For the prediction of missing values (collaborative filtering) it improves the currently best known results.

ICML Conference 2002 Conference Paper

Discriminative Feature Selection via Multiclass Variable Memory Markov Model

• Noam Slonim
• Gill Bejerano
• Shai Fine
• Naftali Tishby

NeurIPS Conference 2002 Conference Paper

Maximum Likelihood and the Information Bottleneck

• Noam Slonim
• Yair Weiss

 that defines partitions over the values of The information bottleneck (IB) method is an information-theoretic formulation, this method constructs for clustering problems. Given a joint distribution a new variable that are informative. Maximum likelihood (ML) of mixture models is a standard statistical about approach to clustering problems. In this paper, we ask: how are the two methods related? We define a simple mapping between the IB problem and the ML prob- lem for the multinomial mixture model. We show that under this mapping the problems are strongly related. In fact, for uniform input distribution over or for large sample size, the problems are mathematically equivalent. Specifically, in these cases, every fixed point of the IB-functional defines a fixed point of the (log) likelihood and vice versa. Moreover, the values of the functionals at the fixed points are equal under simple transformations. As a result, in these cases, every algorithm that solves one of the problems, induces a solution for the other.

NeurIPS Conference 2001 Conference Paper

Agglomerative Multivariate Information Bottleneck

• Noam Slonim
• Nir Friedman
• Naftali Tishby

The information bottleneck method is an unsupervised model independent data organization technique. Given a joint distribution peA, B), this method con(cid: 173) structs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. In a recent paper, we introduced a general princi(cid: 173) pled framework for multivariate extensions of the information bottleneck method that allows us to consider multiple systems of data partitions that are inter-related. In this paper, we present a new family of simple agglomerative algorithms to construct such systems of inter-related clusters. We analyze the behavior of these algorithms and apply them to several real-life datasets.

UAI Conference 2001 Conference Paper

Multivariate Information Bottleneck

• Nir Friedman
• Ori Mosenzon
• Noam Slonim
• Naftali Tishby

The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. The information bottleneck has already been applied to document classification, gene expression, neural code, and spectral analysis. In this paper, we introduce a general principled framework for multivariate extensions of the information bottleneck method. This allows us to consider multiple systems of data partitions that are inter-related. Our approach utilizes Bayesian networks for specifying the systems of clusters and what information each captures. We show that this construction provides insight about bottleneck variations and enables us to characterize solutions of these variations. We also present a general framework for iterative algorithms for constructing solutions, and apply it to several examples.

NeurIPS Conference 2000 Conference Paper

Data Clustering by Markovian Relaxation and the Information Bottleneck Method

• Naftali Tishby
• Noam Slonim

We introduce a new, non-parametric and principled, distance based clustering method. This method combines a pairwise based ap(cid: 173) proach with a vector-quantization method which provide a mean(cid: 173) ingful interpretation to the resulting clusters. The idea is based on turning the distance matrix into a Markov process and then examine the decay of mutual-information during the relaxation of this process. The clusters emerge as quasi-stable structures dur(cid: 173) ing this relaxation, and then are extracted using the information bottleneck method. These clusters capture the information about the initial point of the relaxation in the most effective way. The method can cluster data with no geometric or other bias and makes no assumption about the underlying distribution.

NeurIPS Conference 1999 Conference Paper

Agglomerative Information Bottleneck

• Noam Slonim
• Naftali Tishby

We introduce a novel distributional clustering algorithm that max(cid: 173) imizes the mutual information per cluster between data and giv(cid: 173) en categories. This algorithm can be considered as a bottom up hard version of the recently introduced "Information Bottleneck Method". The algorithm is compared with the top-down soft ver(cid: 173) sion of the information bottleneck method and a relationship be(cid: 173) tween the hard and soft results is established. We demonstrate the algorithm on the 20 Newsgroups data set. For a subset of two news(cid: 173) groups we achieve compression by 3 orders of magnitudes loosing only 10% of the original mutual information.