Author name cluster

Xinwei Deng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

NeurIPS Conference 2022 Conference Paper

Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization

Qing Guo
Junya Chen
Dong Wang
Yuewei Yang
Xinwei Deng
Jing Huang
Larry Carin
Fan Li

Successful applications of InfoNCE (Information Noise-Contrastive Estimation) and its variants have popularized the use of contrastive variational mutual information (MI) estimators in machine learning. While featuring superior stability, these estimators crucially depend on costly large-batch training, and they sacrifice bound tightness for variance reduction. To overcome these limitations, we revisit the mathematics of popular variational MI bounds from the lens of unnormalized statistical modeling and convex optimization. Our investigation yields a new unified theoretical framework encompassing popular variational MI bounds, and leads to a novel, simple, and powerful contrastive MI estimator we name FLO. Theoretically, we show that the FLO estimator is tight, and it converges under stochastic gradient descent. Empirically, the proposed FLO estimator overcomes the limitations of its predecessors and learns more efficiently. The utility of FLO is verified using extensive benchmarks, and we further inspire the community with novel applications in meta-learning. Our presentation underscores the foundational importance of variational MI estimation in data-efficient learning.

PDF Details

AAAI Conference 2018 Conference Paper

Graph Scan Statistics With Uncertainty

Jose Cadena
Arinjoy Basak
Anil Vullikanti
Xinwei Deng

Scan statistics is one of the most popular approaches for anomaly detection in spatial and network data. In practice, there are numerous sources of uncertainty in the observed data. However, most prior works have overlooked such uncertainty, which can affect the accuracy and inferences of such methods. In this paper, we develop the ﬁrst systematic approach to incorporating uncertainty in scan statistics. We study two formulations for robust scan statistics, one based on the sample average approximation and the other using a max-min objective. We show that uncertainty signiﬁcantly increases the computational complexity of these problems. Rigorous algorithms and efﬁcient heuristics for both formulations are developed with justiﬁcation of theoretical bounds. We evaluate our proposed methods on synthetic and real datasets, and we observe that our methods give signiﬁcant improvement in the detection power as well as optimization objective, relative to a baseline.

PDF Details