Author name cluster

U. N. Niranjan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

2 author rows

ECAI Conference 2023 Conference Paper

Revisiting the Efficiency-Accuracy Tradeoff in Adapting Transformer Models via Adversarial Fine-Tuning

Minjia Zhang
U. N. Niranjan
Yuxiong He

Adversarial fine-tuning (i. e. , training on adversarial perturbed inputs) has demonstrated promising results in improving the accuracy of natural language understanding tasks. However, the improved accuracy does not come for free but is accompanied by a significantly prolonged training time, limiting their applicability to larger and more complex models. This work revisits the efficiency-accuracy trade-off in adversarial fine-tuning by systematically analyzing if adversarial fine-tuning methods, in conjunction with several efficiency optimizations, are suitable for adapting pre-trained Transformer models for natural language understanding tasks. Our results show that multiple design choices are crucial in determining the efficiency-accuracy trade-off, and we introduce a method, ScaLA, that achieves better accuracy-vs-speed trade-off than prior methods. We show in experiments that our proposed method attains up to 14. 7× adaptation speedups on BERT, RoBERTa, and T5, while achieving comparable accuracy to existing methods.

Details

UAI Conference 2019 Conference Paper

Guaranteed Scalable Learning of Latent Tree Models

Furong Huang
U. N. Niranjan
Ioakeim Perros
Robert Chen 0001
Jimeng Sun 0001
Anima Anandkumar

We present an integrated approach to structure and parameter estimation in latent tree graphical models, where some nodes are hidden. Our overall approach follows a “divide-and-conquer” strategy that learns models over small groups of variables and iteratively merges into a global solution. The structure learning involves combinatorial operations such as minimum spanning tree construction and local recursive grouping; the parameter learning is based on the method of moments and on tensor decompositions. Our method is guaranteed to correctly recover the unknown tree structure and the model parameters with low sample complexity for the class of linear multivariate latent tree models which includes discrete and Gaussian distributions, and Gaussian mixtures. Our bulk asynchronous parallel algorithm is implemented in parallel and scales logarithmically with the number of variables and linearly with dimensionality of each variable.

Details

UAI Conference 2017 Conference Paper

Provable Inductive Robust PCA via Iterative Hard Thresholding

U. N. Niranjan
Arun Rajkumar
Theja Tulabandhula

The robust PCA problem, wherein, given an input data matrix that is the superposition of a low-rank matrix and a sparse matrix, we aim to separate out the low-rank and sparse components, is a well-studied problem in machine learning. One natural question that arises is that, as in the inductive setting, if features are provided as input as well, can we hope to do better? Answering this in the affirmative, the main goal of this paper is to study the robust PCA problem while incorporating feature information. In contrast to previous works in which recovery guarantees are based on the convex relaxation of the problem, we propose a simple iterative algorithm based on hard-thresholding of appropriate residuals. Under weaker assumptions than previous works, we prove the global convergence of our iterative procedure; moreover, it admits a much faster convergence rate and lesser computational complexity per iteration. In practice, through systematic synthetic and real data simulations, we confirm our theoretical findings regarding improvements obtained by using feature information.

Details

JMLR Journal 2015 Journal Article

Online Tensor Methods for Learning Latent Variable Models

Furong Huang
U. N. Niranjan
Mohammad Umar Hakeem
Animashree Anandkumar

We introduce an online tensor decomposition based approach for two latent variable modeling problems namely, (1) community detection, in which we learn the latent communities that the social actors in social networks belong to, and (2) topic modeling, in which we infer hidden topics of text articles. We consider decomposition of moment tensors using stochastic gradient descent. We conduct optimization of multilinear operations in SGD and avoid directly forming the tensors, to save computational and storage costs. We present optimized algorithm in two platforms. Our GPU-based implementation exploits the parallelism of SIMD architectures to allow for maximum speed-up by a careful optimization of storage and data transfer, whereas our CPU-based implementation uses efficient sparse matrix computations and is suitable for large sparse data sets. For the community detection problem, we demonstrate accuracy and computational efficiency on Facebook, Yelp and DBLP data sets, and for the topic modeling problem, we also demonstrate good performance on the New York Times data set. We compare our results to the state-of-the-art algorithms such as the variational method, and report a gain of accuracy and a gain of several orders of magnitude in the execution time. [abs] [ pdf ][ bib ] &copy JMLR 2015. ( edit, beta )

PDF Details