Author name cluster

Rahul Singh

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

10 papers

2 author rows

AAAI Conference 2026 Conference Paper

Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learning

Avik Kar
Rahul Singh

We study infinite-horizon average-reward reinforcement learning for continuous space Lipschitz Markov decision processes (MDPs) in which an agent can play policies from a given set Φ. The proposed algorithms efficiently explore the policy space by “zooming” into the “promising regions” of Φ, thereby achieving adaptivity gains in the performance. We upper bound the regret as O ̃(T^(1-d_(eff.)^(-1) ) ), where d_(eff.) = d_z^Φ+2 for our model-free algorithm PZRL-MF and d_(eff.) = 2d_S + d_z^Φ+ 3 for our model-based algorithm PZRL-MB. Here, d_S is the dimension of the state space, and d_z^Φ is the zooming dimension given a set of policies Φ. d_z^Φ is an alternative measure of the complexity of the problem, and it depends on the underlying MDP as well as on Φ. Hence, the proposed algorithms exhibit low regret in case the problem instance is benign and/or the agent competes against a low-complexity Φ (that has a small d_z^Φ). When specialized to the case of finite-dimensional policy space, we obtain that d_(eff.) scales as the dimension of this space under mild technical conditions; and also obtain d_(eff.) = 2, or equivalently O ̃(√T) regret for PZRL-MF, under a curvature condition on the average reward function that is commonly used in the multi-armed bandit (MAB) literature.

PDF Details DOI

JMLR Journal 2025 Journal Article

On the Utility of Equal Batch Sizes for Inference in Stochastic Gradient Descent

Rahul Singh
Abhinek Shukla
Dootika Vats

Stochastic gradient descent (SGD) is an estimation tool for large data employed in machine learning and statistics. Due to the Markovian nature of the SGD process, inference is a challenging problem. An underlying asymptotic normality of the averaged SGD (ASGD) estimator allows for the construction of a batch-means estimator of the asymptotic covariance matrix. Instead of the usual increasing batch-size strategy, we propose a memory efficient equal batch-size strategy and show that under mild conditions, the batch-means estimator is consistent. A key feature of the proposed batching technique is that it allows for bias-correction of the variance, at no additional cost to memory. Further, since joint inference for large dimensional problems may be undesirable, we present marginal-friendly simultaneous confidence intervals, and show through an example on how covariance estimators of ASGD can be employed for improved predictions. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2025. ( edit, beta )

PDF Details

TMLR Journal 2023 Journal Article

Signed Graph Neural Networks: A Frequency Perspective

Rahul Singh
Yongxin Chen

Graph convolutional networks (GCNs) and its variants are designed for unsigned graphs containing only positive links. Many existing GCNs have been derived from the spectral domain analysis of signals lying over (unsigned) graphs and in each convolution layer they perform low-pass filtering of the input features followed by a learnable linear transformation. Their extension to signed graphs with positive as well as negative links imposes multiple issues including computational irregularities and ambiguous frequency interpretation, making the design of computationally efficient low pass filters challenging. In this paper, we address these issues via spectral analysis of signed graphs and propose two different signed graph neural networks, one keeps only low-frequency information and one also retains high-frequency information. We further introduce magnetic signed Laplacian and use its eigendecomposition for spectral analysis of directed signed graphs. We test our methods for node classification and link sign prediction tasks on signed graphs and achieve state-of-the-art performances.

PDF Details

NeurIPS Conference 2022 Conference Paper

Augmented RBMLE-UCB Approach for Adaptive Control of Linear Quadratic Systems

Akshay Mete
Rahul Singh
P. R. Kumar

We consider the problem of controlling an unknown stochastic linear system with quadratic costs -- called the adaptive LQ control problem. We re-examine an approach called ``Reward-Biased Maximum Likelihood Estimate'' (RBMLE) that was proposed more than forty years ago, and which predates the ``Upper Confidence Bound'' (UCB) method, as well as the definition of ``regret'' for bandit problems. It simply added a term favoring parameters with larger rewards to the criterion for parameter estimation. We show how the RBMLE and UCB methods can be reconciled, and thereby propose an Augmented RBMLE-UCB algorithm that combines the penalty of the RBMLE method with the constraints of the UCB method, uniting the two approaches to optimism in the face of uncertainty. We establish that theoretically, this method retains ${\mathcal{O}}(\sqrt{T})$ regret, the best known so far. We further compare the empirical performance of the proposed Augmented RBMLE-UCB and the standard RBMLE (without the augmentation) with UCB, Thompson Sampling, Input Perturbation, Randomized Certainty Equivalence and StabL on many real-world examples including flight control of Boeing 747 and Unmanned Aerial Vehicle. We perform extensive simulation studies showing that the Augmented RBMLE consistently outperforms UCB, Thompson Sampling and StabL by a huge margin, while it is marginally better than Input Perturbation and moderately better than Randomized Certainty Equivalence.

PDF Details

AAAI Conference 2022 Conference Paper

Reinforcement Learning Augmented Asymptotically Optimal Index Policy for Finite-Horizon Restless Bandits

Guojun Xiong
Jian Li
Rahul Singh

We study a finite-horizon restless multi-armed bandit problem with multiple actions, dubbed as R(MA)2 B. The state of each arm evolves according to a controlled Markov decision process (MDP), and the reward of pulling an arm depends on both the current state and action of the corresponding MDP. Since finding the optimal policy is typically intractable, we propose a computationally appealing index policy entitled Occupancy-Measured-Reward Index Policy for the finite-horizon R(MA)2 B. Our index policy is well-defined without the requirement of indexability condition and is provably asymptotically optimal. We then adopt a learning perspective where the system parameters are unknown, and propose R(MA)2 B-UCB, a generative model based reinforcement learning augmented algorithm that can fully exploit the structure of Occupancy-Measured-Reward Index Policy. Compared to existing algorithms, R(MA)2 B-UCB performs close to offline optimum, as well as achieves a sub-linear regret and a low computational complexity all at once. Experimental results show that R(MA)2 B-UCB outperforms existing algorithms in both regret and running time.

PDF Details

NeurIPS Conference 2019 Conference Paper

Kernel Instrumental Variable Regression

Rahul Singh
Maneesh Sahani
Arthur Gretton

Instrumental variable (IV) regression is a strategy for learning causal relationships in observational data. If measurements of input X and output Y are confounded, the causal relationship can nonetheless be identified if an instrumental variable Z is available that influences X directly, but is conditionally independent of Y given X and the unmeasured confounder. The classic two-stage least squares algorithm (2SLS) simplifies the estimation problem by modeling all relationships as linear functions. We propose kernel instrumental variable regression (KIV), a nonparametric generalization of 2SLS, modeling relations among X, Y, and Z as nonlinear functions in reproducing kernel Hilbert spaces (RKHSs). We prove the consistency of KIV under mild assumptions, and derive conditions under which convergence occurs at the minimax optimal rate for unconfounded, single-stage RKHS regression. In doing so, we obtain an efficient ratio between training sample sizes used in the algorithm's first and second stages. In experiments, KIV outperforms state of the art alternatives for nonparametric IV regression.

PDF Details

AAAI Conference 2012 Conference Paper

Sustaining Economic Exploitation of Complex Ecosystems in Computational Models of Coupled Human-Natural Networks

Neo Martinez
Perrine Tonnin
Barbara Bauer
Rosalyn Rael
Rahul Singh
Sangyuk Yoon
Ilmi Yoon
Jennifer Dunne

Understanding ecological complexity has stymied scientists for decades. Recent elucidation of the famously coined "devious strategies for stability in enduring natural systems" has opened up a new field of computational analyses of complex ecological networks where the nonlinear dynamics of many interacting species can be more realistically mod eled and understood. Here, we describe the first extension of this field to include coupled human natural systems. This extension elucidates new strategies for sustaining extraction of biomass (e. g. , fish, forests, fiber) from ecosystems that account for ecological complexity and can pursue multiple goals such as maximizing economic profit, employment and carbon sequestration by ecosystems. Our more realistic modeling of ecosystems helps explain why simpler "maxi mum sustainable yield" bioeconomic models underpinning much natural resource extraction policy leads to less profit, biomass, and biodiversity than predicted by those simple models. Current research directions of this integrated natu ral and social science include applying artificial intelligence, cloud computing, and multiplayer online games.

PDF Details

ICRA Conference 1998 Conference Paper

Object Skeletons from Sparse Shapes in Industrial image Settings

Rahul Singh
Nikolaos P. Papanikolopoulos
Vladimir Cherkassky

Presents a method for computing the shape skeleton of planar objects in presence of noise occurring inside the image regions. Such noise may be due to poor control of lighting conditions, incorrect thresholding or image subsampling. Binary images of objects with such noise exhibit sparseness (lack of connectivity), within their image regions. Such non-contiguity may also be observed in thresholded images of objects which consist of regions having varying albedo. The problem of obtaining the skeletal description of sparse shapes is ill posed in the sense of conventional skeletonization techniques. We propose a skeletonization method which is based on obtaining the shape skeleton by evolving an approximation of the principal curve of the shape distribution. Our method is implemented as a batch mode Kohonen self-organizing map algorithm and involves iterating the following two steps: (1) Voronoi tessellation of the data, (2) kernel smoothing on the Voronoi centroids. Adjacency relationships between the Voronoi regions are obtained by computing a Delaunay triangulation of the centroids. The Voronoi centroids are connected by a minimum spanning tree after each iteration. The final shape skeleton is obtained by joining centroids which are disjoint in the spanning tree, but have adjacent Voronoi regions. The skeletal descriptions obtained with the method are invariant to translation, rotation, and scale changes of the shape. The potential of the method is demonstrated on industrial objects having varying shape complexity under different imaging conditions.

Details

IROS Conference 1998 Conference Paper

Pose alignment of an eye-in-hand system using image morphing

Rahul Singh
Richard M. Voyles
David Littau
Nikolaos P. Papanikolopoulos

Positioning an eye-in-hand robotic system with respect to a static target is a challenging research problem since it involves recognition of the object and the desired pose at which alignment is to occur, planning a trajectory for the robot to attain this pose, and careful calibration of the system and the environment. In this paper we introduce a unified framework based on image morphing, to address the above problems and apply it to the task of translational and rotational alignment of an eye-in-hand system to planar objects or planar projections of 3D objects. In our method the desired manipulator pose for each object is defined and stored as a view of the object taken from this pose. The identity of an unknown object in the workspace is established by morphing its image to the views in the database, and using a quantification of the morph as a dissimilarity measure. The synthetic images generated during the morph are used guide an eye-in-hand system to the desired pose. The framework can accommodate partially occluded or deformable targets and smooth trajectories can be generated since an arbitrary number of intermediate images can be used.

Details

ICRA Conference 1997 Conference Paper

Recognition of 2D shapes through contour metamorphosis

Rahul Singh
Ioannis T. Pavlidis
Nikolaos P. Papanikolopoulos

A novel method for 2D shape recognition is proposed. The method employs as a dissimilarity measure the degree of morphing between a test shape and a reference shape. A physics-based approach substantiates the degree of morphing as a deformation energy and casts the problem as an energy minimization problem. The method operates upon key segmentation points that are provided by an appropriate segmentation algorithm. The recognition paradigm is invariant to translation, rotation, and scaling. It can handle both convex and non-convex shapes. The proposed system exhibits robust recognition behavior and real-time performance in a series of experiments. The experiments also highlight the ability of the method to recognize deformable shapes.

Details