Arrow Research search

Author name cluster

Rahul V Kulkarni

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers
1 author row

Possible papers

6

PRL Workshop 2025 Workshop Paper

Average-Reward Reinforcement Learning with Entropy Regularizatio

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

The average-reward formulation of reinforcement learning (RL) has drawn increased interest in recent years due to its ability to solve temporally-extended problems without discounting. Independently, RL algorithms have benefited from entropy-regularization: an approach used to make the optimal policy stochastic, thereby more robust to noise. Despite the distinct benefits of the two approaches, the combination of entropy regularization with an average-reward objective is not well-studied in the literature and there has been limited development of algorithms for this setting. To address this gap in the field, we develop algorithms for solving entropyregularized average-reward RL problems with function approximation. We experimentally validate our method, comparing it with existing algorithms on standard benchmarks for RL.

PDF Details

RLC Conference 2025 Conference Paper

Average-Reward Soft Actor-Critic

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

The average-reward formulation of reinforcement learning (RL) has drawn increased interest in recent years for its ability to solve temporally-extended problems without relying on discounting. Meanwhile, in the discounted setting, algorithms with entropy regularization have been developed, leading to improvements over deterministic methods. Despite the distinct benefits of these approaches, deep RL algorithms for the entropy-regularized average-reward objective have not been developed. While policy-gradient based approaches have recently been presented for the average-reward literature, the corresponding actor-critic framework remains less explored. In this paper, we introduce an average-reward soft actor-critic algorithm to address these gaps in the field. We validate our method by comparing with existing average-reward algorithms on standard RL benchmarks, achieving superior performance for the average-reward criterion.

PDF Details

EWRL Workshop 2025 Workshop Paper

Average-Reward Soft Actor-Critic

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

The average-reward formulation of reinforcement learning (RL) has drawn increased interest in recent years for its ability to solve temporally-extended problems without relying on discounting. Meanwhile, in the discounted setting, algorithms with entropy regularization have been developed, leading to improvements over deterministic methods. Despite the distinct benefits of these approaches, deep RL algorithms for the entropy-regularized average-reward objective have not been developed. While policy-gradient based approaches have recently been presented for the average-reward literature, the corresponding actor-critic framework remains less explored. In this paper, we introduce an average-reward soft actor-critic algorithm to address these gaps in the field. We validate our method by comparing with existing average-reward algorithms on standard RL benchmarks, achieving superior performance for the average-reward criterion.

PDF

RLJ Journal 2025 Journal Article

Average-Reward Soft Actor-Critic

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

The average-reward formulation of reinforcement learning (RL) has drawn increased interest in recent years for its ability to solve temporally-extended problems without relying on discounting. Meanwhile, in the discounted setting, algorithms with entropy regularization have been developed, leading to improvements over deterministic methods. Despite the distinct benefits of these approaches, deep RL algorithms for the entropy-regularized average-reward objective have not been developed. While policy-gradient based approaches have recently been presented for the average-reward literature, the corresponding actor-critic framework remains less explored. In this paper, we introduce an average-reward soft actor-critic algorithm to address these gaps in the field. We validate our method by comparing with existing average-reward algorithms on standard RL benchmarks, achieving superior performance for the average-reward criterion.

PDF Details

RLJ Journal 2024 Journal Article

Boosting Soft Q-Learning by Bounding

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

An agent’s ability to leverage past experience is critical for efficiently solving new tasks. Prior work has focused on using value function estimates to obtain zero-shot approximations for solutions to a new task. In soft $Q$-learning, we show how any value function estimate can also be used to derive double-sided bounds on the optimal value function. The derived bounds lead to new approaches for boosting training performance which we validate experimentally. Notably, we find that the proposed framework suggests an alternative method for updating the $Q$-function, leading to boosted performance.

PDF Details

RLC Conference 2024 Conference Paper

Boosting Soft Q-Learning by Bounding

  • Jacob Adamczyk
  • Volodymyr Makarenko
  • Stas Tiomkin
  • Rahul V Kulkarni

An agent’s ability to leverage past experience is critical for efficiently solving new tasks. Prior work has focused on using value function estimates to obtain zero-shot approximations for solutions to a new task. In soft $Q$-learning, we show how any value function estimate can also be used to derive double-sided bounds on the optimal value function. The derived bounds lead to new approaches for boosting training performance which we validate experimentally. Notably, we find that the proposed framework suggests an alternative method for updating the $Q$-function, leading to boosted performance.

PDF Details
v2026.09.13