Author name cluster

Tianshu Chu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

3 papers

2 author rows

ICML Conference 2024 Conference Paper

SPABA: A Single-Loop and Probabilistic Stochastic Bilevel Algorithm Achieving Optimal Sample Complexity

Tianshu Chu
Dachuan Xu
Wei Yao
Jin Zhang

While stochastic bilevel optimization methods have been extensively studied for addressing large-scale nested optimization problems in machine learning, it remains an open question whether the optimal complexity bounds for solving bilevel optimization are the same as those in single-level optimization. Our main result resolves this question: SPABA, an adaptation of the PAGE method for nonconvex optimization in (Li et al. , 2021) to the bilevel setting, can achieve optimal sample complexity in both the finite-sum and expectation settings. We show the optimality of SPABA by proving that there is no gap in complexity analysis between stochastic bilevel and single-level optimization when implementing PAGE. Notably, as indicated by the results of (Dagréou et al. , 2022), there might exist a gap in complexity analysis when implementing other stochastic gradient estimators, like SGD and SAGA. In addition to SPABA, we propose several other single-loop stochastic bilevel algorithms, that either match or improve the state-of-the-art sample complexity results, leveraging our convergence rate and complexity analysis. Numerical experiments demonstrate the superior practical performance of the proposed methods.

Details

ICLR Conference 2020 Conference Paper

Multi-agent Reinforcement Learning for Networked System Control

Tianshu Chu
Sandeep Chinchali
Sachin Katti

This paper considers multi-agent reinforcement learning (MARL) in networked system control. Specifically, each agent learns a decentralized control policy based on local observations and messages from connected neighbors. We formulate such a networked MARL (NMARL) problem as a spatiotemporal Markov decision process and introduce a spatial discount factor to stabilize the training of each local agent. Further, we propose a new differentiable communication protocol, called NeurComm, to reduce information loss and non-stationarity in NMARL. Based on experiments in realistic NMARL scenarios of adaptive traffic signal control and cooperative adaptive cruise control, an appropriate spatial discount factor effectively enhances the learning curves of non-communicative MARL algorithms, while NeurComm outperforms existing communication protocols in both learning efficiency and control performance.

Details

AAAI Conference 2018 Conference Paper

Cellular Network Traffic Scheduling With Deep Reinforcement Learning

Sandeep Chinchali
Pan Hu
Tianshu Chu
Manu Sharma
Manu Bansal
Rakesh Misra
Marco Pavone
Sachin Katti

Modern mobile networks are facing unprecedented growth in demand due to a new class of trafﬁc from Internet of Things (IoT) devices such as smart wearables and autonomous cars. Future networks must schedule delay-tolerant software updates, data backup, and other transfers from IoT devices while maintaining strict service guarantees for conventional realtime applications such as voice-calling and video. This problem is extremely challenging because conventional trafﬁc is highly dynamic across space and time, so its performance is signiﬁcantly impacted if all IoT trafﬁc is scheduled immediately when it originates. In this paper, we present a reinforcement learning (RL) based scheduler that can dynamically adapt to trafﬁc variation, and to various reward functions set by network operators, to optimally schedule IoT trafﬁc. Using 4 weeks of real network data from downtown Melbourne, Australia spanning diverse trafﬁc patterns, we demonstrate that our RL scheduler can enable mobile networks to carry 14. 7% more data with minimal impact on existing trafﬁc, and outperforms heuristic schedulers by more than 2×. Our work is a valuable step towards designing autonomous, “selfdriving” networks that learn to manage themselves from past data.

PDF Details