Author name cluster

Kazumune Hashimoto

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

EAAI Journal 2026 Journal Article

Signal temporal logic-based neural network for driving task generation in Advanced Driver Assistance Systems

Kazumune Hashimoto
Yusuke Yokokawa
Norika Arai
Xun Shen
Xingguo Zhang
Pongsathorn Raksincharoensak

In this paper, we propose an Advanced Driver Assistance Systems (ADAS) framework that utilizes Signal Temporal Logic (STL) to formally represent driving tasks based on environmental information. A neural network architecture, the STL generator network, is designed to generate tasks that enhance interpretability and guide either driver actions or autonomous control inputs. By integrating coarse task determination and parameter specification networks, the system provides structured task recommendations to improve safety and skill development. The proposed method’s effectiveness is validated through hardware-in-the-loop simulations, demonstrating its potential to enhance driver performance and system transparency in complex driving environments.

Details DOI

NeurIPS Conference 2024 Conference Paper

Flipping-based Policy for Chance-Constrained Markov Decision Processes

Xun Shen
Shuo Jiang
Akifumi Wachi
Kazumune Hashimoto
Sebastien Gros

Safe reinforcement learning (RL) is a promising approach for many real-world decision-making problems where ensuring safety is a critical necessity. In safe RL research, while expected cumulative safety constraints (ECSCs) are typically the first choices, chance constraints are often more pragmatic for incorporating safety under uncertainties. This paper proposes a \textit{flipping-based policy} for Chance-Constrained Markov Decision Processes (CCMDPs). The flipping-based policy selects the next action by tossing a potentially distorted coin between two action candidates. The probability of the flip and the two action candidates vary depending on the state. We establish a Bellman equation for CCMDPs and further prove the existence of a flipping-based policy within the optimal solution sets. Since solving the problem with joint chance constraints is challenging in practice, we then prove that joint chance constraints can be approximated into Expected Cumulative Safety Constraints (ECSCs) and that there exists a flipping-based policy in the optimal solution sets for constrained MDPs with ECSCs. As a specific instance of practical implementations, we present a framework for adapting constrained policy optimization to train a flipping-based policy. This framework can be applied to other safe RL algorithms. We demonstrate that the flipping-based policy can improve the performance of the existing safe RL algorithms under the same limits of safety constraints on Safety Gym benchmarks.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Long-Term Safe Reinforcement Learning with Binary Feedback

Akifumi Wachi
Wataru Hashimoto
Kazumune Hashimoto

Safety is an indispensable requirement for applying reinforcement learning (RL) to real problems. Although there has been a surge of safe RL algorithms proposed in recent years, most existing work typically 1) relies on receiving numeric safety feedback; 2) does not guarantee safety during the learning process; 3) limits the problem to a priori known, deterministic transition dynamics; and/or 4) assume the existence of a known safe policy for any states. Addressing the issues mentioned above, we thus propose Long-term Binary-feedback Safe RL (LoBiSaRL), a safe RL algorithm for constrained Markov decision processes (CMDPs) with binary safety feedback and an unknown, stochastic state transition function. LoBiSaRL optimizes a policy to maximize rewards while guaranteeing long-term safety that an agent executes only safe state-action pairs throughout each episode with high probability. Specifically, LoBiSaRL models the binary safety function via a generalized linear model (GLM) and conservatively takes only a safe action at every time step while inferring its effect on future safety under proper assumptions. Our theoretical results show that LoBiSaRL guarantees the long-term safety constraint, with high probability. Finally, our empirical results demonstrate that our algorithm is safer than existing methods without significantly compromising performance in terms of reward.

PDF Details DOI

NeurIPS Conference 2023 Conference Paper

Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithms

Akifumi Wachi
Wataru Hashimoto
Xun Shen
Kazumune Hashimoto

Safe exploration is essential for the practical use of reinforcement learning (RL) in many real-world scenarios. In this paper, we present a generalized safe exploration (GSE) problem as a unified formulation of common safe exploration problems. We then propose a solution of the GSE problem in the form of a meta-algorithm for safe exploration, MASE, which combines an unconstrained RL algorithm with an uncertainty quantifier to guarantee safety in the current episode while properly penalizing unsafe explorations before actual safety violation to discourage them in future episodes. The advantage of MASE is that we can optimize a policy while guaranteeing with a high probability that no safety constraint will be violated under proper assumptions. Specifically, we present two variants of MASE with different constructions of the uncertainty quantifier: one based on generalized linear models with theoretical guarantees of safety and near-optimality, and another that combines a Gaussian process to ensure safety with a deep RL algorithm to maximize the reward. Finally, we demonstrate that our proposed algorithm achieves better performance than state-of-the-art algorithms on grid-world and Safety Gym benchmarks without violating any safety constraints, even during training.

PDF Details