Safe Reinforcement Learning Using Black-Box Reachability Analysis
release_iigq3ndf5zfuxlob6hmpxigrea
by
Mahmoud Selim, Amr Alanwar, Shreyas Kousik, Grace Gao, Marco Pavone, Karl H. Johansson
2022
Abstract
Reinforcement learning (RL) is capable of sophisticated motion planning and
control for robots in uncertain environments. However, state-of-the-art deep RL
approaches typically lack safety guarantees, especially when the robot and
environment models are unknown. To justify widespread deployment, robots must
respect safety constraints without sacrificing performance. Thus, we propose a
Black-box Reachability-based Safety Layer (BRSL) with three main components:
(1) data-driven reachability analysis for a black-box robot model, (2) a
trajectory rollout planner that predicts future actions and observations using
an ensemble of neural networks trained online, and (3) a differentiable
polytope collision check between the reachable set and obstacles that enables
correcting unsafe actions. In simulation, BRSL outperforms other
state-of-the-art safe RL methods on a Turtlebot 3, a quadrotor, a
trajectory-tracking point mass, and a hexarotor in wind with an unsafe set
adjacent to the area of highest reward.
In text/plain
format
Archived Files and Locations
application/pdf 2.7 MB
file_io7pim5hcnbjpj5glhtqgzhz5y
|
arxiv.org (repository) web.archive.org (webarchive) |
2204.07417v2
access all versions, variants, and formats of this works (eg, pre-prints)