Risk Scheduling-based Optimistic Exploration for Distributional Reinforcement Learning

Jihwan Oh, Joonkee Kim, Se-Young Yun

http://doi.org/10.5626/JOK.2023.50.2.172

Distributional reinforcement learning demonstrates state-of-the-art performance in continuous and discrete control systems with the features of variance and risk, which can be used to explore action space. However, the exploration method employing the risk property is hard to find, although numerous exploration methods in Distributional RL employ the variance of the return distribution for an action. This paper presents risk scheduling approaches that explore risk levels and optimistic behaviors from a risk perspective in Distributional reinforcement learning. We demonstrate the performance (win-rate) enhancement of the DMIX, DDN, and DIQL algorithms, which integrate Distributional reinforcement learning into a multi-agent system using risk scheduling in a multi-agent setting with comprehensive experiments.