Cross-Entropy Planning with Prior Updates

HyeongJoo Hwang, Youngsoo Jang, Jaeyoung Park, Kee-Eung Kim

http://doi.org/10.5626/JOK.2020.47.1.88

This paper introduces a method of cross-entropy planning which updates prior probability for planning optimization process. Cross-entropy planning is a popular method in online planning and involves the extraction of samples from a simulation environment and selection of optimal action based on the values of the extracted samples. The performance of the cross-entropy planning is limited due to involvement of optimization processes without usage of previous planning results. We propose a method that updates prior probabilities for the optimization process based on the action sequences acquired from the cross-entropy planning. The proposed method improves the performance of cross-entropy planning with progression of planning epoch. We evaluated the proposed method based on the comparison with the cross-entropy planning in a physical-based simulation (OpenAI Gym) environment.