RNN-based Body Posture Estimation Method for 3D UI in Virtual Reality

Wonjun Seong, Byungmoon Kim, BoYu Gao, Jini Kwon, HyungSeok Kim

http://doi.org/10.5626/JOK.2020.47.4.362

Providing intuitive and natural interface is crucial in virtual reality applications. There have been many studies on proper 3D interfaces that require the accurate tracking of body posture. Based on previous studies, we propose an effective body posture estimation method for 3D interfaces. In this study, we focused on the body-fixed UI, especially movements of the upper body. To track body posture, it is necessary to apply additional sensors such as RGB/RGB-D camera, magnetic/optical trackers, etc. The goal of this study was to track the body posture with conventional virtual reality devices only. We applied conventional HMD with head tracker and hand-held controllers. With these three trackers which are not directly attached to the body, an RNN-based method is proposed to effectively estimate the upper body pose. Experiments shows that the proposed method could track the upper body position and orientation within 5% of the error rate, compared with the explicit tracker data. The proposed method could be applied to design body-fixed UI or body-related interfaces without additional devices, which would enhance the accessibility of virtual reality applications.