Transformer-Based Head Motion Prediction Algorithm Using Image Generation Model

Hyogeun Byun, Moonsoo Jeong, Sungkil Lee

http://doi.org/10.5626/JOK.2024.51.7.601

Motion-to-photon latency in virtual reality based on head-mounted display can cause discomfort such as cyber sickness due to a lag between a user's physical movement and the image output, potentially disrupting users’ immersion. Traditional methods to reduce this latency involve manually analyzing head motion trends or predicting head motion with recurrent neural networks. However, these models faced long-term dependency issues in remembering information over sequences and limitations in parallel processing. In this paper, images are also used in the decoding process through an image generation model. A deep learning model used in natural language processing is highly scalable when using it as a prediction model. Accordingly, the model proposed in this study could use additional data to predict the user’s head motion and thereby, outperforms the existing models.