TY  - JOUR
T1  - Real-time Multimodal Audio-to-Tactile Conversion System for Playing or Watching Mobile Shooting Games
AU  - Mun, Minjae 
AU  - Yun, Gyeore 
AU  - Park, Chaeyong 
AU  - Choi, Seungmoon 
JO  - Journal of KIISE, JOK
PY  - 2023
DA  - 2023/1/14
DO  - 10.5626/JOK.2023.50.3.228
KW  - multimodal haptics
KW  - human-computer interaction
KW  - user experience
KW  - mobile system
KW  - real-time audio-to-tactile conversion
AB  - This study presents a real-time multimodal audio-to-tactile conversion system for improving user experiences when users play or watch first-person shooting games with a mobile device. The system detects whether sounds from the mobile devices are appropriate to provide haptic feedback in real-time and provides vibrotactile feedback, mainly used for conventional haptic feedback, and impact effects of short and strong force as well. To this end, we confirmed the suitability of the impact haptic feedback compared to the vibrotactile feedback for shooting games. We implemented two types of impulsive sound detectors using psychoacoustic features and a support vector machine. We found that our detectors outperformed the one from a previous study. Lastly, we conduct a user study to evaluate our system. Results showed that our system could significantly improve user experiences.