Sohyeon Kim  Ji-Hyeong Han

10.5626/JOK.2023.50.12.1111

residential area safety Human action recognition video frame selection computer vision multi-modality

Residential area closed-circuit televisions (CCTVs) need human action recognition (HAR) to predict any accidents and crucial problems. HAR model must be not only accurate but also light and fast to apply in the real world. Therefore, in this paper, a cross-modal PoseC3D model with a frame selection method is proposed. The proposed cross-modal PoseC3D model integrates multi-modality inputs (i.e., RGB image and human skeleton data) and trains them in a single model. Thus, the proposed model is lighter and faster than previous works such as two-pathway PoseC3D. Moreover, we apply the frame selection method to use only the meaningful frames based on differences between frames instead of using the whole frame of a video. AI Hub open dataset was used to verify the performance of proposed method. The experimental results showed that the proposed method achieves similar or better performance and is much lighter and faster than those in the previous works.