Hand Gesture Classification Based on Nonaudible Sound Using Convolutional Neural Network

Abstract EN

Recognizing and distinguishing the behavior and gesture of a user has become important owing to an increase in the use of wearable devices, such as a smartwatch.

This study is aimed at proposing a method for classifying hand gestures by creating sound in the nonaudible frequency range using a smartphone and reflected signal.

The proposed method converts the sound data, which has been reflected and recorded, into an image within a short time using short-time Fourier transform, and the obtained data are applied to a convolutional neural network (CNN) model to classify hand gestures.

The results showed classification accuracy for 8 hand gestures with an average of 87.75%.

Additionally, it is confirmed that the suggested method has a higher classification accuracy than other machine learning classification algorithms.