A new approach combining CNN, RNN, and an improved Otsu threshold method for detecting hand gestures in people with thumb finger size problems and hand tremors

The recognition of hand gestures involves the application of mathematical algorithms to detect human movements, with diverse applications in communication for the hearing impaired, human–computer interaction, autonomous driving, and virtual environments. This research presents a comprehensive approach to identifying dynamic hand gestures, which is particularly beneficial for individuals with finger disabilities. In addition, those with hand tremors may encounter challenges when using human–computer interaction devices. The proposed technique enhances the sensitivity of these devices through an advanced Otsu segmentation method. It begins by isolating hand gestures from complex backgrounds using this sophisticated segmentation algorithm and incorporates motion data derived from RGB video sequences. The gestures are then transformed into texture and contour characteristics, which are subsequently input into a hybrid architecture that combines a convolutional neural network (CNN) with a recurrent neural network (RNN). Our findings demonstrate that the proposed method achieves superior segmentation results to existing alternatives and can detect finger joint interactions and tremors with high sensitivity. When comparing the traditional Otsu method with our proposed Otsu method, our results indicate an improvement of 6.3% in accuracy for the CNN and RNN classifiers. The performance of the novel Otsu method has been evaluated and compared using accuracy and sensitivity metrics, yielding significant results.