SIGN LANGUAGE TO TEXT AND SPEECH CONVERSION USING IMAGE PROCESSING AND MACHINE LEARNING
Abstract
Communication with people with hearing impairments (deaf and dumb) is a big challenge in our current society, this can be attributed to the fact that their means of communication (sign language or hand gestures locally) require an interpreter in any case. This paper consists of image processing and machine learning methods. Our goal is to design a human-computer interface system that can accurately recognize the speech of deaf-mutes. In this paper, a vision-based hand gesture recognition system was designed, as the hand plays a vital communicative role for deaf-mute people. Reference is made to various techniques available for manual tracing, segmentation, feature extraction, and classification. In this paper, images are captured using a webcam and processed using image processing techniques (CNN). Convolutional neural network that automatically extracts discriminatory spatiotemporal features from the raw video stream without any prior knowledge, avoiding design features. Here, the captured gestures are stored in folders consisting of the gesture itself. When the hand gesture matches the stored images, the best match will be converted to text and speech.
Keywords
Sign Language Recognition, Convolution Neural Network, Image Processing.