Malaysian Sign Language Real-Time Tutorial using CNN Algorithm

Abstract

The communication challenges faced by people with hearing and speech impairments in Malaysia are made worse by effective resources for learning Malaysian Sign Language (MSL). The Malaysian Sign Language Real Time Tutorial (MASRETT) is an instructional website designed to decrease the communication gap between both disabled and non-disabled. MASRETT is developed to assist in learning fundamental MSL virtually besides reducing the communication barriers which leads to a better and stronger social relation in Malaysia. Agile software development is the methodology of choice for creating this system since it facilitates the iterative identification and correction of mistakes. The artificial intelligence (AI) model which is convolutional Neural Network (CNN) algorithm is used to recognize the sign language input. The findings indicate that MASRETT significantly enhances accessibility and effectiveness of MSL learning particularly for individuals with lack access to traditional classes. However, the results showed that only 33% of the signatures are recognized using CNN algorithm for real time signs detection. It is observed that the results are not significant because of the limitation in data and inefficient model usage and can be improved using improved algorithm as in future work.