Interactive sign language interpreter using skeleton tracking
The aim of this paper is to introduce an interactive communication system that will benefit both people with hearing and verbal difficulties to convey in the form of the sign language naturally. The idea is to provide two ways of communication between two users by converting sign language to voice a...
| 总结: | The aim of this paper is to introduce an interactive communication system that will benefit both people with hearing and verbal difficulties to convey in the form of the sign language naturally. The idea is to provide two ways of communication between two users by converting sign language to voice and text and provides means of returning communication feedback whereby the other party can speak or key-in text and translates it into sign language movement performed by a three-dimensional (3D) model. A Microsoft Kinect device is used to captures the sign movements by optimizing the skeleton tracking algorithm to understand specific hands movements and dictates using the pre-recorded gesture library to digitized voice and using the same apparatus, speech is translated back into sign language. Research leads in helping the disables have been carried out extensively and majority focuses on only using single type of motion sensing technology such as TOBII (eye tracking) and LEAP (leap motion) which are either costly or limited to a small workable space. Microsoft Kinect technology would be a genuinely equipment used to create a cost-effective and capable technology prototype that enables sign-language communication between signer and non-signer, thus, offers translation into Bahasa Malaysia text. |
|---|---|
| ISSN: | 21801843 |