Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Introduction
Sign language is a collection of gestures, postures, movements, and facial expressions used by deaf people. The Brazilian sign language is Libras. The use of Libras has been increased among the deaf communities, but is still not disseminated outside this community. Sign language recognition is a field of research, which intends to help...
Context in source publication
Similar publications
The listening or hearing impaired (deaf/dumb) people use a set of signs, called sign language instead of speech for communication among them. However, it is very challenging for non-sign language speakers to communicate with this community using signs. It is very necessary to develop an application to recognize gestures or actions of sign languages...
Wearable strain sensors, capable of continuously detecting human movements, hold great application prospects in sign language gesture recognition to alleviate the daily communication barriers of the deaf and mute community. However, the unsatisfactory strain sensing performance (such as low sensitivity, narrow detection range, etc.) and wearing dis...
American sign language is an important communication way to convey information among the deaf community in North America and is primarily used by people who have hearing or speech impairments. The deaf community faces a struggle in schools and other institutions because they usually consist primarily of hearing people. Besides, deaf people often fe...
One of the best ways of communication between deaf people and hearing people is based on sign language or so-called hand gestures. In the Arab society, only deaf people and specialists could deal with Arabic sign language, which makes the deaf community narrow and thus communicating with normal people dif ficult. In addition to that, studying the p...
The use of Filipino Sign Language (FSL) has contributed to the improvement of communication of deaf people; however, the majority of the population in the Philippines does not understand FSL. The study explored computer vision in obtaining the images and deep learning techniques in building the automated FSL recognition model to bridge the communic...
Citations
... Em uma o fundo era completamente branco e na outra as capturas eram feitas por mapa de profundidade. Os autores alcançaram 99,17% de acurácia usando um modelo baseado em rede neural convolucional (CNN) no conjunto de dados introduzido em [7] e 99,13% no conjunto em [8]. Rao Rezende et al. [4], que elaborou o conjunto de dados MINDS-Libras usado neste trabalho. ...
... AlQattan and Sepulveda (2017) used discrete wavelet transform for feature extraction and attained an accuracy of 75% and 76% using LDA and SVM classifiers. Costa et al. (2017) proposed three stages (hand segmentation, feature extraction, and classification) automatic method for recognising hand configurations of Brazilian sign language. They attain an accuracy of 96.31% using 12,200 images of Libras and 200 gestures of each hand configuration. ...
... AlQattan and Sepulveda (2017) used discrete wavelet transform for feature extraction and attained an accuracy of 75% and 76% using LDA and SVM classifiers. Costa et al. (2017) proposed three stages (hand segmentation, feature extraction, and classification) automatic method for recognising hand configurations of Brazilian sign language. They attain an accuracy of 96.31% using 12,200 images of Libras and 200 gestures of each hand configuration. ...
... Different studies approach sign language recognition as a hand configuration classification problem by identifying the pose performed in an image or a sequence of images [3]. However, these studies do not address the applicability of this knowledge for accessibility applications because they work with hand images isolated with human assistance in controlled environments [4], [5]. For instance, there is no constant illumination or static background in a real-world scenario, making it difficult to achieve automatic hand isolation. ...
... The main common characteristics are: (1) the type of sign and (2) sensors used for hand pose/movement acquisition. The primary types of signs in literature are static and dynamic, where static signs consider a hand pose [4], [6]- [8] and dynamic signs consider the sequence of movements in combination with the hand poses [9]. For sensors, there are different sensors considered in previous works, from wearable devices [10] to depth cameras [4], [6]- [8]. ...
... The primary types of signs in literature are static and dynamic, where static signs consider a hand pose [4], [6]- [8] and dynamic signs consider the sequence of movements in combination with the hand poses [9]. For sensors, there are different sensors considered in previous works, from wearable devices [10] to depth cameras [4], [6]- [8]. This study considers static signs (poses) as hand configurations in images and videos (frames) from RGB cameras. ...
Despite the recent advancements in deep learning, sign language recognition persists as a challenge in computer vision due to its complexity in shape and movement patterns. Current studies that address sign language recognition treat hand pose recognition as an image classification problem. Based on this approach, we introduce HandArch, a novel architecture for real-time hand pose recognition from video to accelerate the development of sign language recognition applications. Furthermore, we present Libras91, a novel dataset of Brazilian sign language (LIBRAS) hand configurations containing 91 classes and 108,896 samples. Experimental results show that our approach surpasses the accuracy of previous studies while working in real-time on video files. The recognition accuracy of our system is 99% for the novel dataset and over 95% for other hand pose datasets.
... A database is a set of structured information that relates to itself (numbers, images or videos). Previously, in [5,35,39], for recording data of Libras signs, the authors have used multimodal sensors such as RGB-D (Red, Green, Blue and Depth). Others have addressed the problem through gloves and wearable sensors [59]. ...
Sign language recognition is considered the most important and challenging application in gesture recognition, involving the fields of pattern recognition, machine learning and computer vision. This is mainly due to the complex visual–gestural nature of sign languages and the availability of few databases and studies related to automatic recognition. This work presents the development and validation of a Brazilian sign language (Libras) public database. The recording protocol describes (1) the chosen signs, (2) the signaller characteristics, (3) the sensors and software used for video acquisition, (4) the recording scenario and (5) the data structure. Provided that these steps are well defined, a database with more than 1000 videos of 20 Libras signs recorded by twelve different people is created using an RGB-D sensor and an RGB camera. Each sign was recorded five times by each signaller. This corresponds to a database with 1200 samples of the following data: (1) RGB video frames, (2) depth, (3) body points and (4) face information. Some approaches using deep learning-based models were applied to classify these signs based on 3D and 2D convolutional neural networks. The best result shows an average accuracy of 93.3%. This paper presents an important contribution for the research community by providing a publicly available sign language dataset and baseline results for comparison.
... In order to exemplify the last comment, it will be described a classification framework for 61 static symbols presented in Costa et al. [10]. The hand segmentation was performed by using the resources of a Kinect and the video sequence. ...
... More recently, Costa Filho et al. [13] applied a large set of image descriptors in preprocessing, achieving more than 95% on 20 signs recognition. In the second experiment, using 2745 signs, the best result found was 40.25%. ...
In Brazil, only 15% of deaf people are fluent in Brazilian Portuguese. Although Brazilian law says that public spaces should provide access in Libras, deaf still face lots of issues accessing customer services. Also, there is no tool able to allow them to talk in Libras via customer service communication channels, such as chats. Thus, this work aims to help deaf to access the customer services of a big computer manufacturer by providing a way for them to communicate in Libras while the call center attendant receives a translation in Brazilian Portuguese. The method to translate Libras to Portuguese includes the use of a CNN. The results in recognizing and classifying signs (captured from video) reached over 90% of the accuracy in the training phase.
... Um sistema para reconhecimento totalmente automatizado de 61 configurações de mão em Libras utilizando um classificador de novidade proposto previamente pelos autores onde as técnicas de extração de atributos podem ser a twodimensional LDA ou a two-dimensional PCA é apresentado em [14]. Os autores utilizam o Microsoft Kinect para capturar as imagens produzindo uma nova base de dados de 12.200 imagens com 200 repetições de cada uma das 61 configurações de mãos realizadas por 10 pessoas diferentes. ...
... A tese de mestrado em [15] aborda a utilização de redes neurais convolucionais (CNN) na classificação de 61 configurações de mãos de LIBRAS. O trabalho utiliza a mesma base de dados aplicada em [14] e testa 3 variações de arquitetura: 1) Alexnet [16]; 2) Variação da arquitetura apresentada em [17]; e 3) Variação da arquitetura apresentada em [18], combinadas com as técnicas de regularização. Desta forma são gerados 12 modelos, cada uma das 3 arquiteturas sem regularização, com regularização dropout, com regularização L2 e com dropout+L2. ...
... Uma base de dados de 61 configurações de mão pode ser encontrada em [14]. A base foi executada por 10 pessoas diferentes com 200 repetições cada, totalizando 12.200 imagens. ...
... So, Neural net is not suitable for real time skin detection [16]. K Nearest Neighbour algorithm was used with PCA and achieve 96% accuracy [17]. analysis of different tools and techniques with respect to time and accuracy. ...
... Recently, alternative solutions have been developed to help solving the communication problem among hearing impaired people and a listener. As for example, the use of systems that use depth maps obtained by a Kinect sensor to do the recognition of the configurations of the hands [11]. It is an interesting method to help overcoming this barrier. ...
