No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
“Hasta Mudra”: An interpretation of Indian sign hand gestures
Abstract
Hasta Mudra - a Sanskrit word resembles as a hand gestures which are practiced individually or as a series of gestures flowing one into the next. This paper gives glimpse of Indian Sign Language, its dialects and varieties and recent efforts in the direction of its standardiz ation. Also a proposed methodology is discussed to recognize static single hand gestures of a subset of Indian sign language. The present achievements provide the basis for future applications with the objective of supporting the integration of deaf people into the hearing society. The proposed recognition system aims for signer-independent operation and utilizes a single web camera for data acquisition to ensure user-friendliness. The goal is to create a system which can identify gestures of human hand and use them to convey information without the use of interpreter. This paper has summarized the study of Indian sign language and its varieties. Also a simple recognition system is proposed.
... Hearing impaired and hearing people have a major gap between their source of communication [24,74]. A major reason for the communication barriers is that majority of hearing impaired users are illiterate [11,27,69,80,85]. To overcome the communication gap between the hearing impaired and hearing people there is an urgent need of translator(s) [62,77]. ...
Sign language (SL) is the best suited communication medium for hearing impaired people. Even with the advancement of technology, there is a communication gap between the hearing impaired and hearing people. The aim of this research work is to bridge this gap by developing an automatic system that translates the speech to Indian Sign Language using Avatar (SISLA). The whole system works in three phases: (i) The first phase includes the speech recognition (SR) of isolated words for English, Hindi and Punjabi in speaker independent environment (ii) The second phase translates the source language into Indian Sign Language (ISL) (iii) HamNoSys based 3D avatar represents the ISL gestures. The four major implementation modules for SISLA include: requirement analysis, data collection, technical development and evaluation. The multi-lingual feature makes the system more efficient. The training and testing speech sample files for English (12,660, 4218), Hindi (12,610, 4211) and Punjabi (12,600, 4193) have been used to train and test the SR models. Empirical results of automatic machine translation show that the proposed trained models have achieved the minimum accuracy of 91%, 89% and 89% for English, Punjabi and Hindi respectively. Sign language experts have also been used to evaluate the sign error rate through feedback. Future directions to enhance the proposed system using non-manual SL features along with the sentence level translation has been suggested. Usability testing based on survey results confirm that the proposed SISLA system is suitable for education as well as communication purpose for hearing impaired people.
... The suggested model is implemented as a combination of two algorithm layers that anticipate the final sign from the user. In the first layer, to obtain the processed image during extracting features, apply the gaussian blur filter and threshold to the image captured using OpenCV [15]. This modified image is sent to the CNN prediction model, but if a character is identified for more than 50 frames, it is displayed and used to create the word. ...
The ancient way of sign language is most natural forms of communication. The recognition of sign is place a key role in research field. The development and improvement on this kind of work need more and more new techniques to analyze the accurate results. Many people don't know it and interpreters are hard to come by, we developed a real-time technique for finger spelling-based American Sign Language using neural networks. In our technique, the hand is first sent through a filter, and then it is passed through a classifier, which analyses the class of hand movements. For each alphabet the proposed model has a 96 percent accuracy rate. This model mainly implemented for Dumb and Deaf people for communication.
... A pooling layer is another building block of CNN [8]. The main aim of pooling is to downsample the image matrix into a smaller matrix. ...
... The obtained sentence is conveyed to a normal user in the text and speech form. [9] [11]. ...
... Futane, P. R., et al., [6] in their work proposed two approaches, one for device based and the second is vision based for Indian sign language recognition. In the device based approach they used an electronic glove for sign input. ...
Sign language recognition is helpful in communication between signing people and non-signing people. Various
research projects are in progress on different sign language recognition systems worldwide. The research is limited to a
particular country as there are country wide variations available. The idea of this project is to design a system that can
interpret the Indian sign language in the domain of numerals accurately so that the less fortunate people will be able to
communicate with the outside world without need of an interpreter in public places like railway stations, banks, etc. The
research presented here describes a system for automatic recognition of Indian sign language of numeric signs which are in
the form of isolated images, in which only a regular camera was used to acquire the signs. To use the project in real
environment, first we created a numeric sign database containing 5000 signs, 500 images per numeral sign. Direct pixel
value and hierarchical centroid techniques are used to extract desired features from sign images. After extracting features
from images, neural network and kNN classification techniques were used to classify the signs. The result of these
experiments is achieved up to 97.10% accuracy.
... In this approach a whole lot of 3D model data is used which makes the system clumsy. Attempt to automatic translation of static as well as dynamic gestures of ISL with image processing features such as skin tone detection spatial filter velocimetry and temporal tracing are used [15] [16]. The power spectrum representation of each gesture is given as motion prints. ...
This paper proposes a robust approach to recognize hand gesture
which involves face parts like chin, cheeks, eyes and head in the context of sign
language recognition and also attacks the problem of interference between face
and hand. ICondensation algorithm is used to track the face, skin color
segmentation is applied on face to eliminate eyes, and simple four quadrant
multi clue information of face is obtained. Simultaneously two hands are
tracked by another ICondensation module and BRIEF feature descriptors are
extracted from hand. The multi clue information from four quadrants of face is
identified whenever intersection of two tracking modules occurs. This
intersection information provides which part of the face is being referred by the
hand. Along with BRIEF feature descriptor, face position is used as feature for
the gesture recognition. SVM multi-class classifier is used for continuous hand
gestures classification. Experimentation is carried out with Indian Sign
Language and found that the proposed approach outperforms the other existing
methods with the recognition rate of 93.21%.
... Lyy (X, Da) (5) where lxx, Ixy and Iyy are the second-order partial derivatives of the image evaluated at the point x and aD is the Gaussian scale of the partial derivatives. Note that both the Hessian matrix and the related second moment matrix have been applied in several other interest operators (e.g., the Harris, Harris-affine, and Hessian-affine detectors) to find image positions where the local image geometry is changing in more than one direction. ...
This paper proposes an automatic gesture recognition
approach for Indian Sign Language (ISL). Indian sign language
uses both hands to represent each alphabet. We propose an
approach which addresses local-global ambiguity identification,
inter-class variability enhancement for each hand gesture. Hand
region is segmented and detected by YCbCr skin color model
reference. The shape, texture and finger features of each hand are
extracted using Principle Curvature Based Region (PCBR)
detector, Wavelet Packet Decomposition (WPD-2) and complexity
defects algorithms respectively for hand posture recognition
process. To classify each hand posture, multi class non linear
support vector machines (SVM) is used, for which a recognition
rate of 91.3% is achieved. Dynamic gestures are classified using
Dynamic Time Warping (DTW) with the trajectory feature vector
with 86.3% recognition rate. The performance of the proposed
approach is analyzed with well known classifiers like SVM, KNN
& DTW. Experimental results are compared with the
conventional and existing algorithms to prove the better efficiency
of the proposed approach.
Gesture-based communication acknowledgment is useful in correspondence between marking individuals and non-marking individuals. Different research projects are underway on various communication via gesture acknowledgment frameworks around the world. The examination is restricted to specific countries as there are country-wide varieties accessible In this paper the hand signals compared to ISL English letters in order are caught through a webcam. In the caught outlines the hand is divided and the neural organizations are utilized to perceive the letters in order. The elements like points made between fingers, the number of fingers that are completely opened, completely shut, or semi-shut, and recognizable proof of each finger contribute to the neural organization. Trial and error were accomplished for single-hand letter sets and the results are summed up. In this paper, we have mentioned a comparison of various techniques for using recognition of gestural language like a neural network, CNN, deep learning, artificial neural network, and autoencoder work done to determine better results.
Gesture-based communication acknowledgment is useful in correspondence between marking individuals and non-marking individuals. Different research projects are underway on various communication via gesture acknowledgment frameworks around the world. The examination is restricted to a specific country as there are countrywide varieties accessible. In this paper, the hand signals compared to ISL English letters in order are caught through a webcam. In the caught outlines, the hand is divided and the neural organizations are utilized to perceive the letters in order. The elements like points made between fingers, the number of fingers that are completely opened, completely shut, or semi-shut, and recognizable proof of each finger are utilized as contributions to the neural organization. Trial and error were accomplished for single-hand letter sets, and the results are summed up. In this paper, we have mentioned a comparison of various techniques for using recognition of gestural language like a neural network, CNN, deep learning, artificial neural network, and autoencoder work done for determining better results.
Sign language is a type of language that includes postures and body motions in addition to hand gestures. For ages, sign language was the only way to connect with each other. But in early times, without the knowledge of different varieties of language, it became hard to communicate. Now as the world is becoming more advanced and digitalised, deaf and blind people find the basic mode of communication more disrupting and uneasy. To resolve this issue, Sign language recognition/interpreter system becomes a necessity to help the people in need. This is possible because to Machine Learning and Human Computer Interaction (HCI).
This project helps mute people to interact with the rest of the world using sign language. Communication is an important aspect of human beings. Interaction between normal people and an impaired person is very difficult because of communication barriers. This work includes a voice-based and text-based interaction approach. An interpreter is responsible for helping impaired individuals understand what is being said in a variety of situations. The main feature of this project is that it can be used to predict sign language and translate it to voice so that normal people can understand sign language without the need of learning them. In this project, gesture to speech translation is done for Hindi as well as Tamil language, Hindi being the nation's most spoken language, and Tamil being the language used in most parts of southern India has increased the requirement of this model. For future scope, the model created in this project can be used to develop an application that involves not one but two-way communication.
Communication for dumb and deaf community is done by the sign language using hand and finger movements. Adding grammar and emotion to a sentence require lips movement, eye gaze, and facial expressions. In this paper, we have discussed many research papers related to Indian sign language gesture recognition. For static gestures 80.2–99.73% and for dynamic gestures 72.3–99.08% result in accuracy rates found with some certain specific environmental conditions. Papers are analyzed based on dataset acquisition method, feature extraction techniques, and classification techniques with their comparative graphs tables. Some paper lag facial expression with hand gesture, hence further research is required in this direction. From the entire techniques, deep learning convolution neural network (CNN), support vector machine (SVM), and hidden Markov model (HMM) were found to give the better result as compared to others.KeywordsSign languageGesture recognitionNeural network (NN)Multimodal frameworkDeep learning
Nowadays, email is an important medium of communication used by almost everyone whether for official or personal purposes, and this has encouraged some users to exploit this medium to send spam emails either for marketing purposes or for potentially harmful purposes. The massive increase in the number of spam messages led to the need to find ways to identify and filter these emails, which encouraged many researchers to produce work in this field. In this paper, we present a method for identifying and detecting spam email messages based on their contents. The approach uses the mutual information contents method to define the relationship between the text the email contains and its class to select the most frequently used text in spam emails. The random forest classifier was used to classify emails into legitimate and spam due to its performance and the advantage of overcoming the overfitting issue associated with regular decision tree classifiers. The proposed algorithm was applied to a dataset containing 3000 features and 5150 instances, and the results obtained were carefully studied and discussed. The algorithm showed an outstanding performance, which is evident in the accuracy obtained in some cases, which reached 97%, and the optimum accuracy which reached 96.4%.
Workflow scheduling in clouds refers to mapping workflow tasks to the cloud resources to optimize some objective function. Workflow scheduling is a crucial component behind the process for optimal workflow enactment. It is a well-known NP-hard problem and is more challenging in the heterogeneous computing environment. Cloud environments confront several issues, including energy consumption, implementation time, emissions of heat and CO and running costs. The increasing complexity of the workflow applications forces researchers to explore hybrid approaches to solve the workflow scheduling problem. Efficient and effective cloud workflow planning is one of the most important approaches to address the above difficulties and make optimal use of resources. This study suggests energy awareness, based on the methodology whale optimization algorithm (WOA). Our objective is to decrease the energy consumption and maximize the throughput of computational workflows which impose a considerable loss on the quality of service guarantee (QoS). The proposed method is compared with other standard state-of-the-art techniques to analyze its performance.KeywordsWhale optimization algorithmCloud computingEnergyThroughputCostPhysical machine
Mudras are considered as spiritual gestures in the religious sense and hold a very important place in the cultural and spiritual space in India. Images are the symbolic representations of divinity in religious artwork and their origins are conveyed through the religions and spiritual beliefs. Such gestures also have some specific meaning in the Buddhist religion. It refers to some of the events in the life of Buddha or denotes special characteristics of the Buddha deities. In recent years, automatic identification of these gestures has gained a greater interest from the machine learning community. This would help to identify the various deities that exist in the Buddhist religion, leading to digital preservation of cultural heritage art. This paper provides a framework that recognizes the Buddhist hand gesture or Hasta Mudra. The morphological features are extracted from the gesture employing geometric parameters. The experimental results show that utilising geometric features and using k-Nearest Neighbor (kNN) as a classifier, an approximately 70% recognition rate is achieved.
Cultural patterns have been generated from the prehistoric period and have percolated down over the ages with evolution and constant evaluation by successive generations. Such patterns are created using mechanical automation. This article is an attempt to capture such cultural patterns using iconometric analysis. In this article, first, a brief study of various technical research carried out on cultural heritage related to Buddhist iconography has been presented. And then, a computational technique has been proposed to digitize the method that is followed by the Thangka practitioners for drawing the portrait of Buddha on Thangka. The methodology can be used for preservation and learning of the patterns of special significance in this rich art form. These pattern generation techniques can be used as a learning aid in the study of subjects and can generate newer interpretations from patterns in numerous Buddha images and paintings that have been developed over the ages and by various sects within the Buddhist practice. This method, when used to churn out rich cultural heritage artefacts, can lead to the development of centralized digital archives which provide opportunities to the common people to get access to such heritage objects which otherwise may have been hard to lay eyes upon. This will also help in scientific authentication of the present repository of historical artefacts. The article discusses the automated generation of Tibetan thangka−based Buddha faces, satisfying the iconometric proportion as used by different Buddhist schools of thought. The images generated by this method can be considered as authentic Buddha Images. This paper also provides the geometry for the reconstruction of an authentic Buddha image using the key facial features of Buddha. This research can help the traditional learners of Thangka painting to use modern pattern recognition techniques for skill enhancement.
Sign languages are natural languages used by hearing impaired people which use several means of expression for communication in day to day life. It relates letters, words, and sentences of a spoken language to gesticulations, enabling them to communicate among themselves. The deaf community can interact with normal people with an automation system that can associate signs to the words of speech. This will support them to enhance their abilities and make them aware of doing better for the mankind. A vision based system that provides a feasible solution to Indian Sign Language (ISL) recognition of static gestures is presented in this paper. The proposed method doesn’t require that signers wear gloves or any other marker devices to simplify the process of hand segmenting. After modeling and analysis of the input hand image, classification method is used to recognize the sign. The classification is done using Computational Neural networks(CNN). Detection using CNN is rugged to distortions such as change in shape due to camera lens, different lighting conditions, various poses, presence of occlusions, horizontal and vertical shifts, etc. We are able to recognize 5 ISL gestures with a recognition accuracy of 90.55 %.
The sign language is the essential communication method between the deaf and dumb people. In this paper, the authors present a vision based approach which efficiently recognize the signs of Indian Sign Language (ISL) and translate the accurate meaning of those recognized signs. A new feature vector is computed by fusing Hu invariant moment and structural shape descriptor to recognize sign. A multi-class Support Vector Machine (MSVM) is utilized for training and classifying signs of ISL. The performance of the algorithm is illustrated by simulations carried out on a dataset having 720 images. Experimental results demonstrate that the proposed approach can successfully recognize hand gesture with 96% recognition rate.
Sign language is widely used by individuals with hearing impairment to communicate with each other conveniently using hand gestures. However, non-sign-language speakers find it very difficult to communicate with those with speech or hearing impairment since it interpreters are not readily available at all times. Many countries have their own sign language, such as American Sign Language (ASL) which is mainly used in the United States and the English-speaking part of Canada. The proposed system helps non-sign-language speakers in recognising gestures used in American Sign Language. The system described in this paper is implemented using MATLAB. In this approach, firstly, the signs are captured using a webcam. The images captured are then processed further and the features are extracted from the captured images using PCA. Comparison of the features is done using Euclidean Distance with the training sets. Minimum Euclidean distance helps to recognise the character. This system will enable non-sign-language speakers to better understand and communicate with those with impaired hearing.
An efficient human computer interaction is assuming utmost importance in our daily lives. Human beings can communicate mainly by vision and sound. Human can recognize the meaningful expressions of motion using hand gesture. Hand Gesture is the most important to exchange ideas, messages, thoughts etc among deaf and dumb people. This paper discusses a simple recognition algorithm that recognizes the numbers from 0 to 10 using thresholding. The overall algorithm has three main steps: image capture, apply threshold and recognizing the number. The assumption is made that user must wear color hand gloves.
The Sign Language is a method of communication for deaf-dumb people. This paper presents the Sign Language Recognition system capable of recognizing 26 gestures from the Indian Sign Language by using MATLAB. The proposed system having four modules such as: pre-processing and hand segmentation, feature extraction, sign recognition and sign to text and voice conversion. Segmentation is done by using image processing. Different features are extracted such as Eigen values and Eigen vectors which are used in recognition. The Principle Component Analysis (PCA) algorithm was used for gesture recognition and recognized gesture is converted into text and voice format. The proposed system helps to minimize communication barrier between deaf-dumb people and normal people.
In this paper a Sign Language Recognition system has been proposed. The first step of this system is to create a database of Indian Sign Language. This is done by acquiring the videos from the signers while they are performing the hand gestures. Next step is Hand tracking and Segmentation. This is performed in order to extract features from a particular gesture. A three step algorithm has been used in the proposed system to get better quality hand tracking and segmentation. This algorithm works on motion tracking, edge detection and skin color detection. The system is implemented successfully and results are presented in this paper. The results demonstrate working of motion tracking, edge detection and skin color detection individually as well as their combined effect.
Sign language recognition has been a topic of research since the first data glove was developed. Many researchers have attempted to recognize sign language through various techniques. However, none of them have ventured into the area of Pakistan Sign Language (PSL). The Boltay Haath project aims at recognizing PSL gestures using Statistical Template Matching. The primary input device is the DataGlove5 developed by 5DT. Alternative approaches use camera-based recognition which, being sensitive to environmental changes are not always a good choice. This paper explains the use of Statistical Template Matching for gesture recognition in Boltay Haath. The system recognizes one handed alphabet signs from PSL.
In automatic sign language translation, one of the main problems is the usage of spatial information in sign language and its proper representation and translation, e.g. the handling of spatial reference points in the signing space. Such locations are encoded at static points in signing space as spatial references for motion events.
We present a new approach starting from a large vocabulary speech recognition system which is able to recognize sentences of continuous sign language speaker independently. The manual features obtained from the tracking are passed to the statistical machine translation system to improve its accuracy. On a publicly available benchmark database, we achieve a competitive recognition performance and can similarly improve the translation performance by integrating the tracking features.
Computer recognition of sign language is an important research problem for enabling communication with hearing impaired people. This paper introduces an efficient and fast algorithm for identification of the number of fingers opened in a gesture representing an alphabet of the American Sign Language. Finger Detection is accomplished based on the concept of Boundary Tracing and Finger Tip Detection. The system does not require the hand to be perfectly aligned to the camera or use any special markers or input gloves on the hand. Index Terms—Boundary Tracing, computer access for disabled, finger detection, image processing, sign language recognition.
The major challenges that sign language recognition (SLR) now faces are developing methods that solve large-vocabulary continuous sign problems. In this paper, transition-movement models (TMMs) are proposed to handle transition parts between two adjacent signs in large-vocabulary continuous SLR. For tackling mass transition movements arisen from a large vocabulary size, a temporal clustering algorithm improved from k-means by using dynamic time warping as its distance measure is proposed to dynamically cluster them; then, an iterative segmentation algorithm for automatically segmenting transition parts from continuous sentences and training these TMMs through a bootstrap process is presented. The clustered TMMs due to their excellent generalization are very suitable for large-vocabulary continuous SLR. Lastly, TMMs together with sign models are viewed as candidates of the Viterbi search algorithm for recognizing continuous sign language. Experiments demonstrate that continuous SLR based on TMMs has good performance over a large vocabulary of 5113 Chinese signs and obtains an average accuracy of 91.9%
Boltay Hath : Pakistan sign language
- Alvi Alim Khaleed
Alim Khaleed Alvi, "Boltay Hath : Pakistan sign language",
Thesis Report, Sir Syed University of Engineering &
Technology, Pakistan, 2001.