No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Video gestures identification and recognition using Fourier descriptor and general fuzzy minmax neural network for subset of Indian sign language
Abstract
Sign languages are natural languages that use to communicate with deaf and mute people. There exist different sign languages in the world. But we focused on Indian Sign Language which is on the way of standardization & very less work has been done on it so far. We have focused on Indian sign language history and progress in this domain and work carried out by various researchers in Indian Sign language recognition. Also we have proposed an approach that will convert the video of full sentence gesture of Indian sign language to text. It will initially identify individual words from the video & convert them on to text. Finally, the system will process those words to form a meaningful sentence in compliance with the simple grammar rules.
... Sharma et al. suggested an American Sign Language alphabet recognition system to distinguish highly ambiguous and complicated gestures based on extracting features from boundary tracing descriptor [8]. Besides, other similar works on static sign language recognition are also notable [9], [10], [11], [12], and [13]. ...
... Based on Eqs. (7)(8)(9)(10)(11), mass center of an item of digital image f(x, y) in M 9 N frame is computable though zero and first order moments [17,18] (Fig. 8). ...
... To compute hand orientation angle, we need first order moments (10,11) and second order moments (12)(13)(14)(15)). ...
A sign language is a language which uses manual communication and body language to convey meaning, as opposed to acoustically conveyed sound patterns. This can involve simultaneously combining hand shapes, orientation and movement of the hands, arms or body, and facial expressions to fluidly express a speaker’s thoughts. Sign language is a preliminary communication way for individuals with hearing and speech problems. Considering that more than a hundred million people all around the world are annoyed by hearing loss and impaired speech, it is needed to design a system for automatic sign language interpreter as an interface between deaf-and-dumb and ordinary people can feel it strongly. Given this, in this article we aimed to design such a computer vision-based translation interface. Farsi sign language recognition is one of the most challenging fields of study is given because of some features such as the wide range of similar gestures, hands orientation, complicated background, and ambient light variation. A Farsi sign language recognition system is based on computer vision which is capable of real-time gesture processing and is independent of the signer. Furthermore, there is no need to use glove or marker by the signer in the proposed system. After hand segmentation from video frames, the proposed algorithm extracts boundary of the dominant hand to compute the normalized accumulation angle and represents the boundary, so that the invariance to transition and scale change of the features is realized at this stage. Afterward, Fourier coefficients amplitude is extracted as preferred features at the frequency domain, while invariance to rotation of the features is added at this point. Then the frequency features, as extracted features for gesture recognition, are applied to inputs of feed-forward multilayer perception neural network. The proposed method is presented to make recognition system independent of the signer and retrofit it against signer’s distance changes from camera using features of powerful invariant extraction against transition, scale change, and rotation. Data classification is carried out by three classifiers including Bayes, K-NN, and neural network. Performance of the classifiers was also compared. Training set of gestures comprised 250 samples for 10 gestures and 5 positions and orientations that were performed by 5 individuals. Recognition results showed an outstanding recognition rate of the system.
... The classifiers are trained using labeled datasets, where each gesture is associated with a specific label. Through the learning process, the classifiers are able to generalize and recognize previously unseen gestures based on the learned patterns and relationships in the training data [18]. ...
... By combining the powerful features of these libraries, it becomes possible to capture frames, identify key points representing the hand, and employ classifiers to accurately predict the gestures being made. The utilization of OpenCV and OpenPose provides developers with a comprehensive toolkit to build robust and efficient gesture recognition systems, paving the way for enhanced communication and accessibility for individuals who rely on sign language [15,18].While image processing techniques can be powerful tools for sign language prediction, they are not without their limitations and drawbacks. Here are some of the main drawbacks associated with relying solely on image processing for sign language predictions. ...
This book series will be the great platform to share ideas and knowledge among the Industry experts, researchers and academics. Several current and upcoming frontier technologies, innovative solutions, research results, as well as enterprises related to internet of things and their applications will be published.
... Sign language is a method of communication between normal and deaf-mute people. FMNN and its variants have been used to also build a sign language recognition system [89,90,137]. This system identifies the signs and converts them to text. ...
... This drawback is considered and partly solved by Gabrys and Bargiela [28], Nandedkar and Biswas [38], Nandedkar and Biswas [75], Wang et al. [30]. [91,133] Biometrics Character recognition [92] Emotion recognition [128][129][130][131][132] Face detection [106] Hand gesture recognition [105] Human action recognition [66] Human motion recognition [134,135] Numeral character recognition [89,90,137] Sign language recognition [93][94][95] Signature recognition [96,97,136] Speaker/speech recognition [132] Wood defect detection, Face detection [155] Cybersecurity Attack intention recognition [50,51,60] Intrusion detection [102,168] General purpose Colour image segmentation [150,151] Colour video segmentation [154] Image retrieval [169] Object recognition [152,153] Shadow detection and removal [170] Wood defect detection [28,46,103,124,125,171,172] Industrial application Fault detection in a water system [45] Fault detection in cooling system [39] Fault detection in energy systems [173] Fault detection of ball bearing [99,101,[120][121][122][123][174][175][176][177] Fault detection of induction motors [43,70,126] Fault detection of oil pipeline [59] Motor fault detection [67,127] Pipeline defect detection [68,69] Power quality monitoring [144] Medical application Acute Coronary Syndrome [146] Brain glioma [139,140] Cervical cancer [141] Classification of chronic leukaemia [31] Classifying breast cancer patients [147,148] Fall detection [178] Genetics [48,145] Heart diseases [142,143] Liver disease [138] Lung cancer [178] Lung nodules detection [179] Medical diagnosis [180] Patient admission prediction [181] Signal recognition of brain [47] Stroke [149] Wheelchair control [160] Other Agricultural circular economy region division [165] Autonomous vehicle navigation [166] Fault detection in rail vehicle systems [164] Classification of music by composer [182] Clustering of radar pulses [163] Clustering of the satellite infrared images [71] Financial problems [162] Flood forecasting [167] Imputation of missing values in surveys of voting intention polls [160] Special education of learning disabled learners [156] Robotics Fault detection [157] Human-robot affective relationship [158] Human-robot affective relationship ...
The amount of digital data in the universe is growing at an exponential rate with the rapid development of digital information, and this reveals new machine learning methods. Learning algorithms using hyperboxes are a subsection of machine learning methods. Fuzzy min–max neural network (FMNN) are one of the most common and advanced methods using hyperboxes. FMNN is a special type of NeuroFuzzy system that combines the artificial neural network and fuzzy set into a common framework. This paper conducts an extensive bibliometric and network analysis of FMNN literature. Two hundred and sixty-two publications are analysed from the period of 1992–2022. Several analyses are realized in order to identify trends, challenges and key points in a more scientific and objective way that affect the development of knowledge in the FMNN domain. It can be seen from bibliometric analysis that there is rapid development in the last 10 years. Social network analysis results show that Chee Peng Lim is the most active author in the network. Besides, the modifications of FMNN are generally developed for classification. However, there are still potential future research opportunities for clustering.
... The dataset comprising of 182 (26×7) images of twenty six alphabets (seven each) was collected from "Indian Sign Language Dictionary", released by Ramakrishna Mission Vidyalaya College of Education, Coimbatore in 2001 . [9] [14] Samples were previously available in the form of videos. So, these videos were converted into still frames using window media player. ...
... Following shape features are used in this research, the reason for picking these shape signatures is as they are frequently involved in recent FD executions and have been shown best for regular shape representation [14]. ...
Language has a prime role in communication between persons, in learning, in distribution of concepts and in preserving public contacts. The hearing-impaired have to challenge communication obstacles in a mostly hearing-capable culture. There are hundreds Sign Languages that are used all around the world today .The Sign Languages are established depending on the country and area of the deaf public. The aim of sign language recognition is to offer an effectual and correct tool to transcribe hand gesture into text. It can play a vital role in the communiqué between deaf and hearing society. Sign language recognition (SLR), as one of the significant research fields of human–computer interaction (HCI), has produced more and more interest in HCI society. Since, artificial neural networks are best suited for automated pattern recognition problems; they are used as a classification tool for this research. Back propagation is the most important algorithm for training neural networks. But, it easily gets trapped in local minima leading to inaccurate solutions. Therefore, some global search and optimization techniques were required to hybridize with artificial neural networks. One such technique is Genetic algorithms that imitate the principle of natural evolution. So, in this article, a hybrid intelligent system is proposed for sign language recognition in which artificial neural networks are merged with genetic algorithms. Results show that proposed hybrid model outperformed the existing back propagation based system.
... Indo-Pakistani Sign Language (IPSL) is the predominant sign language variety in South Asia, used by at least several hundred thousand deaf signers (2003). [1] Reference [3] has discussed about the Indian sign language (ISL), its history and the progress made in the development of ISL. The details of the deaf population in India over the last five decades are given in the Table. ...
... NUMBER OF DEAF PEOPLE IN INDIA[2,3] ...
The major challenge that faces Indian Sign Language recognition now is developing methods/processes that will scale well in a better shape with increasing research findings by various researchers and supports by various institutions & government organization working for the standardization of sign language. Sign language recognition is best expresses through the gestures. Gesture is the most important to exchange ideas, messages, thoughts etc among deaf and dumb people. Gestures are a basic concept of communication and were used by humans even before speech developed; they have the potential to be a huge fortification to an intuitive human-computer communication. In this work, we would like to present a framework to support a simple step by step approach to research, analysis and designing sign gesture-based data acquisition and recognition. We would like to propose sign gesture-based data acquisition framework for collecting input from multiple input sources. This framework further distils the multiple input sources into a generic engine from which the input is read and processed by a gesture data acquisition processor, an abstraction of data processing functionality such as parsing and sign gesture analysis will be done. Further it will be outputted into various possible options. This framework will supports and keep track of multiple sources input gestures including physical gestures, methods/approaches for processing mechanism, output technologies and user goals-as the basis from which the framework may extends as per the application requirements. Each section will be well defined and tested using pragmatic experiments supported with qualitative approaches and methods. The framework is intended as a helping aid to guide research, analysis and design, and presents a generic skeleton for providing a theoretical understanding of gesture sign data acquisition, processing and recognition. Further a simple method of codeword assignment with features like angle, orientations are identified and a theoretical finding about recognition is discussed.
... In video based sign language translation, sign data set were collected from the deaf and dumb persons then recognizing the words individually by capturing the frames and it will be converted to sentence [4] and then system processed individual words to form a meaningful sentence with the grammar rules. Arabic sign language DOI Number: 10.5958/0976-5506.2018.00144.4 ...
Generally speaking and hearing are two simple practices of communication. The deaf and dumb people experiences difficult to communicate with normal people. To resolve this difficulty Indian Sign Language (ISL) recognition system is developed. Our Indian Sign Language system uses both hand gesture image and NAM (Non-Audible-Murmur) speech to enhance accuracy of recognition system. From input image hand sign feature are taken by DWT (Discrete-Wavelet-Transform) after preprocessing. And from NAM speech features are taken by MFCC (Mel-Frequency-Cepstral-Coefficients).In this work we implemented fusing the image and audio features and as well as fusing the classification techniques for better recognition. We used HMM (Hidden-Markov-Model) and ANN (Artificial-Neural-Network) for classification. The experimental results shows maximum average recognition rate as 70.19% of the ISLR system while fusing sign image and NAM features with ANN classifier, 79.72% with HMM classifier and 88.84% while fusing classifiers. © 2018, Indian Journal of Public Health Research and Development. All rights reserved.
In recent years, a growing number of healthcare organizations have been affected by data breaches. In healthcare entities, preventing data security incidents mainly requires a comprehensive understanding of the context and the factors contributing to such threats. Utilizing publicly available federal data, we examine, from a spatio-temporal perspective, the factors and the context associated to data breaches targeting healthcare sector in the United States (US) between 2009 and 2021. We shed light on healthcare data breaches that occurred during the Covid19 Pandemic era to gain a better understanding of the impact of the global context on data breach incidents. In our findings, the hacking and IT incidents are the most common types of breaches targeting specifically Healthcare providers entities. Victim healthcare organizations are more frequent in Texas, California and Florida. Over the course of years, Emails and Network servers have become the main targeted breach locations. By understanding and identifying factors related to data breach incidents, healthcare IT security professionals can mitigate the risk of leaks.KeywordsHealthcare data breachesdata securitydata analysistime series analysis
Neural networks have been widely used in many application areas such as power systems, weather forecasting, face recognition, behaviour analysis, fingerprint identification, healthcare, fault detection, flood monitoring system, and surveillance system, and navigation. However, classic neural network methods are insufficient for real-life applications. To tackle with this problem, using the learning capability of neural networks and deduction capability of fuzzy systems, neural networks and fuzzy sets are incorporated called as NeuroFuzzy. The Fuzzy min-max neural network (FMNN) is a special type of NeuroFuzzy. In this paper, a bibliometric analysis is conducted on FMNN literature. We consider the studies that are published in the last decade due to there is a jump in this field in the last 10 years. Social network analysis results show that Chee Peng Lim, is the most influential researcher in the network. The Neurocomputing is found to be the most influential journal, publishing 12% of all publications in this field. In addition, the International Conference on Computing, Communication, and Networking Technologies is the most influential conference on FMNN. The findings of this paper can draw a road map for researchers in the FMNNs.KeywordsFuzzy Min-Max Neural NetworkHyperboxClassificationClustering
Sign language gesture recognition is a vast area of research as it is a means of communication between the people without hearing disability and the deaf and dumb people. Many researchers have contributed to develop systems which will detect the signs of letters and words and generate a meaningful output which will help the two communities to interact with each other. This paper is a review of various sign language recognition techniques which are generated to solve the problem of sign language recognition along with their advantages and disadvantages. The aim of this research is to find out best possible system to be implemented for sign language recognition which has the highest accuracy and requires minimum or no hardware.KeywordsSign language recognition (SLR)Artificial intelligence networkNeural and fuzzy logicCombinational neural networks (CNN)Word error rate (WER)
This study is proposed to review the sign language recognition system based on different classifier techniques. Mostly the Neural Network and Deep Learning-based classifiers were utilized to recognize the different sign languages and this survey is proposed to review the best classifier model to represent sign language recognition (SLR). We focused mainly on deep learning techniques and also on Arabic sign language recognition systems. Numerous classifiers like CNN, RNN, MLP, LDA, HMM, ANN, SVM, KNN and more were implemented to the SLR system. Each classifier is reviewed with the recognition accuracy, in which the deep learning-based classifiers executed the optimal recognition result as contrasted to the other types of classifiers.
In present bailiwick of sales promotion, advertisements act as an accelerating puppet to lure the customers to think about their products. The escalated competition between product promoters engulfed them to go back to very purpose of advertisement which is to inform, attract and create awareness about their products among the competitors. The momentous study examines the interrelationship between advertisements, attitude toward advertisements and purchase intention. Further some, this study concentrates on the mediating role of attitude towards advertisements in between the purchase intention and advertisements. The study was conducted among 320 common people from various walks of life in Chennai city was used for the study. The hypotheses were examined by linear regression and Sobel test. Findings reveled that both advertisements and attitude towards advertisements have a positive influence on purchase intention and also the mediating element attitude towards advertisements have a partial relationship between advertisements and purchase intentions. © 2018, Indian Journal of Public Health Research and Development. All rights reserved.
Human beings interact with each other either using a natural language channel such as words, writing, or by body language (gestures) e.g. hand gestures, head gestures, facial expression, lip motion and so on. As understanding natural language is important, understanding sign language is also very important. The sign language is the basic communication method within hearing disable people. People with hearing disabilities face problems in communicating with other hearing people without a translator. For this reason, the implementation of a system that recognize the sign language would have a significant benefit impact on deaf people social live. In this paper, we have proposed a marker-free, visual Indian Sign Language recognition system using image processing, computer vision and neural network methodologies, to identify the characteristics of the hand in images taken from a video trough web camera. This approach will convert video of daily frequently used full sentences gesture into a text and then convert it into audio. Identification of hand shape from continuous frames will be done by using series of image processing operations. Interpretation of signs and corresponding meaning will be identified by using Haar Cascade Classifier. Finally displayed text will be converted into speech using speech synthesizer.
This paper proposes a real time approach to recognize gestures of sign language. The input video to a sign language recognition system is made independent of the environment in which signer is present. Active contours are used to segment and track the non-rigid hands and head of the signer. The energy minimization of active contours is accomplished by using object color, texture, boundary edge map and prior shape information. A feature matrix is designed from segmented and tracked hand and head portions. This feature matrix dimensions are minimized by temporal pooling creating a row vector for each gesture video. Pattern classification of gestures is achieved by implementing fuzzy inference system. The proposed system translates video signs into text and voice commands. The training and testing of Fuzzy Inference system is done using Indian Sign Language. Our data base has 351 gestures with gesture repeated 10 times by 10 different users. We achieved a recognition rate of 96% for gestures in all background environments.
The Sign Language is a method of communication for deaf - dumb people. This paper proposes a method that provides a basis for the development of Sign Language Recognition system for one of the south Indian languages. In the proposed method, a set of 32 signs, each representing the binary ‘UP’ & ‘DOWN’ positions of the five fingers is defined. The images are of the palm side of right hand and are loaded at runtime i.e. dynamic loading. The method has been developed with respect to single user both in training and testing phase. The static images have been pre-processed using feature point extraction method and are trained with 10 numbers of images for each sign. The images are converted into text by identifying the finger tip position of static images using image processing techniques. The proposed method is able to identify the images of the signer which are captured dynamically during testing phase. The results with test images are presented, which show that the proposed Sign Language Recognition System is able to recognize images with 98.125% accuracy when trained with 320 images and tested with 160 images.
Automatic gesture recognition could be used to provide feedback in a computer-based learning environment to practice sign language skills for the deaf. In the present work the gestures made by the deaf as a communication aid, as per the Indian Sign Language were captured and a method was evolved to recognize them as English alphabets. After segmenting and removing the noise, the gestures were recognized in two stages. In the first stage, a common feature vector was calculated for all the alphabets, which captured the properties of overall shape of the gesture. If the gesture passed the first stage, then in the second stage a feature vector was calculated for that part of the gesture that actually represented the alphabet. The approach was evaluated by training the module on data of five users and testing on data of another five users. The dataset was characterized by varied user characteristics including clothing and skin tones. The module is user-independent and could be used for education as well as for entertainment purposes.
This paper describes a general fuzzy min-max (GFMM) neural network which is a generalization and extension of the fuzzy min-max clustering and classification algorithms of Simpson (1992, 1993). The GFMM method combines supervised and unsupervised learning in a single training algorithm. The fusion of clustering and classification resulted in an algorithm that can be used as pure clustering, pure classification, or hybrid clustering classification. It exhibits a property of finding decision boundaries between classes while clustering patterns that cannot be said to belong to any of existing classes. Similarly to the original algorithms, the hyperbox fuzzy sets are used as a representation of clusters and classes. Learning is usually completed in a few passes and consists of placing and adjusting the hyperboxes in the pattern space; this is an expansion-contraction process. The classification results can be crisp or fuzzy. New data can be included without the need for retraining. While retaining all the interesting features of the original algorithms, a number of modifications to their definition have been made in order to accommodate fuzzy input patterns in the form of lower and upper bounds, combine the supervised and unsupervised learning, and improve the effectiveness of operations. A detailed account of the GFMM neural network, its comparison with the Simpson's fuzzy min-max neural networks, a set of examples, and an application to the leakage detection and identification in water distribution systems are given.
A Vision Based Recognition of Indian Sign Language Alphabets and Numerals Using B-Spline Approximation
- M Geetha
- U Manjusha
A Multilingual Multimedia Indian Sign Language dictionary Tool
- Tirthankar Dasgupta
- Sambit Shukla
- Sandeep Kumar
- Sunny Diwakar
- Anupam Basu