-
IEEE Transactions on Systems, Man, and Cybernetics, Part A. 01/2007; 37:1-9.
-
[show abstract]
[hide abstract]
ABSTRACT: This paper presents a Chinese sign language/spoken language dialog system based on the technique of large vocabulary continuous Chinese sign language recognition (SLR) and Chinese sign language synthesis (SLS), which is new development for HandTalker. In the SLR module, a fuzzy decision tree with heterogeneous classifiers is presented for large vocabulary signer-independent SLR, and then large vocabulary continuous SLR based on transition movement models is proposed. In SLS module, three key techniques: realistic 3D facial animation and gesture retargeting technique and synchronization modal on gesture and lip motion are employed to improve sign language synthesis vividness.
Control, Automation, Robotics and Vision Conference, 2004. ICARCV 2004 8th; 01/2005
-
[show abstract]
[hide abstract]
ABSTRACT: In sign language recognition, using subwords instead of whole signs as basic units scales well with increasing vocabulary size. However, there are no subwords defined in the signs' lexical forms. How to automatically extract subwords is a challenging issue. In this paper, a novel approach is proposed to automatically extract these subwords from Chinese sign language (CSL). Signs can be broken down into several segments using hidden Markov models in which each state represents one segment. Temporal clustering algorithm is presented to extract subwords from these segments. The 238 subwords are automatically extracted from 5113 signs, and they can be used as the basic units for large vocabulary CSL recognition with good performance.
Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on; 09/2004
-
[show abstract]
[hide abstract]
ABSTRACT: The major challenges that sign language recognition (SLR) now faces are developing methods that solve large vocabulary continuous sign problems. In this paper, large vocabulary continuous SLR based on transition movement models is proposed. The proposed method employs the temporal clustering algorithm to cluster a large amount of transition movements, and then the corresponding training algorithm is also presented for automatically segmenting and training these transition movement models. The clustered models can improve the generalization of transition movement models, and are very suitable for large vocabulary continuous SLR. At last, the estimated transition movement models, together with sign models, are viewed as candidate models of the Viterbi search algorithm for recognizing continuous sign language. Experiments show that continuous SLR based on transition movement models has good performance over a large vocabulary of 5113 signs.
Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on; 06/2004
-
[show abstract]
[hide abstract]
ABSTRACT: The major difficulty for large vocabulary sign recognition lies in the huge search space due to a variety of recognized classes. How to reduce the recognition time without loss of accuracy is a challenging issue. In this paper, a fuzzy decision tree with heterogeneous classifiers is proposed for large vocabulary sign language recognition. As each sign feature has the different discrimination to gestures, the corresponding classifiers are presented for the hierarchical decision to sign language attributes. A one- or two- handed classifier and a hand-shaped classifier with little computational cost are first used to progressively eliminate many impossible candidates, and then, a self-organizing feature maps/hidden Markov model (SOFM/HMM) classifier in which SOFM being as an implicit different signers' feature extractor for continuous HMM, is proposed as a special component of a fuzzy decision tree to get the final results at the last nonleaf nodes that only include a few candidates. Experimental results on a large vocabulary of 5113-signs show that the proposed method dramatically reduces the recognition time by 11 times and also improves the recognition rate about 0.95% over single SOFM/HMM.
IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans 06/2004; · 2.12 Impact Factor
-
Advances in Multimedia Information Processing - PCM 2004, 5th Pacific Rim Conference on Multimedia, Tokyo, Japan, November 30 - December 3, 2004, Proceedings, Part II; 01/2004
-
Proceedings of the 6th International Conference on Multimodal Interfaces, ICMI 2004, State College, PA, USA, October 13-15, 2004; 01/2004
-
[show abstract]
[hide abstract]
ABSTRACT: We present a Chinese sign language dialog system (CSLDS) based on the technique of large vocabulary continuous Chinese sign language recognition (CSLR) and Chinese sign language synthesis (CSLS). This system can show the advance technology on gesture recognition and synthesis well and can apply to more powerful system combined with speech recognition and synthesis technology, which then can allow the convenient communication between deaf and hearing society.
Analysis and Modeling of Faces and Gestures, 2003. AMFG 2003. IEEE International Workshop on; 11/2003
-
Proceedings of the 5th International Conference on Multimodal Interfaces, ICMI 2003, Vancouver, British Columbia, Canada, November 5-7, 2003; 01/2003
-
J. Comput. Sci. Technol. 01/2003; 18:131-138.
-
[show abstract]
[hide abstract]
ABSTRACT: An improved maximum entropy language model (IMELM) is presented based on three respects of language modeling (LM) improvement: the solution of long dependences, the integration of language knowledge into LM, and the general framework that combines all kinds of language knowledge. The proposed model combines trigram with base phrase structure knowledge in this paper. Trigram is used to capture the local relation between words, while base phrase structure knowledge is considered to represent the long-distance relations between syntactical structures. The knowledge of syntax, semantics and word is integrated in the maximum entropy framework. The experimental results show that the proposed model has a 24% improvement in perplexity over the conventional trigram model.
Signal Processing, 2002 6th International Conference on; 09/2002
-
[show abstract]
[hide abstract]
ABSTRACT: Sign language recognition is to provide an efficient and accurate mechanism to transcribe sign language into text or speech. State-of-the-art sign language recognition should be able to solve the signer-independent continuous problem for practical applications. A divide-and-conquer approach, which takes the problem of continuous Chinese Sign Language (CSL) recognition as subproblems of isolated CSL recognition, is presented for signer-independent continuous CSL recognition. In the proposed approach, the improved simple recurrent network (SRN) is used to segment the continuous CSL. The outputs of SRN are regarded as the states of hidden Markov models (HMM) in which the Lattice Viterbi algorithm is employed for searching for the best word sequence. Experimental results show that the SRN/HMM approach has a better performance than the standard HMM.
Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on; 06/2002
-
[show abstract]
[hide abstract]
ABSTRACT: The aim of sign language recognition is to provide an efficient and accurate mechanism to transcribe sign language into text or speech. State-of-the-art sign language recognition should be able to solve the signer-independent problem for practical application. In this paper, a hybrid SOFM/HMM system, which combines self-organizing feature maps (SOFMs) with hidden Markov models (HMMs), is presented for signer-independent Chinese sign language recognition. We implement the SOFM/HMM sign recognition system. Meanwhile, results from the HMM-based system are provided as comparison. Experimental results show the SOFM/HMM system increases the recognition accuracy by 5% than the HMM-based one. Furthermore, a self-adjusting recognition algorithm is also proposed for improving the SOFM/HMM discrimination. When it is applied to the SOFM/HMM system it can improve the recognition accuracy by 1.9%. All experiments were performed in real-time with the dictionary size 208
Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 2001. Proceedings. IEEE ICCV Workshop on; 02/2001
-
Gesture and Sign Languages in Human-Computer Interaction, International Gesture Workshop, GW 2001, London, UK, April 18-20, 2001, Revised Papers; 01/2001
-
[show abstract]
[hide abstract]
ABSTRACT: In sign language recognition (SLR), the major challenges now are developing methods that solve signer-independent continuous sign problems. In this paper, SOFM/HMM is first presented for modeling signer-independent isolated signs. The proposed method uses the self-organizing feature maps (SOFM) as different signers' feature extractor for continuous hidden Markov models (HMM) so as to transform input signs into significant and low-dimensional representations that can be well modeled by the emission probabilities of HMM. Based on these isolated sign models, a SOFM/SRN/HMM model is then proposed for signer-independent continuous SLR. This model applies the improved simple recurrent network (SRN) to segment continuous sign language in terms of transformed SOFM representations, and the outputs of SRN are taken as the HMM states in which the lattice Viterbi algorithm is employed to search the best matched word sequence. Experimental results demonstrate that the proposed system has better performance compared with conventional HMM system and obtains a word recognition rate of 82.9% over a 5113-sign vocabulary and an accuracy of 86.3% for signer-independent continuous SLR.
Pattern Recognition.
