Joint optimization of word alignment and epenthesis generation for Chinese to Taiwanese sign synthesis.

Home Network Technology Center, Industrial Technology Research Institute, N200, ITRI Bldg. R1, No. 31, Gongye 2nd Rd., Annan District, Tainan City 709, Taiwan, ROC.
IEEE Transactions on Pattern Analysis and Machine Intelligence (Impact Factor: 4.8). 02/2007; 29(1):28-39. DOI: 10.1109/TPAMI.2007.15
Source: PubMed

ABSTRACT This work proposes a novel approach to translate Chinese to Taiwanese sign language and to synthesize sign videos. An aligned bilingual corpus of Chinese and Taiwanese Sign Language (TSL) with linguistic and signing information is also presented for sign language translation. A two-pass alignment in syntax level and phrase level is developed to obtain the optimal alignment between Chinese sentences and Taiwanese sign sequences. For sign video synthesis, a scoring function is presented to develop motion transition-balanced sign videos with rich combinations of intersign transitions. Finally, the maximum a posteriori (MAP) algorithm is employed for sign video synthesis based on joint optimization of two-pass word alignment and intersign epenthesis generation. Several experiments are conducted in an educational environment to evaluate the performance on the comprehension of sign expression. The proposed approach outperforms the IBM Model 2 in sign language translation. Moreover, deaf students perceived sign videos generated by the proposed method to be satisfactory.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: One of the aims of Assistive Technologies is to help people with disabilities to communicate with others and to provide means of access to information. As an aid to Deaf people, we present in this work a production-quality rule-based machine system for translating from Spanish to Spanish Sign Language (LSE) glosses, which is a necessary precursor to building a full machine translation system that eventually produces animation output. The system implements a transfer-based architecture from the syntactic functions of dependency analyses. A sketch of LSE is also presented. Several topics regarding translation to sign languages are addressed: the lexical gap, the bootstrapping of a bilingual lexicon, the generation of word order for topic-oriented languages, and the treatment of classifier predicates and classifier names. The system has been evaluated with an open-domain testbed, reporting a 0.30 BLEU (BiLingual Evaluation Understudy) and 42% TER (Translation Error Rate). These results show consistent improvements over a statistical machine translation baseline, and some improvements over the same system preserving the word order in the source sentence. Finally, the linguistic analysis of errors has identified some differences due to a certain degree of structural variation in LSE.
    Computer Speech & Language 05/2014; 28(3):788-811. · 1.46 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The article describes technical and user-interface issues of transferring the contents and functionality of the CD-ROM version of the Slovenian sing language dictionary to the web. The dictionary of Slovenian sign language consist of video clips showing the demonstra- tion of signs that deaf people use for communication, text description of the words corresponding to the signs and pictures illustrating the same word/sign. A new technical solution—a video sprite—for concatenating subsections of video clips necessary for their smooth display on most available platforms was developed. The contents of the dictionary which were re-edited are combined also with other resources available on the web. Added were also new exercises for learning the sign language.
    First International Conference, SouthCHI 2013; 07/2013
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: A novel Italian Sign Language MultiWordNet (LMWN), which integrates the MultiWordNet (MWN) lexical database with the Italian Sign Language (LIS), is presented in this paper. The approach relies on LIS lexical resources which support and help to search for Italian lemmas in the database and display corresponding LIS signs. The lexical frequency analysis of the lexicon and some newly created signs approved by expert LIS signers are also discussed. The larger MWN database helps to enrich the variety and comprehensiveness of the lexicon. We also describe the approach which links the Italian lemmas and LIS signs to extract and display bilingual information from the collected lexicon and the semantic relationships of LIS Signs with MWN. The users can view the meanings of almost one fourth of the lemmas of MWN in LIS.
    Expert Systems with Applications 10/2013; · 1.85 Impact Factor

Full-text (2 Sources)

Available from
Jun 2, 2014