The signspeak project-bridging the gap between signers and speakers

TID, Granada, Spain
Journal of Speech Language and Hearing Research - J SPEECH LANG HEAR RES 01/2010;
Source: DBLP


The SignSpeak project will be the first step to approach sign language recognition and translation at a scientific level already reached in similar research fields such as automatic speech recognition or statistical machine translation of spoken languages. Deaf communities revolve around sign languages as they are their natural means of communication. Although deaf, hard of hearing and hearing signers can communicate without problems amongst themselves, there is a serious challenge for the deaf community in trying to integrate into educational, social and work environments. The overall goal of SignSpeak is to develop a new vision-based technology for recognizing and translating continuous sign language to text. New knowledge about the nature of sign language structure from the perspective of machine recognition of continuous sign language will allow a subsequent breakthrough in the development of a new vision-based technology for continuous sign language recognition and translation. Existing and new publicly available corpora will be used to evaluate the research progress throughout the whole project.

Download full-text


Available from: Onno Crasborn,
  • Source
    • "It's applicable for sign language recognition. [4] [5] [6]. One of easily and rapidly tracking algorithm is Bounding-Box. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we used a Replication Function (R. F.)for improvement tracking with dynamic programming. The R. F. transforms values of gray level [0 255] to [0 1]. The resulting images of R. F. are more striking and visible in skin regions. The R. F. improves Dynamic Programming (D. P.) in overlapping hand and face. Results show that Tracking Error Rate 11% and Average Tracked Distance 7% reduced
    06/2014; 7(3). DOI:10.14445/22315381/IJETT-V7P254
  • Source
    • "Recognizing signs of visual-gestural languages like ASL is a very active field [21] [3] [16] [18] [27] [23] [28]. For instance, the SignSpeak project [4] aims at developing vision-based technology for translating continuous sign language to text. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In this work, we present a system for recognizing letters and finger-spelled words of the American sign language (ASL) in real-time. To this end, the system segments the hand and estimates the hand orientation from captured depth data. The letter classification is based on average neighborhood margin maximization and relies on the segmented depth data of the hands. For word recognition, the letter confidences are aggregated. Furthermore, the word recognition is used to improve the letter recognition by updating the training examples of the letter classifiers on-line.
    IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, Spain, November 6-13, 2011; 11/2011
  • Source
    • "and Joshua ( About speech generation from sign language, in the Computer Science department of the RWTH, Aachen University, P. Dreuw supervised by H. Ney is making a significant effort in recognizing continuous sign language with a new vision-based technology ([9]; [10]). This paper describes the development of a Spanish Sign Language into Spoken Spanish translation system in a real domain: the Driver's License and Identity Document renewal. "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper describes the development of a Spoken Spanish generator from Spanish Sign Language (LSE – Lengua de Signos Española) in a specific domain: the renewal of Identity Document and Driver's license. The system is composed of three modules. The first one is an interface where a deaf person can specify a sign sequence in sign-writing. The second one is a language translator for converting the sign sequence into a word sequence. Finally, the last module is a text to speech converter. Also, the paper describes the generation of a parallel corpus for the system development composed of more than 4,000 Spanish sentences and their LSE translations in the application domain. The paper is focused on the translation module that uses a statistical strategy with a phrase-based translation model, and this paper analyses the effect of the alignment configuration used during the process of word-based translation model generation. Finally, the best configuration gives a 3.90% mWER and a 0.9645 BLEU.
Show more