Evgeny Matusov

• Dr. rer. nat.
• Lead Science Architect Machine Translation at AppTek

This paper reports on the shared tasks organized by the 21st IWSLT Conference. The shared tasks address 7 scientific challenges in spoken language translation: simultaneous and offline translation, automatic subtitling and dubbing, speech-to-speech translation, dialect and low-resource speech translation, and Indic languages. The shared tasks attra...

To participate in the Isometric Spoken Language Translation Task of the IWSLT 2022 evaluation, constrained condition, AppTek developed neural Transformer-based systems for English-to-German with various mechanisms of length control, ranging from source-side and target-side pseudo-tokens to encoding of remaining length in characters that replaces po...

This paper addresses the problem of evaluating the quality of automatically generated subtitles, which includes not only the quality of the machine-transcribed or translated speech, but also the quality of line segmentation and subtitle timing. We propose SubER - a single novel metric based on edit distance with shifts that takes all of these subti...

In simultaneous machine translation, the objective is to determine when to produce a partial translation given a continuous stream of source words, with a trade-off between latency and quality. We propose a neural machine translation (NMT) model that makes dynamic decisions when to continue feeding on input or generate output words. The model is co...

In this work, we explore the usefulness of target factors in neural machine translation (NMT) beyond their original purpose of predicting word lemmas and their inflections, as proposed by García-Martínez et al., 2016. For this, we introduce three novel applications of the factored output architecture: In the first one, we use a factor to explicitly...

In this work, we customized a neural machinetranslation system for translation of subtitles in the domain of entertainment. The neural translation model was adapted to the subtitling content and style and extended by a simple, yet effective technique for utilizing inter-sentence context for short sentences such as dialog turns. The main contributio...

We empirically investigate learning from partial feedback in neural machine translation (NMT), when partial feedback is collected by asking users to highlight a correct chunk of a translation. We propose a simple and effective way of utilizing such feedback in NMT training. We demonstrate how the common machine translation problem of domain mismatc...

We present the first real-world application of methods for improving neural machine translation (NMT) with human reinforcement, based on explicit and implicit user feedback collected on the eBay e-commerce platform. Previous work has been confined to simulation experiments, whereas in this paper we work with real logged feedback for offline bandit...

In this paper, we introduce a hybrid search for attention-based neural machine translation (NMT). A target phrase learned with statistical MT models extends a hypothesis in the NMT beam search when the attention of the NMT model focuses on the source words translated by this phrase. Phrases added in this way are scored with the NMT model, but also...

In this paper, we discuss different methods which use meta information and richer context that may accompany source language input to improve machine translation quality. We focus on category information of input text as meta information, but the proposed methods can be extended to all textual and non-textual meta information that might be availabl...

In this paper, we introduce a hybrid search for attention-based neural machine translation (NMT). A target phrase learned with statistical MT models extends a hypothesis in the NMT beam search when the attention of the NMT model focuses on the source words translated by this phrase. Phrases added in this way are scored with the NMT model, but also...

In this paper we study the impact of using images to machine-translate user-generated e-commerce product listings. We study how a multi-modal Neural Machine Translation (NMT) model compares to two text-only approaches: a conventional state-of-the-art attentional NMT and a Statistical Machine Translation (SMT) model. User-generated product listings...

In this paper, we study how humans perceive the use of images as an additional knowledge source to machine-translate user-generated product listings in an e-commerce company. We conduct a human evaluation where we assess how a multi-modal neural machine translation (NMT) model compares to two text-only approaches: a conventional state-of-the-art at...

In this paper, we propose an effective way for biasing the attention mechanism of a sequence-to-sequence neural machine translation (NMT) model towards the well-studied statistical word alignment models. We show that our novel guided alignment training approach improves translation quality on real-life e-commerce texts consisting of product titles...

An important challenge to statistical machine translation (SMT) is the lack of parallel data for many language pairs. One common solution is to pivot through a third language for which there exist parallel corpora with the source and target languages. Although pivoting is a robust technique, it introduces some low quality translations. In this pape...

Abstract In this paper, we propose a selective combination approach of pivot and direct statistical machine translation (SMT) models to improve translation quality. We work with Persian-Arabic SMT as a case study. We show positive results (from 0.4 to 3.1 BLEU on different direct training corpus sizes) in addition to a large reduction of pivot t...

The usual approach to improve the interface between automatic speech recognition (ASR) and machine translation (MT) is to use ASR word lattices for translation. In comparison with the previous research along this line, this paper presents an efficient algorithm for lattice-based search in MT. This algorithm utilizes confusion network information to...

In this paper, we propose a novel model for scoring reordering in phrase-based statisti-cal machine translation (SMT) and success-fully use it for translation from Farsi into En-glish and Arabic. The model replaces the distance-based distortion model that is widely used in most SMT systems. The main idea of the model is to penalize each new de-viat...

RWTH participated in the System Combi- nation task of the Fourth Workshop on Sta- tistical Machine Translation (WMT 2009). Hypotheses from 9 German!English MT systems were combined into a consen- sus translation. This consensus transla- tion scored 2.1% better in BLEU and 2.3% better in TER (abs.) than the best sin- gle system. In addition, cross-l...

RWTH participated in the shared transla- tion task of the Fourth Workshop of Sta- tistical Machine Translation (WMT 2009) with the German-English, French-English and Spanish-English pair in each transla- tion direction. The submissions were gen- erated using a phrase-based and a hierar- chical statistical machine translation sys- tems with appropri...

In this paper, we deal with the problem of a large number of unaligned words in auto- matically learned word alignments for ma- chine translation (MT). These unaligned words are the reason for ambiguous phrase pairs extracted by a statistical phrase-based MT system. In translation, this phrase am- biguity causes deletion and insertion er- rors. We...

This chapter shows how outputs of multiple machine translation (MT) systems may be combined in order to improve translation quality. It presents a method that computes a consensus translation from the aligned MT hypotheses. The method utilizes the enhanced alignment procedure of Matusov et al. (2006), as well as its extensions. The proposed system...

The Internet gives us access to a wealth of information in languages we don't understand. The investigation of automated or semi-automated approaches to translation has become a thriving research field with enormous commercial potential. This volume investigates how Machine Learning techniques can improve Statistical Machine Translation, currently...

This paper describes an approach for computing a consensus translation from the outputs of multiple machine translation (MT) systems. The consensus translation is computed by weighted majority voting on a confusion network, similarly to the well-established ROVER approach of Fiscus for combining speech recognition hypotheses. To create the confusio...

Progress in both speech and language processing has spurred efforts to support applications that rely on spoken rather than written language input. A key challenge in moving from text-based documents to such spoken documents is that spoken language lacks explicit punctuation and formatting, which can be crucial for good performance. This article de...

We present the RWTH phrase-based statistical machine translation system designed for the translation of Arabic speech into English text. This system was used in the Global Autonomous Language Exploitation (GALE) Go/No-Go Translation Evaluation 2007. Using a two-pass approach, we first generate n-best translation candidates and then rerank these can...

Confusion networks are a simple representa- tion of multiple speech recognition or transla- tion hypotheses in a machine translation sys- tem. A typical operation on a confusion net- work is to find the path which minimizes or maximizes a certain evaluation metric. In this article, we show that this problem is gener- ally NP-hard for the popular BL...

Recent papers have described machine translation (MT) based on an automatic post-editing or serial combination strategy whereby the input language is first translated into the target language by a rule-based MT (RBMT) system, then the target language output is automatically post-edited by a phrase-based statistical machine translation (SMT) system....

RWTH's system for the 2008 IWSLT evaluation consists of a combination of different phrase-based and hierarchical statistical machine translation systems. We participated in the translation tasks for the Chinese-to-English and Arabic-to-English language pairs. We investigated different prepro-cessing techniques, reordering methods for the phrase-bas...

This paper shows how ASR word lattices can be translated even when exhaustive reordering is required for good transla-tion quality. We propose a method for labeling lattice word hypotheses with position information derived from a confusion network (CN). This information is effectively used in the sta-tistical phrase-based machine translation (MT) s...

Progress in both speech and language processing has spurred efforts to support applications that rely on spoken—rather than written—language input. A key challenge in moving from text-based documents to such “spoken documents ” is that spoken language lacks explicit punctuation and formatting, which can be crucial for good performance. This paper d...

This paper investigates the influence of automatic sentence boundary and sub-sentence punctuation prediction on machine translation (MT) of automatically recognized speech. We use prosodic and lexical cues to determine sentence boundaries, and successfully combine two complementary approaches to sen- tence boundary prediction. We also introduce a n...

We describe experiments with Czech-to-English phrase-based machine translation. Several techniques for improving translation quality (in terms of well-established measure BLEU) are evaluated. In total, we are able to achieve BLEU of 0.36 to 0.41 on the examined corpus of Wall Street Journal texts, outperforming all other systems evaluated on this l...

This paper describes state-of-the-art interfaces between speech recognition and machine translation. We modify two different machine translation systems to effectively process dense speech recognition lattices. In addition, we describe how to fully integrate speech translation with machine translation based on weighted finite-state transducers. Wit...

This paper describes a novel method for computing a consensus translation from the outputs of multiple machine trans-lation (MT) systems. The outputs are combined and a possibly new transla-tion hypothesis can be generated. Simi-larly to the well-established ROVER ap-proach of (Fiscus, 1997) for combining speech recognition hypotheses, the con-sens...

The IBM Models (Brown et al., 1993) enjoy great popularity in the machine translation community because they offer high quality word alignments and a free implementation is available with the GIZA++ Toolkit (Och and Ney, 2003). Several methods have been developed to overcome the asymmetry of the alignment generated by the IBM Models. A remaining di...

This paper focuses on the interface between speech recognition and machine translation in a speech translation system. Based on a thorough theoretical framework, we exploit word lattices of automatic speech recognition hypotheses as input to our trans- lation system which is based on weighted finite-state transduc- ers. We show that acoustic recogn...

This paper presents novel approaches to reordering in phrase-based statistical ma-chine translation. We perform consistent reordering of source sentences in train-ing and estimate a statistical translation model. Using this model, we follow a phrase-based monotonic machine transla-tion approach, for which we develop an ef-ficient and flexible reord...

A Chinese sentence is represented as a sequence of charac-ters, and words are not separated from each other. In statisti-cal machine translation, the conventional approach is to seg-ment the Chinese character sequence into words during the pre-processing. The training and translation are performed afterwards. However, this method is not optimal for...

This paper presents a phrase-based speech translation system that combines phrasal lexicon, language, and acoustic model features in a loglinear model. Automatic speech recognition and machine translation are coupled by using large word lattices as the input for translation. For the first time, all features are directly integrated into the decoding...

Word-aligned bilingual corpora are an important knowledge source for many tasks in natural language processing. We improve the well-known IBM alignment models, as well as the Hidden-Markov alignment model using a symmetric lex-icon model. This symmetrization takes not only the standard translation direc-tion from source to target into account, but...

In this paper, we address the word alignment problem for statistical machine translation. We aim at creating a sym-metric word alignment allowing for reli-able one-to-many and many-to-one word relationships. We perform the iterative alignment training in the source-to-target and the target-to-source direction with the well-known IBM and HMM alignme...

This paper deals with the task of statistical machine transla- tion of spontaneous speech using a limited amount of training data. We propose a method for selecting relevant additional training data from other sources that may come from other domains. We present two ways to solve the data sparse- ness problem by including morphological information...

Topic segmentation, i.e. the combined task of document segmentation and topic identification, is an interesting issue both from a theoretical point of view as well as for practical applications. Previous studies have mainly focussed on applications exposing rather weak correlations regarding the topic order (e.g. broadcast news). In this work, we c...

This paper presents a novel automatic sentence segmentation method for evaluating machine translation output with pos-sibly erroneous sentence boundaries. The algorithm can pro-cess translation hypotheses with segment boundaries which do not correspond to the reference segment boundaries, or a completely unsegmented text stream. Thus, the method is...

We give an overview of the RWTH phrase-based statistical machine translation system that was used in the evaluation campaign of the International Workshop on Spoken Lan-guage Translation 2005. We use a two pass approach. In the first pass, we gen-erate a list of the N best translation candidates. The second pass consists of rescoring and reranking...

In this paper, we describe AppTek's new APT machine trans-lation system that we employed in the IWSLT 2010 evalua-tion campaign. This year, we participated in the Arabic-to-English and Turkish-to-English BTEC tasks. We discuss the architecture of the system, the preprocessing steps and the experiments carried out during the campaign. We show that c...

This paper describes how word alignment information makes machine translation more effi-cient. Following a statistical approach based on finite-state transducers, we perform reordering of source sentences in training using automatic word alignments and estimate a phrase-based translation model. Using this model, we translate monotonically taking a...

In this paper we present the ongoing work at RWTH Aachen University for building a speech-to-speech translation system within the TC-Star project. The corpus we work on consists of parliamentary speeches held in the European Plenary Sessions. To our knowledge, this is the first project that focuses on speech-to-speech translation applied to a real-...

In this paper, we describe the RWTH statistical machine translation (SMT) system which is based on log-linear model combination. All knowledge sources are treated as feature functions which depend on the source language sentence, the target language sentence and possible hidden variables. The main feature of our approach are the alignment templates...

We give an overview of the RWTH phrase-based statistical machine translation system that was used in the evaluation campaign of the International Workshop on Spoken Lan- guage Translation (IWSLT) 2006. The system was ranked first with respect to the BLEU measure in all language pairs it was used Using a two-pass aproach, we first generate the N bes...

In recent years, there has been dramatic progress in both speech and language processing, which spurs efforts to combine speech and language technologies in spoken document processing applications. However, speech is different from text in many respects, most notably in the lack of explicit punctuation and formatting. Thus, spoken document processi...

In this paper, we present SAIC's hybrid ma-chine translation (MT) system and show how it was adapted to the needs of our customer – a major global fashion company. The adaptation was performed in two ways: off-line selec-tion of domain-relevant parallel and monolin-gual data from a background database, as well as on-line incremental adaptation with...

Machine translation of spoken language is a challenging task that involves several natural language processing (NLP) software modules. Human speech in one natural language has to be first automatically transcribed by a speech recognition system. Next, the transcription of the spoken utterance can be translated into another natural language by a mac...

With the recent remarkable success of statistical machine translation (SMT) systems, the question arises: how can the linguistic knowledge locked up in older, rule-based machine translation (RBMT) systems most conveniently be incorporated in these systems? In many cases, RBMT systems represent an investment of several man-years of applied expertise...