No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Post-Editing Machine Translated Text in a Commercial Setting: Observation and Statistical Analysis
Abstract
Machine translation systems, when they are used in a commercial context for publishing purposes, are usually used in combination with human post-editing. Thus understanding human post-editing behaviour is crucial in order to maximise the benefit of machine translation systems. Though there have been a number of studies carried out on human post-editing to date, there is a lack of large-scale studies on post-editing in industrial contexts which focus on the activity in real-life settings. This study observes professional Japanese post-editors’ work and examines the effect of the amount of editing made during post-editing, source text characteristics, and post-editing behaviour, on the amount of post-editing effort. A mixed method approach was employed to both quantitatively and qualitatively analyse the data and gain detailed insights into the post-editing activity from various view points. The results indicate that a number of factors, such as sentence structure, document component types, use of product specific terms, and post-editing patterns and behaviour, have effect on the amount of post-editing effort in an intertwined manner. The findings will contribute to a better utilisation of machine translation systems in the industry as well as the development of the skills and strategies of post-editors.
... In contrast to Tatsumi's (2010Tatsumi's ( , 2009) studies, the present study measures the average PE speed by both source and target sentence length because post-editing requires the post-editors to focus on both source text and target outputs. The results in Table 8 shows the average PE speed in the PEMT tasks by sentence length. ...
... The results indicate that the non-native trainee translators performed the fastest when translating long sentences, with an average of 18 words per minute in both translation directions. Not only does this contradict Hypothesis 4 but also the findings of the previous studies (Koponen, 2016;Tatsumi, 2009Tatsumi, , 2010Tatsumi and Roturier, 2010) because the analysis of the present research data also revealed similar results when measuring the average PE speed by the source sentence length. The results in Table 10 indicate that the average PE speed for the context matches (100-101%) is the slowest with an average of 11 and 6 words per minute respectively. ...
... In contrast to Guerberof's (2012) findings, the present study revealed that the average PE speed for the MT matches is slower than the 85-94% TM matches. In the PETM+MT tasks, Despite not being able to reach the average daily productivity for full human translation (at least 2,000 words per day), the non-native trainee translators managed to reach the average daily productivity for post-editing, which is at least 5,000 words (Koponen, 2016;Tatsumi, 2009Tatsumi, , 2010Tatsumi and Roturier, 2010), which revealed that very long sentences slowed down their post-editors' speed. Regardless, the results of the present study also support the findings of the previous studies, suggesting that short sentences slow down the PE speed. ...
Translation into and between foreign languages has become a common practice in the professional setting. However, this translation directionality has yet to be thoroughly explored, especially when post-editing is involved. The present study conducts experiments on the application of machine translation (MT) and translation memory (TM) in a translation classroom setting. A group of Malay speakers, who are non- native speakers of Arabic and English, used MemoQ 2014 to translate technical Arabic and English texts by post-editing raw MT and modified TM outputs containing several errors. The non-native trainee translators’ productivity was measured and the quality of the translation was assessed through error analysis approach based on the MeLLANGE error typology so that it could provide a comprehensive analysis of the types of errors commonly found in the non-native trainee translators’ translations. The error annotation also aims to provide guidelines for translators who work with the Arabic-English language pair and non-native translators.
The present study revealed that the translation technologies helped improve the non- native translators’ speed and quality. The study also discovered that syntactic and lexical errors are the most problematic in the PE tasks. The trainee translators tend to overlook the errors that were caused by cross-linguistic influence, such as articles, gender, number and the conjunction “wa”. However, this could have been avoided if the participants revised their translations thoroughly because most of the errors are minor. The study also revealed that the non-native trainee translators could be as productive as the professional native translators because they managed to reach the average daily productivity for professional translators, which is at least 5,000 words per day.
... Due to these positive results, interest in PE is strong in both research and the translation industry Tatsumi [2010]; Koponen [2016]. ...
... The authors present results averaged per translation condition, by-subject and -item. Tatsumi [2010] presents a long study analyzing PE effort in English-to-Japanese translation. To this end, amongst other things, a number of multiple linear regression models are presented, including various source-side characteristics, number of edit operations, and translators as őxed effects. ...
... Our review shows that a wide range of experimental designs were used in CAT research, which are mostly restricted to either by-item or by-subjects analyses with the notable exceptions of [Tatsumi, 2010], [Scheepers and Schulz, 2016], [Green et al., 2013a], [Green et al., 2014b] and [Läubli et al., 2013], the latter three works establishing linear mixed-effects models for analysis of CAT user studies. These statistical models enable to account for by-item and by-subject variance [Clark, 1973;Forster and Dickinson, 1976]. ...
Automatic translation of natural language is still (as of 2017) a long-standing but unmet promise. While advancing at a fast rate, the underlying methods are still far from actually being able to reliably capture syntax or semantics of arbitrary utterances of natural language, way off transporting the encoded meaning into a second language. However, it is possible to build useful translating machines when the target domain is well known and the machine is able to learn and adapt efficiently and promptly from new inputs. This is possible thanks to efficient and effective machine learning methods which can be applied to automatic translation. In this work we present and evaluate methods for three distinct scenarios: a) We develop algorithms that can learn from very large amounts of data by exploiting pairwise preferences defined over competing translations, which can be used to make a machine translation system robust to arbitrary texts from varied sources, but also enable it to learn effectively to adapt to new domains of data; b) We describe a method that is able to efficiently learn external models which adhere to fine-grained preferences that are extracted from a constricted selection of translated material, e.g. for adapting to users or groups of users in a computer-aided translation scenario; c) We develop methods for two machine translation paradigms, neural- and traditional statistical machine translation, to directly adapt to user-defined preferences in an interactive post-editing scenario, learning precisely adapted machine translation systems. In all of these settings, we show that machine translation can be made significantly more useful by careful optimization via preference learning.
... Rather than making use of evaluation scores, some previous studies have To the present author's knowledge, further information on potential connections between MT quality and PE effort can only be found in studies that do not explore MT quality itself, but rather the amount of editing implemented. This is the case of the study carried out by Tatsumi (2010), for example, who found that scores reflecting edit distance are significantly correlated with PE time. However, since Tatsumi instructed post-editors to carry out light PE, not asking them to render the post-edited text 'stylistically sophisticated' (ibid., 82), it is debatable if these scores can reflect the quality of the MT output. ...
... One of the research questions addressed by Tatsumi (2010) concerned the impact of different PE operations on PE speed. To identify these operations, she conducted a qualitative analysis of the PE process and proposed a typology of the different types of edits it involves. ...
... Similarly to Krings's (2001), Tatsumi's (2010) typology reflects how different PE operations are performed. In Tatsumi's case, however, only edits actually implemented are taken into account. ...
This thesis investigates the expenditure of cognitive effort in post-editing of machine translation. A mixed-method approach involving the use of eye movements, subjective ratings and think-aloud protocols was adopted for the investigation. The project aims at revealing connections between cognitive effort and variables including linguistic characteristics of the source text and the machine-translation output, post-editors’ individual traits, different linguistic aspects of the activity attended to during the task, and the quality of the post-edited texts, assessed by human translators in terms of fluency (linguistic quality) and adequacy (faithfulness to the source text). Two tasks were conducted to pursue these aims: one involving eye tracking and a self-report scale of cognitive effort, and another carried out by a different, but comparable, sample of participants, under a think-aloud condition. Results indicate that variables such as an automatic machine-translation quality score and source-text type-token ratio are good predictors of cognitive effort in post-editing. The relationship between cognitive effort and post-editors’ traits was found to be a complex one, with significant links in this respect only appearing in the context of interactions between variables. A complex relationship was also found between editing behaviour and the quality of the post-edited text: the number of changes implemented was found to have a generally positive association with post-edited fluency, though cognitive effort was found to be negatively correlated with both the fluency and adequacy of the post-edited texts. Mental processes involving grammar and lexis were significantly related to the levels of cognitive effort expended by participants. These were also the aspects most frequently attended to in the activity. From a methodological perspective, despite the criticisms received by the think-aloud method in previous research, empirical data obtained in this project indicates that think-aloud protocols correlate with eye movements and subjective ratings as measures of cognitive effort.
http://hdl.handle.net/10443/3130
... Even though Koehn (2009) found post-editing to be as productive as other methods of translation, participants did not enjoy it and did not have the idea it was useful. Comparable to translators' desire to learn more about MT found in Fulford (2002), Tatsumi (2010) found that participants wanted to learn how MT worked and what the expected level of final quality was. There seems to be a trend of a more positive and flexible attitude towards MT and post-editing (de Almeida, 2013;Garcia, 2010;Lee & Liao, 2011;Tatsumi, 2010), with Garcia's and Lee & Liao's participants even thinking they would perform better when post-editing compared to when translating from scratch. ...
... Comparable to translators' desire to learn more about MT found in Fulford (2002), Tatsumi (2010) found that participants wanted to learn how MT worked and what the expected level of final quality was. There seems to be a trend of a more positive and flexible attitude towards MT and post-editing (de Almeida, 2013;Garcia, 2010;Lee & Liao, 2011;Tatsumi, 2010), with Garcia's and Lee & Liao's participants even thinking they would perform better when post-editing compared to when translating from scratch. Especially with customised MT systems, participants seem to feel that post-editing was faster, and that the MT output was useful (Green, Heer, & Manning, 2013). ...
... O'Brien (2002), for example, listed four reasons why post-editing should be taught to translators: (1) to meet the increasing demand for translation; (2) because post-editing skills are different from translation skills; (3) to make future translators more comfortable with post-editing and thus more tolerant; and (4) to improve the uptake of machine translation technology. Both Fulford (2002) and Tatsumi (2010) noted that translators want to learn more about MT and its limitations, which is an additional reason to add post-editing and MT to translator training. ...
To keep up with the growing need for translation in today's globalised society, post-editing of machine translation is increasingly being used as an alternative to regular human translation. While presumably faster than human translation, it is still unsure whether the quality of a post-edited text is comparable to the quality of a human translation, especially for general text types. In addition, there is a lack of understanding of the post-editing process, the effort involved, and the attitude of translators towards it. This dissertation contains a comparative analysis of post-editing and human translation by students and professional translators for general text types from English into Dutch. We study process, product, and translators' attitude in detail. We first conducted two pretests with student translators to try possible experimental setups and to develop a translation quality assessment approach suitable for a fine-grained comparative analysis of machine-translated texts, post-edited texts, and human translations. For the main experiment, we examined students and professional translators, using a combination of keystroke logging tools, eye tracking, and surveys. We used both qualitative analyses and advanced statistical analyses (mixed effects models), allowing for a multifaceted analysis. For the process analysis, we looked at translation speed, cognitive processing by means of eye fixations, the usage of external resources and its impact on overall time. For the product analysis, we looked at overall quality, frequent error types, and the impact of using external resources on quality. The attitude analysis contained questions about perceived usefulness, perceived speed, perceived quality of machine translation and post-editing, and the translation method that was perceived as least tiring. One survey was conducted before the experiment, the other after, so we could detect changes in attitude after participation. In two more detailed analyses, we studied the impact of machine translation quality on various types of post-editing effort indicators, and on the post-editing of multi-word units. We found that post-editing is faster than human translation, and that both translation methods lead to products of comparable overall quality. The more detailed error analysis showed that post-editing leads to somewhat better results regarding adequacy, and human translation leads to better results regarding acceptability. The most common errors for both translation methods are meaning shifts, logical problems, and wrong collocations. Fixation data indicated that post-editing was cognitively less demanding than human translation, and that more attention was devoted to the target text than to the source text. We found that fewer resources are consulted during post-editing than during human translation, although the overall time spent in external resources was comparable. The most frequently used external resources were Google Search, concordancers, and dictionaries. Spending more time in external resources, however, did not lead to an increase in quality. Translators indicated that they found machine translation useful, but they preferred human translation and found it more rewarding. Perceptions about speed and quality were mixed. Most participants believed post-editing to be at least as fast and as good as human translation, but barely ever better. We further discovered that different types of post-editing effort indicators were impacted by different types of machine translation errors, with coherence issues, meaning shifts, and grammatical and structural issues having the greatest effect. HTER, though commonly used, does not correlate well with more process-oriented post-editing effort indicators. Regarding the post-editing of multi-word units, we suggest 'contrast with the target language' as a useful new way of classifying multi-word units, as contrastive multi-word units were much harder to post-edit. In addition, we noticed that research strategies for post-editing multi-word units lack efficiency. Consulting external resources did lead to an increased quality of post-edited multi-word units, but a lot of time was spent in external resources when this was not necessary. Interestingly, the differences between human translation and post-editing usually outweighed the differences between students and professionals. Students did cognitively process texts differently, having longer fixation durations on the source text during human translation, and more fixations on the target text during post-editing, whereas professional translators' fixation behaviour remained constant. For the usage of external resources, only the time spent in dictionaries was higher for students than for professional translators, the usage of other resources was comparable. Overall quality was comparable for students and professionals, but professionals made fewer adequacy errors. Deletions were more noticeable for students than for professional translators in both methods of translation, and word sense issues were more noticeable for professional translators than for students when translating from scratch. Surprisingly, professional translators were often more positive about post-editing than students, believing they could produce products of comparable quality with both methods of translation. Students in particular struggled with the cognitive processing of meaning shifts, and they spent more time in pauses than professional translators. Some of the key contributions of this dissertation to the field of translation studies are the fact that we compared students and professional translators, developed a fine-grained translation quality assessment approach, and used a combination of state-of-the-art logging tools and advanced statistical methods. The effects of experience in our study were limited, and we suggest looking at specialisation and translator confidence in future work. Our guidelines for translation quality assessment can be found in the appendix, and contain practical instructions for use with brat, an open-source annotation tool. The experiment described in this dissertation is also the first to integrate Inputlog and CASMACAT, making it possible to include information on external resources in the CASMACAT logging files, which can be added to the CRITT Translation Process Research Database. Moving beyond the methodological contributions, our findings can be integrated in translation teaching, machine translation system development, and translation tool development. Translators need hands-on post-editing experience to get acquainted with common machine translation errors, and students in particular need to be taught successful strategies to spot and solve adequacy issues. Post-editors would greatly benefit from machine translation systems that made fewer coherence errors, meaning shift errors, and grammatical and structural errors. If visual clues are included in a translation tool (e.g., potentially problematic passages or polysemous words), these should be added to the target text. Tools could further benefit from integration with commonly used external resources, such as dictionaries. In the future, we wish to study the translation and post-editing process in even more detail, taking pause behaviour and regressions into account, as well as look at the passages participants perceived as the most difficult to translate and post-edit. We further wish to gain an even better understanding of the usage of external resources, by looking at the types of queries and by linking queries back to source and target text words. While our findings are limited to the post-editing and human translation of general text types from English into Dutch, we believe our methodology can be applied to different settings, with different language pairs. It is only by studying both processes in many different situations and by comparing findings that we will be able to develop tools and create courses that better suit translators' needs. This, in turn, will make for better, and happier, future generations of translators.
... Tatsumi (2009) makes one of the first attempts at statistically predicting PE time and shows that, in addition to automatic evaluation metrics (AEMs) used as an index of amount of edits, sentence-complexity features were necessary to increase the fit of regression models used in the analysis. Tatsumi (2010) also shows that a ST complexity score provided by the MT system SYSTRAN correlated well with PE time. In regard to findings in Tatsumi (2010), Green et al. (2013) point out that the statistical approach used fail to more solidly inform the matter of effort prediction in PE because the regression technique implemented does not account for variation between participants in the study and the wider population. ...
... Tatsumi (2010) also shows that a ST complexity score provided by the MT system SYSTRAN correlated well with PE time. In regard to findings in Tatsumi (2010), Green et al. (2013) point out that the statistical approach used fail to more solidly inform the matter of effort prediction in PE because the regression technique implemented does not account for variation between participants in the study and the wider population. In accounting for such variation, mixed-effects models have been considerably favoured in the literature (Balling 2008;Baayen et al. 2008;Green et al. 2013). ...
... ST, raw MT and emerging TT. Based on automatic MT evaluation, O'Brien (2011) shows a linear relationship between eye-tracking measures and AEMs previously tested in Tatsumi (2010), but final quality and ST-complexity variables are not measuredaspects that are addressed in the present study. ...
Identifying indices of effort in post-editing of machine translation can have a number of applications, including estimating machine translation quality and calculating post-editors’ pay rates. Both source-text and machine-output features as well as subjects’ traits are investigated here in view of their impact on cognitive effort, which is measured with eye tracking and a subjective scale borrowed from the field of Educational Psychology. Data is analysed with mixed-effects models, and results indicate that the semantics-based automatic evaluation metric Meteor is significantly correlated with all measures of cognitive effort considered. Smaller effects are also observed for source-text linguistic features. Further insight is provided into the role of the source text in post-editing, with results suggesting that consulting the source text is only associated with how cognitively demanding the task is perceived in the case of those with a low level of proficiency in the source language. Subjects’ working memory capacity was also taken into account and a relationship with post-editing productivity could be noticed. Scaled-up studies into the construct of working memory capacity and the use of eye tracking in models for quality estimation are suggested as future work.
... This followed a study by O'Brien (2006b) that had indicated a correlation between these two categories. Tatsumi (2010) shows a comparison of TM and MT matches in her study (although she uses a low volume of fuzzy matches in comparison to the volume of MT output used) and she concludes that the speed for processing MT matches lies within the fuzzy match range of editing 75 percent TM matches (English to Japanese). However, it needs to be taken into account that when looking at other studies, the language combinations and engines are different and thus it is difficult to directly compare results. ...
... Translators with less or no experience in post-editing were the slowest group but again the differences were not significant. This seems to be different from our previous findings (Guerberof 2008) and from the findings by De Almeida and O 'Brien (2010), although more in line with the findings in Tatsumi (2010). However, the numbers of participants in those studies are lower, to the extent that one post-editor has a great impact in the whole group, whereas in this project there were 24 translators with different experience and also speed. ...
Machine-translated segments are increasingly included as fuzzy matches within the translation-memory systems in the localisation workflow. This study presents preliminary results on the correlation between these two types of segments in terms of productivity and final quality. In order to test these variables, we set up an experiment with a group of eight professional translators using an on-line post-editing tool and a statistical-based machine translation engine. The translators were asked to translate new, machine-translated and translation-memory segments from the 80-90 percent value range using a post-editing tool without actually knowing the origin of each segment, and to complete a questionnaire. The findings suggest that translators have higher productivity and quality when using machine-translated output than when processing fuzzy matches from translation memories. Furthermore, translators' technical experience seems to have an impact on productivity but not on quality.
... automatic metric scores: Offersgaard et al. (2008), Tatsumi (2010) and Koponen (2012), Tatsumi and Roturier (2010), O'Brien (2011), ; confidence scores: Specia (2009aSpecia ( . 2009bSpecia ( , 2011 and He et al. (2010aHe et al. ( , 2010b, to name just a few. ...
... Translators with less or no experience in post-editing were the slowest cluster but again the differences were not significant. This seems to be different from our previous findings (Guerberof 2008) and from the findings by De Almeida and O'Brien (2010), although more in line with the findings in Tatsumi (2010). However, the numbers of participants in those studies are lower, to the extent that one post-editor has a great impact in the whole group, whereas in this project there were 24 translators. ...
In this chapter, we present results on the impact of professional experience on the task of post-editing. These results are part of a larger research project where 24 translators and three reviewers were tested to obtain productivity, words per minute, and quality data, errors in final target texts, in the post-editing of machine translation (MT) and fuzzy match segments (in the 85 to 94 range). We will discuss here the results on the participants’ experience according to their responses in a post-assignment questionnaire and explain how they were grouped into different clusters in order to correlate firstly the experience with speed according to the words per minute in the different match categories: Fuzzy matches, MT matches (MT output) and No match and secondly, to correlate them with the quality provided by measuring the errors marked by the three reviewers in each match category. Finally, conclusions will be drawn in relation to the experience and the resulting speed and number of errors.
... This followed a study by O'Brien (2006b) that had indicated a correlation between these two categories. Tatsumi (2010) shows a comparison of TM and MT matches in her study (although she uses a low volume of fuzzy matches in comparison to the volume of MT output used) and she concludes that the speed for processing MT matches lies within the fuzzy match range of editing 75 percent TM matches (English to Japanese). However, it needs to be taken into account that when looking at other studies, the language combinations and engines are different and thus it is difficult to directly compare results. ...
... Translators with less or no experience in post-editing were the slowest group but again the differences were not significant. This seems to be different from our previous findings (Guerberof 2008) and from the findings by De Almeida and O 'Brien (2010), although more in line with the findings in Tatsumi (2010). However, the numbers of participants in those studies are lower, to the extent that one post-editor has a great impact in the whole group, whereas in this project there were 24 translators with different experience and also speed. ...
This study presents empirical research on no-match, machine-translated and translation-memory segments, analyzed in terms of translators’ productivity, final quality and prior professional experience. The findings suggest that translators have higher productivity and quality when using machine-translated output than when translating on their own, and that the productivity and quality gained with machine translation are not significantly different from the values obtained when processing fuzzy matches from a translation memory in the 85-94 percent range. The translators’ prior experience impacts on the quality they deliver but not on their productivity. These quantitative findings are triangulatedwith qualitative data from an online questionnaire and from one-to-one debriefings with the translators.
... Although we used a higher number of participants in a different language combination, we share the use of a similar quality framework to analyze the number and type of errors. Tatsumi (2010) studies the speed of nine professional English to Japanese post-editors and analyzes the amount of editing made during the process, as well as the influence of the source text on speed and type of edits. ...
... This followed the study mentioned earlier by O'Brien (2006b) that had indicated a correlation between these two categories. Tatsumi (2010) shows a comparison of TM and MT matches in her study (although she uses a low volume of fuzzy matches in comparison to the volume of MT output used) and she concludes that the speed for processing MT matches lies within the fuzzy match range of editing 75 percent TM matches (English to Japanese). ...
This article presents results on the correlation between machine-translated and fuzzy matches segments in terms of productivity and final quality in the context of a localization project. In order to explore these two aspects, we set up an experiment with a group of twenty four professional translators using an online post-editing tool and a customized Moses machine translation engine with a BLEU score of 0.60. The translators were asked to translate from English to Spanish, working on no-match, machine-translated and translation memory segments from the 85-94 % value, using a post-editing tool, without actually knowing if the segment came from machine translation or from translation memory. The texts were corrected by three professional reviewers to assess the final quality of the assignment. The findings suggest that translators have higher productivity and quality when using machine-translated output than when translating without it, and that this productivity and quality is not significantly different from the values obtained when processing fuzzy matches from translation memories in the range 85-94 %.
... Additionally, translators should understand source text characteristics such as length, function, topic, specialization, structure, vocabulary, difficulty, and syntax, as well as target text requirements ) Munkova et al., 2020;Tatsumi, 2010(. However, hesitation, limited technology skills, lack of experience, slow processing speed, and knowledge gaps are factors that can affect translators' competence )Tatsumi, 2010(. ...
The engagement of Yemeni undergraduate students in an effective translation process using machine translation (MT) including Google translate (GT) seems to be affected by certain factors which may have positive or negative effect on the translation of Arabic texts into English. This study aimed to explore factors affecting Yemeni undergraduate students' translation of Arabic texts into English when using GT. To achieve this objective, a qualitative approach was employed by means of an interview to gather data from students about factors that may affect their translation. Collected data were analyzed following a thematic analysis method. Findings revealed fostering and impeding factors that affect students' translation when using GT. Fostering factors included positive attitude toward MT, awareness of GT limitations, dissatisfaction with GT translations and think-aloud strategy. On the contrary, impeding factors involved challenges faced by participants, lack of competence in using GT, negative effect of think-aloud and negative attitude toward GT. The study provides insights to educators about what may affect students' translation using GT which can be considered in training translation students on how to use GT effectively. Future research could examine factors that may affect students' use of other machine translation tools, including artificial intelligence (AI).
... The researchers used time as an objective and easily quantifiable measure to answer RQ1. This is because time is considered a simple numerical measure rather than a complex function (Tatsumi, 2010). To obtain the necessary data, the researchers referred to the participants' start and end times recorded in Translog-II, as shown in Figure 3. Subsequently, the researchers calculated the time spent on each approach by subtracting the start time from the end time. ...
Motivated by the technological advancements in computer-assisted translation (CAT) tools and the notable lack of academic research regarding their application in the Arabic-translation context, this study aims to investigate the differences in translators’ performance when comparing traditional human translation and post-edited CAT tool-generated text in terms of speed and effort. This study investigates the performance of professional translators in Saudi Arabia through traditional translation from scratch (TFS) and post-editing (PE) approaches. Data was collected from nine translators with 5–12 years of experience who had exposure to CAT tools. The participants translated an Arabic educational article into English using both methods. This study utilized Phrase CAT and Translog-II software to analyze the participants’ time and keystrokes. The results indicate that PE was significantly faster than TFS, with PE requiring 65.1% less time. PE also demanded significantly fewer keystrokes, suggesting lower technical effort. Correlations between keystrokes and time indicate a strong positive relationship in PE, implying that more technical effort correlates with increased temporal effort. These findings emphasize the efficiency of PE in enhancing productivity and suggest the importance of CAT tools and PE training for translators to meet industry demands effectively. Furthermore, this study underscores the need for continuous updates in CAT tool courses and the integration of PE training to prepare translators for constantly evolving technological landscapes.
... However, many problems also appear while using machine systems for translation as machine-translated texts are still far from publishable quality except in some narrow domains [1]. Therefore, correction by human is necessary to make machine translation output more understandable and accurate [2]. This has led to the currently heated issue of post-editing. ...
Nowadays, post-editors are in great demand in China. Therefore, this research aims to study the post-editing performance of English-major undergraduates from NingboTech University. The results indicate that: Overall, the post-editing ability of English-major undergraduates is still not sufficient for professional post-editing tasks; Their post-editing performance is related to the type of texts, and they have better performance in post-editing informative texts, while they are relatively weaker in the post-editing of expressive and vocative texts; Their post-editing quality is related to their dependence on machine translation, and the group with higher post-editing quality has a relatively lower average dependence on machine translation, but no proportional correlation is found between the two; The errors in students’ post-editing versions can be mainly classified into language competence-related errors and translation competence-related errors. Based on the results, several pedagogical implications for post-editing teaching in the future are discussed.KeywordsMachine TranslationPost-editingStudent TranslatorsTranslation Pedagogy
... The reason is that due to the large amount of human-manual work, the final editing step is largely ignored. Thus automatic post-editing (APE) might be involved in this final step, helping to reduce the human effort in editing the translated text (Tatsumi, 2010). To the best of our knowledge, there is no previous study on APE for Vietnamese. ...
Automatic post-editing (APE) is an important remedy for reducing errors of raw translated texts that are produced by machine translation (MT) systems or software-aided translation. In this paper, we present the first attempt to tackle the APE task for Vietnamese. Specifically, we construct the first large-scale dataset of 5M Vietnamese translated and corrected sentence pairs. We then apply strong neural MT models to handle the APE task, using our constructed dataset. Experimental results from both automatic and human evaluations show the effectiveness of the neural MT models in handling the Vietnamese APE task.
... -aiguiser l'esprit critique des étudiants par rapport aux avantages et aux limites toujours existantes de la TA neuronale ; -leur apprendre à ne jamais se fier aveuglément aux propositions de la machine et à ne jamais faire l'économie du retour au TS lorsqu'ils post-éditent ; -attirer leur attention sur les erreurs récurrentes, et donc prévisibles, de la TA, ce que recommandent, entre autres, Čulo et al. (2014), Daems (2016), Depraetere (2010), Killman (2018) et Tatsumi (2010. Former les étudiants à la PE ne permettrait pas uniquement d'en faire de bons post-éditeurs, mais viendrait également enrichir leurs compétences en traduction. ...
... Segments that involved the processing of particles, pronouns, and conditional expressions, particularly when these items had to be manually added or removed in the TT, were found to have substantially lower productivity rates. Omissions, particularly of pronouns, have previously been commonly found in English to Japanese machine translations (Tatsumi 2010), necessitating manual addition by the translator/post-editor. We hope that, in this study, we have added a useful application of relevance theory to translation, while evincing the need for a more fine-grained analysis within this language pair, particularly due to the fluid distinction between the two kinds of encoded meaning in relevance theory. ...
As the profession of translation has become more technologized, translators increasingly work within an interface that combines translation from scratch, translation memory suggestions, machine translation post-editing, and terminological resources. This study analyses user activity data from one such interface, and measures temporal effort for English to Japanese translation at the segment level. Using previous studies of translation within the framework of Relevance Theory as a starting point, various features and edits were identified and annotated within the texts, in order to find whether there was a relationship between their prevalence and translation effort. Although this study is exploratory in nature, there was an expectation based on previous studies that procedurally encoded utterances would be associated with greater translation effort. This expectation was complicated by the choice of a language pair in which there has been little research applying relevance theory to translation, and by contemporary research that has made the distinction between procedural and conceptual encoding appear more fluid than previously believed. Our findings are that some features that lean more towards procedural encoding (such as prevalence of pronouns and manual addition of postpositions) are associated with increased temporal effort, although the small sample size makes it impossible to generalise. Segments translated with the aid of translation memory showed the least average temporal effort, and segments translated using machine translation appeared to require more effort than translation from scratch.
... In addition to the general PE guidelines above, there are other sources of PE guidelines which are either language-dependent or aim-specific. Such guidelines include, for example, the GALE PE guidelines (WEB, b), PE guidelines with a focus on Japanese (Tatsumi, 2010), ACCEPT's guidelines for monolingual and bilingual post-editing (ACCEPT, 2011), language dependent (English-Spanish) PE guidelines (Rico and Ariano, 2014), PE guidelines for BOLT Machine Translation Evaluation (WEB, c), and PE guidelines for lay post-editors in an online community (Mitchell, 2015). ...
With the popular use of machine translation technology in the translation industry, post-editing has been widely adopted with the aim of improving target text quality. Every post-editing project needs to have specific guidelines for translators to comply with, since the guidelines may help clients and LSPs to set clear expectations, and save time and effort for translators. Different organizations make their own rules according to their needs. In this paper, we focus on comparing five sources of post-editing guidelines, and point out their overlaps and differences.
... Het zijn onder andere deze vragen waarop we binnen LT³ (de afdeling Taaltechnologie van de vakgroep Vertalen, Tolken en Communicatie aan de Universiteit Gent) door middel van het ROBOT-project (LT³, 2012) een antwoord trachten te vinden. Hoewel eerder onderzoek erop wees dat post-editing bijvoorbeeld binnen softwarelokalisatie tot snellere vertalingen kan leiden (Guerberof, 2009;Plitt & Masselot, 2010;Tatsumi, 2010), is er weinig onderzoek over het gebruik van post-editing bij algemene teksttypes terug te vinden. Het is dan ook het doel van ons project om een beter inzicht te krijgen in beide vertaalprocessen en -producten, in beide vertaalrichtingen, zowel voor studenten als professionele vertalers. ...
... Based on a field study, Federico et al. (2012) found that the post-editing effort decreases when translators are supplied with TM matches as well as with MT matches, and that most translators achieve substantial time savings when they are offered MT matches in addition to TM matches. Tatsumi (2010) investigated the editing speed and the degree of editing. For instance, she found that it is faster to edit MT matches than to edit 75-79 % fuzzy matches. ...
Though we lack empirically-based knowledge of the impact of computer-aided translation (CAT) tools on translation processes, it is generally agreed that all professional translators are now involved in some kind of translator-computer interaction (TCI), using O'Brien's (2012) term. Taking a TCI perspective, this paper investigates the relationship between machines and humans in the field of translation, analysing a CAT process in which machine-translation (MT) technology was integrated into a translation-memory (TM) suite. After a review of empirical research into the impact of CAT tools on translation processes, we report on an observational study of TCI processes in one particular instance of MT-assisted TM translation in a major Danish translation service provider (TSP). Results indicate that the CAT tool played a central role in the translation process. In fact, the study demonstrates that the translator's processes are both restrained and aided by the tool. As to the restraining influence, the study shows, for example, that the translator resists the influence of the tool by interrupting the usual segment-by-segment method encouraged by translation technology. As to the aiding influence, the study indicates that the tool helps the translator conform to project and customer requirements.
... 3 Regardless of the technology translators use, in many domains they are no longer translating texts from scratch but simply editing them segment by segment, which perhaps turns them into de facto post-editors. Over the past decade translation studies research has started to pay more attention to the PE process, starting with Krings (2001), and to investigate how PE can improve current practices within the translation industry (O' Brien 2006Brien , 2011Garcia 2010;Ramos 2010;Tatsumi 2010). One area requiring further investigation is the training of translators in PE, with particular focus on PE guidelines for training purposes. ...
There is a growing interest in machine translation (MT) and post-editing (PE). MT has been around for decades, but the use of the technology has grown significantly in the language industry in recent years, while PE is still a relatively new task. Consequently, there are currently no standard PE guidelines to use in translator training programmes. Recently, the first set of publicly available industry-focused PE guidelines (for 'good enough' and 'publishable' quality) were developed by Translation Automation User Society (TAUS) in partnership with the Centre for Global Intelligent Content (CNGL), which can be used as a basis on which to instruct post-editors in professional environments. This paper reports on a qualitative study that investigates how trainee translators on an MA course, which is aimed at preparing the trainees for the translation industry, interpret these PE guidelines for publishable quality. The findings suggest trainees have difficulties interpreting the guidelines, primarily due to trainee competency gaps, but also due to the wording of the guidelines. Based on our findings we propose training measures to address these competency gaps. Furthermore, we provide post-editing guidelines that we plan to use for our own post-editing training.
... With the translation industry exponentially growing , more hope is vested in the use of machine translation (MT) to increase translators' productivity (Rinsche and Portera-Zanotti, 2009). Though post-editing MT has proven to increase productivity and even quality for certain text types (Tatsumi, 2010), research on the usability of post-editing for more general texts is rather limited. The research presented in this paper is a pilot study conducted as part of the ROBOT-project 1 , a project designed to gain insight in the differences between human translation and the post-editing of machine translation . ...
Existing translation quality assessment (TQA) metrics have a few major draw-backs: they are often subjective, their scope is limited to the sentence level, and they do not take the translation situation into account. Though suitable for a gen-eral assessment, they lack the granularity needed to compare different methods of translation and their respective translation problems. In an attempt to solve these issues, a two-step TQA-approach is pre-sented, based on the dichotomy between adequacy and acceptability. The proposed categorization allows for easy customiza-tion and user-defined error weights, which makes it suitable for different types of translation assessment, analysis and com-parison. In the first part of the paper, the approach is explained. In the second part of the paper, the approach is tested in a pilot study designed to compare human translation with post-editing for the trans-lation of general texts (newspaper articles). Inter-annotator results are presented for the translation quality assessment task as well as general findings on the productiv-ity and quality differences between post-editing and human translation of student translators.
This article examines the efficiency and consistency of Trados Studio in Arabic-English legal translation by analyzing the post-editing practices of 15 female Saudi translation students. The legal texts used are moderately complex, laden with Saudi-specific legal terminology and Islamic rules, sourced from Adel Azzam Saqf Al-Hait's "The Reliable Guide to Legal Translation" (2012). The Trados Studio Machine Translation Post-Editing Questionnaire (TMTPEQ) was employed to gather users’ insights. The study utilizes mixed methods, combining qualitative content analysis of Trados Studio-translated texts and student post-editing tasks with descriptive quantitative analysis of the questionnaire. The results reveal the ongoing challenge of achieving precision and extra-linguistic significance in written texts, especially in legal translation, exacerbated by specialized terminology, cultural nuances, and complex syntax. They also underscore the significance of qualitative assessments in post-edited translations, emphasizing the multifaceted role of translators encompassing linguistic, cultural, and specialized content precision and functional and legal equivalence.
The modern translation industry is using machine translation post-editing (MTPE) widely, and the translation industry in the Arab World is following the global lead. However, while MTPE training is offered in many language pairs around the world, MTPE training in English-Arabic is still not officially offered in translation training programmes in the Arab World. The aim of this study is to evaluate the effectiveness of MTPE training in a female undergraduate translation programme in Saudi Arabia by examining students’ opinions about MTPE and comparing its productivity and quality with an established practice in the translation classroom, i.e., human translation (HT). To achieve its aim, this study used a mixed-method design of the ‘Kirkpatrick Model of Learning Evaluation’. Focus group discussions and retrospective pre-test surveys were used to examine students’ opinions as well as a pre-post experiment which involved two groups of students (29 in the control group and 31 in the experimental group) that was used to compare the productivity of students and the quality of translated texts when using MTPE as compared with HT. Students’ opinions that were revealed through the pre-intervention focus group discussions were generally mixed with a preference shown in favour of HT, except for translation speed as most of the students thought that MTPE was the faster method of translation. As for the survey, students’ pre-intervention responses supported those opinions revealed in the focus group discussions. However, post-intervention responses revealed a statistically significant shift towards more acceptance of MTPE training and use, indicating that the more students learned about the features of MT and MTPE skills and practiced them, the more positive their opinions became. Statistical results from comparing students’ productivity showed a medium effect size which indicates that MTPE cannot be ignored as a method to increase productivity in translation. The effectiveness of MTPE in translation quality was evaluated by measuring error count and error type. Error count analysis indicated that students who used MTPE have increased scores in a similar manner to those who used HT but not more. The analysis of error type showed that while MTPE helped students avoid deletion and technical errors, the number of errors relating to accuracy, comprehension and grammar were more frequent in Arabic MTPE translated texts.
While translation has always required the ability to find information, currently this process has moved almost entirely into the digital realm. The universal revolution in translation, which happened many years ago, has transformed the profession into something resembling piloting an airplane because of the numerous tools to aid the process and help find information (Gouadec 2007: 263). Information mining and the use of other tools, such as machine translation, has become fixed points in translation curricula, but there remains a scarcity of research into both of these aspects when related to translation trainees.
In line with the translation process research paradigm, this thesis is an attempt to bridge this gap in research and to discuss information searching in the process of translation and post- editing. The aim of this project is to investigate translation trainees and EFL students as they interact with machine translation and online resources during translation and post-editing tasks for two text types (operative-technical and informative-medical, cf. Reiss 1976). The first objective of the thesis is to examine whether both groups put more effort into information searching when translating than when post-editing. Two indicators of effort have been used to test this hypothesis: time spent in applications (temporal effort) and average fixation duration (cognitive effort). The results show that the task type significantly influences the amount of temporal effort put into the use of online resources – both on the global level of all resource categories considered together and for some of them considered separately. No such effect has been found for the cognitive effort indicators. The second hypothesis in the study posits that translation trainees exert more temporal and cognitive effort in both translation and post-editing than EFL students. Again, the results show that this can only be partially confirmed. Significant differences exist only for temporal effort variables: the time spent on Wikipedia and language reference websites (like the Polish language advice centre, Poradnia językowa PWN). In both cases trainees spent more time consulting these resources. The interaction of the group and task effect was found in the use of monolingual dictionaries and it turns out that EFL students put more effort into consulting them. The third hypothesis focuses on the range of consulted online resources in relation to task type and group membership. Contrary to expectations, there is no effect of either group or task on the range of consulted resources. For the fourth hypothesis, accuracy in translating source text rich points is examined. Contrary to the expected group effect on accuracy scores, there is no statistically significant difference between the groups in
190
terms of how accurate they were. There is also no significant correlation between the accuracy of translations and the percentage of rich points (i.e. focal words or phrases) researched by a participant online. The fifth hypothesis concerns the relationship between the attitude towards machine translation and the percentage of time spent in online resources in relation to the whole task time during post-editing – the results show there is no statistically significant correlation between these variables, even for a follow-up correlational analysis between total task time and attitude scores. For the sixth hypothesis, an indicator of perceived effort is correlated with time spent in various online resource categories. The results reveal positive correlations with select temporal effort categories with reference to groups, tasks and texts as well as for each of these variables separately. For the last hypothesis, the correlation between the perceived effort indicator and the range of consulted online resources is examined. The results show a significant positive correlation only for one of the researched text types, i.e. a product description (operative-technical) – regardless of group membership or task type performed.
The results indicate that the relationship between effort, accuracy, and attitude in information searching during translation and post-editing is intensely nuanced. The findings of this study may be particularly valuable for translation trainers and translation process researchers. Although this project is limited in scope, it might provide a prelude into more extensive and focused studies of information searching in relation to translation training and translator competence development – and how machine translation influences the translation process as well. Examining the information searching process in translation students and incorporating self-reflection into translation pedagogy is likely to be beneficial for training more self-aware professionals, ready to commence the journey of life-long learning as translators.
This book explains the concept, framework, implementation, and evaluation of controlled document authoring in this age of translation technologies. Machine translation (MT) is routinely used in many situations, by companies, governments, and individuals. Despite recent advances, MT tools are still known to be imperfect, sometimes producing critical errors. To enhance the performance of MT, researchers and language practitioners have developed controlled languages that impose restrictions on the form or length of the source-language text. However, a fundamental, persisting problem is that both current MT systems and controlled languages deal only with the sentence as the unit of processing. To be effective, controlled languages must be contextualised at the document level, consequently enabling MT to generate outputs appropriate for their functional context within the target document. With a specific focus on Japanese municipal documents, this book establishes a framework for controlled document authoring by integrating various research strands including document formalisation, controlled language, and terminology management. It then presents the development and evaluation of an authoring support system, MuTUAL, that is designed to help non-professional writers create well-organised documents that are both readable and translatable. The book provides useful insights for researchers and practitioners interested in translation technology, technical writing, and natural language processing applications.
A Lektorálástudomány – fordításban című tanulmánykötet – a sorozat előző kötete, a Fordítástudomány – fordításban folytatásaként – válogatott írásokat kínál az olvasók számára a lektorálással kapcsolatos kutatások szaktekintélyeitől. Különlegessége abban rejlik, hogy a nemzetközi szakirodalom képviselői ezúttal magyar nyelven szólalnak meg a könyv lapjain, az Eötvös Loránd Tudományegyetem szakfordító hallgatóinak tolmácsolásában. A kiválasztott tanulmányok híven tanúskodnak a lektoráláskutatás sokrétűségéről, összefoglalják az eddigi eredményeket, és áttekintést nyújtanak a fő elméleti dilemmákról. Érdekes gondolatokat olvashatunk a lektorálás terminológiájáról, a lektorok beavatkozásairól, a lektori kompetencia meghatározó összetevőiről, a lektorálás módszertani kérdéseiről, valamint a sajtófordításban megjelenő transzeditálásról.
A kötet mindazok érdeklődésére számot tarthat, akik gyakorlatban művelik a fordítást, és szívesen tájékozódnak a szakmájukkal kapcsolatban folyó tudományos vizsgálódásokról, különös tekintettel a lektorálásra; akik maguk is tudományos alapokon kutatják a fordítás folyamatait és jellemzőit; és azok a jövendő szakfordító hallgatók is haszonnal forgathatják, akik saját tanulmányaikhoz keresnek fordítástudományi témájú szövegeket a lektorálásról – magyar fordításban.
The sharp rise in the use of technology tools in the translation process has rendered human translators more invisible than ever. The importance of the role played by human translators in translation, however, cannot be denied or understated. This paper aims to examine the primary factors influencing the work of human translation combined with translation technology tools. Therefore, the paper provides an overview of the translation and language industries and insights into translation industry standards, quality concerns and the most frequently used tools, as they are aspects that influence and condition the translator's work today. The final and main section in this work emphasizes an increasingly common trend in translation: a human-assisted machine translation model based on the post-edition of the output from machine translation systems. By analyzing market studies, surveys and papers on the aforementioned aspects, this article confirms that the role of human translators in technology-driven translation processes will be as central in the future as it is today.
As part of a larger research project on productivity and quality in the post-editing of machine-translated and translation-memory outputs, 24 translators and three reviewers were asked to complete an on-line questionnaire to gather information about their professional experience but also to obtain data on their opinions about post-editing and machine translation. The participants were also debriefed after finalising the assignment to triangulate the data with the quantitative results and the questionnaire. The results show that translators have mixed experiences and feelings towards machine-translated output and post-editing, not necessarily because they are misinformed or reluctant to accept its inclusion in the localisation process but due to their previous experience with various degrees of output quality and to the characteristics of this type of projects. The translators were quite satisfied in general with the work they do as translators, but not necessarily with the payment they receive for the work done, although this was highly dependent on different customers and type of tasks.
Today technology is part and parcel of professional translation, and translation has therefore been characterised as Translator-Computer Interaction (TCI) (O’Brien 2012). Translation is increasingly carried out using Translation Memory (TM) systems which incorporate machine translation (MT), referred to as MT-assisted TM translation, and in this type of tool, translators switch between editing TM matches and post-editing MT matches. It is generally assumed that translators’ attitudes towards technology impact on this interaction with the technology. Drawing on Eagly/Chaiken’s (1995) definition of attitudes as evaluations of entities with favour or disfavour and on qualitative data from a workplace study of TCI, conducted as part of a PhD dissertation (Bundgaard 2017) and partly reported on in Bundgaard et al. (2016), this paper explores translator attitudes towards TCI in the form of MT-assisted TM translation. In doing so, the paper has a particular focus on the disfavour towards TCI expressed by translators. Moreover, inspired by Olohan (2011), who applies Pickering’s “mangle of practice” theory and analyses resistance and accommodation in TCI, the paper focuses on how translators accommodate resistances offered by the tool. The study shows that the translators express disfavour towards MT in many respects, but also acknowledge positive aspects of the technology and expect MT to play a significant role in their future working lives. The translators do not make many positive or negative comments about TM which might indicate that TM is a completely integrated part of their processes. The translators seem to have a flexible and pragmatic attitude towards TCI, adapting to the tool’s imperfections and accommodating its resistances.
Este trabajo pone de manifiesto la importancia de la postedición (PE) en el panorama actual de producción de contenido técnico multilingüe, analizando el impacto de esta actividad humana entroncada en paradigmas de automatización de la traducción, desde el punto de vista de los proveedores de servicios lingüísticos (agencias y traductores autónomos) y con respecto a su interrelación con otros componentes y recursos tecnológicos de la traducción. Se aportan valoraciones de otros expertos así como ciertas recomendaciones para aquellos proveedores que todavía no hayan abordado la postedición como servicio lingüístico.
Despite all the criticisms on the “poor” quality of Machine Translation (MT), the use of MT for information circulation is not to be ignored. Meanwhile, recent MT systems have been equipped with increasing customizability, allowing users to tailor the software to specific texts for better output. This thesis intends to investigate the output of MT for specialized language and its lexical customization: a case study of using SYSTRAN to translate abstracts of domain-specific academic articles. The translation of abstracts is not only suitable for MT, but largely meaningful given the massive demand for information-oriented translation.
Starting from a general discussion of the theoretical perspectives regarding MT usage, translation, and specialized language, the thesis points out a conceptual gap in MT evaluation and takes the functionalist perspective with information assimilation as the primary purpose. The case study then uses the 2014 volume of Applied Optics for an in-depth analysis of the MT system’s raw output and lexical customization. The totally 1,328 abstracts are first systematically sampled for a preliminary discussion of the Source Text (ST) features concerning lexical ambiguity, together with SYSTRAN’s disambiguation results. This is followed by further lexical error analysis, providing the basis for glossary modification in SYSTRAN. The comparison between the initial translation and the customized translation shows significant improvement beyond the lexical issues, and indicates the effectiveness of the customization conducted.
Further discussions beyond the sample show that the modified glossary entries are representative of the entire journal, as well as other similar journals. This is conducted via corpus-based investigations of Applied Optics and two other journals of the same kind –– Optics Express and Optics Letters.
SYSTRAN’s automatic process of lexical customization is also discussed, where the results show considerable inaccuracy but can shed light on what words to add into the User Dictionary before translating an ST.
These investigations not only argue for a proper perspective of MT use, but also provide an implication for the practical methods for lexical customization.
A book review of The Routledge Encylopedia of Translation Technology by Chan Sin-wai, written for The Journal of Specialised Translation.
Language translation is slow and expensive, so various forms of machine assistance have been devised. Automatic machine translation systems process text quickly and cheaply, but with quality far below that of skilled human translators. To bridge this quality gap, the translation industry has investigated post-editing, or the manual correction of machine output. We present the first rigorous, controlled analysis of post-editing and find that post-editing leads to reduced time and, surprisingly, improved quality for three diverse language pairs (English to Arabic, French, and German). Our statistical models and visualizations of experimental data indicate that some simple predictors (like source text part of speech counts) predict translation time, and that post-editing results in very different interaction patterns. From these results we distill implications for the design of new language translation interfaces.
This poster is a preliminary report of our experiments for detecting semantically shifted terms between different domains for the purposes of new concept extraction. A given term in one domain may represent a different concept in another domain. In our approach, we quantify the degree of similarity of words between different domains by measuring the degree of overlap in their domain-specific semantic spaces. The domain-specific semantic spaces are defined by extracting families of syntactically similar words, i.e. words that occur in the same syntactic context. Our method does not rely on any external resources other than a syntactic parser. Yet it has the potential to extract semantically shifted terms between two different domains automatically while paying close attention to contextual information. The organization of the poster is as follows: Section 1 provides our motivation. Section 2 provides an overview of our NLP technology and explains how we extract syntactically similar words. Section 3 describes the design of our experiments and our method. Section 4 provides our observations and preliminary results. Section 5 presents some work to be done in the future and concluding remarks.
Since sentences in patent texts are long, they are difficult to translate by a machine. Although statistical machine translation is one of the major streams of the field, long patent sentences are difficult to translate not using syntactic analysis. We propose the combination of a rule based method and a statistical method. It is a rule based machine translation (RMT) with a statistical based post editor (SPE). The evaluation by the NIST score shows RMT+SPE is more accurate than RMT only. Manual checks, however, show the outputs of RMT+SPE often have strange expressions in the target language. So we propose a new evaluation measure NMG (normalized mean grams). Although NMG is based on n-gram, it counts the number of words in the longest word sequence matches between the test sentence and the target language reference corpus. We use two reference corpora. One is the reference translation only the other is a large scaled target language corpus. In the former case, RMT+SPE wins in the later case, RMT wins.
This paper analyzes the translation qual- ity of machine translation systems for 10 language pairs translating between Czech, English, French, German, Hungarian, and Spanish. We report the translation quality of over 30 diverse translation systems based on a large-scale manual evaluation involv- ing hundreds of hours of effort. We use the human judgments of the systems to analyze automatic evaluation metrics for translation quality, and we report the strength of the cor- relation with human judgments at both the system-level and at the sentence-level. We validate our manual evaluation methodol- ogy by measuring intra- and inter-annotator agreement, and collecting timing informa- tion.
This paper describes two studies on the effectiveness of Controlled Language (CL) rules for MT. Both studies investigated the language pair English-German and used corpora from the IT domain. However, they differ in terms of the MT engines employed (Systran vs. IBM WebSphere) and the evaluative methodologies used. Study A examines the effectiveness of CL rules by measuring temporal, technical and post-editing effort. Study B examines the effectiveness of rules by measuring comprehensibility. Both Study A and Study B concluded that some CL rules had a high impact for MT while other rules had a moderate, low or no impact. The results are compared in order to determine what, if any, common conclusions can be drawn. Our conclusions are that rules governing misspelling, incorrect punctuation, sentences longer than 25 words, and the use of personal pronouns with no antecedent in a sentence had a high impact on both post-editing effort and comprehensibility. Further, we found that the use of personal pronouns with antecedents in the same sentence and stand-alone demonstrative pronouns had a low impact, while the rule advocating the use of "in order to" in purposive clauses had no impact in either study. The paper also discusses contrasting results for both studies.
Nagao (8) suggests to model machine translation as an imitation of a process that can be observed in human translation behavior. However, not much work has been done in an MT context that looks into human translation processes. This paper seeks to introduce a research method for the investigation of human translation behavior based on User-Activity Data (UAD) which consists of the translator's recorded keystroke and eye-movement behavior and which can be re- played and analysed off-line. The paper gives the background of this technique and an example on a English-to-Danish translation. Our goal is to elaborate and investigate cognitively grounded basic translation concepts which are material- ized and traceable in the UAD.
This paper describes a tool designed to assess the machine translatability of English source texts by assigning a translatability index to both individual sentences and the text as a whole. The tool is designed to be both stand-alone and integratable into a suite of other tools which together help to improve the quality of professional translation in the preparatory phase of the translation workflow. Assessing translatability is an important element in ensuring the most efficient and cost effective use of current translation technology, and the tool must be able to quickly determine the translatability of a text without itself using too many resources. It is therefore based on rather simple tagging and pattern matching technologies which bring with them a certain level of indeterminacy. This potential disadvantage can, however, be offset by the fact that an annotated version of the text is simultaneously produced to allow the user to interpret the results of the checker.
This paper evaluates the translation quality of machine translation systems for 8 language pairs: translating French, German, Spanish, and Czech to English and back. We carried out an extensive human evaluation which allowed us not only to rank the different MT systems, but also to perform higher-level analysis of the evaluation process. We measured timing and intra- and inter-annotator agreement for three types of subjective evaluation. We measured the correlation of automatic evaluation metrics with human judgments. This meta-evaluation reveals surprising facts about the most commonly used methodologies.
In this paper, we present an overview of the issues in designing a controlled language, the implementation of a controlled language checker, and the deployment of KANT Con- trolled English for multilingual machine translation. We also discuss some success criteria for introducing controlled language. Finally, future vision of KANT controlled language development is discussed.
In this paper, we propose a new statistical Japanese dependency parser using a cascaded chunking model. Conventional Japanese statistical depen-dency parsers are mainly based on a probabilistic model, which is not always efficient or scalable. We propose a new method that is simple and efficient, since it parses a sentence deterministically only de-ciding whether the current segment modifies the segment on its immediate right hand side. Experi-ments using the Kyoto University Corpus show that the method outperforms previous systems as well as improves the parsing and training efficiency.
Variations in group sub-languages evolve quickly and are a key marker of social boundaries such as those between profes-sions, workgroups, tribes and families. In this paper we present a quantitative analy-sis of the effects of group structure on lan-guage use in naturalistic interaction. The data come from text chat interactions in an online social community. Using sta-tistical techniques developed for the anal-ysis of authorship attribution we use this corpus to test three accounts of the emer-gence of group sub-languages: a) local co-ordination mechanisms b) network topol-ogy and c) influential individuals. The re-sults indicate that it is influential individu-als who have the strongest effects on sub-language convergence.
This paper summarises the results of a pilot project conducted to investigate the correlation between automatic evaluation metric scores and post-editing speed on a segment by segment basis. Firstly, the results from the comparison of various automatic metrics and post-editing speed will be reported. Secondly, further analysis is carried out by taking into consideration other relevant variables, such as text length and structures, and by means of multiple regression. It has been found that different automatic metrics achieve different levels and types of correlation with post-editing speed. We suggest that some of the source text characteristics and machine translation errors may be able to account for the gap between the automatic metric scores and post-editing speed, and may also help with understanding human post-editing process.
We investigate the problem of estimating the quality of the output of machine translation systems at the sentence level when reference translations are not available. The focus is on automatically identifying a threshold to map a continuous predicted score into "good" / "bad" categories for filtering out bad-quality cases in a translation post-edition task. We use the theory of Inductive Confidence Machines (ICM) to identify this threshold according to a confidence level that is expected for a given task. Experiments show that this approach gives improved estimates when compared to those based on classification or regression al-gorithms without ICM.
We examine a new, intuitive measure for evaluating machine-translation output that avoids the knowledge intensiveness of more meaning-based approaches, and the labor-intensiveness of human judg-ments. Translation Edit Rate (TER) mea-sures the amount of editing that a hu-man would have to perform to change a system output so it exactly matches a reference translation. We show that the single-reference variant of TER correlates as well with human judgments of MT quality as the four-reference variant of BLEU. We also define a human-targeted TER (or HTER) and show that it yields higher correlations with human judgments than BLEU—even when BLEU is given human-targeted references. Our results in-dicate that HTER correlates with human judgments better than HMETEOR and that the four-reference variants of TER and HTER correlate with human judg-ments as well as—or better than—a sec-ond human judgment does.
In this paper, we describe an Automatic Rule Refiner that, given online post-editing information, traces errors back to problem-atic translation rules and proposes concrete fixes to such rules. Evaluation results on an English-to-Spanish Transfer-based MT sys-tem show that this approach generalizes beyond sentences corrected by bilingual speakers to unseen data. We show that by applying automatic refinements, higher translation accuracy can be achieved as measured by automatic evaluation metrics.
In this paper we investigate the possibility of evaluating MT quality and fluency at the sentence level in the absence of reference translations. We measure the correlation between automatically-generated scores and human judgments, and we evaluate the per-formance of our system when used as a classifier for identifying highly dysfluent and ill-formed sentences. We show that we can substantially improve on the correlation between language model perplexity scores and human judgment by combining these perplexity scores with class probabilities from a machine-learned classifier. The classifier uses linguis-tic features and has been trained to distinguish human translations from machine transla-tions. We show that this approach also performs well in identifying dysfluent sentences.
This article describes the combination of a SYSTRAN system with a "statistical post-editing" (SPE) system. We document qualitative analysis on two experiments performed in the shared task of the ACL 2007 Workshop on Statistical Machine Translation. Comparative results and more integrated "hybrid" techniques are dis-cussed.
This paper investigates the relationships among controlled language (CL), machine translation (MT) quality, and post-editing (PE). Previous research has shown that the use of CL improves the quality of MT. By extension, we assume that the use of CL will lead to greater productivity or reduced PE effort. The paper examines whether this three-way relationship among CL, MT quality, and PE holds. Beginning with a set of CL rules, we determine what types of CL rules have the greatest cross-linguistic impact on MT quality. We create two sets of English data, one which violates the CL rules and the other which conforms to them. We translate both sets of sentences into four typologically different languages (Dutch, Chinese, Arabic, and French) using MSR-MT, a statistical machine translation system developed at Microsoft. We measure the degree of impact of CL rules on MT quality based on the difference in human evaluation as well as BLEU scores between the two sets of MT output. Finally, we examine whether the use of CL improves productivity in terms of reduced PE effort, using character-based edit-distance.
A user-friendly interface to search bilingual resources is of great help to NLP developers as well as pure-linguists. Using bilingual re- sources is difficult for linguists who are unfa- miliar with computation, which hampers capa- bilities of bilingual resources. NLP developers sometimes need a kind of workbench to check their resources. The online interface this paper introduces can satisfy these needs. In order to implement the interface, this research deals with how to align Korean and Japanese phras- es and interpolates them into the original bi- lingual corpus in an automatic way.
In Japanese dependency parsing, Kudo's relative preference-based method (Kudo and Matsumoto, 2005) outperforms both deterministic and probabilistic CKY-based parsing methods. In Kudo's method, for each dependent word (or chunk) a log-linear model estimates relative preference of all other candidate words (or chunks) for being as its head. This cannot be considered in the deterministic parsing methods. We propose an algorithm based on a tournament model, in which the relative preferences are directly modeled by one-on-one games in a step-ladder tournament. In an evaluation experiment with Kyoto Text Corpus Version 4.0, the proposed method outperforms previous approaches, including the relative preference-based method.
This work deals with the application of confidence measures within an interactive-predictive machine translation system in order to reduce human effort. If a small loss in translation quality can be tolerated for the sake of efficiency, user effort can be saved by interactively translating only those initial translations which the confidence measure classifies as incorrect. We apply confidence estimation as a way to achieve a balance between user effort savings and final translation error. Empirical results show that our proposal allows to obtain almost perfect translations while significantly reducing user effort.
We investigate novel types of assistance for human translators, based on statistical machine translation methods. We developed
the computer-aided tool Caitra that makes suggestions for sentence completion, shows word and phrase translation options, and allows postediting of machine
translation output. We carried out a study of the translation process that involved non-professional translators that were
native in either French or English and recorded their interaction with the tool. Users translated 192 sentences from French
news stories into English. Most translators were faster and better when assisted by our tool. A detailed examination of the
logs also provides insight into the human translation process, such as time spent on different activities and length of pauses.
For this work we have carried out a number of analysis experiments comparing raw MT output produced by Microsofts Treelet MT engine (Quirk et al., 2005) with its human post-edited counterpart, for EnglishGerman and EnglishFrench. Through these experiments we identify a number of interesting post-editing patterns, both textual (string-based) and constituent-based. In this paper we discuss our analysis methodologies, present some of our results and provide information on how this type of analysis can be of benefit to translation systems and posteditors, with a view to improving initial MT output and consequently post-editor productivity. In addition, we also discuss the MT and post-editing workflow at Microsoft and results from MT post-editing pilots for a number of different language pairs.
We aim at carrying out an empirical study to clar-ify if texts checked with a controlled language (CL) checker are indeed more translatable than other texts which are not compliant with the CL rule set, evaluating thus the degree of success of the appli-cation of such a restricted language with regard to its machine oriented features. For such an eval-uation we adapt the FEMTI-Framework (Hovy et al., 2002) to our needs and divide our evaluation in two parts: selection of resources for the evalua-tion of a CL, and evaluation of the CL. In this ar-ticle only the findings and results of the first part are presented. In order to simulate a real context of work, we use the system MULTILINT, a sophisti-cated language checker developed by the Institut der Gesellschaft zur Förderung der Angewandten Infor-mationsforschung e.V. an der Universität des Saar-landes (IAI). Furthermore, we use automatic trans-lations made with a Machine Translation System.
SDL, in association with the International Association for Machine Translation (IAMT) and Association for Machine Translation Americas (AMTA), ran a survey which was completed by over 385 individuals in global businesses. The results were fascinating and definitely show an increased interest in the use of automated translation over the last two years. Those surveyed included individuals from companies such as Cisco, Dell, HP, Motorola, Oracle, Panasonic, Philips, Siemens, Sun and Vodafone. 71% of respondents were of managerial level and above, including Vice President and Senior Vice President from different parts of the business – sales and marketing, technical documentation, localization and IT and engineering. Business motivation and utilization Most of the respondents were familiar with the concept of automated translation, with only 7% being unfamiliar with it. 52% of respondents have considered using automated translation in business. Making up that number, 15% are actually using automated translation and 8% plan to use it – so 23% -nearly a quarter of respondents -are either using it or plan to use it. Interestingly, and possibly surprising low, only 30% say that they do not have plans to use it as they have concerns about the quality. The question was asked, "How likely are you to use automated translation now, compared to two years ago?" 40% of respondents said they were more likely now versus two years ago to use automated translation. That is a substantial industry shift and shows quite a development in interest and trust in this technology and its uses. Additionally, 27% said they felt the same as two years ago and would plan to use it, with only 3% less likely now than two years ago. 30% did not change their opinion and would still not use it. So 57% of respondents would currently be in the position to consider automated translation. [8th AMTA conference, Hawaii, 21-25 October 2008] 394 The business motivations for using automated translation – the benefits that have always been provided by this technology – are the requirement to translate at less cost and in less time. However, nearly 40% said that a business motivation for using this technology would be to provide consistency of brand, since the technology always translates the same terms consistently.
This paper focuses on one aspect of controlled authoring in a localization and Machine-Translation context: the treatment of GUI options, which abound in the procedural sections of IT technical documentation. GUI options are technical terms that refer to the Software User Interface. The length and complexity of GUI options is a major problem for numerous NLP tasks, including MT. GUI options which have not been identified by NLP applications typically lead to erroneous analyses of sentences. However, few authors have focused on the identification and tagging of GUI options in IT documentation. This paper delineates an approach based on a controlled language checker that benefits both the human authoring process and Machine Translation.
This article aims at redefining automatic translation within the translation workflow process, and how a new translation software product has been able to revolutionize the field of Machine Translation (MT) and MT Postediting. The term Postediting (PE) is by far most commonly associated as a task that is related to Machine Translation (MT). In basic terms, the task of the posteditor is to edit, modify and/or correct pre-translated text that has been processed by a machine translation system from a source language into (a) target language(s). The notion of Postediting has been adequately defined as the "term used for the correction of machine translation output by human linguists / editors." (Senez, 1998). PE has also been briefly defined elsewhere as the "correction of a pre-translated text rather than translation from scratch." (Wagner, 1985). Up until present, all PE production projects to my knowledge have run documents in batch mode through a machine translation system and then provide the raw output texts to human posteditors. These posteditors use existing third party desktop publishing software (e.g., Microsoft Word, Arbortext SGML editor, etc) to conduct the PE tasks in a process that is referred to as "passive postediting" because it requires the posteditor to adapt this new type of translation task to software that was unfortunately not designed for such a task. As a result, some efforts have been made to help improve the passive PE process by trying to outline and draw up a list of PE criteria. The Postediting Special Interest Group (Postediting SIG) of the Association for MT in the Americas (AMTA), a group that was set up in 1998 (Allen, 1999), has been concerned with the absence of publicly available postediting guidelines although MT integration has moving forward in significant ways during the past few years. Given this absence of specific PE guidelines, each MT implementation project has been forced to invent their own methods and strategies for conducting the PE tasks. All existing publicly available guidelines are often very general and basic, such as those proposed by Löffler-Laurian (1996:93-94): 1) criterion of situation and document type, 2) criterion of necessity, and 3) criterion of comprehensibility. There have been distinctions made between Rapid PE, Minimal PE, and Full PE, yet these terms, and the tasks that accompany them, are often insufficient due to the lack of detailed criteria for the actual PE task. It is not uncommon that posteditors who are asked to conduct Rapid or Minimal PE actually end up making changes to a pre-translated document which are actually closer to the Full Postediting side of the spectrum. The simple reason for this is that everyone wants to produce the best translated document possible, and to retain translation jobs with the same client in the future, including PE jobs.
Machine Translation (MT) is rapidly pro- gressing towards quality levels that might make it appropriate for broad user popula- tions in a range of scenarios, including gist- ing and post-editing in unconstrained do- mains. For this to happen, the field may however need to switch gear and move away from its current technology driven paradigm to a more user-centered approach. In this paper, we discuss how ethnographic techniques like Contextual Inquiry could help in that respect, by providing re- searchers and developers with rich informa- tion about the world and needs of potential end-users. We discuss how data from Con- textual Inquiries with professional transla- tors was used to concretely and positively influence several R&D projects in the area of Computer Assisted Translation technolo- gy. These inquiries had many benefits, in- cluding: (i) grounding developers and re- searchers in the world of their end-users, (ii) generating new technology ideas, (iii) se- lecting between competing development project ideas, (iv) finding how to alleviate friction for important ideas that go against the grain of current user practices, (v) evalu- ating existing or experimental technologies, (vi) helping with micro level design deci- sion, (vii) building credibility with transla- tors, and (viii) fostering multidisciplinary discussion between researchers.
For a professional user of MT, quality, performance and cost efficiency are critical. It is therefore surprising that only little attention - both in theory and in practice - has been given to the task of post-editing machine translated texts. This paper will focus on this important user aspect and demonstrate that substantial savings in time and effort can be achieved by implementing intelligent automatic tools. Our point of departure is the PaTrans MT-system, developed by CST and used by the Danish translation company Lingtech. An intelligent post-editing facility, Ape, has been developed and added to the system. We will outline and discuss this mechanism and its positive effects on the output. The underlying idea of the intelligent post-editing facility is to exploit the lexical and grammatical knowledge already present in the MT-system's linguistic components. Conceptually, our approach is general, although its implementation remains system specific. Surveys of post-editor satisfaction and cost-efficiency improvements, as well as a quantitative, benchmark-based evaluation of the effect of Ape demonstrate the success of the approach and encourage further development.
Various typologies of technical and scientifical texts have already been proposed by authors involved in multilingual transfer problems. They were usually aimed at a better knowledge of the criteria for deciding if a document has to be or can be machine translated. Such a typology could also lead to a better knowledge of the typical errors occuring, and so lead to more appropriate post-editing, as well as to improvements in the system.Raw translations being usable, as they are quite often for rapid information needs, it is important to draw the limits between a style adequate for rapid information, and an elegant, high quality style such as required for information large dissemination. Style could be given a new definition through a linguistic analysis based on machine translation, on communication situations and on the users' requirements and satisfaction.
We have produced a test collection for ma- chine translation (MT). Our test collection in- cludes approximately 2000000 sentence pairs in Japanese and English, which were extracted from patent documents and can be used to train and evaluate MT systems. Our test col- lection also includes search topics for cross- lingual information retrieval, to evaluate the contribution of MT to retrieving patent docu- ments across languages. We performed a task for MT at the NTCIR workshop and used our test collection to evaluate participating groups. This paper describes scientific knowledge ob- tained through our task.
Problems inherent in current machine translation systems have been reviewed and have been shown to be inherently inconsistent. The present paper defines a model based on a series of human language processing and in particular the use of analogical thinking. Machine translation systems developed so far have a kind of inherent contradiction in themselves. The more detailed a system has become by the additional improvements, the clearer the limitation and the boundary will be for the translation ability. To break through this difficulty we have to think about the mechanism of human translation, and have to build a model based on the fundamental function of language processing in the human brain. The following is an attempt to do this based on the ability of analogy finding in human beings.
In large reference corpora representativeness is attempted through carefully selected sampling and sheer size. The situation is different with special language corpora in that their very nature limits them in size. Their representativity is measured by reference to external selection criteria, generally following bibliographic classifications, which tend to be subjective. In order to overcome subjectivity in specialised corpora, a corpus-directed system of internal selection using lexical criteria is proposed. The aim is not to create rigid boundaries but to see clearly what is actually present in the corpus. The method adopted is demonstrated on a corpus consisting of research articles from specialised journals and conference proceedings in the field of plant biology. Restricted collocational networks are used to isolate prototypical groupings within the corpus. It is shown that audience is an important factor in strong and weak prototypical groupings in theme and domain specific corpora. Articles addressing domain specialists through a journal tend to be more central than those presented to a theme-specific discourse community through conference proceedings.
We argue that it is useful for a machine translation system to be able to provide the user with an estimate of the translation quality for each sentence. This makes it possible for bad translations to be filtered out before post-editing, to be highlighted by the user interface, or to cause an interactive system to ask for a rephrasing. A system providing such an estimate is described, and examples from its practical application to an MT system are given.
The approach presented here enables Japanese users with no knowledge of English or legal English to generate patent claims in English from a Japanese-only interface. It exploits the highly determined structure of patent claims and merges Natural Language Generation (NLG) and Machine Translation (MT) techniques and resources as realized in the AutoPat and PC-Transfer applications. Due to its tuned MT engine, the approach can be seen as a human-aided machine-translation (HAMT) system circumventing major obstacles in full-scale Japanese-English MT. The approach is fully implemented on a large scale and will be commercially released in autumn 2005.
The translation of prepositions is often considered one of the more difficult tasks within the field of machine translation. We describe an experiment using transformation-based learning to induce rules to select the appropriate target language preposition from aligned bilingual data. Results show an F-Score of 84.9%, to be compared with a baseline of 75.5%, where the most frequent translation alternative is always chosen.
This paper is a case study of the deployment of new MT technology aiming at improving the overall Post-Editing (PE) experience. One of the main challenges in having MT output post-edited within a localization workflow is being able to meet ever increasing quality ex-pectations, especially from a terminology perspective. In Symantec"s localization process we constantly seek to overcome this obstacle by refining our MT output in a num-ber of ways. Several layers of customization have therefore been added over the years to our commercial MT system (such as User Dictionaries, Translation Stylesheets and Au-tomated Post-Editing). Despite obtaining sub-stantial quality gains with these techniques, improvements are still sought to minimize PE effort. The deployment of a novel technology, based on SYSTRAN's new hybrid approach, is presented in this paper. This paper focuses on the technical and linguistic challenges as-sociated with the integration of this new tech-nology into an existing MT workflow.
Web-based translation memory (TM) is a recent and little-studied development that is changing the way localisation projects
are conducted. This article looks at the technology that allows for the sharing of TM databases over the internet to find
out how it shapes the translator’s working environment. It shows that so-called pre-translation—until now the standard way
for clients to manage translation tasks with freelancers—is giving way to web-interactive translation. Thus, rather than interacting
with their own desktop databases as before, translators now interface with each other through server-based translation memories,
so that a newly entered term or segment can be retrieved moments later by another translator working at a remote site. The
study finds that, while the interests of most stakeholders in the localisation process are well served by this web-based arrangement,
it can involve drawbacks for freelancers. Once an added value, technical expertise becomes less of a determining factor in
employability, while translators lose autonomy through an inability to retain the linguistic assets they generate. Web-based
TM is, therefore, seen to risk disempowering and de-skilling freelancers, relegating them from valued localisation partners
to mere servants of the new technology.
Prepositions as functional words play an important role in the construction of phrases and sentences. They connect typically
a noun or a pronoun to another word and show their relationships in a sentence. They are a kind of “small” word and can not
stand alone in the sentences. However, they often pose great problems in the identification of their relationships to other
constituents. Prepositional phrase (PP) attachment is especially problematic in machine translation. Besides, as prepositions
by themselves are semantically very ambiguous, the translation of these words poses challenge to machine translation (MT)
too. This paper discusses how some of the PP attachments and other problems concerned with PPs are resolved in an experimental
English-Chinese MT system by applying controlled language technique that we have developed in the context of safety-critical
applications. Examples are extracted from a small parallel bilingual corpus focused on medical protocols in English and Chinese,
where the usages, in particular some of the syntactic structures, vary a lot in many aspects. It is suggested that such ambiguities
can be resolved by means of lexical information, such as subcategorizations, selection constraints and other additional information.
User feedback is considered to be a critical element in the information seeking process, especially in relation to relevance assessment. Current feedback techniques determine content relevance with respect to the cognitive and situational levels of interaction that occurs between the user and the retrieval system. However, apart from real-life problems and information objects, users interact with intentions, motivations and feelings, which can be seen as critical aspects of cognition and decision-making. The study presented in this paper serves as a starting point to the exploration of the role of emotions in the information seeking process. Results show that the latter not only interweave with different physiological, psychological and cognitive processes, but also form distinctive patterns, according to specific task, and according to specific user.
The title sentences of scientific and engineering papers are analyzed by simple parsing strategies, and only eighteen fundamental sentential structures are obtained from ten thousand titles. Title sentences of physics and mathematics of some databases in English are translated into Japanese with their keywords, author names, journal names and so on by using these fundamental structures. The translation accuracy for the specific areas of physics and mathematics from INSPEC database was about 93%.
A machine translation system of English to Japanese is described, which has been used in 24-hour direct satellite broadcasting by NHK to translate "World News."In order to treat a wide scope of news sentences, the system is provided with more than 100,000 lexical entries as well as about 3,000 grammatical rules which can robustly analyze various types of undefined words. It is also effective in translation of news sentences to preprocess proper nouns, to resolve structural ambiguities by weighting grammatical rules, and to select appropriate words using semantic markers. The operational experiments on machine translation in satellite broadcasting are briefly discussed.
In this paper, we present a method that improves Japanese dependency parsing by using large-scale statistical information. It takes into account two kinds of informa- tion not considered in previous statistical (machine learning based) parsing meth- ods: information about dependency rela- tions among the case elements of a verb, and information about co-occurrence re- lations between a verb and its case ele- ment. This information can be collected from the results of automatic dependency parsing of large-scale corpora. The results of an experiment in which our method was used to rerank the results obtained using an existing machine learning based parsing method showed that our method can im- prove the accuracy of the results obtained using the existing method.
This paper describes the latest version of speech-to-speech translation systems de- veloped by the team of NICT-ATR for over twenty years. The system is now ready to be deployed for the travel domain. A new noise-suppression technique notably im- proves speech recognition performance. Corpus-based approaches of recognition, translation, and synthesis enable coverage of a wide variety of topics and portability to other languages.
The adoption of Machine Translation technology for commercial applications is hampered by the lack of trust associated with machine-translated output. In this paper, we describe TrustRank, an MT system enhanced with a capability to rank the quality of translation outputs from good to bad. This enables the user to set a quality threshold, granting the user control over the quality of the translations. We quantify the gains we obtain in translation quality, and show that our solution works on a wide variety of domains and language pairs.
In human translation, translators first make draft translations and then modify and edit them. In the case of experienced translators, this process involves the use of wide-ranging expert knowledge, which has mostly remained implicit so far. Describing the difference between draft and final translations, therefore, should contribute to making this knowledge explicit. If we could clarify the expert knowledge of translators, hopefully in a computationally tractable way, we would be able to contribute to the automatic notification of awkward translations to assist inexperienced translators, improving the quality of MT output, etc. Against this backdrop, we have started constructing a corpus that indicates patterns of modification between draft and final translations made by human translators. This paper reports on our progress to date.