ArticlePDF Available

Abstract

Many problems in the translation industry revolve around customer dissatisfaction with someone’s translation quality and disputes between translators when one is revising the work of the other. To determine whether someone has attained translation quality, one must be able to measure it. To measure translation quality, one must be able to define it. And to define translation quality, one must be able to define both translation and quality. Our two prior articles have addressed these terms respectively. Those articles do not resolve disagreements about these terms; indeed, we authors have unresolved disagreements among ourselves. However, the articles outline how different definitions can radically affect people’s expectations about translation and quality. This third article builds on the first two and presents two contrasting definitions of translation quality, a narrow one and a broad one. The narrow definition matches with the narrow definition of translation in the first article and an emphasis on the transcendent view of quality in the second article, while the broad definition matches with the broad definition of translation the first article and an integration of the five approaches in quality management from the second article. The authors strongly disagree about which definition of translation quality is better for the translation industry. Rather than pretending to be the end of this discussion, it hopes to encourage a continuation of the debate that has taken place among the authors.
413-420
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
Defining Translation Quality1
Geoffrey S. Koby (KSU)
Paul Fields (LinguaTech)
Daryl Hague (BYU)
Arle Lommel (independent scholar)
Alan Melby (LTAC)
ABSTRACT
Many problems in the translation industry revolve around customer dissatisfaction with
someone’s translation quality and disputes between translators when one is revising the
work of the other. To determine whether someone has attained translation quality, one
must be able to measure it. To measure translation quality, one must be able to define it.
And to define translation quality, one must be able to define both translation and quality.
Our two prior articles have addressed these terms respectively. Those articles do not re-
solve disagreements about these terms; indeed, we authors have unresolved disagree-
ments among ourselves. However, the articles outline how different definitions can radical-
ly affect people’s expectations about translation and quality. This third article builds on the
first two and presents two contrasting definitions of translation quality: a narrow one and a
broad one. The narrow definition matches with the narrow definition of translation in the
first article and an emphasis on the transcendent view of quality in the second article,
while the broad definition matches with the broad definition of translation in the first article
and an integration of the five approaches in quality management from the second article.
The authors strongly disagree about which definition of translation quality is better for the
translation industry. Rather than pretending these articles are the end of this discussion,
the authors hope to encourage a broad-based continuation of the debate that has taken
place among the authors.
Keywords: Translation quality, Specifications, Metrics, Error categories, Broad, Narrow
RESUM (Tot definint qualitat en traducció)
Molts dels problemes de la indústria de la traducció giren al voltant de la insatisfacció del
client amb la qualitat d'alguna traducció, així com amb conflictes entre traductors quan un
revisa el treball d'un altre. Per determinar si algú ha arribat a la qualitat de traducció
pactada, un ha de poder mesurar-la. Per poder mesurar la qualitat de la traducció, un ha
de poder definir-la. I per definir la qualitat de la traducció, un ha de poder definir dos
conceptes, traducció i qualitat. Els nostres dos articles anteriors han abordat aquests dos
conceptes respectivament. Aquests articles no resolen les discrepàncies entre nosaltres.
No obstant això, els articles exposen com les diferents definicions poden afectar
radicalment les expectatives sobre traducció i qualitat. Aquest tercer article es basa en els
dos articles anteriors i presenta dues definicions confrontades de qualitat en traducció:
una més concreta i una altra més àmplia. La definició més concreta combina la definició
1 This article is the first of trilogy of papers published in this issue: Melby et al., Fields et al., and Koby et al.
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
414
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
més estricta de traducció del primer article amb la visió transcendent de qualitat del segon
article. En canvi, la definició més àmplia combina la definició àmplia de traducció del pri-
mer article amb una integració dels cinc enfocaments sobre gestió de la qualitat del segon
article. Els autors estan totalment en desacord pel que fa quina de les definicions de
qualitat de traducció és millor per a la indústria de la traducció. En lloc d'optar per tancar
aquest debat amb aquests articles, els autors esperen haver donat peu a continuar amb el
debat d'una manera més amplia de com s'ha produït entre ells.
Paraules clau: qualitat en traducció, especificacions, mètricas, categories d’error, narrow,
broad.
RESUMEN (La definición de calidad en traducción)
Muchos de los problemas de la industria de la traducción giran en torno a la insatisfacción
del cliente con la calidad de alguna traducción, así como con conflictos entre traductores
cuando uno revisa el trabajo de otro. Para determinar si alguien ha alcanzado la calidad
de traducción pactada, uno debe poder medirla. Para poder medir la calidad de la
traducción, uno debe poder definirla. Y para definir la calidad de la traducción, uno debe
poder definir ambos conceptos, traducción y calidad. Nuestros dos artículos anteriores
han abordado estos dos conceptos respectivamente. Dichos artículos no resuelven las
discrepancias entre nosotros. No obstante, los artículos exponen cómo las diferentes
definiciones pueden afectar radicalmente las expectativas sobre traducción y calidad. Este
tercer artículo se basa en los dos artículos anteriores y presenta dos definiciones
confrontadas de calidad en traducción: una más concreta y otra más amplia. La definición
más concreta combina la definición más estricta de traducción del primer artículo con la
visión transcendente de calidad del segundo artículo. En cambio, la definición más amplia
combina la definición amplia de traducción del primer artículo con una integración de los
cinco enfoques sobre gestión de la calidad del segundo artículo. Los autores están
totalmente en desacuerdo en cuanto cuál de las definiciones de calidad de traducción es
mejor para la industria de la traducción. En lugar de optar por zanjar este debate con
estos artículos, los autores esperan haber dado pie a continuar con el debate de un modo
más amplio de como se ha producido entre ellos.
Palabras clave: calidad en traducción, especificaciones, métricas, categorías de error,
narrow, broad.
Introduction
With respect to translation, our first article reflects our consensus that definitions can be
placed within a landscape” defined by two axes: scope (narrow-to-broad) and specifications
(absolute-to-relative). The “scope” axis asks how narrowly or broadly one defines the term
translation. The answer to that question will determine whether a given activity, such as local-
ization or transcreation, falls within translation’s purview. The “specifications” axis asks
whether all of a requester’s requirements must be explicitly stated. An “absolute” stance as-
serts that some requirements are always understood and absolute and constant and can
therefore remain unstated, while a “relative” stance asserts that in general, the best practice is
to explicitly state all requirements as specifications, because they can vary from project to
project. For both ends of the specifications axis, some degree of accuracy and fluency is re-
quired. The difference is whether the minimum levels depend on audience or purpose or both.
In our second article, we present multiple approaches to the question of how one defines
quality. Based on Garvin (1984), we describe five approaches to quality (transcendent, prod-
uct, user, production, and value), and we briefly present the contributions of several people
who have shaped the discipline of quality management, according to which quality is what the
customer perceives relative to expectations combining all five approaches. The second article
concludes by addressing the issue of quality management’s relevance to translation. The
authors disagree, for example, on whether raw or lightly post-edited machine translation that
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
415
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
meets the specifications for the project (that could probably not have been met using profes-
sional human translation) can be called a quality translation, even though it accomplishes the
stated purpose and meets the expectations of the audience. This disagreement is related to
two uses of the term “quality,” one in the discipline of quality management and the other in
general usage, which tends toward a transcendent perspective. More fundamentally, the au-
thors disagree sharply on whether the principles of quality management should be adopted by
the translation industry.
In the present article, we build on the prior two articles to address translation quality and
the ramifications of how people define that term. In doing so, we first list some points on
which we generally agree. We then present two contrasting definitions of translation quality
narrow and broadfollowed by a point-by-point analysis demonstrating how people support-
ing these opposing definitions evaluate significant translation-quality issues. After this analy-
sis, we discuss the question of whether specifications need to be explicit and mention a few
possible areas of research. We conclude with the most basic point of agreement among the
authors: that they agree to strongly disagree with each other without walking away from the
debate and to encourage a wider participation in this important debate about the nature of
translation quality, with an eye to developing useful translation quality metrics.
Some Points of Agreement with Respect to Translation Quality
While we have disagreements among ourselves with respect to defining translation quality,
we do agree on several issues. The first concerns how the language industry defines “cus-
tomers.” This term includes both the party requesting the translation work and any parties
who utilize the work productthe end-users. Thus, one can distinguish between these two
types of customers as the requester and the end-user. The person or organization producing
a translation for the customer(s) is the provider. Thus, we have agreed to use the terms re-
quester, provider, and end-user for clarity.
Our second point of agreement concerns any product that can be understood to qualify as
a translation. All translations carry an expectation of some degree of accuracy and fluency.
Accuracy is a bilingual notion referring to the correspondence between the source and target
text, while fluency is a monolingual notion referring to properties of the target text such as
grammar, spelling, and cohesion.
Our third point of agreement pertains to the possibility of perfect” accuracy and fluency.
These two characteristics are always in tension because languages are not mechanical re-
encodings of each other. Some slippage is inevitable, especially when sources move beyond
banal propositional content (“the cat sat on the mat”) to anything where cultural expectations
are an important factor. Perfection is therefore not a reasonable goal. If one rejects the notion
of perfect accuracy and fluency, one can still propose a goal of “maximal” fluency and accur a-
cy. This goal may seem fairly clear in isolation, but there are cases where a less-than-elegant
target solution is best because a requester prioritizes fidelity to the source formulation (e.g.,
foreignized translation). Similarly, where the source text is defective in some fashion (e.g., it
was produced by a second-language learner), a tension is created between accuracy and
fluency. Should some infelicities in the source text be reflected in the target text, or should the
target text improve on the source text and hide its flaws?
Our fourth point of agreement pertains to providers’ responsibility to requesters and end-
users (the audience). Providers should endeavor to understand requesters’ purpose and take
audience and purpose into account during the translation process. This emphasis on respect-
ing a translation’s purpose has a long tradition in translation studies, beginning with Skopos
theory (associated most closely with Hans Vermeer) and continuing with Functionalism (as-
sociated most closely with Christiane Nord).
Our last point of agreement concerns our conviction that both the language industry and
translation studies urgently need a method to measure translation quality as objectively as
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
416
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
possible. That method should emphasize identifying problems that can be corrected. Any
effort to measure translation quality is doomed to confusion without an explicit definition of
translation quality.
Two Definitions of Translation Quality: Broad and Narrow
Given our own lack of consensus on defining translation in the first article and defining
quality in the second article, readers should not be surprised that we have not reached con-
sensus on a definition of translation quality. Below we present two contrasting definitions of
translation quality, along with arguments in favor of one or the other. We label these defini-
tions “broad” and “narrow.”
A Broad Definition of Translation Quality
Concerning the “scope” axis described above, a broad definition of translation categorizes
many activities as “translation.” These include full translation (that is, segment by segment),
summary translation, localization, transcreation, and even gisting (typically a rough translation
done by a human or a machine translation system to help people decide whether a profes-
sional translation is needed). As to the “specifications” axis, this broad view asserts that no
absolute specifications apply to all translation projects. Specifications should therefore be
made explicit whenever feasible, relative to the audience and purpose. This broad view sup-
ports the following definition of translation quality:
“A quality translation demonstrates accuracy and fluency required for the audience and
purpose and complies with all other specifications negotiated between the requester and
provider, taking into account end-user needs.
This broad definition fits with approaches to quality described in the second article. First,
because the definition demands that all translations share innate characteristicsnamely,
certain degrees of accuracy and fluency—it acknowledges a “transcendent” approach to qual-
ity, as described by Garvin and the second article in this series. Second, because accuracy
and fluency are characteristics that can be measured, this broad definition fits Garvin’s prod-
uct-based approach as well. Third, because accuracy and fluency must reflect levels “re-
quired” by audience and purpose, the definition matches Juran’s notion that a quality product
must fit its purpose. Fourth, the definition includes compliance with specifications, an empha-
sis found in both Crosby and Garvin (the “production approach”, which Garvin calls the man u-
facturing approach). Fifth, the definition acknowledges the end-user, recalling Garvin’s user-
based approach and Feigenbaum’s emphasis.
Although the definition does not explicitly address Garvin's value-based perspective, it
acknowledges the need for negotiation between requester and provider on all aspects of the
perceived value of the translation service, including target content requirements, delivery
date, and cost. These aspects are part of structured translation specifications, as described in
the appendix to the first article.
A Narrow Definition of Translation Quality
With respect to the scope axis described above, a narrow definition of translation views
translation as text-centric. Other activities common to the language industry, such as summa-
rization, transcreation, gisting, and certain aspects of localization, do not qualify as “transla-
tion” under this view. In addition, when machine translation is involved, this view requires a
sufficient level of post-editing to ensure minimal levels of accuracy and fluency. Concerning
the specifications axis, a narrow view posits that many requirements are absolute, i.e., subject
to invariable minimum levels, although there are many variables that must be specified (i.e.,
the traditional translation brief). The foregoing assumptions support a narrow definition of
translation quality as follows:
“A high-quality translation is one in which the message embodied in the source text is
transferred completely into the target text, including denotation, connotation, nuance, and
style, and the target text is written in the target language using correct grammar and word
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
417
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
order, to produce a culturally appropriate text that, in most cases, reads as if originally writ-
ten by a native speaker of the target language for readers in the target culture.”
This narrow definition, from a quality management perspective, could be viewed as a spe-
cial case of the broad definition, with various specifications predetermined. Projects outside
the scope of the narrow definition of translation are likewise outside the scope of the narrow
definition of translation quality. Thus, these two definitions of translation quality differ in their
range of practical applications. From the perspective of a narrow view of the definition of
translation, the broad definition of translation quality is too broad and should be viewed as
covering translation quality management rather than just translation quality. The choice of one
definition of translation quality over the other will influence the framework for developing
translation quality metrics and have practical consequences for measuring translation quality
(TQ). The rest of this article presents additional TQ issues in a debate format outlining how a
broad or narrow definition shapes one’s answer.
Some Additional Points of Debate
Should specifications always be explicitly stated?
Both the translation industry and translation studies urgently need a way to compare dif-
ferent sorts of translation as objectively as possible, with an emphasis on identifying prob-
lems. The ultimate goal in all cases is improving accuracy (adequate transfer of meaning) and
fluency (correct target-language usage). However, some hold that this goal must be achieved
within the context of agreed specifications, while others hold that extensive specifications
going beyond the traditional translation brief may not be necessary in all cases.
The broad TQ view implies that providers should work with the requester to understand
the requester’s requirements and the end-users’ needs. Furthermore, those requirements and
needs should be stated as formal specifications. Creating specifications is a discovery pro-
cess that brings unstated requirements into focus. As such, there is no such thing as a “zero-
requirements” work product.
In contrast, the narrow TQ view implies that explicit specifications are not always needed.
For example, requesters often do not know what the specifications are or should be. Further-
more, from a narrow view of TQ, most translation assignments fall within the “normal” stand-
ards of professional human translation, for which explicit specifications going beyond a gen-
eral description of the text type and possibly the target audience (i.e., legal topic, into US
English) are unnecessary, as the text types are typically translated into largely equivalent text
types in the target language. Finally, providers who seek to establish explicit specifications
about every issue can irritate the requester without improving the resulting translation.
Do providers have an ethical responsibility to meet minimum service levels regardless of
what the requester asks?
Many people affirm that often neither the end-user nor the requester knows what a particu-
lar project actually requires. Under such circumstances, providers have an ethical mandate to
deliver a translation product that meets certain minimum standards that are absolute, that is,
independent of the type of the audience and purpose. A narrow TQ view would support the
existence of such an ethical mandate, while a broad TQ view would not. Instead, the broad
view would encourage providers to negotiate requirementsperhaps even a set of default
requirementswith the requester.
Monolingual requesters may initially focus on price and be willing to accept a “rough”
translation. Unfortunately, these requesters frequently misunderstand just how rough a raw or
lightly post-edited machine translation can be. Furthermore, they may not anticipate the com-
prehension challenges monolingual end-users face. In such cases, requesters often complain
about the quality received, even though the translation complies with specifications. Given
this situation, some people argue that even if a requester asks for something that does not
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
418
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
meet minimum standards, the translator still has an ethical responsibility to meet that mini-
mum. Furthermore, others may argue that for most situations, providers would be wrong to
knowingly submit something that does not meet a minimal level of accuracy and fluency. A
contrasting view is that translators who deliver a higher level of accuracy and fluency than
identified in the specifications are ethically obligated to inform the requester, especially if this
variation involves additional cost to the requester.
Regardless of whether a requester approves a set of specifications where “style” is not
considered important, requesters frequently change their minds when the final translation is
delivered. In a world where the customer is always right, referring back to these specifications
is cold comfort for the provider. From a practical perspective, therefore, providers have a
strong incentive to make the translation as fluent and accurate as possible even if the re-
quester claims not to care. In some cases, however, requesters may willingly accept low lev-
els of accuracy and fluency if the provider first sends some sample pages and provides other
clear communication to confirm shared expectations.
Is there a particular set of error categories that should be included in every TQ metric?
Both the narrow and broad views agree that people can use multiple error categories, in-
cluding the broad categories of accuracy and fluency, and specific error categories within
broader error categories, to create a TQ metric. Disagreement sets in when one attempts to
list the error categories that are absolutely necessary. A narrow view of TQ holds that any
translation quality metric must necessarily include at least the following error categories: (1)
meaning transfer; (2) terminology; (3) domain-specific writing quality; (4) domain-independent
target-language accuracy. In contrast, a broad view would simply argue that some situa-
tionse.g., time-sensitive intelligence workmight not require all of these error categories.
The parties can therefore negotiate these error categories as they desire.
Can stakeholders establish minimal acceptable service levels for general situations?
Perhaps one way to address quality concerns that would satisfy both broad and narrow
TQ views would be defining explicit minimal acceptable service levels for general situations
and making these levels known to both requesters and providers. Parties could then negotiate
higher thresholds (e.g., for translating a nuclear disarmament treaty) or lower ones. But if
lower thresholds are defined, the parties should do so consciously, and the request should be
motivated. A help-desk translator’s job brief, for example (see second bullet point below for
details), could read as follows: “We know that you are normally expected to meet certain re-
quirements for fluency, but we want you to disregard them and instead focus only on making
sure that certain keywords and relationships are represented accurately so that the database
can locate the appropriate answers.” This brief could then acknowledge that minimal profes-
sional standards exist, but that there is a good reason to break them in this case. So perhaps
quality evaluation standards could provide guidance for generally accepted quality levels and
explicitly state that deviations must be agreed upon by all parties in advance.
How does one measure TQ when informed requesters deliberately accept less-than-
maximal accuracy and fluency?
One can imagine a translation request in which specifications do not demand the transla-
tor’s best and instead require substantial deviations from an ideal situation. Examples are
relatively easy to find in time-sensitive materials, such as the following:
An intelligence analyst is producing translations of intelligence intercepts. The infor-
mation is highly time-critical, and if the analyst takes time to revise and fix non-critical
problems, the translation will quickly lose value. So, the specifications might actually
state that any spelling, grammar problems, etc., that do not compromise the ability of
the intended audience (e.g., military leaders) to make their decisions should NOT be
fixed.
A second example is perhaps more to the point in the commercial world:
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
419
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
MT is used to try to match user queries to an existing database of support items. If MT
is not able to identify a valid match, then (and only then) human translators are looped
in. The human translator (known as an “operator”) then has to decide whether to try to
fix the MT output or retranslate it from scratch. If the operator chooses to retranslate,
the retranslation’s ONLY short-run value is in identifying relevant items in the data-
base. From a business perspective, any time spent on improving accuracy or fluency
beyond what is needed to identify the relevant database content is wasted. Because
this translation will never be used for an outside client, the text’s appearance does not
matter at all. Therefore, the operator should NOT clean it up or polish it in any way.
From the industry client’s perspective, the operator represents a rather high-value position
for a translator. By employing the operator, providers can significantly expand client support.
Providers simply could not afford to do so with purely human translation, nor could they pro-
vide it at all with pure MT. The operator’s job is not a task for dummies, but requires great
skill. Specifically, the operator provides an absolutely vital service that requires very fast turn-
around and considerable understanding of products and services--and the MT system in-
volved. But the translation output would not meet normal industry standards and in fact
SHOULD NOT do so. Meeting normal standards for accuracy and fluency adds no value and
in fact subtracts value by slowing the human operator down.
Proponents of a broad definition of translation quality point out that if stakeholders want to
define quality in a way that reflects the scenarios in which translators may work (now and in
the future), those scenarios must include varied requirements. Proponents of a broad defini-
tion therefore find the narrow definition of translation quality too rigid and not sufficiently com-
prehensive, since it is associated with a narrow definition of translation that categorizes sum-
mary translation, localization, and light post-editing, among other language industry activities,
as secondary or additional activities that are separate from translation per se.
Proponents of the narrow definition of translation quality point out that a narrow definition
is consistent with the intuition of many professional translators and reflects the way they think
and work. A few years ago, there was a serious debate at the NATO translation office in
Brussels. One of the authors was there. After several hours of discussion a veteran translator
rejected a broad definition of translation quality very similar to the one given in this article, on
the basis that a transcendent view of quality is the only possible view for a professional hu-
man translator. She and other proponents of the narrow definition of translation quality take
the position that the broad definition of translation includes aspects that are separate from
translation and can be performed by monolinguals.
Conclusion
Throughout this three-part series, we have addressed the challenges of defining transla-
tion, quality, and translation quality. We have found that “translation” can be located within a
landscape defined by axes of scope and specifications. The authors have not reached con-
sensus on whether a broad or narrow definition of translation is better for the entire range of
stakeholders in the translation industry: requesters, providers, and end-users. Their only point
of agreement in the first article is that it is it important for any two people talking about transla-
tion quality to first identify where their respective definitions are “planted” in the landscape we
describe, in order to avoid talking past each other.
Concerning the applicability of the discipline of quality management to the translation in-
dustry, the authors also failed to reach consensus. Their only points of agreement are (a) that
it matters deeply whether the translation industry adopts or rejects all of quality management
or emphasizes a transcendent view and (b) that a decision between emphasizing a quality
management perspective and a transcendent perspective cannot be made independent of a
choice regarding the definition of translation, including a discussion of whether there are ab-
solute specifications that are independent of audience and purpose. In the present article, two
DEFINING TRANSLATION QUALITY
Geoffrey S. Koby, Paul Fields, Daryl Hague, Arle Lommel, Alan Melby
420
Número 12, Traducció i qualitat
Revista Tradumàtica: tecnologies de la traducció . desembre 2014 . ISSN: 1578-7559
http://revistes.uab.cat/tradumatica
Els continguts de la revista estan subjectes a una llicència Creative Commons (CC BY 3.0)
contrasting definitions of translation quality have been presented. Proponents of the broad
definition see the narrow definition as a special case of the broad definition. Proponents on
the narrow definition are not ready to accept this claim.
Concerning future research related to translation quality, statistical analyses are needed to
show the breakdown of the various types of translation and peri-translation activities.2 Addi-
tional research is needed on how the language industry (which itself must be carefully de-
fined) perceives traditional text translation, localization, and other forms of adaptation and
how the industry is evolving. How much of the language industry falls under the narrow defini-
tion of translation and how is market segmentation changing? Which of the 21 translation
parameters in the appendix to the first article are considered part of translation quality, as a
product and as a service, to which stakeholders? And how does relevance of parameters
relate to one’s choice of a broad or narrow definition of translation and translation quality?
Regardless of research on how people in the translation industry currently perceive trans-
lation quality, the authors agree that progress on systems to develop and apply translation
quality metrics will be hindered unless everyone working on them lays out three definitions:
Definition of translation (narrow, broad)
Definition of quality (relative to Garvin’s five approaches, embracing all or some)
Definition of translation quality (narrow, broad)
The authors are not indifferent about which definitions are used. Some of us strongly be-
lieve that adopting the broad definition of translation quality and all that goes with it in quality
management is very important to progress in the translation industry, in particular unifying the
industry. However, others of us strongly believe that narrow definitions are sufficient.
Two things are certain: (1) translation quality metrics must be built on a well-defined foun-
dation including at least clearly stated definitions of translation, quality, and translation quality;
and (2) the important debate about which kind of definition to use, broad or narrow, is not
over.
The authors fervently hope that the trio of articles concluding with this one will promote fur-
ther meaningful and beneficial debate among language industry professionals.
References
Crosby, Philip B. Quality Is Free: The Art of Making Quality Certain. New York: McGraw-Hill,
1979.
Deming, W. Edwards. Out of the Crisis. Cambridge, MA: MIT Press, 1986.
Feigenbaum, Armand V. Total Quality Control. New York: McGraw-Hill, 1983.
Garvin, David A. “What Does Product Quality Really Mean?” Sloan Management Review 26.1
(1984): 25-43.
Juran, Joseph M., and Joseph A. DeFeo. Juran’s Quality Handbook: The Complete Guide to
Performance Excellence. 6th ed. New York: McGraw-Hill, 2010.
Nord, Christiane. Translating as a Purposeful Activity: Functionalist Approaches Explained.
Manchester: St. Jerome, 1997.
Vermeer, Hans. “Skopos and Commission in Translatorial Action.” The Translation Studies
Reader. Ed. Lawrence Venuti. 2nd ed. New York: Routledge, 2004. 227-238.
2 One problem in this case is a lack of reliable statistics on the translation industry. It has been argued (by
Durban, for instance) that much of the high-end or “premium” market work is performed by boutique translation
agencies or individual translators who do not report statistics.
... Although translation quality is of utmost concern in human translation (ISO, 2015;Koby et al., 2014;Mellinger, 2018), the eye movements of readers have not been treated as a measure of quality and studied in the way they were in machine translation (MT) research (Colman et al., 2022;Doherty et al., 2010;Kasperavičienė et al., 2020;Kasperė et al., 2023;Stymne et al., 2012). The effect of errors on the readability and comprehensibility of human translation (HT) remains under-researched (Kruger & Kruger, 2017). ...
Article
Texts are translated to be read and provide access to otherwise inaccessible information or experiences. Scant empirical interest in how translations are read and received by readers is surprising in the context of our knowledge about the features of translations, and the systematic ways in which they differ from originally written texts. In this paper, we explore the impact of translation quality on the reading experience by analysing the cognitive effort involved in reading and text comprehension. Two groups of participants (n = 64) were eye-tracked as they read either a low-quality translation (with errors) or a high-quality translation (without errors) of the same source text. Overall, the errors contributed to longer dwell time when reading the entire text but did not significantly affect the participants’ comprehension scores. A more in-depth analysis of the impact of translation errors on the reading experience shows that it depends on the amount of confusion errors cause to the reader when building a coherent model of the entire text.
... However, since MT is application-oriented in nature, calculability and measurability should be at the core of its quality assessment. One widely adopted view of MT quality is proposed by Koby et al. (2014), which regards accuracy and fluency as the two determinants of translation quality. Alignment with human translation is also stressed, and Hassan et al. (2018) argue that only when MT shows no significant difference from human translations as measured by some scores can it be counted as high-quality translation. ...
Preprint
Full-text available
Large language models have demonstrated parallel and even superior translation performance compared to neural machine translation (NMT) systems. However, existing comparative studies between them mainly rely on automated metrics, raising questions into the feasibility of these metrics and their alignment with human judgment. The present study investigates the convergences and divergences between automated metrics and human evaluation in assessing the quality of machine translation from ChatGPT and three NMT systems. To perform automatic assessment, four automated metrics are employed, while human evaluation incorporates the DQF-MQM error typology and six rubrics. Notably, automatic assessment and human evaluation converge in measuring formal fidelity (e.g., error rates), but diverge when evaluating semantic and pragmatic fidelity, with automated metrics failing to capture the improvement of ChatGPT's translation brought by prompt engineering. These results underscore the indispensable role of human judgment in evaluating the performance of advanced translation tools at the current stage.
... The results of the evaluation revealed that the MT system produced several translation errors, covering different linguistic aspects including lexis, syntax, semantics, pragmatics, orthography, and stylistics, which hindered the process of transferring the accurate meaning of the source texts in fluent well-structured translations which definitely go against the translation specifications, Table 1 and 2, that were set by the researchers before the evaluation. These results contradict the concept of translation quality as defined by Koby et al. (2014), who defined translation quality as reproducing accurate and fluent translation results for the target audience that can serve the original purpose and comply with all other specifications negotiated between the requester and provider, while considering the needs of the end-users. Consequently, Instagram's NMT system is not capable of producing translations that are well-structured, properly convey the intended message, and preserve the aesthetic value of the literary texts. ...
Article
Full-text available
Addressing the global increase in social media users, platforms such as Instagram introduced automatic translation to broaden information dissemination and improve cross-cultural communication. Yet, the accuracy of these platforms' machine translation systems is still a concern. Therefore, this paper aims to explore the potential of Neural Machine Translation utilized by Instagram in producing high-quality translations. In doing so, this study attempts to scrutinize the reliability of Instagram's "See Translation" feature in the translation of literary texts from Arabic to English. A selection of auto-translated Instagram captions is analyzed through the identification, classification, and assignment of error types and penalty points, utilizing the MQM core typology. Subsequently, the Overall Quality Score of the error-based analysis is calculated automatically using the ContentQuo platform. Furthermore, the study investigates whether Instagram Neural Machine Translation can effectively convey the intended message within literary texts. From 30 purposively selected Instagram captions with literary content, the study found Instagram's machine translation lacking in 90% of cases, particularly in accuracy, fluency, and style. Among these, 61 errors were identified: 26 in fluency, 25 in accuracy, and 10 in style, adversely affecting the quality and failing to convey the original message. The findings suggest a need for enhanced algorithms and linguistic architecture in Neural Machine Translation systems to better recognize linguistic variants and text genres for more accurate and fluent translations.
Article
Full-text available
Translation quality assessment has always attracted a great deal of scholarly attention, which has resulted in several translation assessment tools/rubrics. Yet very few were observed to incorporate translation solutions as evaluative parameters and, to the author’s best knowledge, none was identified to assess the quality of translations into Turkish. These two features help the tool presented herein stand out and make a substantial contribution to the related literature. The tool was originally available in the author’s doctoral dissertation (Yildiz, 2016), but an improved version was proposed in this paper. It was built on translation errors and translation solutions. To be able to judge a solution’s acceptability, 25 rich points (PACTE, 2009) were identified in the excerpted manual. The rich points are located in the first part of the tool, while the second portion was solely based on the erroneous translation segments. The tool incorporates two types of errors – i.e., mechanical and transfer errors. The paper also proposes a grading table, featuring solution- and error-based grades in exponential increments. The minor, major, and critical errors are penalized with (-2), (-4), and (-8) points, whereas partially acceptable and acceptable solutions are awarded (+2) and (+4) points, respectively. The grading table is accompanied by a rubric to describe how the degrees of errors and solutions can be operationalized, which is believed to promote objectivity and inter-rater reliability. The tool notably contributes to the literature with these two components as well. It is thought to be usable by translation schools, translators associations, and translation companies.
Thesis
Full-text available
Recent language technology developments have disrupted the translation and interpreting professions. However, the focus has been on using more computational power and training larger language models, often neglecting the users of such technology (do Carmo and Moorkens 2022). To date, the goal of technology development has been the creation of an intelligent agent that emulates human behaviour to increase automation. As a response, a novel technology design framework has gained a foothold recently: human-centered artificial intelligence, where instead of human replacement, the aim is to produce a powerful tool that augments human capabilities, enhances performance, and empowers users, who are at all instances in supervisory control of such systems (Shneiderman 2022). If applied to machine translation (MT), we can talk about human-centered, augmented MT (HCAMT). This shift, moving from emulation to empowerment, places humans at the centre of AI/language technology. This PhD thesis presents the concept of Machine Translation User Experience (MTUX) as a way to foster HCAMT. Consequently, we conduct a longitudinal user study with 11 professional translators in the English-Spanish language combination that analyses the effects of traditional post-editing (TPE) and interactive post-editing (IPE) on MTUX, translation quality and productivity. MTUX results suggest that translators prefer IPE to TPE because they are in control of the interaction in this new form of translator-computer interaction and feel more empowered in their interaction with MT. Productivity results also suggest that translators working with IPE report a statistically significantly higher productivity than when working with TPE. Quality results also indicate that translators offer more fluent translations in IPE, and equally adequate translations in both post-editing modalities. All these results allow for reflection on the potential adoption of IPE as a more HCAMT post-editing modality, which empowers the users, who have been increasingly reluctant to interact with machine translation post-editing in industry workflows (Cadwell, O’Brien, and Teixeira 2018). This PhD thesis establishes the methodology for fostering HCAMT tools, systems and workflows through the study of MTUX. The successful implementation of HCAMT in translation and interpreting may lead to sustainable, diverse, and ethically sound development in MT systems and other technological tools through a wide variety of users and use-cases.
Article
Full-text available
La traducción automática neuronal (TAN) ha mejorado de manera notable la calidad de los textos traducidos y en la actualidad es una herramienta más a disposición del traductor. A pesar de ello, el lenguaje natural tiene algunas particularidades que presentan un desafío para este tipo de motores. Tal es el caso de los culturemas (Nord, 1997, p. 34) debido a su estrecha vinculación con la cultura y la idiosincrasia de cada lengua. Constituyen un excelente ejemplo del uso de culturemas los textos gastronómicos. En este contexto, el presente estudio de caso persigue evaluar en qué medida el uso de las herramientas de TAN (DeepL y Google Translate) puede servir de ayuda para traducir los culturemas propios de esta tipología textual. Se pretende asimismo determinar si existe un tipo de culturema más problemático para la TAN y establecer posibles patrones de acuerdo con las propuestas de traducción ofrecidas. Para ello, trabajamos con un corpus de culturemas gastronómicos españoles extraídos del portal oficial de turismo de España y analizamos las traducciones al alemán y al francés que ofrecen los citados motores de TAN. El estudio detecta que ciertos parámetros de calidad actuales de la TAN, considerados errores, no lo son cuando se traducen culturemas. Los que presentan mayor dificultad para la TAN son los que contienen palabras diatópicas o construcciones idiomáticas. Por último, el análisis del francés y del alemán ha permitido vislumbrar similitudes en relación con el comportamiento de la máquina en ambos idiomas. Los resultados apuntan, pues, a que todavía es imprescindible la intervención humana para traducir y evaluar la mayor parte de culturemas gastronómicos de manera adecuada. De ahí, la necesidad de trabajar en nuevas métricas que permitan adaptarse a cada encargo.
Book
Full-text available
Localization is everywhere in our digital world, from apps to websites or games. Our interconnected digital world functions in part thanks to invisible localization processes that allow global users to engage with all sorts of digital content and products. This book presents a comprehensive overview of the main theoretical, practical, and methodological issues related to localization, the technological, textual, communicative, and cognitive process by which interactive digital texts are prepared to be used in contexts other than those of production. Localization in Translation provides an interdisciplinary introduction to the main practical and theoretical issues involved in localizing software, web, video games, and apps. It discusses the many technological, cultural, linguistic, quality, economic, accessibility, and user-reception issues related to the different localization types. It also provides an updated overview of localization in an ever-changing technological landscape marked by advances in neural machine translation and AI. Each chapter includes a basic summary, key questions, a final section with discussion and assignments, as well as additional readings. Online resources with additional questions and assignments are included on the Routledge Translation Studies portal. This is the essential textbook for advanced undergraduates and graduates in translation studies and translation professionals engaged in localization practice.
Book
This bestselling text is a comprehensive overview of functionalist approaches to translation in English. Christiane Nord, one of the leading figures in translation studies, explains the complexities of theories and terms in simple language with numerous examples. Covering how the theories developed, illustrations of the main ideas, and specific applications to translator training, literary translation, interpreting and ethics, Translating as a Purposeful Activity concludes with a concise review of both criticisms and perspectives for the future. Now with a Foreword by Georges Bastin and a new chapter covering the recent developments and elaborations of the theory, this is an essential text for students of translation studies and for translator training.
Article
Product quality is rapidly becoming an important competitive issue.
Article
Article
In this article, the author reviews and synthesizes the varying definitions of product quality arising from philosophy, economics, marketing, and operations management. He then goes on to build an eight-dimensional framework to elaborate on these definitions. Using this framework, he addresses the empirical relationships between quality and variables such as price, advertising, market share, cost, and profitability.
Article
El cambio de estilo administrativo y el tipo de relaciones de la industria con el gobierno son los temas centrales de esta obra. Deming ofrece una teoria de administración basada en 14 puntos básicos en el quehacer administrativo entre los que se encuentran : crear constancia en el propósito de mejorar el produto y el servicio, adoptar e implantar el liderazgo, eliminar las barreras que privan a las personas de sentirse orgullosas de su trabajo. Eliminar la calificación anual o el sistema de méritos entre otros.
Juran's Quality Handbook: The Complete Guide to Performance Excellence. 6 th ed
  • Joseph M Juran
  • Joseph A Defeo
Juran, Joseph M., and Joseph A. DeFeo. Juran's Quality Handbook: The Complete Guide to Performance Excellence. 6 th ed. New York: McGraw-Hill, 2010.