![](/images/graphics-bg.png)
Post-édition de ta neuronale à la DGT et qualité des textes finaux: étude de cas
Other Title(s)
Neural machine translation post-editing in DGT and final text quality: a case study
Author
Source
Issue
Vol. 21, Issue 1 (31 Aug. 2022), pp.77-98, 22 p.
Publisher
University of Oran 2 Mohamed Ben Ahmad
Publication Date
2022-06-30
Country of Publication
Algeria
No. of Pages
22
Main Subjects
Languages
English Language and Literature
Topics
Abstract EN
This article aims at presenting the results of a case study carried out in collaboration with the European commission’s directorate general for translation.
this study analyses the quality of contents post-edited from neural machine translation (NMT) proposals (etranslation NMT engine) by translators with varied translation experience levels.
two types of participants were recruited: “blue book” interns (i.e.
recently graduated translators taking part in a 5-month paid internship in DGT) and in-house translators.
in order to proceed with this analysis, we used an evaluation grid created by French researchers toudic et al.
(2014), and containing nine error categories, as well as four types of effects which guide raters when they attribute severity penalties to errors.
the reliability of this tool was verified by an interrater agreement score: 583 revision marks were compared in terms of 1) severity penalty, 2) category and 3) raw MT responsibility by two investigators.
as far as methodology is concerned, for each source text, a NMT proposal from the etranslation engine was post-edited by a DGT translator (10 participants; 7 in-house translators and 3“blue book” interns) and revised by a DGT colleague.
this procedure follows the typical DGT workflow: texts are usually first translated by a translator, then systematically revised by a colleague from the same (or sometimes, a different) translation unit.
the evaluation of PE text quality was thus carried out through the revision marks introduced in the PE texts.
each of these revision marks was categorised and was attributed a penalty score ranging from 1 (minor) to 5 (critical), according to the perceived distortion of the original message and intention that the source text is supposed to convey.
severity penalties were then normalised using a 100-word basis, in order for the results to be comparable between participants and texts: a total penalty score was computed for each text, and then accordingly divided to reach a 100-word penalty score.
these normalised scores enabled us to compare the perceived quality of the texts provided by our participants.
though our results cannot be generalised, since the study presented here is a case study for which no significance score could be computed (not enough data), several conclusions were reached: the overall PE text quality is higher in participants with high experience levels (senior translators) than in junior translators; participants with lower experience levels produce PE texts containing more fidelity and terminology problems than their more experienced counterparts, and professional experience does not seem to have an influence on the proportion of errors directly caused by NMT proposals.
several organisational constraints limited the scope of our study.
first, the modest number of participants did not provide for significant results.
hence, a deeper study could be carried on with more volunteers, in order to reach more generalisable results.
secondly, each participant provided us with an uneven number of texts and PE words.
this is due to the very nature of our study, in the framework of which translators provided us with texts coming from their daily translation tasks, which limits the quantity of collected data but increases natural validity.
furthermore, the authentic context in which this study was implemented did not enable us to collect process data: further studies could include said data, which would provide for more representative results and provide us with an insight in translators’ cognitive processes when post-editing.
in this context, eye-tracking data could be collected, and methods such as questionnaires and think-aloud protocols could be implemented in order to link process data to the quality scores obtained in our study.
finally, studying additional language pairs would be relevant, since NMT quality tends to vary according to these.
Abstract FRE
Dans cet article, nous nous proposons de présenter les résultats d’une étude de cas menée conjointement avec la dgt de la commission européenne.
cette étude a pour objet l’analyse de la qualité des contenus post-édités à partir d’une traduction automatique neuronale (TAN) chez des traducteurs aux différents niveaux d’expérience.
aux fins de cette analyse, nous nous sommes appuyé sur une grille d’évaluation vérifiée au moyen d’une mesure d’accord interévaluateurs.
pour chaque texte source, une proposition de TAN issue du moteur etranslation a été post-éditée par un traducteur de l’institution (10 participants, dont 7 fonctionnaires et 3 stagiaires «blue book») et révisée par un collègue.
pour mener à bien notre analyse, nous avons évalué la qualité des textes post-édités à l’aune des marques de révision insérées dans les versions post-éditées, marques que nous avons classées en catégories et auxquelles nous avons attribué une pénalité de sévérité allant de 1 (mineur) à 5 (critique).
les pénalités ont ensuite été normalisées sur une base de 100 mots, afin de rendre les scores comparables entre les participants.
malgré le caractère non généralisable des résultats, nous avons pu conclure que la qualité des contenus post-édités est supérieure chez les traducteurs expérimentés de notre cohorte, que les participants les moins expérimentés produisent des textes post-édités contenant davantage de problèmes de sens ou de terminologie par rapport à leurs homologues plus expérimentés, et que l’expérience professionnelle ne détermine pas la proportion de phénomènes erronés directement provoqués par la TAN.
American Psychological Association (APA)
Pires, Loic De Faria. 2022. Post-édition de ta neuronale à la DGT et qualité des textes finaux: étude de cas. Revue traduction et langues،Vol. 21, no. 1, pp.77-98.
https://search.emarefa.net/detail/BIM-1442019
Modern Language Association (MLA)
Pires, Loic De Faria. Post-édition de ta neuronale à la DGT et qualité des textes finaux: étude de cas. Revue traduction et langues Vol. 21, no. 1 (Aug. 2022), pp.77-98.
https://search.emarefa.net/detail/BIM-1442019
American Medical Association (AMA)
Pires, Loic De Faria. Post-édition de ta neuronale à la DGT et qualité des textes finaux: étude de cas. Revue traduction et langues. 2022. Vol. 21, no. 1, pp.77-98.
https://search.emarefa.net/detail/BIM-1442019
Data Type
Journal Articles
Language
French
Notes
Includes appendices: p. 98
Record ID
BIM-1442019