Cultural references and machine translation: a methodology for evaluation The case of administrative texts in the area of migration
Article Sidebar
Main Article Content
Abstract
Cultural references constitute one of the great challenges for machine translation (MT). Despite the existence of numerous ad hoc studies on this technology, the number of cases where it can be applied exceeds the limits of current research. MT of a language's own cultural manifestations is one of the fields that have been little explored so far. Consequently, there is a need to review the evaluation methods often used to determine the validity of the texts produced by MT, with a focus on cultural referents on the field of migration. In this line, the article presents a methodological proposal for MT evaluation which is based primarily on the following qualitative data: fluency, accuracy, and acceptability. To these, quantitative data is added on the perception of these same criteria. First, the cultural references are contextualised in the framework of MT and artificial intelligence. Then, the different evaluation methods of automatic and manual evaluation are presented, with the definition of a specific methodology for the evaluation of cultural references. To illustrate this methodology from a practical point of view, a case of evaluation of cultural references is shown by means of an exploratory study carried out with administrative texts in the field of migration.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).References
Almaghout, H. y Specia, L. (2-6 de septiembre 2013). A CCG-based quality estimation metric for statistical machine translation [póster]. MT Summit XIV, Niza, Francia. https://aclanthology.org/2013.mtsummit-posters.4.pdf
Bender, E. M. (11 de mayo de 2022) Look behind the curtain: Don’t be dazzled by claims of ‘artificial intelligence. The Seattle Times. https://www.seattletimes.com/opinion/look-behind-the-curtain-dont-be-dazzled-by-claims-of-artificial-intelligence/
Bender, E. M., Gebru, T., McMillan-Major, A., y Shmitchell, S. (2021). On the dangers of stochastic parrots Can Language Models Be Too Big? [comunicación]. Proceedings of the 2021 ACM Conference on Fairness, Accountability and Transparency, (pp. 610–623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922 DOI: https://doi.org/10.1145/3442188.3445922
Candel-Mora, M. A. (2022). Fine-tuning machine translation quality-rating scales for new digital genres: The case of user-generated content. ELUA Estudios de Lingüística Universidad de Alicante, 38, 117-136. https://doi.org/10.14198/elua.21900 DOI: https://doi.org/10.14198/ELUA.21900
Canfora, C. y Ottmann, A. (2020). Risks in neural machine translation. Translation Spaces, 9(1), 58–77. https://doi.org/10.1075/ts.00021.can DOI: https://doi.org/10.1075/ts.00021.can
Castilho, S.; Doherty, S.; Gaspari, F. y Moorkens, J. (2018). Approaches to Human and Machine Translation Quality Assessment. En J. Moorkens, S. Castilho, F. Gaspari y S. Doherty (Eds.), Translation Quality Assessment from Principles to Practice (pp. 9-38). Springer International. https://doi.org/10.1613/jair.1.12007. DOI: https://doi.org/10.1007/978-3-319-91241-7_2
Conde Ruano, J. T. (2022). Calidad. En Enciclopedia de traducción e interpretación (ENTI). https://www.aieti.eu/enti/quality_SPA/
El-Madkouri Maataoui, M. (2016). El discurso del lenguaje jurídico-administrativo español: análisis y perspectivas. En M. Eurrutia Cabrero (Coord.). El lenguaje jurídico y administrativo en el ámbito de la extranjería: Estudio multilingüe e implicaciones socioculturales (pp. 127-164). Peter Lang.
Google Translator. https://translate.google.es/
Koponen, M., Mossop, B. Robert, I. S. y Scocchera, G. (Eds.) (2021) Translation, revision and post-editing. Routledge. DOI: https://doi.org/10.4324/9781003096962
Luque Nadal, L. (2009) Los culturemas: ¿unidades lingüísticas, ideológicas o culturales? Language Design, 11, 93-120.
Martindale, M. y Carpuat, M. (2018). Fluency Over Adequacy: A Pilot Study in Measuring User Trust in Imperfect MT. En C. Cherry y G. Neubig (Eds.) Proceedings of the 13thConference of the Association for Machine Translation in the Americas (Volume 1: Research Track), (pp. 13-25). Association for Machine Translation in the Americas, https://www.aclweb.org/anthology/W18-1803
Microsoft Translator. https://translate.google.es/
Mihalache, I. (2021). Human and Non-Human Crossover: Translators Partnering with Digital Tools. En R. Desjardins, C. Larsonneur y P. Lacour (Eds.) When Translation Goes Digital. Case Studies and Critical Reflections (pp. 19-44). Palgrave Macmillan. DOI: https://doi.org/10.1007/978-3-030-51761-8_2
Molina, L. (2006). El otoño del pingüino: análisis descriptivo de la traducción de los culturemas. Publicaciones de la Universidad Jaime I.
MyMemory. https://guides.matecat.com/my
Nitzke, J.; Hansen-Schirra, S. y Canfora C. (2019). Risk management and post-editing competence. JoSTrans. The Journal of Specialised Translation, 31, 239-259.
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama,K., Ray, S., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder,P., Christiano, P., Leike, J. ,Lowe, R. (2022). Training language models to follow instructions with human feedback. https://doi.org/10.48550/arXiv.2203.02155
Papineni, K., Roukos, S., Ward T., y Zhu, W. (julio 2002). BLEU: a method for automatic evaluation of machine translation. [Presentación de comunicación]. ACL-2002: 40th Annual meeting of the Association for Computational Linguistics (pp 311–318). DOI: https://doi.org/10.3115/1073083.1073135
Portal de Migraciones del Ministerio de Inclusión, Seguridad Social y Migraciones. https://www.inclusion.gob.es/web/migraciones/home
Ricart Vayá, A. y Jordán Enamorado, M. A. (2022). Traducción automática y crisis humanitaria: análisis de la eficacia de Google Translate en la comunicación con refugiados ucranianos en España. Revista Tradumàtica, 20, 96-114. https://doi.org/10.5565/rev/tradumatica.306 DOI: https://doi.org/10.5565/rev/tradumatica.306
Rico Pérez, C. (2020). Translation technologies for the aid-chain. En F. M. Federici y S. O’Brien (Eds.) Translation in Cascading Crises (pp. 112-131). Routledge. DOI: https://doi.org/10.4324/9780429341052-6
SAE (2001). SAEJ2450 Translation quality metrics. http://www.apex-translations.com/documents/sae_j2450.pdf
Sánchez Ramos, M. M. y Rico Pérez, C. (2020). Traducción automática. Conceptos clave, procesos de evaluación y técnicas de posedición. Comares.
Secretaría de Estado de Migraciones. (2022). Unidad de grandes empresas y colectivos estratégicos. https://www.inclusion.gob.es/web/unidadgrandesempresas/sobre-nosotros
Tesseur, W. (2017). The translation challenges of INGOs. Professional and non-professional translation at Amnesty International. Translation Spaces, 6(2), 209–229. https://doi.org/10.1075/ts.6.2.02tes DOI: https://doi.org/10.1075/ts.6.2.02tes
Tesseur, W. (2022). Translation as Social Justice: Translation Policies and Practices in Non-Governmental Organisations (1.ª ed.). Routledge. https://doi.org/10.4324/9781003125822 DOI: https://doi.org/10.4324/9781003125822-1
Torrijos Caruda, C. (2022). Inteligencia artificial y traducción al español. Proyección, riesgos y responsabilidad. Puntoycoma, 174, 31–40. https://www.aieti.eu/noticias/introduce-una-noticia/
Valli, P. (26-27 de noviembre de 2015). The TAUS Quality Dashboard [Presentación de comunicación]. Proceedings of the 37th Conference Translating and the Computer, (pp. 127–136). https://aclanthology.org/2015.tc-1.17.pdf
Venuti, L. (1995): The Translator’s Invisibility, Routledge.
Vetere, G. (2021). Textnology. Imminent. Research Report.https://imminent.translated.com/textnology
Vieira, L. N. (2019). Post-editing of machine translation. O'Hagan, M. (Ed.), The Routledge handbook of translation and technology (1.ª ed.), pp. 319-337). Routledge. https://doi.org/10.4324/9781315311258 DOI: https://doi.org/10.4324/9781315311258-19
Way, A. (2018). Quality Expectations of Machine Translation. Moorkens, J., Castilho, S., Gaspari, F., y Doherty, S. (Eds.), Translation Quality Assessment from Principles to Practice (pp. 159-178). Springer International. https://doi.org/10.1613/jair.1.12007 DOI: https://doi.org/10.1613/jair.1.12007