ISSN: 1579-9794
Hikma 22(1) (2023), 219 - 248
An Eye-Tracking Study of Cognitive Effort in Processing of
Lexical Features in Students and Experts
Estudio del esfuerzo cognitivo en el procesamiento de
características léxicas en estudiantes y expertos utilizando
una metodología de seguimiento de ojos
M
ONTSERRAT BERMÚDEZ BAUSELA
mbermudez@flog.uned.es
Universidad Nacional de Educación a Distancia (UNED)
T
ABEA DE WILLE
tabea.dewille@ul.ie
University of Limerick
Fecha de recepción: 01/07/2022
Fecha de aceptación: 27/10/2022
Abstract: In this paper, it is our aim to observe the impact that translator
training and experience have on different groups of participants (novice,
trained and experienced) when evaluating a translation. We will be doing so
by measuring the cognitive effort invested by the participants in the processing
of lexical features applying an eye-tracking methodology. Participants will be
presented with several translated versions from English into Spanish done by
translators with different levels of training and experience. This paper offers a
detailed description of the experiment carried out. In it, we were also able to
observe that while there are common patterns in the three groups, training
and experience does have an impact on their behaviour when reading and
assessing the different translated versions. We have been able to observe a
link between these two factors and the amount of cognitive effort, which is
higher in the group of students than in the group of experts, with trained
participants leading the numbers, which we believe confirms the skill
acquisition model proposed by Dreyfus (2004). Also, that extrinsic information
is an element of disruption that influences the decisions made by participants,
the amount of cognitive effort employed and how those lexical features have
been processed.
Keywords: Cognitive effort, Eye-tracking, Translator training, Assessment
competence, Translation process
220 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
Resumen: El objetivo de este artículo es observar el impacto que la formación
y la experiencia del traductor ejercen sobre diferentes grupos de participantes
(principiantes, con formación y con experiencia) a la hora de evaluar una
traducción. Para ello, mediremos el esfuerzo cognitivo empleado por estos en
el procesamiento de características léxicas mediante la aplicación de una
metodología de seguimiento de ojos. Se presentará a los participantes con
una serie de versiones traducidas del inglés al español realizadas por
traductores con distintos niveles de formación y experiencia. El artículo ofrece
una descripción detallada del experimento llevado a cabo. En él, se puede
observar que, además de detectar unos patrones comunes en los tres grupos,
la formación y la experiencia han influido en su comportamiento a la hora de
proceder con la lectura y la evaluación de las distintas versiones de
traducción. Hemos podido observar un vínculo entre estos dos factores y la
cantidad de esfuerzo cognitivo empleado, la cual es mayor en el grupo de
estudiantes que en el grupo de expertos, siendo los participantes con
formación los que registran un mayor esfuerzo, lo cual creemos que confirma
el modelo de adquisición de destrezas propuesto por Dreyfus (2004). De la
misma forma, la información extrínseca representa un elemento disruptivo
que ha condicionado las decisiones de los participantes, la cantidad de
esfuerzo cognitivo y cómo se han procesado las características léxicas.
Palabras clave: Esfuerzo cognitivo, Seguimiento de ojos, Formación del
traductor, Competencia evaluadora, Proceso de traducción
I
NTRODUCTION
While much has been written about the competences a translator
should acquire (for example, PACTE, 2000, 2003; Kelly, 2002), less is known
about the impact translator training and experience have on the process and
cognitive effort invested in evaluation of translation.
In this paper we present a study on the cognitive effort invested by three
different groups of participants (novice, trained and experts) in processing
lexical features through the use of eye-tracking methodology. By asking these
three groups of participants to evaluate three translations and choosing which
was their preferred version, we observed the impact training and experience
had on the approach to the evaluation taken by different participant groups.
Further, we provided either correct or incorrect information on the professional
status of the translators to participants with the goal of observing how such
information would influence their choices and evaluation process. For the sake
of brevity, we refer to scenarios where no such extrinsic information was
provided as NoInfo, where correct information was provided as GoodInfo, and
incorrect information as BadInfo throughout. Finally, we observed a number
Monserrat Bermúdez Bausela y Tabea De Wille 221
Hikma 22(1) (2023), 219 - 248
of patterns that were common across all participant groups regardless of
training or experience.
In our reflections on the link between the competences acquired, the
stage of the acquisition of those competences and the ability to assess the
quality of a translation, we draw on the Dreyfus and Dreyfus five-stage model
of adult skill acquisition (Dreyfus & Dreyfus, 1986; Dreyfus, 2004) and
Chesterman’s (2016) adaptation to Translation.
1. H
YPOTHESIS/EXPECTATIONS AND OBJECTIVES
H01. When processing lexical features, we expect some common
patterns in the behaviour of all groups of participants:
• H01.1 There is a preference for certain words as focus of
attention.
• H01.2 Participants compare keywords, sometimes in a
sequential manner.
• H01.3 Given NoInfo, participants expend more cognitive effort
on lexical words than grammatical words.
H02. Having received formal training has an impact on the behaviour of
participants when they are reading and assessing the three different
translated versions. Cognitive effort is higher in the group of students than in
the group of experts:
• H02.1 Students expend more cognitive effort than experts on
the evaluation process.
• H02.2 Mistranslations and difficult to translate words require
more cognitive effort on the part of the group of students, while
experts are more efficient (that is, they spend less time on
those units, which translates into less cognitive effort).
H03. Extrinsic information will be an element of disruption:
• H03.1 When meta words are included in the scenarios,
participants pay attention to them, that is, they register long
durations and are included in sequential comparisons.
The specific objectives in which we think the aforementioned might be
materialised are the following:
1) Study the cognitive effort made by participants in the
processing of lexical features and check whether there is a
difference in novice, trained and expert groups.
2) Observe if, despite the possible differences, there are common
patterns or common behaviours in the three groups.
222 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
3) Learn if the extrinsic information provided exerts some kind of
influence in the decisions/the amount of cognitive effort/how
those lexical features are processed.
2. L
ITERATURE REVIEW
In a previous study to this one, presented in De Wille and Bermúdez-
Bausela (2018), we investigated ‘perceived quality’ and how strongly an
individual is influenced by extrinsic cues, that is, factors such as price, brand
name, level of advertising (Zeithaml, 1988, p. 6; Sirohi, McLaughlin & Wittink
1998, p. 226; Vantamay, 2007, p. 114) or, in the case that concerns us here,
whether the translation has been performed by a novice student, a trained
student or a professional translator. This is opposed to intrinsic attributes,
which are usually information inherent to the nature of the translation process
in itself. Examples include issues concerning the right meaning, grammar,
terminology, or style.
While a number of factors such as status, remuneration, education,
prestige or visibility (Paloposki, 2016, p. 18; Pym, Orrego-Carmona & Torres-
Simón, 2016) play a role in delineating a ‘professional’ from a ‘non-
professional’ translator, we are particularly interested in the relationship
between being a ‘professional translator’ and training: Does a professional
translator need to have received formal training to be considered so?
Translation training can take many forms (Pym, 2011, p. 313): experience
would be at the first level for all those translators that have learnt on the job,
to short-term courses which tend to focus on the acquisition of specific skills
required for a particular niche, and, finally, long-term training programmes
offered by mainly Universities. As stated previously, in this paper we would
like to test the hypothesis that having received formal training has an impact
on how participants approach the translation evaluation process.
We believe there is a close link between the skills and competences
acquired during the degree and the ability to assess the quality of a translation
as part of that training. We would like to highlight the five-stage model of adult
skill acquisition suggested by Dreyfus and Dreyfus (1986) and Dreyfus (2004),
since our intention is to establish a link between assessment abilities and
translator training. The professional translator is an expert, i.e., with
translational competence. But how is this expertise acquired? Dreyfus and
Dreyfus (1986) differentiate five stages. In the first three (novice, advanced
beginner and competence), trainees mainly acquire and follow rules, process
information and make some choices; their actions and behaviour are
conscious. In the last two (proficiency and expertise), intuition and experience
take over and actions are more automatic and unconscious. This development
from novice to expert stage is, according to Dreyfus (2004), marked by a range
Monserrat Bermúdez Bausela y Tabea De Wille 223
Hikma 22(1) (2023), 219 - 248
of emotions in the intermediate stages, especially that of the competent
learner (which is the third stage out of five). While the novice learner is
provided with rules and examples, the «learning can be carried out in a
detached, analytic frame of mind as the student follows instructions and is
given examples» (Dreyfus, 2004, p. 177). However, with increased experience
the learner can become overwhelmed and «performance becomes nerve-
wracking and exhausting» (Dreyfus, 2004, p. 178) as they are able to
recognize an increasing number of potentially relevant elements and
procedures but are lacking a sense of what is important for the situation they
are in. While the proficient performer sees what needs to be done and then
decides how to do it, the expert not only sees what needs to be achieved but
also immediately how to achieve it due to a «vast repertoire of situational
discriminations» (Dreyfus, 2004, p. 180).
Chesterman (2016, p. 145-166) applies the Dreyfus and Dreyfus (1986)
five-stage model to translation. The key term here would be ‘conscious
awareness’, a concept that is applied in all five stages: in the early stages of
novice, advanced beginner and competent performer, because the trainee is
mainly following rules and processing information; in the higher stages of
proficiency and expertise, because results need to be monitored and
improved. The expert translator has internalised routines, operating in an
automatic way, and brings about ‘deliberative rationality’ only when the
situation requires so.
In the process of translation training, assessment abilities will have
been acquired in the middle and the last stages, thanks to experience and
having absorbed all the competences in translation. Chesterman (2016, p.
159-160) is of the opinion that contrastive analysis becomes relevant at this
stage as it helps students take an analytical view on the source text and the
target text. Having cultivated the required skills, their judgements will be more
reliable. In this light, we argue that conscious engagement translates into
cognitive effort: the more conscious engagement, the more the cognitive
effort. This cognitive effort can then be visualised and measured using eye-
tracking methodology.
Eye-tracking helps to investigate different aspects of cognitive
processing, in our case, determining the cognitive effort spent by our
participants on particular words. This is based on the assumption that there is
a correlation between what the eye is looking at and the amount of cognitive
effort devoted to its processing (Just & Carpenter, 1980, p. 330; Schäffner &
Shuttleworth, 2013, p. 102). Muñoz Martín (2014, p. 60) uses the term ‘mental
load’ to describe this concept. He reports on studies conducted in research
efforts related to mental load and the comprehension and production of texts,
and states that given complex mental activities, mental resources or mental
224 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
capacity (which are limited) can be overloaded, which we think might be the
reason why the assessment process runs differently and takes different
amounts of time for each group of participants.
Pavlović and Hvelplund (2009) have investigated directionality in
translation processes, conducting an experiment that involves student and
professional subjects. They reach the conclusion that cognitive effort invested
in the processing of the translation is greater than that invested in the
processing of the original in both directions of translation. Jakobsen and
Hvelplund (2008) have conducted an experiment using eye-tracking
methodology with novice and professional translators who are presented with
some translation-oriented tasks. They then measured the cognitive effort
employed by each group of participants in terms of gaze duration. One of their
conclusions is that professionals spend more time on the translation itself and
on its revision than on the source text, while students spend more time trying
to comprehend the source text. Also applicable to our own study is Schaeffer,
Paterson, McGowan, White, and Malmkjaer’s (2017) work, which reports on
the relationship between eye movement measures and words that have more
than one translation alternative, as is our case on many occasions. They also
compare the eye movement behaviour when reading for comprehension and
when reading for translation. Dragsted and Carl (2013) carried out a study
involving students and professional translators with the purpose of analysing,
first of all, common features to all of them regardless of their expertise and,
then, to investigate individual behavioural characteristics and their possible
categorisation. In several moments of our own research we have reached
similar conclusions, in particular, that some novices behave similarly to what
is expected from experts and the other way around, at least under some
conditions. The goal of these authors is to try to extract features that might
describe the translator behaviour and to establish a relationship between the
style of the translator and their level of expertise (whether professional or
novice). They reached several interesting conclusions, among them, that
translators are characterised by their individual profiles and that their
behaviour remains quite constant regardless of the complexity of the texts.
The authors point to the fact that this might also be the case regardless of
other external factors (we add here that one of these external factors could be
the extrinsic information that the participants were presented with in our study,
and we will extract our own conclusions on this point). Schaeffer et al. (2019)
carried out a study with students and professional translators to measure their
revision competence in translation applying eye-tracking. They study the
behaviour in each group of participants in relation to the correction of mistakes
and reading, reaching the conclusion that professional translators are more
efficient when it comes to error recognition and correction because they
prioritise what they are searching and reading, being able to differentiate
Monserrat Bermúdez Bausela y Tabea De Wille 225
Hikma 22(1) (2023), 219 - 248
between types of errors and adequating their strategies to it, while «students
take longer to correct errors as compared to professional translators»
(Schaeffer et al., 2019, p. 600), which is especially worth noting for our study.
In our case, we will also be expecting more time devoted to mistranslations in
the group of the students. The list of relevant studies is long and this section
does not try to be a comprehensive collection of them. For a more extensive
review of eye tracking methodology applied to process-oriented translation
research, we recommend the overviews carried out by Walker (2021) and
Hvelplund (2017).
As part of our expectations and objectives, we are interested in studying
whether the participant has a tendency to focus on some particular words that
might have caught their attention for several reasons and if they compare
them, going from the same word in the ST and one or several of the three
possible equivalents in the target language (we will call this ‘keyword
sequential movements’ as we have not found an existing term in the literature
that describes exactly what we want to study). Also, we want to check if this
is a common pattern among participants. In this light, Rayner (1998, p. 377)
points out that when a word is the unit of analysis, the two most frequently
used measures in eye-tracking are the first fixation duration and the gaze
duration on a word (that would be «the sum of all fixations made on a word
prior to a saccade [rapid movements of the eyes] to another word»). According
to this author, «readers' gaze durations are longer on low-frequency words
than on high-frequency words» (Rayner, 1998, p. 378), which, we believe,
might also be applied to lexical words and grammar words.
Following Muñoz Martín:
When fixations are observed on ST words, it makes sense that the
translator is engaged in ST reading and when fixations are observed
on TT words, it makes sense that the translator is engaged in tasks
related to the processing of the TT. (2014, p. 209)
Assuming that there is a link between visual focus and cognitive focus,
our aim is to study the cognitive effort employed in the lexical features. Rayner
(1998, p. 387) points to a study conducted by Frazier and Rayner (1982) who
demonstrated that when readers encountered a word that indicated that their
prior interpretation of the sentence might be wrong, they often made a
«regression» (defined by Rayner (1998, p. 375) as «right-to-left movements
along the line or movements back to previously read lines») to that word to
find disambiguating information. In this sense, Rayner believes that many
regressions are due to comprehension failures. He mentions that when these
words are ‘refixated’, they receive additional fixations before the reader leaves
the word (Rayner, 1998, p. 387). There is evidence to suggest that high-
226 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
frequency words are ‘skipped’ more frequently than low frequency words.
Based on this, we have formulated our hypothesis that participants will focus
on lexical words and skip grammatical words.
There are several textual variables that seem to influence where the
readers’ eyes move to next, one of them would be word length: the longer the
word, the longer the duration, which increases the probability that the reader
re-fixates on it. Two other factors that influence fixation time on a word are
word frequency, already mentioned, and contextual constraints (Rayner,
1998, p. 387). Regarding contextual constraint (understood as the relative
predictability of the context by Schwanenflugel and LaCount, 1988), when
words are highly constrained by the preceding context, they are skipped more
frequently than words that are not, or are fixated for less time (Rayner, 1998,
pp. 387-388). The author mentions other variables that influence fixation time
on a word or the pattern of eye movements: semantic relationships between
words, repetition effects, morphemic units, anaphora and coreference, lexical
ambiguity, phonological ambiguity, discourse factors and stylistic conventions,
and syntactic disambiguation (Rayner, 1998, p. 390).
The terminology that we will be using in the paper, such as Fixation
Point (FP), First Fixation Point (FFP), Duration of Fixation Point (DFP), Area
of Interest (AOI) and regressions (or revisits), among others, will be further
explained and developed in the data analysis. For the purposes of this study,
we assume that they are all indicators of the cognitive effort used by each
group of participants (and also of individual participants in the three groups)
in their assessment of the translations.
3. M
ETHODOLOGY AND DATA COLLECTION
As the previous sections have shown, eye-tracking is a widely used
method for investigating cognitive effort and processes during the evaluation
task. It affords researchers a more direct way of measuring cognitive effort
than alternatives such as Think Aloud Protocols. While these have their
applications, they require an additional cognitive step from participants that
involves verbally describing their research. Our concern with this approach
was that this added step could influence the results by calling the process
undertaken into the conscious awareness of the participants as opposed to
merely observing the process as is the case for the eye-tracking approach
(Saldanha & O’Brien, 2014, p. 124). Furthermore, due to the mixed language
background of the research team, participants would have either had to
describe their process in English (their L2) or the protocols would have
required translation for analysis and publication, which would have added a
further step distancing the cognitive efforts from the data analysis. While all
known approaches don’t directly measure cognitive effort (Saldanha &
Monserrat Bermúdez Bausela y Tabea De Wille 227
Hikma 22(1) (2023), 219 - 248
O’Brien, 2014, p. 113), we believe eye-tracking is the most appropriate
approximation in the context of our research. However, it does not lack its
limitations which we have sought to mitigate as much as possible through the
nature of the setup outlined below. O’Brien (2009) documents some
challenges that researchers who are interested in applying eye-tracking
methodology to translation research might be faced with, such as
environment, participants, ethics, data explosion and validity, some of which
we will discuss below as they relate to our study. Muñoz Martín (2014, p. 5)
also comments on the potential difficulties (reliability, validity and appropriate
use of these research tools and methods) imposed on translators.
3.1 Overview of the setup
Individual participants were assigned to three groups based on their
level of training and experience. Each participant was then shown the same
set of MS Powerpoint slides on a screen. Those slides included source text
(ST), three translations or target texts (TT) and in some cases information
(correct or incorrect) on who had conducted the translation. We refer to this
information as «extrinsic» or «meta words». Participants were asked to
verbally select which of the three translations on each slide they thought was
best. Their preference was recorded by one of the researchers present in the
room and their eye-movements during evaluation were recorded using eye-
tracking technology.
Participants did not receive instructions on how to evaluate the
translations or whether or not to consider the extrinsic information provided.
3.2 Participants
Members of the student body and teaching staff at the University
Alfonso X el Sabio in Madrid were invited to participate in this study.
Participants donated their time freely and were not compensated or rewarded.
The setup was piloted with two participants and the data collected excluded
from analysis. This was to allow the researchers to increase their familiarity
with the setup and to resolve issues with the physical space and instructions
given to participants.
The 24 participants included in data collection and analysis were
grouped into three categories, based on their level of training received at that
point and their working experience (Table 1).
Participant grouping
Number
Percentage
Novice evaluators (1st/2nd year students)
12 (5 1st, 7 2nd year)
50%
228 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
Trained evaluators (3rd/4th year
students)
6 (4 3rd year, 2 4th year)
25%
Expert evaluators (lecturers)
6
25%
Table 1. Distribution of participants
Source. Elaborated by the authors
We considered students in years 1 and 2 of their studies to be novice
evaluators, and students in years 3 and 4, trained evaluators. The rationale
behind this division is based on the competences that the particular groups of
participant students taking part in this case study acquire during their degree.
Our idea of competence is the one outlined by Kelly (2002, p. 14) according
to whom the translation competence is a ‘macro-competence’ that «comprises
a set of capacities, skills, knowledge and even attitudes that professional
translators combine and that intervene in translation as an expert activity».
During their first year of the degree, students mainly developed linguistic and
communication skills in their working languages (two foreign languages),
something that continued to be reinforced in their second year, when students
also acquired cultural competences and were introduced to their first
translation and interpreting subjects, along with translation theory and
linguistics courses. In their third and fourth years, they developed interpreting
and translation skills, they acquired textual and translation competences in
professional and specialised fields of translation (literature, business, legal,
commercial, journalistic, audiovisual, scientific, technical, and so forth), along
with technical skills covered in courses on Computer Assisted Translation
Tools and localisation. In their third year they had a specific revision course in
which students were provided with the necessary skills to conduct a linguistic
and textual analysis applied to the revision of texts. Students were made
aware of the difference between revision and correction, types of error in
translation, the process of revision and industry quality standards. Based on
these competences acquired by our students in their degree on Translation
and Interpreting, we assume the progressive acquisition of new skills and
competences. While first and second-year students had been mainly exposed
to linguistic and cultural competences, the third and fourth-year students had
covered all the competences in translation: linguistic, cultural, textual,
translation, technical and documentary (for models on the acquisition of
translation competences, see Hurtado Albir, 2015).
Expert evaluators were lecturers at the University Alfonso X el Sabio in
Madrid. In terms of formal training, 60% of them had a degree and/or a
doctorate in Philology (English Studies mainly, and one in German Studies),
while the other 40% had a degree and/or a doctorate in Translation and
Monserrat Bermúdez Bausela y Tabea De Wille 229
Hikma 22(1) (2023), 219 - 248
Interpreting Studies. This is not a surprising distribution since the degree in
Translation and Interpreting in Spain was not very common before the 1990s
and translators who started their career during that period usually hold
degrees in Philology and have afterwards acquired specific competences in
Translation through advanced degrees and experience. Going back to our
group of experts, 80% of them had more than 20 years of experience teaching
Translation in Universities and over 20 years of being professional translations
in the workplace.
3.3. Environment setup
Participants completed the task individually and only the two
researchers were present in the room with them. Participants were asked to
sit at a previously set up desk with a computer monitor elevated to allow them
to comfortably read the text on the screen. The room was well lit, with a mix
of daylight and artificial light. One researcher provided instructions and
support for participants while the other operated the eye-tracking software and
monitored the data collection. Although using an eye-tracking bar rather than
head-mounted eye-tracking hardware may lower accuracy of the
measurements (O’Brien, 2009), we used an eye-tracking bar placed below the
screen that participants were viewing, with the goal of improving ecological
validity as this technology is less intrusive to participants. Specifically, we used
eye-tracking technology by Gazepoint, and associated software for recording
the data and initial analysis. We used the built-in calibration procedure for the
system and were able to confirm successful calibration with a test procedure
on our recording screen. In some instances, calibration was not successful
even after multiple attempts and participants had to be excluded from the
study.
3.4. Setup of the stimulus
For this study, we used a selection of text created and translated for the
study previously described in De Wille and Bermúdez-Bausela (2018). The
text used for the study had been created by one of the authors of this paper
and proofread by a native speaker who was trained and experienced in editing
texts. Its features were:
• No previous translation was available.
• Short paragraphs (two to three sentences each) with individual
headings (between two and 13 words each) to avoid as far as
possible issues with missing references, context or consistency
that may arise from segmenting longer sections.
• General subject matter without requiring high levels of
specialisation.
230 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
The text was then independently translated by six Spanish translation
students and lecturers (two from each group). The groups were:
• Novice translators: first-year translation students who had not
yet received training in translation.
• Trained translators: fourth-year translation students who had
received in-depth training in translation but had not gained real-
life translation experience.
• Expert translators: lecturers at the University Alfonso X el Sabio
in Madrid. Both had many (25 and 15 respectively) years of
teaching experience in the translation degree and were trained
and experienced in judging language quality.
All translators were told that their translations would be used for a study
but not what the exact nature of the study was. The translations were screened
(but not edited) by the researchers to ensure translations were mostly
complete and did not include additional text such as translator comments.
For this eye-tracking study, individual sentences were laid out on MS
PowerPoint slides. Text on the individual slides was limited to one single ST
sentence per slide and three translations. Each translation had been
conducted by a different translator as described above. Text was laid out in
large font sizes (between 27 and 44 px) with the goal of increasing the
accuracy of eye-tracking data.
The initial slides did not include extrinsic information on the translator
(NoInfo). These were followed by slides that included the correct information
on the professional status of the translator. We refer to this extrinsic
information as GoodInfo going forward. GoodInfo was positioned above the
target texts and included the phrases ‘Novice Translator’, ‘Trained Translator’
and ‘Trained + Experienced Translator’ without further explanation on those
phrases (Figure 1). The third set of slides also included extrinsic information,
but the extrinsic information provided did not match the professional status of
the translator who had done the translation. So, a novice translator’s TT might
be labelled as ‘Trained Translator’ and that of the expert translator as ‘Novice
Translator’, etc. In order to facilitate the comprehension of the analysis and
discussion, we include all the texts shown to participants as follows.
Monserrat Bermúdez Bausela y Tabea De Wille 231
Hikma 22(1) (2023), 219 - 248
Figure 1. Sample slide as shown to participants
Source. Elaborated by the authors
Scenario 1 NoInfo:
• ST: This is a demanding programme and you will receive a lot
of material and opportunities to learn.
• TT1: Este es un programa muy demandado, por lo que vas a
recibir mucho trabajo y vas a tener muchas oportunidades para
aprender.
• TT2: Este es un programa exigente, por lo que recibirás
muchos materiales y tendrás muchas oportunidades para
aprender.
• TT3: Este es un programa exigente y recibirás mucho material
y muchas oportunidades para aprender.
Scenario 2 NoInfo:
• ST: It can be tempting to try and do everything perfectly and
thereby get behind on the material.
• TT1: Puede ser tentador intentar hacer todo a la perfección y
como resultado, puedes quedarte atrás con el trabajo.
• TT2: Puede resultar tentador intentar hacer todo a la
perfección, pero así podrías quedarte rezagado.
• TT3: Puede resultar tentador intentar y hacer todo
perfectamente y, de este modo, apoyarse en el material.
232 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
Scenario 3 GoodInfo:
• ST: Think ahead to the dissertation but don’t panic.
• TT1: Trained + Experienced Translator: Ten la tesina en mente
pero que no cunda el pánico.
• TT2: Novice Translator: Mira de cara a tu tesis, pero no entres
en pánico.
• TT3: Trained Translator: Pensar de cara a la tesis pero sin
miedo.
Scenario 4 GoodInfo:
• ST: Make use of resources available to you like the library and
magazine subscriptions
• TT1: Novice Translator: Haz uso de los recursos disponibles
para ti, como la biblioteca o las revistas.
• TT2: Trained + Experienced Translator: Utiliza los recursos
disponibles como la biblioteca o las suscripciones de revistas.
• TT3: Trained Translator: Use los recursos disponibles como la
biblioteca y las suscripciones de revistas.
Scenario 5 BadInfo
• ST: Try to apply what you learn
• TT1: Trained Translator: (Done by Novice) Intentar poner en
práctica lo que aprendas.
• TT2: Novice Translator: (Done by Experienced) Pon en
práctica lo que aprendes.
• TT3: Trained + Experienced Translator: (Done by Trained)
Intentar aplicar lo que aprendas.
Scenario 6 BadInfo
• ST: In some modules you will need to take a written exam at
UL.
• TT1: Novice Translator: (Done by Experienced) Algunos
módulos tienen exámenes presenciales que realizarás en la
UL.
• TT2: Trained Translator: (Done by Novice) En algunas
unidades tendrás que hacer un examen escrito en la
Universidad de Limerick.
• TT3: Trained + Experienced Translator (Done by Trained): En
algunos módulos deberá hacer un examen escrito en la
universidad
Monserrat Bermúdez Bausela y Tabea De Wille 233
Hikma 22(1) (2023), 219 - 248
4. RESULTS
4.1 Data analysis
The following sections focus on four aspects as measurements of
cognitive effort: First Fixation Points (FFP), Last Fixation Points (LFP),
Keyword Sequential Movements (KSM) and duration of fixations.
For the duration of first and last fixation point we established a threshold
of 0.29 seconds and summarised which words the participants as a whole or
per participant group fixated on primarily. We expected that participants would
compare keywords in the different translations to a varying extent depending
on their professional status. We expected that expert evaluators would expend
less cognitive effort, and therefore compare keywords less than other
participant groups. We counted the number of KSMs that indicated the
number of movements between keywords in a group. We counted a KSM
when participants went linearly from a particular word in ST, to the equivalent
in one or more of the translations.
We were interested in two aspects:
• The range of KSMs registered. We have established three
ranges: high, medium and low. When it is labelled ‘high’, it
means that we have counted 1 for each participant who has
registered KSM in both the scenarios presented according to
the information provided (NoInfo, GoodInfo, BadInfo). Likewise,
when the level is ‘medium’, we have counted 1 for each
participant who has registered one KSM in one of the scenarios
presented. Finally, ‘low’ level means that the participant has not
registered any KSMs for either of the two scenarios.
• Nature of the keywords in the SMs. The other issue is whether
the words in the KSMs have been lexical, grammatical or meta
words and if this is an issue to bear in mind in our analysis.
In doing so, we would like to check if experts register less KSMs than
students and if meta words are frequent in the KSMs, based on the
assumption that the higher the KSMs the more the cognitive effort.
4.2 Main observations
We have summarised our findings in three tables: Table 2 shows first
fixations and last fixations including the most frequent words, distribution
between lexical, grammatical and meta words. Table 3 shows KSMs. We first
discuss high-level observations and then provide details on how participants
approached mistranslations, difficult words and meta information in more
detail. Table 4 shows keywords and the average duration spent on them.
234 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
We have made the following high-level observations:
1) Overall, there were common patterns across all groups regarding
words that participants spent the most cognitive effort on, whether
that related to their first fixation points or last fixation points. We
believe that a lot of these commonalities are due to specific aspects
inherent in those words, such as being particularly difficult to
translate or containing mistranslations.
2) There was a smaller variety in FFPs chosen than in LFPs across
participant groups.
3) FFPs were more likely to be an English word in the ST, while LFPs
were mostly from one of the three translated versions.
4) Participants spent more time on LFPs, in comparison to FFPs,
before saying out loud their preferred translated version.
5) There was a strong correlation between LFPs and the translated
version finally chosen by the participants. In most cases,
participants fixated on a word in the scenario they chose last, with
experts doing so most frequently (93.3%), followed by trained
evaluators (83.3%) and then novice evaluators (75%).
6) Across all scenarios, experts spent the least amount of time on the
evaluation, followed by novice evaluators and then trained
evaluators with the most amount of time. Novice and trained
evaluators also spent more cognitive effort in the form of
movements between keywords (Table 3) than expert evaluators.
Experts also spent less time, on average, on the LFPs, while the
trained students tended to spend more time on them.
7) In the majority of the cases, both the FFP and the LFP were a
lexical word. After that, the FFP constituted more grammatical
words than meta words, while for LFP it was the opposite in the
numbers, with the meta words being more frequent than the
grammatical words. We calculated these averages without taking
into account the NoInfo scenarios.
8) Participants, in general, focus on one word and compare them in
the different versions. This was especially pronounced for
keywords on which participants spent a higher amount of time. In
GoodInfo and BadInfo scenarios, those words were frequently meta
words, which tells us that this information prompted the interest of
participants.
Monserrat Bermúdez Bausela y Tabea De Wille 235
Hikma 22(1) (2023), 219 - 248
Table 2. First and Last Fixation Points and distribution in categories
Source. Elaborated by the authors
236 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
Table 3. Keyword Sequential Movements (KSMs)
Source. Elaborated by the authors
Monserrat Bermúdez Bausela y Tabea De Wille 237
Hikma 22(1) (2023), 219 - 248
Table 4. Keywords and average durations
Source. Elaborated by the authors
4.2.1 Mistranslations
The texts shown to participants contained 4 mistranslations:
1) In scenario 1 (NoInfo), in the first translated version (TT1). The word
‘demanding’ would usually be translated as ‘exigente’, not as
‘*demandado’, which could be considered a false friend.
KW1 KW2
KW3 KW4 KW5
demanding
opportunities receive material programme
demandado
oportunidades
recibir materiales programa
exigente recibirás
Av.
Fixation
Duration
(sec.)
7.607 5.292
4.204 3.432 3.006
%
32.3% 22.5% 17.9% 14.6% 12.8%
get behind
thereby
tempting try material
quedarse
atrás
(como)
resultado
tentador intentar trabajo
quedarte
rezagado
(de este)
modo
apoyarse
Av. FD (sec.) 8.959
5.098 4.524 4.104 3.592
%
27.6%
15.7% 13.9% 12.6% 11.1%
Think ahead
don't panic
(Extrinsic
Information)
dissertation
Mira de cara
no entres en
pánico
Experienced
Translation
tesina
Pensar de
cara
que no cunda
el pánico
tesis
Ten en mente sin miedo
Av. FD (sec.)
7.908 60.414 5.719 5.166
%
32% 24% 23% 21%
available
library
subscriptions resources
disponibles biblioteca subscripciones recursos
suscripciones
Av. FD (sec.)
6.32 5.77 4.21 3.977
%
31.20%
28.50% 20.80% 19.60%
Apply
Aprendes
poner en
práctica
aprendas
pon en
práctica
Av. FD (sec.)
4.698 3.846
%
55%
45%
written exam to take modules
(Extrinsic
Information)
some
examen
escrito
realizarás
módulos
Experienced
Translation
algunos
hacer unidades
algunas
Av. FD (sec.)
9.257 5.085 3.925 3.516 3.455
%
37% 20% 15% 14% 14%
none
none
none
none
none
KW
KW
KW
KW
KW
KW
238 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
2) In scenario 2 (NoInfo), in the third translated version (TT3). The
expression ‘get behind’ could be translated as ‘quedarse atrás’ or
‘quedarse rezagado’, but not as ‘*apoyarse’, which has a different
meaning and, therefore, could be a mistake labelled false sense.
3) In scenario 3 (GoodInfo) in the second (TT2) and third (TT3)
translated version. The word ‘dissertation’ has been wrongly
translated as ‘*tesis’ (PhD dissertation).
4) In scenario 5 (BadInfo) in the first (TT1) and third (TT3) version
there is a grammatical mistake since it has been translated as an
infinitive form (‘*intentar’). However, this would be so strictly in
grammatical terms since, lexically speaking, the translation of the
imperative form ‘Try’ into Spanish would be ‘Intenta’, not ‘Pon’. So,
we are also interested in knowing what participants have prioritised,
whether grammatical form or lexical choice.
We observed a strong focus from all three participant groups on the
mistranslated words ‘*demandado’ and ‘*apoyarse’ compared to other words
in the different scenarios with an average duration of 0.5 and 0.6 seconds
relative to our cut-off threshold of 0.29 for a fixation point. This was not the
case for ‘*tesis’, which did not register very high fixation points. This could be
due to the fact that it is not such an obvious mistake as the first two ones. For
the fourth mistranslation in scenario 5, we observed a strong focus on the
mistranslation ‘*Intentar’, as opposed to the version in imperative form ‘pon’
(Table 5).
In all instances, trained evaluators spent the highest amount of
cognitive effort on the mistranslated words relative to the other groups,
followed by novice in second and experts in third position in all but one
instance.
Focus on
mistrans.
‘*demandado’
S1
Focus on
mistrans.
‘*apoyarse’
S2
Focus on
mistrans.
‘*tesis’
S3
Focus on
mistrans.
‘*Intentar’
TT1 and TT3
S5
Novice
Pos 3: 0.524
Pos 2: 0.298
Pos 2: 0.200
Pos 2: 0.679
Trained
Pos 1: 0.604
Pos 1: 0.306
Pos 1: 0.272
Pos 1: 0.848
Expert
Pos 2: 0.598
Pos 3: 0.190
Pos 3: 0.088
Pos 3: 0.594
Table 5. Focus on mistranslations and positions
Source. Elaborated by the authors
Monserrat Bermúdez Bausela y Tabea De Wille 239
Hikma 22(1) (2023), 219 - 248
4.2.2 Difficult phrases
We had assumed that the focus of participants was going to be on
lexical words, and this also applied to the FFP and the LFP in most cases.
However, in scenario 2 NoInfo, participants placed a strong focus (high
number of FFPs) on the word ‘thereby’, which is a discourse marker. A
possible reason for this focus is that discourse markers are quite frequent in
English but not so much in Spanish, which makes the word more challenging
to translate than other grammatical words. Similarly, we have observed a
focus on ‘get behind’ and possible equivalents, which does not have a unique
or straight-forward translation in Spanish.
Other examples of participants encountering difficult translations were
‘Think ahead’ in scenario 3 GoodInfo and in scenario 6 BadInfo ‘examen’/
‘exámenes’, maybe because in one version it has been translated as
‘exámenes presenciales’ and in the other two as ‘examen escrito’ and
‘exámenes escritos’.
We believe that the focus on ‘Think ahead’ and its translated versions:
‘Mira de cara’, ‘Pensar de cara’ and ‘Ten en mente’ is because of the difficulty
that the translation of ‘Think ahead’ entails, which does not have a
straightforward translation into Spanish due to the inner complexity of the
construction ‘verb + preposition’ from English into Spanish, in which the
preposition in English can indicate ‘action’ and in Spanish it cannot. The
suggested versions differ among one another and we believe that this is the
reason why more Fixation Points have been devoted to this keyword.
Participants also picked up on difficult words that were translated
correctly: In scenario 4 GoodInfo, both ‘available’ and its translation
‘disponibles’ were the most common keywords selected by all groups in
general, followed by ‘library’, ‘subscriptions’ and ‘resources’, along with their
translations. There is no particular linguistic issue regarding the choice of
these words, only to point out that in Spanish, both ‘subscripciones’ and
‘suscripciones’ are correct spellings.
Regarding scenario 6 BadInfo, it is interesting to note several aspects.
The first keyword is ‘written exam’ and its suggested translations. The second
keyword, ‘to take’, also presents different variants. In this regard, we have
been able to determine that when a specific word presents different
translations, it represents a focus of difficulty or a challenge for the participant,
something that we can appreciate looking at the DFPs over those variants.
This is the same case with keyword 3 ‘modules’ and its equivalents ‘módulos’
and ‘unidades’. This time, keyword 4 has been the word ‘Experienced’, which
is part of the extrinsic information provided to the participant, but only for this
specific one, not for the other two of them, which leads us to think that the
240 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
participant was particularly interested in the translation given by the
Experienced translator (even though it had been done by a Novice translator)
(Table 4).
4.2.3. Meta words
Overall, participants have spent less cognitive effort on meta words
(extrinsic information about professional status of the translator) than on
lexical or grammatical words, by all our measures. However, given that the
information of meta words is less complex and does not require comparison
between different versions or with a ST, we do not think that this is a very
meaningful observation in itself. Instead, we focus our analysis on the
comparison between participant groups and scenarios rather than absolute
cognitive effort. When extrinsic information was first provided in scenario 3
GoodInfo, it became the centre of attention for all groups of participants, likely
due to the novelty of additional information being introduced. However, this
effect seems to have worn off by the next scenario, where we observed low
FFPs on extrinsic information (Scenario 4, GoodInfo).
We also observed differences between participant groups:
While all groups were influenced by the introduction of extrinsic
information, this was particularly the case for experts in BadInfo scenarios.
For the LFPs, meta words were more frequently the focus in the case of
Novices and Experts. However, extrinsic information was the FFP for the
BadInfo scenarios in few instances.
Extrinsic information also had an impact on the number of KSMs for
each group:
1) Novice students seem to have a preference for meta words in
GoodInfo scenarios, while for BadInfo scenarios they show an
equal preference for lexical and meta words.
2) Trained students, given GoodInfo scenarios, express an equal
preference for lexical and meta words, while in BadInfo scenarios,
they choose lexical words.
3) Experts show a preference for meta words in GoodInfo scenarios
and there is almost a tie between lexical words and meta words for
BadInfo scenarios.
Monserrat Bermúdez Bausela y Tabea De Wille 241
Hikma 22(1) (2023), 219 - 248
5. DISCUSSION
5.1 Common patterns in the behaviour of all groups of participants
H01.1. There is a preference for certain words as the focus of attention.
This has been so for all groups of participants. Several reasons may be behind
this preference:
• Words that do not have a unique translation and, therefore,
have had more than one equivalent. Example: ‘dissertation’.
• Combination of words in English whose structure for a Spanish
speaker is quite rare and, therefore, a direct or literal translation
is not possible. This would be the case of ‘Think ahead’.
• Words which have been mistranslated, such as ‘demanding’ for
‘*demandado’.
Other common patterns that have also been observed are the following:
• Participants chose the same words as FFPs. There were a lot
of similarities too for LFPs, even though there is a greater
variety of words in this latter case. Apart from this, it is worth
noting that most of the FFPs have been in English (ST), while
the LFPs have been mostly from the three translated versions
(TTs).
• Participants spend more time on the LFP than on the FFP. We
believe that after having processed all the information on the
slide, participants know what they want to focus on. This was
reinforced by the expectation of having to state their
preference, so it might make sense to dwell a bit longer on the
last word.
H01.2 Participants compare keywords, sometimes in a sequential
manner. This expectation has been true. Participants, in general, focus on one
word and like to compare them in the different versions. This has sometimes
been done in a sequential way, and the study of keywords shows that, even if
not always sequential, the interest in some particular words is common due to
the long durations that they register.
Let us remember that Rayner (1998, p. 390) used the textual variable
«semantic relationships between words» and, for the purposes of our own
study, we have assumed an «equivalent relationship between words in the ST
and in the TTs» that the participants have compared and looked at.
H01. 3 Participants choose the version they have looked at in the last
place. This has been the case in 84% of the cases, so it is an expectation that
has turned out to be true. There is a clear correlation between the LFP and
242 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
the version finally chosen by the participant, regardless of the type of scenario
and participant group.
H01.4 Given NoInfo, participants expend more cognitive effort on lexical
words than grammatical words. This has been so in the vast majority of the
cases. We have only observed one exception with the discourse marker
‘thereby’.
We conclude that, when processing lexical features, there were
common patterns in the behaviour of all groups of participants. One of our
objectives was to study whether participants have a tendency to focus on
some particular words. We expected that lexical words would accumulate
longer fixation durations than grammatical words, similar to Rayner’s (1998)
assumption that readers’ gaze durations are longer on low-frequency words
than on high-frequency words, and have shown this to be the case.
5.2 The impact of training on the behaviour of participants: cognitive effort
is higher in the group of students than in the group of experts
H02.1 First Fixation Points and Last Fixation Points require more
cognitive effort on the part of the group of students. The average of durations
for FFPs and LFPs has revealed that, in general terms and regardless of the
scenario (NoInfo, GoodInfo or BadInfo), it is the experts who spend less time
on average on the FFP, while it is the trained participants who devote more
time to it, and novice students are somewhere in between.
H02.2 Mistranslations and difficult to translate words require more
cognitive effort on the part of the group of students, while experts are more
efficient (that is, they spend less time on those units, which translates into less
cognitive effort). We have been able to identify that the group of trained
participants have spent longer, on average, on the mistranslated words than
experts, who are the ones spending the least on them (and novices were
generally in the middle between the other two groups), which we believe
confirms the skill acquisition model proposed by Dreyfus (2004), which
describes the emotional involvement of learners in intermediate stages, which
leads them to spend more cognitive effort than novice learners, while experts
are the most efficient. Their nonreflective involvement and intuition are
dominant. They are self-aware of their own profession (Chesterman, 2016, p.
161). That would easily explain why they require less cognitive effort in the
assessment task: they are more confident in themselves. A key concept here
would be ‘conscious engagement’ (Chesterman, 2016, pp. 148-149) in
relation to cognitive effort. It has turned out to be true that the participants
engaged in ‘deliberative rationality’ when the situation required it (words that
posed a difficulty in translation and mistranslations). We believe that where
there is deliberative rationality there is higher cognitive effort, and this could
Monserrat Bermúdez Bausela y Tabea De Wille 243
Hikma 22(1) (2023), 219 - 248
be the reason why trained students require more cognitive effort than the other
two groups: they have required critical conscious engagement. For them, it is
no longer ‘information processing’, but also and mainly goal-oriented decision
making and problem solving; however, they do not have yet the confidence
and experience of experts. Our objective of studying the cognitive effort in the
processing of lexical features through the use of eye-tracking methodology
has been accomplished. Rayner (1998) pointed to the textual variable ‘lexical
ambiguity’ to explain that readers increase their fixation time on ambiguous
words, which, in our case, are words that are particularly difficult to translate
and mistranslations. Schaeffer et al. (2019) reached a similar inference in their
study, concluding that professional translators are more efficient in terms of
error recognition and the evaluation process in general, as they have the
needed skills that help them prioritise, evaluate and apply the adequate
strategies. Other studies that support the fundamental points of our findings
are those conducted by Schaeffer et al. (2017), Jakobsen and Hvelplund
(2008) or Dragsted and Carl (2013).
Also, we have observed a higher number of KSMs regarding novices
and trained students, which translates into a higher cognitive effort than in the
case of experts. Students have devoted more time comparing words than
experts, registering the highest number of movements.
We can conclude that having received formal training has had an
impact on the behaviour of participants when they are reading and assessing
the three different translated versions. We have observed a link between
formal training and the amount of cognitive effort, as observed in the
behaviour of participants, and a close relationship between the Dreyfus and
Dreyfus model, the concept of ‘conscious awareness’ and cognitive effort.
5.3 Extrinsic information as an element of disruption
H03.1 When meta words are included in the scenarios, participants pay
attention to them, that is, they register long durations and are included in
sequential comparisons. This has been true in all scenarios and in all
situations, registering a long duration of Fixation Points and Keyword
Sequential Movements where participants frequently compared them in
sequential order. Even though lexical words have registered the highest
number of durations, we also observed cognitive effort expended on meta
words. We saw this in the study of keywords, where for scenarios 4 GoodInfo
and 6 BadInfo, meta words were also keywords (particularly the meta word
‘Experienced’ and its cluster ‘Experienced Translation’), which is indicative of
the fact that the participant was particularly interested in this piece of
information.
244 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
We have observed that novice students have a clear preference for
meta words in this comparison; trained students focus slightly more on lexical
words and experts have an equal preference for lexical words and meta words
for KSMs. Across all groups, participants especially paid attention to meta
information in the first scenario it was introduced. Overall, they favoured
lexical words, followed by grammatical words as FFPs and meta words in
LFPs.
If we focus on the participants' choice, we can see that all groups have
been influenced by the introduction of meta words in the GoodInfo and
BadInfo scenarios. We would like to point out that the influence of extrinsic
information over the choice of participants for one particular translation has
been dealt with previously. In De Wille and Bermúdez-Bausela (2018), the
researchers found out that those participants with higher levels of expertise
were less influenced by extrinsic information (the translator’s professional
background) than those with lower levels of expertise. Expert participants
overall selected the expert translation, at the same time that they took into
account both extrinsic information and intrinsic attributes. However, when they
observed a discrepancy between both, they relied more heavily on the intrinsic
attributes, discarding the extrinsic cues. On the other hand, there was a
greater element of disagreement among trained participants, who were
situated in the mid-range between experts and novices in terms of reliance on
extrinsic cues. Finally, novice participants were heavily influenced by extrinsic
information, being the only group that chose the false expert translations
(BadInfo) more often that the actual expert translations. All this indicates that
certainty in the evaluation process is linked to the levels of expertise and the
training received.
C
ONCLUSION
One of our initial research questions was whether having received
formal training and the level of expertise influences the way in which one
approaches the translation evaluation process in terms not only of agility but
also in terms of decisiveness, efficiency and how influenceable one might be.
Also, we wanted to go a step beyond and see whether this was linked to the
model of adult skill acquisition. We worked with a sample of participants
(novice, trained and experts) and we measured the amount of cognitive effort
that each group employed in the processing of lexical features through an eye-
tracking methodology. The study has allowed us to detect certain trends, such
as the fact that experts spent the least amount of time on the evaluation,
followed by novice, while trained participants expended the most amount of
time; or the long fixation points over difficult phrases and mistranslations,
which required more cognitive effort on the part of the students than on the
experts (efficiency). However, and notwithstanding the previous findings, we
Monserrat Bermúdez Bausela y Tabea De Wille 245
Hikma 22(1) (2023), 219 - 248
observed, as researches before us did, that not everything was different in the
behaviour of groups, as there were common features shared by all of them,
such as the fact that certain words caught their attention, or that they all liked
to compare them in the different versions (even sequentially), not to speak of
the effect and influence that extrinsic information (meta words) exercised on
the groups, particularly the first time that the cues appeared on the slides.
While the number of participants in our study (24) was too small to be
able to generalize the findings, we note that our participant sample was similar
in size to that of other studies as described for example in Dragsted and Carl
(2013), O’Brien (2009), Pavlović and Hvelplund (2009), and Schaeffer et al.
(2019). Based on our experience with eye-tracking as a data collection
method and subsequent analysis of the large volume of data generated, we
believe that this method is highly relevant for the development of hypotheses
and observation of tendencies, but would be difficult to realize with participant
numbers that allow for general observations.
We would like to make a final observation, which also relates to the
question of generalizability of our findings: Although it is quite evident that
there are different tendencies between the groups of novices, trained and
experts, it would be too simplistic to limit or explain it all in terms of groups of
participants. Within each group we have been able to ascertain some common
behaviours among certain individuals that were common throughout the three
groups. Bearing in mind that we worked with a closed group of participants,
well-known to us, we believe that it is not only a matter of professional
background, expertise or amount of training that defines the behaviour of a
participant, but also the individual character and personality of each of them,
which has an impact on the participant behaviour. Along the same lines as
Dragsted and Carl (2013), in several moments of our own research, we have
seen novices behaving similarly to what would be expected from experts and
the other way around. Let us remember as well that these authors pointed to
the fact that translators are characterised by their individual profiles and that
their behaviour remains quite constant regardless of the complexity of the
texts and they leave open the possibility that this might also be so regarding
other external factors. Nonetheless, the overall conclusion is that training has
been a determining factor in our study; groups of participants have been
recognized in their behaviour while processing lexical features, but there have
been important common patterns too. In this process, extrinsic information has
exerted an influence in the decisions, in the amount of cognitive effort and, in
general, how those lexical features have been processed. As a closing
reflection to this section, we believe that we can talk about coherent behaviour
in the assessment process and that eye-tracking has proved to be very useful
in attaining our objectives.
246 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
REFERENCES
Chesterman, A. (2016). Memes of translation: The spread of ideas in
translation theory. John Benjamins.
De Wille, T., & Bermúdez-Bausela, M. (2018). Quality Perceptions and
Professional Status in Translation. Clina: Revista Interdisciplinaria de
Traducción, Interpretación y Comunicación Intercultural, 4(2), 103-122.
doi:10.14201/clina201842103122
Dragsted, B., & Carl, M. (2013). Towards a classification of translation styles
based on eye-tracking and keylogging data. Journal of Writing
Research, 5(1), 133-158. doi:10.17239/jowr-2013.05.01.6
Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind Over Machine. The Free Press.
Dreyfus, S. E. (2004). The Five-Stage Model of Adult Skill Acquisition. Bulletin
of Science, Technology & Society, 24(3), 177-181
https://www.bumc.bu.edu/facdev-medicine/files/2012/03/Dreyfus-skill-
level.pdf
Frazier, L., & Rayner, K. (1982). Making and correcting errors during sentence
comprehension: Eye movements in the analysis of structurally
ambiguous sentences. Cognitive Psychology, 14, 178-210.
Hurtado Albir, A. (2015). The Acquisition of Translation Competence.
Competences, Tasks, and Assessment in Translator Training. Meta,
60(2), 256-280. doi:10.7202/1032857ar
Hvelplund, K. T. (2017). Eye Tracking in Translation Process Research. In J.
W. Schwieter & A. Ferreira (Eds.), The Handbook of Translation and
Cognition (pp. 248-264). John Wiley & Sons.
Jakobsen, A. L., & Hvelplund Jensen, K. T. (2008). Eye movement behaviour
across four different types of reading task. In S. Göpferich, A. L.
Jakobsen & I. M. Mees (Eds.), Looking at Eyes: Eye-Tracking Studies
of Reading and Translation Processing (pp. 103-124).
Samfundslitteratur Press. doi:10.7202/1011266ar
Just, M.A., & Carpenter, P.A. (1980). A Theory of Reading: From Eye
Fixations to Comprehension. Psychological Review, 87(4), 329-354.
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.677.6396&
rep=rep1&type=pdf
Kelly, D. (2002). Un modelo de competencia traductora: bases para el diseño
curricular. Puentes, 1, 9-20. http://wpd.ugr.es/~greti/revista-
puentes/pub1/02-Kelly.pdf
Monserrat Bermúdez Bausela y Tabea De Wille 247
Hikma 22(1) (2023), 219 - 248
Muñoz Martín, R. (2014). A blurred snapshot of advances in translation
process research. MonTI Special Issue - Minding Translation, 49-84.
doi:10.6035/MonTI.2014.ne1.1
O’Brien, S. (2009). Eye tracking in translation process research:
methodological challenges and solutions. In I.M. Mees, F. Alves & S.
Göpferich (Eds.), Methodology, Technology and Innovation in
Translation Process Research (pp. 251-266). Samfundslitteratur.
PACTE. (2000). Acquiring translation competence: Hypotheses and
methodological problems in a research project. In A. Beeby, D.
Ensinger & M. Presas (Eds.), Investigating translation (pp. 99-106).
John Benjamins.
PACTE. (2003). Building a Translation Competence model. In F. Alves (Ed.),
Triangulating Translation: Perspectives in Process Oriented Research
(pp. 43-66). John Benjamins.
Paloposki, O. (2016). Translating and translators before the professional
project. Special issue of JoSTrans, 25, 15-32.
https://www.jostrans.org/issue25/art_paloposki.pdf
Pavlović, N., & Hvelplund, K. T. (2009). Eye tracking translation directionality.
Translation research projects, 2, 93-109.
http://www.intercultural.urv.cat/media/upload/domain_317/arxius/TP2/j
ensenpavlovic.pdf
Pym, A. (2011). Translation research terms: a tentative glossary for moments
of perplexity and dispute. In A. Pym (Ed.), Translation research projects
3, 75-110. Intercultural Studies Group.
Pym, A, Orrego-Carmona, D., & Torres-Simón, E. (2016). Status and
technology in the professionalization of translators. Market disorder and
the return of hierarchy. The Journal of Specialised Translation, 25, 33-
53. https://www.jostrans.org/issue25/art_pym.pdf
Rayner, K. (1998). Eye Movements in Reading and Information Processing:
20 Years of Research. Psychological Bulletin, 124(3), 372-422.
http://www.infinitychallenge.com/clamlist/Rayner_1998.pdf
Saldanha, G., & O’Brien, S. (2014). Research Methodologies in Translation
Studies. Routledge
Schäffner, C., & Shuttleworth, M. (2013). Metaphor in translation: Possibilities
for process research. Target, 25(1), 93-106.
https://doi.org/10.1075/target.25.1.08shu
248 An Eye-Tracking Study of Cognitive Effort in Processing of Lexical Features […]
Hikma 22(1) (2023), 219 - 248
Schaeffer, M., Paterson, K. B., McGowan, V. A., White, S. J., & Malmkjaer, K.
(2017). Reading for translation. In A. L. Jakobsen & B. Mesa-Lao (Eds.),
Translation in Transition: Between cognition, computing and technology
(pp. 17-53). John Benjamins.
Schaeffer, M., Nitzke, J., Tardel, A., Oster, K., Gutermuth, S., & Hansen-
Schirra, S. (2019). Eye-tracking revision processes of translation
students and professional translators. Perspectives, 27(4), 589-603.
doi:10.1080/0907676X.2019.1597138
Schwanenflugel, P. J., & LaCount, K. L. (1988). Semantic relatedness and the
scope of facilitation for upcoming words in sentences. Journal of
Experimental Psychology: Learning, Memory, and Cognition, (14)2,
344-354.
Sirohi, N., McLaughlin, E. W., & Wittink, D. R. (1998). A Model of Consumer
Perceptions and Store Loyalty Intentions for a Supermarket Retailer.
Journal of Retailing 74(2), 223-245.
Vantamay, S. (2007). Understanding of Perceived Product Quality: Reviews
and Recommendations. BU Academic Review, (6)1, 110-117.
https://www.bu.ac.th/knowledgecenter/epaper/jan_june2007/Somphol.
pdf
Walker, C. (2021). An Eye-Tracking Study of Equivalent Effect in Translation.
The Reader Experience of Literary Style. Palgrave Macmillan.
Zeithaml, V. A. (1988). Consumer Perceptions of Price, Quality, and Value: A
Means-End Model and Synthesis of Evidence. The Journal of
Marketing, 52(3), 2-22. https://doi.org/10.2307/1251446