Content uploaded by Ali H. Al-Hoorie
Author content
All content in this area was uploaded by Ali H. Al-Hoorie on Mar 28, 2021
Content may be subject to copyright.
THE ROLE OF TASK ENGAGEMENT AND
PERSISTENCE IN LEARNER’S WRITTEN
TASK PERFORMANCE
Ali H. Al-Hoorie
Phil Hiver
AAAL, March 2021
INTRODUCTION
•Engagement: increasingly important in contemporary language learning research
•Cognitive, emotional, behavioral (and sometimes social)
•The “action verb”, or the tangible manifestation of motivation.
•Learning occurs due to engagement not motivation.
•Chain: Motivation => Engagement => Learning
•Hiver, Al-Hoorie, and Mercer (2021); Hiver, Al-Hoorie, Vitta, and Wu (2021)
•Mindset: one’s personal theory of intelligence; also increasingly important for the field.
•Lou and Noels (2019)
•Fixed mindset (entity theory): belief that ability is fixed, you either have it or you don’t [undesirable]
•Growth mindset (incremental theory): belief that ability is malleable, if you don’t have it yet work
hard [desirable]
•Papi et al. (2019)
METHOD
•Participants
•N= 396
•Roughly 50% Colombian, 50% Chinese
•College students
•Learning English as L2
•Snowballing sampling
METHOD
•Design
METHOD
•Instruments
•Cognitive Engagement – refers to students’ level of investment of attention and effort in L2 learning, and
includes students’ self-regulated learning as they exert the necessary effort for comprehension of complex
ideas or mastery of difficult skills in the L2 domain.
•Motivation and Engagement Scale (Martin, 2007)
•5 items
•Cronbach’s α= .91, 95% CI [.90, .93]
•McDonald’s ω= .92, 95% CI [.90, .93]
METHOD
•Instruments
•Fixed Mindset –refers to the extent to which learners believe that language learning skill is innate or
unchangeable.
•Papi et al.(2019) based on Dweck (1999)
•4 items
•Cronbach’s α= .84, 95% CI [.82, .87]
•McDonald’s ω= .85, 95% CI [.81, .88]
METHOD
•Instruments
•Baseline Tasks –an accuracy task created using an adapted C-Test (Lamb, 2012) with no time pressure.
•Accuracy operationalized as: a) ratio of correct lexical choices for content words (lexical accuracy) and b) ratio of the
number of errors to the total number of words (syntactic accuracy).
•The C-Test typically tests a unidimensional general language proficiency, but one that draws heavily on lexical and
grammatical competence (Eckes & Grotjahn, 2007) which is why we use this as a proxy for accuracy. For these
reasons, we do not expect greater time on task to affect learners’ scores on this accuracy task.
•Scoring: For this measure, we counted all accurate answers for a full point. No partial points were possible: responses
that are misspelled or grammatically incorrect received zero points. There are 75 total possible points in this section.
METHOD
•Instruments
•Effort Tasks 1 & 2 –a fluency task that provides the learner with a set of thematically related content words
and requires them to write as many sentences as possible based on the words in the set.
•Fluency operationalized as: a) amount and b) rate of language production
•Scoring: For the measure of fluency-amount, we counted total number of words produced across all T-units. For the
measure of fluency-rate, we divided the total number of words produced by time on task.
METHOD
•Instruments
•Effort Task 3 –a global accuracy task that provides learners with sets of five or six content and function
words and requires them to choose four words and use them to make a correct sentence.
•Scoring: we counted all accurate answers for a full point (e.g., “He always goes camping”). Partial points (e.g., 0.75,
0.5, 0.25) were awarded for legitimate attempts to produce a response that contained grammatical errors (e.g., “He
goes to camping”).
•Responses that were at the word level or below, not meaningful, or incoherent (e.g., “ddddd”; “go my yes”; “travel”)
received zero points.
METHOD
•Instruments
•Frustration Task 1 –a set of clearly unrelated content words (e.g., wood, Internet, magic) to write as many
sentences as possible based on the words in the set in only 45 seconds. Voluntarily clicking “CONTINUE”
instead of “FINISH” revealed additional sets of words. We used this measure as a proxy for learners’
persistence and effort under frustration-inducing task conditions.
•Scoring: we scored each attempt to provide a response as one full point (total of 10 pts), and each voluntary click to
reveal the next set of words as one-half point (total of 5 points). There were 15 total possible points.
METHOD
•Instruments
•Frustration Task 2 –provides learners with sets of eight colored content and function words (e.g.,
flooded the high night rain was easily house) and requires them to choose four words and use them to
make a correct sentence. However, learners cannot make two words of the same color touch each other in
any sentence they produce. We used this measure as an additional proxy for learners’ persistence and effort
under frustration-inducing task conditions.
•Scoring: we scored each attempt to provide a legitimate response (four words placed in a correct order to make a
meaningful sentence, and no two words with the same color are touching) as one full point. A limited number of
possibilities exist for each of the 10 questions; any answers not listed among the range of possible answers received
zero (0) points. There were 10 possible points.
QUALITY OF LANGUAGE-PRODUCTION DVS
•Coded using the Tool for the Automatic Analysis of Syntactic Sophistication and Complexity
(TAASSC), and the Tool for the Automatic Analysis of Lexical Diversity (TAALED).
•Scoring: Syntactic complexity is examined through length of production units (i.e., mean length of T-unit
[MLTU]), phrasal complexity (i.e., number of coordinate phrases per T-unit [CP/T]), and amount of
subordination (i.e., number of dependent clauses per T-unit [DC/T]).
•Scoring: Lexical complexity measures will include lexical diversity (i.e., Giraud index of content words
[RTTR_CW] and the measure of textual lexical diversity [MTLD_CW]).
METHOD
•Data analysis
•Multiple hierarch regression
•Baseline Task in Step 1; Engagement and Mindset in Step 2
•Assumptions checked; a few outliers excluded
RESULTS
RESULTS
•Effort Tasks
•Fixed Mindset:
•Unexpected positive relationship with Accuracy
•Expected positive relationship with Amount and Rate
•As if learners with a fixed mindset are concerned with
accuracy at the expense of fluency.
•Cognitive Engagement:
•Expected positive relationship with Rate but not
Syntactic Complexity.
•GREEN = expected, RED = unexpected,
•YELLOW = borderline
RESULTS
•Frustration Tasks
•Fixed Mindset:
•Expected negative relationship with Accuracy
•As if they quit once the task become hard.
•Cognitive Engagement:
•Expected positive relationship with Accuracy,
Persistence, and Lexical Complexity.
•GREEN = expected, YELLOW = unclear
DISCUSSION
•Our field needs more attention to engagement in actual tasks, not just self-reports.
•Proficiency is a serious confound that has not received much attention to date.
•More than half of the results were non-significant; significant results tended to be weak (β< .20).
•Fixed mindset learners seem to emphasize accuracy over fluency, but they quit once the task
becomes hard.
REFERENCES
•Eckes, T. & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23, 290-325.
•Hiver, P., Al-Hoorie, A., & Mercer, S. (Eds.). (2021). Engagement in the second language classroom. Multilingual Matters.
•Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2021). Engagement in language learning: A systematic review of 20 years of research
methods and definitions. Language Teaching Research. Online First.
•Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices
of syntactic sophistication (Doctoral Dissertation). Retrieved from http://scholarworks.gsu.edu/alesl_diss/35.
•Kyle, K. & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly,
49, 757-786.
•Lamb, M. (2012). A self-system perspective on young adolescents’ motivation to learn English in urban and rural settings. Language
Learning, 62, 997–1023.
•Lou, N. M., & Noels, K. A. (2019). Language mindsets, meaning-making, and motivation. In M. Lamb, K. Csizér, A. Henry, & S. Ryan (Eds.),
The Palgrave handbook of motivation for language learning (pp. 537–559). Palgrave.
•Martin, A.J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach.
British Journal of Educational Psychology, 77, 413-440.
•Papi, M., Rios, A., Pelt, H., & Ozdemir, E. (2019). Feedback-seeking behavior: Basic components and motivational antecedents. The
Modern Language Journal, 103, 205–226.
THANK YOU!
hoorie_a@jic.edu.sa
phiver@fsu.edu