Conference PaperPDF Available

The role of task engagement and persistence in learner's written task performance

Authors:
THE ROLE OF TASK ENGAGEMENT AND
PERSISTENCE IN LEARNER’S WRITTEN
TASK PERFORMANCE
Ali H. Al-Hoorie
Phil Hiver
AAAL, March 2021
INTRODUCTION
Engagement: increasingly important in contemporary language learning research
Cognitive, emotional, behavioral (and sometimes social)
The “action verb”, or the tangible manifestation of motivation.
Learning occurs due to engagement not motivation.
Chain: Motivation => Engagement => Learning
Hiver, Al-Hoorie, and Mercer (2021); Hiver, Al-Hoorie, Vitta, and Wu (2021)
Mindset: one’s personal theory of intelligence; also increasingly important for the field.
Lou and Noels (2019)
Fixed mindset (entity theory): belief that ability is fixed, you either have it or you don’t [undesirable]
Growth mindset (incremental theory): belief that ability is malleable, if you don’t have it yet work
hard [desirable]
Papi et al. (2019)
METHOD
Participants
N= 396
Roughly 50% Colombian, 50% Chinese
College students
Learning English as L2
Snowballing sampling
METHOD
Design
METHOD
Instruments
Cognitive Engagement – refers to students’ level of investment of attention and effort in L2 learning, and
includes students’ self-regulated learning as they exert the necessary effort for comprehension of complex
ideas or mastery of difficult skills in the L2 domain.
Motivation and Engagement Scale (Martin, 2007)
5 items
Cronbachs α= .91, 95% CI [.90, .93]
McDonald’s ω= .92, 95% CI [.90, .93]
METHOD
Instruments
Fixed Mindset refers to the extent to which learners believe that language learning skill is innate or
unchangeable.
Papi et al.(2019) based on Dweck (1999)
4 items
Cronbachs α= .84, 95% CI [.82, .87]
McDonald’s ω= .85, 95% CI [.81, .88]
METHOD
Instruments
Baseline Tasks an accuracy task created using an adapted C-Test (Lamb, 2012) with no time pressure.
Accuracy operationalized as: a) ratio of correct lexical choices for content words (lexical accuracy) and b) ratio of the
number of errors to the total number of words (syntactic accuracy).
The C-Test typically tests a unidimensional general language proficiency, but one that draws heavily on lexical and
grammatical competence (Eckes & Grotjahn, 2007) which is why we use this as a proxy for accuracy. For these
reasons, we do not expect greater time on task to affect learners’ scores on this accuracy task.
Scoring: For this measure, we counted all accurate answers for a full point. No partial points were possible: responses
that are misspelled or grammatically incorrect received zero points. There are 75 total possible points in this section.
METHOD
Instruments
Effort Tasks 1 & 2 a fluency task that provides the learner with a set of thematically related content words
and requires them to write as many sentences as possible based on the words in the set.
Fluency operationalized as: a) amount and b) rate of language production
Scoring: For the measure of fluency-amount, we counted total number of words produced across all T-units. For the
measure of fluency-rate, we divided the total number of words produced by time on task.
METHOD
Instruments
Effort Task 3 a global accuracy task that provides learners with sets of five or six content and function
words and requires them to choose four words and use them to make a correct sentence.
Scoring: we counted all accurate answers for a full point (e.g., “He always goes camping”). Partial points (e.g., 0.75,
0.5, 0.25) were awarded for legitimate attempts to produce a response that contained grammatical errors (e.g., “He
goes to camping”).
Responses that were at the word level or below, not meaningful, or incoherent (e.g., ddddd”; “go my yes”; “travel”)
received zero points.
METHOD
Instruments
Frustration Task 1 a set of clearly unrelated content words (e.g., wood, Internet, magic) to write as many
sentences as possible based on the words in the set in only 45 seconds. Voluntarily clicking “CONTINUE”
instead of “FINISH” revealed additional sets of words. We used this measure as a proxy for learners’
persistence and effort under frustration-inducing task conditions.
Scoring: we scored each attempt to provide a response as one full point (total of 10 pts), and each voluntary click to
reveal the next set of words as one-half point (total of 5 points). There were 15 total possible points.
METHOD
Instruments
Frustration Task 2 provides learners with sets of eight colored content and function words (e.g.,
flooded the high night rain was easily house) and requires them to choose four words and use them to
make a correct sentence. However, learners cannot make two words of the same color touch each other in
any sentence they produce. We used this measure as an additional proxy for learners’ persistence and effort
under frustration-inducing task conditions.
Scoring: we scored each attempt to provide a legitimate response (four words placed in a correct order to make a
meaningful sentence, and no two words with the same color are touching) as one full point. A limited number of
possibilities exist for each of the 10 questions; any answers not listed among the range of possible answers received
zero (0) points. There were 10 possible points.
QUALITY OF LANGUAGE-PRODUCTION DVS
Coded using the Tool for the Automatic Analysis of Syntactic Sophistication and Complexity
(TAASSC), and the Tool for the Automatic Analysis of Lexical Diversity (TAALED).
Scoring: Syntactic complexity is examined through length of production units (i.e., mean length of T-unit
[MLTU]), phrasal complexity (i.e., number of coordinate phrases per T-unit [CP/T]), and amount of
subordination (i.e., number of dependent clauses per T-unit [DC/T]).
Scoring: Lexical complexity measures will include lexical diversity (i.e., Giraud index of content words
[RTTR_CW] and the measure of textual lexical diversity [MTLD_CW]).
METHOD
Data analysis
Multiple hierarch regression
Baseline Task in Step 1; Engagement and Mindset in Step 2
Assumptions checked; a few outliers excluded
RESULTS
RESULTS
Effort Tasks
Fixed Mindset:
Unexpected positive relationship with Accuracy
Expected positive relationship with Amount and Rate
As if learners with a fixed mindset are concerned with
accuracy at the expense of fluency.
Cognitive Engagement:
Expected positive relationship with Rate but not
Syntactic Complexity.
GREEN = expected, RED = unexpected,
YELLOW = borderline
RESULTS
Frustration Tasks
Fixed Mindset:
Expected negative relationship with Accuracy
As if they quit once the task become hard.
Cognitive Engagement:
Expected positive relationship with Accuracy,
Persistence, and Lexical Complexity.
GREEN = expected, YELLOW = unclear
DISCUSSION
Our field needs more attention to engagement in actual tasks, not just self-reports.
Proficiency is a serious confound that has not received much attention to date.
More than half of the results were non-significant; significant results tended to be weak (β< .20).
Fixed mindset learners seem to emphasize accuracy over fluency, but they quit once the task
becomes hard.
REFERENCES
Eckes, T. & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23, 290-325.
Hiver, P., Al-Hoorie, A., & Mercer, S. (Eds.). (2021). Engagement in the second language classroom. Multilingual Matters.
Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2021). Engagement in language learning: A systematic review of 20 years of research
methods and definitions. Language Teaching Research. Online First.
Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices
of syntactic sophistication (Doctoral Dissertation). Retrieved from http://scholarworks.gsu.edu/alesl_diss/35.
Kyle, K. & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly,
49, 757-786.
Lamb, M. (2012). A self-system perspective on young adolescents’ motivation to learn English in urban and rural settings. Language
Learning, 62, 9971023.
Lou, N. M., & Noels, K. A. (2019). Language mindsets, meaning-making, and motivation. In M. Lamb, K. Csizér, A. Henry, & S. Ryan (Eds.),
The Palgrave handbook of motivation for language learning (pp. 537559). Palgrave.
Martin, A.J. (2007). Examining a multidimensional model of student motivation and engagement using a construct validation approach.
British Journal of Educational Psychology, 77, 413-440.
Papi, M., Rios, A., Pelt, H., & Ozdemir, E. (2019). Feedback-seeking behavior: Basic components and motivational antecedents. The
Modern Language Journal, 103, 205226.
THANK YOU!
hoorie_a@jic.edu.sa
phiver@fsu.edu
... Noticeably, engagement is one of the constructs that has been repeatedly reported in the literature as related to students' perceived difficulty. However, reviewing literature regarding students' engagement reveals variations in its definitions and scopes (Bond et al., 2020;Hiver et al., 2021). Skinner et al. (2009), for example, conceptualized engagement as energized, directed and sustained actions. ...
Article
Submitted Currently, research on second language acquisition (SLA) and psychology of language learning has conceived foreign language learning as a complex system of beliefs, thoughts, emotions, and actions that contribute immensely to shape students’ learning experiences. The significance of such research lies in the fact that they provide a holistic perspective of FL learning by considering the multiple and interrelated interactions of beliefs, emotional and cognitive factors, rather than examining any single variable in isolation. Therefore, this study seeks to provide such insights by examining the interaction between perception of task difficulty and cognitive engagement in an EFL writing classroom, taking into consideration the possible effects of foreign language enjoyment (FLE) and mindset on such relationship. Data were collected from 218 Saudi undergraduate students using a questionnaire. Results revealed that: a) students’ perception of task difficulty is linked to their cognitive engagement; b) FLE acts as a buffer against the negative effects of difficult EFL writing tasks on students’ cognitive engagement; c) growth mindset has no moderating role in the relationship between perceived difficulty and cognitive engagement.
Article
Full-text available
At the turn of the new millennium, in an article published in Language Teaching Research in 2000, Dörnyei and Kormos proposed that ‘active learner engagement is a key concern’ for all instructed language learning. Since then, language engagement research has increased exponentially. In this article, we present a systematic review of 20 years of language engagement research. To ensure robust coverage, we searched 21 major journals on second language acquisition (SLA) and applied linguistics and identified 112 reports satisfying our inclusion criteria. The results of our analysis of these reports highlighted the adoption of heterogeneous methods and conceptual frameworks in the language engagement literature, as well as indicating a need to refine the definitions and operationalizations of engagement in both quantitative and qualitative research. Based on these findings, we attempted to clarify some lingering ambiguity around fundamental definitions, and to more clearly delineate the scope and target of language engagement research. We also discuss future avenues to further advance understanding of the nature, mechanisms, and outcomes resulting from engagement in language learning.
Book
Full-text available
This book defines engagement for the field of language learning and contextualizes it within existing work on the psychology of language learning and teaching. Chapters address broad substantive questions concerned with what engagement is or looks like, and how it can be theorized for the language classroom; methodological questions related to the design, measurement and analysis of engagement in language classrooms and beyond; as well as applied issues examining its antecedents, factors inhibiting and enhancing it, and conditions fostering the reengagement of language learners who have become disengaged. Through a mix of conceptual and empirical chapters, the book explores similarities and differences between motivation and engagement and addresses questions of whether, how and why learners actually do exert effort, allocate attention, participate and become involved in tangible language learning and use. It will serve as an authoritative benchmark for future theoretical and empirical research into engagement within the classroom and beyond, and will be of interest to anyone wishing to understand the unique insights and contributions the topic of engagement can make to language learning and teaching.
Article
Full-text available
This study investigated the concept of corrective feedback in second language learning as a learning resource, recasting it as feedback-seeking behavior. Dweck’s (1999) mindsets, Korn and Elliot’s (2016) achievement goals, and Ashford’s (1986) model of feedback-seeking behavior were re-operationalized in the context of language learning. Questionnaire data from 287 college students studying foreign languages in the United States confirmed that learners make calculated decisions regarding whether to seek feedback, by what method, and from what source, based on their own perceptions of the costs and values associated with different feedback-seeking strategies, which are, in turn, largely predicted by the learners’ language mindsets and achievement goals. Learners with a growth language mindset and development-approach goals sought feedback using both monitoring and inquiry methods and from teachers and others. Learners with a fixed language mindset and demonstration goals sought feedback only by method of inquiry but from different sources depending on the valence (approach vs. avoidance) of their goals.
Article
Full-text available
This study seeks to examine a multidimensional model of student motivation and engagement using within- and between-network construct validation approaches. The study tests the first- and higher-order factor structure of the motivation and engagement wheel and its corresponding measurement tool, the Motivation and Engagement Scale - High School (MES-HS; formerly the Student Motivation and Engagement Scale). The study draws upon data from 12,237 high school students from 38 Australian high schools. The hypothesized 11-factor first-order structure and the four-factor higher-order structure, their relationship with a set of between-network measures (class participation, enjoyment of school, educational aspirations), factor invariance across gender and year-level, and the effects of age and gender are examined using confirmatory factor analysis and structural equation modelling. In terms of within-network validity, (1) the data confirm that the 11-factor and higher-order factor models of motivation and engagement are good fitting and (2) multigroup tests showed invariance across gender and year levels. In terms of between-network validity, (3) correlations with enjoyment of school, class participation and educational aspirations are in the hypothesized directions, and (4) girls reflect a more adaptive pattern of motivation and engagement, and year-level findings broadly confirm hypotheses that middle high school students seem to reflect a less adaptive pattern of motivation and engagement. The first- and higher-order structures hold direct implications for educational practice and directions for future motivation and engagement research.
Article
Full-text available
This study examined the motivation to learn English of Indonesian junior high school pupils, 13–14 years of age, in three distinct contexts: a metropolitan city, a provincial town, and a rural district. Utilizing Dörnyei's second language (L2) Motivational Self System as the theoretical framework, this study employed a 50-item questionnaire to sample the views of 527 learners, and a C-test to measure their current proficiency in English. Motivation was found to be similar in strength and character in the two urban settings but significantly different in the rural setting. A positive view of the experience of learning English was the strongest predictor of both motivated learning behavior and L2 proficiency, whereas the Ideal L2 self was only a significant factor among the metropolitan group.
Article
This study explores the construct of lexical sophistication and its applications for measuring second language lexical and speaking proficiency. In doing so, the study introduces the Tool for the Automatic Analysis of LExical Sophistication (TAALES), which calculates text scores for 135 classic and newly developed lexical indices related to word frequency, range, bigram and trigram frequency, academic language, and psycholinguistic word information. TAALES is freely available; runs on Windows, Mac, and Linux operating systems; and has a simple graphic user interface that allows for batch processing of .txt files. The tool is fast, reliable, and outputs results to a comma-separated value file that can be accessed using spreadsheet software. The study examines the ability of TAALES indices to explain the variance in human judgments of lexical proficiency and speaking proficiency for second language (L2) learners. Overall, these indices were able to explain 47.5% of the variance in holistic scores of lexical proficiency and 48.7% of the variance in holistic scores of speaking proficiency. This study has important implications for second language acquisition, for assessing L2 learners' productive skills (writing and speaking), and for L2 pedagogy. Limitations and future directions are also discussed.
Article
What C-tests actually measure has been an issue of debate for many years. In the present research, the authors examined the hypothesis that C-tests measure general language proficiency. A total of 843 participants from four independent samples took a German C-test along with the TestDaF (Test of German as a Foreign Language). Rasch measurement modelling and confirmatory factor analysis provided clear evidence that the C-test in question was a highly reliable, unidimensional instrument, which measured the same general dimension as the four TestDaF sections: reading, listening, writing and speaking. Moreover, the authors showed that language proficiency was divisible into more specific constructs and that examinee proficiency level differentially influenced C-test performance. The findings have implications for the multicomponentiality and fluidity of the C-test measurement construct.
Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication (Doctoral Dissertation
  • K Kyle
• Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication (Doctoral Dissertation). Retrieved from http://scholarworks.gsu.edu/alesl_diss/35.
The Palgrave handbook of motivation for language learning
  • N M Lou
  • K A Noels
• Lou, N. M., & Noels, K. A. (2019). Language mindsets, meaning-making, and motivation. In M. Lamb, K. Csizér, A. Henry, & S. Ryan (Eds.), The Palgrave handbook of motivation for language learning (pp. 537-559). Palgrave.