No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Exploring multiple profiles of highly rated learner composition
Abstract
Recent research has come a long way in describing the linguistic features of large samples of written texts, although a satisfactory description of L2 writing remains problematic. Even when variables such as proficiency, language background, topic, and audience have been controlled, straightforward predictive relationships between linguistic variables and quality ratings have remained elusive, and perhaps they always will. We propose a different approach. Rather than assuming a linear relationship between linguistic features and quality ratings, we explore multiple profiles of highly rated timed compositions and describe how they compare in terms of their lexical, grammatical, and discourse features. To this end, we performed a cluster analysis on two sets of timed compositions to examine their patterns of use of several linguistic features. The purpose of the analysis was to investigate whether multiple profiles (or clusters) would emerge among the highly rated compositions in each data set. This did indeed occur. Within each data set, the profiles of highly rated texts differed significantly. Some profiles exhibited above-average levels for several linguistic features, whereas others showed below-average levels. We interpret the results as confirming that highly rated texts are not at all isometric, even though there do appear to be some identifiable constraints on the ways in which highly rated timed compositions may vary.
... In order to investigate the internal variation of these texts, a broad base of linguistic features can first be examined. The texts can then be grouped based on their use of these features into multiple profiles (Jarvis, Grant, Bikowski, & Ferris, 2003). ...
... These profiles denote groups of texts that show within group similarity in regards to the use of a set of linguistic features but between group difference (Jarvis et al., 2003). Multiple profiles have previously been identified in academic texts (Friginal, Li, & Weigle, 2014;Jarvis et al., 2003: ). ...
... These profiles denote groups of texts that show within group similarity in regards to the use of a set of linguistic features but between group difference (Jarvis et al., 2003). Multiple profiles have previously been identified in academic texts (Friginal, Li, & Weigle, 2014;Jarvis et al., 2003: ). Friginal et al. (2014), building on Jarvis et al. (2003), provide a broad base of features that can be used to help discern different textual profiles. ...
Writing about art and the teaching of art writing are becoming important issues in the art world. Furthering our understanding of art texts can inform the practice and teaching of art writing. In an effort to expand linguistic understanding of art writing, this study examines a corpus of 180 online explanatory art museum texts from the online collections of nine US museums with a functional profiles perspective. Cluster analysis is used to group the 180 texts into functional profiles based on their use of 21 linguistic features. The cluster analysis resulted in five clusters that are interpreted functionally: Cluster 1 (n=56), descriptive information; Cluster 2 (n=30), expanded form-adding interpretation and process; Cluster 3 (n=40), contextualizing; Cluster 4 (n=8), process and interpretation with agency; and Cluster 5 (n-46), narrative focus. It is posited that this understanding of online explanatory art museum texts can inform the teaching of future museum professionals.
... Given these findings, at least within the context of this study, the prospect of using such automated evaluation for high stakes summative assessment decisions is clearly untenable. This finding is not unexpected in light of previous work that has made clear the difficulty of establishing linear relationships between the linguistic features of texts and assessments of L2 writing quality (Bulté & Housen, 2014;Crossley & McNamara, 2014;Jarvis, Grant, Bikowski, & Ferris, 2003). Moreover, computer evaluations of written texts cannot account for all linguistic, semantic, or discourse features that contribute to the quality of a text such as metaphor or disciplinary knowledge (Graesser et al., 2011). ...
... p < .001). This may represent an instance of what Jarvis et al. (2003) refer to as complementarity, where although there are "a number of linguistic features that contribute to the overall quality of a written text, high levels of some features may bring about low levels of other features" (p. 399). ...
... Such corpora can include learners' texts written in response to a range of tasks, the automated indices of the texts, and measures of locally relevant human assessments of text quality. As there are various contextual factors that influence the patterns of linguistic features used in student texts (Jarvis et al., 2003;Weigle & Friginal, 2015), it makes sense that the practical application of automated evaluation in the language classroom needs to be based on data gathered at the local level. Over time, an analysis of such corpora may enable the context-specific relationships between automated indices and human evaluations of text quality to be more fully understood and more effectively applied. ...
Despite the current potential to use computers to automatically generate a large range of text-based indices, many issues remain unresolved about how to apply these data in established language teaching and assessment contexts. One way to resolve these issues is to explore the degree to which automatically generated indices, which are reflective of key measures of text quality, align with parallel measures derived from locally relevant, human evaluations of texts. This study describes the automated evaluation of 104 English as a second language texts through use of the computational tool Coh-Metrix, which was used to generate indices reflecting text cohesion, lexical characteristics, and syntactic complexity. The same texts were then independently evaluated by two experienced human assessors through use of an analytic scoring rubric. The interrelationships between the computer and human generated evaluations of the texts are presented in this paper with a particular focus on the automatically generated indices that were most strongly linked to the human generated measures. A synthesis of these findings is then used to discuss the role that such automated evaluation may have in the teaching and assessment of second language writing.
... The features can be summarized as (1) noun types/lexical-word types, (2) complex nominals/clauses, (3) complex nominals/T-units, (4) total nominal clauses, and (5) total clauses embedded in noun phrases (Halliday, 1993;Halliday & Martin, 1993;Lu, 2011;Wolfe-Quintero, Inagaki, & Kim, 1998). Based on the features above, nouns, complex nominals, and clausal noun modifiers (e.g., relative clauses) have been empirically shown to be reliable indicators of L2 writing development (Ferris, 1994;Ginther, 2000;Jarvis, Grant;Bikowski, & Ferris, 2003;Lu, 2011). Nevertheless, the role of phrasal features in NPs in measuring L2 writing development is still not fully understood. ...
... The features can be summarized as (1) noun types/lexical-word types, (2) complex nominals/clauses, (3) complex nominals/T-units, (4) total nominal clauses, and (5) total clauses embedded in noun phrases (Halliday, 1993;Halliday & Martin, 1993;Lu, 2011;Wolfe-Quintero, Inagaki, & Kim, 1998). Based on the features above, nouns, complex nominals, and clausal noun modifiers (e.g., relative clauses) have been empirically shown to be reliable indicators of L2 writing development (Ferris, 1994;Ginther, 2000;Jarvis, Grant;Bikowski, & Ferris, 2003;Lu, 2011). Nevertheless, the role of phrasal features in NPs in measuring L2 writing development is still not fully understood. ...
... In a qualitative review of the Chinese students' essays, we noticed that the students with higher TOEFL writing scores used more relative clauses. This finding is consistent with previous studies, in which it has been demonstrated that L2 writing proficiency influences the frequency of relative clauses (Ferris, 1994;Ginther, 2000;Jarvis, Grant, Bikowski, & Ferris, 2003;Wolfro-Quintero, Inagaki, & Kim, 1998). A qualitative review of the Chinese students' writings also revealed that higher-proficiency students use more relative clauses with diverse relativizers (e.g., which, who, that, whose), whereas lowerproficiency students use fewer relative clauses and are limited to using which as relativizers. ...
See JEAP Volume 38, March 2019:
https://www.sciencedirect.com/science/article/pii/S1475158518300171
... In addition to the number of formal errors in the texts and the use of syntactic and cohesive devices, these characteristics include text length, the complexity and sophistication of the words used in the text, and the variety of the words in the text. Of particular relevance to the present study, metrics related to the quantity, range, and variety of the vocabulary used in a text-that is, metrics related to the text's lexical richness-are important predictors of writing quality as judged by experts (e.g., Engber, 1995;Grobe, 1981;Jarvis, 2002;Jarvis, Grant, Bikowski, & Ferris, 2003;Malvern, Richards, Chipere, & Durán, 2004;McNamara, Crossley, & McCarthy, 2010;Nold & Freedman, 1977). 1 Yet, as Jarvis (2013a) points out, despite the proliferation of lexical richness metrics in quantitative linguistics, it is not clear how well these do indeed measure aspects of lexical richness. To elaborate, lexical richness metrics are generally evaluated on the basis of how sensitive they are to text length (e.g., McCarthy & Jarvis, 2007, 2013Treffers-Daller, 2013;Tweedie & Baayen, 1998) and how well they correlate with variables representing other constructs, such as essay quality, language proficiency or other writer and talker characteristics (e.g., Daller, van Hout, & Treffers-Daller, 2003;Treffers-Daller, 2013;Treffers-Daller, Parslow, & Williams, 2016). ...
... lexical richness should be linearly related to perceived lexical richness, so it may be worthwhile to experiment with statistical models that do not assume linear relationships between the indices and the perceptions. Finally, it is conceivable that the text-based indices stand in non-additive relationships to lexical richness perceptions (on the possibility of interacting effects in judgements of writing quality, see Jarvis et al., 2003), so it may also be worthwhile to allow for interaction effects in the models. ...
... While Jarvis' (2013bJarvis' ( , 2017 are the only studies that concerned lexical richness ratings by untrained raters, a number of studies sought to establish how well text-based indices predict expert ratings of overall text quality. Here, metrics associated with the variability Engber, 1995;Grobe, 1981;Jarvis, 2002;Kuiken & Vedder, 2014;Malvern et al., 2004;McNamara et al., 2010), rarity Guo, Crossley, & McNamara, 2013;Malvern et al., 2004;McNamara et al., 2010), and volume dimensions (Grobe, 1981;Jarvis et al., 2003;Nold & Freedman, 1977) helped to account for variance in the ratings. Additionally, variability, rarity, and volume metrics can be used as predictors for related constructs such as lexical proficiency and overall language proficiency (e.g., Crossley, Cobb, & McNamara, 2013;Crossley, Salsbury, & McNamara, 2011;Crossley, Salsbury, McNamara, & Jarvis, 2010;Kyle & Crossley, 2015;Laufer & Nation, 1995). ...
We investigated how well readers’ perceptions of the lexical richness of short
texts can be predicted on the basis of automatically computable indices of
the texts’ lexical properties. 3,060 French, German and Portuguese texts
(between 9 and 284 words long) written by 8- to 10-year-olds were rated
for their lexical richness by between 3 and 18 uninstructed raters, and over
150 indices were derived from these texts. We found that the ratings could
to a substantial degree be predicted on the basis of these indices and that
the accuracy with which the ratings of shorter texts could be predicted was
comparable to that of longer texts. For French and German, the greatest
predictive power was attained by opaque models with scores of predictors,
but models with fewer predictors based on a 6-dimensional framework of
lexical richness perception or even with a single, easily computed predictor,
Guiraud’s index, fared only slightly worse.
... As mentioned in Section 2.1, previous studies often examined the effect of picture prompts in terms of individual linguistic features of writing products. However, Jarvis et al. (2003) found that writing quality may depend more on linguistic features used together than on individual features. Despite some of the features (e.g. ...
... In all, the findings in this regard corroborate to some extent the argument that patterns of linguistic co-occurrence compensate for the limitation of individual linguistic features and can be used to further analyze writing quality (e.g. Jarvis et al. 2003;. Specifically, the results echo the previous findings that functional dimensions can reflect the variations in writing products by different task features (e.g. ...
... Nevertheless, the results are generally in line with earlier findings that linguistic features used together better captured writing quality than separate features (e.g. Jarvis et al. 2003;Biber & Gray 2013;Biber et al. 2016). Therefore, the findings of this study add further proof to the -linguistically motivated‖ and -parsimonious‖ nature of functional dimensions (Biber et al. 2016) and their applicability to the analysis of task-induced variation in essay quality. ...
To explore the validity of picture-prompt writing assessment, the study investigated the influence of caption type (i.e. narrative, abstract, and zero) in the picture prompt on EFL writing quality of Chinese college students in terms of holistic scores and functional dimensions (i.e. Dimensions of Involvement, Narration, Elaborated reference, Persuasion, Abstractness, and On-line informational elaboration). ANOVA, MANOVA, and linear regression analysis were conducted and results showed that (a) participants performed significantly better with the abstract caption; (b) Dimensions of Involvement and Abstractness significantly distinguished essays with the abstract caption from those with other types; (c) Dimension of Persuasion significantly predicted ratings of essays with the narrative caption, while Dimension of Narration significantly predicted ratings of essays with the abstract caption. Finally, implications for picture-prompt writing assessment and instruction were discussed.
... In order to investigate the internal variation of these texts, we can first examine a broad base of linguistic features. The texts can then be grouped based on their use of these features into 'multiple profiles' (Jarvis et al., 2003). ...
... These profiles denote groups of texts that show within-group similarity with regard to the use of a set of linguistic features but betweengroup difference (Jarvis et al., 2003). Multiple profiles have previously been identified in academic texts (Jarvis et al., 2003;and Friginal et al., 2014). ...
... These profiles denote groups of texts that show within-group similarity with regard to the use of a set of linguistic features but betweengroup difference (Jarvis et al., 2003). Multiple profiles have previously been identified in academic texts (Jarvis et al., 2003;and Friginal et al., 2014). Friginal et al. (2014), building on Jarvis et al. (2003), provide a broad base of features that can be used to help discern different textual profiles. ...
Writing about art and the teaching of art writing are becoming important issues in the art world. Furthering our understanding of art texts can inform the practice and teaching of art writing. In an effort to expand linguistic understanding of art writing, this study examines a corpus of 180 online explanatory art museum texts from the online collections of nine US museums using a functional profiles perspective. I use cluster analysis to group the 180 texts into functional profiles based on their use of twenty-one linguistic features. The cluster analysis resulted in five clusters that are interpreted functionally: Cluster 1 (n=56), ‘descriptive information’; Cluster 2 (n=30), ‘expanded form – adding interpretation and process’; Cluster 3 (n=40), ‘contextualising’; Cluster 4 (n=8), ‘process and interpretation with agency’; and Cluster 5 (n=46), ‘narrative focus’. It is posited that this understanding of online explanatory art museum texts can inform the teaching of future museum professionals.
... To understand second language writing proficiency, it is important to consider the linguistic development of students (e.g. Crossley, Salsbury, & McNamara, 2012;Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1999;Jarvis, Grant, Bikowski, & Ferris, 2003;Lu, 2011). In practice, the results of research have shown that lexical complexity and grammatical complexity are two main problematic features in students' writing Lu, 2011Lu, , 2012. ...
... Crossley & McNamara, 2009;Lu, 2011Lu, , 2012Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998), these two features are indicators of students' second language writing proficiency, as they may use complex grammar and lexicon differently depending on their levels of proficiency. However, Ferris (1994), Grant and Ginther (2000), Jarvis et al. (2003), and Lu (2011) (Ellis & Yuan, 2004;Lu, 2011;Ortega, 2003;Sotillo, 2000;Way, Joiner, & Seaman, 2000;Yang, Lu, & Weigle, 2015). First Language (L1) differences can also considerably influence second language writing performance. ...
Language assessment literacy is defined as the ‘language instructor’s familiarity with testing definitions and the application of this knowledge to classroom practices in general and specifically to issues related to assessing language’. Recent times have witnessed a phenomenal increase in the role of teachers in language testing and assessment. They are involved in assessing students and evaluating assessment data to improve student learning and their own instructional strategies (Malone, 2013). This highlights the significance of teachers’ assessment literacy in influencing how they construct and implement high-quality assessment instruments that adequately assess students’ specified learning outcomes (e.g. Stiggins, 2002). In Saudi Arabia, research indicates major gaps in EFL teachers’ assessment literacy and therefore this study investigates Saudi EFL teachers’ development of assessment tasks and the extent that these tasks are aligned with student learning outcomes. The study examines the gaps in teachers’ knowledge of assessment practices in classrooms and any incongruity between the tasks and learning outcomes. Suggestions are made for how teachers’ language assessment literacy can be improved through future professional development courses.
... While we know that advanced learners are more capable of detecting and correcting a higher number of errors (Kol et al., 2018), we do not yet know what kind of translation errors are detected and corrected by learners of varying proficiency levels. Previous research makes it clear that when writing in the L2, advanced learners are more likely to produce longer texts (Crossley & McNamara, 2012;Jung et al., 2019), display greater lexical variety (Grant & Ginther, 2000;Jarvis et al., 2003), and generate syntactically complex and accurate texts with a higher number of nominalization/subordination (Grant & Ginther, 2000;Liu & Li, 2016) and error-free T-units (Hwang, 2012). Whether such linguistic features can also be observed for MT post-editing remains to be seen. ...
... These findings suggest that the ways in which MT output is used and analyzed is determined by learners' proficiency as learners become more critical of and succeed in detecting and correcting bigger error units with increasing proficiency. This is not surprising considering previous research findings on the effect of proficiency in L2 writing in which advanced learners produced longer, more complex, and more accurate sentences with diverse vocabulary (Crossley & McNamara, 2012;Grant & Ginther, 2000;Hwang, 2012;Jarvis et al., 2003;Jung et al., 2019;Shin & Kim, 2014). While advanced learners did not necessarily produce longer revisions overall, they were willing to revise larger units and write more in their own words. ...
... To understand second language writing proficiency, it is important to consider the linguistic development of students (e.g. Crossley, Salsbury, & McNamara, 2012;Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1999;Jarvis, Grant, Bikowski, & Ferris, 2003;Lu, 2011). In practice, the results of research have shown that lexical complexity and grammatical complexity are two main problematic features in students' writing Lu, 2011Lu, , 2012. ...
... Crossley & McNamara, 2009;Lu, 2011Lu, , 2012Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998), these two features are indicators of students' second language writing proficiency, as they may use complex grammar and lexicon differently depending on their levels of proficiency. However, Ferris (1994), Grant and Ginther (2000), Jarvis et al. (2003), and Lu (2011) (Ellis & Yuan, 2004;Lu, 2011;Ortega, 2003;Sotillo, 2000;Way, Joiner, & Seaman, 2000;Yang, Lu, & Weigle, 2015). First Language (L1) differences can also considerably influence second language writing performance. ...
... A number of scholars favor an analytic scale for the fine-grained score inferences and positive washback while others prefer a holistic scale for practical considerations. In terms of score inferences, analytic scales are arguably more suitable for capturing different learner profiles, which are more reflective of the complex and imbalanced nature in L2 writing development (Hamp-Lyons, 1991Jarvis, Grant, Bikowski, & Ferris, 2003;Weigle, 2009;Yan, Kim, & Kotnarowski, forthcoming). They can also offer more diagnostic information and lead to positive washback on teaching and learning (Chan, Inoue, & Taylor, 2015;Hamp-Lyons, 1991Knoch, 2011). ...
... By providing the binary, analytic scale along with the holistic scale, the current rating method provides additional guidelines and focuses raters' attention on each of the major criteria of the holistic scale. With the addition of the binary, analytic scale, the scale will be able to provide diagnostic information about learner profiles, which are essential in L2 writing assessment (Hamp-Lyons, 1991Jarvis et al., 2003;Weigle, 2009;Yan et al., forthcoming). ...
This two-phased, sequential mixed-methods study investigates how raters are influenced by different rating scales on a college-level English as a second language (ESL) writing placement test. In Phase I, nine certified raters rated 152 essays using a holistic, profile-based scale; in Phase II, they rated 200 essays using a binary, analytic scale developed based on the holistic scale and 100 essays using both rating scales. Ratings were examined both quantitatively through Rasch modeling and qualitatively via think-aloud protocols and semi-structured interviews. Findings from Phase I revealed that, despite satisfactory internal consistency, the raters demonstrated relatively low rater agreement and individual differences in their use of the holistic scale. Findings from Phase II showed that the binary, analytic scale led to much improvement in rater consensus and rater consistency. Another finding from Phase II suggests that the binary, analytic scale helped the raters deconstruct the holistic scale, reducing their cognitive burden. This study represents a creative use of a binary, analytic scale to guide raters through a holistic rating scale. Implications regarding how a rating scale affects rating behavior and performance are discussed. Introduction In performance-based assessment, test scores are influenced by not only the examinee's ability and task but also the rater and rating scale. In order to make appropriate inferences about test-takers' true abilities, such effects need to be minimised or at least understood. Much research has focused on rater effects and rating conditions that affect the evaluation of test performance, suggesting that rating is 'a complex, error-prone cognitive process' as well as how they use different scoring criteria (Cumming, 1990; Cumming, Kantor, & Powers, 2001). While rater training is recommended to reduce such variability among raters (Weigle, 1994, 1998), studies have reported that raters exhibit idiosyncrasy even after training (Vaughan, 1991). In addition to rater-internal effects, the choice of rating scale also affects how raters rate. In this paper, we define 'rater behavior' as raters' rating processes and how they interact with different aspects of examinee performances and rating scales, and 'rater performance' as rater reliability, including both inter-and intra-rater agreement and consistency. We also differentiate three categories of rating scales: holistic, analytic and binary scales. Despite extensive discussions on the strengths and weaknesses of holistic vs. analytic scales in relation to rater reliability and score use, few studies have examined how binary scales influence rater behavior and performance. This study examines how the introduction of a binary, analytic rating scale affects rater behavior and performance.
... To understand second language writing proficiency, it is important to consider the linguistic development of students (e.g. Crossley, Salsbury, & McNamara, 2012;Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1999;Jarvis, Grant, Bikowski, & Ferris, 2003;Lu, 2011). In practice, the results of research have shown that lexical complexity and grammatical complexity are two main problematic features in students' writing Lu, 2011Lu, , 2012. ...
... Crossley & McNamara, 2009;Lu, 2011Lu, , 2012Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998), these two features are indicators of students' second language writing proficiency, as they may use complex grammar and lexicon differently depending on their levels of proficiency. However, Ferris (1994), Grant and Ginther (2000), Jarvis et al. (2003), and Lu (2011) (Ellis & Yuan, 2004;Lu, 2011;Ortega, 2003;Sotillo, 2000;Way, Joiner, & Seaman, 2000;Yang, Lu, & Weigle, 2015). First Language (L1) differences can also considerably influence second language writing performance. ...
https://www.palgrave.com/gp/book/9783030532536
... Furthermore, research broadened towards the investigation of factors, which might cause differences in the reported observations. Examples for this are analyses of heterogeneous learner profiles, due to varying first language (L1) backgrounds (Crossley & McNamara 2011), or differences in learning strategies (Jarvis et al. 2003). ...
... There have been several follow-up syntheses of complexity research, which may be consulted for a more recent overview over the topic, (Malvern et al. 2004), the Guiraud index (Guiraud 1960), and the advanced Guirad index (Daller, Hout & Treffers-Daller 2003). For a more detailed discussion of these indices, please see Jarvis et al. 2003. In summary, the following general trends are often assumed to hold for increasing proficiency: On low and intermediate proficiency levels, learners tend to write increasingly longer sentences, which might either indicate globally increased sentence complexity or higher fluency, and use more subordination (Lu 2011;Norris & Ortega 2009;Ortega 2003). ...
This thesis analyses German L2 writing proficiency by means 398 complexity measures from the domains of i) language use; ii) human language processing; iii) discourse and encoding of meaning; and three subdomains of theoretical linguistics, namely iv) syntax; v) lexicon and semantics; and vi) morphology; with an additional focus on task effects on language complexity in learner corpora. To account for these, available task information from the Merlin and the Falko Georgetown corpus as well as retrospectively annotated cognitive and functional task factors are included in all analyses.
A descriptive cross-corpus investigation of over 100 features shows, how complexity measures exhibit diverging sensitivity to heterogeneous task backgrounds across complexity domains and linguistic constructs. Furthermore, a series of ordinal Generalized Additive Models (GAMs) illustrates, how interactions between cognitive or functional task factors with selected complexity measures significantly improve model fit; and give highly interesting insights into task effects; as well as valuable prompts for further research in more controlled settings.
On Merlin, classification experiments all ordinal GAMs reach F1 scores of at least 71% by using only 13 complexity measures. The best classification results are obtained for a model including also performance effects, which outperforms previously reported results with F1 scores of 85%.
... To understand second language writing proficiency, it is important to consider the linguistic development of students (e.g. Crossley, Salsbury, & McNamara, 2012;Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1999;Jarvis, Grant, Bikowski, & Ferris, 2003;Lu, 2011). In practice, the results of research have shown that lexical complexity and grammatical complexity are two main problematic features in students' writing Lu, 2011Lu, , 2012. ...
... Crossley & McNamara, 2009;Lu, 2011Lu, , 2012Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998), these two features are indicators of students' second language writing proficiency, as they may use complex grammar and lexicon differently depending on their levels of proficiency. However, Ferris (1994), Grant and Ginther (2000), Jarvis et al. (2003), and Lu (2011) (Ellis & Yuan, 2004;Lu, 2011;Ortega, 2003;Sotillo, 2000;Way, Joiner, & Seaman, 2000;Yang, Lu, & Weigle, 2015). First Language (L1) differences can also considerably influence second language writing performance. ...
The ability to write well in English has become a sought-after skill in Saudi Arabia over the last four decades as the status of English has become increasingly important for business and education (Alrashidi & Phan, 2015). However, writing proficiency continues to lag behind learners’ proficiency in other skills and in doing so, its teaching and assessment are subject to continuous debate (Grami, 2010). Considering this, there has been a recent shift in research to examine how technology can play a role in improving learners’ writing proficiency. This chapter therefore carries out a meta-analysis to examine how technology has acted as an intervention that can improve writing for university learners. The meta-analysis considers this use in line with measures of study quality and frames its discussion of effectiveness around these measures. The chapter concludes by recommending how future research studies in this area should be conducted and reported.
... In addition, lexical complexity measures have also been used as valid development measures in L2 writing. It is viewed in terms of lexical density (the proportion of content words to total words), lexical sophistication (the proportion of unusual/infrequent and advanced words), and lexical variation (the range of vocabulary) (Read, 2000), and studies have found measures of lexical variation, such as the number of different words (NDW) and transformations of Type-Token Ratio (TTR), as well as lexical sophistication measures to be highly indicative of L2 development (Crossley et al., 2011;Grant & Ginther, 2000;Jarvis et al., 2003;McNamara et al., 2010). ...
... show that complex nominalization linearly increase across proficiency levels and are one of the best discriminators between proficiency levels. Moreover, the relationship between high-scoring essays and lexical variation/diversity is well attested and unquestioned (Grant & Ginther, 2000;Jarvis et al., 2003). confirm Fotos' (1991) observation that what cloze tests measure can be variable depending on the proficiency level of the test-takers. ...
... In fact, a great number of empirical studies have shown that text length is a rather consistent predictor of writing quality, both among L1 (e.g., Malecki & Jewell, 2003;Olinghouse Jung, Crossley, & McNamara, 2019). For example, using cluster analysis, Jarvis et al. (2003) examined 21 linguistic features (e.g., text length, diversity of vocabulary, hedges, etc.) in 338 timed compositions by L2 writers from two sets of data and found that all clusters of highly rated texts in both data sets showed longer than average text lengths. A similar endeavor was made by Jung, Crossley, and McNamara (2019) In addition to quantitative evidence, empirical studies on the importance of text length have also offered qualitative evidence by collecting comments from the raters after rating or eliciting concurrent verbalizations during the rating process. ...
... & Leaird, 2009) and L2 writers (e.g., Al-Mudhi, 2019;Jarvis, Grant, Bikowski, & Ferris, 2003; ...
In the past three decades, the construct of second language (L2) writing complexity has been theorized and refined in both second language acquisition (SLA) (Crossley, 2020; Housen, De Clercq, Kuiken, & Vedder, 2019; Lu, 2011; Norris & Ortega, 2009) and Systemic Functional Linguistics (SFL) research (Byrnes, 2009; Ryshina-Pankova, 2015; Schleppegrell, 2004). The general consensus is that lexical and syntactic variations are regarded as signs of advanced academic writing. The contemporary legal writing pedagogy, however, is informed by the Plain English Movement (Benson, 1985; Dorney, 1988; Felsenfeld, 1981), which largely discourages the use of overly complex structures and “elegant variations.” The recommendation to use plain English in legal writing thus poses a challenge to the theoretical consensus in SLA and SFL research and raises a question about the conceptualizations and assessment of writing complexity in academic legal writing classrooms. This dissertation consists of three interrelated studies and aims to address this contradiction by examining the development and assessment of writing complexity (i.e., lexis, syntax, and discourse) in 246 hypothetical legal essays written by 31 international (LL.M.) students over one-year of legal language study at the Georgetown University Law Center. In Study 1, I used a structural, corpus-based approach and tracked the changes of 31 students’ lexical and syntactic complexity in six data collection points over one year and compared the complexity indices with those benchmarked by eight model essays. In Study 2, I offer an in-depth discussion of four students’ distinct developmental trajectories of discourse complexity, which I analyzed through the system of engagement (Martin & White, 2005) from the SFL perspective. Finally, in Study 3, I adopted a mixed-methods approach to investigate two legal instructors’ conceptualizations of writing complexity in the classroom setting. Results showed that overall students were able to write with significantly more sophisticated words and, in the second semester, significantly shorter sentences, a pattern consistent with the pedagogical focus of the program. Additionally, the four individual trajectories of discourse complexity indicate that, even when starting with a similar proficiency level, some students constructed increasingly complex legal discourse by actively engaging different legal voices through dialogic expansion and contraction, while others only made marginal progress. Consistent with the instructors’ conceptualizations of writing complexity, structurally more complex essays were not necessarily of higher text quality. In fact, the instructors’ assessment of text quality was found to be influenced by a number of external factors, such as students’ performance in comparison to others and their academic progress in the program. I conclude by discussing some additional insights that emerged from the dissertation such as the boundaries between text modeling and plagiarism, as well as the role of text length in assessing timed writing. Finally, I call for a critical reflection on the pedagogical principles of Plain English and highlight the value of an integrated structural-functional approach to holistically understand the construct of L2 writing complexity in academic legal writing.
... Word length increases with age and quality in L1 writers (Berman & Nir-Sagiv, 2007;Grobe, 1981;Malvern et al., 2004;Massey & Elliott, 1996;Massey et al., 2005;Myhill, 2009;Olinghouse & Leaird, 2009). For L2 writers there is conflicting evidence, with some studies finding that word length is related to proficiency (Cumming et al., 2005;Jarvis et al., 2003) or to longitudinal development (Hou et al., 2016), and others finding no effect (Crossley et al., 2010;Knoch et al., 2015;Verspoor et al., 2012;Vidakovic & Barker, 2010). It is important to note, however, that the construct which these measures tap is rather ambiguous since mean word length is likely to reflect a combination of use of low frequency words and morpho-syntactic complexity (e.g. ...
Recently-developed tools which quickly and reliably quantify vocabulary use on a range of measures open up new possibilities for understanding the construct of vocabulary sophistication. To take this work forward, we need to understand how these different measures relate to each other and to human readers’ perceptions of texts. This study applied 356 quantitative measures of vocabulary use generated by an automated vocabulary analysis tool ( Kyle & Crossley, 2015 ) to a large corpus of assignments written for First-Year Composition courses at a university in the United States. Results suggest that the majority of measures can be reduced to a much smaller set without substantial loss of information. However, distinctions need to be retained between measures based on content vs. function words and on different measures of collocational strength. Overall, correlations with grades are reliable but weak.
... Their series of studies were differentiated not only in their analytic techniques, but also in their data types: Torrance et al. (1994) used the questionnaire on students' recall of ordinary habits, whereas Torrance et al. (2000) depended on a retrospective questionnaire that enabled students to answer on just a previous process. In the domain of L2 writing research, Jarvis, Grant, Bikowski, and Ferris (2003) investigated multiple profiles of highly rated timed compositions and explored the relationships with their linguistic features. ...
This study aimed to identify undergraduate students' writing profiles based on quality of planning output, revision changes between the first and final drafts, and quantity of Internet searches while writing digitally. To investigate the performance of each profile, the differences in text quality according to students' profile membership were also examined. 260 Korean undergraduate students participated in the study and wrote an opinion essay. Latent profile analysis and one-way MANOVA were adopted as analytic tools. Four profiles emerged: Revision-based, Plan-based, Search-based, and Correction-based Writers. Correction-based Writers who showed many surface-level changes represented the majority. Revision-based Writers who showed many sentential and textual changes outperformed the other three profiles; the other profiles did not show significant differences in text quality between each other. The research findings corroborated evidence for the existing issue of planning vs. revising strategies and presented educational implications based on analyzing the current state of undergraduate students' writing.
Keywords digital writing; planning; revision; Internet search; latent profile analysis
... Research has been published on the features of highly rated compositions in the TOEFL (Jarvis et al., 2003), the role of argument in IELTS writing (Coffin, 2004) and the washback of IELTS tests on the education systems and societies in which they operate (Green, 2006(Green, , 2007. In addition, Raimes (1990) explored causes for concern with the writing section of the TOEFL and Moore and ...
... However, no single predictor variable can fully account for L2 English writing performance. Researchers are recommended to examine multiple profiles of English L2 writing in the future, across proficiency levels or NNS and NS groups (e.g., Jarvis, Grant, Bikowski, & Ferris, 2003;Friginal, Li, & Weigle, 2014). More recently, Yan and Staples (2016) and advocated a multi-dimensional analysis of L2 writing because co-occurring lexico-grammatical features were more reliable for describing and capturing L2 writing proficiency. ...
This study aims to investigate the relationship between the noun phrase complexity of advanced Chinese EFL learners' integrated writing and the score assigned by expert raters. Their written performance was also compared with those of native English speakers (NS) at university level with particular reference to the use of noun phrases. Results showed that there was a moderately positive correlation between the use of complex nominals in test-takers' writing and the corresponding score. More specifically, non-native speakers of English (NNS) and NS groups differed significantly in the majority of noun phrase complexity measures. The implications are discussed concerning noun phrase complexity as a more reliable measure of syntactic complexity for an integrated writing writing test.
... Word length increases with age and quality in L1 writers (Berman & Nir-Sagiv, 2007;Grobe, 1981;Malvern et al., 2004;Massey & Elliott, 1996;Massey et al., 2005;Myhill, 2009;Olinghouse & Leaird, 2009). For L2 writers there is conflicting evidence, with some studies finding that word length is related to proficiency (Cumming et al., 2005;Jarvis et al., 2003) or to longitudinal development (Hou et al., 2016), and others finding no effect (Crossley et al., 2010;Knoch et al., 2015;Verspoor et al., 2012;Vidakovic & Barker, 2010). It is important to note, however, that the construct which these measures tap is rather ambiguous since mean word length is likely to reflect a combination of use of low frequency words and morpho-syntactic complexity (e.g. ...
... The results of the Centroid hierarchical cluster analysis successfully matched our previous cluster definition hypothesis as it organized the sixteen traits studied into the three clusters (Fig. 1). This analysis proceeds from each cognitive process constituting its own cluster, to all of the processes being iteratively and progressively combined into a single global cluster (Jarvis et al., 2003;Norušis, 2012). Then, we selected the iteration that best represented the three clusters that we had previously determined by examining the agglomeration stages and coefficients obtained (Fig. 1). ...
Tools allowing to understand the evolution of donkey populations in time, the future trends that these populations describe, and the factors conditioning such trends, become invaluably critical when aiming at preserving and later recovering such populations from their endangerment status. Basing on the characteristic lack of information regarding the genealogical background of donkey populations and taking a particular breed as an example, it is possible to infer a model to assess the genetic and demographical structure of other international endangered donkey populations. Then, we can plot selection strategies to implement once such populations have reached the sufficient number of individuals, and are supported by solid enough structures. Microsatellite-tested pedigree analyses were carried out to study the genetic diversity, structure and historical evolution of the Andalusian donkey breed since the 1980s. Despite mean inbreeding was low, highly inbred animals were present. The effective population size based on individual inbreeding rate was about half when based on individual coancestry rate. Nei's distances and equivalent subpopulations number indicated differentiated farms in a highly structured population. Although genetic diversity loss since the founder generations could be considered small, intraherd breeding policies and the excessive contribution of few ancestors to the gene pool could lead to narrower pedigree bottlenecks. Long average generation intervals could be considered when reducing inbreeding. Wright's fixation statistics indicated slight inbreeding between farms. Pedigree shallowness suggested applying new breeding strategies to reliably estimate descriptive parameters and control the negative effects of inbreeding, which could indeed, mean the key to preserve such valuable animal resources avoiding the extinction they potentially head towards. Diversity studies render especially important in donkeys as they reveal the genetic background in the populations and the starting point for making decisions on whether to apply conservation or breeding plans in this functionally misallocated species. Once genetic diversity parameters are balanced, finding new niches for donkeys becomes potentially the most relevant aim to approach in the midterm future for the species. Selection strategies in donkeys are approached from three different perspectives; donkey-assisted therapy and therapeutic riding, fertility and disease resistance, not only as a way to widen the functional spectrum of opportunities of donkeys but also to lengthen their useful lives, and improve their life quality and welfare. Studying the specific genetic background behind functional traits enables quantifying the degree in which such features pass from jacks and jennies onto the new foal generations. As a genetic term, environment means all influences other than inherited factors. Controlling the environmental factors conditioning the expression of certain functional features help to build animal models shedding light in the genetic fraction involved in such functional traits. The functional performance of 300 microsatellite-assisted parentage tested donkeys was studied using REML and Gibbs sampling Bayesian methods for the obtention of genetic parameters and breeding values using BLUP methodology. The first functional niche for which donkeys may be well-suited is linked to their special psychological nature and physical characteristics as facilitators of learning processes and for the development of key life skills and confidence building for a wide spectrum of vulnerable people. Therapeutic riding and asinotherapy take advantage of the physical and psychological interaction between donkeys and patients given the potential application of donkey's characteristics and abilities for the treatment of specific human disorders. The selection of donkeys when the breeding criteria is their suitability for equine-assisted therapies was implemented following two different approaches; the selection for coping styles and cognitive processes and the selection for gaits and kinetics. Aiming at developing suitable models seeking the consolidation of equine assisted-therapy breeding criteria, we studied 29 factors that may potentially influence several cognitive processes in donkeys. These factors not only affect donkeys' short-term behaviour but may also determine their long-term cognitive skills from birth. Thus, animal behaviour becomes a useful tool to obtain past, present or predict information from the situation of a certain animal in a particular area. Operant conditioning and Qualitative Behavioural Assessment (QBA) synergism can provide valuable information about animals' extinction/learning and emotional status. All noncognitive animal inherent features significantly affected four variables (P<0.001), although some were not linearly correlated. On the other hand, the effect power of meteorological factors ranged from 7.9% for the birth season on learning (P<0.05) to 38.8% for birth moon phase on mood (P<0.001). Psychometric testing enables quantifying animal cognitive capabilities and their genetic background. Among these cognitive capabilities, the study of problem-solving coping styles achieves a special relevance as it brings together the need genetically select donkeys displaying a neutral reaction during training, given its implication with handler/rider safety and trainability. Heritabilities for coping style traits were moderate, 0.18 to 0.21. Phenotypic correlations between intensity and mood/emotion or response type were -0.21 and -0.25, respectively. Genetic correlations between the same variables were -0.46 and -0.53, respectively. Phenotypic and genetic correlations between mood/emotion and response type were 0.92 and 0.95, respectively. Principal components and Bayesian analyses were used to compute the variation in cognitive capabilities explained by 13 cognitive processes and their genetic parameters, respectively. Heritabilities ranged 0.06 to 0.38 suggesting the same patterns previously reported for humans and other animal species. By contrast, when considering the selection for therapeutic riding, gaits' heritability estimates ranged from 0.53 to 0.67 for walk and trot, respectively. Genetic correlations ranged from 0.28 to 0.42, for walk/trot and amble/trot, respectively. Our results suggest that gait genetic lines could be developed. Among other breeding criteria, disease resistance and reproduction offer two functional niches to consider given their relationship with donkey life quality and welfare. Breeding programs selecting for disease resistance could address food safety and quality issues in products such as donkey milk, and may be perceived to be more humane. Cutaneous habronematidosis (CH) is a highly prevalent parasitic seasonally recurrent skin disease causes distress and relapsing wounds to the animals. CH hypersensibility heritability was 0.0346. Genetic parameters and breeding values for functional traits enable planning strategies for endangered donkey breed preservation and breeding what may turn into a measure to improve animal welfare indirectly.Multiple births in equids are dangerous situations that compromise the life of the dam and offspring. However, embryo collection techniques take advantage of individuals whose multiple ovulations allow flushing more fertilized embryos from the oviduct. Heritabilities ranged from 0.18 to 0.24. Genetic and phenotypic correlations ranged from 0.496 to 0.846 and 0.206 to 0.607, respectively.
... Acquiring native-like competence in second language (L2) writing is perhaps the greatest challenge faced by L2 learners. Much comparative and quantitative research in the past two decades has endeavored to discover linguistic features and characteristics of proficient L2 writing (e.g., Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1999;Grant & Ginther, 2000;Jarvis, Grant, Bikowski, & Ferris, 2003;Lu, 2011;Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998). These studies commonly employ linguistic complexity (e.g., length of production units, lexical complexity, syntactic complexity) as a predictor for proficient L2 writing, but their results are sometimes inconsistent and/or inconclusive (e.g., Becker, 2010;Biber, Gray, & Poonpon, 2011;Ortega, 2003;Taguchi, Crawford, & Wetzel, 2013;Wolfe-Quintero et al., 1998). ...
Using Biber’s (1988) multidimensional analysis, this study investigates textual variation in second language (L2) learners’ writing at different proficiency levels, and attempts to identify any developmental progression. The study used a corpus of 5200 argumentative essays written by 2600 students learning English as an L2. The results indicate that advanced L2 writing is fundamentally different from less advanced L2 writing: Advanced learners’ writing is closer to native speakers’ written discourse, while less advanced learners’ writing is closer to native speakers’ spoken discourse. The patterns of development vary across different sets of textual features. Informational (as opposed to involved) production and impersonal (as opposed to nonimpersonal) style showed gradual development as the learners’ proficiency increases. Nonnarrative (as opposed to narrative) production, elaborated (as opposed to situation-dependent) reference, and overt expression of persuasion did not show significant differences across the proficiency levels. The article offers pedagogical implications for practices of L2 writing instruction.
... Laying out research implications in the Journal of Second Language Writing 2015 special issue on L2 writing complexity, Ortega posits that "if syntactic complexity grows as writers become increasingly more capable of using the additional language with linguistic maturity, so will they also write with more communicative and rhetorical flexibility" (p. 87.) Yet, while studies into the relationship between syntactic complexity and L2 writing quality exist (e.g., Biber, Gray, & Staples, 2016;Bulté & Housen, 2014;Crossley & McNamara, 2014;Taguchi, Crawford, & Wetzel, 2013;Yang et al., 2015), as do studies which take features of syntactic complexity into account of broader profiles of high-rated writing (e.g., Friginal & Weigle, 2014;Jarvis, Grant, Bikowski, & Ferris, 2003), extremely limited scholarship has been dedicated to the relationship between syntactic complexity and assessed writing quality in L2 student writing in the context of first-year writing (FYW) courses (Staples & Reppen, 2016, is a notable exception). ...
This study explores the relationship between syntactic complexity and writing quality in assessed source-based research papers produced by ESL undergraduate writers in a first-year writing course through a combination of holistic and fine-grained measures of complexity. The analysis is based on a corpus of 280 student papers across three grade tiers: high, mid, and low. A one-way MANOVA was used to explore the statistical significance of differences of five commonly used syntactic complexity measures (assessed using Lu’s Second Language Syntactic Complexity Analyzer, 2010) across these grade tiers. Results reveal little variation in clausal subordination and coordination, but statistically significant lower complex nominal densities, mean length of clauses (phrasal measures), and mean length of T-units (global measure) in low-rated papers. Analysis of complex nominal composition using the Stanford Tregex with differences assessed with a one-way MANOVA shows that the highest densities of complex nominal types are present in high-rated papers, with statistical significance in adjectival pre-, prepositional post-, and participle modification, and the lowest densities in low-rated papers. While clausal complexity did not demonstrate a relationship with assessed quality, both global and phrasal complexity features appear to be important components. We conclude with implications for syntactic complexity research and ESL composition pedagogy.
You can view this article for free until May 20th with the following link: https://authors.elsevier.com/a/1YpoU3n6UYaaTi
... and word frequency (r = −.61) are highly correlated with holistic scores of lexical proficiency. Taken together, these studies suggest that lexical diversity is an important indicator of writing proficiency (Crossley & McNamara, 2012), although the reported effect sizes are usually low to moderate (Crossley & McNamara, 2012;Engber, 1995;Grant & Ginther, 2000;Jarvis, Grant, Bikowski, & Ferris, 2003). ...
Numerous studies have examined the relationship between lexical features of students’ compositions and judgements of text quality. However, the degree to which teachers’ judgements are influenced by the quality of vocabulary in students’ essays with regard to their assessment of other textual characteristics is relatively unexplored. This experimental study investigates the influence of lexical features on teachers’ judgements of English as a second language (ESL) argumentative essays. Using analytic and holistic rating scales, English pre-service teachers (N=37) in Switzerland assessed four essays of different proficiency levels in which the levels of lexical diversity and sophistication had been experimentally varied. Coh-Metrix software was used to manipulate the level of lexical diversity, as measured by MTLD and D, and the Tool for the Automatic Analysis of Lexical Sophistication (TAALES) software was used to obtain differing levels of lexical sophistication, as measured by word range. The results suggested that texts with greater lexical diversity and sophistication were assessed more positively concerning their overall
quality as well as the analytic criteria ‘grammar’ and ‘frame of essay’. The implications of this study for classroom practice and teacher education are discussed.
... That being said, more recent studies (Nisbett, Peng, Choi and Norenzayan, 2001;Jarvis, Grant, Bikowski and Ferris, 2003;Graesser, McNamara, Louwerse and Cai, 2004;Crossley and McNamara, 2012;Robinson, Navea and Ickes, 2013;Pennebaker, Chung, Frazee, Lavergne, and Beaver, 2014) focus on profiling candidates based on their writings. Concentrating on the syntactical level, Haswell (2000) found that advanced writers constructed more complex clauses and sentences. ...
Fierce competitiveness of admission to highly respected universities and companies has created the need to evaluate and profile successful candidates. Traditional evaluation of candidates involving their grades is no longer satisfactory and does not provide a clear image of a candidate. There should be a way to profile a candidate and predict one’s success ranging from a success in a course to a working position. Our research aimed to study potential correlations of students’ writings and their revision exam grades acquired in English for Specific Purposes courses. 203 university engineering students submitted their introductory essays for the computational (LIWC) and statistical (SPSS) analysis. The results showed that the higher achievers used more scholarly vocabulary and verbose sentences. They were task-oriented thinking in a categorical way. On the other hand, the lower achievers used less complex sentences and were other-oriented, demonstrating a verbal dynamic style. However, stereotypical behaviour was only exhibited by the male students, i.e. due to societal pressure, the female students put more effort into their task completion, compensated for the lack of knowledge attempting to establish themselves in the male-dominated field.
... It focuses on the strengths of writing but not deficits, weaknesses, or problems (Jarvis, Grant, Bikowski, & Ferris, 2003), and is unsuitable for diagnostic purposes as defined earlier. ...
To date, only a handful of studies in language testing and assessment utilized Cognitive Diagnostic Modeling (CDM) for diagnostic assessment of L2 academic writing. Even fewer studies tracked the development of L2 academic writing over time. This study adopted CDM to profile the English academic writing performance of first-year university students and tracked the changes of their writing profiles over an academic writing course. This study involved 472 first-year undergraduate students from a university in Hong Kong. Participants were involved in a 13-week English academic writing course, for which they produced three essays for assessment at the beginning (Time 1: individual, timed essay), during (Time 2: pair co-construction, untimed essay), and the end (Time 3: individual and timed essay). At each assessment point, students’ writing was assessed by nine EAP instructors against a fine-grained diagnostic checklist for academic writing. The diagnostic data were analyzed via the reduced RUM to generate for each student a multi-dimensional diagnostic profile on five writing attributes. Salient writing profiles was identified for each assessment point and their changes were tracked over the three observations. The study identified four major performance patterns out of the 25 posterior probability of mastery profiles at each assessment point, and multiple transition trajectories of such profiles from Time 1 to 3 and from Time 2 to 3. Comparing Time 1 and 3, students’ writing performance improved on the two attributes of ‘task fulfilment’ and ‘vocabulary use’, but deteriorated in terms of ‘grammar’ and ‘organization’. Performance on ‘mechanics’ has marginal but insignificant decrease. Besides generating individualized feedback to students, the CDM-based diagnostic assessment can provide useful information for course planning, instruction, and evaluation. Limitations of this study and further directions will be discussed at the end.
... Metodológicamente, a partir del software informático de acceso libre AntConc y Paramtext, se cuantificó (a) el total de palabras clasificándolas según su categoría morfológica, así como (b) el total de types y tokens de cada producción para comprobar: en primer lugar (i) la fluidez léxica (wolfe-Quintero, et al. 1998;Jarvis, et al. 2003;Agustín Llach 2011) y en segundo lugar, (ii) la diversidad léxica (Read 2000;Laufer 2003;Pallotti 2015). A continuación se utilizó el software estadístico de acceso libre G-Stat 2.0 para comprobar si las diferencias observadas, esto es, mayor número tanto de Types (6157 en el M1 y 8592 en el M2) como de Tokens, eran estadísticamente significativas. ...
La experiencia profesional adquirida en un contexto educativo norteamericano se ha convertido en una oportunidad para acercarnos a un planteamiento diferente en torno a la corrección del discurso escrito. Así es como, en un marco dominado por el aumento del número de estudiantes estadounidenses que aprenden español como lengua extranjera (LE), se impone conceder una especial atención a la expresión escrita de los alumnos de Primaria. En este sentido, en el presente trabajo nos proponemos como objetivo dar cuenta de las diferencias y similitudes que existen entre dos sistemas educativos respecto a la corrección de producciones escritas en una lengua extranjera.
a los que acuden los profesores de LEs del mencionado contexto para la corrección y el feedback del discurso escrito de sus alumnos. Consideramos que el análisis que llevaremos a cabo nos permitiría observar en qué medida podemos adoptarlos y adaptarlos a nuestras necesidades didáctico-prácticas en la perspectiva de la mejora de la enseñanza-aprendizaje de ELE en nuestro contexto educativo.
... Coh-Metrix (Graesser, McNamara, Louwerse, & Cai, 2004) is a computational tool that measures various surface-and deep-level text features related to comprehension including lexicon, syntax, and discourse levels (McNamara, Graesser, McCarthy, & Cai, 2014). Unlike traditional measures of textual features such as sentence and word count (Connor, 1990;Jarvis, Grant, Bikowski, & Ferris, 2003;Nation, 2001), Coh-Metrix examines the meaning of the text and its underlying discourse. Coh-Metrix provides statistical measures corresponding to three levels of mental representation when readers are engaged with texts: the surface structure (lexicon and syntax), the textbase (a mental representation of the text that maintains the text meaning), and the situation model (the "subject matter" described in the text including topics, persons, and locations) (McNamara et al., 2014). ...
This study aims to invoke a theoretical model to link the linguistic features of text complexity, as measured by Coh-Metrix, and text quality, as measured by human raters. One hundred and sixty three Chinese EFL learners wrote sample expository and persuasive essays that were marked by four trained raters using a writing scale comprising Word Choice, Ideas, Organization, Voice, Conventions, and Sentence Fluency traits. The psychometric reliability of the writing scores was investigated using many-facet Rasch measurement. Based on the construction–integration (CI) model of comprehension, three levels of mental representation were delineated for the essays: the surface level (lexicon and syntax), the textbase, and the situation model. Multiple proxies for each level were created using Coh-Metrix, a computational tool measuring various textual features. Using structural equation modeling (SEM), the interactions between the three levels of representation, text quality, and tasks were investigated. The SEM with the optimal fit comprised 23 observed Coh-Metrix variables measuring various latent variables. The results show that tasks affected the situation model and several surface level latent variables. Multiple interactions were identified between writing quality and levels of representation, such as the Syntactic Complexity latent variable predicting the situation model and the situation model latent variable predicting Conventions and Organization. Implications for writing assessment research are discussed.
... Besides vocabulary and grammar, it is more than necessary to investigate other linguistic features of L2 learners in the exploration of how L2 proficiency develops. Accordingly, there have been numerous attempts to identify various linguistic characteristics of L2 quality in terms of qualitative methods (e.g., Crossley, Salsbury & McNamara, 2012;Ferris, 1994;Jarvis et al., 2003;Lu, 2011). In these studies, various measures such as lexical density, lexical diversity, lexical variation, mean length of Tunit, and clause per T-unit have been widely used as major indices of L2 proficiency. ...
With the wide spread of the Common European Framework of Reference (CEFR) scales, many studies attempt to apply them in routine teaching and rater training, while more evidence regarding criterial features at different CEFR levels are still urgently needed. The current study aims to explore complexity features that distinguish and predict CEFR proficiency levels in oral performance. Using a quantitative/corpus-based approach, this research analyzed lexical and syntactic complexity features over 80 transcriptions (includes A1, A2, B1 CEFR levels, and native speakers), based on an interview test, Standard Speaking Test (SST). ANOVA and correlation analysis were conducted to exclude insignificant complexity indices before the discriminant analysis. In the result, distinctive differences in complexity between CEFR speaking levels were observed, and with a combination of six major complexity features as predictors, 78.8% of the oral transcriptions were classified into the appropriate CEFR proficiency levels. It further confirms the possibility of predicting CEFR level of L2 learners based on their objective linguistic features. This study can be helpful as an empirical reference in language pedagogy, especially for L2 learners’ self-assessment and teachers’ prediction of students’ proficiency levels. Also, it offers implications for the validation of the rating criteria, and improvement of rating system.
... A possible explanation might be that pairwise comparisons are more prone and sensitive to text length, as longer texts were often rated as more qualitative and better texts. In educational research, text length has often been proven to have a significant relationship with text quality (Jarvis et al., 2003;Lee et al., 2009). However, when comparing texts to each other (like pairwise comparisons do), text length is an easy criterion to use. ...
Assessing argumentative writing skills is not a straightforward task, as multiple elements need to be considered. In function of providing feedback to students and keeping track of their progress, evaluating argumentative texts in a suitable, valid and efficient way is important. In this state-of-the-art exploratory study, 130 argumentative texts written by eleventh graders were assessed by means of three different rating procedures (i.e., absolute holistic rating, comparative holistic rating, and absolute analytic rating). The main aim of this study is twofold. First, we aim to examine the correlations between the three rating procedures and to study the extent to which these procedures differ in assigning scores. In doing so, the more innovative approach of pairwise comparisons is compared to more established assessment methods of absolute holistic and analytic rating. Second, we aim to identify key characteristics that determine the quality of an argumentative text, independent of the rating procedure used. Furthermore, key elements of mid-range, weak and strong argumentative texts were studied in detail. The results reveal low to moderate agreement between the different procedures, indicating that all procedures are suitable to assess the quality of an argumentative text; each procedure, however, has its own qualities and applicability.
... The difficulty of accounting for the contribution of this measure to the interpretation of this component can be justified by its low coefficient (0.22) in comparison with the loadings of the other positive variables (0.45-0.52). 17 Jarvis et al. (2003) is an example of a feature-driven L2 study that compares proficiency levels of students from different backgrounds. They carried out an agglomerative hierarchical cluster analysis of the frequencies of 21 linguistic features representative of length (text length, mean word length), lexical diversity (type/token ratio), lexical reinforcement (hedges, amplifiers, emphatics, downtoners), lexical involvement/objectification (nouns and nominalisations, personal/impersonal pronouns, verbal tenses, demonstratives), syntactic sophistication (frequency of conjuncts, passives, adverbial/relative subordination, complementation), etc., in two sets of L2 English placement compositions and essays written by Arabic-, Chinese-, Japanese-, and Spanish-speaking university-level students. ...
... From the early 2000s on, researchers have also turned their attention to cross-sectional designs of learner language at different proficiency levels (Grant & Ginther, 2000) and, more recently, longitudinal designs (Belz & Vyatkina, 2005;Byrnes, Maxim, & Norris, 2010;Connor-Linton & Polio, 2014;Vyatkina, Hirschmann, & Golcher, 2015). Multivariate approaches to the study of SLW have increased as well, moving analyses beyond individual variables to constellations of variables that are related to L2 development, using factor analysis and cluster analysis (Biber, Gray, & Staples, 2016;Friginal, Li, & Weigle, 2014;Gries, 2018;Jarvis, Grant, Bikowski, & Ferris, 2003;Weigle & Friginal, 2015;Yan & Staples, 2020). In addition, SLW research outside of English has begun to flourish with studies such as the development of L2 pragmatics through use and awareness of German modal particles (Belz & Vyatkina, 2005) and the genre-based curricular design for advanced writing development in foreign languages (Byrnes et al., 2010). ...
... We also included Measure of Textual, Lexical Diversity (MTLD), which was automatically calculated using the online tool TAALED (Kyle, n.d., "NLP for the Social Sciences"). These measures were included as AWL provides information on lexical sophistication, a measure of complexity concerning one's breadth and depth of lexical knowledge (Jarvis et al., 2003); WF has the potential to predict L2 proficiency (Crossley et al., 2013); and MTLD, unlike the typetoken ratio, is a measure of lexical diversity that is unaffected by text length (McCarthy & Jarvis, 2010). sentence complexity (C/S). ...
Understanding the effects of second language writers’ topic familiarity on their subsequent writing performance has been a subject of great interest to researchers in second language acquisition and TESOL. However, prior studies have tended to suffer from multiple methodological limitations (e.g., no interrater reliability, limited or no control over important variables), which may call into question their results. In this conceptual replication, we highlight some of these issues by revisiting an influential study by He and Shi (2012), which explored the effects of ESL students’ topic familiarity on their writing performance as assessed by measures of text quality and complexity, accuracy, and fluency (CAF). Apart from highlighting the differences in our studies’ designs (e.g., the CAF measures and statistical analyses used), we highlight the similarities and differences in our findings. Finally, we close with a call for increased replication research in this area.
... This fact can not only help L2 researchers identify relevant (taskinduced) linguistic features, but it also illustrates an important similarity between learner and non-learner performance; namely, that certain linguistic features are associated with certain functions that are, in this sense, not dependent on L1 or L2 performance. Secondly, the significance of identifying linguistic features that co-occur and work together to fulfil specific discourse functions has been identified in previous SLA work (Jarvis, Grant, Bikowski, & Ferris 2003;Friginal, Li, & Weigle 2014) and MD analysis is a fruitful and empirically valid way to identify co-occurring features and interpret them functionally. With the caveat that an RA perspective towards analyzing task performance in task-based L2 research has its own challenges and requires deliberate planning in designing corpora of task-based L2 data, we have illustrated in this section that MD approaches of these types, together with the lexico-grammatical and lexical bundle analyses, can locate and interpret linguistic differences between written and spoken language or can begin to tell TBLT researchers something about "task language. ...
For over 30 years, corpus research on register variation has expanded our understanding of language use by illustrating how linguistic features co-occur and vary in different situations of use ( Biber & Conrad 2019 ). Over the same period, Task-based Language Teaching (TBLT) has provided a theoretical and empirical basis for research in instructed Second Language Acquisition/SLA ( Ellis 2012 ). This paper illustrates how the methods and approaches used in register analysis offer a useful framework for understanding critical issues in TBLT (e.g., describing tasks and interpreting task performance). The paper compares register analysis and TBLT and then draws upon recent empirical work demonstrating how a register approach (a) identifies a wider range of linguistic and non-linguistic variables than are generally in TBLT; and, (b) provides a useful framework to functionally interpret task performance. The paper ends by discussing how a register perspective can benefit future areas of investigation in task-based second language research.
... The identification of valid and reliable indices of syntactic complexity that can significantly predict second language (L2) writing quality has received much research attention in the past few decades (e.g., Biber, Gray, & Staples, 2016;Crossley, 2020;Crossley & McNamara, 2014;Ferris, 1994;Frase, Faletti, Ginther, & Grant, 1997;Grant & Ginther, 2000;Jarvis, Grant, Bikowski, & Ferris, 2003;Kim & Crossley, 2018;Kyle & Crossley, 2018;Yang, Lu, & Weigle, 2015). Holistic T-unit-based and clausal indices of syntactic complexity that do not differentiate subtypes of clausal or phrasal structure (e.g., T-units per sentence, clauses per T-unit, and dependent clauses per clause) have played a long role in investigations into L2 writing development and L2 writing quality (Lu, 2011(Lu, , 2017Ortega, 2003;Wolfe-Quintero, Inagaki, & Kim, 1998). ...
This study investigated the extent to which traditional vs. fine-grained indices of syntactic complexity could predict second language (L2) raters' quality ratings of 581 application letters and 595 argumentative essays produced by college-level Chinese English as a foreign language (EFL) learners. Our results showed that fine-grained indices explained a larger proportion of variance in both letter and essay quality than traditional indices. More importantly, however, our results revealed substantial genre effects on the explanatory power of traditional and fine-grained indices as well as on the specific types of indices that significantly predicted L2 writing quality. Furthermore, our results differed from previous findings with respect to the effect size of traditional indices for essay quality as well as the relative importance of fine-grained phrasal and clausal complexity indices in essay quality. These findings call for the need to systematically assess the predictive power of syntactic complexity indices across different genres.
... Fluency in writing is typically assessed by productivity, that is, the total number of words produced in an essay or other written assessment given the same allocation of writing time for all participants . Most studies that have measured the total number of words have found significant differences between "better" writing, usually longer, and "weaker" writing, typically shorter (Jarvis, Grant, Bikoski, & Ferris, 2003). CAF studies have used these concepts to assess discourse-level writing and oral development in L2L studies (Michel, 2017) and SHLLs (e.g., Marqués-Pascual, 2011, see chapter 10, this volume). ...
"Heritage Speakers of Spanish and Study Abroad" is an edited volume that provides emerging research on heritage speakers of Spanish in immersion contexts in theoretical, empirical, and programmatic terms.
This edited collection seeks to expand our understanding of heritage speakers of Spanish by incorporating research on their linguistic, sociolinguistic, and pragmatic development during and after a sojourn abroad, by discussing the complexities of their identity formation and negotiation during immersive stays, and by highlighting programmatic innovations that could be leveraged to better serve diverse learners in study abroad contexts.
This volume advances the fields of both heritage language education and research on immersion study in a variety of ways, and will be of interest to scholars of applied linguistics, sociolinguistics, second language acquisition, and educational linguistics, especially those interested in study abroad programming and Spanish for heritage speakers.
... The difficulty of accounting for the contribution of this measure to the interpretation of this component can be justified by its low coefficient (0.22) in comparison with the loadings of the other positive variables (0.45-0.52). 17 Jarvis et al. (2003) is an example of a feature-driven L2 study that compares proficiency levels of students from different backgrounds. They carried out an agglomerative hierarchical cluster analysis of the frequencies of 21 linguistic features representative of length (text length, mean word length), lexical diversity (type/token ratio), lexical reinforcement (hedges, amplifiers, emphatics, downtoners), lexical involvement/objectification (nouns and nominalisations, personal/impersonal pronouns, verbal tenses, demonstratives), syntactic sophistication (frequency of conjuncts, passives, adverbial/relative subordination, complementation), etc., in two sets of L2 English placement compositions and essays written by Arabic-, Chinese-, Japanese-, and Spanish-speaking university-level students. ...
This paper investigates 'linguistic complexity' in academic language. The notion of complexity, as understood here, is approached by considering a number of lexical, structural, and syntactic features as generated by computational tools. The hypotheses are 'learner academic language deviates from native language as regards linguistic complexity' and 'proficient learner language involves a greater degree of complexity'. The data on L2 academic language are retrieved from the Written Corpus of Learner English (WRICLE), which includes academic writings by Spanish university students from the A2 to the C2 level of the Common European Framework of Reference for Languages. The L1 data comprise A-level essays extracted from the Louvain Corpus of Native English Essays (LOCNESS). The software used are Coh-Metrix and the L2 Syntactic Complexity Analyzer (L2SCA). Coh-Metrix provides basic cohesion, lexical, syntactic, and semantic-discoursive features, as well as other metrics of lexical diversity and readability. L2SCA automates the analysis of complexity by using different measures of lexical density, taken from the literature on L1 and L2 acquisitions. Principal Component Analysis and an ordered logistic regression analysis are used to determine the most significant groupings of features (or variables) in the dataset.
Poiché si tratta di elementi linguistici «poco salienti a livello percettivo» (Chiapedi, 2010), gli articoli rappresentano una delle aree più problematiche nell’insegnamento dell’italiano L2, specie nel caso degli apprendenti la cui madrelingua non possiede una tale categoria lessicale. Benché nel parlato una persona possa comunicare in modo più o meno efficace pur utilizzando gli articoli in modo erroneo, nella scrittura questo tipo di errori diventa particolarmente visibile in quanto la definitezza non può essere espressa che verbalmente, e quindi una debole padronanza del sistema di articoli L2 potrebbe minare il rendimento accademico degli studenti. Infatti, sul piano pedagogico, molti studiosi richiamano l’attenzione sul fatto che l’insegnamento degli articoli L2 è un compito che viene spesso trascurato oppure mal eseguito: la differenza tra definitezza e specificità viene trattata molto di rado, e ancora meno la funzione espletiva dell’articolo nei soggetti preverbali. L'obiettivo di questo contributo è quello di proporre un modello semplificato per l'insegnamento degli articoli italiani L2, sviluppato principalmente a partire dell’ipotesi della fluttuazione elaborata da Ionin, Ko, Wexler (2004), nonché sulla base della corretta interpretazione dell’articolo determinativo come operatore iota (ing. ι-operator; Chierchia, 1998) e dell’approccio euristico proposto da Testa (2021). Il modello qui proposto, dunque, si propone ridurre il numero di regole da memorizzare per gli studenti di madrelingue prive di articoli, dato che l’elevato numero di regole descritte nelle grammatiche tradizionali potrebbe sovraccaricare le risorse cognitive degli apprendenti, difficoltando così il processo di acquisizione.
The purpose of this study is to investigate the developmental writing patterns of English as a foreign language (EFL) students across different score levels through a TOEFL Junior® writing task. The sample of this study comprised 2,016 young adolescent EFL students, ranging from 10 to 15 years old, who took the TOEFL Junior test. We performed a detailed analysis of student essays to identify possible patterns in the development of young adolescent EFL students’ academic writing ability. Linguistic characteristics, such as lexical, syntactic, and discourse features, as well as organizational and content features specific to the argumentative writing genre, were examined by using automated writing evaluation tools and human ratings. The results of descriptive statistics, significance testing, and regression analyses yielded notable patterns for linguistic accuracy and complexity variables. Certain linguistic and content-related features also were found to be highly associated with the writing quality of the young EFL students in the sample. Implications for the instruction and assessment of young adolescent EFL students’ academic writing ability are discussed.
To explore factors that may affect learners’ longitudinal developmental trajectories, we investigated whether L2 learners show divergent trajectories of second-language writing development at a group level when they iterate a narrative writing task procedure in class from the perspective of complex dynamic systems theory. One-hundred five university students who were learning English as a foreign language in Japan and spoke Japanese as a mother tongue performed a timed narrative-writing task procedure once a week over one academic year (30 times). A growth mixture model was used to identify three groups with different developmental patterns: stagnating, steadily increasing, and markedly increasing. A general growth mixture model was used to indicate that the three groups had different achievements not only in writing performance but also in the course grade. It was also revealed that the grouping was influenced by the type of reflecting the students did after writing a composition. These findings suggest that individuals orient and act on a task differently, resulting in divergent developmental trajectories even at the group level. The results also indicate that detailed, rather than simple and superficial, reflection on one’s writing may promote learning through task iteration.
This study examines an under-explored interactive register of discipline-specific online academic forum posts in higher education. 881 academic forum posts written by postgraduate students were analysed, using an additive multi-dimensional (MD) analysis. Four dimensions of linguistic variation were compared across the two first language backgrounds (L1 Chinese and L1 English), three subregisters (application design, business scenario, and discussion), and grades. The results indicated that the forum posts written by two L1 backgrounds, three subregisters, and grades significantly differed in terms of dimensions of linguistic variation. The findings have important implications for writing instruction and assessment in higher education. One of the significant implications of the present study is that both first and second language writers need to have mastery of informational and elaborated discourse features (nouns, phrasal coordination) to write successful academic forum posts at graduate level. Writing instructors may provide successful model texts for relatively new registers so that L2 writers can meet the multiple demands of online academic forum posts, including interacting with peers and supporting their ideas with academic literature.
This study is an attempt to report the varying profiles of Complexity, Accuracy, and Fluency (CAF) in students’ writing. In practice, the written performance of 22 Iranian freshman students at university was analysed. Having taught the writing skills during a semester, the researcher collected and analysed the final written performance of test takers through CAF rubrics to report their final examination writing profiles. Results of the study revealed that the relative contribution of each feature could to some extent assist to inform understandings of the problematic areas in written performance. In sum, the findings of this study may corroborate the very idea of using analytic measures to determine objective writing assessment. The wider discussion of this study refers to using Cognitive Diagnostic Assessment (CDA) in the written performance of students as a complement to the CAF profile analysis as a way of determining the processes, strategies, and knowledge students possess and activate in producing their exam-based writing scripts.
This study reports on the revision of a rating scale for an ESL writing placement test and demonstrates how collaboration with teachers can both enhance the assessment practice within an ESL writing program and raise teachers’ awareness about assessment literacy. Following a data-driven approach to scale development, teachers participated in a three-stage revision process, where they were asked to (1) reflect on the range of writing performances in ESL courses, (2) evaluate sample essays from the test and revise descriptors for the new scale, and (3) pilot-rate new essays using the new scale. During the first stage, both teachers and testers recognized that test takers display different strengths and weaknesses in argument development and lexico-grammar. However, when evaluating sample essays, teachers seemed to weigh argument development more heavily, whereas testers placed a higher value on lexico-grammatical accuracy. Additionally, when rating argument development, some teachers relied heavily on surface/structural rhetorical features rather than essay content. These contrasts resulted in conflicting ratings on certain essay profiles. Through several rounds of discussion, these differences were eventually mitigated by creating separate criteria for argument development and lexico-grammar. The revised scale strikes a better balance between argument development and lexico-grammar, more accurately covering the range of writing performances among test takers. The revision process standardized the conceptualization and operationalization of writing quality, shifting teachers’ focus from surface rhetorical features to essay content. In return, collaboration with teachers enhanced testers’ understanding of the local instructional contexts. Teachers’ involvement promoted collaborative assessment-related dialogues and practices within the ESL program, strengthening the alignment across curriculum, instruction, and assessment.
This research focuses on the phonetic distinctions between Arabic and Turkish and the student's work. Based on ACTFL writing competence requirements, the morphological and syntactical differences between Arabic and Turkish in writing an essay about the student's summer vacation. Also, what is the impact of L1 on L2 and L3 writing proficiency? Additionally, the phonological and morphological differences between Arabic and Turkish, and syntactical variations in Arabic and Turkish. Lastly, a grade 11 trilingual IB student is a case study of the research. Also, CBI (content-based instruction method) and different methods could help students enhance their writing proficiency.
The English article system can be taught as a binary division between classification (a and Ø) and identification (the). All the other elements of article usage can be understood within this framework, allowing a one form/one function correspondence for a and the. Furthermore, the notions of classification and identification can be introduced as distinct concepts before the various rules for article usage are taught. This simplified schema is presented as a pedagogical tool for selecting the appropriate article, a universally acknowledged difficulty for nonnative speakers of English.
As the computer is rapidly finding its way into classrooms around the world at all levels of education, teachers are trying to find effective ways to integrate this technology into their curriculum. While the effectiveness of using word processing in the teaching of writing is acknowledged, there is still no general consensus on how to use, or even whether to use, asynchronous electronic mail, leaving a number of questions unanswered. For example, when given comparable academic tasks, do students produce similar texts in the two media or do they write differently according to the medium used? In order to determine whether the medium has an effect on the language that the students produce, a discourse analysis of comparable word-processed and e-mail writing assignments was carried out, focusing on twelve cohesive features and on text length. The students involved in the study were enrolled in a higher-intermediate English as a Foreign Language course at a university in the United States. The results indicate that two of the cohesive features, as well as text length, differentiated e-mail and word-processed writing. It was also found that, while they tended to write shorter texts in both media, Arab students tended to use more of some of the cohesive features than Asian students.
This paper describes the acquisition, frequency, and function of the English articles (a, the, and Ø—the zero article). It explains the two types of zero article (zero and null) and shows how these occur in alternation with a and the. It then provides a framework for the presentation of the articles in the classroom at the three general levels of beginning, intermediate, and advanced proficiency.
This paper describes the cluster analysis technique and how it can be used to support menu interface design for in-vehicle multimedia applications. Cluster analysis and similar types of classifying technqiues have proven effective for developing simple menu interfaces. This paper extendes the use of the cluster analysis technique to a more complex system that consists of 201 generic functions. These functions are representative of those being incorporated into near-term multimedia products. Study results show promise for using cluster analysis as a tool for incorporating the user's organizational structure into the design of a complex menu architecture. Cluster analysis may also benefit the automotive menu designer by providing a means for partitioning menu tasks into chunkable units that can be easily accessed by the driver in single glances.
Much of the research on L2 article acquisition has investigated the effects of semantic, syntactic, and discourse universals on the sys- tematicity and variability of learners' article use. The present paper looks at systematicity from the combined perspective of two putative discourse universals related to topic continuity (e.g., Givon, 1983) that have been addressed only separately in past studies of article acquisition: the tendency to mark the distinction between topics and comments (e.g., Huebner, 1983) and the tendency to mark the dis- tinction between new, continuous, and reintroduced NP referents (e.g., Chaudron & Parker, 1990). The present study examines how well these discourse universals account for the patterns of article use and nonuse found in narratives written by 199 Finnish-speak- ing and 145 Swedish-speaking adolescent learners of English. The quantitative results of the study cast some doubt on learners' sensi- tivity to the topic-comment distinction and also suggest that learners' tendency to mark distinctions between new, continuous, and reintro- duced NP referents is influenced by the prominence of such distinc- tions in the L1. The quantitative results are supported by a qualitative analysis of a subset of the data that suggests numerous other ele- ments that are needed to characterize the systematicity of individual learners' interlanguage article systems.
This article describes an experiment that observed the effects of dialogue journaling through electronic mail on the language produced by learners of Spanish as a second language, compared with the paper-and-pencil version of the technique. The authors statistically analyzed the quality and quantity of discourse generated via the electronic and the traditional (i.e., paper-and-pencil) medium. The primary objective was to determine whether the use of electronic mail had any effect on grammatical accuracy, appropriate use of vocabulary, and language productivity. In addition, the participants completed a written survey at the end of the semester that elicited their opinions of the program's effectiveness. It was found that the electronic version of dialogue journals had a significantly positive effect on the amount of language generated by the students, and that it improved students' attitude towards learning and practicing the target language. However, the electronic version of dialogue journals did not seem to pose any significant advantage over the paper-and-pencil version with regard to lexical and grammatical accuracy.
The use of generic the and its alternatives 0 and a(n) is investigated at a range of discourse levels in a series of articles from Scientific American. Generic the was found to occur less frequently than generic 0 but much more frequently than generic a(n). It was the most likely of the three generic articles to occur with noun phrases in subject position. It was more likely to be found in the first sentence of a paragraph than in the last, and it was more likely to be found in introductions and conclusions than elsewhere. Finally, generic thewas more likely than a(n) or 0 to mark the topic of a scientific essay and other NPs that contribute to the author's argument. These usages appear to reflect the apparent focused, causal nature of generic the, in contradistinction to the more descriptive generic 0 and a(n).
Provided are the Introduction, Chapter 1, and Chapter 2. This 510-page text introduces future teachers of English to the major elements of English grammar in a systematic fashion using step-by-step procedures, charts and diagrams, numerous exercises, and a unique problem-solving approach that prepares the teacher to present grammar to students with confidence and clarity.
Metadiscourse is a construct that is increasingly important in both composition and reading research. The purpose of this study was to explore, first, how metadiscourse can enhance the writer's awareness of readers' needs and, second, how the use of metadiscourse is related to the quality of the texts that students produce. In this quasi-experimental study, university-level student writers in the experimental class were taught metadiscourse in addition to a process method, while those writers in the control class were taught composition through only a process method. Pre- and post-treatment student papers were analyzed to determine whether metadiscourse usage was different and how the interpersonal, textual, and ideational components of the texts in the two groups were affected. The results of the analysis indicated that the experimental group benefited from instruction about metadiscourse: Students in the experimental group produced essays that received significantly higher grades than those in the control group. Qualitative in-depth analyses of the essays of the experimental students further showed that this improvement can be attributed to the use of metadiscourse markers, which made the texts more accommodating toward readers, and to the strengthening of the ideational as well as the interpersonal and textual meanings of the texts. These results suggest that teaching students to use metadiscourse may be an important way to improve their writing skills.
Four writing samples were obtained from 638 applicants for admission to U.S. institutions as undergraduates or as graduate students in business, engineering, or social science. The applicants represented three major foreign language groups (Arabic, Chinese, and Spanish), plus a small sample of native English speakers. Two of the writing topics were of the compare and contrast type and the other two involved chart and graph interpretation. The writing samples were scored by 23 readers who are English as a second language specialists and 23 readers who are English writing experts. Each of the four writing samples was scored holistically, and during a separate rating session two of the samples from each student were assigned separate scores for sentence-level and discourse-level skills. Representative subsamples of the papers also were scored descriptively with the Writer's Workbench computer program and by graduate-level subject matter professors in engineering and the social sciences.
The differential success of second/foreign language learners suggests a need to examine in detail what strategies successful language learners employ. An indication is given of what these strategies might consist of and a list of several widely recognized good learner strategies is given. In addition to the need for research on this topic, it is suggested that teachers can already begin to help their less successful students improve their performance by paying more attention to learner strategies already seen as productive.
This project had three main objectives: (1) to establish a database of essays written by different language groups on a variety of topics for the Test of Written English (TWE) that can be used in future research; (2) to summarize, analyze, and compare the linguistic properties of those essays; and (3) to determine how the TWE performance of language groups relates to essay styles. As part of the first objective of this project we created a database of 1,737 essays, a data matrix of essay variables, files containing sorted phrases and vocabularies of different language groups, and files of the common and unique vocabulary items for each pair of language groups.
This paper uses the agglomerative hierarchical cluster analysis as a grouping procedure to classify relatively similar regional economies into clusters. The hierarchical cluster technique permits a set of macroeconomic indicators to be jointly assessed, as a basis for identifying similar groups of economies better poised for monetary integration. In particular, economies of the European and Asia-Pacific regions were assessed respectively. The cluster results supported a natural 'core' and 'periphery' group of European countries prior to the formation of the European monetary union (EMU). When the EMU convergence criteria were applied to the Asia- Pacific economies, it is found that they were not sufficient to sort the heterogeneous regional economies into relatively similar clusters. Due to the diversity of the Asia-Pacific region, a practical approach towards regional monetary integration would be to begin with smaller clusters first. The rationale is that the economic costs of monetary integration reduce with the degree of similarity or convergence of its potential union members.
This paper reports the impact of metadiscourse, a rhetorical domain that regulates the communicative function of language, on sixth‐grade students’ learning and attitudes. Metadiscourse directing the reader how to regard the author and the text was divided into two types, informational and attitudinal, and inserted in a social studies textbook chapter in either interpersonal or impersonal voice. Students were placed in high‐vocabulary or low‐vocabulary groups on the basis of test scores; in low‐comfort or high‐comfort groups on the basis of comfort index scores. The significant interaction results showed differential effects of metadiscourse on students’ learning. Low‐comfort students learned more when informational metadiscourse was presented in interpersonal voice, but high‐comfort ones learned less. The results for attitudinal metadiscourse were the opposite. The effect on attitudes was that students who read just one type of metadiscourse were more tolerant of opinions than those who read both types or none at all. The issues concerning interaction and metadiscourse are clearly in need of discussion.
Following up on recent work by Malvern and Richards (1997, this issue; McKee et al., 2000) concerning the measurement of lexical diversity through curve fitting, the present study compares the accuracy of five formulae in terms of their ability to model the type-token curves of written texts produced by learners and native speakers. The most accurate models are then used to consider unresolved issues that have been at the forefront of past research on lexical diversity: the relationship between lexical diversity and age, second language (L2) instruction, L2 proficiency, first language (L1) background, writing quality and vocabulary knowledge. The participants in the study comprise 140 Finnish-speaking and 70 Swedish-speakinglearners of English, and an additional group of 66 native English speakers. The data include written narrative descriptions of a silent film, and the results show that two of the curve-fitting formulae provide accurate models of the type-token curves of over 90% of the texts. The texts for which accurate models were obtained were subjected to further analyses, and the results indicate a clear relationship between lexical diversity and amount of instruction, but a more complicated relationship between lexical diversity and L1 background, writing quality and vocabulary knowledge.
Metadiscourse refers to writers' discourse about their discourse—their directions for how readers should read, react to, and evaluate what they have written about the subject matter. In this study the authors divided metadiscourse into textual metadiscourse (text markers and interpretive markers) and interpersonal metadiscourse (hedges, certainty markers, attributors, attitude markers, and commentary). The purpose was to investigate cultural and gender variations in the use of metadiscourse in the United States and Finland by asking whether U.S. and Finnish writers use the same amounts and types and whether gender makes any difference. The analyses revealed that students in both countries used all categories and subcategories, but that there were some cultural and gender differences in the amounts and types used. Finnish students and male students used more metadiscourse than U.S. students and female students. Students in both countries used much more interpersonal than textual metadiscourse with Finnish males using the most and U.S. males the least. The study provides partial evidence for the universality of metadiscourse and suggests the need for more cross-cultural studies of its use and/or more attention to it in teaching composition.
Hedging is a well-documented feature of spoken discourse as a result of its role in qualifying categorical commitment and facilitating discussion. Its use in academic writing has received less attention, however, and we know little about the functions it serves in different research fields and particular genres. Hedging is a significant communicative resource for academics since it both confirms the individual's professional persona and represents a critical element in the rhetorical means of gaining acceptance of claims. Hedges allow writers to anticipate possible opposition to claims by expressing statements with precision, caution, and diplomatic deference to the views of colleagues. Based on a contextual analysis of 26 articles in molecular biology, this paper argues that hedging in scientific research writing cannot be fully understood in isolation from social and institutional contexts and suggests a pragmatic framework which reflects this interpretive environment.
Although similarities and differences between speech and writing have often been studied, contradictory claims concerning the linguistic relationship between the two modes are still common. These contradictions can arise from basing global conclusions on restricted methodologies-such as assigning undue weight to individual linguistic features, or to choice of particular text samples and text types. The present study uses a 'multi-feature/multi-dimension' approach, which includes a broad range of linguistic features and text types in a single quantitative analysis, to provide a global description of similarities and differences among spoken/written text types in English. The distribution of 41 linguistic features in 545 text samples of approximately 2000 words each is subjected to factor analysis (a multivariate statistical technique). Three underlying textual dimensions are identified: Interactive vs. Edited Text, Abstract vs. Situated Content, and Reported vs. Immediate Style. To demonstrate the value of the multi-feature/multi-dimension approach, the specific findings of earlier studies are reconciled within the model proposed here.
Part I. Background Concepts and Issues: 1. Introduction: textual dimensions and relations 2. Situations and functions 3. Previous linguistic research on speech and writing Part II. Methodology: 4. Methodological overview of the study 5. Statistical analysis Part III. Dimensions and Relations in English: 6. Textual dimensions in speech and writing 7. Textual relations in speech and writing 8. Extending the description: variations within genres 9. Afterword: applying the model Appendices.
Students often see academic writing as an alien form of literacy designed to disguise the author and deal directly with facts. Style guides and textbooks commonly portray scholarly writing as a kind of impersonal, faceless discourse, and EAP teachers direct students to remove themselves from their texts. But how realistic is this advice? In this article I briefly explore the most visible expression of a writer's presence in a text: the use of exclusive first person pronouns. I show that not all disciplines follow the same conventions of impersonality, and that in fact there is considerable scope for the negotiation of identity in academic writing. I argue that by treating academic discourse as uniformly impersonal we actually do a disservice to our students, and that as teachers, we might better assist them by raising their awareness of the options available to them as writers.
The ways that writers distinguish their opinions from facts and evaluate the certainty of their assertions is central to the meaning of academic texts, yet this is an area that second language students often find extremely problematic. In this paper I examine the view that the items writers use to modify their claims, commonly referred to as hedges and boosters, may actually be unnoticed by L2 readers, a phenomenon Low (1996) calls the 'Lexical Invisibility Hypothesis'. Data is presented from a small retrospective think-aloud study which explores how 14 Cantonese L1 undergraduates respond to hedges and boosters in an academic text. The discussion is supported by questionnaire data which seeks to determine learners' awareness of the meanings of these forms. The results suggest that while the subjects generally attended to the boosters, hedges did seem to be more invisible.
A study compared the types of information included in English narratives written by monolingual American students, English narratives by bilingual Mexican students, and Spanish narratives by monolingual Mexican students, and possible differences by grade level. Subjects were in six groups: monolingual (English-speaking) American; bilingual Mexican; and monolingual (Spanish-speaking) Mexican, in fourth and ninth grades. Data were drawn from written narratives elicited with a silent animated film of an animal fable, a total of 10 stories in each of the 6 analysis groups. Results indicate that although general narrative structure did not differ for Mexicans and Americans, Mexicans provided more information about emotional states and personal traits. With respect to developmental differences, the younger writers tended to include fewer physical and action details, provide fewer causal links between actions and states, and provide significantly less information about characters' emotional states and physical and personal traits. Bilinguals also showed a developmental lag in the second language with respect to inclusion of information about setting and action details, and in proportion of propositions allocated to different parts of the story. No evidence of first-to-second-language transfer appeared. Contains 29 references. (MSE)
This study focuses on the impact of different variables on the nature of language performance in the context of task-based instruction. Characteristics of tasks are discussed, and then a framework is offered that can organize the nature of task-based instruction and relevant research. The framework is used to generate predictions regarding the effects of three different tasks (Personal Information Exchange, Narrative, and Decision-Making) and three different implementation conditions for each task (unplanned, planned but without detail, detailed planning) on the variables of fluency, complexity, and accuracy. The study reports strong effects of planning on fluency and clear effects also on complexity, with a linear relationship between degree of planning and degree of complexity. However, a more complex relationship was discovered between planning and accuracy, with the most accurate performance produced by the less detailed planners. In addition, interactions were found between task type and planning conditions, such that the effects of planning were greater with the Narrative and Decision-Making tasks than with the Personal Information Exchange task. The results are discussed in terms of an attentional model of learning and performance and highlight the importance of tradeoff effects between the goals of complexity and accuracy in the context of the use of limited capacity attentional resources. The study contributes to the development of cognitive models of second language performance and addresses a number of pedagogic issues.(Received August 25 1995)
The study of metadiscoursal components of academic texts, through which writers organise, interpret and evaluate content matter, provides one means of examining the relationship between writer and reader. This paper explores one grammatical feature of metadiscourse, clauses with an anticipatory it and extraposed subject (as in ‘It is interesting to note that no solution is offered’). This feature is compared in two computerised corpora of text, one consisting of published journal articles from the field of Business Studies and the second of MBA student dissertations written by non-native speakers of English. It-clauses are found to have four main interpersonal roles in hedging, marking the writer's attitude, emphasis, and attribution. The main differences between the two corpora are in the use of it-clauses to persuade readers of the validity of claims, with student writers making an apparently greater and more overt persuasive effort, and stating propositions more forcefully. Proposals are made on why this might be the case, taking into account the different writer-reader relationships in journal articles and dissertations. Implications of the findings for the teaching of academic writing are presented.
In scientific English it is conventional for present tense to signal general truth (scientific universality); the past tense is then used to report the author's own research actions and findings. Both NS and NNS scientific English may, however, depart from the present tense convention - for different reasons. Data collected from a reception study in which 45 readers from eight countries evaluated and annotated the same three Discussion sections written by Dutch biologists (but not yet corrected by an NS) form the basis for a discussion of these reasons. The NNSs' competence in English, mother tongue interference and non-anglophone tense conventions for reporting past events are dealt with. The readers' responses to the preponderance of present tense in the texts appeared to be inconsistent. Possible reasons for this are suggested and the implications of the findings for writers, teachers, editors and reviewers are discussed.
This study examined the extent to which a computerized tagging program was able to capture proficiency level differences of second language (L2) learners' essays. A sample of 90 Test of Written English (TWE) essays, written at three levels of proficiency as defined by TWE ratings, were tagged for features of essay length, lexical specificity (type/token ratio and average word length), lexical features (e.g., conjuncts, hedges), grammatical structures (e.g., nouns, nominalizations, modals), and clause level features (e.g., subordination, passives). The results indicate that computerized tagging can be used to reveal detailed differences among proficiency levels, but that additional coding into the program or tagging by hand is necessary to gain a more complete picture of differences in L2 students' writing.
Nonnative speakers (NNSs) of English in U.S. colleges and universities often hove difficulty writing adequate academic prose. One research area which has sought to identify and solve the problems of English as a Second Language (ESL) writing is contrastive rhetoric: the study of texts written in English by native speakers (NSs) of different languages to determine syntactic and rhetorical differences. This study examined 768 essays written in English by native speakers of Arabic, Chinese, Spanish, and English in order to determine whether distinctive, quantifiable differences in the use of four cohesion devices existed between and among the four language backgrounds. The corpus consisted of four essay prompts: two topic types and two topic tasks for each topic. The Writer's Workbench (WWB), a computer text-analysis program originally developed by AT&T/Bell Laboratories, was used to analyze the four cohesion variables in the corpus. Results of the analyses showed frequent co-occurrence of certain cohesion devices that differed significantly between and among language backgrounds and between topic types.
Process theories have been extremely influential in the evolution of L2 writing instruction. Responding to purely formal views of writing, proponents borrowed the techniques and theories of cognitive psychology and L1 composition to refine the ways we understand and teach writing. While remaining the dominant pedagogical orthodoxy for over 30 years, however, process models have for some time found themselves under siege from more socially-oriented views of writing which reject their inherent liberal individualism. Instead, genre approaches see ways of writing as purposeful, socially situated responses to particular contexts and communities. In this paper, I discuss the importance of genre approaches to teaching L2 writing and how they complement process views by emphasising the role of language in written communication.
A text is composed of two parts: propositional content and metadiscourse features. Metadiscourse features are those facets of a text which make the organization of the text explicit, provide information about the writer's attitude toward the text content, and engage the reader in the interaction. In this study, we analyze the metadiscourse in persuasive essays written by English as a second language (ESL) university students. Half of the essays received good ratings and half received poor ratings. Differences between the two sets were found in the number of words, number of T-units, and density of metadiscourse features. When features were analyzed as a proportion of number of T-units, differences were found in all categories. Furthermore, the good essays showed a greater variety of metadiscourse features within each category than the poor essays. It is proposed that skilled writers have an awareness of the needs of their readers and control the strategies for making their texts more considerate and accessible to the reader. Poor writers, on the other hand, are not able to generate considerate texts.
This study investigated the efficacy of integrating task-based e-mail activities into a process-oriented ESL writing class. In particular, it examined the linguistic characteristics of 132 pieces of e-mail writing by ESL students in tasks that differed in terms of purpose, audience interaction and task structure. The analysis focused on the linguistic features of the students' e-mail writing at different levels, i.e. syntactic complexity, lexical complexity and grammatical accuracy. Computerized text analysis programs were used to ensure internal consistency of the linguistic analysis. Statistical analysis of the results using the repeated measures analysis of variance and post hoc contrast tests showed significant syntactic, lexical and grammatical differences in the students' e-mail writing of the different tasks. Specifically, in e-mail tasks involving audience interaction, students tended to produce syntactically and lexically more complex texts, and in tasks which allowed students self-selection of topics and content, students also tended to use more complex sentences and richer and more diverse vocabulary. However, an interesting trade-off effect was observed between linguistic complexity and grammatical accuracy in the students' e-mail writing, indicating the complexity of the second language writing process. The study provides pedagogical implications for designing effective e-mail tasks for enhancing second language writing development.
The term discourse analysis has been used interchangeably in two separate contexts — spoken discourse (i.e., multiple-source dialogic) and written discourse (i.e., single-source monologic). Such a distinction, however, oversimplifies the situation; while there are obvious overlaps between the two, to some extent each has evolved in its own direction. Written discourse analysis, the subject of our discussion, is obviously closely connected with work in literacy, but it implicates a great heterogeneity of topics and approaches, including at least some from psycholinguistics and sociolinguistics. Discourse analysis, in the sense in which we are using it, emerged in the early 1970s. A modern history of written discourse analysis is perhaps best covered within a 40–50-year time span. In the course of that time, a number of new and emerging disciplines and research fields have contributed to systematic analyses of the linguistic features and patterns occurring in written texts. At the same time, other continuing disciplines have provided contributions that have been important and are ongoing. It should be fairly evident that any attempt to cover such a broad spectrum of views and disciplines would not be appropriate in a single article. We therefore intend to limit the scope of this paper to analyses of written discourse that explore the actual structuring of the text via some consistent framework. Our goal is to highlight and describe historically the various efforts to find the structures and linguistic patterns in texts that contribute to how they are understood, interpreted, and used. It seems to us that, in order to comprehend what has happened in the context of L2 writing research, it is necessary to understand the extensive work that has been done in discourse analysis.
Thesis (Ph. D.)--Northern Arizona University, 1994. Includes bibliographical references.
Counts of register features have produced useful data on varieties of written English and on differences between writers of differing writing and language proficiency. This article reports an attempt to extend this procedure to an evaluation of the same foreign-language writers at two different stages It shows that the procedure can differentiate the products of developing writers at two relatively close points in time, and that a more detailed examination of significant changes can be revealing about patterns of learning The major changes were from features of spoken English to those more typical of formal writing, both in surface detail and in more fundamental characteristics. There was less change in complexity of construction or variety of vocabulary improved correctness in the structures used was balanced by errors in new structures being attempted. The subjects had been discriminating in their acceptance of academic style and actively sensitive to genre and other requirements
This study reports the association of myelodysplasia with Turner syndrome.
An 11-year-old girl with Turner syndrome was found to have mild macrocytic anemia that persisted during 2 years.
Examination of the bone marrow revealed dyserythropoietic features with multinucleation consistent with refractory anemia. Levels of hemoglobin F were also markedly elevated (57%). She also had transient neutropenia and thrombocytopenia, as well as abnormal platelet function studies. The hematopoietic abnormalities were mild and may have been missed were she not followed for her hypertension and aortic coarctation.
Myelodysplastic syndromes in children are frequently associated with chromosomal abnormalities, but an association with Turner syndrome has not been previously described. This could be due to the fact that mild hematopoietic abnormalities in these patients may not be investigated.
Lexis in composition: A performance analysis of Swedish learners’ written English
- M Linnarud
Linnarud, M. (1986). Lexis in composition: A performance analysis of Swedish learners' written
English. Malmö, Sweden: CWK Gleerup.
Cluster analysis and the identification of learner types
- P Skehan
Skehan, P. (1986). Cluster analysis and the identification of learner types. In V. Cook (Ed.),
Experimental approaches to second language learning (pp. 81-94). Oxford: Pergamon.