ArticlePDF Available

Using global complexity measures to assess second language proficiency: Comparing CLIL and non-CLIL learners of English and Dutch in French-speaking Belgium

Authors:
A preview of the PDF is not available
... Section 4.3.2.2.) give an indication of their receptive vocabulary. For more information on their overall written proficiency, we refer to Bulon et al. (2017) and Van Mensel et al. (forthcoming). These studies showed different results according to the target language: the Dutch CLIL students outperformed the non-CLIL students with respect to lexical diversity and lexical and syntactic complexity, while the English CLIL students outperformed the non-CLIL students only with respect to lexical diversity and syntactic complexity. ...
... Besides the multiple-choice test, the participating students were also invited to write an e-mail to a friend, telling them about a party they had been to or about their holiday (15-25 lines). These written materials were used to calculate various measures of written proficiency (Bulon et al. 2017). We used the MTLD (Measure of Textual Linguistic Diversity) calculated on the learners' written productions as a proxy for productive TL vocabulary knowledge. ...
Article
Languages differ in their preferences for particular intensifying constructions. While intensifying adjectival compounds (IACs) (e.g. ijskoud, ice-cold) are productively used to express intensification in Dutch and English, in French this construction is hardly productive. Consequently, French-speaking learners may encounter difficulties acquiring IACs in Dutch/English. As part of a research project on CLIL in French-speaking Belgium, we explore the effect of CLIL on the acquisition of IACs in the target language (TL) Dutch/English through a multiple-choice test. The results confirm that CLIL students (learning English/Dutch) develop greater receptive knowledge of these constructions. Furthermore, the more frequent IACs are more likely to be recognized by the learners. Moreover, even when the CLIL effect is considered alongside other factors, such as the students' extracurricular exposure to the TL and their overall vocabulary, CLIL is still an important predictor of the learners' receptive knowledge of English IACs, in addition to productive and receptive vocabulary. By contrast, current informal contact with the TL and receptive vocabulary are significant predictors of learners' receptive knowledge of Dutch IACs, but CLIL does not significantly contribute to the regression model for the latter language.
... extracurricular exposure to the target language) are included, while it remains present in the English one. This converges with similar results obtained for the productive skills of these same pupils presented in Van Mensel et al. (2020b) as well as for their overall target language proficiency (Bulon et al. 2017), in that the gap between the CLIL and the non-CLIL group, if present, appears to be larger for learners of Dutch than for learners of English. ...
Article
Introduction of the Special Issue ‘Assessing CLIL: A multidisciplinary approach’. Please find a copy here: https://www.tandfonline.com/eprint/KQNKDSBBVU7V2MD3S2FK/full?target=10.1080/13670050.2021.1935441
... В эпоху глобализации английский язык как основное средство коммуникации должен активно осваиваться как в письменной, так и в устной форме. Как один из самых важных мировых языков, английский язык предназначен не только для академических нужд студентов, преподавателей или исследователей, но и для ищущих работу, рабочих, деловых людей, предпринимателей, государственных служащих и членов общества [10][11][12]. В рамках государственного экономического университета студенты профиля «Туристский бизнес» имеют большую, по сравнению со студентами других профилей неязыкового вуза, мотивацию к овладению иностранным языком, а именно не только ради хороших оценок по предмету, но и для того, чтобы иметь надежные коммуникативные навыки, чтобы в дальнейшем получить более привлекательные предложения на работу. ...
Article
Full-text available
This paper discusses ways to optimize the process of students professional foreign language competence development at a non-linguistic university, based on students language and speech training at the economic university training program of Tourism business, which combines a comprehensive business training with entrepreneurial approach to future qualified professionals working in the leisure and tourism industry. This involves acquisition of necessary competencies by students, aiming them at the prospect of becoming entrepreneurs and managers who are able to realize their working potential both at the national and international level. This, in turn, imposes quite high requirements for professional language training at a non-linguistic university and implies fluency in a foreign language by graduates. An additional catalyst for the Samara Region and a dozen other regions of the Russian Federation was the 2018 FIFA World Cup, which stimulated the formation of modern infrastructure of the tourism industry and confirmed relevant changes in the field of higher education. Globalization and the Internet have changed all aspects of foreign language training, including at a non-language university, especially for the tourism and hotel industry. Many of the strategies that have been applied in higher education until recently are no longer effective at present because of new and constantly evolving technical capabilities.
... The MulTINCo database has already been fruitfully exploited in several studies. Bulon et al. (2017) look at the global written proficiency of CLIL and Non-CLIL learners in the two L2s (English/Dutch) and in their L1 (French) using a number of lexical and syntactic measures. In a follow-up study, examine the influence of different types of input (CLIL, formal instruction and informal contact) on the learners' variability in writing. ...
Article
Full-text available
Whilst the links between learner corpus research (LCR) and Second Language Acquisition (SLA) have long been debated, McEnery et al. (2019. “Corpus Linguistics, Learner Corpora, and SLA: Employing Technology to Analyze Language Use.” Annual Review of Applied Linguistics 39: 74-92. doi:10.1017/S0267190519000096) claim that learner corpus data are not yet sufficiently integrated in SLA research. This article aims to go one way towards bridging the LCR/SLA gap by illustrating the benefits of collecting and analyzing data sets that better document multiliteracy practices. We first contextualize our work within the field of LCR where calls for more multidimensional data sets have been made. We then present a new database called MulTINCo - Multilingual Traditional, Immersion, and Native Corpus - collected in the framework of a project on Content and Language Integrated Learning in French-speaking Belgium. As our data set contains rich metadata and blends corpus data with other data types, we illustrate its potential for SLA research. In Sections 3 and 4, we describe the data collected and the interface. In the last section of the paper, we wrap up with a discussion on the methodological assets of such multidimensional data sets for SLA studies, and present directions for future research.
... At present a large-scale project is being conducted in French-speaking Belgium, correlating linguistic performance with cognitive, educational and socio-affective variables in Dutch and English CLIL(Bulon, Hendrikx, Meunier, & Van Goethem, 2017). ...
Article
Full-text available
Content and language integrated learning (CLIL) has expanded in Europe, favored by the large body of research, often showing positive effects of CLIL on L2 development. However, critical voices have recently questioned whether these positive findings apply to any language, given that most research focuses on English. Taking into account this concern, the present study investigated the (productive and receptive) vocabulary development in L2 English and L2 French of the same group of learners within a CLIL context. The aim was not to evaluate the benefits of CLIL over non-CLIL, but, instead, to examine whether vocabulary gains in CLIL learning are language-dependent. More specifically, this study included 75 Flemish eight-grade pupils who had CLIL lessons in both English and French. The results show that although the pupils have a larger English vocabulary, the level of improvement (from pretest to posttest) is not different across the languages. The findings indicate that within CLIL vocabulary knowledge also develops in languages other than English.
... Focus group dataThe observational and focus group data collected enable us to carry out more qualitative analyses and to provide us with more interpretative elements for the various analyses carried out.The availability of multiple data types makes it possible to investigate the interplay between various aspects in SLA. Whilst some of the studies carried out in the framework of the project include purely linguistic analyses (seeBulon et al. 2017 ...
Article
Full-text available
The paper first briefly reviews some of the research data and methods used to track development in Second Language Acquisition (SLA) studies. The focus is then narrowed down to data and methods in learner corpus research (LCR) and arguments for the use mixed-method research approaches are presented. The third section consists in a concrete illustration of how such mixed-methods can be implemented through the presentation of a multidisciplinary project on the acquisition of L2s in immersive and non-immersive settings. The last section includes concluding remarks.
... Focus group dataThe observational and focus group data collected enable us to carry out more qualitative analyses and to provide us with more interpretative elements for the various analyses carried out.The availability of multiple data types makes it possible to investigate the interplay between various aspects in SLA. Whilst some of the studies carried out in the framework of the project include purely linguistic analyses (seeBulon et al. 2017 ...
Conference Paper
Full-text available
The paper first briefly reviews some of the research data and methods used to track development in Second Language Acquisition (SLA) studies. The focus is then narrowed down to data and methods in learner corpus research (LCR) and arguments for the use mixed-method research approaches are presented. The third section consists in a concrete illustration of how such mixed-methods can be implemented through the presentation of a multidisciplinary project on the acquisition of L2s in immersive and non-immersive settings. The last section includes concluding remarks.
Thesis
RESUME : Ce mémoire prend sa source dans le domaine des Sciences de l’Éducation et celui de la Psychologie Linguistique. Il aborde la question de l’acquisition d’un langage et d’une seconde langue en portant une attention particulière à l’appropriation des connaissances lexicales. L’apprentissage des langues étrangères demeure une préoccupation actuelle en matière d’Éducation. La présente étude s’intéresse à un type de pédagogie proposée en Fédération Wallonie-Bruxelles : l’immersion linguistique. Ainsi, ses fondements seront explicités et comparés à ceux de l’acquisition de la langue maternelle. Pour aborder la partie empirique de ce mémoire, une distinction sera faite entre l’immersion précoce et l’immersion tardive. Le design de recherche imaginé vise la comparaison entre trente-quatre élèves issus des deux groupes énoncés. Ainsi, un pré-test rendant compte de la connaissance du vocabulaire des élèves des deux groupes sera donné. Ensuite, dix-huit expressions écrites d’élèves d’immersion précoce seront comparées aux seize élèves d’immersion tardive en termes de variété lexicale. Les résultats de la recherche ont pu révéler une différence significative de connaissance de base du vocabulaire pour les 1000 premiers mots utilisés en anglais entre les deux groupes grâce au pré-test donné. Ensuite, différentes comparaisons de moyennes ont été faites entre les deux groupes quant à leur connaissance du vocabulaire classé selon le Cadre Européen des Langues. Le logiciel utilisé est « Text Inspector » d’ « English Profile, Cambridge. » Une différence entre les deux groupes a été significative concernant le pourcentage obtenu de niveau A1 pour les expressions écrites non-corrigées. Aussi, une différence entre les deux groupes s’est révélée quant aux résultats de variété du vocabulaire de niveau A2 uniquement, et lorsque les expressions écrites ne sont pas corrigées. De manière générale, ces résultats ont été nuancés et nous ne sommes pas parvenu à établir de distinction majeure entre les deux groupes concernant leurs connaissances du vocabulaire. Finalement, la recherche pourrait être poursuivie après une temps d’apprentissage plus conséquent mais pourrait également se concentrer sur d’autres composantes de maitrise d’une langue étrangère : les compréhensions à la lecture ou à l’audition ou encore lors de productions orales.
Article
The present study analyses the impact of a bilingual Content and Language Integrated Learning (CLIL) programme vis-à-vis a regular monolingual programme on the development of different aspects of L2 learners’ linguistic (syntactic, morphological and lexical) complexity. Five pupils enrolled in a Dutch–English CLIL programme in a secondary school in the Netherlands are compared with five peers following the mainstream programme with English as a Foreign Language (EFL) teaching. The longitudinal development of these ten pupils’ linguistic complexity in L2-English is investigated by means of six complexity measures calculated for each of eleven writing tasks collected over a period spanning their first nineteen months of secondary education. Linear mixed models are used to estimate the effects of time and programme type on the pupils’ L2 complexity. The results indicate that both groups of learners significantly increase the complexity of their L2 writing over the course of the study. Surprisingly, only limited effects of programme type (CLIL vs non-CLIL) are found, despite considerable differences in the quantity and quality of instructional exposure to the target language, suggesting that for these pupils increased and more varied instructional exposure to the L2 in the CLIL programme did not lead to significantly different L2 productions in terms of linguistic complexity. Several possible explanations for these findings are considered and the implications for CLIL research are discussed.
Article
Full-text available
As part of a project on Content and Language Integrated Learning (CLIL) in French-speaking Belgium, this study aims to explore the impact of formal and informal input on learners' variability in writing, and to compare two target-language conditions (Dutch and English) in CLIL and non-CLIL settings in French-speaking Belgium. A regression model shows that CLIL is a significant predictor of L2 outcomes for both target languages, but that the relative impact of formal and informal input differs depending on the target language. In short, the amount of formal language exposure predicts the outcomes of the written productions of the learners of English, and the frequency of informal exposure those of the learners of Dutch. We argue that this observation is likely related to the difference in status that each of these languages holds among the pupils in our sample. The findings thus highlight the importance of the L2 status in research on CLIL, since different L2s can yield different results.
Article
Full-text available
This study examined cross-language transfer of morphological awareness in Chinese–English bilingual children. One hundred and thirty-seven first to fourth graders participated in the study. The children were tested on parallel measures of compound awareness, vocabulary, word reading and reading comprehension in Chinese and English. They also received measures of English derivational awareness, English phonological awareness and nonverbal reasoning. Structural equation modelling was used to compare a baseline model with only within-language paths to a model with cross-language paths. The cross-language model fit significantly better than the within-language model, suggesting transfer of morphological awareness between English and Chinese. In particular, we observed a bidirectional relationship between English compound awareness and Chinese vocabulary. Furthermore, English compound awareness was a significant predictor of Chinese reading comprehension. The conditions that support transfer of morphological awareness and the impact of transfer on literacy development in Chinese and English are discussed.
Article
Analysing participants' perceptions of a Content and Language Integrated Learning (CLIL) experience is essential in order to provide a clear picture of whether CLIL is effectively implemented. This becomes extremely relevant in primary school contexts, where the way the different groups involved in an educational experience feel tends to be highly related to its potential success. The present paper presents an overview of the students', CLIL teachers' and parents' perceptions after a year of CLIL implementation in five primary schools obtained by means of opinion questionnaires and interviews. Children, except for low achievers, report general satisfaction, mainly because they realise their use of English is meaningful and feel capable of understanding concepts in English. Teachers' main concerns included their students' low level of English, lack of materials, lack of institutional and peer support and not being competent enough in the content subject. Parents unrealistically perceive CLIL as the only solution to their children's low level of English or fear it might be a threat to the students' L1 and content knowledge. Although a great effort on the part of all parties involved in the process is required, CLIL is already perceived as a positive practice that promotes motivation, learning and interest in the foreign language.
Article
The relationship between lexical diversity and overall quality of test takers' task performances is claimed, explicitly or implicitly, in rating scales of many large-scale language tests including MELAB (Michigan English Language Assessment Battery), as well as in automated evaluation systems. This paper reports an empirical study that aimed to understand the lexical diversity of MELAB test takers' writing and speaking performances. In particular, it analyzed a representative sample of MELAB 2004-2005 archive data to investigate the relationships between lexical diversity and overall quality scores on test takers' writing and speaking task performances and general language proficiencies, the differences between test takers' writing and speaking lexical diversity, and to what extent different topics and topic types of the writing prompts affected lexical diversity measures. It was found that D (Malvern & Richards, 1997)—a measure of lexical diversity—had statistically significant positive correlation with the overall quality ratings of both writing and speaking performances as well as the test takers' general language proficiencies. Nevertheless, the significant relationships did not bear out equally for every subgroup of the sample in terms of gender, first language background, purpose of taking the MELAB test, and topics of the writing prompts. The different topics and topic types of the writing prompts had significant effects on lexical diversity—especially when the test takers were highly familiar with the topics—even after controlling the test takers' writing abilities and overall language proficiency. The lexical diversity of test takers' writing and speaking performances were approximately at the same level; however, D was found to be more able to predict test takers' speaking than their writing performances. The implications of these findings are discussed with specific reference to the use of lexical diversity measures to inform test validation and the development of lexical diversity parameters in the emerging area of automated evaluation systems in language testing. Lexical diversity has been considered an illuminative predictor of learners' general
Article
In this article, we examine current practices in the measurement of syntactic complexity to illustrate the need for more organic and sustainable practices in the measurement of complexity, accuracy, and fluency (CAF) in second language production. Through in-depth review of examples drawn from research on instructed second language acquisition, we identify and discuss challenges to the evidentiary logic that underlies current approaches. We also illuminate critical mismatches between the interpretations that researchers want to make and the complexity measures that they use to make them. Building from the case of complexity, we point to related concerns with impoverished operationalizations of multidimensional CAF constructs and the lack of attention to CAF as a dynamic and interrelated set of constantly changing subsystems. In conclusion, we offer suggestions for addressing these challenges, and we call for much closer articulation between theory and measurement as well as more central roles for multidimensionality and dynamicity in future CAF research.
Article
In this study I evaluate the cumulative evidence on the use of syntactic complexity measures as indices of college-level L2 writers' overall proficiency in the target language. Based on a synthesis of twenty-five studies, I arrive at several substantive findings. First, I conclude that the relationship between L2 proficiency and L2 writing syntactic complexity varied systematically across studies depending on whether a second or a foreign language learning context was investigated and whether proficiency was defined by programme level or by holistic rating. Second, aggregating available cross-sectional findings, I propose critical magnitudes for between-proficiency differences in syntactic complexity for four measures. Finally, I interpret the limited longitudinal evidence to suggest that an observation period of roughly a year of college- level instruction is probably needed for substantial changes in the syntactic complexity of L2 writing to be observed. I conclude the paper by discussing implications of these findings for future primary research.
Article
In 1987 an early total immersion program based on the Canadian model was established in Vaasa/Vasa, Finland. It is evident that the functional approach of the immersion program has consequences for the teacher, the test methods and the language usage of the children. This paper is mainly centred on an analysis of six‐year‐old Finnish‐speaking children's second language acquisition during their first year of Swedish immersion in kindergarten. A conclusion to be drawn from the analysis is that a very important didactic task for the teacher is to provide the children with meaningful routines, in which the role of the second language, both quantitatively and qualitatively is gradually increased. The importance of context‐bound situations is illustrated by the code‐switching strategies in the children's language usage. In the second part of the paper further evidence of the importance of building up ritualised context‐bound situations with standardised language usage is demonstrated on the basis of the children's first second language utterances.
Article
This study investigated the relationships among 3 types of vocabularyknowledge (passive, controlled active, and free active) within the same individuals, taking 4variables into consideration: passive vocabulary size, language learning context, second (L2) orforeign (FL), length of residence in L2 context and, among the Canadians, knowledge of French.Participants were adult learners of English in Israel (N = 79) and in Canada (N = 103) at different proficiency levels. We used The Levels Test for passivevocabulary size, a Controlled Active Vocabulary Test and The Lexical Frequency Profile (forlexical richness in free written expression). We found that the 3 dimensions of vocabularyknowledge developed at different rates. Active, particularly free active, vocabulary developedmore slowly and less predictably than did passive vocabulary. Furthermore, the relationshipsamong the 3 dimensions of vocabulary knowledge differed between the 2 learning contexts.Although passive vocabulary was always significantly larger than controlled active and freeactive, the passive-active vocabulary gap was smaller in the FL than in the L2 context. Thebenefits of residence in an L2 context only began to appear after about 2 years, as passivevocabulary was activated and the gap reduced. In the Canadian context, knowledge of Frenchwas an asset at the earlier stages of ESL learning.
Article
This article presents a study examining how narrative structure and narrative complexity might influence the performance of second language learners. Forty learners of English in London and sixty learners in Teheran were asked to retell cartoon stories from picture prompts. Each performed two of four narrative tasks that had different degrees of narrative structure (loose or tight) and of storyline complexity (with or without background events). Results support the findings of previous research that tight task structure is connected to increased accuracy and that narratives involving background information give rise to more complex syntax. A comparison of the data from the London and Teheran cohorts showed that the learners in London used significantly more complex syntax and diverse vocabulary even though they did not differ from the Teheran learners in other performance dimensions.