No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Perspectives on validity: A historical perspective on Language Testing Conference abstracts
Abstract
looks at abstracts from LTRC conferences 1979-1995 to discover how far new perspectives on validity have entered the field second language testing.
... To svoje uvjerenje iznio je i mnogo godina kasnije tvrdeći kako se pragmatička kompetencija zapravo ne razlikuje ni od stvarne (neidealizirane) jezične kompetencije (znanja o određenim jezicima) ni od komunikacijske kompetencije razmatrane sa sociolingvističkoga ili stilističkoga gledišta. Mišljenja je, naime, da su sociokulturne razlike uloge i statusa kodirane, gramatički ili leksički, u strukturu određenih jezika (nekih više nego drugih), iz čega proizlazi da jezično kompetentna osoba mora posjedovati ne samo jezični i sociolingvistički nego i sociokulturni element jezične kompetencije (Lyons 1998.). U skladu s tako širokim gledanjem na jezičnu kompetenciju govornika, Lyons (ibid.) ...
... No tu je nejasnoću pokušao otkloniti na nešto drukčiji način od Hymesa. Lyons (1998.) uvodi pojam jezik sustav (engl. ...
PREDGOVOR
Komunikacijska kompetencija pojam je s kojim se često susreću stručnjaci u svom profesionalnom zanimanju za jezik i bavljenju njime. Osobito je učestala uporaba toga pojma u području obrazovanja. Na svim obrazovnim razinama komunikacijska se kompetencija ističe kao jedan od važnijih ob-razovnih ciljeva. U jezikoslovnim nastavnim predmetima, napose u stranim jezicima, komunikacijska kompetencija nije samo glavni obrazovni ishod nego se opis njezina razvoja provlači kao crvena nit i kroz sadržaj nastav-noga plana i programa. S obzirom na važnost i učestalost uporabe pojma komunikacijske kompetencije, gotovo trivijalnim čini se pitanje što ona zapravo jest. Ipak, razgovori autorice ove knjige sa studentima diplomskih i poslijediplomskih studija te nastavnicima jezika otkrili su da se komuni-kacijska kompetencija shvaća i opisuje vrlo općenito – kao sposobnost u-porabe jezika u različitim komunikacijskim situacijama. U tom je smislu i pitanje o onome što čini sposobnost jezične uporabe u različitim komuni-kacijskim situacijama često ostajalo dijelom ili u potpunosti bez odgovora.
Glavna je svrha ove knjige ponuditi odgovore na navedena pitanja. Ti se odgovori temelje na spoznajama koje se proizašle iz višedesetljetnih teorij-skih rasprava i empirijskih istraživanja prirode komunikacijske kompeten-cije u raznim znanstvenim disciplinama – lingvistici, sociolingvistici, psi-holingvistici, psihologiji te, posebice, u dvama mlađim znanstvenim pod-ručjima jezičnoga istraživanja kao što su ovladavanje inim jezikom i jezi-čno testiranje.
Jezično je testiranje (engl. language testing) područje znanstvenoga istra-živanja koje se intenzivno počelo razvijati 80-ih godina prošloga stoljeća i ubrzo se potom profiliralo u zasebnu disciplinu u okviru primijenjene ling-vistike. U posljednjih dvadesetak godina došlo je do prave eksplozije za-nimanja za to znanstveno područje, što se odražava u velikom broju istra-živanja i publikacija. Postalo je očigledno da ta mlada disciplina u pogledu teorijskih, metodoloških i praktičnih dostignuća može mnogo ponuditi srodnim disciplinama, a jedna od tih sestrinskih disciplina upravo je ovla-davanje inim jezikom (engl. second language acquisition), etablirana znan-stvenoistraživačka disciplina koja, uz glotodidaktiku, sve više postaje teo-rijskim temeljom suvremene nastave drugoga i stranoga jezika.
Do kraja 80-ih godina na istraživanja u tim dvjema disciplinama gledalo se kao na potpuno različite načine stjecanja spoznaja u okviru primijenjene lingvistike. Krajem osamdesetih godina prošloga stoljeća pojavile su se prve studije u kojima je bilo uočljivo približavanje tih dviju poddisciplina primijenjene lingvistike kako u pogledu teorijskih pitanja tako i u pogledu metodologije istraživanja. Obje discipline imaju isti osnovni predmet istra-živanja – jezična znanja i sposobnosti ili, preciznije rečeno, učenikov me-đujezik, o kojem se u objema disciplinama zaključci izvode na temelju sus-tavnoga promatranja učenikove jezične izvedbe. No tom glavnom predme-tu istraživanja pristupa se različito u područjima ovladavanja inim jezikom i jezičnoga testiranja. Istraživanja u okviru ovladavanja inim jezikom često su longitudinalnoga karaktera i bave se primarno opisom i objašnjavanjem načina razvoja jezičnih znanja i sposobnosti, usredotočujući se pritom na analizu međujezika, tj. procesa i čimbenika koji su uključeni u njegov raz-voj. Istraživanja u okviru jezičnoga testiranja usmjerena su prema pronala-ženju modela komunikacijske kompetencije koji bi postao temeljom opisa i vrednovanja te znanja i sposobnosti učenika na određenom stupnju njego-va jezičnoga razvoja. Iz toga proizlazi da interes istraživača u okviru ovla-davanja inim jezikom zaokuplja razvoj jezika te svi čimbenici i procesi koji utječu na taj razvoj, dok je zanimanje istraživača u jezičnome testiranju usmjereno prema ishodu ovladavanja jezikom, tj. analizi komponenata ko-je čine jezičnu sposobnost (usp. Bachman 1989., Shohamy 1998.). Shodno tomu, navedene dvije discipline koriste se, djelomice, različitom metodo-logijom istraživanja. U području ovladavanja inim jezikom naglasak je na lingvističkim analizama jezičnih izričaja, proučavanju pojedinačnih slu-čajeva ovladavanja jezikom, eksperimentima i slično, dok je za područje jezičnoga testiranja karakteristična primjena testova i njihova kvantitativna analiza, iako se u posljednje vrijeme sve učestalije provodi i kvalitativna analiza, npr. analiza učenikovih odgovora na testu te verbalnih izvještaja o procesu testiranja.
Unatoč navedenim razlikama, te dvije discipline dijele relativno široko po-lje zajedničkoga interesa, a u njemu razumijevanje prirode komunikacijske kompetencije čini jednu od glavnih dodirnih točaka. Budući da su se teo-retičari i istraživači u području jezičnoga testiranja do sada iscrpnije bavili navedenom problematikom i pritom došli do vrlo značajnih spoznaja ne samo o načinu mjerenja, analize i vrednovanja jezične sposobnosti u da-nome kontekstu testiranja nego i o strukturi komunikacijske kompetencije korisnika jezika, ponudivši niz definicija i teorija komunikacijske kompe-tencije, teorijsko i empirijsko polazište za ovu knjigu pružila je, većim dije-lom, upravo ta disciplina. No rezultati i implikacije opisanih istraživanja imaju velik značaj i za teoriju usvajanja i poučavanja stranoga jezika.
Kako je, dakle, jedna od temeljnih metoda za stjecanje uvida u prirodu komunikacijske kompetencije testiranje korisnika jezika, cilj je ove knjige pružiti odgovor i na pitanje kako pristupiti testiranju i vrednovanju komu-nikacijske kompetencije ispitanika, tj. na što obratiti pozornost u procesu istraživanja komunikacijske kompetencije u stranome jeziku. A zadnji je, ali ne i manje važan, cilj ukazati na implikacije pojave i teorijskoga odre-đenja pojma komunikacijske kompetencije, kao i istraživanja prirode ko-munikacijske kompetencije, na praksu poučavanja stranoga jezika.
S obzirom na navedene ciljeve, knjiga je prije svega namijenjena studenti-ma jezikoslovnih poslijediplomskih studija te nastavnicima jezika, a među njima osobito nastavnicima stranoga jezika. Ona bi, međutim, mogla biti zanimljiva i sustručnjacima koji se bave istraživanjima usvajanja i pouča-vanja komunikacijske kompetencije, i to zbog toga što pruža osvrt na do-sadašnja istraživanja i daje smjernice za buduća teorijska i empirijska istra-živanja prirode komunikacijske kompetencije korisnika jezika.
Knjiga je podijeljena u četiri poglavlja: Komunikacijska kompetencija i srodni koncepti, Struktura komunikacijske kompetencije, Vrednovanje ko-munikacijske kompetencije te Implikacije teorijskih i empirijskih istraživa-nja koncepta komunikacijske kompetencije za poučavanje stranoga jezika. Prva tri poglavlja proizašla su iz neobjavljene disertacije „Komunikacijska kompetencija u stranom jeziku“ (Bagarić 2007.). U ovoj su knjizi, među-tim, ona formalno dorađena i sadržajno dopunjena spoznajama koje auto-rici nisu bile dostupne u trenutku pisanja disertacije, kao i spoznajama do kojih su znanstvenici, uključujući i samu autoricu, došli tijekom nekoliko proteklih godina.
Slijedi prikaz poglavlja u knjizi.
U prvome se poglavlju prikazuje proces razvoja koncepta i teorije/teorija komunikacijske kompetencije od kraja 1960-ih godina do danas. Kako je riječ o terminološki i definicijski teško uhvatljivom konceptu, jedini put za sagledavanje svih aspekata njegove kompleksnosti bilo je prikazivanje procesa određenja pojma komunikacijske kompetencije. Stoga se prvo opi-suje uvođenje koncepta komunikacijske kompetencije u lingvistički di-skurs i utjecaj raznih uskolingvističkih i nelingvističkih disciplina na nje-govo oblikovanje u sadržajnome smislu. Nakon toga detaljno se prikazuje razvoj koncepta u okviru primijenjene lingvistike. Pritom se poseban nag-lasak stavlja na odnos koncepta komunikacijske kompetencije s drugim, gotovo u potpunosti istoznačnim, konceptima, npr. s konceptom jezičnoga umijeća, komunikacijske jezične sposobnosti, komunikacijske sposobnosti i komunikacijske jezične kompetencije ili sa srodnim konceptima kao što su koncepti interakcijske kompetencije, interkulturalne komunikacijske kompetencije i drugi, koji su prethodili konceptu komunikacijske kompe-tencije ili su se pojavili nakon njega, a uslijed pokušaja prilagođavanja i ograničavanja njegove uporabe u određenome kontekstu. U nastavku su knjige stoga svi koncepti koji se značenjski u izrazito velikoj mjeri prekla-paju s konceptom komunikacijske kompetencije, koji se neprestano razvija, svrstani pod opći nazivnik komunikacijske kompetencije. Pritom nisu promijenjeni izvorni nazivi tih koncepata. Štoviše, u cijeloj se knjizi u pri-jevodu pojmova s engleskoga ili njemačkoga jezika nastojala zadržati zna-čenjska bliskost s izvornim nazivom, često odabirom posuđenice u hrvat-skome jeziku, a s ciljem da čitatelji dobiju što vjerniji uvid u slojevitost određenoga koncepta i razvoj znanstvene misli u vezi s tim konceptom.
Drugo poglavlje donosi pregled razvoja modela komunikacijske kompe-tencije, kao i relevantnih empirijskih istraživanja koncepta komunikacijske kompetencije odnosno istoznačnih koncepata. Pregled modela obuhvaća prikaz i osvrt na modele komunikacijske kompetencije koji su nastali od trenutka nastanka koncepta komunikacijske kompetencije do danas. Svrha je pregleda pružiti uvid u proces oblikovanja strukture komunikacijske kompetencije i načine tumačenja interakcije među njezinim komponenta-ma. Navedeni pregled prati opis empirijskih istraživanja iz kojih su izravno proizašli prijedlozi nekih modela komunikacijske kompetencije, npr. Olle-rov jednodimenzionalni model jezičnoga umijeća ili Bachmanov i Palme-rov model komunikacijske jezične sposobnosti. Prikaz istraživanja u svije-tu i kod nas dopunjen je opisom istraživanja koja nisu izravno utjecala na oblikovanje modela, ali je njihov utjecaj bio (ili će biti) značajan za razumi-jevanje strukture komunikacijske kompetencije.
U trećem se poglavlju, nakon uvoda u ključne pojmove u okviru jezičnoga testiranja i kratkoga povijesnoga pregleda jezičnoga testiranja, razmatraju određena teorijska pitanja, kao npr. pitanje što testirati i procjenjivati – kompetenciju ili performansu, ali i brojna metodološka pitanja vezana uz problematiku testiranja i procjenjivanja komunikacijske kompetencije na koja su upozorila dosadašnja istraživanja.
Četvrto, ujedno i završno, poglavlje knjige sažima implikacije teorijskih i empirijskih istraživanja komunikacijske kompetencije za praksu poučava-nja stranoga jezika. U tom će poglavlju čitatelj pronaći poveznicu između koncepta komunikacijske kompetencije i pojave komunikacijskoga pristu-pa poučavanju jezika te se upoznati s kritikama toga pristupa i njegovim mogućim razvojem u budućnosti.
Na kraju, autorica upućuje veliku zahvalu svima koji su joj svojim upitima i zanimanjem za sadržaj neobjavljene disertacije pružili poticaj za objavlji-vanje ove knjige.
... Such a trend has apparently been reflected in the research agenda of this scholarly community. For example, an abstract analysis conducted by Hamp-Lyons and Lynch (1998) using data from the Language Testing Research Colloquium from 1979 to 1994, revealed that studies on validity in language assessment reflected predominantly postpositivist thinking at the time with a strong dependence on psychometric methods to support claims of validity. Similarly, Chalhoub-Deville and Deville (2008) conducted an analysis of articles in the two major language assessment journals (i.e., Language Assessment and Language Assessment Quarterly) from their beginnings to 2005, and concluded that quantitative research approaches were dominantly used in the publications. ...
... This strong orientation toward quantitative research methodology stems from the dominant research interest in language assessment revolving around the development of score-valid and score-reliable large-scale standardized tests (Chalhoub-Deville & Deville, 2008;Hamp-Lyons & Lynch, 1998). Hamp-Lyons (2000), describing the developments in the philosophy and epistemology of language testing, stated that the epistemology of early language tests was shaped by a faith in psychometrics and positivism due to a strong influence from intelligence tests. ...
Although an epistemic change often is labeled as a shift, some researchers representing the social sciences consider this paradigm shift as a paradigm expansion (e.g., Pollack, 2007) because epistemological and methodological diversity allows researchers to address a wider range of questions. The third paradigm, mixed methods (MM) research, is claimed to provide a more holistic picture of a research problem by combining two different data sources— quantitative and qualitative—in a single study (Creswell & Plano Clark, 2007; Greene, 2007; Teddlie & Tashakkori, 2009). This article discusses how mixed methods research approaches have been used to enrich the results and to enhance the rigor of classroom-based language assessment investigations, drawing on both the language testing and assessment (hereafter, language assessment) and classroom assessment literature in second language education. The article opens with a brief overview of the methodological evolution in language assessment research. Then, focusing on MM research studies that investigated various facets of classroom-based language assessment (CBLA) practices, the main part of this article outlines a proposal of MM methodology as an appropriate methodology for research on CBLA, particularly in a context where a new form of assessment is implemented. The article closes with a discussion of challenges that MM researchers might face, and a proposal of MM research methodology as an appropriate research approach for CBLA scholars, especially in settings where both the validation of assessment and the explanation of the phenomenon are required.
... One possible reason for the study's findings is that if requirements of test validation are not satisfied, tests cannot measure what they are intended to measure (Fulcher & Davidson, 2007;McNamara, 2006). Furthermore, study's results support this longstanding view that test validation incorporates three essential elements, including content, construct, and criterion (Bachman & Palmer, 2010;Hamp-Lyons & Lynch, 1998). In alignment with Messick (1994Messick ( , 1996, the study's findings may be explained from the viewpoint that validity gives meaning to test scores; that is, validity evidence shows that test performance is closely linked to performance outside the class. ...
This study aims to disclose the Iranian university teachers’ perceptions of the fundamentals of language assessment literacy (LAL). To this aim, using purposive sampling, eighteen university teachers from two Iranian universities were invited to participate in semi-structured interviews. Their viewpoints were audio-recorded, transcribed, and subjected to a phenomenographic analysis. Findings yielded two overarching LAL domains: knowledge (e.g., having an acceptable level of digital LAL, satisfying ethical requirements, benefiting more from performance assessment, considering students’ individual differences, making assessment valid, assuring that tests are reliable, and having an acceptable level of pedagogical content knowledge) and skills (e.g., involving students in assessment, using alternative assessment methods, employing logically traditional assessment methods, informing students about test results, administering tests in standardized ways, using valid grading procedures, and bringing positive wash-back effects). After discussing the results, the study concludes with proposing a range of implications for different testing stakeholders and highlighting some avenues for further research.
... In language testing, models of language have been a frequent focus of attention over the lifetime of the Language Testing Research Colloquium (Hamp-Lyons & Lynch, 1998). For the past two decades, much systematic research has examined the construct validation of the concept of communicative competence in second language education (e.g., Bachman & Palmer, 1996;Harley, Allen, Cummins, & Swain, 1990;Jamieson, Jones, Kirsch, Mosenthal, & Taylor, 2000;Palmer, Groot, & Trosper, 1981) and in language testing (e.g., Milanovic, Saville, Pollitt, & Cook, 1996;Swain, 1985;Wesche, 1981). ...
This study responds to the Test of English as a Foreign Language™ (TOEFL ® ) research agenda concerning the need to understand the processes and knowledge that test‐takers utilize. Specifically, it investigates the strategic behaviors test‐takers reported using when taking the Speaking section of the TOEFL iBT™ (SSTiBT). It also investigates how the reported strategic behaviors differed across integrated and independent tasks in the SSTiBT, as well as the relationship between test‐takers' reported strategic behaviors and their performance on the tasks as determined by their test scores. The participating students were 14 graduate and 16 undergraduate engineering students whose first language was Chinese.
The results indicate that test‐takers reported using 49 separate strategies when completing the SSTiBT tasks. Of the five strategy categories, the metacognitive, communication, and cognitive strategies were proportionally the most frequently reported. The interrelationships among these three categories were negative. Undergraduates reported using significantly more communication strategies, whereas graduates reported using significantly more cognitive and affective strategies. No statistically significant differences were found in reported strategy use across proficiency levels. The integrated tasks were more alike with respect to reported strategy use than were the independent and integrated tasks. Furthermore, the integrated tasks elicited a wider variety of reported strategy use than the independent tasks. Overall, we found no relationship between the total number of reported strategic behaviors and total test score on the SSTiBT.
We conclude that strategy use is integral to performing SSTiBT tasks and should therefore be considered as part of the construct of communicative performance. However, the relationship between strategy use and test performance is varied and is due to complex interactions among test‐taker characteristics, tasks, and contexts.
... Validity, from this perspective, is the degree to which the specific performance or test score matches constructs or meanings that educators have predetermined as the object of assessment. Comparing current generalized understandings with new, contextualized interpretations is done in a reciprocal fashion rather than in the scientific hypothesis-testing mode; that is, there is a give and take between one interpretation and the next that results in a reshaping or reconceptualization of both (Hamp-Lyons & Lynch, 1998). ...
Assessment is a powerful tool which can shape curriculum, teaching and learning. One of the major topics which has been the focus of so much debate among the scholars is the concept of ethics in educational assessment systems. The purpose of this paper is to touch upon the issue of ethics in language assessment from validity perspectives. To ensure effective planning, implementation, and evaluation of learning, certain principles and guidelines for beliefs and behaviors are adopted. These principles and guidelines are identified as ethical standards. The central question to be addressed here is whether any test can be defended as ethical, or moral. This study also examines professional ethics, specifically ethical principles as they relate to educational assessment. And finally, some suggestions and guidelines will be offered for applying ethics in educational assessment to maximize validity and fair application of language test scores.
... In language testing, models of language have been a frequent focus of attention over the lifetime of the Language Testing Research Colloquium (Hamp-Lyons & Lynch, 1998). For the past two decades, much systematic research has examined the construct validation of the concept of communicative competence in second language education (e.g., Bachman & Palmer, 1996;Harley, Allen, Cummins, & Swain, 1990;Jamieson, Jones, Kirsch, Mosenthal, & Taylor, 2000;Palmer, Groot, & Trosper, 1981) and in language testing (e.g., Milanovic, Saville, Pollitt, & Cook, 1996;Swain, 1985;Wesche, 1981). ...
This study responds to the Test of English as a Foreign Language[TM] (TOEFL[R]) research agenda concerning the need to understand the processes and knowledge that test-takers utilize. Specifically, it investigates the strategic behaviors test-takers reported using when taking the Speaking section of the TOEFL iBT[TM] (SSTiBT). It also investigates how the reported strategic behaviors differed across integrated and independent tasks in the SSTiBT, as well as the relationship between test-takers' reported strategic behaviors and their performance on the tasks as determined by their test scores. The participating students were 14 graduate and 16 undergraduate engineering students whose first language was Chinese. The results indicate that test-takers reported using 49 separate strategies when completing the SSTiBT tasks. Of the five strategy categories, the metacognitive, communication, and cognitive strategies were proportionally the most frequently reported. The interrelationships among these three categories were negative. Undergraduates reported using significantly more communication strategies, whereas graduates reported using significantly more cognitive and affective strategies. No statistically significant differences were found in reported strategy use across proficiency levels. The integrated tasks were more alike with respect to reported strategy use than were the independent and integrated tasks. Furthermore, the integrated tasks elicited a wider variety of reported strategy use than the independent tasks. Overall, we found no relationship between the total number of reported strategic behaviors and total test score on the SSTiBT. We conclude that strategy use is integral to performing SSTiBT tasks and should therefore be considered as part of the construct of communicative performance. However, the relationship between strategy use and test performance is varied and is due to complex interactions among test-taker characteristics, tasks, and contexts. Appendices include: (1) A List of Strategic Behaviors; (2) Analysis of Test Scores; (3) Pretest Proficiency Screening; (4) Individual Profile Questionnaire; (5) Stimulated Recall Instructions; (6) Coding Scheme; (7) Results of Normality Tests; (8) Descriptive Statistics for Individual Strategies by Test-Taker Study Level; (9) Descriptive Statistics for Individual Strategies by Task Group; (10) Descriptive Statistics for Individual Strategies by Task; and (11) Excerpts Illustrating Impact of Stimulated Recalls on Test-Takers' Behavior. (Contains 19 tables, 4 figures, and 16 notes.)
... Of all stakeholders in testing events, test takers surely have the highest stake of all. And yet Hamp-Lyons and Lynch (1998) found that in the papers given by language testers at the main annual meeting, test takers' voices are those least heard. Clearly, participation by all test takers is both impractical and unnecessary; yet there seem good reasons to include test takers' points of view within test development and implementation consultation. ...
The paper explores some of the developments in the philosophy and epistemology of language testing with particular reference to the notion of ‘responsibility’. This term is used to include the ethical practices of the individual language testing practitioner; the professional standards to which language testing as professional practice should be held; and the accountability to society and by society that should drive all those involved in the industry of creating, administering, consuming, preparing people for, and critiquing tests. The paper argues that as the millennium changes, language testing is at a point of change too. Contending with complexities on multiple levels, the future is still unclear, but we can see that it will take us in some directions quite different from the past.
Nos documentos oficiais que orientam o ensino no Brasil, o objetivo primeiro da avaliação é promover a aprendizagem (BRASIL, 2018; 2019; SILVA; FURTOSO, 2017). No entanto, o que se percebe na prática é uma avaliação de caráter quase que exclusivamente somativo, e usada como uma ferramenta de controle, poder e punição (SCARAMUCCI, 2006). Este contraste é tanto uma representação quanto uma consequência da falta de espaço para a avaliação em cursos de formação de professores de línguas (QUEVEDO-CAMARGO, 2020), uma questão também identificada em outros contextos (FULCHER, 2012). Essa lacuna na formação de professores evidencia a necessidade de promover o letramento em avaliação em contexto de línguas (LAL) desses profissionais. O LAL se refere aos conhecimentos, habilidades e princípios necessários para que práticas avaliativas bem embasadas sejam realizadas (GIRALDO; 2018; 2020; INBAR-LOURIE, 2008a). Considerando a lacuna identificada, o principal objetivo desta tese foi identificar o LAL de formadoras de professores de inglês, buscando implicações positivas na práxis (FREIRE, 2019) e nos contextos dessas formadoras. Os objetivos específicos versaram sobre os componentes do LAL norteadores da práxis das formadoras, o potencial de ações para a promoção do LAL, e as implicações dessa promoção. Para isso, adotou-se uma metodologia qualitativa crítica caracterizada como estudo de caso (COHEN; MANION; MORRISON, 2018). As participantes foram 15 professoras formadoras de sete universidades públicas do estado do Paraná que participaram de uma oficina online em avaliação, com duração de 20 horas, planejada com base em suas necessidades. Os instrumentos de geração de dados foram a) gravações transcritas da oficina; b) artefatos de aprendizagem; e c) questionários. Os dados passaram por um processo de análise de conteúdo (BARDIN, 2016; SALDAÑA, 2013) parcialmente facilitado pelo software Atlas.ti. As categorias de análise foram as tendências observadas na literatura, bem como aquelas originadas pelos dados. Os resultados apontam que, de forma geral, os três componentes do LAL se mostram presentes nos documentos norteadores dos contextos, mas, na práxis das formadoras, os conhecimentos e as habilidades se destacam, enquanto os princípios ficam em segundo plano, o que representa uma tendência já identificada na literatura (DAVIES, 2008). As ações da oficina que mais promoveram o LAL das formadoras foram as que fomentaram a colaboração entre as participantes, que culminou no estabelecimento de uma comunidade de prática (WENGER, 1998), e aquelas que promoveram seus conhecimentos em LAL, o que difere da literatura com professores em formação (GIRALDO; MURCIA, 2019). As implicações reportadas mais recorrentes da promoção do LAL foram a ampliação de conhecimentos e princípios. Ademais, algumas formadoras relataram ações no sentido de promover o LAL de outros protagonistas, como seus alunos ou colegas. Assim, advogamos a favor de mais iniciativas para o desenvolvimento colaborativo e situado do LAL de professoras formadoras. Essa promoção pode ter um efeito cascata em outros protagonistas, além de ser essencial para que mudanças em concepções e práticas ocorram, com vistas a um processo avaliativo mais coerente, bem fundamentado, justo e orientado para a aprendizagem na formação inicial e continuada de professores/as de línguas no Brasil.
Language testing has been defined as one of the core areas of applied linguistics because it tackles two of its fundamental issues: the need to define and reflect on the appropriateness of Second Language Acquisition models and constructs by data analysis from language tests and the importance of facing the ethical challenge deriving from the social and political role language tests play nowadays. Language testing has thus a twofold impact in a variety of contexts. In the first instance, it constitutes a scientific impulse for which research is needed to develop and implement the technical design of tests. Secondly, language testing has also become subject of debating because the use and interpretation of test results introduce ethical issues concerning the concept of ‘fairness’ in the construction, administration, evaluation and interpretation of language tests. In fact, language tests are always designed and used to make decisions on the basis of a process in which information about test takers is gathered from an observed performance under test conditions. This inevitably leads to the development of codes of ethics in educational testing environments and to the elaboration of theories of validity and validation.
Owing to their scope, and decisiveness, Ph. D. program entrance exams (PPEE) ought to demonstrate acceptable reliability and validity. The current study aims to examine the reliability and validity of the new Teaching English as a Foreign Language (TEFL) PPEE from the perspective of both university professors and Ph. D. students. To this end, in-depth unstructured interviews were conducted with ten experienced TEFL university professors from four different Iranian state universities along with ten Ph. D. students who sat both the new and old PPEEs. A detailed content analysis of the data suggested that the new exam was assumed to establish acceptable reliability through standardization and consistency in administration and scoring procedures. Conversely, the new exam was perceived to demonstrate defective face, content, predictive, and construct validities. This study further discusses the significance and implications of the findings in the context of Iranian TEFL higher education.
Speaking tasks involving peer-to-peer candidate interaction arc increasingly being incorporated into language proficiency assessments, in both large-scale international testing contexts, and in smaller-scale. for example course-related. ones. This growth in the Popularity and use of paired and group orals has stimulated research, particularly into the types of discourse produced and the possible impact of candidate background factors on performance. However, despite the fact that the strongest argument for the validity of peer-to-peer assessment lies in the claim that such tasks allow for the assessment of a broader range of interactional skills than the more traditional interview-format tests do, there is surprisingly little research into the judgements that are made of such performances. The fact that raters, and rating criteria, are in a crucial mediating position between output and Outcomes, warrants investigation into how raters construe the interaction in these tasks. Such investigations have the potential to inform the development of interaction-based rating scales and ensure that validity claims are moved beyond the content level to the construct level. This paper reports the findings of a verbal protocol study of teacher-raters viewing the paired test discourse of 17 beginner dyads in a University-based Spanish as a foreign language course. The findings indicate that the raters identified three interaction parameters: non-verbal interpersonal communication, interactive listening, and interactional management. The findings have implications for our understanding of the construct of effective interaction in paired candidate speaking tests, and for the development of appropriate rating scales.
The thought of Samuel Messick has influenced language testing in 2 main ways: in proposing a new understanding of how inferences made based on tests must be challenged, and in drawing attention to the consequences of test use. The former has had a powerful impact on language-testing research, most notably in Bachman's work on validity and the design of language tests. Messick's writing on test consequences has informed debate on ethics, impact, accountability, and washback in language testing in the work of several researchers. But the character of Messick's work challenges us in many additional ways. Messick located validity theory in the area of values. This article explores the implications of this position, highlighting the social construction of language test constructs. Language test constructs are increasingly the target of policy, a development that threatens to render traditionally conceived validation work of only marginal relevance. The less obvious, covert social construction of language test constructs is explored in the light of Butler's theory of performativity. The article concludes with a consideration of recent adaptations of Messick's work in the influential validation models of Mislevy and Kane, and considers their failure to address questions of values and the social context of assessment properly. Tackling these questions is the ongoing challenge of Messick's legacy.
There has been great recent interest in the social and policy context of language testing and assessment, to the point where it represents perhaps the most significant area of new thought and debate in the field of language testing. The topic has also produced a ferment of activity, reflected in policy statements, special symposia, and special issues of professional journals devoted to aspects of the subject. These discussions have been accompanied by the beginnings of a movement away from the traditional positivist, asocial paradigm for language testing research as traced in Lynch and Hamp-Lyons (1996; Hamp-Lyons and Lynch in press).
ResearchGate has not been able to resolve any references for this publication.