ArticlePDF Available
Open scholarship in applied linguistics: What, why,
and how
Meng Liu1* , Sin Wang Chong2, Emma Marsden3, Kevin McManus4,
Kara Morgan-Short5, Ali H. Al-Hoorie6, Luke Plonsky7, Cylcia Bolibaugh3,
Phil Hiver8, Paula Winke9, Amanda Huensch10 and Bronson Hui11
University of Cambridge, Cambridge, UK,
University of Edinburgh, Edinburgh, UK,
University of York, York, UK,
State University, University Park, USA,
University of Illinois Chicago, Chicago, USA,
Royal Commission for Jubail and
Yanbu, Jubail, Saudi Arabia,
Northern Arizona University, Flagstaff, USA,
Florida State University, Tallahassee, USA,
Michigan State University, East Lansing, USA,
University of Pittsburgh, Pittsburgh, USA and
University of Maryland,
College Park, USA
*Corresponding author. Email:
(Received 6 August 2022; accepted 14 August 2022)
1. Introduction
In November 2021, the UNESCO Recommendation on Open Science was adopted by the UNESCO
General Conference. This marks a significant milestone in the development of Open Science as the
new norm for research, with intended influences on national laws and practices of the 193 UN
Member States and far-reaching implications beyond the UN. It is against such a background that
Open Applied Linguistics (, a new Research Network affiliated with the
International Association of Applied Linguistics (AILA), was established in early 2022 and a two-day
online symposium on open scholarship in applied linguistics was held in June 2022.
Organized by Meng Liu and Sin Wang Chong, convenors of Open Applied Linguistics, the sym-
posium was the inaugural event of the network and was sponsored by Cambridge University Press
& Assessment. There were 19 sessions by 31 speakers and presenters across career stages, covering
a wide range of topics. While our symposium included speakers from neighbouring fields such as
psychology, due to limited space, we selected six sessions that directly concentrated on applied linguis-
tics to include in this report.
2. What do we mean by open scholarship?
Open scholarship is the theme of our symposium. Specifically, we follow the UNESCO (2021) defin-
ition with our own adaptations:
Open [scholarship] is defined as an inclusive construct that combines various movements and
practices aiming to make multilingual [scholarly] knowledge openly available, accessible, and
reusable for everyone; to increase [scholarly] collaborations and sharing of information for the
benefits of [scholarship] and society; and to open the processes of [scholarly] knowledge creation,
evaluation, and communication to societal actors beyond the traditional [scholarly] community.
It comprises all [scholarly] disciplines and aspects of scholarly practices, including basic and
applied sciences, natural and social sciences and the humanities, and it builds on the following
key pillars: open [scholarly] knowledge, open [scholarship] infrastructures, research communica-
tion, open engagement of societal actors, and open dialogue with other knowledge systems. (p. 7)
While we mostly use open scholarship,open research, and open scienceinterchangeably, we delib-
erately replaced the term sciencewith scholarshipin the above definition to highlight our aim to
© The Author(s), 2022. Published by Cambridge University Press
Language Teaching (2023), 56, 432437
doi:10.1017/S0261444822000349 Published online by Cambridge University Press
promote a more inclusive and critical understanding of open science. Readers familiar with the history
of open science in psychology know that it has traditionally been associated with quantitative research
and it was only recently that alternative, equally valid epistemologies have received more attention in
this space (see Bennett, 2021, for example, for a discussion on open science from the perspective of
feminist psychology). A recent survey within applied linguistics (Liu & De Cat, in press) also revealed
some concerns from qualitative researchers on open science having unduly quantitative connotations.
Nonetheless, both the UNESCO recommendations and our symposium demonstrate that open science
can be and should be understood as more inclusive. By using open scholarship, a term less commonly
used than open science, we hope to inspire curiosity, mitigate misconceptions, and facilitate a more
inclusive understanding of open scholarship/research/science (henceforth OS).
3. Selected talks and panels from the symposium
3.1 Open research practices: Value versus sustainability
Marsden and Bolibaughs session focused on open research practices. Marsden started the session with
the rationales for and the value of OS. Rationales for OS can be philosophical, social, and scientific,
ranging from epistemic responsibility and social equity to enhanced rigour, reliability, and validity.
Within applied linguistics, there have been several initiatives to promote OS. IRIS (Instruments and
Data for Research in Language Studies; Marsden et al., 2016), an open repository for instruments
and data established in 2011, is one of the first initiatives to promote open materials and data, and
soon IRIS will increase its scope from a focus on second languages to all language-related studies.
The Open Accessible Summaries in Language Studies (OASIS; Marsden et al., 2018a) is another exem-
plary initiative for research accessibility. While much progress has been made and journals are con-
tinuing to join the initiative, there are persisting challenges in terms of sustainability (e.g., active
engagement from journal editors, cost of infrastructures, time required for data preparation). It is
also important to acknowledge that epistemology may affect the rationales for OS and its potential
benefits some epistemologies and methodologies are not necessarily compatible with, for example,
replication and fully anonymized data sharing.
In Part II of this session, Bolibaugh zoomed in on open data. Reiterating the rationales for open
data, she called for a move from the verification-focused approach (Open Research 1.0) to a more syn-
thetic and collaborative approach (Open Research 2.0). In the case of open data, Open Research 2.0
would entail reusing data for purposes such as synthesis and reconceptualization, in addition to veri-
fication. Surveying the IRIS database, it was found that while the prevalence of open data is on the rise,
the usability and completeness judged according to the FAIR (findable, accessible, interoperable, and
reusable) principles are less than ideal. To move to Open Research 2.0, we still need consensus on what
constitutes data for reproducing published findings, recognition for OS efforts at the institutional and
award levels, training support, and funding for infrastructure.
To conclude, to keep applied linguistics at the forefront of OS, sustained commitment is needed
from all levels in the community, which will understandably be a gradual process.
3.2 TESOLgraphics: Opening up scholarly communication
The session by Chong extended the notion of OS in applied linguistics to include practitioners. This
session reported on an ongoing initiative, TESOLgraphics (, which produces open-
access, one-page infographic summaries of secondary research in all topics of TESOL (Teaching
English to Speakers of Other Languages) to create opportunities for English teachers to engage with
research and develop their evidence-based teaching practices.
Chong began the session by reviewing research on researcher-practitioner relationships in language
education, highlighting that both researchers and practitioners are willing to engage in professional
exchange. Chong discussed the usefulness of secondary research, in particular research syntheses
Language Teaching 433 Published online by Cambridge University Press
(i.e., literature reviews conducted using systematic methodologies), in fostering the research-practice
nexus (Chong, 2019; Chong & Plonsky, 2021).
Introducing the notion of praxisfrom dialectical materialism (Poehner & Inbar-Lourie, 2020),
Chong problematized the nature of current research-practice dialogues. Most research-practice activ-
ities are unidirectional, with researchers imparting knowledge to practitioners. This points to the need
for two-way communication between researchers and practitioners in TESOL, forming an evidence
ecosystem comprising producers, synthesisers, and consumers of evidence (Shepherd, 2014): while
researchers offer empirical and theoretical insights into issues pertaining to teaching and learning,
experience and pragmatic knowledge of practitioners evaluate ecological validity of research findings,
making the focus of educational research more relevant to practitioners.
Responding to the need for more researcher-practitioner dialogues, TESOLgraphics will enter a
new phase of development, inviting English teachers to produce teaching materials and resources
(e.g., lesson plans) inspired by specific infographic summaries. Going alongside the teaching resources
are short video clips where teachers designing the materials reflect on how research findings are trans-
lated into concrete teaching ideas.
3.3 Replication research in applied linguistics: What, why, and how
McManustalk focused on replication research in applied linguistics. Several new initiatives in our field
point to an increased awareness about the need, place, and value of replication research in the growth
and credibility of the discipline (e.g., methodological guides, journal special issues). Yet, as reviews
have noted (e.g., Marsden et al., 2018b; Porte, 2012), support and resources are needed so that
researchers can design, conduct, and report high quality replication studies. This session responded
to that need by introducing novice and experienced researchers to the replication research process.
The key topics introduced were: why replication studies are needed, what a replication study is,
what distinguishes a replication study from other types of research studies, and how to go about
designing a replication study in the field of applied linguistics (see McManus, 2021,in press-a,in
press-b; Porte & McManus, 2019). A brief summary of this sessions main parts is provided here.
To start, a replication repeats a studys methodology with or without changes followed by system-
atic comparison to better understand the nature, repeatability, and generalizability of its findings.
Replication therefore aims to systematically reconsider, refine, extend, and sometimes limit previous
research findings. Because replication is defined by comparison, replication researchers must system-
atically draw comparisons at each stage of the process (design, data handling, reporting), including
using comparative language in reporting and planning how comparisons will be handled in the ana-
lyses. In addition, the more that we change in a replication, the more difficult comparison becomes.
We should therefore aim to keep as much similarity between the studies as possible. Lastly, because
replication is important for all empirical disciplines, clear labelling should be used throughout, includ-
ing the use of replicationin the studys title and abstract.
3.4 Registered reports as an open research practice
In this talk, Morgan-Short discussed Registered Reports (RRs; Center for Open Science, n.d.;
Chambers & Tzavella, 2022; Marsden et al., 2018c) as an open research practice that can improve
the validity, reliability, and replicability of applied linguistics research. RRs is an article type in
which authors motivate and design their study and submit it for peer review BEFORE data collection.
If a study is well motivated and has a valid design protocol, it receives In Principle Acceptance, mean-
ing that the journal agrees to publish the study after data collection regardless of the outcomes of the
results, as long as the authors adhere to the planned protocol and make reasonable conclusions about
the results. Upon receiving In Principle Acceptance, researchers preregister their protocol. Then, they
collect and analyse their data and write their full manuscript, which is reviewed for adherence to the
preregistered protocol and the reasonableness of the conclusion before being published.
434 Meng Liu et al. Published online by Cambridge University Press
Morgan-Short presented emerging metascience suggesting that RRs are effective in attenuating
publication bias and are associated with higher computational reliability, validity, and increased
replication efforts (for more information, see Chambers & Tzavella, 2022). In addition, common con-
cerns about RRs were addressed. For example, exploratory analyses are not restricted in RRs as long as
preregistered confirmatory analyses are reported as planned. Morgan-Short also discussed the concern
that the publication time course of RRs is shifted because of the peer review process that occurs before
data collection, which may be a particular challenge for researchers with inflexible deadlines. Finally,
ideas, examples, and resources for moving forward with RRs were considered, especially the need for
funding support and for more journals to offer RRs, following the lead of Language Learning,
Bilingualism: Language and Cognition, and Second Language Research, where RRs are available as
an article type. To conclude, Morgan-Short noted that the benefits of RRs often greatly outweigh
the challenges and encouraged researchers to adopt this practice to bring about increased validity,
reliability, and replicability to their own research and to our field.
3.5 Open science initiatives: The Postprint Pledge
The session by Al-Hoorie, Hiver, and Nosek was concerned with OS initiatives, specifically promoting
open access in order to democratize scholarship, provide equitable access to researchers from the
Global South, and remove obstacles to accessing the latest literature. According to the classic technol-
ogy adoption lifecycle model (Rogers, 1995), a sociological framework describing the process of innov-
ation acceptance, individuals vary in the speed at which they adopt innovations and initiatives. The
first groups are INNOVATORS and EARLY ADOPTERS. These are then followed by EARLY MAJORITY and
LATER MAJORITY, and finally by LAGGARDS. Building on this model, the session launched the Postprint
Pledge, an initiative for applied linguists and second language acquisition researchers to share the
accepted versionsof their manuscripts. According to the policies of journal publishers, authors are
usually permitted to post their postprints online. Authors may not be aware that they still have the
right to share the accepted versions of their manuscripts, believing that once they sign off the copy-
right agreement, they lose all rights to the manuscript.
As part of this initiative, and to make it more concrete for authors wishing to share the accepted
versions of their manuscripts, a list of 60 applied linguistics journals was compiled and their copyright
policies were reviewed through Sherpa Romeo, an online resource that aggregates the open access pol-
icies of various publishers. Analysis of the copyright agreements of these 60 journals showed that most
publishers have no restrictions on authors sharing postprints, whether on their personal websites and
on nonprofit repositories, though a minority of publishers require an embargo period of one year and
in a few cases two years. The list detailing the copyright policies of each journal in relation to sharing
postprints is currently hosted at, where interested researchers can
see the list of signatories and pledge to share their postprints.
3.6 Speaking openly on open science: A panel discussion
The panel discussion by Plonsky, Winke, Huensch, and Hui sought to bring in a range of voices that
might help us to reckon with some of the new ways of thinking and action that accompany a way for-
ward for the field. There are many facets to embracing an ethos of OS. For the full range of benefits
associated with OS to accrue, we as a culture and community need to shift not only our actions but our
thinking as well. These adjustments, both major and minor, can be seen and felt in virtually all we do
as scholars. The points of view that the panel shared included, among others, those of researcher,
journal editor, manuscript reviewer, researcher trainer, grant writer, grant reviewer, graduate program
director, mentor, and recent graduate.
In an event such as this one, instead of focusing on what OS can bring to our field, the panel spoke
openly on the pushback that has been and continues to be observed in the face of OS initiatives. For
instance, one commonly raised concern is the fear of being scoopedor having a result overturned via
Language Teaching 435 Published online by Cambridge University Press
re-analysis. There may also be the hesitation to freely share something (e.g., data) that one has very
painstakingly collected, cleaned, coded, and prepared. The panel argued that more of a community
or synthetic mindset is needed and that it is an ethical duty to engage in practices that are best for
the collective construction of knowledge rather than solely for career advancement.
In addition to deliberating on potentially thorny issues applied linguists all face such as data shar-
ing, the panel also luxuriated in response to a prompt to dream about where the field might be in
terms of OS ten years from now. Among other visions and hopes, the group imagined scholarly
and institutional structures, incentives, workflow tools, and training protocols with an OS orientation
baked in. Achieving consistency in code books (e.g., variable naming conventions) and in script anno-
tation, for instance, present serious challenges to these dreams, but we see them as worthwhile if not
These discussions, though useful, can sometimes drift into the very conceptual and abstract. In
order to reorient to more practical territory, the session was concluded by each panellist citing concrete
steps toward embracing OS. These included, among others, posting materials and data on IRIS or OSF,
writing an OASIS summary, identifying as part of the OS movement on personal websites, posting
about OS practices on social media, taking the Postprint Pledge (see Section 3.5), and encouraging
journal editors to integrate OS practices into their journalspolicies. Such a list of activities, though
incomplete, may seem daunting. Consequently, the panel urged those who might feel overwhelmed
to put aside the notion that OS is an all or nothing endeavour. The path that will lead to the greatest
uptake in OS at both the field and individual levels and the one that will ultimately prove most effect-
ive for applied linguistics is one of incrementalism.
4. Conclusion
Our symposium addressed the what, why, and how of OS: Initiatives such as IRIS, OASIS,
TESOLgraphics, and the PostPrint Pledge directly promote research accessibility and social equity,
and they indirectly facilitate OS practices such as replication and registered reports, which, in turn,
directly improve research validity, reliability, and reproducibility. A common thread across all is an
ethos of openness, not merely in terms of research practices, but also in the open acknowledgement
of and engagement with the many tensions and challenges regarding OS in applied linguistics, from
epistemological and cultural to psychological and practical. This, we believe, is the ethos that can truly
build us up as a field. While our symposium represents only one of the initial steps in a long journey,
we hope to inspire more aligned and sustained efforts across all levels to incrementally and effectively
advance our field towards a more open future.
Acknowledgements. We would like to thank the other speakers and presenters who contributed to this symposium and
who, due to limited space, we were unable to include in this report (in order of appearance at the symposium): Curtis
Sharma, Inge Alferink, Junlan Pan, Carl Blyth, David Birdsong, Mark Amengual, Kiera McNeice, Dessi Kirilova, Jessica
Miorini, Madeleine Pownall, Flávio Azevedo, Hassan Nejadghanbar, Guangwei Hu, Yaoko Matsuoka, Hiroyuki Ida, Laura
Gurzynski-Weiss, Lara Bryfonski, Derek Reagan, and Brian Nosek.
Bennett, E. A. (2021). Open science from a qualitative, feminist perspective: Epistemological dogmas and a call for critical
examination. Psychology of Women Quarterly,45(4), 448456. doi:10.1177/03616843211036460
Center for Open Science. (n.d.). Registered Reports. Center for Open Science. Retrieved from
Chambers, C. D., & Tzavella, L. (2022). The past, present and future of registered reports. Nature Human Behaviour,6(1),
2942. doi:10.1038/s41562-021-01193-7
Chong, S. W. (2019). The role of research synthesis in facilitating researchpedagogy dialogue. ELT Journal,74(4), 484487.
Chong, S. W., & Plonsky, L. (2021). A typology of secondary research in applied linguistics. OSF. doi:10.31219/
436 Meng Liu et al. Published online by Cambridge University Press
Liu, M., & De Cat, C. (in press). Open science in applied linguistics: A preliminary survey. In L. Plonsky (Ed.), Open science in
applied linguistics. John Benjamins.
Marsden, E., Alferink, I., Andringa, S., Bolibaugh, C., Collins, L., Jackson, C., Kasprowicz, R., OReilly, D., & Plonsky, L.
(2018a). Open Accessible Summaries in Language Studies (OASIS) [Database]. Retrieved from https://www.oasis-data-
Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository of instruments for research into second languages:
Advancing methodology and practice. In A. Mackey, & E. Marsden (Eds.), Advancing methodology and practice: The
IRIS repository of instruments for research into second languages ( pp. 121). Routledge.
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018b). Replication in second language research: Narrative
and systematic reviews and recommendations for the field. Language Learning,68(2), 321391. doi:10.1111/lang.12286
Marsden, E., Morgan-Short, K., Trofimovich, P., & Ellis, N. C. (2018c). Introducing registered reports at language learning:
Promoting transparency, replication, and a synthetic ethic in the language sciences. Language Learning,68(2), 309320.
McManus, K. (2021). Are replication studies infrequent because of negative attitudes? Insights from a survey of attitudes and
practices in second language research. Studies in Second Language Acquisition,114. doi:10.1017/S0272263121000838
McManus, K. (in press-a). Replication and open science in applied linguistics research. In L. Plonsky (Ed.), Open science in
applied linguistics. John Benjamins.
McManus, K. (in press-b). Replication research in instructed SLA. In L. Gurzynski-Weiss, & Y. Kim (Eds.), Instructed second
language acquisition research methods (pp. 103122). John Benjamins.
Poehner, M. E., & Inbar-Lourie, O. (2020). Toward a reconceptualization of second language classroom assessment. Springer.
Porte, G. (2012). Replication research in applied linguistics. Cambridge University Press.
Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. Routledge.
Rogers, E. M. (1995). Diffusion of innovations (4th ed.). Free Press.
Shepherd, J. P. (2014). How to achieve more effective services: The evidence ecosystem. What Works Network, Cardiff
University. Retrieved from
UNESCO. (2021). UNESCO recommendation on open science ( pp. 136). Retrieved from
Cite this article: Liu, M. et al (2023). Open scholarship in applied linguistics: What, why, and how. Language Teaching,56(3),
Language Teaching 437 Published online by Cambridge University Press
... In order to facilitate this, the instrument has been made available on the OSF ( By sharing measures through sites such as this, researchers can help each other to replicate studies and move the field of SLA forwards (Gass et al., 2021;Liu et al., 2021). ...
Full-text available
Self-efficacy has emerged as a popular construct in second language research, especially in the frontline and practitioner-researcher spaces. A troubling trend in the relevant literature is that self-efficacy is often measured in a general or global manner. Such research ignores the fact that self-efficacy is a smaller context-driven construct that should be measured within a specific task or activity where time, place, and purpose domains are considered in the creation of the measurement. Task-based language teaching researchers have also largely neglected the affective factors that may influence task participation, including self-efficacy, despite its potential application to understanding task performance. In this report, we present an instrument specifically developed to measure English as a foreign language students' self-efficacy beliefs when performing a dialogic, synchronous, quasi-formal group discussion task. The instrument's underlying psychometric properties were assessed (N = 130; multisite sample from Japanese universities) and evidence suggested that it could measure a unidimensional construct with high reliability. The aggregate scale constructed from the instrument's items also displayed a central tendency and normal unimodal distribution. This was a positive finding and suggested that the instrument could be useful in producing a self-efficacy measurement for use in the testing designs preferred by second language researchers. The potential applications of this instrument are discussed while highlighting how this report acts as an illustration for investigators to use when researching self-efficacy.
... The momentum for open science is on the rise in applied linguistics. In 2022 alone, there were already several exciting developments: Open Science in Applied Linguistics (Plonsky, in press), the first edited volume on this topic, was being prepared for publication, with several postprints shared online; Open Applied Linguistics (, a newly established research network affiliated with the International Association of Applied Linguistics (AILA) held a two-day symposium to promote open science in applied linguistics (Liu et al., 2023); Following the symposium, AILA officially endorsed the open science guidelines and example practices statement by Open Applied Linguistics (Liu & Chong, 2022), formally recognising open science's value to our field; the Postprint Pledge, an initiative calling for applied linguists to share accepted manuscripts online, was launched (Al-Hoorie & Hiver, 2023); two calls for papers were circulated: one by Studies in Second Language Acquisition on replication in second language research, one by Language Testing on open science practices in language testing and assessment. As we entered into 2023, the momentum continued to grow: in April, Language Learning published a conceptual review along with open peer commentaries, examining whether and why open research practices are the future for the study of language learning; the 56th annual conference of the British Association for Applied Linguistics (BAAL) is set for August, with a theme centred on "opening up applied linguistics". ...
Full-text available
Comments from the reviewing panel: “An extremely timely, thoughtful, and well-informed discussion in applied linguistics and beyond concerning open science. The essay seeks to engage with a broad audience and to dispel some of the persisting myths around open science and would be read fruitfully by many in the field.”
... Furthermore, Jones and Hafner reinforce all chapters with heuristic case studies produced by themselves, fully available e-resources (an annotated glossary of key concepts, links to important websites, videos, etc.), and updated teaching activities to facilitate students' reflection on their digital practices and promote research-pedagogy dialogues. All these efforts are reminiscent of the open scholarship that Liu et al. (2022) have been calling for to advance the agenda of more research accessibility in applied linguistics. In this respect, I believe that Jones and Hafner have contributed to promoting open research in our field through inspiring curiosity, mitigating misconceptions, and fostering an inclusive and critical understanding of what counts as a digitally literate person in the globalized world. ...
Full-text available
Since the first edition of this book was published in 2012, technological innovations and social changes in the past decade have prompted the authors to take the audiences on a new journey to digital literacies. Understanding Digital Literacies: A Practical Introduction (2nd Edition), by Rodney H. Jones and Christoph A. Hafner, reexamines the concept of digital literacies and addresses emerging issues (e.g., digital surveillance, algorithmic impact on literacy practices). Compared with its first edition, this twelve-chapter volume features two completely new chapters (Chapters 6 and 12) and updated resources and activities for individual reflection and pedagogical practices. Reading this second edition, I found I was able to deepen my understanding of the constantly changing landscape of digital literacies. I also felt that the fresh insights provided by Jones and Hafner could enlighten language education researchers and practitioners to rethink what makes a literate person in the digital age. In so doing, we may better facilitate students’ interrogation of digital media as a part of pedagogical imperatives in times of mobility and change.
This systematic review of 170+ journal articles showcases the current trends and developments in Turkey-based applied linguistics and language education research between 2016–2022. The current review presents similarities to the previous reviews (Alptekin & Tatar, 2011; Aydınlı & Ortaçtepe, 2018) in terms of trends and practices that indicate a vibrant research scene and a community of practice in Turkey within language education and applied linguistics research. While certain research areas such as instructional technologies remain widely popular among Turkey-based scholars, there has been growing interest in multiculturalism, social justice language education, critical pedagogy, and culturally responsive teaching. It was also clear in our analysis that the demands by the Higher Education Council for academic promotion exacerbated some of the already-existing challenges also noted in the previous reviews. Some of the concerns include pre- and post-test study designs that focus on practical concerns and outcomes rather than on processes that would lead to conceptual or theoretical development; and lack of engagement with broader (inter)disciplinary debates. We hope that this review will help establish conversations among fellow scholars in terms of future directions that applied linguistics and language education research in Turkey can take in order to contribute to the larger discussions in the field.
Open Research practices can be relevant for every stage of the research process, from conceptualisation through to dissemination. Our purpose is to discuss key facets of Open Research and highlight its rationales, infrastructures, behaviours, and challenges. Part I describes the potential of Open Research initiatives to address problems relating to reliability, validity, replicability, bias, scope (inclusivity), and impact. Part II identifies challenges, such as the speed and cost of Open Research; the usability of open data and materials; the difficulties of conducting replication research; and the economics and sustainability of open access and Open Research generally. We seek to provide examples of good practice, describe and evaluate emerging innovations, and envision change. Part III considers ongoing co-evolutions of culture, infrastructure, and behaviours, and acknowledges the limitations of both our review and Open Research practices. We hope to engender discussion about pathways that yield valid findings about how humans learn languages.
Full-text available
The first official step toward Open Science (OS) in Applied Linguistics (AL) was arguably the establishment of IRIS, a discipline-specific data and instrument repository, back in 2011 (Marsden et al., 2016). Since then, there have been active efforts to improve methodological rigour and research quality (e.g., Gass et al., 2021; Plonsky, 2014; Plonsky et al., 2020), as well as initiatives to enhance research transparency, reproducibility and accessibility, such as Registered Reports at Language Learning (Marsden, Morgan-Short, et al., 2018) and the Open Accessible Summaries in Language Studies (OASIS) initiative (Marsden, Alferink, et al., 2018). Despite these efforts, however, a recent survey (Isbell et al., 2022) on applied linguists with regards to scientific misconduct and questionable research practices (QRPs) painted a less-than-rosy picture: Of the 351 respondents, 94% admitted to at least one form of QRPs in quantitative research. While there have been surveys indicating a general recognition of the value and relevance of OS (cited in Marsden, 2019) and specific OS practices such as replication (McManus, 2021) to the AL community, the high prevalence of QRPs reported in the recent survey (Isbell et al., 2022) indicates a misalignment between the attitude to and the adoption of OS practices. This points to a knowledge gap regarding what prevents applied linguists from engaging in OS practices despite the recognition of their value. We seek to address this gap by surveying applied linguists not only on their OS attitudes and practices, but also on the perceived barriers to OS practices.
Full-text available
This chapter addresses the role and place of replication research and open science practices in advancing theory building and new research directions in the field of applied linguistics. The chapter begins by describing what replication research is, what the most common types of replication study are, and why carrying out replication matters. Close attention is paid throughout to the ways in which replication benefits from and contributes to a variety of open science initiatives, including open materials, open access and preprints, and preregistration.
Full-text available
Replication is a research methodology designed to verify, consolidate, and generalize knowledge and understanding within empirical fields of study. In second language studies, however, reviews share widespread concern about the infrequency of replication. A common but speculative explanation for this situation is that replication studies are not valued because they lack originality and/or innovation. To better understand and respond to the infrequency of replication in our field, 354 researchers were surveyed about their attitudes toward replication and their practices conducting replication studies. Responses included worldwide participation from researchers with and without replication experience. Overall, replications were evaluated as relevant and valuable to the field. Claims that replication studies lack originality/innovation were not supported. However, dissemination issues were identified: half of published replication studies lacked explicit labeling and one quarter of completed replications were unpublished. Explicit labeling of replication studies and training in research methodology and dissemination can address this situation.
Full-text available
Secondary research is burgeoning in the field of Applied Linguistics, taking the form of both narrative literature review and especially more systematic research synthesis. Clearly purposed and methodologically sound secondary research contributes to the field because it provides useful and reliable summaries in a given domain, facilitates research dialogues between sub-fields, and reduces redundancies in the published literature. It is important to understand that secondary research is an umbrella term which includes numerous types of literature review. In this commentary, we present a typology of 13 types of well-established and emergent types of secondary research in Applied Linguistics. Employing a four-dimensional analytical framework, focus, review process, structure, and representation of text of the 13 types of secondary research are discussed, supported with examples. This article ends with recommendations for conducting secondary research and calls for further inquiry into field-specific methodology of secondary research.
Full-text available
Doing Replication Research in Applied Linguistics is the only book available to specifically discuss the applied aspects of how to carry out replication studies in Second Language Acquisition. This text takes the reader from seeking out a suitable study for replication, through deciding on the most valuable form of replication approach to its execution, discussion, and writing up for publication. A step-by-step decision-making approach to the activities guides the reader/student through the replication research process from the initial search for a target study to replicate, through the setting up, execution, analysis, and dissemination of the finished work.
Registered Reports are a form of empirical publication in which study proposals are peer reviewed and pre-accepted before research is undertaken. By deciding which articles are published based on the question, theory and methods, Registered Reports offer a remedy for a range of reporting and publication biases. Here, we reflect on the history, progress and future prospects of the Registered Reports initiative and offer practical guidance for authors, reviewers and editors. We review early evidence that Registered Reports are working as intended, while at the same time acknowledging that they are not a universal solution for irreproducibility. We also consider how the policies and practices surrounding Registered Reports are changing, or must change in the future, to address limitations and adapt to new challenges. We conclude that Registered Reports are promoting reproducibility, transparency and self-correction across disciplines and may help reshape how society evaluates research and researchers. Registered Reports were introduced a decade ago as a means for improving the rigour and credibility of confirmatory research. Chambers and Tzavella overview the format’s past, its current status and future developments.
Open science serves to address core issues that are unique to quantitative methods in psychology, though it is typically presented as an appropriate framework for psychological research in general. In the present article, I critically examine the context within which open science operates as I bring that perspective into dialogue with priorities and goals of research that are both qualitative and feminist. I orient this examination in response to the question: what does open science mean for research methodologies that have historically been a home for transgressive and radical question-asking? Questioning the purposes of key tenets like replication and statistical significance—values systems that pose central distinctions between quantitative and qualitative methods—begs bigger questions for feminist psychologists. What counts as science? What counts as a valid epistemology? How can we avoid a further marginalization of epistemologies deemed less valid? I explore these questions, followed by a possible reimagining of our field’s engagement with open science in which I present seven suggestions to practically guide this endeavor.
Readers Respond is a forum for individuals to contribute their own reactions, perspectives, or experiences in relation to a specific article or articles published in a recent issue of ELT Journal. These views are not necessarily those of the Editor, the Editorial Panel, or the Publisher.
This book responds to the call for praxis in L2 education by documenting recent and ongoing projects around the world that see partnership with classroom teachers as the essential driver for continuing to develop both classroom assessment practice and conceptual frameworks of assessment in support of teaching and learning. Taken together, these partnerships shape the language assessment literacy, the knowledge and skills required for theorizing and conducting assessment activities, of both practitioners and researchers. While united by their orientation to praxis, the chapters offer considerable diversity with regard to languages taught, learner populations included (varying in age and proficiency level), specific innovations covered, research methods employed, and countries in which the work was conducted. As a whole, the book presents a way of engaging in research with practitioners that is likely to stimulate interest among not only language assessment scholars but also those studying second language education and language teacher education as well as language teaching professionals themselves.