Content uploaded by Ali H. Al-Hoorie
Author content
All content in this area was uploaded by Ali H. Al-Hoorie on Jun 16, 2023
Content may be subject to copyright.
Content uploaded by Sin Wang Chong
Author content
All content in this area was uploaded by Sin Wang Chong on Aug 22, 2022
Content may be subject to copyright.
RESEARCH IN PROGRESS
Open scholarship in applied linguistics: What, why,
and how
Meng Liu1* , Sin Wang Chong2, Emma Marsden3, Kevin McManus4,
Kara Morgan-Short5, Ali H. Al-Hoorie6, Luke Plonsky7, Cylcia Bolibaugh3,
Phil Hiver8, Paula Winke9, Amanda Huensch10 and Bronson Hui11
1
University of Cambridge, Cambridge, UK,
2
University of Edinburgh, Edinburgh, UK,
3
University of York, York, UK,
4
Penn
State University, University Park, USA,
5
University of Illinois Chicago, Chicago, USA,
6
Royal Commission for Jubail and
Yanbu, Jubail, Saudi Arabia,
7
Northern Arizona University, Flagstaff, USA,
8
Florida State University, Tallahassee, USA,
9
Michigan State University, East Lansing, USA,
10
University of Pittsburgh, Pittsburgh, USA and
11
University of Maryland,
College Park, USA
*Corresponding author. Email: ml858@cam.ac.uk
(Received 6 August 2022; accepted 14 August 2022)
1. Introduction
In November 2021, the UNESCO Recommendation on Open Science was adopted by the UNESCO
General Conference. This marks a significant milestone in the development of Open Science as the
new norm for research, with intended influences on national laws and practices of the 193 UN
Member States and far-reaching implications beyond the UN. It is against such a background that
Open Applied Linguistics (openappliedlinguistics.org), a new Research Network affiliated with the
International Association of Applied Linguistics (AILA), was established in early 2022 and a two-day
online symposium on open scholarship in applied linguistics was held in June 2022.
Organized by Meng Liu and Sin Wang Chong, convenors of Open Applied Linguistics, the sym-
posium was the inaugural event of the network and was sponsored by Cambridge University Press
& Assessment. There were 19 sessions by 31 speakers and presenters across career stages, covering
a wide range of topics. While our symposium included speakers from neighbouring fields such as
psychology, due to limited space, we selected six sessions that directly concentrated on applied linguis-
tics to include in this report.
2. What do we mean by open scholarship?
Open scholarship is the theme of our symposium. Specifically, we follow the UNESCO (2021) defin-
ition with our own adaptations:
Open [scholarship] is defined as an inclusive construct that combines various movements and
practices aiming to make multilingual [scholarly] knowledge openly available, accessible, and
reusable for everyone; to increase [scholarly] collaborations and sharing of information for the
benefits of [scholarship] and society; and to open the processes of [scholarly] knowledge creation,
evaluation, and communication to societal actors beyond the traditional [scholarly] community.
It comprises all [scholarly] disciplines and aspects of scholarly practices, including basic and
applied sciences, natural and social sciences and the humanities, and it builds on the following
key pillars: open [scholarly] knowledge, open [scholarship] infrastructures, research communica-
tion, open engagement of societal actors, and open dialogue with other knowledge systems. (p. 7)
While we mostly use ‘open scholarship’,‘open research’, and ‘open science’interchangeably, we delib-
erately replaced the term ‘science’with ‘scholarship’in the above definition to highlight our aim to
© The Author(s), 2022. Published by Cambridge University Press
Language Teaching (2023), 56, 432–437
doi:10.1017/S0261444822000349
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
promote a more inclusive and critical understanding of open science. Readers familiar with the history
of open science in psychology know that it has traditionally been associated with quantitative research
and it was only recently that alternative, equally valid epistemologies have received more attention in
this space (see Bennett, 2021, for example, for a discussion on open science from the perspective of
feminist psychology). A recent survey within applied linguistics (Liu & De Cat, in press) also revealed
some concerns from qualitative researchers on open science having unduly quantitative connotations.
Nonetheless, both the UNESCO recommendations and our symposium demonstrate that open science
can be and should be understood as more inclusive. By using open scholarship, a term less commonly
used than open science, we hope to inspire curiosity, mitigate misconceptions, and facilitate a more
inclusive understanding of open scholarship/research/science (henceforth OS).
3. Selected talks and panels from the symposium
3.1 Open research practices: Value versus sustainability
Marsden and Bolibaugh’s session focused on open research practices. Marsden started the session with
the rationales for and the value of OS. Rationales for OS can be philosophical, social, and scientific,
ranging from epistemic responsibility and social equity to enhanced rigour, reliability, and validity.
Within applied linguistics, there have been several initiatives to promote OS. IRIS (Instruments and
Data for Research in Language Studies; Marsden et al., 2016), an open repository for instruments
and data established in 2011, is one of the first initiatives to promote open materials and data, and
soon IRIS will increase its scope from a focus on second languages to all language-related studies.
The Open Accessible Summaries in Language Studies (OASIS; Marsden et al., 2018a) is another exem-
plary initiative for research accessibility. While much progress has been made and journals are con-
tinuing to join the initiative, there are persisting challenges in terms of sustainability (e.g., active
engagement from journal editors, cost of infrastructures, time required for data preparation). It is
also important to acknowledge that epistemology may affect the rationales for OS and its potential
benefits –some epistemologies and methodologies are not necessarily compatible with, for example,
replication and fully anonymized data sharing.
In Part II of this session, Bolibaugh zoomed in on open data. Reiterating the rationales for open
data, she called for a move from the verification-focused approach (Open Research 1.0) to a more syn-
thetic and collaborative approach (Open Research 2.0). In the case of open data, Open Research 2.0
would entail reusing data for purposes such as synthesis and reconceptualization, in addition to veri-
fication. Surveying the IRIS database, it was found that while the prevalence of open data is on the rise,
the usability and completeness judged according to the FAIR (findable, accessible, interoperable, and
reusable) principles are less than ideal. To move to Open Research 2.0, we still need consensus on what
constitutes data for reproducing published findings, recognition for OS efforts at the institutional and
award levels, training support, and funding for infrastructure.
To conclude, to keep applied linguistics at the forefront of OS, sustained commitment is needed
from all levels in the community, which will understandably be a gradual process.
3.2 TESOLgraphics: Opening up scholarly communication
The session by Chong extended the notion of OS in applied linguistics to include practitioners. This
session reported on an ongoing initiative, TESOLgraphics (tesolgraphics.com), which produces open-
access, one-page infographic summaries of secondary research in all topics of TESOL (Teaching
English to Speakers of Other Languages) to create opportunities for English teachers to engage with
research and develop their evidence-based teaching practices.
Chong began the session by reviewing research on researcher-practitioner relationships in language
education, highlighting that both researchers and practitioners are willing to engage in professional
exchange. Chong discussed the usefulness of secondary research, in particular research syntheses
Language Teaching 433
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
(i.e., literature reviews conducted using systematic methodologies), in fostering the research-practice
nexus (Chong, 2019; Chong & Plonsky, 2021).
Introducing the notion of ‘praxis’from dialectical materialism (Poehner & Inbar-Lourie, 2020),
Chong problematized the nature of current research-practice dialogues. Most research-practice activ-
ities are unidirectional, with researchers imparting knowledge to practitioners. This points to the need
for two-way communication between researchers and practitioners in TESOL, forming an evidence
ecosystem comprising producers, synthesisers, and consumers of evidence (Shepherd, 2014): while
researchers offer empirical and theoretical insights into issues pertaining to teaching and learning,
experience and pragmatic knowledge of practitioners evaluate ecological validity of research findings,
making the focus of educational research more relevant to practitioners.
Responding to the need for more researcher-practitioner dialogues, TESOLgraphics will enter a
new phase of development, inviting English teachers to produce teaching materials and resources
(e.g., lesson plans) inspired by specific infographic summaries. Going alongside the teaching resources
are short video clips where teachers designing the materials reflect on how research findings are trans-
lated into concrete teaching ideas.
3.3 Replication research in applied linguistics: What, why, and how
McManus’talk focused on replication research in applied linguistics. Several new initiatives in our field
point to an increased awareness about the need, place, and value of replication research in the growth
and credibility of the discipline (e.g., methodological guides, journal special issues). Yet, as reviews
have noted (e.g., Marsden et al., 2018b; Porte, 2012), support and resources are needed so that
researchers can design, conduct, and report high quality replication studies. This session responded
to that need by introducing novice and experienced researchers to the replication research process.
The key topics introduced were: why replication studies are needed, what a replication study is,
what distinguishes a replication study from other types of research studies, and how to go about
designing a replication study in the field of applied linguistics (see McManus, 2021,in press-a,in
press-b; Porte & McManus, 2019). A brief summary of this session’s main parts is provided here.
To start, a replication repeats a study’s methodology with or without changes followed by system-
atic comparison to better understand the nature, repeatability, and generalizability of its findings.
Replication therefore aims to systematically reconsider, refine, extend, and sometimes limit previous
research findings. Because replication is defined by comparison, replication researchers must system-
atically draw comparisons at each stage of the process (design, data handling, reporting), including
using comparative language in reporting and planning how comparisons will be handled in the ana-
lyses. In addition, the more that we change in a replication, the more difficult comparison becomes.
We should therefore aim to keep as much similarity between the studies as possible. Lastly, because
replication is important for all empirical disciplines, clear labelling should be used throughout, includ-
ing the use of ‘replication’in the study’s title and abstract.
3.4 Registered reports as an open research practice
In this talk, Morgan-Short discussed Registered Reports (RRs; Center for Open Science, n.d.;
Chambers & Tzavella, 2022; Marsden et al., 2018c) as an open research practice that can improve
the validity, reliability, and replicability of applied linguistics research. RRs is an article type in
which authors motivate and design their study and submit it for peer review BEFORE data collection.
If a study is well motivated and has a valid design protocol, it receives ‘In Principle Acceptance’, mean-
ing that the journal agrees to publish the study after data collection regardless of the outcomes of the
results, as long as the authors adhere to the planned protocol and make reasonable conclusions about
the results. Upon receiving In Principle Acceptance, researchers preregister their protocol. Then, they
collect and analyse their data and write their full manuscript, which is reviewed for adherence to the
preregistered protocol and the reasonableness of the conclusion before being published.
434 Meng Liu et al.
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
Morgan-Short presented emerging metascience suggesting that RRs are effective in attenuating
publication bias and are associated with higher computational reliability, validity, and increased
replication efforts (for more information, see Chambers & Tzavella, 2022). In addition, common con-
cerns about RRs were addressed. For example, exploratory analyses are not restricted in RRs as long as
preregistered confirmatory analyses are reported as planned. Morgan-Short also discussed the concern
that the publication time course of RRs is shifted because of the peer review process that occurs before
data collection, which may be a particular challenge for researchers with inflexible deadlines. Finally,
ideas, examples, and resources for moving forward with RRs were considered, especially the need for
funding support and for more journals to offer RRs, following the lead of Language Learning,
Bilingualism: Language and Cognition, and Second Language Research, where RRs are available as
an article type. To conclude, Morgan-Short noted that the benefits of RRs often greatly outweigh
the challenges and encouraged researchers to adopt this practice to bring about increased validity,
reliability, and replicability to their own research and to our field.
3.5 Open science initiatives: The Postprint Pledge
The session by Al-Hoorie, Hiver, and Nosek was concerned with OS initiatives, specifically promoting
open access in order to democratize scholarship, provide equitable access to researchers from the
Global South, and remove obstacles to accessing the latest literature. According to the classic technol-
ogy adoption lifecycle model (Rogers, 1995), a sociological framework describing the process of innov-
ation acceptance, individuals vary in the speed at which they adopt innovations and initiatives. The
first groups are INNOVATORS and EARLY ADOPTERS. These are then followed by EARLY MAJORITY and
LATER MAJORITY, and finally by LAGGARDS. Building on this model, the session launched the ‘Postprint
Pledge’, an initiative for applied linguists and second language acquisition researchers to share the
‘accepted versions’of their manuscripts. According to the policies of journal publishers, authors are
usually permitted to post their postprints online. Authors may not be aware that they still have the
right to share the accepted versions of their manuscripts, believing that once they sign off the copy-
right agreement, they lose all rights to the manuscript.
As part of this initiative, and to make it more concrete for authors wishing to share the accepted
versions of their manuscripts, a list of 60 applied linguistics journals was compiled and their copyright
policies were reviewed through Sherpa Romeo, an online resource that aggregates the open access pol-
icies of various publishers. Analysis of the copyright agreements of these 60 journals showed that most
publishers have no restrictions on authors sharing postprints, whether on their personal websites and
on nonprofit repositories, though a minority of publishers require an embargo period of one year and
in a few cases two years. The list detailing the copyright policies of each journal in relation to sharing
postprints is currently hosted at ali-alhoorie.com/postprint-pledge, where interested researchers can
see the list of signatories and pledge to share their postprints.
3.6 Speaking openly on open science: A panel discussion
The panel discussion by Plonsky, Winke, Huensch, and Hui sought to bring in a range of voices that
might help us to reckon with some of the new ways of thinking and action that accompany a way for-
ward for the field. There are many facets to embracing an ethos of OS. For the full range of benefits
associated with OS to accrue, we as a culture and community need to shift not only our actions but our
thinking as well. These adjustments, both major and minor, can be seen and felt in virtually all we do
as scholars. The points of view that the panel shared included, among others, those of researcher,
journal editor, manuscript reviewer, researcher trainer, grant writer, grant reviewer, graduate program
director, mentor, and recent graduate.
In an event such as this one, instead of focusing on what OS can bring to our field, the panel spoke
openly on the pushback that has been and continues to be observed in the face of OS initiatives. For
instance, one commonly raised concern is the fear of being ‘scooped’or having a result overturned via
Language Teaching 435
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
re-analysis. There may also be the hesitation to freely share something (e.g., data) that one has very
painstakingly collected, cleaned, coded, and prepared. The panel argued that more of a community
or synthetic mindset is needed and that it is an ethical duty to engage in practices that are best for
the collective construction of knowledge rather than solely for career advancement.
In addition to deliberating on potentially thorny issues applied linguists all face such as data shar-
ing, the panel also luxuriated in response to a prompt to dream about where the field might be in
terms of OS ten years from now. Among other visions and hopes, the group imagined scholarly
and institutional structures, incentives, workflow tools, and training protocols with an OS orientation
baked in. Achieving consistency in code books (e.g., variable naming conventions) and in script anno-
tation, for instance, present serious challenges to these dreams, but we see them as worthwhile if not
necessary.
These discussions, though useful, can sometimes drift into the very conceptual and abstract. In
order to reorient to more practical territory, the session was concluded by each panellist citing concrete
steps toward embracing OS. These included, among others, posting materials and data on IRIS or OSF,
writing an OASIS summary, identifying as part of the OS movement on personal websites, posting
about OS practices on social media, taking the Postprint Pledge (see Section 3.5), and encouraging
journal editors to integrate OS practices into their journals’policies. Such a list of activities, though
incomplete, may seem daunting. Consequently, the panel urged those who might feel overwhelmed
to put aside the notion that OS is an all or nothing endeavour. The path that will lead to the greatest
uptake in OS at both the field and individual levels and the one that will ultimately prove most effect-
ive for applied linguistics is one of incrementalism.
4. Conclusion
Our symposium addressed the what, why, and how of OS: Initiatives such as IRIS, OASIS,
TESOLgraphics, and the PostPrint Pledge directly promote research accessibility and social equity,
and they indirectly facilitate OS practices such as replication and registered reports, which, in turn,
directly improve research validity, reliability, and reproducibility. A common thread across all is an
ethos of openness, not merely in terms of research practices, but also in the open acknowledgement
of and engagement with the many tensions and challenges regarding OS in applied linguistics, from
epistemological and cultural to psychological and practical. This, we believe, is the ethos that can truly
build us up as a field. While our symposium represents only one of the initial steps in a long journey,
we hope to inspire more aligned and sustained efforts across all levels to incrementally and effectively
advance our field towards a more open future.
Acknowledgements. We would like to thank the other speakers and presenters who contributed to this symposium and
who, due to limited space, we were unable to include in this report (in order of appearance at the symposium): Curtis
Sharma, Inge Alferink, Junlan Pan, Carl Blyth, David Birdsong, Mark Amengual, Kiera McNeice, Dessi Kirilova, Jessica
Miorini, Madeleine Pownall, Flávio Azevedo, Hassan Nejadghanbar, Guangwei Hu, Yaoko Matsuoka, Hiroyuki Ida, Laura
Gurzynski-Weiss, Lara Bryfonski, Derek Reagan, and Brian Nosek.
References
Bennett, E. A. (2021). Open science from a qualitative, feminist perspective: Epistemological dogmas and a call for critical
examination. Psychology of Women Quarterly,45(4), 448–456. doi:10.1177/03616843211036460
Center for Open Science. (n.d.). Registered Reports. Center for Open Science. Retrieved from https://www.cos.io/initiatives/
registered-reports
Chambers, C. D., & Tzavella, L. (2022). The past, present and future of registered reports. Nature Human Behaviour,6(1),
29–42. doi:10.1038/s41562-021-01193-7
Chong, S. W. (2019). The role of research synthesis in facilitating research–pedagogy dialogue. ELT Journal,74(4), 484–487.
doi:10.1093/elt/ccaa046
Chong, S. W., & Plonsky, L. (2021). A typology of secondary research in applied linguistics. OSF. doi:10.31219/osf.io/msjrh
436 Meng Liu et al.
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
Liu, M., & De Cat, C. (in press). Open science in applied linguistics: A preliminary survey. In L. Plonsky (Ed.), Open science in
applied linguistics. John Benjamins.
Marsden, E., Alferink, I., Andringa, S., Bolibaugh, C., Collins, L., Jackson, C., Kasprowicz, R., O’Reilly, D., & Plonsky, L.
(2018a). Open Accessible Summaries in Language Studies (OASIS) [Database]. Retrieved from https://www.oasis-data-
base.org
Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository of instruments for research into second languages:
Advancing methodology and practice. In A. Mackey, & E. Marsden (Eds.), Advancing methodology and practice: The
IRIS repository of instruments for research into second languages ( pp. 1–21). Routledge.
Marsden, E., Morgan-Short, K., Thompson, S., & Abugaber, D. (2018b). Replication in second language research: Narrative
and systematic reviews and recommendations for the field. Language Learning,68(2), 321–391. doi:10.1111/lang.12286
Marsden, E., Morgan-Short, K., Trofimovich, P., & Ellis, N. C. (2018c). Introducing registered reports at language learning:
Promoting transparency, replication, and a synthetic ethic in the language sciences. Language Learning,68(2), 309–320.
doi:10.1111/lang.12284
McManus, K. (2021). Are replication studies infrequent because of negative attitudes? Insights from a survey of attitudes and
practices in second language research. Studies in Second Language Acquisition,1–14. doi:10.1017/S0272263121000838
McManus, K. (in press-a). Replication and open science in applied linguistics research. In L. Plonsky (Ed.), Open science in
applied linguistics. John Benjamins.
McManus, K. (in press-b). Replication research in instructed SLA. In L. Gurzynski-Weiss, & Y. Kim (Eds.), Instructed second
language acquisition research methods (pp. 103–122). John Benjamins.
Poehner, M. E., & Inbar-Lourie, O. (2020). Toward a reconceptualization of second language classroom assessment. Springer.
Porte, G. (2012). Replication research in applied linguistics. Cambridge University Press.
Porte, G., & McManus, K. (2019). Doing replication research in applied linguistics. Routledge.
Rogers, E. M. (1995). Diffusion of innovations (4th ed.). Free Press.
Shepherd, J. P. (2014). How to achieve more effective services: The evidence ecosystem. What Works Network, Cardiff
University. Retrieved from https://orca.cardiff.ac.uk/id/eprint/69077/1/2014_JPS_What_Works.pdf
UNESCO. (2021). UNESCO recommendation on open science ( pp. 1–36). Retrieved from https://unesdoc.unesco.org/ark:/
48223/pf0000379949.locale=en
Cite this article: Liu, M. et al (2023). Open scholarship in applied linguistics: What, why, and how. Language Teaching,56(3),
432–437. https://doi.org/10.1017/S0261444822000349
Language Teaching 437
https://doi.org/10.1017/S0261444822000349 Published online by Cambridge University Press
