Reporting on Qualitative Components of A Research Project

Abstract

div class="page" title="Page 1">
In this brief commentary, I first overview the strengths and weaknesses of qualitative research in education and also briefly consider mixed methods. I then offer a few suggestions for reporting on particular methodological components and the results of qualitative research. I do not address the reporting on other significant components such as the literature review and theoretical framework, given that these aspects are generic to both qualitative and quantitative research. The points I make are brief and are by no means exhaustive. Nevertheless, it is hoped that they offer some guidance for authors in education who adopt primarily qualitative methods.

</div

Full text

International Journal of Pedagogy and Teacher Education
Journal Homepage: jurnal.uns.ac.id/ijpte
PAPER | 69 p-ISSN: 2597-7792 / e-ISSN: 2549-8525
DOI: https://dx.doi.org/10.20961/ijpte.v5i2.55896
International Journal of Pedagogy and Teacher Education – 5(2) – 2021, pp50-pp53
Reporting on Qualitative Components of a Research Project
Lyn D. English1*
1 School of Teacher Education & Leadership , Faculty of Creative Industries, Education & Social Justice, Queensland
University of Technology, Australia
ARTICLE INFO
ABSTRACT
Article History
Received: Oct 21, 2021
1st Revision: Oct 24, 2021
Available Online: Oct 30th, 2021
In this brief commentary, I first overview the strengths and weaknesses of qualitative
research in education and also briefly consider mixed methods. I then offer a few
suggestions for reporting on particular methodological components and the results of
qualitative research. I do not address the reporting on other significant components such
as the literature review and theoretical framework, given that these aspects are generic
to both qualitative and quantitative research. The points I make are brief and are by no
means exhaustive. Nevertheless, it is hoped that they offer some guidance for authors in
education who adopt primarily qualitative methods.
Keywords:
methodology
qualitative reporting
qualitative research
*Corresponding Author
Email address:
l.english@qut.edu.au
How to cite: English, L. D. (2021). Reporting on Qualitative Components of a Research Project. International Journal of
Pedagogy and Teacher Education, 5(2), 50-53. https://dx.doi.org/10.20961/ijpte.v5i2.55896
1. INTRODUCTION
Qualitative research methodology has expanded significantly in the last few decades, with several articles
(e.g. Walsham, 2006), books and book chapters addressing the topic (e.g. Bikner-Ahasbahs, Knipping, & Presmeg,
2015; Kaiser & Presmeg, 2019). Mixed methods research has also grown in popularity as researchers capitalise
on the strengths of both qualitative and quantitative approaches (e.g. Buchholtz, 2019). Choosing an appropriate
research methodology depends on the research questions asked and the theoretical framework adopted.
Furthermore, as Cai et al. (2019) emphasised, authors need to justify the research methods chosen.
2. WHY QUALITATIVE RESEARCH?
Qualitative research is undertaken when one wants to discover or learn more about a topic or a particular
phenomenon; it is commonly used to understand people’s experiences and perspectives (Johnson & Christensen,
2017). Qualitative data are far more diverse and complex than quantitative data. Qualitative data may comprise
transcripts of video/audio recordings of classroom interactions, face-to-face interviews, and peer group
reporting, together with students’ written documents. Given the nature of the extensive data corpus, qualitative
research can be very time-consuming and is thus usually confined to a smaller number of cases than would be
addressed in quantitative research. On the other hand, the smaller number of cases in qualitative research
enables greater insights into complex situations or dynamic processes. However, it is usually not possible to
generalise the results or to verify/falsify hypotheses as one can with quantitative research (Buchholtz, 2019). To
draw on the strengths of both research forms, a mixed methods approach is frequently adopted. Kelle and
Buchholtz’s (2015) metaphor for this approach is worth citing:
Quantitative methods provide us with a general picture of the surface of the research field, while
qualitative research can be used to drill deep holes into the field yielding the information necessary for in -
depth explanations (p. 354).
In-depth explanations are most frequently derived from qualitative content analysis, which can apply to
both qualitative and mixed methods approaches. Irrespective of the methodology chosen, it is essential that the
theoretical framework, together with the research questions, inform decisions about the design and conduct of
an education study, including those from mathematics and STEM education. Hence, in reporting a qualitative
study, it is important to indicate how and why the research methodology was adopted.

Vol 5 Issue 2 Oct 2021 Reporting on Qualitative Components of A Research Project - 51 -
PAPER | 69 p-ISSN: 2597-7792 / e-ISSN: 2549-8525
DOI: https://dx.doi.org/10.20961/ijpte.v5i2.55896
3. REPORTING ON QUALITATIVE RESEARCH METHODOLOGY
Given the diversity and complexity of qualitative research, the quality of its reporting is especially important
(Twining, Heller, Nussbaum, & Tsai, 2017). Detailing the nature, number and background of the participants, the
population from which they were drawn, and how they were selected should appear in the methodology section
of a report. Where appropriate, this section should also describe the instruments administered, including their
appropriateness for addressing the research questions, how the instruments were developed or sourced, their
reliability for yielding the required data, and samples of items (or the entire instrument). Furthermore,
information on who conducted the data collection, who was present and the nature of the relationships between
them is often recommended (e.g. Twining et al., 2017). In other words, the context in which the research took
place needs to be included. When authors omit aspects or all of this information, it can be difficult for reviewers
to determine the validity of the results and any subsequent claims made (English, 2019).
Likewise, reports on studies involving treatments or classroom interventions should describe them in
sufficient detail to enable their key features to be identified and applied in interpreting the results (AERA, 2006).
The types of approaches adopted, examples of instructional materials or treatments implemented, and the
duration and frequency of implementation or administration should also be stated. Again, the nature of the
intervention should be guided/supported by the theoretical framework, which enables the reader to see how
the study stemmed from its conceptual foundations.
The nature of the data analysis and how it was conducted are important features to include in the report.
As previously noted, qualitative content analysis is one of the favoured means of data analysis. Several articles
address the nature of qualitative content analysis, describing it as a systematic approach where the focus is on
working with categories (codes) and developing a category system (or a coding frame; Kuckartz, 2019). It is
important to provide examples of responses to illustrate the categories adopted in the coding process. It is
beyond the scope of this commentary to undertake a detailed discussion of qualitative content analysis, but it is
worth noting just a few points from Kuckartz (pp. 195–196). These include:
• The categories of the final coding frame should be described as precisely as possible and the coding
procedure should be reliable, that is, at least two coders agree in their coding. Categories for which no
agreement can be reached must be excluded from the analysis: ‘Content analysis stands and falls by its
categories’ (Kuckartz, 2019, p. 195).
• Qualitative content analysis is conducted in several phases, ranging from preparing data, creating
categories (which often comprises several cycles) and undertaking analysis, through to writing a report
and presenting the results. Qualitative content analysis thus means more than just data coding. While
coding is a key step in the analysis, it is ultimately preparation for the subsequent analytical steps.
• The actual analysis phase ‘consists of summarizing the data, and constantly comparing and contrasting
the data. The analysis techniques can be qualitative as well as quantitative.’ (Kuckartz, p. 195).
• The quantitative analyses may comprise comparing the frequency of certain categories and/or
subcategories for certain groups.
4. REPORTING RESULTS
In reporting the results of data analysis, it is essential to keep in mind the research questions being
investigated. It should be made clear how the analysis addresses the research questions and leads to the
outcomes (AERA, 2006). One approach to reporting the results is to organise them according to each research
question, that is, revisit each question in turn (assuming more than one question). Only those results that actually
answer the questions should be included. Other pertinent findings might emerge from the data analysis, in which
case they could be incorporated within the discussion and cited as unanticipated outcomes. Such results could
serve as one area for further research.
Ensuring that all claims and conclusions made are supported by the data is especially important (AERA,
2006) and is an aspect that can be easily overlooked even by experienced researchers. It is easy to over-interpret
a study’s findings, that is, to make claims that are not supported by the data produced. In documenting data
outcomes, it is recommended that tables should be used only when they clarify or summarise outcomes involving
multiple data points (Saracho, 2013). Generally, the fewer the tables the better, as too many can detract from a
manuscript and extend its length, which can be problematic for those journals that have a strict page limit. While
tables should be readily interpreted, the messages they convey need to be summarised in the related text.

English, L. D. Reporting on Qualitative Components of A Research Project - 52 -
PAPER | 69 p-ISSN: 2597-7792 / e-ISSN: 2549-8525
DOI: https://dx.doi.org/10.20961/ijpte.v5i2.55896
The inclusion of appropriate anecdotes to illustrate aspects of the results can enhance the reporting. Such
anecdotes could be drawn from various data transcripts, teacher and student interviews, students’ written
explanations, and so on. The recent article by Budiyanto et al. (2021) exploring computational thinking in early
childhood education provides powerful examples of such anecdotes. Conducted in children’s homes due to
Covid-19 restrictions, the study revealed how young learners’ home environment and interactions with their
family supported their development of computational thinking. Anecdotes from parent interviews supported the
authors’ three primary constructs on the role of educational robots in early childhood. Other examples of
anecdotal evidence in interpretive research also appear in the case studies explored by Budiyanto, Prananto, and
Tan (2019). Their article highlights how case studies can enable in-depth investigation and rich descriptions.
In concluding this commentary, I think it is worth noting the words of a highly experienced reviewer in her
assessment of a journal submission:
… let me offer my perspective on what constitutes quality empirical research, whether quantitative,
qualitative, or mixed. Fundamentally research is undertaken to question, explore, probe, or to disprove;
ideally research is undertaken with an open mind and with researchers’ inevitable biases held in check to
the extent possible. This means that research can be distinguished from persuasive or promotion writing
where the objective can be to convince, support, or endorse. In empirical research, assumptions are not to
be forwarded but tested and scrutinized.
The reviewer’s words are worth keeping in mind as we embark on any research programme and
subsequently report on its findings.
REFERENCES
American Educational Research Association (2006). Standards for reporting on empirical social science research.
Educational Researcher, 35(6), 33–40. https://doi.org/10.3102/0013189X035006033
Bikner-Ahsbahs, A., Knipping, C. & Presmeg, N. (Eds.) (2015). Approaches to qualitative research in mathematics
education (pp. 75–101). Berlin: Springer
Buchholtz, N. (2019). Planning and conducting mixed methods studies in mathematics educational research. In
G. Kaiser, and N. Presmeg (Eds.), Compendium for early career researchers in mathematics education (pp.
131–152). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-15636-7_6
Budiyanto, C., Prananto, A., & Tan, F. T. (2019). Designing embedded case study research approach in educational
research. International Journal of Pedagogy and Teacher Education, 3(1), 1–18.
https://doi.org/10.20961/ijpte.v3i1.15002
Budiyanto, C., Shahbodin, F., Ulin Khoirul Umam, M., Isnaini, R., Rahmawati, A., & Widiastuti, I. (2021).
Developing computational thinking ability in early childhood education: The influence of programming toy
on parent-children engagement. International Journal of Pedagogy and Teacher Education, 5(1), 19–25.
https://doi.org/10.20961/ijpte.v5i1.44397
Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., ... Hiebert, J. (2019). Choosing and justifying
robust methods for educational research. Journal for Research in Mathematics Education, 50(4), 342–348.
English, L. D. (2019). Removing obstacles to quality research publishing. In K. Leatham (Ed.), Designing,
conducting, and publishing quality research in mathematics education (pp. 199–211). Cham, Switzerland:
Springer.
Johnson, R. B., & Christensen, L. B. (2017). Educational research: Quantitative, qualitative, and mixed approaches
(6th ed.). Thousand Oaks, CA: Sage.
Kaiser, G., & Presmeg, N. (2019) (Eds.). Compendium for early career researchers in mathematics education.
Cham, Switzerland: Springer.
Kelle, U., & Buchholtz, N. (2015). The combination of qualitative and quantitative research methods in
mathematics education: A “Mixed Methods” study on the development of the professional knowledge of
teachers. In A. Bikner‐Ahsbahs, C. Knipping, and N. Presmeg (Eds.), Approaches to qualitative research in
mathematics education: Examples of methodology and methods (pp. 321–361). Dordrecht: Springer.
Kuckartz, U. (2019). Qualitative text analysis: A systematic approach. In G. Kaiser, and N. Presmeg (Eds.),
Compendium for early career researchers in mathematics education (pp. 181–197). Cham, Switzerland:
Springer. https://doi.org/10.1007/978-3-030-15636-7_8

Vol 5 Issue 2 Oct 2021 Reporting on Qualitative Components of A Research Project - 53 -
PAPER | 69 p-ISSN: 2597-7792 / e-ISSN: 2549-8525
DOI: https://dx.doi.org/10.20961/ijpte.v5i2.55896
Saracho, O. N. (2013). Writing research articles for publication in early childhood education. Early Childhood
Education Journal, 41, 45–54. https://doi.org/10.1007/s10643-012-0564-3
Twining, T., Heller, R. S., Nussbaum, M., & Tsai, C. (2017). Some guidance on conducting and reporting qualitative
studies. Computers & Education, 106, A1–A9. https://doi.org/10.1016/j.compedu.2016.12.002
Walsham, G. (2006). Doing interpretive research. European Journal of Information Systems, 15, 320–330.
https://doi.org/10.1057/palgrave.ejis.3000589