I Never Promised You a Rose Garden: When Research Questions Ought to Change

Chapter (PDF Available) · July 2016with 738 Reads
DOI: 10.1108/S0897-301620160000024003
In book: Research in Organizational Change and Development, pp.47-82
Abstract
This chapter addresses the common assumption that research questions are fixed at the outset of a study and should remain stable thereafter. We consider field-based organizational research and ask whether and when research questions can legitimately change. We suggest that change can, does and indeed should occur in response to changes in the context within which the research is being conducted. Using an illustrative example we identify refinement and reframing as two distinct types of research question development. We conclude that greater transparency over research question evolution would be a healthy development for the field.
I Never Promised You a Rose Garden:
when research questions ought to change
Robert MacIntosh
Heriot-Watt University
robert.macintosh@hw.ac.uk
Jean M. Bartunek
Boston College
bartunek@bc.edu
Mamta Bhatt
IESEG School of Management
m.bhatt@ieseg.fr
Donald MacLean
University of Glasgow
donald.maclean@glasgow.ac.uk
Abstract
This chapter addresses the common assumption that research questions are fixed at
the outset of a study and should remain stable thereafter. We consider field-based
organizational research and ask whether and when research questions can legitimately
change. We suggest that change can, does and indeed should occur in response to
changes in the context within which the research is being conducted. Using an
illustrative example we identify refinement and reframing as two distinct types of
research question development. We conclude that greater transparency over research
question evolution would be a healthy development for the field.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
Robert MacIntosh is Professor of Strategic Management at Heriot-Watt University
where he is also Head of the School of Management and Languages. He has a PhD in
Engineering and researches the development of strategy in senior teams as well as
organizational change. He sits on the councils of the British Academy of
Management, the Chartered Association of Business Schools, the board of Turning
Point Scotland and is the co-founder of Stridesite which provides on-line resources for
those developing strategy.
Jean M. Bartunek is the Robert A. and Evelyn J. Ferris Chair and Professor of
Management and Organization at Boston College. Her PhD in social and
organizational psychology is from the University of Illinois at Chicago. She is a past
president and fellow of the Academy of Management. Her research interests center
on organizational change and academic-practitioner relationships. Jean is currently
an associate editor of the Academy of Management Review and the Journal of Applied
Behavioral Science.
Mamta Bhatt is an Assistant Professor of Management at the IESEG School of
Management, Paris. She has a PhD in Organization Studies from Boston College. Her
research interests include individual and organizational identity, organizational
identification, non-traditional work arrangements (e.g. contingent work,
telecommuting), and cross-cultural issues. In particular, her work focuses on the
implications of context (e.g. organizational change and crisis situations, etc.) and
relationships for identity construction and identification.
2
Donald MacLean received a BSc in Physics from the University of Strathclyde, a
PhD in optoelectronics from the University of Cambridge and an MBA from Kingston
University. He spent ten years working in the global optoelectronics industry before
joining the University of Glasgow in 1993 where he is now a professorial research
fellow in the Adam Smith Business School. He has published extensively on strategy,
transformation and complexity theory in a range of international journals including
the Strategic Management Journal, The Journal of Management Studies, Organization
Studies and Human Relations. His latest book “Strategic Management: Strategist at
work” (Palgrave 2015) is aimed particularly at practitioners.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
Introduction
Can research questions sometimes legitimately change over time during the
conduct of field-based, longitudinal action research? If the answer to this question is
yes, how and why might such change happen?
Management researchers have shown considerable interest in the generation of
research questions (e.g. Alvesson & Sandberg, 2013b), yet there has been almost no
attention on how research questions might legitimately evolve over time. Indeed, in
our view such evolution1 frequently does happen but goes unreported.
In this chapter, we consider the case of an action research project that was
addressing significant societal issues, to show how and why research questions might
evolve. In this project the research questions changed multiple times for reasons
beyond the control of the investigators. Through our review of this illustrative
example, we open for discussion concerns which are often present beneath the surface
of many research projects using this and other methodological approaches but which
are rarely discussed in scholarly writing. We provide a foundation for our exploration
below.
There can be little doubt that the field of organization and management
research is experiencing a period of rapid change. Global growth in business schools
has resulted in more scholars submitting to peer reviewed journals, and the
established top-tier journals have experienced a “disproportional increase in the[ir]
rejection rate” (Alvesson & Sandberg, 2013a: 127). For scholars adopting a
qualitative approach to their research, Pratt argues that the “lack of consensus around
the evaluation criteria used for [qualitative] research means that it is difficult to
publish” (2008: 482). Action research is even more difficult to publish in top-tier
1 Throughout this chapter, research questions are described as evolving in the sense that they are
subject to a process of gradual development. This is distinct from the connotation of variation,
selection and retention associated with evolution through, for example, natural selection.
4
management journals; indicatively, only one formal action research paper (Lüscher &
Lewis, 2008) has been published in such journals in recent years.
Amis and Silk contend that an assessment of research quality isinseparable
from the ontological and epistemological foundations of the research project” (2008:
457). In that spirit, this paper considers the case of action research. Using a specific
action research project as an example, we examine processes through which
researchers might (need to) develop and refine research questions in studies that are
both field based (Czarniawska, 2014) and inductive (O’Gorman and MacIntosh,
2015). In doing so, we pay particular attention to the temporal dimensions of research
questions and challenge some of the dominant orthodoxy relating to when, where and
how research questions are established. Further, we also reflect upon the wider
implications of an evolutionary perspective on research questions for research taking
a different epistemological, ontological and methodological approach. This reflection
has implications for the quality of action research and other types of field-based
research projects.
The chapter is organized as follows. We first review contemporary ideas on
the ways in which researchers identify and specify research questions in order to
develop publishable research. We then present some historical foundations for why it
has been considered so important to elaborate research questions completely before a
study starts.
Following this, we focus on the development of research questions in the
context of action research. In that context, we offer an illustration of a longitudinal
action research project aimed at delivering bilingual education in an elementary
school. This case features key transitions where the empirical context shifted during
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
the research process, thus requiring some evolution of the original research questions.
Whilst such transitions may be exactly what one might expect when studying
organizational change, they nonetheless introduce significant challenges in
accomplishing publishable research.
Identifying and specifying appropriate research questions
For aspiring researchers, the ability to publish rests on overcoming what
Ketokivi and Mantere describe as the challenge of “drawing theoretical conclusions
from empirical data in a manner that is credible” (2010: 315). Research questions
offer one narrative device linking theory and data and they are commonplace in
written accounts of research. To meet the criteria for publication in top-tier journals,
researchers must demonstrate that they are advancing knowledge by delivering a
novel and worthwhile contribution. One approach for doing so is to pose a significant
or “big” research question (Peng, 2004; Tsui, Zhao & Abrahamson, 2007).
Implicitly, if not explicitly, such a question is posed at the outset of a study.
According to the extant literature, initial research questions are most often
formulated by means of gap spotting (Alvesson and Sandberg, 2011) and somewhat
less often by systematic literature reviews (Denyer and Tranfield, 2009). A third
strategy for research question formulation entails counterfactual reasoning
(Cornelissen & Durand, 2012) where researchers are encouraged to rethink base
assumptions and reframe explanations by challenging orthodox ideas. This approach
is considered most likely to lead to “big” questions. We summarize these strategies
below and move on to problematize their underlying assumptions.
Gap Spotting. In order to build a contribution, Alvesson and Sandberg argue
that most scholars engage in gap-spotting, since this allows them to make the
argument that they are contributing by “filling an important gap in the literature”
6
(2011: 250; 2013b). (They note that even if assumption challenging underpinned the
development of guiding research questions, the questions themselves are often
presented as based on gap-spotting (Alvesson & Sandberg, 2013b) thus introducing
the notion of temporality.) Alvesson and Sandberg (2011: 249) acknowledge that gap-
spotting rarely involves “a simple identification of obvious gaps in a given body of
literature;” instead it is often a “complex” and “constructive” process. Nevertheless,
they view gap-spotting as inherently narrow in what it may accomplish.
Systematic literature reviews. Systematic literature reviews are relatively
common in fields like medicine and engineering, where core concepts can be clearly
and consistently defined in ways which facilitate cross-study learning (Tranfield et al.,
2003). In recent years, management scholars have embraced more structured
approaches to reviewing the literature since such reviews help them to both locate
their own work and identify points of difference from extant work (Rousseau &
Gunia, 2015). Indeed, it would be unusual now for a published literature review in a
journal such as the International Journal of Management Reviews to omit a
description of the stages and processes by which papers were identified, categorized
and either included or excluded.
Counterfactual reasoning. Some scholars develop research questions by
moving beyond the extension or corroboration of existing theories to question the
very foundation of existing explanatory frameworks. By pursuing alternative
explanations that may challenge or even break existing concepts, constructs,
assumptions and relationships, researchers extend theoretical understanding in new
ways (Durand and Vaara, 2009; Folger and Turillo, 1999). The imaginative use of
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
‘what if questions (Cornelissen & Durand, 2014) might lead researchers to
understand the phenomena under examination anew.
Notably, each of these three strategies gap spotting, systematic literature
reviews, and counterfactual reasoning - implicitly, if not explicitly posits that research
questions are established and fixed at the outset. Indeed, Alvesson and Sandberg
(2013b) suggest that the best approach to generating high quality research questions is
to “deliberately and systematically identify and challenge the assumptions underlying
the existing literature”. This is done through “(1) identifying a domain of literature;
(2) identifying and articulating the assumptions underlying this domain; (3) evaluating
these; (4) developing an alternative assumption ground; (5) considering it in relation
to its audience; and (6) evaluating the alternative assumption ground” (op cit: 56).
There is no mention of the possibility or legitimacy of research question evolution.
The focus is still on a fairly rational, linear and logical trajectory which starts with the
literature, moves to the articulation of a research question and ends with the
elaboration of a contribution that flows from answering the research question.
The importance of developing and changing research questions
But there is much more to the development of research questions, even
initially, than is typically acknowledged in the literature, in part because gap spotting
is so common and so accepted as a basis for research questions. Since the idea that
research questions area beginning point for their research” (Agee, 2009: 431) is a
taken-for-granted assumption (Alvesson & Sandberg, 2013b), the evolution of
research questions is rarely considered in depth. Scholarly articles typically spend
very little time discussing research questions beyond the somewhat literature-led
rhetorical strategy that questions flow from gaps, challenge assumptions, and/or build
on systematic reviews.
8
This standard approach ignores the fact that the generation of appropriate
research question(s) for particular phenomena, especially in field settings, is difficult.
For instance, Leung and Lapum (2005: 3) describedthe struggle to an apt research
question (as) an arduous journey”. Glick, Miller and Cardinal (2007: 818) added that
in a field such as organizational behavior, “with weak paradigm development,
individuals face tremendous uncertainty in choosing research questions and methods
that will allow contributions in the published literature.” This is the case regardless of
whether the approach taken is based on gap-spotting, systematic review, or
counterfactual reasoning. Further, there has typically been an expectation that
research questions, once formally developed, must stay as they are. Not to do so is to
raise ethical quandaries.
Approaches to social science research based on the scientific method typically
follow Karl Popper’s (1972; 2002) positivist philosophy. Popper argued that the first
step in scientific research is generating a hypothesis, or, more precisely, a null
hypothesis that can be tested and (hopefully) falsified. He believed that, from a
scientific perspective, at least, there is really no such thing as pure observation
(Banerjee et al., 2009). Further, it is not possible to “prove” a particular hypothesis.
However, it is possible to show the likelihood of the null hypothesis being falsified.
Historically, the scientific method and especially the importance of testing and
(possibly) falsifying a priori hypotheses have served as the foundation for most
scholarly research in social science. It has also served as the foundation for ethical
standards in social science research (e.g. Bakker et al., 2012; Sterba, 2006). The
ethical standards built up around the scientific method have fostered the importance of
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
determining a research question at the beginning of a research project and not
changing this later.
Accordingly, it is deemed inappropriate to collect (primarily quantitative) data,
run a number of statistical tests, see which hypotheses are supported by the data, and
then describe the hypotheses as if they were formulated before data collection. This is
sometimes referred to as a fishing expedition or as HARKing (hypothesizing after the
results are known; Wasserman, 2013) and represents a clear ethical violation for
many. This is the case for good reason. Running a large number of statistical tests
(especially if these are exploratory rather than confirmatory; Wasserman, 2013)
increases the likelihood of type I error rates (Simmons et al., 2011; Bakker et al.,
2012). The American Statistical Association states that “Running multiple tests on the
same data set at the same stage of an analysis increases the chances of obtaining at
least one invalid result. Selecting the one ‘significant’ result from a multiplicity of
parallel tests poses a grave risk of reaching incorrect conclusions. Failure to disclose
the full extent of tests and their results in such a case would be highly misleading”
(Committee on Professional Ethics, 1999, p. A-8). Consistent with this approach,
Institutional Review Boards typically ask for a specification of hypotheses at the time
of the initial proposal.
Perhaps because of the many ways this pattern is reinforced, establishing a
research question and hypotheses prior to data collection, and then leaving these
largely untouched thereafter, has become an engrained expectation for many scholars.
As Stephens, Barton and Haslett (2009: 466) suggested, the steps of positivist
approaches, “observation, hypothesis, experimentation and generalization (have
become) indoctrinated into all particular methods of science.”
10
If the assertion of indoctrination is accurate, it is perhaps unsurprising that
there is little discussion of the processes through which research questions are initially
constructed, there is even less discussion of the processes through which such
questions may (legitimately and ethically) evolve once research has begun. There are
many possible motivations for this. Agee (2009), for example, argues that researchers
often develop initial, fixed, research questions for no other reason than that funders or
potential collaborators expect to see them. In our experience doctoral committees
often expect this, and many journal editors and reviewers share this expectation.
Even Alvesson and Sandberg (2013b: 43), whose entire book is devoted to
constructing research questions, say very little about how such questions might
evolve, with the exception of recognizing that sometimes “there is no clear initial
research question or there are gradually or abruptly changing research objectives
that perhaps last for several years”. The lack of recognition of change in research
questions remains true even in volumes that speak formally about qualitative, field-
based research (cf. Given, 2008) which flows from very different epistemological and
ontological origins.
The positive consequences of allowing room for research questions to evolve
In our view, the lack of explicit consideration of evolution of research question
significantly limits possibilities for learning from research projects, especially those
that are field based. This is so because the very word research carries subtle but
significant connotations.
First, research is defined as the systematic study of a topic to enable the
researcher to establish new conclusions (from the Oxford English Dictionary). In
everyday usage, this emphasis on newness relates to our own academic sense of the
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
demand placed by doctoral committees and journal editors alike for novel theoretical
contributions. Hence, a research question may be held as the framing of some, as yet
unknown, aspect of a situation, data set or body of theory.
In addition, empirical researchers working in the field of management and
organization are familiar with a second and co-existing connotation of research. This
occurs whenever we familiarize ourselves with a new empirical context. As
researchers entering the field, we face the challenge of grappling with things which
are at once “strange, irregular and inexplicit” (Geertz, 1973: 10). As we establish how
organizational processes, incentive structures, reporting lines, decision-making
processes and the like operate, or at least appear to operate, or as we study the
processes enacted by those who do these things, we are researching (Evered & Louis,
1981). The subtle connotation of research here is the enmeshed sense of finding out
about, orienting ourselves within, or becoming acculturated to a new setting. This
implies the possibility, and indeed the likelihood, that as scholars become acculturated
in a new setting they may learn that research questions formulated without respect to
this understanding may no longer be suited for use in situ.
Mezias and Starbuck (2003) describe an example of such discovery in a study
exploring managerial perceptions of variables that academics claim are important.
Through a study that lasted over two decades, the researchers showed how they
learned over time that the research questions had to keep changing so as to
accomplish their initial purpose. To elaborate, as the study progressed and the
researchers learned more and more about the settings in which they were working, the
research questions themselves evolved. The initial research questions focused on
variables found in academic prescriptions, e.g. about organization design and strategic
planning, and were very quantitatively oriented. However, the researchers found that
12
some variables discussed in the management literature were very salient to the
managers while other variables made no sense to them. Hence they had to reformulate
the research questions in ways that did make sense. In other words, their study
describes an odyssey of discovering which research questions (and data gathering
approaches based on them) were meaningful for their population at particular points
in time, recognizing that these may, very appropriately, require change over time.
Action Research and the Development of Research Questions
There is also growing recognition that research questions may be particularly
likely to evolve during field-based action research projects, especially when some of
those involved, including actors in the setting, have a personal stake in the projects
(e.g. Bartunek, 2008) and the settings themselves are volatile (Kacen & Chaitin,
2006). While the traditional, theory-led approaches to research see research questions
“set and solved in a context governed by the, largely academic, interests of a specific
community” (Gibbons et al., 1994: 3), engaged forms of research (Van de Ven, 2007)
such as action research involve a considerable level of dialogue between those in a
particular field-based situation and those hoping to study that situation in a way that
contributes to improving the situation as much as it focuses on making a conceptual
contribution. Such situations are typically very complex; as such, it is difficult to
conduct what scholars operating from a positivist epistemology consider truly
rigorous, well controlled research. Further, in action research, as with other forms of
field-based research, neither researchers nor research participants may have complete
control over events that affect the research. This likely affects the formulation and
possible reformulation of research questions.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
However, even in action research and similar forms of engaged research,
consistent with the approach followed by positivistic research, the default position
regarding the timing of research question development appears to favour fixing the
question early. For instance, Bartunek and Louis (1996) describe the framing of the
research question as one of the earliest stages in insider/outsider collaborative
research. Similarly, Carney, Dundon & Léime (2012) argue that research questions
should be rigorously determined in advance, even in community-based action
research projects, in part to enable replicability. Further, Van de Ven states that
“Priorities need to be established by formulating a specific question that will be
addressed in a research project … The research question not only narrows the focus of
a study to manageable dimensions, it also establishes a pragmatic criterion for
evaluating the relevance and quality of a research project” (2007: 88).
In contrast, we believe that it is reasonable to assume that research questions
may, and perhaps should, evolve over time in field-based settings such as those in
which action research is typically carried out. Cordner and Brown (2013) argue, for
example, that field-based research in situations where there are environmental health
threats evokes “moments” of scientific uncertainty (p. 470), including in the choice of
research questions, and that this in turn affects ethical concerns. They note (p. 478)
that “this moment of uncertainty leads to ethical tensions which can be unresolved if
formal ethical guidelines lag behind the development of novel methods or do not
adequately prepare researchers or practitioners to deal with the relevance of findings
for nonscientific purposes.” In other words, they are indicating that ethical guidelines
have not evolved in ways that match necessitated changes in the development of
research questions in such settings. Guidelines designed for hypothesis testing
research are inadequate for field-based action research studies.
14
Likewise, Sasco et al. (2010: 7), discussing the challenge of AIDS-Related
malignancies in sub-Saharan Africa, stated that “By collecting and harmonizing data
from many HIV/AIDS cohorts from Western and Southern countries, this initiative
will address unique and evolving research questions in HIV/AIDS such as its
association with malignancies currently unanswerable by single cohorts.” Further,
Moschitz and Home (2014: 400), in a study of the challenges of innovation for
sustainable agriculture and rural development, critiqued academic researchers who
had “defined the research questions at the time of proposal writing instead of jointly
developing relevant questions at the local level”. They observed that social realities
change and project priorities shift throughout field-based studies like this.
The argument for the evolution of research questions has also been made in
qualitative field research. For instance, Marshall and Rossman (1999: 23) argued that
“In qualitative inquiry, the proposal should reserve some flexibility in research
questions and design because these are likely to change during the research process”.
Similarly, Paulsen (2009: 510) noted that with regard to ethnography, a research
method that like action research focuses on addressing the experiences of participants
in their own settings, “rigid adherence to research questions predicated on advance
knowledge can easily prohibit investigation of what we later find to be most
interesting. In some instances, emphases change entirely as new and interesting
dimensions of a scene reveal themselves”. Also, Van den Hoonaard and van den
Hoonard (2008, p. 186) hint at the importance of delay in setting the research question
when using grounded theory methodologies, noting that in such methodologies it is
not unusual and sometimes more appropriate for a research question not to be settled
on until after at least some data gathering. Weinberg’s ethnographic study that began
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
with the straightforward query: “Why are the nurses crying”, evolved into a book-
length study of money-driven hospitals (2003).
Evolution of research questions and the quality of action research
While some of the extant scholarship that we have described recognizes that
research questions can change, what is not clear is how and why such changes occur.
For instance, studies like those of Sasco et al. (2010) and Moschitz and Home (2014)
focus on the fact that the questions do shift, rather than an exploration of the factors
that lead to such shifts, or what the implications of the shifts may be for the ability to
carry out the research in some volatile settings. For instance, it may be that for
accomplishing high quality action research, especially collaborative action research,
research questions should be flexible by necessity.
To elaborate, while what quality means in action research is not as clearly
specified as in approaches to positivist research, it has been recognized that high
quality action research requires flexibility and evolution. Reason (2006: 197) claimed,
for example, that:
“Good action research emerges over time in an evolutionary and
developmental process, as individuals learn skills of inquiry, as
communities of inquiry develop, as understanding of the issues deepens,
and as practice grows and shifts changes over time. Emergence means
that the questions may change, the relationships may change, the purposes
may change, and what is important may change”.
In a similar vein, Coghlan (2011: 71) argued, based on Shani and Pasmore (1985) that
“[G]ood action research may be judged in terms of four factors: how
the context is assessed; the quality of collaborative relationships between
researchers and members of the system; the quality of the action research
16
process itself as cycles of action and reflection are enacted and that the
dual outcomes reflect some level of sustainability (human, social,
economic, and ecological); and the development of self-help and
competencies out of the action and the creation of new knowledge from the
inquiry”.
Coghlan and Shani (2014: 534) build on the discussion of these four factors
and add that: “Leading an action research project in the present tense requires
simultaneous attention throughout the project to all four factors. For example,
capturing a change in the action research project course due to an unexpected
change in the company strategy due to a new innovation or a strategic opportunity
and how it is impacting the emphasis and/or level of engagement in the inquiry
process and/or its direction generates critical insights of relevance to both the system
and scientific community”.
In other words, in action research settings, while an initial focus may be on
developing research questions through challenging assumptions, gap-spotting or a
systematic review, the process of collaborating with others (typically actors within the
setting) in a specific field situation requires an openness to how the setting itself, as
well as collaboration with others there, may require some reformulations of research
questions in response to ways in which the situation changes. This openness to the
evolution of research questions is very different from “HARKing”. High quality
action research needs to pay attention to possible shifts in settings, as these may affect
the appropriateness of initially formulated research questions. High quality action
research also needs to foster relationships between outside researchers and insider
members of a setting, relationships of “trust, concern for other, equality of influence”,
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
among other characteristics (Coghlan and Shani, 2014: 525). Such relationships may
also require a capacity to adapt the focus of particular studies, especially as insider
members of the setting have knowledge external researchers do not…
While we have made this point abstractly, it is important to explore how such
evolution might take place in a specific field-based action research setting. Fleshing
this out will add to the understanding of what shifts in research questions might mean
in practice in field studies jointly conducted by academics and practitioners. To that
end, we present a case that conveys well what may happen to research questions in
action research projects in a setting which is itself volatile.
Bi-Lingual Education in the Lomond School
The research study that we describe here is a part of a larger study of a Joint
Partnership Program (JPP) between a School of Education (SOE) in a major
university in United States and the public schools (PS) in the neighborhoods near the
SOE. Two of the authors of this paper, Jean and Mamta, conducted an assessment of
this program.
Ten years prior to this assessment, the then Dean of the SOE had received a
substantial gift from an anonymous donor to establish a JPP that would support
faculty and students from the education faculty to partner with teaching staff from
local public schools in action research projects that lasted up to three years. At the
time of the assessment was conducted, 16 such projects had been implemented. They
had addressed topics as diverse as anti-racism training in urban schools, improving
urban children’s self-esteem and self-confidence, implementation and evaluation of a
comprehensive health curriculum in a school-community collaborative, and engaging,
motivating, and increasing elementary school students’ interest in science and
18
engineering while improving conceptual understanding of science and their ability to
write and communicate with others.
The particular action research project we describe here aimed to foster
bilingual education in Portuguese and English at the Lomond School (a school
enrolling students from kindergarten through grade 6). The materials we present
below are based on the data collected for the assessment. These included the initial
proposal for the project, interim project reports submitted during and after the
project’s first, second and third years, publications generated from the project, and,
finally interviews we conducted with three participants after the project was
completed. The interviewees were the principal investigator, Professor Walters, one
of the Lomond School (LS) bilingual teachers, and an SOE associate, a doctoral
student who was working with Professor Walters. Table 1 summarizes key
information related to the project.
---------------------------------
Insert Table 1 about here
----------------------------------
Context of the study. Within the state in which this particular action research
project took place, there was a law at the time that wherever there were: “20 or more
enrolled children of the same language group who cannot do ordinary class work in
English and whose native language is not English and whose parents do not speak
English…schools must teach all required courses in both English and the child’s
native language.” There had been Portuguese-speaking students at the Lomond
School for several years, but that year, there was a sudden increase in the number of
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
these students. This increase triggered a need for the school to implement bilingual
education – in Portuguese and English - much more fully.
Formulation of the initial goal. The principal of the Lomond School (Ms.
Wachter) approached Professor Walters, of the SOE, with a request for help with the
bilingual program. Professor Walters was known for her expertise in bilingual
education; she had engaged in this work for decades and had written multiple articles
and books on creating quality bilingual education. In response to the request from Ms.
Wachter, Professor Walters applied for, and received, a three year JPP grant to guide
program development in bilingual education at the Lomond School.
Ultimately, the project team included Professor Walters other SOE faculty
specializing in bilingual education, doctoral students (referred to here as SOE
associates), Lomond School teachers, para-professionals, and Ms. Wachter, the
principal. Professor Walters was the Principal Investigator (PI) and led the project; her
recent work on bilingual education was used as “study material” for the teachers and
others involved in the project.
Initial research question(s) and purpose. The research team’s initial goal was to
develop and study the development of a new Portuguese Bilingual Education
program. As stated in the proposal that Professor Walters submitted to the SOE for a
JPP grant, the initial research objectives of the project focused on the following:
1) to develop “portfolios documenting the characteristics of the bilingual
education program as well as the school and community context;”
2) to investigate the effects of the creation of these program portfolios on the
quality of the bilingual program with respect to school goals, relation to
community, curriculum, instruction, and student monitoring
20
3) to produce an entry for the national data base - Bilingual Education:
Portraits of Success - for programs that qualify school, curriculum,
instructional and assessment practices.
Though framed as research objectives rather than questions, the clear implication
was that the study was asking: what are the characteristics of effective bi-lingual
education?” Professor Walters and the project team initially proposed to focus on the
Portuguese Bilingual Education program at the Lomond School in the first year and
include other public schools in years 2 and 3 of the grant. Further, as they stated in the
initial proposal submitted to the SOE, they planned “to engage in ongoing inquiry
with respect to what the program has as well as what further work is needed” for
effective bilingual education.
In terms of the typical methods for developing research questions, the research
question described here included components of both gap-spotting and a systematic
literature review. It was certainly aimed at determining characteristics of effective
bilingual education in a new setting, thus filling in a particular gap in the literature. In
addition, it was based on a systematic development of a literature on this topic that
went back over multiple decades and to which Professor Walters had contributed
significantly. To elaborate, the SOE had collaborated with other universities and the
National Association for Bilingual Education on a national “Portraits of Success”
project that entailed studying successful bilingual programs. This action research
project was expected to build on, and add to, this systematic body of literature
Portraits of success - by presenting new illustrations of what effective bilingual
education entails.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
How the project team worked to achieve the goals. To address these goals,
Professor Walters met with the principal and the Lomond School teachers to
understand the teachers’ perspectives and needs. She also had formal meetings with
the SOE associates every two weeks. The purpose of these meetings was to identify
immediate goals and to plan. The SOE associates then visited the Lomond School
and worked on the plan during the next week. Professor Walters also met with the
Public School teachers on a regular basis. She, along with the SOE associates,
prepared the agenda for each meeting; however, it was fluid and changed during the
meetings depending upon the concerns raised by teachers. Teachers were also asked
to complete selected readings from Professor Walters’ book on bilingual education
and to “bring in data from their classrooms (e.g. samples of student work,
instructional materials, daily schedules and self-observations) in order to enhance
discussion” (Source: End of Year 1 report submitted to the SOE).
The SOE associates, along with Professor Walters, worked hand in hand with
the Lomond School teachers. They visited the school regularly to observe the classes
and then talked to the teachers about what they did and teaching strategies they could
adopt in the classroom. In addition, Professor Walters and the SOE associates
conducted professional development activities for the Lomond School teachers. As an
SOE associate indicated in her interview: One of the main issues of the bilingual
program was that teachers were not trained to be bilingual teachers. Most of them ...
learned how to teach in Brazil…. They needed some training in terms of how to
apply the skills learned in Brazil”.
Additional activities undertaken in the first year. During the first year, all of
those involved in the action research project (Lomond School teachers,
paraprofessionals and students, Professor Walters and SOE associates) focused on the
22
first two components of their goals, especially the gathering of information in the
service of developing portfolios. Specifically, they focused on gathering information
on what already existed in the bilingual program at the Lomond School. This included
assessing the potential strengths of each teacher and paraprofessional at the school
and collating relevant background information such as the language and country of
origin of the students and families. They engaged school personnel in data gathering
as well. For instance, a math teacher arranged for her students to survey the language
and country of origin of the students at the Lomond School; the analysis of the
information not only provided an input for the Portfolio but also raised the awareness
of linguistic diversity amongst mainstream teachers who taught only in English and
who tended to be less in favor of bilingual education than those who taught in the
bilingual program.
Similarly, the project team met on a regular basis, to discuss what they were
learning about their students and families and what this implied about their language
and instructional needs. For instance, they learned that the parents agreed with the
teachers that the students should become both bilingual and bi-literate. Further, the
team found that there was “a significant discrepancy in the amount of English and
Portuguese used at each grade levels” (Source: End of Year 1 report submitted to the
SOE). Given this parental preference, the team agreed to the specific goal of having a
balanced amount of instruction in both languages with strong literacy development in
the heritage language. The Lomond School teachers and their SOE collaborators then
worked together to develop a comprehensive program of instruction for each grade
level, creating and revising alternate models until “the team arrived at schedules that
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
contained adequate instruction in each subject area and included an appropriate
amount of instruction in both English and Portuguese” (source: End-year 1 report).
Note that this key focus was arrived at in collaboration with the Lomond
School personnel after the SOE research team entered the field. Yet, it (appropriately)
had an effect on what could and should be studied. An additional focus emerged in a
similar way: “As the teachers reworked their instructional schedules, they became
increasingly aware of how they could work and teach together to best take advantage
of each other's strengths, both in terms of language and instruction” (source: End-
year 1 report).
Finally, there was a parents night near the end of the school year in which the
parents “enjoyed a performance of music and plays in Portuguese and English
organized by the bilingual and music teachers” Ms. Wachter, Professor Walters, and
other teachers responded to the parents’ questions about bilingual education. All in
all, it was a busy first year, one in which a great deal of information was uncovered
about bilingual education needs and in which plans were made to foster bilingual
education.
Revising the research question(s) in the second year: Minor changes
In their JPP report at the end of the second year, the research team listed a
somewhat revised research question: Does completing the information required in
the nomination form of the Portraits of Success Project help a new program in
making decisions about curriculum, instructional and assessment practices? Does it
help program development?” This question was clearly based on the learnings from
the first year. These learnings had led to new activities the second year, and to at least
one additional change in the original research question.
24
Minor changes and their impact on the project. As noted above, the original
plan was to move to expand the scope of the project by incorporating a second school
in the second year of the action research project. However, by the end of the first year,
the team concluded that there was still “much to be done at the Lomond School”
(Source: End of Year 1 report submitted to the SOE) and decided to focus only there.
There were developments in the project, some of which were based on
changes happening in other parts of the school. For instance, as Professor Walters
stated in her interview, the principal had to “reduce one of the mainstream classes.”
This freed up one of the teachers who joined the action research team as the literacy
teacher; she “became an extraordinary force and brought the bridge between the
bilingual and the mainstream program” (Source: Interview with Professor Walters,
Project PI). In addition, the teachers’ collaboration with each other increased. The
Lomond School teacher we interviewed commented “I believe it was the second year,
when … we had combined classes but we used to switch. I would just go and teach
English to this group and the other teacher would come and teach Portuguese for all
subjects, especially literacy.”
Further, the team made additional adjustments to the objectives of their
research when it became apparent that “there was not a systematic approach from the
school or the district to look at assessments of English and that was a very difficult
issue for teachers” (Source: Interview with a doctoral student from SOE). As
Professor Walters also stated in her interview, when the research team started asking
questions about assessment, they “didn’t seem to get any straight answers and the
teachers didn’t seem to know what the assessment policies were” (Source: Interview
with Professor Walters, Project PI). Thus, whilst the focus in Year I had been mostly
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
on learning about school and program characteristics, goals, personnel, curriculum,
and choice of language for instruction, in Year II, the emphasis shifted to instruction
and assessment, specifically concerning literacy, learning techniques and assessment
strategies that would work well in a bilingual setting. Consequently, the team’s
research question was also adjusted to ascertainwhat role does assessment play in
the effective delivery of bi-lingual education?”
Further, the team recommended activities to determine the language and
country of origin of the students at the school in each new academic year, something
that had not been done in the past. Additionally, the team compiled a list of all the
Portuguese books owned by the school and the age group(s) for which they were
appropriate. This led to the realization that these materials were inadequate, with
notable omissions such as dictionaries. In the true spirit of action research, the JPP
grant paid for some books to be purchased, and some SOE personnel sourced
donations of textbooks currently used in Brazil. Native Portuguese speakers studying
at the University where SOE was located volunteered as teaching aides (source: Mid-
year two report).
Meantime, the population of Portuguese speakers in Lomond School’s
catchment area decreased during the second year, because the price of housing had
risen in the geographic area served by the school. The smaller number of students
could potentially threaten the need for the bilingual program.
Revising the research question(s) in the third year: major changes
By the third year of the project the original intention was to focus primarily on
assessment and the institutionalization of insights generated from the first two years
of the study. This was an emergent focus, one that had arisen from discovery of the
importance and complexity of assessment. In particular, the emphasis would be on
26
working with the existing assessment tools, along with public school standards, to
identify the best means of determining how well students in bilingual education in the
Lomond School were doing. To accomplish this objective, the team also decided to
create an informational brochure about the program for parents and a teacher’s
manual outlining basic decisions about teaching, scheduling, and assessment that
could be given to all new staff. These began to be implemented.
Major change in the third year. During November of the third school year of
the project, a ballot initiative was passed in the state that mandated instruction in
English for all bilingual students with limited English ability. There would be
discontinuation of all bilingual education. Instead, the new law required “public
schools to educate …children who cannot do ordinary class work in English and who
either do not speak English or whose native language is not English… through a
sheltered English immersion program, normally not lasting more than one year.
Once a student was able to do regular schoolwork in English, the student would be
transferred to an English language mainstream classroom”.
Impact of the new law on the action research project. This new legislation
had a major impact on the research project, as well as bilingual education more
broadly. The team was forced to abandon the objectives of developing and assessing
the bilingual program altogether. As opposed to the work that had been planned for
the third year of the project (i.e., assessment and the creation of mechanisms to
institutionalize the program), the research team had little choice but change the focus
of their study to reflect the new context in which they were working.
As result of the new law, the teachers who had been in the bilingual program
could not teach in Portuguese anymore. This was distressing for the Lomond School
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
teachers; as Professor Walters mentioned in her interview:the teachers were crying
half of the time because of the uncertainty and what was happening to them and to the
children” (Source: Interview with Professor Walters, Project PI). Their distress
motivated one of the PhD students from the SOE to do her dissertation on the impact
of the legislation on teachers: “[My dissertation) started because…I could see the
teachers struggling” (Source: Interview with SOE associate).
Following the legislative change, the focus of the project evolved from
“improving a bilingual program characteristics to providing teachers with the tools
to teach bilingual students in a monolingual setting” and “what would teachers need
the following year in order to still provide quality instruction to bilingual students”
(Source: Interview with a an SOE associate). Professor Walters conducted workshops
on “how they would have to switch their approach” and, along with her team, focused
on preparing bilingual teachers to use Sheltered English strategies and to pursue
English as a Second Language (ESL) certification (Source: Interview with Professor
Walters, Project PI). In addition, they interviewed Lomond School teachers to
investigate the information available to them in connection with these drastic
curricular changes and to learn about the changes. Further, they developed a course
where 13 bilingual teachers from two Boston Public schools (Lomond School and a
second school) were trained on how to teach bilingual students using English as the
only language of instruction.
The change in goals was evident from the publications coming out of the
project. The presentations and papers focused not only on the process of development
of bilingual program and assessment of bilingual program (i.e., papers based on work
done in the first two years of the project), but also on topics such as the impact of the
new law on Bilingual teachers. While the team did not formally reframe the research
28
question in their third year report, it was evident that the new legislation had a strong
bearing on the activities carried out in conjunction with the project. These were very
different than had been anticipated, and seemed to deal in particular with questions
such as how do teachers involved in bilingual education respond to legislative
changes regarding such education?”, as well as what kinds of supports might be
available to help them.
To summarize, the action research project at the Lomond School involved a
number of events in the course of its three years that resulted in the evolution of
research questions and emphases. These did not evolve because findings supported
other hypotheses better, but rather because circumstances made it impossible to
continue asking the originally posed questions. We discuss the meanings and
implications of this below.
Discussion
Our purpose of reviewing the bi-lingual education case study was to explore
the evolution of research questions in the context of a longitudinal, highly engaged
research relationship in a volatile field setting. As was evident, environmental events
over which the external researchers had no control, as well as discoveries of actual
circumstances at the Lomond School itself required changes in the research questions
addressed. As an authoring team with experience of many such longitudinal research
projects, the four of us believe that several of the characteristics of the bilingual
education case recur across many such projects. In this section, we will discuss some
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
observations about the processual dimensions of research questions which might
otherwise remain hidden from view.
Extant explanations of research question formulation emphasize the
importance of gap-spotting, systematic literature reviews and counterfactual
reasoning. All three of these imply that the genesis of research questions lies in
careful, prior examination of the literature in a way that is largely devoid of
interactions with people involved in a setting or external conditions of that setting,
especially if they might be changing.
Whilst we obviously appreciate the importance of theoretical framing, we
believe that in action research the drive to contribute “to the practical concerns of
people in an immediate problematic situation” (Rapoport, 1970:499). Action research
has been found wanting when assessed against the criteria of positivist science where
theoretical developments can only flow fromdata that can be directly experienced
and verified between independent observers” (Susman and Evered, 1978: 583). The
low likelihood of action research generating theory based on data that meet these
criteria led MacIntosh and Bonnet to speculate that whilst qualitative research more
generally is treated as the methodological “poor cousin,” action research represents
“the poor cousin’s downtrodden neighbor” (2007: 321). Absent directly experience
and independently verifiable data, action research places greater importance on the
ways research questions are formed and reformed. For instance, in the bi-lingual
education study, like most action research, the data to be gathered were integrally
linked with addressing problems experienced by those in the Lomond School, about
which there was genuine concern and an “intention to take action on the basis of the
intervention” (Eden and Huxham, 1996). This case illustrates the need to locate
theoretically informed questions in the context of particular empirical circumstances
30
that are meaningful to both outside researchers and members of a setting in which the
research takes place.
March (2000: 56) implies that this imperative for focusing on the setting and
its members when developing research questions may run counter to the principles of
good research, since “the primary usefulness of management research lies in the
development of fundamental ideas that might shape managerial thinking, not in the
solution of immediate management problems”. However, as we have demonstrated, it
is not unusual in longitudinal organizational research that the initial circumstances of
the study undergo a change in such a way that the original research questions are not
the best way to explore the settings (cf. Mezias & Starbuck, 2003). For instance,
companies, or parts of companies, get acquired or divested; informants get promoted,
move, and/or are made redundant. This is particularly true when studying
organizational change. As illustrated in the case of the bilingual education study,
researchers entered the setting and gathered more information and a better
understanding of the requirements (e.g., about the desire for both bi-lingual and bi-
literate outcomes, and a need for better assessment strategies) than had been
previously available. Similarly, the setting was characterized by challenges (e.g., lack
of instructional material) and changes (e.g., changing house prices in the region; the
change of law which occurred in year 3 of the project) that had not been expected.
Individually and collectively, these changes had a bearing upon how the initial
research question evolved.
We suggest that, especially in action research projects and as depicted in
Figure 1, the (ongoing) development of research question(s) involves the interplay of
three different dynamics: 1) the conceptual basis for the questions, 2) the context and
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
3) a reflexive engagement with the interests of the researchers and other actors in the
setting member who help to compose a research team (Bartunek & Louis, 1996). In
other words, our first observation would be that researchers are involved in a co-
constitutive dialogic (Beech et al., 2010), or dialectical encounter, between the extant
literature, their own interests, and the empirical setting. Further, characteristics of the
setting may well change in small or large ways that may be far beyond the capability
of the researchers or the setting itself to affect, but thinking of the research questions
as affected by the setting makes it possible for the dialectic involving them to shift.
Our second observation is that, on entering the empirical setting, the
dialectical encounter of initially established research questions with the setting and
with researcher interests reflexively determined, may result in three potential
outcomes as shown in Figure 1. It could be that the setting is as expected and remains
stable enough during the conduct of the research and the original research design can
be executed unproblematically; i.e. there is no change in the initial research
question(s).
---------------------------------
Insert Figure 1 about here
----------------------------------
In contrast, it could be that through a process of familiarizing themselves with
and reflecting on the research setting, researchers may realize that the situation is not
as originally expected. In the bi-lingual education case, the researchers learned the
nuanced difference of bi-lingual and bi-literate outcomes as well as realizing the
relative importance of assessment. Given the complex and multifaceted nature of
32
organizations, such nuances may be difficult to articulate before entering the field.
Initial data gathering may reveal subtleties, misapprehensions and new insights that
would have been difficult to glean before entering the setting without a sensitizing
research question. For instance, in the second year of the bi-lingual education project,
the researchers reassessed the original plan to expand the study and include other
schools, because the characteristics they encountered the first year proved to be more
difficult to deal with than they had expected; something that became clear in their
reflection on the setting. Further, there could be changes in some aspects of the
organizations and their larger contexts (e.g., the change in the law as in the bilingual
education case), which may have a bearing upon the framing of research questions.
Hence, the researcher may often gain further nuance and understanding once in situ.
As illustrated in the bilingual education case and depicted in Figure 1, such changes
may result in two distinct types of evolutionary processes in research questions. We
call these evolutionary processes refinement and reframing respectively.
Refinement involves adjustment to the particular focus of a research question
following engagement with the research setting in a way that leaves the original
intention of the research intact. The refined research question(s) results from minor
changes in the initial question(s); they are essentially more fine-tuned questions which
may be more focused, more relevant and more feasible since they take into account
the challenges and constraints imposed by the setting and researchers’ understanding
of them after entering the field. In the bi-lingual education case for example, this
refinement led to an increased focus on the role of assessment in the effective delivery
of learning outcomes. The trigger for this evolutionary step was the gathering of more
detailed knowledge of the research setting once in situ. Of course, a refined research
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
question could still be linked to the process of systematically reviewing the literature
whilst such contextual refinement occurs. For instance, in the bi-lingual education
case presented here, it is clear that the researchers could respond dialogically to their
setting by adjusting the search terms used in their review of the literature, perhaps
incorporating other studies with a stronger focus on assessment methods.
Reframing, on the other hand, involves a substantive and potentially
discontinuous shift in the focus or nature of the research question. Reframing is likely
to occur when, either on entering the research setting, or at some point during the
study, the original research design is rendered unworkable. In these circumstances,
researchers can either choose to abandon the study or reframe their original research
question. In the particular case of the bi-lingual education study described earlier, the
dynamics of the housing market led to decreased demand for bi-lingual education (in
year 2) before a legislative change outlawed the very phenomenon that the researchers
had committed to explore (in year 3). Such substantive shifts in the context may mean
that research questions move from a specific instance to the general case, e.g. from
the study of bi-lingual education to a study of the impact of changes in education
policy on teachers, children and parents. Equally, a substantive shift may be achieved
by changing the unit of analysis or the focus of the research, e.g. from within school
educational practices to local government/school relationships. For instance, in the
case we described above, the researchers shifted their focus from studying the
development and assessment of the bilingual program to developing insights on the
ways in which bilingual teachers could work within the framework of a new law.
Such reframing may be a way to foster a counterfactual reasoning form of research
question generation, since the disruption of the empirical setting may provoke the
34
researcher to question more fundamentally the underlying assumptions and constructs
with which they are operating.
As we have suggested above, our experience in multiple such studies is that
research questions often do evolve, but that the process of their evolution is rarely
acknowledged in the final written account of the research. Figure 1 sets out a process
through which research questions might evolve. Initial research questions, at least as
presented by academic researchers, are typically generated by gap spotting, although
systematic reviews and counterfactual reasoning may also take place. But these
questions encounter both a particular research setting and the members of that setting.
Thereafter, in field-based studies such as the action research project reported here, the
appropriateness of research questions may be influenced by both contextual and
reflexive triggers. The subsequent evolution of research questions can occur
incrementally through refinement (which may involve further iteration of a systematic
literature review) or radically through reframing (which may involve counterfactual
reasoning).
We have experienced doctoral students beginning a dissertation on one topic in
a field setting, for which the originally promised data become unavailable. A different
dynamic occurs when researchers enter the field with one set of expectations only to
discover that these do not reflect either the most important or the most interesting
features of the setting. Action researchers and other scholars of organizational change
may start to study and/or work with a particular change effort only to find that it is
indefinitely postponed. And so on. Rather than pretending that such events do not
occur, it is essential to recognize them as important in revealing key features of the
setting and potentially signaling the possibility of other questions.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
When describing hidden practices of qualitative research, Sutton described the
uncertainty that flows from not knowing “when and how others have done it” (1997:
99). We suggest that researchers, and particularly those new to the practice of
research, would benefit from greater knowledge and clarity about the possible
evolution of research questions when they encounter a field setting and, especially,
ways in which such evolution is both acceptable and meaningful in field research.
One simple solution would be to incorporate, perhaps within the methods section, an
account of whether the research question was (a) stable with no change, (b) the
subject of some refinement or (c) reframed during the conduct of the study and why.
Our suspicion is that many field research projects would fit into categories (b) or (c)
and that a more honest, transparent and reflexive account of the journey towards a
crystalized and stable research question would greatly enrich our field. At the very
least, it would give much more information about the context in which the work is
being done than do virtually all studies now.
Concluding Commentary
Through illustration of the bi-lingual education study, we have shed light on
how and why research questions may evolve once external researchers enter the field.
We have also highlighted the fact that both contextual and reflexive triggers may have
a bearing on this development. Finally, we have noted two types of evolutionary
development, namely refinement and reframing. To conclude, we reflect upon why
researchers often refrain from discussing refinement or reframing of research
question, despite the likelihood that such evolution of research questions is a reality in
many action research projects – as well as other kinds of field research.
We have attributed the expectation of fixed hypotheses (and/or questions) to
the widespread influence of the scientific method and Popper’s notion of
36
“falsificationism”. In his book Objective Knowledge (1972) Popper made it clear that
he was talking about scientific knowledge which is independent of the observer, or
acts of observation, i.e. not “subjectivist” in his terms. For us, there is a danger that
social science research loses sight of Popper’s qualifying statement. Given the
popularity of the scientific method and the fact that many scholars of business and
organization studies originally trained in other disciplines (e.g. our authoring team
features an engineer, a physicist, an experimental social psychologist and a science
graduate), it is understandable for researchers to operate with an expectation that
research questions should be impervious to the effects of the myriad vagaries of
human interaction, i.e. from the processes of observation, reflection and other forms
of data collection that constitute the research itself. Within the scientific and rational
mindset, the data and the processes of collecting the data should not distort the
original research design. The bi-lingual education case challenges this
straightforward chronology by suggesting that it is a somewhat mechanistic, linear
and unnecessarily narrow interpretation of what constitutes robust research practice.
A Popperian view of immutable research questions serves rather well while
researching the dynamics of planets or chemicals. But what happens when the focal
points of enquiry are the very (inter-subjective) processes of human interaction, which
objective enquiry, as outlined above, explicitly regards as barriers to scientific
research? In a fascinating essay on ideology and methodology in the social sciences,
philosopher Alasdair MacIntyre (1998) accounts for the dominant, positivistic
orthodoxies of enquiry as not simply a perhaps strained, but otherwise, innocent
application of the scientific method, which he regards as an ideologically structured
process of bureaucratic control. Consider first his observation:
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
“Methodology then functions so as to communicate one very particular vision
of the social world and one that obscures from view the fundamental levels of
conceptuaIization, conflict, contestability, and unpredictability as they
constitute and operate in that world. It thus has one of the two centrally
important effects of ideology (op cit, p65)”.
He goes on to explain some of the “centrally important effects of ideology”,
namely that the particular view alluded to above operates in the interests of control on
the part of a particular group. Of course, such control offers the benefits of relative
stability, predictability and, in an era of escalating concerns for sound governance,
clear lines of accountability. What is interesting here is that in sharing roots with the
ideology of bureaucracy, it is an abiding view of the human as a rational actor
(MacLean et al., 2015) that underpins conventional research methods.
By rational, what we mean here is the idea that action is conceptualised in
terms of ends, means and conditions. One enters into a situation or conditions (in our
case, a researcher enters the context), with an end in mind (in our case, a research
question) and uses the available means (in our case, research protocols and methods)
to secure the end (answer the question). This is all done in a relatively straightforward
way since both the situation and the means are considered to be open to manipulation
by an intellectually driven process unimpeded by human experiences. If the process
is rational, the root metaphor is the mechanism. And thus we end up back, so to speak,
where we started, with the combination of Cartesian behaviour and Newtonian
mechanics that underpins to the modern scientific era (MacLean and MacIntosh
2012).
Yet, the unwanted “friction” in this Newtonian phenomenon, or the obscured
levels – alluded to by MacIntyre above as “conceptuaIization, conflict, contestability,
38
and unpredictability” - we would argue are the very essence, or hallmark, of good
action research in particular, and field research in general. If we develop MacIntyre’s
argument, the underpinning ideology of conventional research obscures the very
things that we as researchers are seeking to illuminate. For instance, in the case of our
bi-lingual education study, the research team faced a stark choice. Either they must
recognise that the social setting in which they were gathering data was changing
around them in real time, or they must abandon the research since it was not
conforming to original expectations. When studying change, there is inevitability
about such experiences which will likely resonate with established field researchers.
As Susman and Evered (1978) argued, it is simply not appropriate to assess the
scientific merits of action research from within a frame of reference which is
exclusively scientific and rational. Part of the way to avoid this is to operate out of an
ideology with different assumptions about human experience and action that go well
beyond counterfactual reasoning.
Conveniently, much work has already been done on this front. Specifically,
and in relation to the scientific method, complexity theory (Prigogine and Stengers,
1984) has posed a serious and sustained challenge to the centrality, or even possibility,
of prediction in complex systems involving many interacting elements. Instead of a
concern with control, and a design-driven process of predictable execution with
defined outcomes, complexity points to a world that is essentially dynamic,
unpredictable and governed by emergence rather than mechanical execution. Hence,
one way forward is to engage seriously with a new root metaphor for research as a
complex process or system rather than a mechanism. Elsewhere, two of the authors
have used a complexity perspective to conceptualise research as a dynamic which we
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
cannot know in advance and out of which meaningful questions will emerge and
evolve as the system adapts and transforms (MacLean and MacIntosh, 2003).
Having adopted a complexity perspective on the “research” in action research,
the other obvious step would be the adoption of a consistent view of human action to
complete a reframed view of “action research”. This involves introducing an
alternative to human beings and their settings as purely objective, utility-maximising
rational automatons.
We are not proposing the idea that humans and their actions are “irrational”
(though, of course, both often are). Instead, we simply suggest the need for a more
rounded view of humans and human interaction that incorporates our “non-rational”
faculties such as intuition, emotion, and imagination. Notably, these quintessentially
human characteristics which serve us so well in other aspects of our lives are placed
firmly “in the closet” (Sutton, 1997) when we read sanitized accounts of a linear,
logical, rational research process, especially if it is carried out in a field setting.
Another case in point is the work of social theorist Hans Joas (1996) towards
promoting a view of creative action which he claims is much better suited to the times
we in which we live, and which we see as much more able to deal with MacIntyre’s
“obscured levels” of contestability, emergence, unpredictability, etc. Joas draws on
American pragmatist philosophy to depict a view of action which, as an alternative to
the ends-means-conditions framework of rationalism, is organised around situated
social interaction, embodied expression, and, critically for the subject of this paper,
emergent intention. Instead of a fixed and prior intention, Joas argues that intention
itself emerges in the situation, in interaction with others, and influenced by a plethora
of embodied “non-rational” urges such as desires, chance ideas, mood, social
affinities and intuitions.
40
In this chapter, we have argued that the wider research community would
benefit from acknowledging that research questions can, and in many circumstances,
should evolve. What Joas points toward is a much more radical stance within which a
conception of intention itself as emergent utterly changes the status and nature of the
research question. Rather than a fixed entity, it becomes the evolving expression of a
collective intent to understand, as embodied researchers, the situation in which we
find ourselves. In doing so, we create with others in that situation a sense of who we
are, what we are doing together, and what this means. Emerging and evolving
intention is the lifeblood of a research process peopled by fully rounded, emotional,
intuitive and radically social individuals. From this perspective, a closed and fixed
research question becomes a straightjacket which reduces a living collective to an
impoverished mechanical interpretation of that same collective.
In this more radical perspective then, our argument is simple. Research
questions should evolve as researchers reflexively interact with the situation(s) being
studied. We have suggested ways this interaction takes place and identified two
distinct processes by which such evolution can occur (refinement and reframing). We
encourage fellow scholars to be more explicit about how their research methods
enable the most revealing aspects of the settings they are studying.
Refining and Reframing in “creative” action research
Research, conceptualised as creative action, fully acknowledges researchers as
creative human beings who are always in the social process of “becoming” (Ingold,
2013). Turning the focus away from answering a pre-set question perhaps to
“requestioning” what we are enquiring into, in the light of what we are experiencing
in the actual conduct of the research, creative action research pushes us to question
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
the extent to which we are abusing the original Popperian view of scientific method
when we apply it to social settings. A constant openness to change requires an “acute
empirical sensitivity” (Chia 2014), not only to an ever-evolving context or situation,
but to also to each other in our interactions and interpretations.
The increasingly familiar term “emergence” perhaps best sums up what we are
saying about research questions and research practice in general. In this paper, we are
drawing attention to a complex emergent dynamic, whose very nature or direction
cannot be known in advance in detail, and which is uncomfortably resistant to rational
mechanical control. We are calling for a first step in a longer process. By
acknowledging when and for what reasons our research questions evolve we will offer
more transparent access to our research findings. Beyond this transparency, there may
be a need to more fundamentally rethink the philosophical basis on which we ground
our definition of research.
In closing then, we suggest a metaphor that may bring some of our more
academic argumentation to life. Often the focus of scholarly research, at least the type
that is formally accepted by scholars, is on individual rose petals, chosen in advance
as discrete, varying objects whose intricacies may be explored according to pre-set
questions. Such focus enables considerable control. What this mindset ignores, of
course, is that rose petals are only alive and truly beautiful when they are part of the
roses from which they draw life. These roses are in constant development and change
during their lifetimes. An individual research question, formulated in advance, may be
a beautiful rose petal. Ignoring the ever-changing context of that question, the flower
which hosts it, may nurture it, and will eventually change it, is to do a disservice to
the temporality of what is being observed. The rose gives life to the rose petal.
Ignorance of this key fact, whether deliberate or inadvertent, eventually leads in turn
42
to a lack of appreciation of the even more marvellous gift humans are given, a
complete rose garden.
Acknowledgements
The idea for this paper was first rehearsed in a key-note address given by Jean
Bartunek at a Special Conference of the Strategic Management Society on Strategy in
Complex Settings, held in Glasgow in 2013. The authors would also like to
acknowledge all those involved in the action research project we discussed and the
help of Dr Angeliki Papachroni in preparing the final manuscript.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
References
Agee, J. (2009). Developing Qualitative Research Questions: a reflective process.
International Journal of Qualitative Studies in Education, 22(4), 431-447.
Alvesson, M. and Sandberg, J. (2011). Generating Research Questions Through
Problematization. Academy of Management Review, 36(2), 247-271.
Alvesson, M. and Sandberg, J. (2013a). Has Management Studies Lost its Way? ideas
for more imaginative and innovative research. Journal of Management Studies,
50(1), 128-152.
Alvesson, M., and Sandberg, J. (2013b). Constructing research questions: Doing
interesting research. Los Angeles: Sage.
Amis, J. M. and Silk, M.L. (2008). The Philosophy and Politics of Qualitative
Organizational Research. Organizational Research Methods, 11(3), 456-480.
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called
psychological science. Perspectives on Psychological Science. 7(6), 543-554.
Banerjee, A., Chitnis, U. B., Jadhav, S. L., Bhawalkar, J. S., & Chaudhury, S. (2009).
Hypothesis testing, type I and type II errors. Industrial Psychiatry
Journal. 18(2), 127-131
Bartunek, J., & Louis, M. R. (1996). Insider/outsider team research (Vol. 40). Sage
Publications, Inc.
Bartunek, J. M. (2008). Insider/Outsider team research: The development of the
approach and its meanings. In A. B. Shani, N. Adler, S. A. Mohrman, W. A.
Pasmore, & B. Stymne (Eds.), Handbook of collaborative management research (.
73 – 91). Thousand Oaks: Sage.
Beech, N., MacIntosh, R. & MacLean, D. (2010). Dialogues Between Academics and
Practitioners: the role of generative dialogic encounters. Organization Studies
31(9), 1341-1367.
Carney, G. M., Dundon, T., & Léime, Á. N. (2012). Participatory action research with
and within community activist groups: Capturing the collective experience of
Ireland’s Community and Voluntary Pillar in social partnership. Action
Research, 10(3), 313-330.
Chia R. (2014). Reflections on the distinctiveness of European management
scholarship. European Management Journal 32, 683-688.
44
Clark, C., Brody, M., Dillon, J., Hart, P., & Heimlich, J. (2007). The messy process of
research: Dilemmas, process, and critique. Canadian Journal of Environmental
Education (CJEE), 12(1), 110-126.
Coghlan D. (2011). Action research: exploring perspective on a philosophy of practical
knowing. Academy of Management Annals, 5,53–87.
Coghlan, D., & Shani, A. B. (2014). Creating action research quality in organization
development: Rigorous, reflective and relevant. Systemic Practice and Action
Research, 27(6), 523-536.
Cordner, A., & Brown, P. (2013). Moments of uncertainty: Ethical considerations and
emerging contaminants. Sociological Forum 28 (3), 469-494.
Committee on Professional Ethics. (1999). Ethical guidelines for statistical practice.
Retrieved from http://www.amstat.org/about/ethicalguidelines.cfm.
Cornelissen, J. P. and Durand, R. (2012). Beyond novelty: conceptual blending and
causality. Academy of Management Review 37, 152–54
Cornelissen, J. P. and Durand, R. (2014). Moving Forward: Developing Theoretical
Contributions in Management Studies. Journal of Management Studies, 51(6):
995-1022.
Creswell, J. (2007). Qualitative inquiry and research design (2nd ed). Thousand
Oaks, CA: SAGE.
Czarniawska, B. (2014) Social Science Research: from field to desk. London: SAGE.
Denyer, D., & Tranfield, D. (2009). Producing a systematic review. In D. Buchanan &
A. Bryman (Eds.), The Sage handbook of organizational research methods (pp.
671-689). London, United Kingdom: SAGE.
Donaldson, L., Qiu, J. and Nanfeng Luo, B. (2013). For Rigour in Organizational
Management Theory Research. Journal of Management Studies 50(1): 153-172.
Durand, R. and Vaara, E. (2009). Causation, counterfactuals and competitive
advantage. Strategic Management Journal 30, 1245–64.
Eden, C. and Huxham, C. (1996). Action research for management research. British
Journal of Management 7(1), 75 -86.
Evered, R., & Louis, M. R. (1981). Alternative perspectives in the organizational
sciences:“inquiry from the inside” and “inquiry from the outside. Academy of
Management Review 6(3), 385-395.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
Folger, R. and Turillo, C. J. (1999). Theorizing as the thickness of thin abstraction.
Academy of Management Review 24, 742–58.
Geertz, C. (1973). The Interpretation of Cultures. New York: Basic Books.
Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S, Scott, P. and Trow, M.
(1994). The New Production of Knowledge: The Dynamics of Science and
Research in Contemporary Societies. London: SAGE.
Given, L. M. (Ed.). (2008). The Sage encyclopedia of qualitative research methods.
Los Angeles: SAGE.
Glick, W. H., Miller, C. C., & Cardinal, L. B. (2007). Making a life in the field of
organization science. Journal of Organizational Behavior 28(7), 817-835.
Ingold T. (2013). Prospects” in Ingold T and Palsson G (eds) Biosocial Becomings.
Cambridge University Press.
Joas, H. (1996). The Creativity of Action. University of Chicago Press.
Kacen, L., & Chaitin, J. (2006). The Times They are a Changing 1: Undertaking
Qualitative Research in Ambiguous, Conflictual, and Changing Contexts. The
Qualitative Report 11(2), 209-228.
Ketokivi, M. and Mantere, S. (2010). Two Strategies for Inductive Reasoning in
Organizational Research. Academy of Management Review 35(2), 315-333.
Leung, D., & Lapum, J. (2008). A poetical journey: The evolution of a research
question. International Journal of Qualitative Methods 4(3), 63-82.
Lundberg C.C. (1976). Hypothesis creation in organizational behavior research.
Academy of Management Review 1, 5–12.
Lüscher, L. S., & Lewis, M. W. (2008). Organizational change and managerial
sensemaking: Working through paradox. Academy of Management Journal
51(2), 221-240.
MacIntosh, R. and Bonnet, M. (2007). International Perspectives on Action Research:
introduction to the special issue. Management Research News 30(5), 321-324.
MacIntosh, R., and MacLean, D. (2001). Conditioned emergence: researching change
and changing research. International Journal of Operations and Production
Management 21 (10), 1343-1357.
MacIntyre, A.C. (1998). Social Science Methodology as the Ideology of Bureaucratic
Authority in K. Knight, The MacIntyre Reader, Polity Press UK.
MacLean, D. and MacIntosh, R. (2003). Complex Adaptive Social Systems: towards a
theory for practice. In E. Mitleton-Kelly (ed) Complex Systems and
46
Evolutionary Perspectives on Organizations: the application of complexity
theory to organizations Amsterdam: Elsevier Science BV. 149-165.
MacLean, D. and MacIntosh, R. (2012). Strategic Change as Creative Action.
International Journal of Strategic Change Management 4(1): 80-97.
MacLean, D., MacIntosh, R. and Seidl, D. (2015) Rethinking Dynamic Capabilities
from a Creative Action Perspective, Strategic Organization, 13(4), 340-
352.Alvasdfasdfasdfasdf
March, J. (2000). Citigroup’s John Reed and Stanford’s James March on Management
Research and Practice. Academy of Management Executive 14, 52-64.
Marshall, C., & Rossman, G. B. (1999). Designing Qualitative research. Los
Angeles:SAGE.
Mezias, J. M., & Starbuck, W. H. (2003). Studying the accuracy of managers'
perceptions: A research odyssey. British Journal of Management 14, 3-17.
Moschitz, H., & Home, R. (2014). The challenges of innovation for sustainable
agriculture and rural development: Integrating local actions into European
policies with the Reflective Learning Methodology. Action Research 12(4): 392-
404.
O’Gorman, K. and MacIntosh, R. (2015). Research Methods for Business and
Management. 2nd Edition, Oxford: Goodfellow Publishers.
Paulsen, K. E. (2009). Ethnography of the ephemeral: studying temporary scenes
through individual and collective approaches. Social Identities 15(4), 509-524.
Peng, M. W. (2004). Identifying the big question in international business
research. Journal of International Business Studies 35(2), 99-108.
Popper, K. R. (2002). Unended quest: An intellectual autobiography. Psychology
Press.
Pratt, M. G. (2008) Fitting Oval Pegs Into Round Holes: tensions in evaluating and
publishing qualitative research in top-tier North American journals, Qualitative
Research Methods, 11(3), 481-509.
Popper, K. R. (1972) Objective Knowledge. Oxford University Press.
Prigogine, I. & Stengers, I. (1984). Order out of chaos: man’s new dialogue with
nature. New York, Bantram.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
Rapoport, R.N. (1970). Three dilemmas of action research. Human Relations 23, 499-
513.
Reason, P. (2006). Choice and quality in action research practice. Journal of
Management Inquiry 15,187–203
Shani, (Rami) A.B. & Pasmore, W.A. (1985) Organization inquiry: towards a new
model of the action research process. In: Warrick DD (ed) Contemporary
organization development: current thinking and applications. Scott Foresman
and Company, Glenview.
Rousseau, D. M., & Gunia, B. C. 2015 [in press]. Evidence-Based Practice: The
Psychology of EBP Implementation. Annual review of psychology.
Sasco, A. J., Jaquet, A., Boidin, E., Ekouevi, D. K., Thouillot, F., LeMabec, T., &
Dabis, F. (2010). The challenge of AIDS-related malignancies in sub-Saharan
Africa. PLoS One, 5(1), e8621.
Shotter, J. (2006a). Understanding Process from within: an argument for withness
thinking. Organization Studies 27(4), 585-604.
Shotter, J. (2006b). On the Edge of Social Construction: Wittgensteinian inquiries
into organization and management. Qualitative Research in Organizations and
Management 1(3), 189-203.
Shotter, J. (2008).Dialogism and polyphony in Organizing Theorizing in Organization
Studies: Action guiding anticipations and the contentious creation of novelty.
Organization Studies 29(4), 501-524.
Simmons, J. P., Nelson, L. D., & Simonshon, U. (2011). False-positive psychology:
Undisclosed flexibility in data collection and analysis allows presenting
anything as significant. Psychological Science 22, 1359–1366.
Stephens, J., Barton, J., & Haslett, T. (2009). Action research: Its history and
relationship to scientific methodology. Systematic Practice and Action
Research, 22, 463–474.
Sterba, S. K. (2006). Misconduct in the analysis and reporting of data: Bridging
methodological and ethical agendas for change. Ethics & Behavior 16(4), 305-
318.
Susman, G.I. and Evered, R.D. (1978). An assessment of the scientific merits of
action research. Administrative Science Quarterly 23, 582-603.
48
Sutton, R. (1997). The Virtues of Closet Qualitative Research, Organization Science
8(1), 97-106.
Tranfield, D., Denyer, D. and Smart, P. (2003). Towards a Methodology for
Developing Evidence-informed Management Knowledge by Means of a
Systematic Review. British Journal of Management 14, 207-222.
Tsui, A. S., Zhao, S., & Abrahamson, E. (2007). What to study in China? Choosing
and crafting important research questions. Management and Organization
Review 3(2), 171-181.
Van den Hoonaard, D. K., & van den Hoonard, W. C. (2008). Data Analysis. In L.
M., Given (Ed.) The Sage Encyclopedia of Qualitative research methods, pp.
186- 188. Los Angeles: Sage.
Van de Ven, A. H. (2007). Engaged scholarship: A guide for organizational and
social research. New York: Oxford University Press
Weick, K.E. (1989). Theory Construction as Disciplined Imagination. Academy of
Management Review 14, 516-531.
Wasserman, R. (2013). Ethical Issues and Guidelines for Conducting Data Analysis in
Psychological Research. Ethics & Behavior 23(1), 3-15.
Weinberg, D. B. (2003). Code green: Money-driven hospitals and the dismantling of
nursing. Ithaca; Cornell University press.
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
TABLE 1
The Case of Bilingual Education: A Summary
Study phases -Year 1
-Year 2
-Year 3
Key actors
involved
-Professor Walters from SOE (Principal Investigator and leader
of the project)
-SOE associates (primarily doctoral students working with the
project)
-Ms. Wachter, Principal of Lomond School
-Lomond School (LS) teachers
Initial Impetus
for the study
Increase in Portuguese speaking students in a Public School
located in a state that had a law mandating bilingual education if
there were 20 or more enrolled students from the same language
group
The research
question(s)
-Year 1: What are the characteristics of effective bi-lingual
education?
-Year 2: What role does assessment play in the effective
delivery of bi-lingual education?
-Year 3: How can teachers involved in bilingual education
respond to legislative changes regarding such education?
Day to day
activities of
participants
-Bimonthly meetings – Professor Walters with LS personnel
and with SOE associates
-Sharing of information among the LS personnel
-Regular visits to the LS on the part of Professor Walters and
SOE associates
-Observations of classes at the LS
-Discussions and feedback sessions
-Professional development workshops for LS teachers
Key milestones
of the study
-Compilation of background information (language and country
of origin) of students and families
-Creation of a summary of goals of a bilingual program
-Balanced amount of instruction in both Portuguese and English
-Increased collaboration between Portuguese and English
speaking teachers
-Parents’ night with Ms. Wachter and Professor Walters
-Compilation of list of Portuguese books
-Purchase of Portuguese books for the LS
-Creation of informational brochure about the bilingual
program
-Creation of teacher’s manual
-Ongoing development of assessment tools
-Training bilingual teachers to teach in English
Reflexive and
contextual
triggers for the
evolution of
research
questions
-Minor Change(s)
oIncreased understanding of the context (e.g., about lack
of assessment tools, parental preference that students
should be bilingual and bi-literate, inadequate number
of Portuguese books)
oChange in project participants from the Lomond
50
School
oIdentification of discrepancy in the amount of English
and Portuguese teaching at various grade levels
-Major change
oChange in legislation leading to discontinuation of
bilingual education in the state
Implication of
refinement of
action research
project
objectives
-Decision to focus only on one school (not moving to another
school as initially planned)
-Decision to focus on assessment practices and tools
-Engaging in activities like buying Portuguese books
Implication of
reframing of
action research
project
objectives
-Change in focus to help teachers prepare for teaching bilingual
students in English
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
Figure 1
How research questions evolve in field research
Initial research question and purpose based on gap spotting, systematic review
and/or counterfactual reasoning
current research question(s)
Evolutionary Outcomes
No change in research question
Refinement involving minor changes in the research question
-Adjusting the focus
-Becoming more relevant to the particularities of the situation
-Reflecting feasibility in situ
Reframing involving major changes in the research question
-Change of focus (e.g., constructs explored)
-Abandoning the question/research
Contextual Triggers
- Setting characteristics
-Change in the setting/ institutional context
-Challenges/Constraints
-Opportunities
-Expectations of project hosts/funders
Reflexive Triggers
-Increased clarity/ understanding
-More information
-Interest in emerging issues in the setting
-Concerns of setting members
52
R MacIntosh, J Bartunek, M Bhatt and D MacLean, I Never Promised You a Rose Garden: reflections on the evolution of
research questions, in A B Shani and D Mouair (eds), Research in Organizational Change and Development, Proceedings of
the Academy of Management, Volume 2, 47-82, 2016 (forthcoming)
  • Chapter
    For 30 years the series, Research in Organizational Change and Development (ROCD) has provided an extensive range of scholarly research and philosophical reflections on the field of organization development and change (ODC). On the occasion of the 30th anniversary of the first volume, this chapter poses the question as to how we might learn about the philosophy of ODC research from the 24 published volumes. Taking the author's explicit pursuit of the question as a process of interiority, it invites readers to engage with the question themselves and thereby enact interiority within ODC itself.
  • ... Action research work is often used as a vessel for the discovery of what may be unspeakable in conflicts and disagreements or for bridging separations as well as for birth- ing and nurturing never before thought of ideas, innovations, joint ventures and exper- imentations. Thus, refinement and reframing of the research question are processes that need to be an integral part of the process to be managed with the utmost rigour and care (MacIntosh, Bartunek, Bhatt, & MacLean, 2016). ...
    Article
    Lewin’s contribution to social science is extensive. This paper takes Lewin’s notion of defining the field at a given time and brings it into the arena of action research to make explicit links between attending and inquiring in the present tense and the collaborative dynamics of action research within the social space of a change process. Inquiring in the present tense in the field as it exists at a given time has hitherto not been explored explicitly. The paper highlights the unfolding and emergent nature of action research in terms of the inquiry process of first-, second- and third-person practices as they take place in the present tense. It advances a framework that can guide the action researcher in the practice of action research that enhances the integration of present tense into the collaborative effort. In doing so, it extends our knowledge of the richness of the Lewin legacy and elaborates a neglected and necessary mechanism in the field of organization development and action research.
  • Article
    Full-text available
    Dynamic capabilities research aims to explain how firms manage to change in order to sustain their competitive advantage. Yet, despite considerable progress, dynamic capabilities theorizing struggles to capture novelty, which lies at the heart of change. We argue that this is caused by inherent limitations of the ways in which human action has been conceptualized: dynamic capabilities theorizing to date is based on rational or normative notions of actions, neither of which account for novelty in action. Hence, we propose that to overcome this impasse dynamic capabilities research needs to incorporate a concept of creative action into its theoretical apparatus. We elaborate on the positioning of creative action relative to existing dynamic capabilities theorizing and outline its implications for future dynamic capabilities research.
  • Article
    The purpose of the paper is to present a framework that enables action researchers to create quality action research projects within the organization development (OD) domain using the broad criteria of being rigorous, reflective and relevant and so contribute to the realm of practical knowing. What constitutes good quality action research within OD is a difficult question, given the broad range of approaches that operates in a wide variety of settings and with great diversity. It advances specific dimensions by which action researchers can create, review and assess quality in action research work. This integrative framework and criteria are practical tools to enable action researchers to create quality action research in OD.
  • Article
    The linear view of innovation in agricultural contexts is being replaced by systems approaches. Agricultural producers are seen as important actors rather than merely consumers of the technologies that are generated by agricultural research and transferred by education and extension services for subsequent adoption. This new understanding of innovation for sustainable agriculture is however not reflected within the European policy framework, so learning and innovation networks for sustainable agriculture (LINSA) have formed in response. The support needs of these innovation networks are poorly understood which hampers their integration into policy. A participatory action research approach – the Reflective Learning Methodology was developed and applied to address the challenge of up-scaling from local learning and innovation networks to a regional, in this case European, support framework for innovation in sustainable agriculture. We examine the Reflective Learning Methodology by describing and analysing its application in a European level action research project. This approach involves identifying some of the challenges of transdisciplinary research and finding ways of addressing them and critically reflecting on the role of the researcher in participatory action research. With the Reflective Learning Methodology, we have developed a tool, which has the potential to be adapted to research projects in other contexts.
  • Article
    Full-text available
    How do we, as management researchers, develop novel theoretical contributions and, thereby, potentially break new ground in management studies? To address this question, we review previous methodological work on theorizing and advance a typology of the reasoning processes that underlie theoretical contributions and significant advances in management studies. This typology consists of various types of analogical and counterfactual reasoning, ranging from focused thought experiments aimed at prodding existing theory in the direction of alternative assumptions, constructs, and hypotheses to more expansive efforts for inducing new theoretical models and alternative explanations. Applying this typology, we detail the mechanisms behind the formation of novel theoretical contributions and illustrate the currency of our typology through a review of 24 major theoretical breakthroughs in management studies. We conclude the paper by discussing the implications of this typology for our collective efforts in building, elaborating, and expanding theory in management studies.
  • Article
    Science on emerging environmental health threats involves numerous ethical concerns related to scientific uncertainty about conducting, interpreting, communicating, and acting upon research findings, but the connections between ethical decision making and scientific uncertainty are under-studied in sociology. Under conditions of scientific uncertainty, researcher conduct is not fully prescribed by formal ethical codes of conduct, increasing the importance of ethical reflection by researchers, conflicts over research conduct, and reliance on informal ethical standards. This paper draws on in-depth interviews with scientists, regulators, activists, industry representatives, and fire safety experts to explore ethical considerations of moments of uncertainty using a case study of flame retardants, chemicals widely used in consumer products with potential negative health and environmental impacts. We focus on the uncertainty that arises in measuring people's exposure to these chemicals through testing of their personal environments or bodies. We identify four sources of ethical concerns relevant to scientific uncertainty: 1) choosing research questions or methods, 2) interpreting scientific results, 3) communicating results to multiple publics, and 4) applying results for policy-making. This research offers lessons about professional conduct under conditions of uncertainty, ethical research practice, democratization of scientific knowledge, and science's impact on policy.
  • Article
    Despite the huge increase in the number of management articles published during the three last decades, there is a serious shortage of high‐impact research in management studies. We contend that a primary reason behind this paradoxical shortage is the near total dominance of incremental gap‐spotting research in management. This domination is even more paradoxical as it is well known that gap‐spotting rarely leads to influential theories. We identify three broad and interacting key drivers behind this double paradox: institutional conditions, professional norms, and researchers' identity constructions. We discuss how specific changes in these drivers can reduce the shortage of influential management theories. We also point to two methodologies that may encourage and facilitate more innovative and imaginative research and revisions of academic norms and identities.
  • Article
    The reflective and interrogative processes required for developing effective qualitative research questions can give shape and direction to a study in ways that are often underestimated. Good research questions do not necessarily produce good research, but poorly conceived or constructed questions will likely create problems that affect all subsequent stages of a study. In qualitative studies, the ongoing process of questioning is an integral part of understanding the unfolding lives and perspectives of others. This article addresses both the development of initial research questions and how the processes of generating and refining questions are critical to the shaping of a qualitative study.
  • Article
    In the context of management and organization studies, the potential of action research for generating robust actionable knowledge has not been yet realized. While there are historical reasons for this with roots in different philosophies of social science, there are areas of common ground that may be explored fruitfully. This paper works from the insight that there are two key perspectives on action research: one from inside the action research community and one from outside. It explores how action research is a form of science in the realm of practical knowing and that this perspective provides an insight into how the views of action research from inside and from outside may be engaged. It proposes a general empirical method and the notion of interiority, based on the operations of human knowing, with a focus on how we know, rather than on what we know, as a synthesis, whereby the two perspectives on action research may be engaged.
  • Article
    Action research has become increasingly prominent among management researchers as an espoused paradigm used to justify the validity of a range of research outputs. In this paper we introduce and discuss 12 contentions which, we argue, justify an action research project as quality research. The contentions are presented through a discussion of a number of important issues: generality and theory generation, the type of theory development appropriate to action research, the pragmatic focus of action research, designing action research and validity of action research.
  • Article
    Much organizational activity and academic research relies on the accuracy of managers' perceptions. However, few studies have assessed the accuracy of managerial perceptions, and these studies indicate that managers' perceptions are often very inaccurate. This article discusses an odyssey into the study of managerial perceptions spanning two decades and two empirical studies. It depicts the evolution of research questions, samples, study designs, problems with such research and inferences drawn. It also identifies some errors that tend to be especially large and suggests some corrective actions. These corrective actions include using education and training to inform managers about organizational and environmental properties, exploiting improved technology, helping organizations to identify and correct misperceptions and designing robust organizations that can tolerate misperceptions.