ArticlePDF Available

Roxburgh 2018 - On Elegant Design

Authors:
  • Oikoi Pty Ltd
Article

Roxburgh 2018 - On Elegant Design

Abstract

Research design is arguably the most important and genuinely powerful skill a scientist can have. It is here, where problems arising in the real world can be addressed. And yet instead of occupying the primary focus of our working lives, research design is too often left on autopilot. This opinion article discusses the factors driving a overlooking of good design in modern research, and proposes several simple principles that ought to be used to anchor researchers in their study design. Particular attention is paid to applying the 'so what?' test to avoid collecting unnecessary data and thereby overcomplicating analysis and wasting valuable time and resources.
44|Agricultural Science
I
truly believe that there can be beauty in
good research design.
Both of my parents are artists. I grew up in a household
where it was common to debate for hours, or even
genuinely fight, over how a picture ought to be hung
on a wall. My childhood involved countless nights at
gallery openings which taught me to associate the word
‘exhibition’ with extreme and inescapable boredom. But
despite my early misgivings, over time I also came to
believe that good design can make anything beautiful.
Be it a car, an app for your phone, or a work of art, good
design can bring us joy regardless of its form. To behold
something that has been properly designed, when it
is simple, effective, and robust, is to appreciate true
creative elegance. And to me the word ‘elegant’ is what
best articulates the essence of good design.
And this brings me to research design. It is arguably the
most important and genuinely powerful skill a scientist
can have. It is here, where problems arising in the real
world can be addressed. These problems almost always
take the form of questions without answers. If we ask the
right questions and using deliberate, considered scientific
inquiry to gather evidence and analyse it, we can provide
answers that did not previously exist. This is the source
of the power and prestige of science. It’s the very heart
of what we do. And yet instead of occupying the primary
focus of our working lives, research design is too often
left on autopilot. In either the excitement or the ‘business’
of starting a new project, the nuances of design are often
lost. Instead, studies are haphazardly thrown together,
accommodating random and sometimes irrelevant ideas
or methodology. These are often included because of
the preferences of powerful or influential groups (such as
our institutions, journals, funding bodies or even beloved
colleagues). Too often, a project or individual study will
include a subject of inquiry simply to satisfy the funding
body’s pet interest. At other times, questions are thrown
into surveys ‘just in case’ or because it ‘might be useful’.
These kinds of vague decisions during research design
ON ELEGANT DESIGN
Author: Caspar Roxburgh, Research & Storytelling Director
AgImpact International, Freshwater NSW 2096 caspar@agimpact.org
Photograph courtesy of Greg Bortolussi
www.aginstitute.com.au| 45
have contributed to enormous amounts of data being
collected (at great expense). As we all know, this data
mostly ends up abandoned on hard-drives never to see
the light of day. We’ve all written papers or reports where
it feels like only 10% of the study data is actually being
included in the results. I believe our practices around
research design are partly to blame for that trend.
HOW DID THIS HAPPEN?
During my natural sciences education, I was never
explicitly taught research design (the social sciences
are another story). I have not encountered or heard of
university courses in scientific design at an undergraduate
level. Along with writing (which I’ve dealt with previously),
design is a critical skill that has somehow been left
for bachelor-level students to pick up tacitly. Young
scientists-in-making are expected to learn design
principles from their subtle underlying presence in
courses that outline historical scientific discovery and
force feed pre-prepared lab experiments. Even for those
who have completed design courses, the lessons seem to
be lost in practice.
I have a suspicion that research design somehow
dropped off undergraduate curricula because it was seen
as less interesting, less gripping and more difficult than
applied learning through running pre-designed soils,
chemistry or animal experiments. But I would counter that
research design is actually the most exciting, creative
and powerful aspect of science. It is in research design
that we apply the scientific method. It is in design that we
can begin to see how scientific skills can help us solve
problems that we don’t even know we have. It is where we
learn about the greatest strength of science. And I worry
that we’ve failed to impress this great power upon my own
generation of scientists.
WHAT TO DO WHEN NO ONE IS TELLING YOU
WHAT TO DO
Good research design comes down to confidence.
Confidence that the question you are answering has
value. Confidence that even though your research
questions sits within a complex system, you don’t have
to chase the entire story to make it relevant or useful. In
attempting to find that confidence, I would argue for three
important principles in designing a research project that
can be applied universally:
Know what problem your research is aiming to solve.
Determine what question your research can answer
that would solve the problem
Clearly and precisely articulate what must be
measured, the format this measurement will need to
take and the analysis that must be completed on it to
answer your question.
KNOWING THE PROBLEM
Your research project cannot, and will not, save the
world. In research design, we must be honest about
what can, and cannot, be achieved. This means ensuring
that the problem we are setting out to solve can actually
be overcome through a research activity. This usually
means at its heart, it must have a question that needs
to be answered. If the solution to the problem involves
significant non-research activities, it cannot be solved
through research alone and is an improper goal for
designing a project or activity. For example, a research
project cannot ‘lift one million small landholders out of
extreme poverty’. Research alone cannot increase the
profitability of cotton farms by 10%. A scientist isn’t able
to ‘ensure a stable supply of agricultural inputs’ in Africa.
Among other things, such achievements would be likely
to require education programs, adoption of new practices
by farmers, and even changes in government policy.
Researchers do not do this work. And designing our
research using abstract and unachievable goals leads to
vague plans and lack of direction for individual studies.
During the design, more thought needs to go into asking
who will use the information and whether the findings
are actionable. Not all research is actionable, but the
question should always be asked.
KNOWING THE QUESTION
Research is fundamentally about answering questions.
Everything we do is focused on testing the validity of
falsifiable statements (i.e. hypotheses). At its core, this
will always embody a question – is the following statement
true? Of all research design principles, the centrality of
questions to guide our activities is probably the most
obvious. I would think most researchers are familiar
with its place in design. But while research questions
may be common, they are not being appropriately
wielded to ensure data collection is narrowly focused on
delivering core results. Thanks to research conventions,
researchers’ own curiosity and pressure from our
funders, there are strong incentives to add in data
collection that does not address core research questions.
Understandably, we often yield to this pressure. And it is
for this reason that we routinely find ourselves collecting
so much unnecessary data. The solution to this problem
is to look at every piece of data you plan to collect in a
study and ask yourself: ‘Which research question is this
46|Agricultural Science
data going to answer and how will I use it to do so?’. If the
answer isn’t immediately obvious, then the data most likely
aren’t necessary. In my experience, asking teams to justify
their planned data collection can be an extremely difficult
process. But research questions must be our guides. We
must make use of them to determine the scope of our data
collection. By doing this we can save valuable resources
and redirect them to other pressing needs.
SPECIFICS MATTER: KNOWING EXACTLY WHAT
YOU INTEND TO MEASURE
The third principle is one that comes up often in our work
designing and building digital tools for data collection.
Unlike paper and pen data collection, using digital
systems means the format of your data is locked in before
you begin fieldwork. This means if you aren’t clear on
what you are trying to measure or how you are planning
on analysing it, you may end up with useless data. For
example, a classic survey question often included in
a design brief might be ‘how many people live in your
house?’. But a researcher might include this question to
measure the number of family members, the number of
people working on a farm, or the number of people being
supported by a farming enterprise. The problem here is
that the question ‘how many people live in your house’
doesn’t tell you any of those things. In other words, the
specifics matter. By knowing precisely what a variable
is supposed to represent, and how it will be used in the
analysis, survey questions (or measurement methodology)
can be properly designed to make sure the data will allow
you to answer your research question. You’d be surprised
how many researchers I’ve worked with have no sense
of what their data is going to do, and when pressed for a
specific intention for measurement they come up short.
This is the other way that unnecessary data collection
creeps into our work.
THE ‘SO WHAT’ OF RESEARCH AND DATA
COLLECTION
What I have been describing in this article is applying the
‘so what’ test to our research and our data. Because we
fail to take this test seriously in our designs, we collect
far too many data that we do not need. We often have no
clear plan on how we will make use of these data. And
I believe we waste enormous time and resources in this
process. The unnecessary work we do causes confusion
during analysis, and misplaces researcher attention and
effort in rationalising the unnecessary activities. We find
ourselves doing this because we rely on instinct and
convention to design our research instruments for data
collection instead of the principles I outline.
To break out of this habit, my team and I always begin
a research project design by asking ourselves: i) What
problem are we going to solve? ii) What question will we
answer to solve it? and iii) How we are going to use our
data to answer the question? And if there are no obvious
responses to these questions, we begin to reconsider
the value or validity of the entire research project. As
researchers we should have the integrity and confidence
to say ‘no, this isn’t the right question’. To say ‘no, that
isn’t addressing the issue’. While saying ‘no’ in the short
term can make for difficult conversations, it ensures our
work has much greater impact in long term. And very
occasionally, when we really work hard and are even a
little lucky, we design something that I can describe as
elegant.
...I would counter that research design is actually the most exciting,
creative and powerful aspect of science.
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.