Content uploaded by Oscar Montes de Oca
Author content
All content in this area was uploaded by Oscar Montes de Oca on May 13, 2021
Content may be subject to copyright.
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 1
Is it the model or is it the process of using it? Extension officers
evaluate ADOPT as a tool to assist planning in the pastoral
sector
Oscar Montes de Oca Munguia1,2, David J. Pannell1 & Rick Llewellyn1,3
1School of Agriculture and Environment, The University of Western Australia, Crawley WA 6009
2Scion Research, 49 Sala Street, Rotorua 3010, New Zealand
3CSIRO, Glen Osmond SA 5064
Email: oscar.montes@scionresearch.com
Abstract. Agricultural extension professionals are aware of the complexity surrounding farmers’
decisions to adopt a new technology or practice. These extension officers often need to design
strategies to improve adoption though planning processes, which are commonly run
collaboratively by expert groups and through deliberation rather than individually. Models have
been used to assist these deliberations, but it is not clear which aspects of the model or the
deliberative process are more useful for extension planning. In this study, we research how
ADOPT, a model that predicts adoption, may assist decision making in planning for agricultural
extension. In 2018, we used ADOPT in three workshops with extension officers from the pastoral
sector in New Zealand to analyse the adoption of four well-known practices in the industry. We
identified important features of the model and the process used in the workshops and asked
participants to rank their usefulness. The components were: a conceptual model of adoption, a
comparison between the predicted diffusion curve and actual uptake, a sensitivity analysis of
the results, and a structured discussion around these components. We found that using ADOPT
changed participants’ perceptions on the feasibility of forecasting adoption. We also found that
participants believe the process of discussing and using ADOPT was just as important, or more
important, than the model’s results.
Keywords: Technology diffusion, agricultural extension, adoption, modelling, pastoral farming,
ADOPT.
Introduction
Planning for extension in agriculture requires consideration of the complex nature of the process
via which farmers adopt new practices. In this process, many elements dynamically interact with
each other over time and future conditions are uncertain. Interacting elements include the
population of potential adopters, the technology or practice in question and the external context
in which adoption takes place (e.g. biophysical, economic and regulatory conditions). These
factors make each adoption case unique.
Agricultural extension agents are aware that a farmer’s decision to adopt technologies and
practices in agriculture is not as simple as choosing the option offering more economic benefits
(Vanclay 2011). According to Nicholson et al. (2015, p. 1), farm decision-making consists of
'…choosing a path that provides a farming business with acceptable reward for acceptable effort
at an acceptable amount of risk'. For example, some farmers are willing to accept lower profits in
order to maintain a lifestyle and production system that suits their goals and values, while others
might be willing to operate at a high level of risk in order to maximise returns while knowing they
are foregoing other opportunities. Moreover, agriculture depends on biology and climate, so its
performance can be affected by factors beyond operator control. Nicholson et al. (2015) point out
that agriculture operates in one of the most challenging business environments due to the
combination of volatile production and prices. Kaine et al. (2011) support this idea and propose
that landowners normally configure resources with technologies and practices to realise family
and business objectives while managing exogenous constraints. The interdependence of all those
elements imposes restrictions on how farmers can respond to opportunities and constraints and
therefore can influence their decisions on whether to adopt new practices.
On the other hand, one of the aims of extension is to increase and/or accelerate the adoption of
beneficial technologies and practices amongst a target population of potential adopters. The
design of the extension strategy used is often developed in a planning process, which commonly
is done collaboratively rather than individually. Collaborative planning is thus an iterative process
where decisions emerge from discussions amongst a designated group rather than from
individuals working alone. Planning theorists argue that groups are better than individuals at
assessing and choosing amongst options for the future (Innes & Booher 1999). They propose that
the main advantage of group discussions is having the opportunity to expose the group members
to the unique knowledge of each participant about aspects of a problem that they understand
better than anybody else does.
However, due to the complexity of adoption decisions, it is difficult to achieve a common
understanding of what drives adoption, especially when individual knowledge and beliefs differ
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
2 http://www.apen.org.au/rural-extension-and-innovation-systems-journal
substantially from one another (Pannell & Claassen 2020). Wilkinson (2011, p. 47) has suggested
that '…the word "adoption" is so entrenched in the language that everyone who uses the word
thinks they know what is meant by it, but that its interpretation varies. This lack of common
understanding is well documented in the adoption literature (Wauters & Mathijs 2014; Liu et al.
2018) and it can potentially make it difficult for experts to undertake effective planning sessions.
Models of various types have been used successfully to understand complexity and assist decision
making. It has been argued that, when making decisions, the more complex the problem, the
greater the potential benefits of a model (Walker 2000; Vicsek 2002; Epstein 2008; Veldkamp
2009). In order to assist with planning for agricultural extension, a model of adoption needs to
logically reflect the realities of landowners making a choice about adoption (Nicholson et al. 2015)
by considering the interplay between rewards, effort and risk, in terms of both the decision
makers’ preferences and the characteristics of the practice itself (Montes de Oca Munguia &
Llewellyn 2020). The model also needs to clearly identify drivers related to potential adopters and
the practice itself, distinguishing between drivers that are at the disposal of participants to design
extension interventions and drivers that are outside their control. Finally, the model needs to
show the effects that intervention strategies can have in the system under certain contexts.
There are many examples in the agriculture and natural resource management literature showing
the use of different modelling approaches in different types of deliberative processes. Models
include: economic models (e.g. Rosegrant et al. 2002), information flow models (e.g. Fountas et
al. 2006), integrated models (e.g. Antle et al. 2014; Kuehne et al. 2017), and agent-based models
(e.g. Laciana & Oteiza-Aguirre 2014; Schreinemachers & Berger 2011). Collaborative processes
include ‘robust decision-making’ (Lempert et al. 2006; Haasnoot et al. 2013; Kalra et al. 2014;
Maier et al. 2016), ‘theory of change’ approaches (Prinsen & Nijhof 2015; Allen et al. 2017;
Douthwaite & Hoffecker 2017; Thornton et al. 2017) and more recent efforts to develop tools to
facilitate planning for the scaling out of innovations in complex developing country scenarios (e.g.
Sartas et al. 2020).
The literature is not clear in defining what a ‘good deliberative process’ is for extension planning
and the contributing role that models may play. Using a model in group settings requires a degree
of compromise between the individuals’ perception of how the system works and the simplified
representation of the system used in the deliberation. It is therefore inevitable that this would
cause a degree of scepticism that will affect the individuals’ perception of the model’s results.
Nevertheless, participants who are open to the use of models in their deliberations often do it
pragmatically, or as the common aphorism attributed to the statistician George Box states, they
may embrace the view that: 'all models are wrong, but some models are useful' (Box & Lucefio
1998).
The objective of this paper is to improve our understanding of the usefulness of models in planning
for extension, and whether participants perceive the model or the process of using it to be more
useful. There is already evidence in other fields that the successful use of models depends on a
good deliberative process (Jakku & Thorburn 2010), and that '...A good process can survive a bad
model, but a bad process isn’t helped by a good model' (Lempert 2015). Furthermore, Phillips &
Linstone (2016) suggest that the real objective of using models in planning, especially forecasting
models, is not necessarily to be 'right', but to '… help us be better prepared to understand the
range of possibilities and react with flexibility and resilience to future events', and that '…the most
precise forecast is not necessarily the most useful forecast' (p. 163).
In general terms, we considered that the usefulness of a model and a deliberative process were
based on their ability to generate focused technical discussions, as proposed by Forester (1999).
Usefulness thus depends on facilitating detailed, focused technical arguments amongst
participants about the range of options at their disposal to design extension interventions, to
assess the potential performance of these interventions under a specific context, and to
methodically analyse the uncertainty surrounding drivers and their effects on adoption. Thus, we
consider an adoption model to be useful for extension if it can improve participants’ understanding
of the complex environment in which they operate by: a) illustrating how the system works and
identifying key driving forces, b) quantitatively predicting the outcomes from the system in a
particular context, and c) methodically analysing the uncertainty surrounding drivers and their
effects on outcomes.
It is also not clear in the literature if there are specific aspects of a model that can be identified
as being more useful than others. For this research, we identified three model components that
were evaluated independently and are aligned to one of the three points above: a) the model’s
specifications of drivers and their causal relationships (e.g. functional form), b) the model’s output
(e.g. diffusion curve), and c) the outputs’ sensitivity analysis (e.g. scenario evaluation) (Kalra et
al. 2014).
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 3
ADOPT
For this research, we use the ADOPT model (Adoption and Diffusion Outcome Prediction Tool),
developed by Kuehne et al. (2017), in a case study with professional extension specialists working
in the pastoral sector in New Zealand.
ADOPT was selected because we consider it offers four features that can be evaluated separately
in terms of their contribution to effective deliberation: ADOPT includes a conceptual framework
that can be used to illustrate how the system works identifying key driving forces; ADOPT
produces a predicted diffusion curve; ADOPT includes a sensitivity analysis to methodically
analyse the uncertainty surrounding drivers and their effects; and it includes a structured process
that can be used by groups for deliberative design.
The process of using ADOPT consists of three steps. These steps include: first, presenting and
discussing the conceptual model; second, methodically answering a sequence of questions to
define the model’s inputs – this step requires the group to discuss each question, understand its
relevance and reach consensus for each answer; and third, discussing the effects of different
variables on the model’s output using sensitivity analysis.
Most importantly, ADOPT is one of the few tools available that explicitly provides ex-ante adoption
analysis in agriculture. We consider these features can generate the level of technical enquiry
suggested by Forester (1999) for planners to 'assess practically, comparatively and prescriptively
a range of viable options at their disposal'. Given its features, ADOPT may be able to facilitate
discussions about the potential effects of drivers on the adoption of a technology or practice,
where experts in different fields are encouraged to make contributions throughout the process,
as suggested by Innes & Booher (1999).
Methods
Workshops
Three separate workshops were conducted in 2018 with 34 professionals representing a range of
organisations involved in research and extension in the pastoral sector in New Zealand after
human research ethics approval was granted by the University of Western Australia. These
participants have extensive knowledge of the population of pastoral farmers in New Zealand, and
some were also experts on the specific technologies used in this research. Pastoral farming
dominates the rural agribusiness sector in New Zealand, more specifically 'sheep and beef'
farming (producing meat and fibre) and dairy cattle farming. Technology has always played an
important role in the sector. Organisations represented in the workshops were: Dairy NZ (an
industry organisation), the Red Meat Profit Partnership (a research programme), AgResearch (an
agricultural research company), the Ministry for Primary Industries, the Alliance Group (a meat
processing company), Beef & Lamb New Zealand (an industry organisation), and Lincoln
University.
The number of participants for each workshop were 18, 10 and 6, with no repetition. The first two
workshops were attended by dairy industry specialists and the third workshop was attended by
sheep and beef farming specialists.
Extension officers were asked to complete a questionnaire before their workshop and one at the
end, to detect shifts in their perceptions about using models before and after their participation
(Montes de Oca Munguia 2020). Reponses were summarised as a group and used to define
statistical models. The pre-workshop questionnaire was sent alongside the invitation to participate
two weeks prior to the workshop. This questionnaire included questions regarding the participant’s
area or work, their perception of the usefulness and the feasibility of predicting adoption, and
their opinion on the importance of several drivers of adoption to be considered while thinking
about adoption.
The post-workshop questionnaire included questions about their perceptions on the usefulness of
the group discussion generated in the workshop and different components of ADOPT. Participants
were also asked whether their participation in the workshop had changed their opinion about the
ability of a model to forecast adoption and whether their opinion on the importance of different
adoption factors had changed from their original perceptions. Finally, they were asked what
actions they were likely to take in relation to predicting adoption after the workshop. A total of
31 participants completed the pre-workshop questionnaire. Of those, 24 completed the post-
workshop questionnaire.
Participants were made aware that the objective of the workshop was to use their expert opinion
to evaluate the use of ADOPT for extension planning using an example of a well-known technology
available to pastoral farmers in New Zealand. We selected familiar practices as a reference point
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
4 http://www.apen.org.au/rural-extension-and-innovation-systems-journal
rather than a new practice so that participants would be more confident in assessing the model’s
output. We clarified that the intention of the workshop was not to validate the model results
against measured adoption. Current adoption of these practices was estimated using the results
of a survey of pastoral farmers in New Zealand (Montes de Oca Munguia 2020). Each group was
asked to select an example from four available options:
Use of body condition scoring. The assessment of Body Condition Scoring (BCS) is used to
estimate body fat reserves in both cows (visual) and ewes (feeling backbone with fingers and
thumb). BCS is used as a management tool to determine feed requirements and improve
reproductive performance.
Use of pasture management software. Use of computers, tablets and smartphone apps or
programmes to calculate feed demand, feed availability and feed quality for sheep and cattle
at any time of the year and for different levels of production. Information can be used for both
tactical and strategic decisions.
Use of Plantain and/or Lucerne for summer grazing. Plantain and Lucerne are used to increase
the amount and quality of summer feed in grazing systems. Plantain is more often used as a
pasture mix, but it can also be used as a special purpose crop, lasting 2-3 years. Lucerne is
used on soils with low soil moisture holding capacity to increase production.
Use of a formal, audited nutrient management plan. Used to actively manage nutrients (N and
P) on the farm in a formal, audited way. They can be developed in conjunction with a fertiliser
consultant or as part of an environmental plan developed by industry or local government.
Nutrient management may include managing the type, placement and timing of fertiliser
applications; crop rotations; precision application; and stock exclusion from waterways.
All practices have been available in the industry for at least 20 years, but they all showed different
levels of current adoption by the time the workshops took place. The survey found body condition
scoring was the most widespread practice amongst surveyed farmers, reaching 74% adoption.
This was followed by using Plantain and Lucerne for summer grazing (56%), the use of pastoral
management software (45%) and the use of nutrient management plans (38%).
Participants at each workshop were asked to select the practice they were more familiar with or
more interested in. The use of formal, audited nutrient management plans amongst dairy farmers
was used as the example for the first workshop. The use of pasture management software
amongst dairy farmers was used as the example for the second workshop, and the use of a formal,
audited nutrient management plans amongst sheep and beef farmers was used as the example
for the third workshop.
The workshops consisted of three facilitated steps:
A presentation and discussion about ADOPT’s conceptual model, covering the model’s
specifications of drivers and their causal relationships.
Running ADOPT on-line to produce a diffusion curve and compare it with the survey’s measured
adoption for the selected practice.
An exercise to use ADOPT’s sensitivity analysis to adjust the initial forecast to be closer to the
survey’s measured adoption curve, discussing the assumptions behind observed discrepancies.
Conceptual model discussion
The ADOPT conceptual model shown in Figure 1 was presented by the facilitator and discussed by
the group. The presentation also covered the mechanics of using ADOPT in the workshop:
participants were required to discuss and agree on the answers to 22 questions about the
numbered variables in the four quadrants represented in Figure 1. In ADOPT, responses are used
in mathematical functions to predict time to peak adoption and peak adoption level using cause-
effect relationships for each relationship shown in the conceptual model.
The first high-level driver of adoption in the ADOPT model is relative advantage. Theories related
to relative advantage for modelling are well supported in literature and include subjective
expected utility theory (Tversky & Kahneman 1981), prospect theory (Laibson & Zeckhauser
1998), and multi-attribute utility theory (Huang et al. 2011). The application of these theories in
models of decision-making have been reviewed extensively (e.g. Behzadian et al. 2010). The use
of different population orientations (i.e. preferences) to weight characteristics of the practice when
calculating overall relative advantage in the conceptual model illustrates the interplay between
rewards, effort and risk in decision making in agriculture, as proposed by Nicholson et al. (2015).
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 5
Figure 1. The ADOPT conceptual model
Source: Kuehne et al. (2017)
The second high-level driver of adoption in the ADOPT model is learning, which is a key focus for
extension professionals. The ADOPT conceptual model identifies cause-effect relationships
affecting the likelihood of adoption and the time lag from the availability of the innovation to the
decision to adopt it in three stages as suggested by Lindner et al. (1982). They are: first, the
discovery stage – the time it takes for the producer to be aware of the existence of the innovation;
second, the evaluation stage – the time from awareness to first use, on a trial basis; and third,
the trial stage – the time from the initiation of trial use to the acceptance of the innovation. Thus
it covers the stages of awareness, trialling and adoption as outlined by Pannell et al. (2006).
ADOPT results
The process of using ADOPT in all workshops consisted of running the on-line version of ADOPT
and having a 1 to 3-minute group discussion about each of the 22 questions until agreement was
reached on an answer. ADOPT results were then presented, compared with the survey results,
and discussed.
Figure 2 shows an example of the ADOPT questions participants were asked to answer. The
question and its explanation are displayed on the top-right. After group discussion, participants
agreed on an answer (bottom-right) and moved on to the next question. The list of variables and
their correspondence of each variable to a quadrant in the conceptual model is displayed in the
list on the left-hand side of the screen. A printout of the conceptual model (Figure 1) remained
visible to participants throughout the process, so the facilitator could point out the links between
each numbered question and their relationships with other variables.
Table 1 shows, for each workshop, the estimated current uptake level and average time to adopt
from the survey and the ADOPT outputs resulting from the group’s discussion. The table shows
differences between each group’s predictions and the survey estimates of current adoption levels.
Participants were reminded that the intention of the comparison was not for validation purposes,
but rather to reflect on the characteristics of the practice and the population that might be behind
the differences.
Workshop 1 participants produced a peak level prediction relatively close to the estimated current
uptake level, but an unusually long time to peak. In contrast, workshop 2 participants produced
a prediction of the time to reach peak adoption close to estimates of current adoption times but
predicted peak adoption levels were much higher than current uptake levels. Predictions
generated by workshop 3 participants were much higher than current adoption levels and involved
a much longer time to reach the predicted peak than the average time to adoption experienced
by adopters so far.
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
6 http://www.apen.org.au/rural-extension-and-innovation-systems-journal
Figure 2. Example of on-line ADOPT used in Workshop 2
Table 1. Simulated results in relation to current levels of uptake from a recent survey
Workshop
Practice Measured
current
adoption
level from
survey
Average time
to adoption
estimated
from survey
Predicted
peak adoption
level from
participants’
ADOPT run
Predicted time
to near-peak
adoption from
participants’
ADOPT run
1 Audited nutrient
management plans
38% 6 years 59% 21 years
2 Pasture management
software
45% 12 years 94% 13 years
3 Audited nutrient
management plans
38% 6 years 93% 13 years
Sensitivity analysis
The last step in the workshop was an exercise to allow adjustment of participants’ inputs, informed
by ADOPT’s sensitivity analysis and the predictions generated by the initial inputs. The purpose
of this exercise was to allow participants to identify the variables that, if adjusted, would have
the most impact on the outputs. Participants were encouraged to explore different combinations
of variables that would adjust their initial prediction and discuss the assumptions behind their
answers. In this case, the sensitivity analysis identifies the variables that have the most effect on
the model’s predictions of peak level of adoption and time to peak adoption. Figure 3 shows
ADOPT’s sensitivity analysis for Workshop 2.
The horizontal axis shows the question numbers, which correspond to each numbered variable in
the conceptual model. In each graph, the vertical axis shows the step changes that occur in each
output by changing the answer to the question one level up or down. The change in peak adoption
level occurs in terms of adoption percentage and the time to peak adoption is measured in number
of years.
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 7
Figure 3. ADOPT’s sensitivity analysis diagram used to analyse the use of pasture
management software in Workshop 2
Sensitivity analysis allowed participants to make separate adjustments to both their prediction of
peak adoption levels and time to reach peak adoption by exploring different sets of variables (as
illustrated in the conceptual model – Figure 1). For example, participants in workshop 2 aimed at
reducing their predicted peak adoption level while retaining their predicted time to reach peak
adoption. Figure 3 shows that the most sensitive variable for changing the predicted peak adoption
level on the use of pasture management software was the level of profit benefits in years that the
practice is used (question 16), followed by profit benefits in the future (question 17) and
environmental costs and benefits (question 19). Figure 3 shows that, for example, adjusting the
answer to question 16 one level down would reduce the prediction by 21%. In their example, this
would adjust their initial peak adoption prediction from 94% to 73%. Participants agreed to reduce
this variable by one level. After making this adjustment, the process was repeated to decide
whether to make more adjustments to the same variable or adjust other variables, making
constant references to the conceptual model. Table 2 shows, for each workshop, the variables
that were adjusted from the initial run and the resulting adjusted predictions.
The wrap-up discussion after the sensitivity analysis centred on the feasibility of predicting
adoption, the general use of prediction models in extension and the ‘strengths and weaknesses’
of using ADOPT for extension planning. There was discussion about factors that affect the diffusion
of innovations that are not only outside the individual farmer’s control but also outside the scope
of any model. For example, participants highlighted the role of impending legislation in the future
uptake of audited environmental farm management plans. Experts observed that the ‘pressure to
comply’ amongst the farming community has increased recently, and therefore peak adoption
level in a voluntary basis has not been reached for this practice yet. Participants did not feel this
pressure could be captured explicitly in the relative advantage equation.
Table 2. Comparison between initial and adjusted ADOPT predictions for each workshop
Workshop 1 Workshop 2 Workshop 3
Predictions Initial Adjusted Initial Adjusted Initial Adjusted
Peak adoption 59% 48% 94% 66% 93% 35%
Time to near peak adoption 21 years 10 years 13 years 13 years 13 years 12 years
ADOPT answers
1. Profit orientation a minority almost all a minority
2. Environmental orientation about half a minority a majority
3. Risk orientation a majority a minority a majority
4. Enterprise scale almost all almost all almost all
5. Management horizon a majority almost none a majority
6. Short-term constraints almost all almost all almost none
7. Trialing ease moderately easy easily easily moderately
difficult
difficult
8. Innovation complexity very difficult slightly difficult difficult difficult moderately
difficult
9. Observability moderately easily difficult difficult moderately
difficult
10. Advisory support a minority about half about half a majority
11. Group involvement a minority a minority a minority
12. Relevant existing skills & knowledge almost all a majority about half a majority
13. Innovation awareness a minority about half almost all a majority
14. Relative upfront cost of innovation moderate minor large
15. Reversibility of innovation very easily very easily easily
16. Profit benefit in years that it is used small profit
advantage
moderate small moderate small
17. Profit benefit in future moderate profit ad
moderate small small
18. Time for future profit benefits to be realized 3 to 5 years 6 to 10 years 1 -2 years 3 to 5 years 6 to 10 years
19. Environmental impact small no net large small
20. Time for environmental impacts to be realized 3 to 5 years not applicable 1 to 2
21. Risk small reduction small moderate
22. Ease and convenience no change small increase small decrease
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 9
Some participants suggested that ADOPT could be complemented with other models analysing
‘macro’ variables, such as bioeconomic models, but many participants felt that extension planning
should not be ‘run by models’. However, there was also discussion about the strengths of ADOPT.
Participants expressed that the model included the factors than need to be in place to assess the
‘adoptability’ of an innovation. After this discussion participants were asked to fill in the post-
workshop questionnaire and the workshop concluded.
Statistical analysis of workshop questionnaires
The frequency of responses from both pre and post-workshop questionnaires were summarised.
Due to the ordinal nature of the data and the small sample size, we grouped and reported
responses for each question (collected on a scale from 1 to 10) into three intervals: low scores
(e.g. rarely, disagree, not useful), medium scores (e.g. sometimes, moderately agree, moderately
useful) and high scores (e.g. often, agree, useful).
We also conducted non-parametric testing to explore the statistical relationships between
responses. Recognising the limitations arising from our small sample size, we consider these tests
could be used more confidently in subsequent studies with larger samples. Firstly, we conducted
a paired-samples Wilcoxon test (i.e. Wilcoxon signed-rank test) to detect a shift in the perception
of participants using the responses to four questions that were asked both before and after the
workshop. Those questions were:
In your view, is it possible to predict adoption adequately?
In your view, how important are the following factors in driving adoption:
o Characteristics of the farmer.
o Characteristics of the technology or practice.
o External factors, out of farmers’ control.
Secondly, we defined five variables based on the responses of a selection of post-workshop
questions (Table 3). The variables were used to build an ordered logit regression model to identify
the level of contribution of each ADOPT feature in the participant’s perception of usefulness of the
discussion.
Table 3. Variables used to test the usefulness of the workshop components
Question
Dependent variable How useful did you find the workshop to focus technical discussion of adoption?
Independent variables
How do you rate the contribution of the following model features explored today
to the discussion:
Diffusion s-curve
Sensitivity analysis
Conceptual model
Process of group discussion
Results
Pre-workshop responses summary
Participants were asked to identify tasks in their work that required them to think about future
adoption. Many participants were involved in the evaluation of research projects involving farmers
(35%) or in decisions to invest in new technologies or practices (38%). In addition, most
participants were directly involved in designing (88%) and implementing (74%) extension
strategies to increase adoption of beneficial practices.
Participants were asked how often they considered the likely adoption of a technology or practice
in their work. Results showed that 15% of participants seldomly considered adoption, while about
a quarter considered adoption occasionally (24%). Almost two-thirds of participants indicated
they needed to consider adoption regularly (62%).
Prior to the workshop, most participants considered that predicting the adoption of a new
technology or practice was useful. Results showed that 41% of participants considered it very
useful, 44% considered it moderately useful and 15% only slightly useful.
However, while most respondents to the pre-workshop survey considered predictions useful, 38%
did not believe it was possible to predict adoption adequately (Table 4).
When asked about the importance of the characteristics of the farmer in driving adoption, over
two-thirds of participants considered it important (71%), 24% or participants considered it
neutral, and 5% considered it not important. Participants rated the importance of the
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
10 http://www.apen.org.au/rural-extension-and-innovation-systems-journal
characteristics of the technology in driving adoption as important (81%), or neutral (19%).
Finally, opinions before the workshop were more divided amongst participants about the
importance of external factors (factors out of farmers’ control) in driving adoption. Half the group
considered them important (52%), 33% considered them neutral and 14% considered them not
important.
Post-workshop responses summary
After the workshop, participants were asked whether their participation in the workshop changed
the way they understood adoption. More than half of respondents agreed with this statement
(55%), 27% were neutral and 18% of participants disagreed. Table 4 shows the responses to
questions included in both the pre and post-workshop questionnaires. The largest changes to
participants’ perceptions following the workshop were an increase in the perception that it is
possible to predict adoption adequately, and an increase in the importance ascribed to external
factors out of farmers’ control as influences on adoption.
Table 4. Participants’ changes in opinion before and after the workshop
In your view, is it possible to predict adoption
adequately?
Disagree Neutral Agree
Pre-workshop
38% 29% 33%
Post-workshop
14% 24% 62%
In your view, how important are the following
factors in driving adoption
Not important
Neutral Important
Characteristics of the farmer Pre-workshop
5% 24% 71%
Post-workshop
5% 21% 74%
Characteristics of the technology
or practice
Pre-workshop
0% 19% 81%
Post-workshop
0% 5% 95%
External factors, out of farmer’s
control
Pre-workshop
14% 33% 52%
Post-workshop
5% 16% 79%
We also asked participants to rate the usefulness of the workshop and the different features of
ADOPT to generate focused technical discussion of adoption. All participants considered that the
workshop was useful. Table 5 shows how participants rated the usefulness of different components
we evaluated.
Table 5. Participants’ rating of the usefulness of workshop components
How do you rate the usefulness of the following
information and model features explored today
Not useful Neutral Useful
Conceptual model 0% 4% 96%
Process of group discussion 0% 0% 100%
Diffusion S-curve 8% 25% 67%
Sensitivity analysis 4% 13% 83%
All components of the workshop, including the ADOPT conceptual model, ADOPT results and the
process of using it were considered useful by most of the participants. The presentation of the
ADOPT conceptual model was considered useful by most participants and the process of group
discussion was considered useful by all participants. Regarding ADOPT outputs, most participants
considered them useful, with some participants being neutral about them and one or two
participants considered them not useful.
Finally, we asked participants what actions they were likely to take in relation to predicting
adoption after the workshop. A minority (4%) said they would take no action, 25% said they
would reassess their practice, 33% said they would change their approach or advice, and the
majority (58%) said they would seek extra information or training.
Statistical analyses
We had limited success regarding the two analyses used to explore the statistical relationships
between responses, possibly due to the low number of observations (n=24). However, the paired
samples Wilcoxon test detected a significant shift in two of the perceptions of participants listed
in Table 4. The test indicated a significant difference in the perceived ability of models to predict
adoption after the workshop (p = 0.046), and the importance of external factors (p = 0.044).
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 11
According to the test, participants’ opinion did not change significantly regarding the importance
of characteristics of farmers and the practice after the workshop. On the other hand, the
regression analysis aimed at identifying which factors from Table 5 had the most influence on the
rating of the overall usefulness of the workshop did not identify any statistically significant factors
(p<0.05).
Discussion
Was it the model or the process of using it? Our results show that all participants rated the process
of group discussion to be useful, although a minority of participants considered the model to be
not useful. Our results therefore support previous research finding that the process of using
models is considered more valuable than the model itself (Jakku & Thorburn 2010; Lempert
2015).
However, previous studies generally only focus on contrasting model results vs process. We
extended the analysis by investigating three specific aspects of a model that may influence its
usefulness for extension planning: understanding the system, forecasting adoption and evaluating
alternative outcomes. Kalra et al. (2014) identified those three areas in which parties to a decision
often do not know or cannot agree on but did not provide an evaluation of how a model can assist
each one.
In line with previous studies (e.g. Phillips & Linstone 2016), our results showed that the model’s
predicted diffusion curve (i.e. model results) was ranked by participants only as the third most
useful component. Even though ADOPT was able to generate adequate predictions of adoption,
more participants found the other two components even more valuable than the predictions per
se.
The most useful model component was the conceptual model. Participants used it to discuss how
the system works and to identify the key driving forces and relationships behind the adoptability
of an innovation, identifying how drivers affect either the likelihood of adoption, the speed of
adoption or both.
This was followed by the sensitivity analysis. Participants were able to evaluate alternative
outcomes, identifying drivers that where under their control (e.g. advisory support, improve
awareness) and those they cannot control (e.g. the priority given by the farmers to issues such
as profit, risk and the environment). They used sensitivity analysis to discuss how different
adoption strategies could work in practice.
We thus consider that this study can move the discussion of model usefulness further by offering
a more nuanced analysis of a model’s ability to generate focused technical discussions, as
proposed by Forester (1999) and reduce uncertainty in decision making (Kalra et al. 2014).
There are two points not covered in this study that could limit its findings. We did not investigate
how planning occurs in the absence of a model and we did not investigate the role of models in
participants’ learning. Regarding the second point, we believe there is an opportunity to further
understand models as learning tools. Participants in our workshops reported a change in their
understanding of adoption, suggesting that their participation was useful in deepening their
knowledge of the adoption process. They also indicated an intention to change their current
practice because of the workshop, either by reassessing their current practice, changing their
approach or advice, or seeking extra information or training.
Conclusion
The objective of this paper was to improve our understanding of the ‘usefulness’ of models in
planning for extension, and whether extension officers perceived the model or the process of
using it to be more useful. We conducted a series of workshops using ADOPT as the model to
analyse the adoption of well-known practices and asked participants in our workshops to evaluate
the model and the process of using it in assisting extension planning. Our results support previous
research finding that the process of using models is considered more valuable than the model
outputs, but we also offer a more nuanced understanding of model usefulness that goes beyond
using models as ‘black boxes’ to produce adoption projections.
In our workshops, we found that participants have pragmatic attitudes towards the use of models
in planning, even when some felt that extension planning should not be ‘run by models’. At each
workshop, there were multiple discussions about the general use of prediction models and the
‘strengths and weaknesses’ of using them for extension planning. It was concluded that ADOPT
was useful to think about the ‘adoptability’ or ‘adoption potential’ of an innovation, but that
external factors that are outside of the scope of any model should also be kept in mind when
using the results.
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
12 http://www.apen.org.au/rural-extension-and-innovation-systems-journal
Lastly, this study highlights the importance for model developers considering a full range of model
features to better assist extension planning to improve the uptake of beneficial agricultural
innovations.
Acknowledgements
We acknowledge the support of AgResearch and Landcare Research in the development of the
research underpinning this study. We also acknowledge Denise Bewsell from the Red Meat Profit
Partnership and Callum Eastwood and Ina Pinxterhuis from Dairy NZ for their valuable contribution
in the selection of technologies and practices to be included in the survey and workshop
organisation.
References
Allen W, Cruz J & Warburton B 2017, ‘How decision support systems can benefit from a theory of change
approach’, Environmental Management, vol. 59, no. 6, pp. 956–965,
https://dx.doi.org/10.1007/s00267-017-0839-y.
Antle JM, Stoorvogel JJ & Valdivia RO 2014, ‘New parsimonious simulation methods and tools to assess
future food and environmental security of farm populations’, Philosophical Transactions of the Royal
Society of London, vol. 369, no. 1639, pp. 20120280, https://dx.doi.org/10.1098/rstb.2012.0280.
Behzadian M, Kazemzadeh RB, Albadvi A & Aghdasi M 2010, ‘PROMETHEE: A comprehensive literature
review on methodologies and applications’, European Journal of Operational Research, vol. 200, no. 1,
pp. 198–215, https://dx.doi.org/10.1016/j.ejor.2009.01.021.
Box GEP & Lucefio A 1998, ‘Quality quandaries models, assumptions, and robustness’, Quality Engineering,
vol. 10, no. 3, pp. 595-598, https://dx.doi.org/10.080/08982119808919174.
Douthwaite B & Hoffecker E 2017, ‘Towards a complexity-aware theory of change for participatory research
programs working within agricultural innovation systems’, Agricultural Systems, vol. 155, pp. 88–102,
https://dx.doi.org/10.1016/j.agsy.2017.04.002.
Epstein JM 2008, ‘Why Model?’, Journal of Artificial Societies and Social Simulation, vol. 11, no. 412.
Available from: <http://jasss.soc.surrey.ac.uk/11/4/12.html> [18 June 2020].
Forester J 1999, The deliberative practitioner: encouraging participatory planning processes, MIT Press,
Cambridge, MA, USA.
Fountas S, Wulfsohn D, Blackmore BS, Jacobsen HL & Pedersen SM 2006, ‘A model of decision-making and
information flows for information-intensive agriculture’, Agricultural Systems, vol. 87, no. 2, pp. 192–
210, https://dx.doi.org/10.1016/j.agsy.2004.12.003.
Haasnoot M, Kwakkel JH, Walker WE & ter Maat J 2013, ‘Dynamic adaptive policy pathways: A method for
crafting robust decisions for a deeply uncertain world’, Global Environmental Change, vol. 23, no. 2, pp.
485–498, https://dx.doi.org/10.1016/j.gloenvcha.2012.12.006.
Huang IB, Keisler J & Linkov I 2011, ‘Multi-criteria decision analysis in environmental sciences: Ten years of
applications and trends’, Science of the Total Environment, vol. 409, no. 19, pp. 3578–3594,
https://dx.doi.org/10.1016/j.scitotenv.2011.06.022.
Innes JE & Booher DE 1999, ‘Consensus building and complex adaptive systems’, APA Journal, vol. 65, no.
4, pp. 412–423.
Jakku E & Thorburn PJ 2010, ‘A conceptual framework for guiding the participatory development of
agricultural decision support systems’, Agricultural Systems, vol. 103, no. 9, pp. 675–682,
https://dx.doi.org/10.1016/j.agsy.2010.08.007.
Kaine G, Wright V, Cooksey R & Bewsell D 2011, ‘Identifying potential adopters of an agricultural
innovation’, in Changing land management adoption of new practices by rural landholders, eds. DJ
Pannell DJ & F Vanclay, CSIRO Publishing, Collingwood, Vic, pp. 69–86.
Kalra N, Hallegatte S, Lempert RJ, Brown C, Fozzard A, Gill S & Shah A 2014, ‘Agreeing on robust decisions
new processes for decision making under deep uncertainty’, World Bank Policy Research Working Paper,
no. 6906, https://dx.doi.org/doi:10.1596/1813-9450-6906.
Kuehne G, Llewellyn R, Pannell DJ, Wilkinson R, Dolling P, Ouzman J & Ewing M 2017, ‘Predicting farmer
uptake of new agricultural practices: A tool for research, extension and policy’, Agricultural Systems, vol.
156, pp. 115–125, https://dx.doi.org/https://doi.org/10.1016/j.agsy.2017.06.007.
Laciana CE & Oteiza-Aguirre N 2014, ‘An agent based multi-optional model for the diffusion of innovations’,
Physica A: Statistical Mechanics and its Applications, vol. 394, pp. 254–265,
https://dx.doi.org/10.1016/j.physa.2013.09.046.
Laibson D & Zeckhauser R 1998, ‘Amos Tversky and the ascent of behavioral economics’, Journal of Risk
and Uncertainty, vol. 16, no. 1, p. 7.
Lempert RJ, Groves DJ, Popper SW & Bankes SC 2006, ‘A general, analytic method for generating robust
strategies and narrative scenarios’, Management Science, vol. 52, no. 4, pp. 514–528,
https://dx.doi.org/10.1287/mnsc.l.
Lempert RJ 2015, ‘Key note presentation - A new decision science for complex systems: A decade of
enabling tools’, in IIASA Systems Analysis Conference. Luxembourg. Available from:
<https://www.youtube.com/watch?v=a0I4buEvjl4> [15 January 2020].
Lindner RK, Pardey PG & Jarret FG 1982, ‘Distance to information source and the time lag to early adoption
of trace element fertilisers’, Australian Journal of Agricultural Economics, vol. 26, no. 2, pp. 98–133,
https://dx.doi.org/10.1111/j.1467-8489.1982.tb00618.x.
Liu T, Bruins RJF & Heberling MT 2018, ‘Factors influencing farmers’ adoption of best management
practices: A review and synthesis’, Sustainability, vol. 10, no. 2, pp. 1–27,
https://dx.doi.org/10.3390/su10020432.
Rural Extension & Innovation Systems Journal, 2020 16(1) – Research © Copyright APEN
http://www.apen.org.au/rural-extension-and-innovation-systems-journal 13
Maier HR, Guillaume JHA, van Delden H, Riddell GA, Haasnoot M & Kwakkel JH 2016, ‘An uncertain future,
deep uncertainty, scenarios, robustness and adaptation: How do they fit together?’, Environmental
Modelling & Software, vol. 81, pp. 154–164, https://dx.doi.org/10.1016/j.envsoft.2016.03.014.
Montes de Oca Munguia O 2020, Improving the forecasting of adoption in agriculture, PhD thesis, The
University of Western Australia.
Montes de Oca Munguia O & Llewellyn R 2020, ‘The Adopters versus the Technology: Which Matters More
when Predicting or Explaining Adoption?’, Applied Economic Perspectives and Policy, vol. 42, no. 1, pp.
80–91, https://dx.doi.org/10.1002/aepp.13007.
Nicholson C, Long J, England D, Long B, Creelman Z, Mudge B & Cornish D 2015, Farm decision making:
The interaction of personality, farm business and risk to make more informed decisions, Grains Research
and Development Corporation, Kingston ACT.
Pannell D & Claassen R 2020, ‘The Roles of Adoption and Behavior Change in Agricultural Policy’, Applied
Economic Perspectives and Policy, vol. 42, no. 1, pp. 31–41, https://dx.doi.org/10.1002/aepp.13009.
Phillips F & Linstone H 2016, ‘Key ideas from a 25-year collaboration at technological forecasting & social
change’, Technological Forecasting and Social Change, vol. 105, pp. 158–166,
https://dx.doi.org/10.1016/j.techfore.2016.01.007.
Prinsen G & Nijhof S 2015, ‘Between logframes and theory of change: Reviewing debates and a practical
experience’, Development in Practice, vol. 25, no. 2, pp. 234–246,
https://dx.doi.org/10.1080/09614524.2015.1003532.
Rosegrant MW, Ringler C, Msangi S, Sulser TB, Zhu T & Cline SA 2002, International Model for Policy
Analysis of Agricultural Commodities and Trade (IMPACT): Model Description, IFPRI, Washington DC.
Sartas M, Schut M, Proietti C, Thiele G & Leeuwis C 2020, ‘Scaling Readiness : science and practice of an
approach to enhance impact of research for development’, Agricultural Systems, vol. 183, pp. 1–52,
https://dx.doi.org/10.1016/j.agsy.2020.102874.
Schreinemachers P & Berger T 2011, ‘An agent-based simulation model of human-environment interactions
in agricultural systems’, Environmental Modelling and Software, vol. 26, no. 7, pp. 845–859,
https://dx.doi.org/10.1016/j.envsoft.2011.02.004.
Thornton PK, Schuertz T, Forch W, Cramer L, Abreu D, Vermeulen S & Campbell BM 2017, ‘Responding to
global change: A theory of change approach to making agricultural research for development outcome-
based’, Agricultural Systems, vol. 152, pp. 145–153, https://dx.doi.org/10.1016/j.agsy.2017.01.005.
Tversky A & Kahneman D 1981, ‘The Framing of Decisions and the Psychology of Choice’, Science, vol. 211,
no. 4481, pp. 453–458, https://dx.doi.org/10.1126/science.7455683.
Vanclay F 2011, ‘Social principles for agricultural extension in facilitating the adoption of new practices’, in
Changing land management adoption of new practices by rural landholders, eds DJ Pannell & F Vanclay,
CSIRO Publishing, Collingwood, Vic, pp.51–68.
Veldkamp A 2009, ‘Investigating land dynamics: future research perspectives’, Journal of Land Use Science,
vol. 4, no. 1, pp. 5–14, https://dx.doi.org/10.1080/17474230802645592.
Vicsek T 2002, ‘The bigger picture’, Nature, vol. 418, p. 131, https://dx.doi.org/10.1038/418131a.
Walker WE 2000, ‘Policy analysis: a systematic approach to supporting policymaking in the public sector’,
Journal of Multi-Criteria Decision Analysis, vol. 9, no. 1, pp. 11–27, https://dx.doi.org/10.1002/1099-
1360(200001/05)9:1/3<11::AID-MCDA264>3.0.CO;2-3.
Wauters E & Mathijs E 2014, ‘The adoption of farm level soil conservation practices in developed countries:
a meta-analytic review’, International Journal of Agricultural Resources, Governance and Ecology, vol.
10, no. 1, pp. 78–102, https://dx.doi.org/10.1504/IJARGE.2014.061058.
Wilkinson R 2011, ‘The many meanings of adoption’, Changing land management adoption of new practices
by rural landholders, eds. DJ Pannell & F Vanclay, CSIRO Publishing, Collingwood, Vic, pp.39–50.
