Article

Uptake pathways: The potential of Bayesian belief networks to assist the management, monitoring and evaluation of development-orientated research

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

The effectiveness of development assistance has come under renewed scrutiny in recent years. In an era of growing economic liberalisation, research organisations are increasingly being asked to account for the use of public funds by demonstrating achievements. However, in the natural resources (NR) research field, conventional economic assessment techniques have focused on quantifying the impact achieved rather understanding the process that delivered it. As a result, they provide limited guidance for planners and researchers charged with selecting and implementing future research. In response, “pathways” or logic models have attracted increased interest in recent years as a remedy to this shortcoming. However, as commonly applied these suffer from two key limitations in their ability to incorporate risk and assess variance from plan. The paper reports the results of a case study that used a Bayesian belief network approach to address these limitations and outlines its potential value as a tool to assist the planning, monitoring and evaluation of development-orientated research.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The dominant methodology for conducting impact assessments has been economic cost-benefit analysis, which utilizes the concept of social welfare to measure benefits and discounted measures of worth to compare costs with these benefits over time (Henderson and Burn 2004). Quantitative methods for impact assessments require the development of a counterfactual, to postulate the situation without any intervention (White 2010), which is challenging if real communities or farmers are involved, as people cannot simultaneously participate and not participate in a programme (Maredia 2009). ...
... There are many factors and actors that affect the long-term impacts of research projects and, in the case of socioeconomic impacts, intermediaries and the network of actor contributions play key roles in adoption of knowledge (Joly et al. 2015). While impact assessment has a role in project evaluation, commentators (Cracknell 1996;Alston and Pardey 2001;Ekboir 2003;Hall et al. 2003;Henderson and Burn 2004;Mercer 2004;Millstone, Van Zwanenberg and Marshall 2010;White 2010) have identified the following issues regarding its effectiveness for learning lessons: ...
Article
Full-text available
This article reviews approaches commonly used to evaluate official development assistance programmes and projects, considers their utility for lessons learned, and proposes a simple methodology for evaluating the relative success of research projects. While current approaches each have appropriate uses, they do not readily provide a way of systematically evaluating the relative success of multiple projects without the need for complex reviews and analyses. This is a constraint for research managers in donor organizations, such as the Australian Centre for International Agricultural Research (ACIAR), in facilitating organizational learnings, including factors that contribute to project success. The proposed evaluation methodology assesses two dimensions of project success: achievements of the research project, and the impacts arising from the research. Four assessment criteria are identified for each dimension, and are scored largely by reviewing existing project records. The methodology includes suggested evaluation questions and evidence requirements, and generates a scoring matrix. The methodology allows assessors to vary the weighting assigned in scoring, and to interpret outcomes, in terms most relevant to the purpose and context of particular evaluations. It was tested by application to 10 collaborative forestry research projects funded by ACIAR and implemented in Vietnam. The results indicate that the evaluation methodology is useful in identifying the relative success of research projects, particularly for the research achievements dimension. Its usefulness for the research impacts dimension depends on the nature of the research project and its impacts, the timing of the assessment after project completion, and how well the impacts have been documented.
... An evaluation process which includes the viewpoints of stakeholders may also be more appropriate than approaches in which assessments are conducted by external experts using predetermined indicators of success (Cramb and Purcell 2001;Owen 2006). Although the evaluation of development assistance in the past has often neglected the complex processes behind assistance uptake, it is no longer acceptable to disregard them (Henderson and Burn 2004;Henriksen and Barlebo 2007). Hence, systems modelling, which includes breaking problems down into component variables, considering interactions between them, recognising the dangers of narrow model boundaries and the importance of qualitative as well as quantitative data (Sterman 2002), matched the evaluation needs of this program. ...
... The ability of BBNs to accommodate imprecise estimates of qualitative variables is also highly advantageous in extension, which by definition is concerned with subjective human values and attitudes. Unfamiliarity with the concept of probability can limit the extent to which stakeholders can provide probability estimates and a variety of methods have been used to elicit data, e.g. a consensus based approach followed by expert group review (Henderson and Burn 2004) or a semi-structured questionnaire (Cain et al. 2003). Henriksen and Balebo (2007) used stakeholder opinion to populate models of groundwater management, but found that explaining BBNs was demanding and time consuming. ...
Article
Full-text available
This paper presents a systems modelling approach to evaluating the success of an agroforestry extension program in Leyte, the Philippines. During the program, variables which are intrinsic to farmers’ socio-economic and farming systems were found to have influenced the uptake and acceptance of extension advice. Evaluation of the program therefore depended on identifying the variables and their interdependencies and assessing their relative influence on program outputs. For this purpose, a systems approach which encourages breaking systems into component variables, but also acknowledges the context of problems, assisted construction of models. Using both empirical data collected during program activities and input from stakeholders, Bayesian Belief Network software was used to predict critical success factors for four aspects of the overall extension system, namely recruitment, use of written extension materials, farmers’ self-efficacy and retention of participating farmers throughout the program. A key predicted constraint to program recruitment is farmers’ perception of harvest security and while this variable can be partly addressed through dissemination of information on harvesting legislation, title security cannot. Differing levels of farmers’ education result in differences in predicted reading ability, comprehension of extension literature and possible misconstrual of information. The variable most critical to the development of farmers’ self-efficacy is extended on-farm technical assistance and support. KeywordsSelf-efficacy–Harvest certainty–Land tenure–Bayesian belief networks
... Yet, they are highly suited to this context. For example, Henderson and Burn (2004) illustrate how BNs are able to combine information from various sources, such as hard data and expert knowledge, into comprehensive causal models that result in more accurate impact forecasts than using data or expert knowledge alone. ...
Technical Report
Full-text available
The report outlines some Decision Analysis tools for the adaptation and application of Bayesian Networks to agriculture for nutrition development contexts. These are promising as effective strategies for dealing with complex systems, multiple impact pathways, uncertain and incomplete information and other current constraints to meaningful quantitative impact projections.
... Useful examples of the applicability of Bayesian Networks include: environmental management (Low Choy et al., 2005;Henriksen et al., 2007;Ticehurst et al., 2007;Stewart-Koster et al., 2010), reliability engineering (Trucco et al., 2008), software engineering (Fenton et al., 2004) and project management (Khodakarami and Abdi, 2014). In fact, Bayesian Networks have been used for knowledge integration tools and systems analysis in an agricultural context within a number of studies (Henderson and Burn, 2004;Bashari et al., 2008;Florin et al., 2013;Stewart-Koster et al., 2017). ...
Article
A Bayesian Network model has been developed that synthesizes findings from concurrent multi-disciplinary research activities. The model describes the many factors that impact on the chances of a smallholder farmer adopting a proposed change to farming practices. The model, when applied to four different proposed technologies, generated insights into the factors that have the greatest influence on adoption rates. Behavioural motivations for change are highly dependent on farmers' individual viewpoints and are also technology dependent. The model provides a boundary object that provides an opportunity to engage experts and other stakeholders in discussions about their assessment of the technology adoption process, and the opportunities, barriers and constraints faced by smallholder farmers when considering whether to adopt a technology.
... Furthermore, BN models can be generated and parameterized through the use of expert knowledge, integrated into model calculations (Yet et al., 2016;Henderson & Burn, 2004). Many examples have illustrated the use of structured elicitation techniques to gather expert knowledge for generating BN model structures and eliciting model parameters (cf. ...
Article
Full-text available
Governments around the world have agreed to end hunger and food insecurity and to improve global nutrition, largely through changes to agriculture and food systems. However, they are faced with a lot of uncertainty when making policy decisions, since any agricultural changes will influence social and biophysical systems, which could yield either positive or negative nutrition outcomes. We outline a holistic probability modeling approach with Bayesian Network (BN) models for nutritional impacts resulting from agricultural development policy. The approach includes the elicitation of expert knowledge for impact model development, including sensitivity analysis and value of information calculations. It aims at a generalizable methodology that can be applied in a wide range of contexts. To showcase this approach, we develop an impact model of Vision 2040, Uganda's development strategy, which, among other objectives, seeks to transform the country's agricultural landscape from traditional systems to large-scale commercial agriculture. Model results suggest that Vision 2040 is likely to have negative outcomes for the rural livelihoods it intends to support; it may have no appreciable influence on household hunger but, by influencing preferences for and access to quality nutritional foods, may increase the prevalence of micronutrient deficiency. The results highlight the tradeoffs that must be negotiated when making decisions regarding agriculture for nutrition, and the capacity of BNs to make these tradeoffs explicit. The work illustrates the value of BNs for supporting evidence-based agricultural development decisions.
... Forestry (Fo) 6 Cyr et al. (2010), Henderson and Burn (2004), Ordónez-Galán et al., 2009, Rieman et al. (2001), Stassopoulou et al. (1998), Walton and Meidinger (2006) Fisheries (Fi) 5 Axelson et al. (2009), Giles (2008), Haapasaari et al. (2007), Hammond and O'Brien (2001), Uusitalo et al. (2005) ...
... In addition, BBNs can be back transformed to explore how a child node's outcome can be influenced by its parents (Wooldridge and Done 2004). As Henderson and Burn (2004) stated, BBNs or IDs ''model complex causal inter-relationships in a flexible way.'' This flexibility along with the ability to include many types of information makes IDs ideal candidates to logically determine courses of action in risk management. ...
The pesticide policy arena is filled with discussion of probabilistic approaches to assess ecological risk, however, similar discussions about implementing formal probabilistic methods in pesticide risk decision making are less common. An influence diagram approach is proposed for ecological risk-based decisions about pesticide usage. Aside from technical data, pesticide risk management relies on diverse sources, such as stakeholder opinions, to make decisions about what, how, where, and when to spray. Bayesian influence diagrams allow multiple lines of evidence, including process related information from existing data and expert judgment, in 1 inclusive decision model. In ecological risk assessments, data informally incorporated for pesticide usage decisions, such as field and laboratory effect studies along with chemical monitoring and modeling data, can be formally incorporated and expressed in linked causal diagrams. A case study is presented from the perspective of an environmental manager wishing to efficiently control pests while minimizing risk to local aquatic receptors. Exposure modeling results and toxicity studies were incorporated, and an ecological risk assessment was carried out but combined with hypothetical information on spraying efficacy and valuation of outcomes that would be necessary for making risk management decisions. The variables and their links in the influence diagram are ones that are important to a manager and can be manipulated to optimally control pests while protecting nontarget resources.
Article
Full-text available
The current practice of impact evaluation in agricultural research organizations in Eastern and Central Africa often entails isolated quantitative studies that are not linked to research management. We propose a theory-based, qualitative approach to strengthen the capacity of research organizations to reflect upon the process of agricultural innovation and to manage programs so as to obtain greater development impact. Impact pathway analysis provides a holistic view of the change processes that can be linked to a research program. The design of an impact pathway is kept open for revision, and is expected to evolve over time as knowledge is gained about agricultural innovation processes. Impact pathway analysis can accommodate the views of different stakeholders and it can incorporate information drawn from a variety of sources and methods.
Article
Full-text available
A methodological framework was designed to assess the effectiveness of agri-environmental policy measures adopted by the Veneto Region to reduce diffuse water pollution of agricultural origin. Two already existing methodologies were combined in a new flexible approach for policy assessment: Conceptual Modelling and Bayesian Networks (BNs). The former supported the development of a shared conceptual model (a cognitive map) of the agro-ecosystem of the study area; while the latter allowed the development of a probabilistic model coherent with the cognitive map. BNs were selected because they allow analyses with scarce data; they can be updated when further information becomes available, and are easily understandable by layperson. The paper reports the results obtained in the Venice Lagoon Watershed (VLW) case study, where the current agri-environmental measures were assessed in order to identify their effectiveness in terms of reduction of nitrogen releases in water bodies connected to the lagoon ecosystem. Preliminary results obtained by implementing expert opinions in the BN pointed out the likely limited effects of the measures on the declared objective of
Article
Full-text available
Bayesian Belief Network (BBN) modelling may be applied in rural extension in situations where program outputs are influenced by variables which are sequentially influenced by other variables. For a recently completed agroforestry extension program in Leyte the Philippines, BBN modelling of site factors, establishment practices and risk, predicted widely different program outputs for different levels of extension assistance and farmer inputs. In a situation where very little was known about how farmers would respond to offers of extension assistance, monitoring of the program over a period of three years revealed that extended extension assistance was crucial in determining the likely survival and growth of trees. Extended extension assistance was also important for the elimination of unsuitable sites and the use of appropriate establishment procedures. Where extension support was not available, farmers displayed a poor knowledge of the principles of tree growth, planting trees underneath complete canopies and adjacent to mature coconut palms even though they could have been expected to have extensive local knowledge of raising and growing plants. Approximately one third of planting sites were infertile and eroded and growth of newly planted trees on these sites was poor, often because site preparation and maintenance was minimal. Newly established trees were also found to be at risk from fire, typhoon, and grazing and in situations where plantations were destroyed, farmers became antagonistic towards the program. The implications of the BBN modelling for a hypothetically expanded program are that extended assistance and site inspections are necessary to eliminate planting trees on inappropriate and unsuitable sites and to improve establishment practices and weed control in order to avoid plantations of suppressed and chlorotic trees which fail to meet the expectations of farmers, thus impinging on the success of the program.
Article
To determine the antidiabetic mechanism of Bakumondo-inshi (BI), we examined its effects on glucose absorption, alpha-glucosidase activity, sodium-dependent glucose transporter and facilitative glucose transporter isoform 5 (GLUT5) in small intestine. The oral administration of BI into KK-Ay mice caused a significant decrease in the glucose absorption in small intestine. The small intestine content of active glucose transporter isoform (SGLUT) protein content from KK-Ay mouse significantly decreased in the BI-treated KK-Ay mice compared to that in the controls. However, the small intestine content of facilitative glucose transporter isoform, GLUT5 protein content did not change. The alpha-glucosidase activity in small intestine significantly decreased in the BI-treated KK-Ay mice. These results suggest that the antidiabetic effect of BI is derived, at least in part, from a decrease of glucose absorption in small intestine , due to the reduction of SGLUT protein content in total membrane of the small intestine and the reduction of alpha-glucosidase activity. Because of its therapeutic mechanism, BI could be a new category of therapeutic agent for non-insulin dependent diabetic mellitus.
Article
Full-text available
This inductive study offers an examination of 23 cases in which informants from firms engaged in large-scale global projects reported unforeseen costs after failing to comprehend cognitive-cultural, normative, and/or regulative institutions in an unfamiliar host societal context. The study builds on the conceptual framework of institutional theory. The findings, which include propositions and a generic narrative model, contribute to theoretical knowledge of how institutional exceptions arise, how they are resolved, and how they typically involve three general phases: ignorance, sensemaking, and response. The findings also articulate the kinds of institutional transaction costs that an entrant incurs in each of the three phases, and the conditions that lead to the growth of these costs. Journal of International Business Studies (2008) 39, 562–588. doi:10.1057/palgrave.jibs.8400370
Article
A causal network is used in a number of areas as a depiction of patterns of ‘influence’ among sets of variables. In expert systems it is common to perform ‘inference’ by means of local computations on such large but sparse networks. In general, non‐probabilistic methods are used to handle uncertainty when propagating the effects of evidence, and it has appeared that exact probabilistic methods are not computationally feasible. Motivated by an application in electromyography, we counter this claim by exploiting a range of local representations for the joint probability distribution, combined with topological changes to the original network termed ‘marrying’ and ‘filling‐in‘. The resulting structure allows efficient algorithms for transfer between representations, providing rapid absorption and propagation of evidence. The scheme is first illustrated on a small, fictitious but challenging example, and the underlying theory and computational aspects are then discussed.
Article
The practical elicitation of expert beliefs is considered through two contrasting examples. The first example concerns elicitation of engineers' prior beliefs about various quantities relating to the future capital investment need of a water company. Prior beliefs needed to be elicited about very many quantities, but only in the form of prior means, variances and covariances. A computed procedure was required that could be routinely used by engineers unsupervised. The second example is a single application-specific elicitation of the beliefs of hydrogeologists about properties of certain rocks. A full probability specification was ideally to be obtained from a two-day intensive supervised, elicitation with several experts together. The two different approaches used for these problems are described and contrasted, but a common principle of trying to identify and elicit separately the various sources of expert uncertainty is identified.
Article
The practical elicitation of expert beliefs is considered through two contrasting examples. The first example concerns elicitation of engineers' prior beliefs about various quantities relating to the future capital investment need of a water company. Prior beliefs needed to be elicited about very many quantities, but only in the form of prior means, variances and covariances. A computerized procedure was required that could be routinely used by engineers, unsupervised. The second example is a single, application-specific elicitation of the beliefs of hydrogeologists about properties of certain rocks. A full probability specification was ideally to be obtained from a two-day intensive, supervised, elicitation with several experts together. The two different approaches used for these problems are described and contrasted, but a common principle of trying to identify and elicit separately the various sources of expert uncertainty is identified
Book
This book contains chapters relating to forest economics. Included are the following chapters: Forecasting demand and supply of forest resources, products, and services; Wood fiber production; Forestry sector environmental effects.
Article
The practical elicitation of expert beliefs is considered through two contrasting examples. The first example concerns elicitation of engineers' prior beliefs about various quantities relating to the future capital investment need of a water company. Prior beliefs needed to be elicited about very many quantities, but only in the form of prior means, variances and covariances. A computerized procedure was required that could be routinely used by engineers, unsupervised. The second example is a single, application-specific elicitation of the beliefs of hydrogeologists about properties of certain rocks. A full probability specification was ideally to be obtained from a two-day intensive, supervised, elicitation with several experts together. The two different approaches used for these problems are described and contrasted, but a common principle of trying to identify and elicit separately the various sources of expert uncertainty is identified
Eliciting expert beliefs in substantial practical applications The Statistician 47 (Part 1), 21–35 Rethinking Development—Challenges and Opportunities. Speech at the Tenth Ministerial Meeting of UNCTAD, Bangkok, Thailand, 16 February
  • O Hagan
O’Hagan, A., 1998. Eliciting expert beliefs in substantial practical applications. The Statistician 47 (Part 1), 21–35. Wolfensohn, J., 2000. Rethinking Development—Challenges and Opportunities. Speech at the Tenth Ministerial Meeting of UNCTAD, Bangkok, Thailand, 16 February. World Bank, Washington, DC. J.S. Henderson, R.W. Burn/Agricultural Systems 79 (2004) 3–15 15
Netica Application User's Guide Available from <www.norsys.com>. O'Hagan, A., 1998. Eliciting expert beliefs in substantial practical applications
  • Norsys Software
Norsys Software Corp. 1998. Netica Application User's Guide. Norsys Software Corp., Vancouver, BC, Canada. Available from <www.norsys.com>. O'Hagan, A., 1998. Eliciting expert beliefs in substantial practical applications. The Statistician 47 (Part 1), 21–35.
Rethinking Development—Challenges and Opportunities. Speech at the Tenth Ministerial Meeting of UNCTAD
  • J Wolfensohn
Wolfensohn, J., 2000. Rethinking Development—Challenges and Opportunities. Speech at the Tenth Ministerial Meeting of UNCTAD, Bangkok, Thailand, 16 February. World Bank, Washington, DC. J.S. Henderson, R.W. Burn / Agricultural Systems 79 (2004) 3–15
Impact Assessment Study: Final Technical Report R7079
  • J S Henderson
Henderson, J.S., 1999. Impact Assessment Study: Final Technical Report R7079. Forestry Research Programme. Natural Resources International Ltd, Kent, UK.
Adaptation and application of a balanced scorecard approach to research performance assessment: the case of UK-supported miombo woodland research in Malawi
  • S Henderson
  • B Burn
  • J Lowore
  • N Marsland
Henderson, S., Burn, B., Lowore, J., Marsland, N., 2000. Adaptation and application of a balanced scorecard approach to research performance assessment: the case of UK-supported miombo woodland research in Malawi. In: ECART/ASARECA/CTA Workshop on Impact Assessment of Agricultural Research in Eastern and Central Africa, Entebbe, Uganda, 16–19 November 1999. Deutsche Gesell-schaft fur Technische Zusammenarbeit (GTZ) GmbH.
Natural Resources Research: Impact Assessment and Poverty. NRI Policy Series 10
  • S Henderson
Henderson, S., 2000. Natural Resources Research: Impact Assessment and Poverty. NRI Policy Series 10. Natural Resources Institute, University of Greenwich.
Research costs and social returns
  • Griliches
Netica Application User's Guide
  • Corp Norsys Software