No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Risk Scoring Methods for Ranking Hazards to Human Health and Environment – Contaminated Sites as a Case Study
ResearchGate has not been able to resolve any citations for this publication.
Strategic risk appraisal (SRA) has been applied to compare diverse policy level risks to and from the environment in England and Wales. Its application has relied on expert-informed assessments of the potential consequences from residual risks that attract policy attention at the national scale. Here we compare consequence assessments, across environmental, economic and social impact categories that draw on 'expert'- and 'literature-based' analyses of the evidence for 12 public risks appraised by Government. For environmental consequences there is reasonable agreement between the two sources of assessment, with expert-informed assessments providing a narrower dispersion of impact severity and with median values similar in scale to those produced by an analysis of the literature. The situation is more complex for economic consequences, with a greater spread in the median values, less consistency between the two assessment types and a shift toward higher severity values across the risk portfolio. For social consequences, the spread of severity values is greater still, with no consistent trend between the severities of impact expressed by the two types of assessment. For the latter, the findings suggest the need for a fuller representation of socioeconomic expertise in SRA and the workshops that inform SRA output.
Risk assessment and management was established as a scientific field some 30-40 years ago. Principles and methods were developed for how to conceptualise, assess and manage risk. These principles and methods still represent to a large extent the foundation of this field today, but many advances have been made, linked to both the theoretical platform and practical models and procedures. The purpose of the present invited paper is to perform a review of these advances, with a special focus on the fundamental ideas and thinking on which these are based. We have looked for trends in perspectives and approaches, and we also reflect on where further development of the risk field is needed and should be encouraged. The paper is written for readers with different types of background, not only for experts on risk.
The Global Burden of Disease (GBD) study assesses health losses from diseases, injuries, and risk factors using disability-adjusted life-years, which need a set of disability weights to quantify health levels associated with non-fatal outcomes. The objective of this study was to estimate disability weights for the GBD 2013 study.
METHODS: We analysed data from new web-based surveys of participants aged 18-65 years, completed in four European countries (Hungary, Italy, the Netherlands, and Sweden) between Sept 23, 2013, and Nov 11, 2013, combined with data previously collected in the GBD 2010 disability weights measurement study. Surveys used paired comparison questions for which respondents considered two hypothetical individuals with different health states and specified which person they deemed healthier than the other. These surveys covered 183 health states pertinent to GBD 2013; of these states, 30 were presented with descriptions revised from previous versions and 18 were new to GBD 2013. We analysed paired comparison data using probit regression analysis and rescaled results to disability weight units between 0 (no loss of health) and 1 (loss equivalent to death). We compared results with previous estimates, and an additional analysis examined sensitivity of paired comparison responses to duration of hypothetical health states.
FINDINGS: The total analysis sample consisted of 30 230 respondents from the GBD 2010 surveys and 30 660 from the new European surveys. For health states common to GBD 2010 and GBD 2013, results were highly correlated overall (Pearson's r 0·992 [95% uncertainty interval 0·989-0·994]). For health state descriptions that were revised for this study, resulting disability weights were substantially different for a subset of these weights, including those related to hearing loss (eg, complete hearing loss: GBD 2010 0·033 [0·020-0·052]; GBD 2013 0·215 [0·144-0·307]) and treated spinal cord lesions (below the neck: GBD 2010 0·047 [0·028-0·072]; GBD 2013 0·296 [0·198-0·414]; neck level: GBD 2010 0·369 [0·243-0·513]; GBD 2013 0·589 [0·415-0·748]). Survey responses to paired comparison questions were insensitive to whether the comparisons were framed in terms of temporary or chronic outcomes (Pearson's r 0·981 [0·973-0·987]).
INTERPRETATION: This study substantially expands the empirical basis for assessment of non-fatal outcomes in the GBD study. Findings from this study substantiate the notion that disability weights are sensitive to particular details in descriptions of health states, but robust to duration of outcomes.
Background:
In low- and middle-income countries (LMICs), chemical exposures in the environment due to hazardous waste sites and toxic pollutants are typically poorly documented and their health impacts insufficiently quantified. Furthermore, there often is only limited understanding of the health and environmental consequences of point source pollution problems, and little consensus on how to assess and rank them. The contributions of toxic environmental exposures to the global burden of disease are not well characterized.
Objectives:
The aim of this study was to describe the simple but effective approach taken by Blacksmith Institute's Toxic Sites Identification Program to quantify and rank toxic exposures in LMICs. This system is already in use at more than 3000 sites in 48 countries such as India, Indonesia, China, Ghana, Kenya, Tanzania, Peru, Bolivia, Argentina, Uruguay, Armenia, Azerbaijan, and Ukraine.
Methods:
A hazard ranking system formula, the Blacksmith Index (BI), takes into account important factors such as the scale of the pollution source, the size of the population possibly affected, and the exposure pathways, and is designed for use reliably in low-resource settings by local personnel provided with limited training.
Findings:
Four representative case studies are presented, with varying locations, populations, pollutants, and exposure pathways. The BI was successfully applied to assess the extent and severity of environmental pollution problems at these sites.
Conclusions:
The BI is a risk-ranking tool that provides direct and straightforward characterization, quantification, and prioritization of toxic pollution sites in settings where time, money, or resources are limited. It will be an important and useful tool for addressing toxic pollution problems in LMICs. Although the BI does not have the sophistication of the US Environmental Protection Agency's Hazard Ranking System, the case studies presented here document the effectiveness of the BI in the field, especially in low-resource settings. Understanding of the risks posed by toxic pollution sites helps assure better use of resources to manage sites and mitigate risks to public health. Quantification of these hazards is an important input to assessments of the global burden of disease.
While health impact assessment (HIA) has typically been applied to projects, plans or policies, it has significant potential with regard to strategic considerations of major health issues facing society such as climate change. Given the complexity of climate change, assessing health impacts presents new challenges that may require different approaches compared to traditional applications of HIA. This research focuses on the development of health consequence scales suited to assessing and comparing health effects associated with climate change and applied within a HIA framework. This assists in setting priorities for adaptation plans to minimize the public health impacts of climate change. The scales presented in this paper were initially developed for a HIA of climate change in Perth in 2050, but they can be applied across spatial and temporal scales. The design is based on a health effects pyramid with health measures expressed in orders of magnitude and linked to baseline population and health data. The health consequence measures are combined with a measure of likelihood to determine the level of risk associated with each health potential health impact. In addition, a simple visual framework that can be used to collate, compare and communicate the level of health risks associated with climate change has been developed.
For very good reasons, comparing risk is becoming all the rage, but the practice of comparative risk assessment (CRA) has caused battle lines to be drawn. I believe that CRA is neither as pernicious as its detractors claim nor nearly as useful as its proponents allege --particularly as is often practiced. At a time when agencies are being required to set risk-based priorities and to compare might-be-regulated risks to others with which the public is familiar,1 I argue that we need better tools for showing when (if ever) such comparisons are edifying. This article aspires to be an example of what risk assessment practitioners have to offer, if and when the "risk wars" cool down enough to allow a sensible alternative either to no analysis or to runaway analysis as an end in itself. Based on an appendix to one of the last reports to be released by the Congressional Office of Technology Assessment (OTA),2 this paper has been revised to address broadly why CRA is controversial and to offer suggestions for improving it. Thesis Comparing risks is not impossible or immoral, but it is very difficult3 --more so than either supporters or detractors of the practice seem to realize. Whenever analysts, agencies or interested parties decide to juxtapose different hazards and compare their severity, they invite controversy. Many stakeholders view CRA with suspicion because it may usurp fundamentally political choices with the authority of "science" or appear to answer questions it cannot reliably resolve. I too take a skeptical view but reject blanket condemnation that has been partly responsible for dissuading researchers from doing studies such as the OTA report this article originally supplemented. It would be simple, and perhaps sufficient, to conclude that CRA is inevitable and should therefore be done explicitly --and with broad participation from experts and laypeople. I say "inevitable" because, after taking a close look at the various purposes to which risk assessment is put, it seems that virtually all risk assessment is CRA. Standard-setting, the major arena of risk assessment, in fact often entails a risk comparison --that of the existing risk to the reduced risk expected following the contemplated intervention (which may be different in kind or affect a different population). The incessant decisions to focus on some risks and ignore or defer others also involve risk comparison, whether the priorities thus obtained arise via conscious design or by refiex. Even the language of risk communication involves comparison, for few if any of us can really understand the magnitude of a risk of "one in X" or "10-Y" in the abstract without making conscious or subconscious comparison to other risks or situations --to ground these probabilities to an experiential reference point, e.g., an annual risk of one in 1,000,000 is less likely than being struck by lightning, but more likely than being struck by a meteorite.
In this paper, the ability of the National Classification System for Contaminated Sites (NCSCS) to rank contaminated sites has been investigated by comparing NCSCS score ranks to preliminary quantitative risk assessment (PQRA) result ranks. A recently published hierarchical partial order ranking procedure was applied to hazard quotients from PQRAs to generate ranks. Using data from 20 federal contaminated sites, the study showed that sites with low and high NCSCS score ranks correspond reasonably with low and high PQRA results ranks though there is scatter amongst the data in mid-ranges. Although the output from both the NCSCS and PQRAs have significant uncertainties, the study concludes that indeed the NCSCS can be used to rank sites. A figure is provided indicating the probability that a site with a higher NCSCS score will have a higher rank, as per PQRA.
Policy- and decision-makers in government and industry constantly face important decisions without full knowledge of all the facts. They rely routinely on expert advice to fill critical scientific knowledge gaps. There are unprecedented opportunities for experts to influence decisions. Yet even the most experienced can be over-confident and error-prone, and the hidden risk is that scientists and other experts can over-reach, often with good intentions, placing more weight on the evidence they provide than is warranted. This book describes how to identify potentially risky advice, explains why group judgements outperform individual estimates, and provides an accessible and up-to-date guide to the science of expert judgement. Finally, and importantly, it outlines a simple, practical framework that will help policy- and decision-makers to ensure that the advice that they receive is relatively reliable and accurate, thus substantially improving the quality of information on which critical decisions are made.
Information on potentially harmful emerging and legacy chemicals is essential to understand the risks to the environment and inform regulatory actions. The objective of this study was to assess the occurrence, concentration, and distribution of emerging and legacy contaminants across a gradient of land-use intensity and determine the risk posed to aquatic ecosystems. The land-use intensity gradient considered was: background/undeveloped < low-intensity agriculture < high-intensity agriculture < urban residential < urban industrial. Twenty-five sites were sampled for surface water, sediment, and soil. A total of 218 chemicals were analyzed: pesticides, per- and poly-fluoroalkyl substances (PFAS), polybrominated biphenyls and polybrominated diphenyl ethers (PBDEs), phthalates, and short-chain chlorinated paraffins (SCCPs). The risk posed by the analyzed chemicals to the aquatic environment was measured using hazard quotients (HQs), which were calculated by dividing the maximum measured environmental concentration by a predicted no-effect concentration for each chemical. A HQ > 1 was considered to indicate a high risk of adverse effects from the given chemical. A total of 68 chemicals were detected: 19 pesticides, 18 PFAS, 28 PBDEs, two phthalates, and SCCPs (as total SCCPs). There were no significant differences in the overall chemical composition between land uses. However, the insecticide bifenthrin, PFAS, PBDEs, and phthalates were more frequently found in samples from residential and/or industrial sites, suggesting urban land uses are hotspots and potential large-scale sources of these chemicals. Nineteen chemicals had a HQ > 1; most had a restricted spatial distribution limited to high-intensity agriculture and industrial sites in Melbourne. Bifenthrin and the perfluorooctanesulfonic acid (PFOS) had the highest HQs in residential and industrial sites, suggesting an increased risk to aquatic ecosystems in urban settings. The results of this study will enhance future research, predictive methods, and effective targeting of monitoring, and will help guide regulatory management actions and mitigation solutions.
The objective of this study is to review applications of multicriteria decision analysis (MCDA) methods to provide structure for and practical insights on how MCDA methods are applied in different healthcare areas. Four databases (Embase, Medline, Web of Science, and PubMed) were searched from 1980 to February 7, 2018. A scoping review was performed to outline the most frequently applied MCDA methods, in which only case studies were considered, as the objective was to assess the applications of MCDA methods. Seventy case studies were selected. The review showed that the first real‐life application was published in 1990. With regard to areas of application, three aspects were analysed: type of health services (public or private), type of intervention (preventive, diagnostic, or treatment), and healthcare area (policymaking; resource allocation; health technology assessment; pharmaceutics; and hospital, clinical, or patient level). Data revealed that MCDA was principally used in public health services (n = 59). Three types of interventions were identified: prevention (n = 36), diagnosis (n = 32), and treatment (n = 2). An MCDA process of application was then followed to provide practical insights. During the problem‐structuring phase, results showed that processes for selecting stakeholders and criteria differed among healthcare areas. For model building, authors preferred value measurement methods and especially the analytical hierarchy process. Authors of the 70 case studies included and confirmed the usefulness of MCDA in different healthcare contexts. The stakeholders, criteria, and MCDA methods chosen differ from one study to another, answering different research objectives in different healthcare areas. This scoping review highlights the importance of each MCDA step (using Belton and Stewart's framework), from problem structuring (including the selection of stakeholders) to the model building phase (encompassing the selection of the appropriate MCDA method). We recommend structured and justified analysis so that decision makers and stakeholders can feel confident during the application process in order to make final decisions more meaningful. Further work is needed to help researchers and decision makers when choosing a suitable MCDA method.
This study aimed to critically review methods for ranking risks related to food safety and dietary hazards on the basis of their anticipated human health impacts. A literature review was performed to identify and characterize methods for risk ranking from the fields of food, environmental science and socio-economic sciences. The review used a predefined search protocol, and covered the bibliographic databases Scopus, CAB Abstracts, Web of Sciences, and PubMed over the period 1993–2013.
All references deemed relevant, on the basis of of predefined evaluation criteria, were included in the review, and the risk ranking method characterized. The methods were then clustered – based on their characteristics - into eleven method categories. These categories included: risk assessment, comparative risk assessment, risk ratio method, scoring method, cost of illness, health adjusted life years, multi-criteria decision analysis, risk matrix, flow charts/decision trees, stated preference techniques and expert synthesis. Method categories were described by their characteristics, weaknesses and strengths, data resources, and fields of applications.
It was concluded there is no single best method for risk ranking. The method to be used should be selected on the basis of risk manager/assessor requirements, data availability, and the characteristics of the method. Recommendations for future use and application are provided.
This article discusses to what extent risk analysis is scientific in view of a set of commonly used definitions and criteria. We consider scientific knowledge to be characterized by its subject matter, its success in developing the best available knowledge in its fields of study, and the epistemic norms and values that guide scientific investigations. We proceed to assess the field of risk analysis according to these criteria. For this purpose, we use a model for risk analysis in which science is used as a base for decision making on risks, which covers the five elements evidence, knowledge base, broad risk evaluation, managerial review and judgment, and the decision; and that relates these elements to the domains experts and decisionmakers, and to the domains fact-based or value-based. We conclude that risk analysis is a scientific field of study, when understood as consisting primarily of (i) knowledge about risk-related phenomena, processes, events, etc., and (ii) concepts, theories, frameworks, approaches, principles, methods and models to understand, assess, characterize, communicate, and manage risk, in general and for specific applications (the instrumental part).
The risk journal literature lacks a clear and simple account of the conceptual issues involved in determining the overall risk of an action, and in explaining how risk is additive. This article attempts to bring a measure of clarity to these issues in as basic and non‐technical a way as possible. First of all, the view that risk is ‘expected harm’ is explained. The view that risk is a quantitative concept is then defended. The distinction between the risk run by doing action A in respect of possible outcome x, and the overall risk run by doing action A in general is explained, as is the position that the overall risk of A is determined by summing the risks of each possible harm that A could give rise to. The article then explains how risks can be summed over time, as long as the probabilities involved are determined according to probability theory. Finally, the article explains that in a doing a risk‐benefit analysis of A, positive aspects of a possible outcome x, where x is harmful on balance, must be incorporated into x's level of harm rather than incorporated into the benefit side of the risk‐benefit analysis of A.
A deliberative method for ranking risks was evaluated in a study involving 218 risk managers. Both holistic and multiattribute procedures were used to assess individual and group rankings of health and safety risks facing students at a fictitious middle school. Consistency between the rankings that emerged from these two procedures was reasonably high for individuals and for groups, suggesting that these procedures capture an underlying construct of riskiness. Participants reported high levels of satisfaction with their groups’ decision-making processes and the resulting rankings, and these reports were corroborated by regression analyses. Risk rankings were similar across individuals and groups, even though individuals and groups did not always agree on the relative importance of risk attributes. Lower consistency between the risk rankings from the holistic and multiattribute procedures and lower agreement among individuals and groups regarding these rankings were observed for a set of high-variance risks. Nonetheless, the generally high levels of consistency, satisfaction, and agreement suggest that this deliberative method is capable of producing risk rankings that can serve as informative inputs to public risk-management decision making.
Any practical process of risk ranking must group hazards into a manageable number of categories. Defining such categories requires value choices that can have important implications for the rankings that result. Most risk-management organizations will find it useful to begin defining categories in terms of environmental loadings or initiating events. However, the resulting categories typically need to be modified in light of other considerations. Risk-ranking projects can benefit from considering several alternative categorization strategies and drawing upon elements of each in developing their final categorization of risks. In principle, conducting multiple ranking exercises by using different categorizations could be interesting and useful. In practice, agencies are unlikely to have either the resources or patience to do this, but other groups in society might. Done well, such additional independent rankings could add valuable inputs to democratic risk-management decision making.
Risk ranking offers a potentially powerful means for gathering public input to help set risk-management priorities. In most rankings conducted to date, the categories and attributes used to describe the risks have varied widely, the materials and procedures have not been designed to facilitate comparisons among risks on all important attributes, and the validity and reproducibility of the resulting rankings have not been assessed. To address these needs, a risk-ranking method was developed in which risk experts define and categorize the risks to be ranked, identify the relevant risk attributes, and characterize the risks in a set of standardized risk summary sheets, which are then used by lay or other groups in structured ranking exercises. To evaluate this method, a test bed involving 22 health and safety risks in a fictitious middle school was created. This article provides an overview of the risk-ranking method and describes the challenges faced in designing the middle school test bed. A companion article in this issue reports on the validity of the ranking procedures and the level of agreement among risk managers regarding ranking of risks and attributes.
A ranking system for contaminated sites based on comparative risk methodology using fuzzy Preference Ranking Organization METHod for Enrichment Evaluation (PROMETHEE) was developed in this article. It combines the concepts of fuzzy sets to represent uncertain site information with the PROMETHEE, a subgroup of Multi-Criteria Decision Making (MCDM) methods. Criteria are identified based on a combination of the attributes (toxicity, exposure, and receptors) associated with the potential human health and ecological risks posed by contaminated sites, chemical properties, site geology and hydrogeology and contaminant transport phenomena. Original site data are directly used avoiding the subjective assignment of scores to site attributes. When the input data are numeric and crisp the PROMETHEE method can be used. The Fuzzy PROMETHEE method is preferred when substantial uncertainties and subjectivities exist in site information. The PROMETHEE and fuzzy PROMETHEE methods are both used in this research to compare the sites. The case study shows that this methodology provides reasonable results.
This article reports an extension of the Carnegie Mellon risk-ranking method to incorporate ecological risks and their attributes. On the basis of earlier risk-perception studies, we identified a set of 20 relevant attributes for describing health, safety, and environmental hazards in standardized risk summary sheets. In a series of three ranking sessions, 23 laypeople ranked 10 such hazards in a fictional Midwestern U.S. county using both holistic and multiattribute ranking procedures. Results were consistent with those from previous studies involving only health and safety hazards, providing additional evidence for the validity of the method and the replicability of the resulting rankings. Holistic and multiattribute risk rankings were reasonably consistent both for individuals and for groups. Participants reported that they were satisfied with the procedures and results, and indicated their support for using the method to advise real-world risk-management decisions. Agreement among participants increased over the course of the exercise, perhaps because the materials and deliberations helped participants to correct their misconceptions and clarify their values. Overall, health and safety attributes were judged more important than environmental attributes. However, the overlap between the importance rankings of these two sets of attributes suggests that some information about environmental impacts is important to participants' judgments in comparative risk-assessment tasks.
Comparative risk projects can provide broad policy guidance but they rarely have adequate scientific foundations to support precise risk rankings. Many extant projects report rankings anyway, with limited attention to uncertainty. Stochastic uncertainty, structural uncertainty, and ignorance are types of incertitude that afflict risk comparisons. The recently completed New Jersey Comparative Risk Project was innovative in trying to acknowledge and accommodate some historically ignored uncertainties in a substantive manner. This article examines the methods used and lessons learned from the New Jersey project. Monte Carlo techniques were used to characterize stochastic uncertainty, and sensitivity analysis helped to manage structural uncertainty. A deliberative process and a sorting technique helped manage ignorance. Key findings are that stochastic rankings can be calculated but they reveal such an alarming degree of imprecision that the rankings are no longer useful, whereas sorting techniques are helpful in spite of uncertainty. A deliberative process is helpful to counter analytical overreaching.
For many years experimental observations have raised questions about the rationality of economic agents--for example, the Allais Paradox or the Equity Premium Puzzle. The problem is a narrow notion of rationality that disregards fear. This article extends the notion of rationality with new axioms of choice under uncertainty and the decision criteria they imply (Chichilnisky, G., 1996a. An axiomatic approach to sustainable development. Social Choice andWelfare 13, 257-321; Chichilnisky, G., 2000. An axiomatic approach to choice under uncertainty with Catastrophic risks. Resource and Energy Economics; Chichilnisky, G., 2002. Catastrophical Risk. Encyclopedia of Environmetrics, vol. 1. John Wiley & Sons, Ltd., Chicester). In the absence of catastrophes, the old and the new approach coincide, and both lead to standard expected utility. A sharp difference emerges when facing rare events with important consequences, or catastrophes. Theorem 1 establishes that a classic axiom of choice under uncertainty - Arrow's Monotone Continuity axiom, or its relatives introduced by DeGroot, Villegas, Hernstein and Milnor - postulate rational behavior that is [`]insensitive' to rare events as defined in (Chichilnisky, G., 1996a. An axiomatic approach to sustainable development. Social Choice andWelfare 13, 257-321; Chichilnisky, G., 2000. An axiomatic approach to choice under uncertainty with Catastrophic risks. Resource and Energy Economics; Chichilnisky, G., 2002. Catastrophical Risk. Encyclopedia of Environmetrics, vol. 1. John Wiley & Sons, Ltd., Chicester). Theorem 2 replaces this axiom with another that allows extreme responses to extreme events, and characterizes the implied decision criteria as a combination of expected utility with extremal responses. Theorems 1 and 2 offer a new understanding of rationality consistent with previously unexplained observations about decisions involving rare and catastrophic events, decisions involving fear, the Equity Premium Puzzle, [`]jump di
How NOT to Piss Off a Community - How to Work in Low-trust Environments With Integrity and Compassion
- C Dalton
Warning fatigue is not a myth – understanding why people do or don't respond to warnings
- Mackie B.