Michael McWilliams’s research while affiliated with University of Michigan and other places

What is this page?


This page lists works of an author who doesn't have a ResearchGate profile or hasn't added the works to their profile yet. It is automatically generated from public (personal) data to further our legitimate goal of comprehensive and accurate scientific recordkeeping. If you are this author and want this page removed, please let us know.

Publications (5)


Figure 1. Worked example from expert elicitation survey instrument 
Figure 4. Expert mixture probability density function for Campylobacter on poultry
Figure 5. The number of excess deaths caused by ranking risk-mitigation programs based on outbreak data when expert judgment attributions were correct.
Illness incidence and rankings by food
Fatality incidence and rankings by food

+3

Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: A demonstration using foodborne pathogens
  • Article
  • Full-text available

May 2008

·

117 Reads

·

20 Citations

Reliability Engineering & System Safety

·

·

·

Michael McWilliams

This article looks at a new approach to expert elicitation that combines basic elements of conventional expert elicitation protocols with formal survey methods and larger, heterogeneous expert panels. This approach is appropriate where the hazard-estimation task requires a wide range of expertise and professional experience. The ability to judge when to rely on alternative data sources often is critical for successful risk management. We show how a large, heterogeneous sample can support internal validation of not only the experts’ assessments but also prior information that is based on limited historical data.We illustrate the use of this new approach to expert elicitation by addressing a fundamental problem in US food safety management, obtaining comparable food system-wide estimates of the foodborne illness by pathogen–food pair and by food. The only comprehensive basis for food-level hazard analysis throughout the US food supply currently available is outbreak data (i.e., when two or more people become ill from the same food source), but there is good reason to question the portrayal that outbreak data alone gives of food risk. In this paper, we compare results of food and pathogen–food incidence estimates based on expert judgment and based on outbreak data, and we demonstrate a suite of uncertainty measures that allow for a fuller understanding of the results.

Download

Elicitation from Large, Heterogeneous Expert Panels: Using Multiple Uncertainty Measures to Characterize Information Quality for Decision Analysis

June 2007

·

69 Reads

·

25 Citations

Decision Analysis

Decision analysts are frequently called on to help inform decision makers in cases involving considerable uncertainty. In such situations, expert elicitation of parameter values is frequently used to supplement more conventional research. Expert elicitations typically rely on small panels of experts. However, in cases where the information needed for risk management must draw on a broad range of disciplines or types of professional backgrounds and experience, a larger, more heterogeneous expert panel is needed. In this paper we develop a formal protocol and a suite of uncertainty measures for this work. The protocol uses formal survey methods to take advantage of variation in individual expert uncertainty and heterogeneity among experts as a means of quantifying and comparing sources of uncertainty about parameters of interest. We illustrate the use of this protocol with an expert elicitation on the distribution of foodborne illness in the United States across foods. In the survey, experts are asked to attribute illnesses associated with one of eleven major foodborne pathogens to the consumption of one of eleven categories of food. Results show how the distributions of multiple measures of uncertainty (e.g., agreement of experts and uncertainty in knowledge), made feasible by use of a large panel of experts, can help identify which of several types of risk management actions may be most appropriate.


Food category definitions a
Using Expert Elicitation To Link Foodborne Illnesses in the United States to Foods

June 2007

·

84 Reads

·

106 Citations

Journal of Food Protection

U.S. foodborne illness risk analysis would benefit greatly from better information on the relationship between the incidence of foodborne illness and exposure to foodborne pathogens. In this study, expert elicitation was used to attribute U.S. foodborne illnesses caused by the nine FoodNet pathogens, Toxoplasma gondii, and noroviruses to consumption of foods in 11 broad categories. Forty-two nationally recognized food safety experts responded to a formal written expert elicitation survey. For each pathogen, respondents gave their best estimate of the distribution of foodborne illnesses associated with each of the food categories and the 90% confidence bounds on each of their estimates. Based on the work of Paul Mead and his coauthors, food attribution percentage estimates from this study were used to attribute case, hospitalization, and death incidence estimates to foods according to pathogen. These attribution estimates indicate that 15 food-pathogen pairs account for 90% of the illnesses, 25 pairs account for 90% of hospitalizations, and 21 pairs account for 90% of deaths.



Citations (4)


... In this regard, Batz et al. (2012) reported that outbreak-based attribution estimates for Campylobacter, Toxoplasma and Yersinia are not representative; therefore expert-based attribution seems better for these pathogens. Hoffmann et al. (2008) also advocated the use of expert opinion because outbreak data generally are incomplete, whereas using a structured approach for eliciting expert opinion, allowing for assessment of the reliability of the expert-based data, would cover all commoditypathogen combinations. Previous foodborne source attribution studies (Batz et al., 2012;Hoffman et al., 2007;Vally et al., 2016) have reported varied levels of uncertainty around the estimates at the commodity level, which could be expected to be greater when linking a disease rarely reported with sub-product categories. ...

Reference:

Source attribution at the food sub-product level for the development of the Canadian Food Inspection Agency risk assessment model
Informing risk-mitigation priorities using uncertainty measures derived from heterogeneous expert panels: A demonstration using foodborne pathogens

Reliability Engineering & System Safety

... However, this should not diminish the utility of the findings as a first depiction of the diversity of factors related to AMR in the Canadian food chain context, that allows us to identify additional domain experts with whom to engage in future model expansion and validation. Our approach was guided by expert elicitation, a method particularly useful for addressing questions that are difficult to answer via any other means [17,[22][23][24][25], that has been used previously to both qualitatively to rank pathways and build models [26,27], and to produce quantitative estimates [24,25,28]. In the field of enteric pathogen source attribution, for example, when quantitative data are incomplete or unavailable, expert elicitation represents the only possible method for synthesizing knowledge about pathogen transmission [29]. ...

Foodborne Illnesses to Their Food Sources Using Large Expert Panels to Capture Variability in Expert Judgment

... This pool's size and selection process are discussed in various papers (Aspinall, 2010;Hora, 2007;Melnick and Everitt, 2008;Morgan, 2014;Winkler and Clemen, 2004) and references therein). Typically, PoE has 7 to 15 members (e.g., Aspinall and Cooke, 2013); however, larger panels may be appropriate for some projects (Hoffmann et al., 2007). In the case of complex assessments, sub-pools on specific topics may be foreseen. ...

Elicitation from Large, Heterogeneous Expert Panels: Using Multiple Uncertainty Measures to Characterize Information Quality for Decision Analysis
  • Citing Article
  • June 2007

Decision Analysis

... It is estimated that 20 to 30% of campylobacteriosis cases can be attributed to 1 handling, preparing, and consuming of broiler meat (EFSA BIOHAZ Panel, 2010). Probabilistic models of subtyping based on multilocus sequence typing (MLST) of C. jejuni isolates from USA and Australia suggest broilers may constitute as a reservoir, and that contributes to 50 to 80% of human cases (Mead et al., 1999;Hall et al., 2005;Hoffmann et al., 2007;Boysen et al., 2013). In addition to broilers, beef is considered the second most important source for human campylobacteriosis (Skarp et al., 2016;Wainaina et al., 2022). ...

Using Expert Elicitation To Link Foodborne Illnesses in the United States to Foods

Journal of Food Protection