ArticlePDF Available

An Assessment of Listeriosis Risk Associated with a Contaminated Production Lot of Frozen Vegetables Consumed Under Alternative Consumer Handling Scenarios

Authors:

Abstract and Figures

Frozen foods do not support the growth of Listeria monocytogenes (LM) and should be handled appropriately for safety. However, consumer trends regarding preparation of some frozen foods may contribute to the risk of foodborne listeriosis, specifically when cooking instructions are not followed and frozen products are instead added directly to smoothies or salads. A quantitative microbial risk assessment model FFLLoRA (Frozen Food Listeria Lot Risk Assessment) was developed to assess the lot-level listeriosis risk due to LM contamination in frozen vegetables consumed as a ready-to-eat food. The model was designed to estimate listeriosis risk per serving and the number of illnesses per production lot of frozen vegetables contaminated with LM, considering individual facility factors such as lot size, prevalence of LM contamination, and consumer handling prior to consumption. A production lot of 1 million packages with 10 servings each was assumed. When at least half of the servings were cooked prior to consumption, the median risk of invasive listeriosis per serving in both the general and susceptible population was <1.0 × 10−16 with the median (5th, 95th percentiles) predicted number of illnesses per lot as 0 (0, 0) and 0 (0, 1) under the exponential and Weibull-gamma dose-response functions, respectively. In scenarios in which all servings are consumed as ready-to-eat, the median predicted risk per serving was 1.8 × 10−13 and 7.8 × 10−12 in the general and susceptible populations, respectively. The median (5th, 95th percentile) number of illnesses was 0 (0, 0) and 0 (0, 6) for the exponential and Weibull-Gamma models, respectively. Classification tree analysis highlighted initial concentration of LM in the lot, temperature at which the product is thawed, and whether a serving is cooked as main predictors for illness from a lot. Overall, the FFLLoRA provides frozen food manufacturers with a tool to assess LM contamination and consumer behavior when managing rare and/or minimal contamination events in frozen foods. HIGHLIGHTS
Content may be subject to copyright.
Research Paper
An Assessment of Listeriosis Risk Associated with a
Contaminated Production Lot of Frozen Vegetables Consumed
under Alternative Consumer Handling Scenarios
CLAIRE ZOELLNER,
1
*MARTIN WIEDMANN,
2
AND RENATA IVANEK
1
1
Department of Population Medicine and Diagnostic Sciences, College of Veterinary Medicine (ORCID: https://orcid.org/0000-0002-4930-6225 [C.Z.];
https://orcid.org/0000-0001-6348-4709 [R.I.]), and
2
Department of Food Science, College of Agriculture and Life Sciences (ORCID:
https://orcid.org/0000-0002-4168-5662 [M.W.]), Cornell University, Ithaca, New York 14853, USA
MS 19-092: Received 22 February 2019/Accepted 2 September 2019/Published Online 19 November 2019
ABSTRACT
Frozen foods do not support the growth of Listeria monocytogenes (LM) and should be handled appropriately for safety.
However, consumer trends regarding preparation of some frozen foods may contribute to the risk of foodborne listeriosis,
specically when cooking instructions are not followed and frozen products are instead added directly to smoothies or salads. A
quantitative microbial risk assessment model FFLLoRA (Frozen Food Listeria Lot Risk Assessment) was developed to assess
the lot-level listeriosis risk due to LM contamination in frozen vegetables consumed as a ready-to-eat food. The model was
designed to estimate listeriosis risk per serving and the number of illnesses per production lot of frozen vegetables contaminated
with LM, considering individual facility factors such as lot size, prevalence of LM contamination, and consumer handling prior
to consumption. A production lot of 1 million packages with 10 servings each was assumed. When at least half of the servings
were cooked prior to consumption, the median risk of invasive listeriosis per serving in both the general and susceptible
population was ,1.0 310
16
with the median (5th, 95th percentiles) predicted number of illnesses per lot as 0 (0, 0) and 0 (0, 1)
under the exponential and Weibull-gamma dose-response functions, respectively. In scenarios in which all servings are
consumed as ready-to-eat, the median predicted risk per serving was 1.8 310
13
and 7.8 310
12
in the general and susceptible
populations, respectively. The median (5th, 95th percentile) number of illnesses was 0 (0, 0) and 0 (0, 6) for the exponential and
Weibull-Gamma models, respectively. Classication tree analysis highlighted initial concentration of LM in the lot, temperature
at which the product is thawed, and whether a serving is cooked as main predictors for illness from a lot. Overall, the FFLLoRA
provides frozen food manufacturers with a tool to assess LM contamination and consumer behavior when managing rare and/or
minimal contamination events in frozen foods.
HIGHLIGHTS
A tool for frozen food manufacturers to assess listeriosis risk was developed.
Scenarios of low-level L. monocytogenes in frozen vegetables did not typically result in illness.
Listeriosis cases depended on model inputs related to consumer handling and initial concentration.
Scenarios of more testing increased the probability of nding a contaminated lot and reduced risk.
Key words: Consumer behavior; Frozen foods; Listeria monocytogenes; Not ready-to-eat; Quantitative microbial risk
assessment
Frozen vegetables represent approximately a $4.4
billion market in the United States (47) and are typically
intended to be consumed after following validated cooking
instructions provided on the package. Despite the fact that
food products while frozen do not support the growth of
Listeria monocytogenes (LM), frozen foods, including
vegetables, have been associated with product recalls and
outbreaks of listeriosis globally (5, 14, 56). Variability in
temperature and bacterial destruction achieved by micro-
wave cooking has been cited as a contributing risk factor for
foodborne illness in both chilled and frozen food products
(19). The main factors contributing to variability in
microwave cooking are physical and chemical properties
of food products, the abbreviated nature of the thermal
treatment, and the potential for temperature abuse (19).
However, consumer behavior regarding handling and
preparation of frozen foods may also be contributing to
outbreaks of foodborne illness (31, 48). Although frozen
vegetables are considered not ready-to-eat (NRTE) prod-
ucts, meaning they require proper cooking before consump-
tion according to validated instructions on the package and
* Author for correspondence. Tel: 607-253-0361; E-mail:
cez23@cornell.edu.
2174
Journal of Food Protection, Vol. 82, No. 12, 2019, Pages 21742193
https://doi.org/10.4315/0362-028X.JFP-19-092
Published 2019 by the International Association for Food Protection
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
are manufactured under that assumption with regard to
microbiological criteria (MC), these products may also be
consumed as though they are RTE (i.e., without following
on-package cooking instructions to ensure food safety). For
example, trends toward healthier home-cooked foods with
fewer ingredients and innovative mixed vegetable products
and recipe ideas have changed how these frozen products
are consumed (47). Before consumption, frozen vegetables
may be thawed and/or stored at room or refrigerated
temperatures and consumed without fully cooking, for
example in the form of smoothies, salads, and dips.
Preparation of these types of dishes exposes the frozen
vegetables to conditions that may allow LM to grow, if
present, and therefore may increase the risk of listeriosis
(31). The most recent industry guidance from the U.S. Food
and Drug Administration (FDA) on conducting hazard
analysis gives specic examples of frozen RTE vegetables
(e.g., broccoli, carrots, spinach, and kale) and frozen NRTE
vegetables (e.g., potatoes, okra, and winter squash) (55).
The goal of this study was to understand to what extent
consumer preparation methods contrary to package instruc-
tions impact the risk of listeriosis.
Sampling and microbiological testing, good manufac-
turing practices, environmental pathogen monitoring, and
hazard analysis and critical control point programs are used
by the food industry to manage food safety risks, including
contamination with pathogens such as LM. MC are
internationally recognized principles used to determine the
acceptability (quality and/or safety) of a production lot (i.e.,
a quantity of food produced under the same manufacturing
conditions within a specied timeframe) (27). Sampling
plans are then designed to apply MC to a production lot and
are dened by specic components such as the number of
samples to be collected, the analytical method, and the
number of analytical units that should conform to specied
limits for the appropriate product hazard, which could be
based on presence or absence or the level of microorgan-
isms per unit of mass (i.e., concentration). Governments and
regulatory authorities often incorporate MC into regulations
and policies that will trigger mandatory remedial action,
particularly for the pertinent pathogenic microorganisms in
food products that present high risk of illness or death to
consumers (26). In the case of LM, unclear denitions of
RTE and NRTE foods can have serious implications for
applying the appropriate MC and enforcing food safety
policies. Regulations for zero tolerance of LM have been in
place in the United States since the late 1980s and apply to
only RTE food products; RTE foods regulated by the U.S.
Department of Agriculture (USDA) and FDA are consid-
ered adulterated by these agencies when any LM is detected
in either two or one 25-g sample, respectively (49). For
foods in international trade, the Codex Alimentarius
Commission (9, 10) established regulations for LM in
RTE foods depending on whether the food product permits
growth under storage conditions throughout its shelf life and
use. For RTE foods that do not support growth of LM,
,100 CFU/g in ve samples is the microbiological limit for
acceptable product lots. For RTE foods that do support
growth of LM, the zero tolerance policy applies for ve
samples per lot (i.e., absence in 25 g). Denition of an RTE
food that does not support growth of LM is determined
based on scientic studies or justication and is typically
dependent on intrinsic factors such as pH and water activity
but has also historically included frozen storage (57).
Therefore, in some countries frozen foods that contain low-
level (considering prevalence and/or concentration) con-
tamination may not be considered unsafe, because frozen
RTE products would fall under the criterion of ,100 CFU/g
and frozen NRTE products may have no regulatory limits.
Alternative preparations of frozen vegetables by consumers
may contradict these established categorizations of food
products and should be considered in risk assessments at the
global and regulatory levels and at the individual production
facility and/or lot levels. The impact of food safety practices
on public health outcomes can be maintained by routine
evaluation of individual facility and/or product risks, and as
the industry moves toward reliance on data and risk-based
decision making, risk assessments may nd more frequent
application at the manufacturing level. Consequently, to
facilitate more rational food safety management strategies
regarding LM in both frozen RTE and NRTE foods,
industry and others need better tools and data to assess
listeriosis risks associated with individual production lots of
frozen products contaminated with LM.
All of the MC described above for LM follow a two-
class sampling plan for testing products to decide whether to
release or divert a production lot at either the manufacturer
or the port of entry. Two-class sampling plans are used with
presence or absence microbiological testing, and decisions
are based on two numbers: nis the number of sample units
taken and cis the maximum number of sample units that
may exceed the acceptable microbiological specication,
i.e., absence, which is denoted by m. For example, under the
FDA zero-tolerance policy for LM, the two-class sampling
plan is described as n¼1, c¼0, and mis absence in 25 g,
whereas under Codex Alimentarius Commission guidelines
for RTE foods that do not support growth of LM, the
sampling plan would be n¼5, c¼0, and mis ,100 CFU/g
(10, 27). Finished product sampling cannot ensure zero risk
of pathogen contamination in foods and is particularly
limited in its ability to detect low-level contamination or
contamination that is heterogeneously distributed within a
production lot. The choice of nis a compromise between the
desired (or required) stringency and the resources available
and is independent of the size of the production lot; as n
decreases, the chance of accepting a contaminated lot and
the mean concentration of contamination in an accepted lot
increases. The International Commission for the Microbi-
ological Specications of Foods (ICMSF) recommends
increased numbers of samples (i.e., more than ve) for LM
in RTE foods, foods intended for highly susceptible
populations, and foods that have been epidemiologically
linked with listeriosis (23). Data obtained from applying a
two-class sampling plan can be used to estimate the
proportion of the lot that is contaminated and the mean
concentration of the microorganism within the lot, but both
values should be expressed as possible ranges or distribu-
tions (27). Finished product testing for microbial pathogens
is not required for NRTE products (because of the
assumption that the product will be fully cooked before
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2175
consumption) but may be conducted to fulll supply chain
requirements or to verify the efcacy of food safety systems
(25, 27).
Quantitative microbial risk assessment (QMRA) is a
traditional approach for estimating the probability and
severity of food safety risks through four main components:
(i) hazard identication, (ii) exposure assessment, (iii) dose-
response, and (iv) risk characterization. The results of a
QMRA provide risk estimates and comparisons for various
risk management options. QMRAs have been conducted
regarding the risk of listeriosis in RTE foods, ranking the
product categories of highest importance (17, 20, 39, 57, 60)
and focusing on deli meats at manufacturing, retail, and
consumer stages (16, 18, 35, 40, 45, 46, 61), raw milk and
cheeses (2, 32, 51), cold-smoked salmon (33, 41, 44), and
leafy green vegetables (52). Although these risk assess-
ments have appropriately focused on the public health risks
from RTE foods, similar risk assessment tools are needed
for producers of frozen NRTE food products to estimate the
potential risk to their consumers from LM contamination,
even though these products may not support LM growth
under proper storage and handling conditions and/or may
include labeling instructions calling for proper cooking
before consumption.
The QMRA presented here was designed as a decision
support tool, based on a single production lot, for estimating
listeriosis risk related to consumption of NRTE frozen foods
without following proper cooking instructions (i.e., as RTE)
and therefore calculates the risk per lot. The tool and its
interface, named FFLLoRA (Frozen Food Listeria Lot Risk
Assessment), are intended for use by risk managers, perhaps
with a suite of data and risk applications, when making
decisions based on nished product sampling in large
production lots; these results are not to be used as an
estimate for the risk of listeriosis in the overall population
as those of a traditional QMRA may be interpreted. The
FFLLoRA interface is located on the rst tab of the
spreadsheet and contains a brief description of the model,
tables for input of required and optional model parameters,
simulation options, a button to run the model, and
calculation and presentation of key results upon completion
of the simulation. The second tab of the spreadsheet
contains the QMRA model used to estimate the number of
human listeriosis cases that may be caused by a contam-
inated production lot or lots given user-dened inputs: (i)
the number of servings contained in the lot(s), (ii) the
estimated prevalence of contaminated packages in the lot
identied through testing, (iii) an estimate of the concen-
tration of contamination in the lot, and (iv) the anticipated
fraction of servings not cooked according to package
instructions. Comparison of the risk of illness between
servings consumed fully cooked versus not cooked may
provide data for other risk management options. The
FFLLoRA was parameterized for a production lot of frozen
vegetables as a demonstration of its use to assess the risk of
listeriosis.
MATERIALS AND METHODS
Model overview. The FFLLoRA is a one-dimensional,
probabilistic QMRA tool designed to be used by manufacturers
of frozen NRTE products after testing of the nished product for
LM contamination to determine the risk of listeriosis from the
production lot if released for consumption. LM contamination in
frozen vegetables was used to demonstrate the capabilities of the
FFLLoRA. Although properly stored, prepared, and consumed
frozen vegetables will not permit growth of LM and cooking
according to the label instructions will effectively eliminate LM,
certain methods used by consumers for preparation of frozen
vegetables may increase the risk of listeriosis associated with these
products. To demonstrate this and to develop an industry tool to
assess the food safety risk associated with unintended uses of
frozen foods, we present a model pathway for quantifying the
consequences of LM contamination in a lot of frozen vegetable
products under various consumer handling scenarios (Fig. 1). The
FFLLoRA is intended to be used to inform risk management of
LM at the manufacturer and assumes that the product is properly
stored, transported, and sold, thus preventing growth of LM and/or
contamination between manufacturing and home preparation.
Within the interface of the spreadsheet-based QMRA tool, users
are instructed to change input parameters, run the model, and
interpret the results (Supplemental Fig. S1). Key inputs from the
manufacturer (or the user of the model), including the number of
nished product samples taken and the results, are used to infer the
prevalence and concentration of LM contamination in the
designated lot. The change in LM concentration during prepara-
tion and the resulting contamination level in the frozen vegetable
serving at the time of consumption are determined according to the
modeled consumer handling scenarios. The endpoint of the risk
assessment tool is a prediction of the risk of invasive listeriosis per
serving and the number of invasive listeriosis cases attributable to
a simulated production lot for two subpopulations: the susceptible
and healthy (or general) populations. For clarity of description of
the model and its predictions, an arbitrary scenario was chosen to
serve as the baseline model. This baseline scenario assumed that
all servings of frozen vegetables were consumed without
following the labeled cooking instructions (hereinafter referred
to as not cooked or RTE) with a 50%chance of being thawed at
room temperature. Other scenarios were evaluated to address
uncertainty in consumer behavior and model inputs, which
involved changing specic parameters of the baseline model and
documenting their impact. Details of all model variables, baseline
parameter values, and their sources are provided in Table 1.
Hazard identication. LM is a gram-positive, nonspore-
forming bacterial pathogen that can survive freezing and drying
and can grow at temperatures of 0.4 to 458C, pH 4.3 to 9.4, water
activity .0.92, and salt concentration ,10%(24). LM is the
causative agent of listeriosis in humans, with symptoms ranging
from gastrointestinal upset, malaise, and mild fever to meningitis,
encephalitis, and septicemia depending on the hosts immune
function. Worldwide, LM is estimated to cause more than 23,000
cases of invasive listeriosis annually, with an estimated 20 to 30%
case fatality rate in susceptible populations (13). Although LM is
common in natural environments, human listeriosis is primarily a
foodborne illness caused by consumption of LM in food (27).
Because proper heat treatment (.708C) effectively eliminates
Listeria and LM can survive and grow at refrigeration tempera-
tures, postprocessing contamination with LM is a main concern in
refrigerated and frozen food products that do not undergo further
cooking by the consumer before consumption, such as deli meats,
cold-smoked seafood, and fresh cheeses. Recent outbreaks and
product recalls due to LM contamination have been associated
with frozen food products including frozen vegetables between
2015 and 2018 (5, 14, 56). Although the concentration of LM in
2176 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
packages of frozen vegetables was ,100 CFU/g in an outbreak in
Europe from 2015 to 2018 and the frozen vegetable products were
labeled with cooking instructions, 47 cases of invasive listeriosis
and nine deaths were attributed to consumption of frozen corn,
mixed vegetables, spinach, and green beans (14). In a survey of
frozen vegetables manufactured in Portugal, the prevalence of LM
was 0 to 22.6%among frozen sliced red peppers, eggplant, peas,
broccoli, sliced zucchini, and sliced green peppers (36).
Exposure assessment: prevalence and concentration of
LM in a contaminated production lot. The FFLLoRA was
developed to assess the public health impact of product
contamination immediately after completion of the manufacturing
process, so the formulas for the distribution of prevalence and
concentration of LM contamination in a production lot were based
on the nished product sampling immediately after manufacturing
before additional contamination (e.g., in the consumer kitchen)
could have occurred. Because the LM in nished products in the
United States is not typically quantied, limited data are available
on the mean concentration of LM in frozen food products other
than those associated with an outbreak. Therefore, we relied on the
level of assurance that certain nished product sampling plans
provide for discriminating between poor and acceptable lots,
following the notion that testing more samples reduces the risk of
accepting a contaminated lot and also reduces the risk of accepting
a lot with a high mean concentration of contamination (27). For
demonstration purposes, we assumed that the number of packages
in a single production lot of frozen vegetables, n
lot
, was 1 million
and the number of servings per package, n
ser
, was 10, each
weighing 140 g (w).
In the simulations performed, we assumed that the within-lot
sampling plan was designed with appropriate consideration of MC
for LM in RTE foods recommended by the ICMSF (i.e., as
described in the introduction: c¼0 and mis absence in 25 g). This
assumption was made given the modeled scenario of NRTE foods
consumed as RTE and based on the suggested corrective action of
implementing product test and hold following repeated detection
of Listeria spp. on a food contact surface (53, 54). Design of
FIGURE 1. Schematic of the risk assessment model for not ready-to-eat frozen food products consumed without cooking according to
validated instructions on the package (not cooked). The dark shaded box indicates that the model did not consider changes in
contamination with Listeria monocytogenes between manufacturing and at-home use by consumers. Light shaded boxes enclose user
inputs. The dashed box encloses the inputs used in simulation of consumer handling scenarios within the risk assessment process model.
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2177
TABLE 1. Spreadsheet model and variables for Listeria monocytogenes (LM) and a production lot of frozen vegetables with parameter values and their sources for the baseline (exponential)
model
Symbol Cell Variable Value, formula Unit Source, reference
Input data
n
lot
C6 No. of packages per lot 1,000,000 Packages User input
a
n
ser
C7 No. of servings per package 10 Servings User input
wC8 Serving size (g) 140 Gram User input
sC9 No. of servings per meal
b
1 Servings User input
P
S
C10 Proportion of lot consumed by susceptible population
b
0.2 No unit 42, 57
n
test
C11 Sample size (no. of packages tested from lot)
b
5 Packages User input
n
pos
C12 No. of LM-contaminated packages among samples
b
1 Packages User input
p
lot
C13 Prevalence of LM-contaminated packages in lot Beta npos þ1;ntest npos þ1
 No unit Calculated
μ
lot
C14 Arithmetic mean of LM concn 0.53 log CFU/g 26
σ
lot
C15 SD of LM concn 0.8 log CFU/g 59
μ
pack
C16 LM concn in a contaminated package Normal llot;rlot
ðÞ log CFU/g Calculated
p
cook
C17 Probability that a serving is cooked
b
0 No unit User input
p
thaw
C18 Probability that a serving is thawed at room temp
b
0.5 No unit User input
Exposure
kC20 Presence of within-package clustering (yes ¼1; no ¼0) 1 No unit User input
bC21 Within-package clustering parameter
b
1 No unit 37
N
pack
C22 No. of LM cells in a contaminated package 10lpack 3nser 3wCFU Calculated
p
serv
C23 Proportion of LM cells from a contaminated package in a
serving
1=nser if k¼0
Beta b;b3nser 1ðÞ½if k¼1
No unit 37
π
0
C24 Probability of zero cells in serving Cb3nser
ðÞCNpackþb3nser 1ðÞ
½
Cb3nser1ðÞ½Cb3nser þNpack
ðÞ No unit 37
N
serv
C25 No. of LM cells in a serving 1 þBinomial Npack;pserv
 CFU Calculated
μ
serv
C26 LM concn in contaminated serving log Nserv=wðÞ log CFU/g Calculated
Consumer preparation of a serving
C
01
C28 Serving cooked (yes ¼1; no ¼0) Bernoulli pcook
ðÞ No unit Calculated
A
01
C29 Serving thawed (at room temp ¼1; in refrigerator ¼0) Bernoulli pthaw
ðÞif C01 ¼0
0if C01 ¼1
No unit Calculated
T
frig
C30 Refrigerated storage temp
c
Laplace 4:06;2:31ðÞ 8C43
T
room
C31 Room storage temp
d
Pert 14;24;42ðÞ 8C61
t
frig
C32 Refrigerated storage time
e
Lognorm 2;1ðÞ,0tfrig 96 hour 61
t
room
C33 Room storage time
f
Lognorm 0:9;1:331ðÞ,0troom 14 hour 61
LC34 Log reduction of LM due to proper cooking
g
0if C01 ¼0
Normal 7;0:5ðÞif C01 ¼1
log CFU/g (11, 34)
Growth
LPD G28 Lag-phase duration 19:77 2:164 Tfrig

þ0:0673 Tfrig

2if A01 ¼0
19:77 2:164 Troom
ðÞþ0:0673 Troom
ðÞ
2if A01 ¼1
(hour 31
2178 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
TABLE 1. Continued
Symbol Cell Variable Value, formula Unit Source, reference
t
lag
G29 Lag time
h
LPD if C01 ¼0
0if C01 ¼1
hour This model
aG30 Growth model constant 0.0225 =(log CFU/g/h/8C) 31
T
min
G31 Growth model, theoretical minimum temp for LM growth 0.791 8C31
EGR G32 Exponential growth rate of LM ½a3Tfrig Tmin

2if A01 ¼0
a3Troom Tmin
ðÞ½
2if A01 ¼1
(log CFU/g/h 31
δG33 Change in LM concn during 1 h of storage
h
EGR if C01 ¼0
0if C01 ¼1
log CFU/g This model
t0
tot
G34 Storage time minus lag time max 0;tfrig tlag

if A01 ¼0
max 0;troom tlag

if A01 ¼1
hour Calculated
t
tot
G35 Total growth time
h
t0
tot if C01 ¼0
0if C01 ¼1
hour This model
GG36 Growth of LM prior to consumption δ3t
tot
log CFU/g Calculated
δ
tot
G37 Net change in LM concn due to handling Gif C01 ¼0
Lif C01 ¼1
log CFU/g Calculated
μ
con
C35 LM concn at consumption min 8;lservþdtot
ðÞ log CFU/g Calculated
dC36 LM dose consumed Poissonð10lcon 3w3sÞCFU Calculated
DC37 Log LM dose consumed
i
log dþ1ðÞ log CFU Calculated
Dose-response (D-R) by model and subpopulation
r
G
G43 Exponential model (exp), probability for one LM cell to
cause illness in the general population
2.37E14 No unit 58
r
S
G44 Exp, probability for one LM cell to cause illness in the
susceptible population
1.06E12 No unit 58
β
G
G45 Weibull-Gamma model (WG), Gamma distribution
parameter describing heterogeneity of illness from LM
in the general population
1.82Eþ15 No unit 2
β
S
G46 WG, Gamma distribution parameter describing
heterogeneity of illness from LM in the susceptible
population
9.55Eþ10 No unit 2
Probability of invasive listeriosis for two D-R models and subpopulations
P
G,exp
C39 Exp, general population 1 erG3dNo unit Calculated
P
S,exp
C40 Exp, susceptible population 1 erS3dNo unit Calculated
P
G,WG
C41 WG, general population 1 ð1þd2:14=bGÞ0:25 No unit Calculated
P
S,WG
C42 WG, susceptible population 1 ð1þd2:14=bSÞ0:25 No unit Calculated
Probability of invasive listeriosis per serving for two D-R models and subpopulations
P
serG,exp
C44 Exp, general population PG;exp 3plot 31p0
ðÞ No unit Calculated
P
serS,exp
C45 Exp, susceptible population PS;exp 3plot 31p0
ðÞ No unit Calculated
P
serG,WG
C46 WG, general population PG;WG 3plot 31p0
ðÞ No unit Calculated
P
serS,WG
C47 WG, susceptible population PS;WG 3plot 31p0
ðÞ No unit Calculated
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2179
TABLE 1. Continued
Symbol Cell Variable Value, formula Unit Source, reference
Lot consumption
n
G
G49 No. of exposed consumers in general population nlot 3nser=sðÞ31PS
ðÞ Consumers Calculated
n
S
G50 No. of exposed consumers in susceptible population nlot 3ðnser=sÞnGConsumers Calculated
Predicted no. of illnesses per lot for two D-R models
M
G,exp
C49 No. of illnesses under exp model in general population Binomial nG;PserG;exp
 Illnesses Calculated
M
S,exp
C50 No. of illnesses under exp model in susceptible
population
Binomial nS;PserS;exp
 Illnesses Calculated
M
exp
C51 Total no. of listeriosis cases under exp model MG;exp þMS;exp Illnesses Calculated
I
exp
C52 Any illness occurring from contaminated lot, exp model
(yes ¼1; no ¼0)
1if Mexp 1
0otherwise
No unit Calculated
M
G,WG
C53 No. of illnesses under WG model in general population Binomial nG;PserG;WG
 Illnesses Calculated
M
S,WG
C54 No. of illnesses under WG model in susceptible
population
Binomial nS;PserS;WG
 Illnesses Calculated
M
G
C55 Total no. of listeriosis cases under WG model MG;WG þMS;WG Illnesses Calculated
I
WG
C56 Any illness occurring from contaminated lot, WG model
(yes ¼1; no ¼0)
1if MWG 1
0otherwise
No unit Calculated
a
User input that is a xed value chosen by the user according to the users situation. A value has been set here for demonstration purposes.
b
Model parameter that was evaluated in scenario analysis.
c
Median (5th, 95th percentile): 4.08 (1.3, 9.4).
d
Median (5th, 95th percentile): 25 (17, 34).
e
Median (5th, 95th percentile): 11.5 (1.4, 36).
f
Median (5th, 95th percentile): 3.2 (0.25, 10).
g
Median (5th, 95th percentile): 7.0 (6.2, 7.8).
h
Equation returns zero for cooked servings, meaning that the parameter was not applicable.
i
Value of 1 was added to the dose (d) to allow log transformation when the dose was 0 CFU.
2180 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
nished product testing schemes according to the ICMSF is a
standard approach used in the food industry. Based on zero-
tolerance regulations in the United States, we assumed that
nished product testing was based on application of a two-class
presence or absence test for detection of LM. The baseline number
of packages tested per lot, n
test
, was 5, but the impact of taking 10,
20, or 60 samples per lot was also tested in scenario analysis
(Table 2). Based on n
test
, the mean LM concentration of the lot,
μ
lot
, was estimated from the ICMSFs sampling tool (26).We
assumed that LM contamination was not homogeneously distrib-
uted in a contaminated lot and a standard deviation (SD), σ
lot
,of
0.8 log CFU/g was used according to the ICMSF recommendation
when no specic product information is available, for solid foods,
or when contamination may be restricted to the surface of the food
(59). The ICMSFs sampling tool can be used to assess the
performance of a simulated sampling plan, describing the lowest
concentration of an organism that will be detected with a particular
plan and with a particular certainty (26). For example, the ICMSF
sampling tool can provide an estimate of the mean concentration
of contamination in a lot that a chosen sampling plan will detect
and thus reject from release with .95%probability (or accept
with ,5%probability). Because of lack of relevant information on
LM in frozen vegetables, we used this estimated concentration
from the ICMSF sampling tool; however, the user could utilize the
facilitys own estimates of the LM concentration (the mean and
SD) if available. The LM concentration in a contaminated package
(log CFU per gram), μ
pack
, was t with a Normal distribution with
mean μ
lot
and SD σ
lot
(Fig. 2). With n
ser
and w, μ
pack
was used to
derive the number of cells (CFU) in a contaminated package,
N
pack
.
From n
test
and the number of LM-contaminated packages
among the samples, n
pos
, the prevalence of contaminated packages
in the lot, p
lot
, was modeled with a Beta distribution (Table 1 and
Fig. 3). For calculation of the prevalence of contaminated packages
in the baseline model, we considered a situation where n
pos
¼1, but
scenarios where n
pos
¼0 or 2 were also considered during scenario
analysis (Table 2). The model for partitioning of the package of
frozen vegetables into portions for consumption (i.e., servings)
considered clustering of contamination due to clumping or
aggregation of cells in the solid frozen vegetables (37). A Beta-
Binomial distribution was used to model the number of LM cells in
a serving, N
serv
, originating from a contaminated package:
TABLE 2. Parameters and values considered in scenario analysis in the Listeria monocytogenes (LM) risk assessment model for a lot of
frozen vegetables
Symbol Parameter Baseline value Alternative values
sNo. of servings per meal 1 0.5, 2
P
S
Proportion of lot consumed by susceptible population 0.2 0.1, 0.4
n
test
Sample size 5 10, 20, 60
n
pos
No. of LM-contaminated packages among samples 1 0, 2
p
cook
Probability that a serving is cooked 0 1, 0.95, 0.75, 0.5
p
thaw
Probability that a serving is thawed at room temp 0.5 0, 1
bWithin-package clustering parameter
a
1 0.01, 0.1, 5, 10
a
Values of bapproaching 0 indicate increased clustering or heterogeneity in contamination within a package. Values of bapproaching
innity indicate increasingly homogeneous contamination within a package.
FIGURE 2. Probability distributions of the Listeria monocyto-
genes concentration in a contaminated package, μ
pack
(log CFU
per gram), estimated from ice cream outbreak data (solid line) and
predicted for frozen foods considered in this risk assessment based
on accepted lots under a zero-tolerance two-class sampling plan
(i.e., n
test
;c
¼
0, mis absence in 25 g), where n
test
¼
5 (solid light
shaded line), 10 (dashed light shaded line), 20 (dotted shaded
line), or 60 (dashed-dotted shaded line).
FIGURE 3. Predicted probability distributions for prevalence of
contaminated packages in a lot of frozen vegetables, p
lot
, for
different numbers of packages tested, n
test
, and one package
testing positive for Listeria monocytogenes, n
pos
¼
1, where n
test
¼
5 (solid shaded line), 10 (dashed light shaded line), 20 (dotted
shaded line), or 60 (dashed-dotted shaded line).
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2181
Nserv ¼1þBinomial Npack;pserv
 ð1Þ
pserv ¼1=nser if k¼0
Beta b;b3nser 1ðÞ½if k¼1
ð2Þ
where p
serv
is the probability that an LM cell from a contaminated
package is in a serving, depending on the presence of clustering
within a package (k¼0, no clustering; k¼1, clustering). In
equation 2, Beta refers to the Beta distribution, and b(b.0) is the
within-package clustering parameter. No clustering occurs when b
approaches innity, and maximum clustering occurs when b
approaches zero (37). In the absence of information, as a baseline,
we assumed that b¼1 to signify a moderate level of clustering (37),
but this parameter was also tested in scenario analysis (Table 2).
The partitioning of a package into servings allowed for the
possibility that some servings from a contaminated package would
not be contaminated. The probability that a serving from a
contaminated package contains no cells, π
0
, was calculated as
p0¼Cb3nser
ðÞCNpack þb3nser 1ðÞ

Cb3nser 1ðÞ½Cb3nser þNpack

ð3Þ
where Γis the gamma function (37). In the @Risk spreadsheet
model, the calculation was implemented using the GammaLn
Excel built-in function. Depending on the number of servings per
meal, s, N
serv
was scaled to obtain the level (dose) of LM in the
meal consumed. Although a simplication, we assumed that one
serving was used per consumer in the baseline model. This was a
necessary assumption because no data on consumption patterns for
frozen vegetables were available and research continues to be
conducted on modeling of dose-response with repeated exposure.
However, alternative values for the number of servings per meal
(0.5 or 2) were evaluated in scenario analysis (Table 2).
Exposure assessment: consumption. According to the
cooking instructions on frozen vegetable packages, proper
preparation before consumption includes microwave or stovetop
cooking. The distribution of the overall log CFU per gram
reduction, L, in LM due to proper cooking according to packaging
instructions (exposure to a temperature .708C for 2 min) was
given as L~Normal(7, 0.5) (11, 34). Of interest in this risk
assessment was estimation of the effect of not cooking the frozen
vegetables before consumption and instead thawing the serving at
either refrigeration or room temperature. Because of the lack of
consumer behavior data and to demonstrate the capabilities of the
FFLLoRA, we tested ve hypothetical probabilities of cooking a
serving (p
cook
¼1, 0.95, 0.75, 0.50, and 0) (Table 2). For servings
that were not cooked, we tested the difference between thawing
the serving of frozen vegetables (p
thaw
) at room temperature and
thawing in the refrigerator. For the baseline model, we assumed
equal probability of thawing at room and refrigeration tempera-
tures (p
thaw
¼0.50). For example, when p
cook
¼0.95 and p
thaw
¼
0.50, 5%of servings were simulated as not cooked and of those
servings, 50%were simulated as being thawed at room
temperature and the other 50%were simulated as being thawed
at refrigeration temperature. Both p
thaw
¼0 and p
thaw
¼1 were
tested during scenario analysis (Table 2). Because of the lack of
quantitative data about variability of consumer behavior, for
sensitivity analysis both p
cook
and p
thaw
were modeled with a
Uniform distribution from 0 to 1 so that the effect of not cooking
on the risk of illness could be assessed across the full range of
possible situations occurring with equal probabilities.
Home refrigeration and countertop temperatures (T
frig
and
T
room
) and storage times (t
frig
and t
room
) from previous surveys and
risk assessments conducted for RTE products and deli meats (43,
61) were utilized to model the practice of thawing a serving (at
refrigeration or room temperature) (Table 1). Time and temper-
ature data for storage scenarios did not include correlation
between time and temperature, which was consistent with other
risk assessments and studies (12, 43). Growth of LM was modeled
using the exponential growth rate and a two-phase linear model for
the lag-phase duration (31):
ffiffiffiffiffiffiffiffiffi
EGR
p¼aT
iTmin
ðÞ ð4Þ
LPD ¼19:77 2:164ðTiÞþ0:0673ðTiÞ2ð5Þ
where EGR is the exponential growth rate of LM (log CFU/g/h), T
i
is the storage temperature (8C) for i¼frig or room, ais the growth
model constant, T
min
is the theoretical minimum temperature for
LM growth (8C), and LPD is the lag-phase duration (h). For frozen
vegetables, aand T
min
were parameterized as 0.0225 and
0.7918C, respectively (31). The total time LM growth was
permitted in the product before consumption, t
tot
,wasthe
difference between the time thawed and the lag time. The EGR
was multiplied by t
tot
to determine the concentration of LM at the
time of consumption. The maximum LM concentration allowed in
a serving was 8 log CFU/g (31).
Hazard characterization. The risk of invasive listeriosis on
a per serving basis was predicted from the estimated dose of
contamination consumed in a single serving of frozen vegetables
using two dose-response models (exponential and Weibull-
Gamma) and parameter values to capture differences among
dose-response models and susceptibility in subpopulations of
consumers (Fig. 4). The two subpopulations that were considered
were the population of consumers susceptible to LM infection due
to conditions such as pregnancy, compromised immunity, and old
age (susceptible population) relative to the rest of the population
(general) (58, 60). As a baseline, the proportion (P) of the lot
consumed by the susceptible population (S) was set at 20%of
consumers (P
S
¼0.2), which has been used in other LM dose-
response and risk assessment models for U.S. and global
populations (3, 8, 57). Various values for P
S
were also tested in
scenario analysis (Table 2), and the model user can input this
information when known for a particular production lot.
The exponential dose-response model was used in baseline
simulations with the outcome of morbidity, i.e., the probability of
invasive or symptomatic listeriosis, in each subpopulation. The
exponential dose-response model and parameters were derived
from exposure data and annual number of illnesses estimated in
the United States (3) and has been used in previous risk
assessment models of LM contamination in foods (58, 60).
Therefore, the probability of invasive listeriosis was determined
according to:
Pj;exp ¼1erjdð6Þ
where P
j,exp
is the probability of invasive listeriosis in subpop-
ulations j¼general (G), susceptible (S) given an individual
exposure to LM from a dose d(CFU), and the probability that one
LM cell will cause illness in the general and susceptible
populations, r
G
¼2.37E14 and r
S
¼1.06E12, respectively
(58, 60). According to these parameters in the exponential dose-
response model, the median illness probability per serving of 2.4 3
10
11
for the general population at a low dose level of 10
3
CFU
per serving corresponds to one illness in approximately 42 billion
servings. The median illness probability per serving of 1.1 310
9
for the susceptible population at 10
3
CFU per serving corresponds
to one illness in approximately 943 million servings. At the higher
2182 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
dose of 10
9
CFU per serving, the risks per serving are 2.4 310
5
and 1.1 310
3
in the general and susceptible populations,
respectively, corresponding to one illness in approximately 42,195
and 944 servings, respectively. In the FFLLoRA, this exponential
dose-response curve was used to estimate the probability of illness
from contaminated frozen vegetables based on the LM dose
resulting from contamination during manufacturing and prepara-
tion by the consumer.
The second dose-response model used was the exible
Weibull-Gamma model, where the probability of illness is
described by
Pj;WG ¼11þðdcÞ=bj
hi
að7Þ
where P
j,WG
is the average probability of invasive listeriosis in
subpopulation jfor an individual exposure to LM from a dose d.
The parameters αand βare parameters of a Gamma distribution
describing the host-pathogen heterogeneity upon exposure to a
single LM cell. The parameter γdetermines the shape of the
individual dose-response curve, where values of γ.2 result in
sigmoidal curves associated with extremely low probabilities of
illness at low doses and a rapid increase in probability at doses
near the median dose level (17). When γ¼1, the Weibull-Gamma
model reduces to the Beta-Poisson dose-response model. The
parameters αand γwere parameterized irrespective of population,
with α¼0.25 and γ¼2.14, whereas β
j
differed for the general and
susceptible populations: β
G
¼1.82 310
15
and β
S
¼9.55 310
10
,
respectively (2, 17, 33). According to the baseline parameters of
the Weibull-Gamma dose-response model, the median illness
probability per serving of 3.6 310
10
for the general population at
the 10
3
CFU per serving dose corresponds to one illness in
approximately 2.77 billion servings. The median illness probabil-
ity per serving of 6.9 310
6
for the susceptible population at the
10
3
CFU per serving dose corresponds to one illness in
approximately 145,235 servings. At the higher dose of 10
9
CFU
per serving, the risks per serving are 0.89 and 0.99 in the general
and susceptible populations, respectively, corresponding to an
almost 100%probability per serving in both populations. The
Weibull-Gamma model provides a more conservative prediction of
risk per dose than does the exponential model (Fig. 4). In the
FFLLoRA, this Weibull-Gamma dose-response curve was used to
provide an alternative estimate for the probability of illness from
contaminated frozen vegetables based on the LM dose resulting
from contamination during manufacturing and preparation by the
consumer.
Risk characterization. The key results provided by the
FFLLoRA include two main outcomes associated with the
simulated production lot: (i) the probability of illness per serving,
P
ser,ji
(for j¼general and susceptible populations and i¼
exponential and Weibull-Gamma models) and (ii) the predicted
total number of listeriosis illnesses, M
i
(for i¼exponential and
Weibull-Gamma models) (Table 1). For comparison purposes, we
present these outcomes separately for each dose-response model.
The probability of illness per serving, P
ser,ji
,was calculated using
(i) the prevalence of contaminated servings per lot, p
lot
, (ii) the
probability that a serving from a contaminated package was not
contaminated because of clustering, π
0
, and (iii) the output of the
corresponding dose-response model (Table 1). M
i
was calculated
as the sum of illnesses in the susceptible (M
S
) and general (M
G
)
populations, which were determined using the Binomial distribu-
tion based on the number of exposed consumers per lot in the
general (n
G
) and susceptible (n
S
) populations and the correspond-
ing P
ser,ji
(Table 1).
The FFLLoRA spreadsheet-based model was implemented
using @Risk 7.5 software (Palisade Co., Neweld, NY) and is
available upon request from the corresponding author. On the
model interface, users are prompted to provide values for required
and optional model parameters before setting and running a
simulation (Fig. S1). For demonstration purposes, the baseline
model simulations presented here utilized parameter values
previously dened (Table 1) and consisted of 100,000 iterations,
allowing for convergence of the 99th percentile of the main
outcome variable, the total number of illnesses per lot.
Probabilities estimated for the risk of invasive listeriosis per
serving that were smaller than 1 310
16
were automatically set to
zero by the @Risk software because of oating point limitations
and were therefore interpreted as zero or negligible risk. Latin
Hypercube sampling was used to sample input variables from their
corresponding distributions and determine the predicted number of
FIGURE 4. Comparison of invasive liste-
riosis dose-response models, where P
G,exp
and P
S,exp
are the exponential model
probabilities for the general and suscepti-
ble populations (black and shaded solid
lines, respectively) and P
G,WG
and P
S,WG
are the Weibull-Gamma model probabili-
ties for these populations (black and
shaded solid squares, respectively). The
exponential model was chosen for the
baseline model. Inset: a comparison of
the dose-response models plotted on a log
scale to demonstrate differences in risk at
low levels of L. monocytogenes exposure,
as often found in simulated production lots
of frozen vegetables.
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2183
listeriosis cases per lot. Spearman rank correlation coefcient (r)
was used to identify the inputs for which key outcomes were most
sensitive. Classication tree analysis was conducted using the
rpart package in R (50) to identify the series of contamination and
consumption events that led to occurrence of at least one listeriosis
illness in an iteration of a simulated production lot. In
classication tree analysis, the Gini index was utilized as a
measure of node impurity during partitioning, and 20 was chosen
as the minimum number of data points in a branch. In pruning the
classication tree to its optimal size, we used 10-fold cross-
validation to choose the tree with the smallest misclassication
error based on the 1 standard error rule (28).Predictive
performance of the nal classication tree was evaluated through
the 10-fold cross-validation on the overall accuracy in terms of the
proportion of simulated lots with correctly classied presence or
absence of illness and on measures of sensitivity, specicity, and
positive and negative predictive values.
Model validation. The thoroughly reported contamination
data from the 2015 ice cream outbreak (4, 7, 42) provided a useful
check that our risk process model, particularly exposure
assessment, provided sensible estimates for listeriosis and were
used for validation of the structure of the model and the upper tail
of the LM concentration distribution. Although ice cream is not
the same as frozen vegetables, this outbreak data set contains the
only enumeration data from a frozen product that we could nd
and represents an instance where a traditionally low-risk product
was not consumed as originally intended (i.e., a milkshake left to
thaw for a period of time), resulting in cases of listeriosis. Data for
99%prevalence (2,307 of 2,320 samples were positive for LM)
and the arithmetic mean LM concentration (0.9 log CFU/g) within
seven production lots of the ice cream were used as inputs. The SD
for the mean LM concentration was set at 0.3 log CFU/g,
consistent with homogeneous contamination within the lot (27,
37), and within-package clustering was not modeled because the
ice cream was packaged in single-serve portions and two entire
packages were used in preparation of a milkshake for consumption
by a susceptible individual. Because contamination may not be
homogeneous, we tested the impact of this assumption to
determine the best t with the empirical data (data not shown),
using both the literature value of 1.9 log CFU/g and literature
recommendations of 0.8 log CFU/g for the SD of contamination
not homogeneously distributed in a production lot (27, 59). Proper
ice cream handling was modeled as consumption while still
frozen, with no change in the contamination level (neither
prevalence nor concentration) between manufacturing and con-
sumption. For preparation or handling under conditions other than
freezing, LM growth was modeled using the growth model
parameters a¼0.019 and T
0
¼3.6388C with the maximum
population density as 7 log CFU/g (48). The same aand T
0
parameters were used to determine the lag-phase duration from the
equation LPD ¼1/[a(TT
0
)
2
](48). The results of this validation
are described in the Resultssection.
RESULTS
Demonstration of the FFLLoRA tool in evaluation
of the illness risk associated with production lots of
frozen vegetables. The estimated median LM dose
consumed, D, from a 140-g serving of frozen vegetables
originating from a contaminated lot was 0 to 1.5 log CFU
depending on the probability that before consumption the
serving was cooked according to the package instructions
(Fig. 5). Under the exponential dose-response model,
baseline model estimates for the median risk (probability)
FIGURE 5. Predicted Listeria monocytogenes dose consumed, D,
per 140-g serving of frozen vegetables given contamination during
manufacturing and various hypothetical probabilities that a
serving is cooked prior to consumption, p
cook
. Violin plots show
the kernel density around the median, interquartile range, and 5th
to 95th percentile of 100,000 simulations.
TABLE 3. Model predictions for risk of invasive listeriosis per serving
a
due to consumption of frozen vegetables from a production lot
contaminated with Listeria monocytogenes in general and susceptible populations depending on the probability that a serving is cooked
prior to consumption, p
cook
p
cook
General population Susceptible population
Median 95%99%99.9%Median 95%99%99.9%
1.0 0
b
0000000
0.95 0 0 1.2 310
12
2.1 310
11
0 0 5.4 310
11
9.4 310
10
0.75 0 1.2 310
12
1.1 310
11
1.2 310
10
0 5.6 310
11
4.8 310
10
5.4 310
9
0.5 0 3.5 310
12
2.3 310
11
2.7 310
10
0 1.6 310
10
1.0 310
9
1.2 310
8
0.0 1.8 310
13
8.2 310
12
4.8 310
11
5.8 310
10
7.8 310
12
3.7 310
10
2.2 310
9
2.6 310
8
a
Predictions from the exponential dose-response model with predicted percentiles (95th, 99th, 99.9th) for risk of invasive listeriosis per
serving from baseline simulations.
b
Probabilities smaller than 1 310
16
were automatically set to 0 by the @Risk software because of oating point limitations and are
interpreted as negligible risk.
2184 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
of invasive listeriosis per serving in the general and
susceptible populations were negligible (dened as ,1.0
310
16
) for p
cook
0.5. When p
cook
¼0, the median
predicted risks per serving were 1.8 310
13
and 7.8 310
12
in the general and susceptible populations, respectively
(Table 3). Under the exponential dose-response model and
for the number of exposed consumers modeled, these risk
levels did not result in any predicted cases of invasive
listeriosis for the production lot (M¼0; 5th to 95th
percentile: 0 to 0). For the Weibull-Gamma model, the
predicted baseline risk was a median of zero total listeriosis
cases per lot (5th to 95th percentile: 0 to 6); however, a few
iterations indicated the potential for higher case counts (i.e.,
99th to 99.9th percentile: 244 to 39,579 of 10,000,000
servings). Not surprisingly, in these extreme iterations the
majority (99%) of the cases per lot occurred in the
susceptible population. In iterations in which p
cook
¼0
and at least one listeriosis case was predicted under the
Weibull-Gamma dose-response model (i.e., M
WG
1), the
median LM concentration in the lot was 0.50 log CFU/g,
and the median prevalence of contaminated packages was
30%(slightly greater than the median concentration of
0.53 log CFU/g and median prevalence of 26%in lots with
no predicted illness).
For other scenarios when p
cook
0.95, the Weibull-
Gamma dose-response model indicated zero total cases per
lot, except for a few iterations (i.e., top 5%of iterations
with the highest risk) (Table 4). Because each of the
100,000 iterations represented one production lot (each
with 1,000,000 servings), the 95th percentile result can be
interpreted as the predicted number of cases in 5%of
production lots (1 of every 20 lots). Similar interpretations
can be made for the other percentiles provided in Table 4
(i.e., 99%provides predicted cases in 1 of 100 lots and
99.9%provides predicted cases in 1 of 1,000 lots). The
predicted number of cases per lot under the Weibull-
Gamma dose-response model may seem unreasonably high
for frozen vegetables, may not be supported by epidemi-
ological data, and may emphasize the results within the
context of the baseline model parameters, including (i)
p
cook
¼0 (the whole lot was consumed without cooking)
and (ii) p
thaw
¼0.5 (50%of product was thawed at room
temperature), which could be considered a worst-case
scenario for consumption of frozen vegetables without a
proper kill step.
Sensitivity analysis. When p
cook
and p
thaw
were
allowed to vary within Uniform(0, 1), the impact of
variation in the consumer preparation method on dose
consumed was revealed. Sensitivity analysis at the univar-
iate level (Fig. 6) conrmed that cooking the serving (i.e.,
C
01
¼1) reduced (i.e., was negatively correlated with) the
dose consumed, whereas increasing the initial contamina-
tion level from manufacturing (N
serv
, CFU) increased (i.e.,
was positively correlated with) the dose consumed. The
lengths of storage time at refrigeration temperature (t
frig
)
FIGURE 6. Sensitivity of mean dose of Listeria monocytogenes (LM) consumed from frozen vegetables, d, to model input parameters,
where C
01
is whether the serving is cooked (yes
¼
1; no
¼
0), Lis the log reduction of LM due to proper cooking, N
serv
is the number of
LM cells in a serving from a contaminated package, t
room
is the time stored at room temperature, A
01
indicates how the serving is thawed
(at room temperature
¼
1; in fridge
¼
0), T
room
is room temperature, t
frig
is the time stored in the refrigerator, and T
frig
is the temperature
of the refrigerator. The tornado plot ranks the correlation between input parameters and the predicted dose consumed when the
probability that a serving is cooked, p
cook
, and the probability that a serving is thawed at room temperature, p
thaw
, were modeled as
~Uniform(0, 1), which made it possible to consider for this analysis their full range of possible parameter values. Bars show the
Spearman rank correlation coefcient for the input and dose consumed; positive values indicate a positive correlation (i.e., increase in
input value increases dose consumed), and negative values indicate a negative correlation (i.e., increase in input value decreases dose
consumed).
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2185
and room temperature (t
room
) were not correlated (Spearman
r0.02) with LM dose nor was the temperature at both
refrigeration (T
frig
) and ambient (T
room
) storage conditions
(absolute value of Spearman r¼0.01).
The classication tree analysis captured the multifac-
torial nature of the listeriosis risk, where the presence of at
least one illness per lot could be explained by groups of co-
occurring risk factors. The nal tree included strong
TABLE 4. Model predictions for the number of invasive listeriosis cases per production lot of frozen vegetables contaminated with
Listeria monocytogenes depending on the probability that a serving is cooked prior to consumption, p
cook
, and the dose-response model
a
p
cook
Exponential Weibull-gamma
Median 95%99%99.9%Median 95%99%99.9%
1.00000 000 0
0.95 0 0 0 0 0 0 0 42
0.75 0 0 0 0 0 0 11 1,419
0.5 0 0 0 0 0 1 50 9,819
0.0 0 0 0 0 0 6 244 39,579
a
Because each of the 100,000 iterations represents one production lot, the 95th percentile can be interpreted as the predicted number of
cases for 5 of 100 (i.e., 1 of 20) production lots with the baseline contamination level (prevalence and concentration) and according to
consumer handling scenarios. Similar interpretations can be made for the other percentiles (99%provides predicted cases for 1 of 100
lots, 99.9%provides predicted cases for 1 of 1,000 lots). The total number of servings per lot was assumed to be 1,000,000.
FIGURE 7. Classication tree for predicting whether a contaminated production lot of frozen vegetables will cause one or more
listeriosis cases according to consumer handling behavior. Branch labels indicate a rule used for partitioning the data, where μ
lot
is the
arithmetic mean Listeria monocytogenes (LM) concentration in the production lot, t
room
is the time stored at room temperature, T
room
is
the room temperature, C
01
is whether the serving is cooked (yes
¼
1; no
¼
0), and p
lot
is the prevalence of LM-contaminated packages in
the production lot. Each node at the bottom of the tree indicates the predicted outcome per lot (illness or no illness). The intensity of the
node color is proportional to the value predicted at the node. Underneath each node is the predicted probability of illness (e.g., 0.744) and
the percentage of model iterations falling in that node (e.g., 3.1
%
). For example, to interpret the far-right illness node, for a lot with a
mean LM concentration, μ
lot
,
0.71 log CFU/g and the servings from this lot were not cooked (C
01
¼
0), there is a high probability
(74.4
%
) of occurrence of at least one case of listeriosis from the lot; however, this pathway occurred in only 3
%
of model iterations (or
3.1
%
of simulated lots), indicating that this pathway is a high risk but relatively rare.
2186 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
predictors for illness from a production lot when multiple
variables were considered at the same time and indicated
high overall accuracy (mean of 95.4%61.1%). Three
pathways resulted in predicted illness from a contaminated
production lot (Fig. 7). The mean LM concentration in the
lot after manufacturing (μ
lot
) was the most important factor
inuencing the occurrence of illness, as indicated by its
position closest to the root of the tree. For low LM
concentrations at the manufacturer (1.16 μ
lot
,0.335
log CFU/g), the tree predicted that a lot was more likely to
cause one or more illnesses when servings were thawed at
room temperature (14 T
room
,25.68C) for longer than
9.97 h (predicted probability of at least one illness ¼0.766);
however, 0.5%of simulations fell in this pathway
suggesting that this type of thawing was not a common
practice under simulated conditions. For example, the
majority of simulations (84.9%) resulted in production lots
with low mean LM concentrations at manufacturing
(,0.335 log CFU/g) that were consumed within 9.97 h of
preparation and therefore had a low likelihood of causing at
least one illness (predicted probability of any illness ¼
0.024). For higher LM concentrations at manufacturing,
cooking prior to consumption (C
01
) and the prevalence of
contaminated packages in the lot (p
lot
) were important
factors for causing illness. When μ
lot
0.335 log CFU/g but
the serving was properly cooked (C
01
¼1), the tree
predicted that illness was unlikely (,0.001). For a lot with
μ
lot
0.712 log CFU/g and servings not cooked (C
01
¼0),
illness was very likely (predicted probability of 0.744) but
the pathway was not very common (occurred in 3.1%of
iterations). When 0.488 μ
lot
,0.712 log CFU/g, C
01
¼0,
and p
lot
0.178, the lot was likely to cause illness
(predicted probability ¼0.552) but again the pathway was
uncommon (occurred in 1.5%of iterations). The mean (SD)
specicity of the nal classication tree after cross-
validation was 98.3%(1.1%), and the mean sensitivity
was only 53.2%(1.3%), which we suspect was due to low
prevalence of lots with predicted illness (and therefore few
iterations with illnesses). The high specicity gives clear
pathways that lead to no illness. Although the low
sensitivity indicates that the nal tree missed about 50%
of lots with predicted illness in FFLLoRA simulations, the
tree did identify major risk factors for illness, which was the
goal achieved through pruning and provides most relevant
targets for control. The mean (SD) positive and negative
predictive values of the tree were 68.9%(1.2%) and 96.8%
(0.2%), respectively, meaning that the tree correctly
identied two-thirds of lots leading to illness and almost
all lots leading to no illness in FFLLoRA simulations.
Scenario analysis. The results for the log risk of
invasive listeriosis per serving based on alternative
parameter values are summarized for both the general and
susceptible populations in Figure 8, and detailed results are
given in the Supplemental Material. Risk per serving
decreased in both the general and susceptible populations
as n
test
increased from 5 (the baseline model) to 60 per lot,
supporting the assumption that as sample size increases,
both the chance of accepting a contaminated lot and the
mean concentration of contamination in an accepted lot will
FIGURE 8. Median log risk of invasive listeriosis per serving from consumption of frozen vegetables contaminated with Listeria
monocytogenes according to the exponential dose-response model for the general (A) and susceptible (B) populations. Tornado plots
show results from scenario analysis of baseline parameter assumptions, where n
test
is the sample size, bis the within-package clustering
parameter (values approaching 0 indicate maximum clustering and values approaching innity indicate no clustering; clustering is
moderate around b
¼
1), n
pos
is the number of contaminated packages among the samples, sis the number of servings used per meal, and
p
thaw
is the probability that a serving is thawed at room temperature (p
thaw
¼
1, all servings thawed at room temperature; p
thaw
¼
0, all
servings thawed at refrigeration temperature). In each plot, the vertical axis crosses the horizontal axis at the baseline model value for the
median log risk of listeriosis per serving when p
cook
¼
0, n
test
¼
5, n
pos
¼
1, s
por
¼
1, p
thaw
¼
0.5, and b
¼
1.
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2187
decrease. The risk of illness per serving in both the general
and susceptible populations was reduced when within-
package contamination was heterogeneously distributed (b
¼0.1, 0.01), suggesting that in this setting the reduced
probability of a contaminated serving due to clustering had
a stronger overall effect on reducing the risk of listeriosis
per serving than did the associated increase in LM
concentration per contaminated serving due to clustering
of contamination. Changing the fraction of servings that
were not cooked and thawed at room temperature (p
thaw
¼1)
versus in the refrigerator (p
thaw
¼0) had negligible impact
on the risk of invasive listeriosis. Changing the number of
servings per meal (s¼0.5, 2) also resulted in no difference
from the baseline (s¼1) risk of listeriosis. As expected, the
number of positive samples from a production lot was
directly related to risk because the implicit prevalence of
contaminated packages increased. Although still presenting
a low risk, increasing n
pos
to 2 (from 1 in the baseline
model) when n
test
¼5 increased the risk to 12.5 and 10.9
log units in the general and susceptible populations,
respectively. Even with n
pos
¼0, the risk of invasive
listeriosis per serving was estimated at 13.1 and 11.5 log
units in the general and susceptible populations, respective-
ly. Although it may seem counterintuitive to have a dened
risk with zero positive samples, this estimate demonstrates
the impact of uncertainty in the true estimate of prevalence
obtained from such a small sample size (i.e., n
test
¼5) on the
risk of illness. Increasing the sample size (n
test
¼10, 20, or
60), even when n
pos
¼1, provided lower estimates of risk in
the general and susceptible populations compared with the
scenario of n
test
¼5 and n
pos
¼0 (Fig. 8).
Validation of the exposure assessment part of the
FFLLoRA tool with ice cream data. The estimated mean
LM contamination level per 80-g serving of ice cream from
the model was 807 CFU (90th percentile ¼1,540 CFU; 99th
¼3,169 CFU; 99.9th ¼5,295 CFU; 99.99th ¼7,965 CFU),
which was the same order of magnitude as a previously
reported estimate of LM in the ice cream outbreak (mean ¼
620 CFU; 90th ¼1,300 CFU; 99th ¼3,700 CFU; 99.9th ¼
7,400 CFU; 99.99th ¼13,000 CFU) (42). Because two
servings of ice cream were used per milkshake, the
estimated mean LM level per milkshake was obtained by
multiplying the level per 80-g serving by 2 (mean ¼1,614
CFU; 90th ¼3,083 CFU; 99th ¼6,341 CFU; 99.9th ¼
10,743 CFU). These predicted doses can, albeit with very
low probability, cause illness based on both dose-response
curves considered in this model (i.e., exponential and
Weibull-Gamma), thus still presenting a risk to consumers
even if the product were consumed frozen. Under
alternative handling and consumption scenarios considered
in this model, the LM level slightly increased before
consumption and resulted in median predicted doses for the
contaminated lots of 3.4 to 3.5 log CFU, depending on the
fraction of servings not consumed frozen (Fig. S2).
According to the exponential dose-response model, the
estimated probability of illness per serving of 3.4 310
9
for
the susceptible population at the 10
3.5
CFU dose level
corresponds to one illness in approximately 298 million
servings. Under the Weibull-Gamma dose-response model,
the estimated risk of illness per serving of 8.1 310
5
for the
susceptible population at the 10
3.5
CFU dose level
corresponds to one illness in approximately 12,400
servings. Sensitivity analysis illustrated that the initial
contamination level from manufacturing was most highly
correlated with the dose consumed, followed by whether the
serving was thawed at room temperature and the length of
time held at room temperature (Fig. S3). Overall, the results
from the ice cream simulations were consistent with
published data about the ice cream outbreak and therefore
provided validation of the structure and predictions of the
FFLLoRA.
DISCUSSION
The FFLLoRA tool was implemented with simulated
data from a production lot of frozen vegetables contami-
nated with LM to demonstrate use of the model and the
impact of different input and manufacturer-based parameter
values. Simulation results from this tool, considering our
model assumptions, align with previous risk assessments
and market surveys in indicating that fruits, vegetables, and
frozen dairy products present a low risk to U.S. consumers
in terms of the predicted number of cases of listeriosis per
serving per year (57). However, outbreaks are possible and
have occurred recently from apparent low-level contamina-
tion, often spanning several years (13). The extended period
of these outbreaks likely is due to several factors, including
long-term pathogen persistence in the facility (with repeat
product contamination over years) and the long shelf life of
frozen products. In addition to outbreaks, product recalls
without associated human listeriosis cases (56) and
suggestions of abusive consumer handling practices for
frozen foods (48) motivated development of this risk
assessment model. The versatility of the FFLLoRA allows
food manufacturers to assess the risk of listeriosis attributed
to consumersalternative handling or use of a production lot
of a NRTE frozen food product contaminated with low
levels of LM. The FFLLoRA can be used for frozen food
products with validated instructions for cooking, as with
frozen vegetables and entrees, or with safe handling
instructions for thawing, as with frozen fruit and desserts.
The model parameter values provided in Table 1 are specic
to LM and are not intended to apply to other foodborne
pathogens. The results presented here and those obtained
through use of the tool are not intended to represent the
estimated risk in the national population of invasive
listeriosis due to frozen vegetables. Instead, the example
using a production lot of frozen vegetables demonstrates the
types of data generated from the FFLLoRA that may be
used to make decisions about whether to release a
production lot. We designed the FFLLoRA to accommodate
lot-specic information relevant to risks of foodborne
illness, such as the results of nished product testing (29),
the fraction consumed by susceptible populations (3, 8, 57),
the fraction of servings handled in a manner contrary to
stated package instructions (6), and heterogeneity of LM
contamination within the lot and within a package (37).
2188 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
The FFLLoRA model was successfully validated
through comparison with data from a previous outbreak
investigation. Data from the 2015 listeriosis outbreak
associated with ice cream was used in validation of the
risk model pathway for extrapolation to use with frozen
vegetables in smoothies or similar thawed (and not cooked)
applications. Retrospective analysis of the ice cream
products and case data suggested that the susceptible
population consumed a cumulative LM dose of 6.9 to 7.5
log CFU via several milkshakes within a 9-day period (42).
Our model results for the LM contamination level in the
packaged ice cream were the same order of magnitude as
the results of product sampling conducted after the
outbreak. The estimated median LM dose consumed was
3.4 to 3.5 log CFU per milkshake under the various
handling and consumption scenarios considered. Although
the change in contamination during preparation and the
median dose consumed were lower than those in previous
analyses, higher doses such as those estimated by Pouillot et
al. (42) were observed in the tails of simulation distributions
when a fraction of the lot was not consumed frozen. LM
contamination could also have been introduced during
milkshake preparation from sources other than the ice cream
product (e.g., contaminated equipment) (42); however, these
types of external postprocessing contamination sources
were not considered in the risk assessment pathway
presented here. Ice cream is not the same as frozen
vegetables, and use of this product for validation is a
possible limitation to our approach; however, this outbreak
data set is the only one available with enumeration data
from a frozen product and represents a similar scenario of a
traditionally low-risk product that was mishandled during
preparation, resulting in cases of listeriosis.
The baseline FFLLoRA model provided one represen-
tation of contamination in a production lot of frozen
vegetables using plausible values for model parameters and
consumption scenarios. As expected, when frozen vegetable
servings were modeled as being prepared and consumed
according to the package cooking instructions, there was
negligible risk of listeriosis per serving and zero predicted
illnesses per lot. When some servings within the modeled
production lot were not cooked, the results again indicated
that on average no cases of listeriosis were expected from
the production lot. However, the simulation results did
indicate the potential for rare occurrences of large numbers
of cases associated with a single simulated lot; these results
were obtained only when the more conservative Weibull-
Gamma dose-response model was used and likely repre-
sented an overestimation not supported by epidemiological
studies. Although listeriosis is rare compared with other
foodborne illnesses, these results reinforce the importance
of preventing contamination and validating cooking in-
structions and suggest that improved adherence to these
instructions by consumers would have a positive impact on
public health.
According to the baseline model assumptions and
scenarios considered, the nal LM dose consumed was most
sensitive to whether the serving was cooked and to the
initial contamination level from manufacturing. The impact
of consumer handling practices for deli meats in the home
on the risk of listeriosis was previously estimated to
increase mean mortality by 10
6
-fold and was most impacted
by inadequate product storage, particularly the temperature
of home refrigerators (61). When considering our model
inputs independently, the length of thawing time of frozen
vegetables at either room or refrigerated temperatures was
weakly correlated with increased LM doses, and the
temperature uctuations of both ambient (i.e., on a
countertop) and refrigerated storage were not inuential,
suggesting that relatively short exposure times to temper-
atures that can facilitate growth had limited effects when
considered in isolation. However, sensitivity analysis at the
multivariable level (i.e., when we used classication tree
analysis to consider the effect of all model parameters
simultaneously) revealed that consumer handling time and
temperature contributed to the risk of illness depending on
the initial concentration of contamination in the lot,
particularly at low concentrations. The results indicate the
potential multifactorial nature of the risk; multiple co-
occurring risk factors, although generally rare, may lead to
illness. Multifactorial risk means that control strategies may
need to target multiple risk factors or that decision making
may need to involve multiple factors to prevent illness.
Alternative sampling values were also tested through
scenario analysis to demonstrate the exibility of the
spreadsheet model to be used routinely by a manufacturer
and the possible range of risk of listeriosis per serving. The
effect of an increasing sampling effort (i.e., the number of
nished product samples tested per production lot) given the
same result (i.e., one sample testing positive) was a
reduction in the risk of invasive listeriosis per serving due
to the reduced estimated prevalence and concentration of
contamination in the lot. The scenario involving detection
of a single positive sample among 20 or 60 samples tested
predicted a lower risk of listeriosis per serving than when
zero positive samples were detected in only 5 samples tested
(Supplemental Tables S1a and S1b). Although it may seem
counterintuitive to have a dened risk with zero positive
samples detected, these modeled scenarios demonstrate the
impact of uncertainty in the true estimate of prevalence on
the risk of illness. This nding underscores existing industry
guidance that when a relatively small number of samples
tested fails to detect contamination in a lot, the sampling
results should be interpreted with caution given the limited
statistical power of a small sample to detect such low-level
contamination (prevalence and concentration) (21).
The effect of within-package clustering in our model of
partitioning frozen vegetable packages was contrary to that
in previous reports, likely because of the uncertainty in the
infectious dose required to cause listeriosis and because our
evaluation of the subsequent handling of the product after
partitioning (and hence LM growth patterns) differed from
that of previous studies, which utilized food survey data for
RTE foods at retail outlets to derive the estimated LM dose
consumed. Nauta (37) reported that with increased cluster-
ing, prevalence of contaminated subunits (e.g., servings)
decreases and consequently the mean number of cells in
each contaminated subunit increases. Previous QMRAs
have indicated that elimination of units with higher
concentrations of contamination can reduce the number of
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2189
predicted cases by .99%, whereas no comparable reduction
can be anticipated with emphasis on prevalence alone (8).
However, in the scenario analysis reported here, increased
clustering (bapproaching 0) reduced risk compared with
homogeneously distributed contamination among the serv-
ings prepared from a package. Under our assumptions, the
reduced probability of a contaminated serving had a greater
overall effect on reducing the risk of listeriosis per serving
than did the associated increase in a dose per contaminated
serving when clustering of contamination was present.
Thus, an increase in the contamination concentration in an
occasional contaminated serving due to clustering was
offset by a decrease in the consumers probability of
exposure to such a serving because the prevalence of
contaminated servings was lower. However, this result (i.e.,
homogeneous contamination associated with higher listeri-
osis risk) may be particularly applicable only to LM and the
scenario in which the product is not cooked before
consumption but is held at refrigeration or room tempera-
ture, thus allowing low concentrations to increase enough to
cause illness. The relationship between clustering and
number of illnesses is difcult to predict because it depends
on a number of factors, such as the pathogen concentration
in the food units before partitioning, the prevalence of
contaminated food units (packages), the size and variability
of smaller units (servings), and the dose-response relation-
ship for the pathogen. Generally, partitioning of contami-
nated food leads to an increasing heterogeneity of exposure,
thus increasing the variability of individual risk for
consumers (1), and clustering can increase this variability
even further. A practical implication of this nding is that
users of the model may want to run scenarios with and
without clustering and choose a more conservative estimate
of the number of potential cases upon which to base their
risk management decisions. Although the FFLLoRA model
does not currently consider how the production lot became
contaminated, the distribution of contamination within
packages (or within the lot) may be assumed based on
additional information about the production environment
and processes. For example, results from a facilitys routine
Listeria testing program may suggest whether clustered
contamination (as a result of a sporadic, point-source
contamination event) or homogeneous contamination (as a
result of a systemic, widespread contamination event or
contaminated food ingredient) would be more likely.
The FFLLoRA was developed as a tool for managing
nished product testing programs, reducing the need for
market withdrawals and product recalls, and providing data
for making risk-based decisions that meet production and
food safety objectives. Although good manufacturing
practices, including cleaning and sanitation, environmental
monitoring, and raw material supplier verication pro-
grams, should be in place to reduce the risk of contamina-
tion of foods, the FFLLoRA tool could be used to interpret
and compare results from nished product testing of a range
of food products as they relate to risk of listeriosis in the
population of exposed consumers. For frozen NRTE foods
that contain specic safe handling and preparation instruc-
tions on the package, a key use of the risk assessment model
would be assessment of the public health risk associated
with releasing into commerce products containing rare and/
or minimal LM contamination when legally permitted. For
example, in some countries frozen RTE products fall under
regulatory limits of ,100 CFU/g, and frozen NRTE
products may have no regulatory limits. Recommendations
for sampling and testing aimed at detecting LM in
processing plants for frozen vegetables, fruits, and herbs
were quickly published by the European Food Safety
Authority (15) following the 2018 listeriosis outbreak
associated with frozen vegetable products. Because the
current version of the FFLLoRA model does not consider
how the production lot became contaminated during
processing, information from environmental monitoring
could not be directly evaluated in the risk assessment.
However, future research could link the risk assessment
with the contamination dynamics in the facility and provide
sufcient data (e.g., through extensive nished product
testing and/or appropriate modeling tools (62)) for use of
such a risk assessment modeling tool to help industry and
regulatory agencies understand the potential public health
risk associated with frozen NRTE products produced in a
facility where LM has been detected in the environment.
The model also does not incorporate the frequency of
contaminated lots within a manufacturer or across manu-
facturers. However, if such low-level lot contamination
were to occur regularly or be widespread across the
industry, a large number of consumers would be exposed,
requiring action beyond use of the FFLLoRA tool. The
results of the frozen vegetable scenarios and model
assumptions presented here indicate that providing appro-
priate food safety messages on the package of NRTE frozen
foods and ensuing consumer education to build recognition
for these messages can have a signicant impact on
consumer behavior. However, consumers may still not
follow validated cooking instructions due to perceived
health benets or new consumption trends. To account for
these behavioral digressions, data obtained from QMRAs,
such as the FFLLoRA, can inform a manufacturers
decision-making process. The results may even guide
manufacturers to inform consumers, particularly those
among susceptible populations, of the risks associated with
consumption of undercooked or uncooked NRTE frozen
foods in a manner similar to consumer warnings that
currently exist for shellsh and rare hamburger.
When considering the FFLLoRA model as a reliable
risk management tool, some limitations encountered during
model development and parameterization must be ad-
dressed. Starting from the exposure assessment, we
encountered data limitations for prevalence and concentra-
tion of LM in production lots of frozen vegetables. We
relied on the ICMSFs sampling tool to estimate the
concentration of contamination based on the manufacturers
sampling plan criteria, mainly the number of nished
product samples. The FFLLoRA model did not consider the
design or selection of nished product samples (e.g.,
random or stratied random sampling), which can impact
detection of heterogeneously distributed contamination
(e.g., Cronobacter spp. in powdered infant formula) (30).
With respect to consumer consumption habits regarding
adherence to package instructions and the temperature of
2190 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
home refrigerators and freezers, data are scarce and have
been requested by risk assessors and researchers (38).We
assumed that a manufacturer would know the approximate
fraction of servings from a lot that would not be properly
cooked according to labeled instructions; however, this
information may be proprietary and not easily accessible or
reliable. The FFLLoRA tool did not consider introduction
of LM from the home kitchen or other postprocessing
sources or from other ingredients during meal preparation or
multiple LM exposures within a period of time. These
simplications were necessary given the available informa-
tion and data on frozen vegetables and were justied
because the goal of this model was to assess the risk for the
manufacturer; contamination occurring postmanufacturing
would not be a risk attributable to the manufacturers. For
hazard and risk characterization, the dose-response function
and parameter values chosen are important when interpret-
ing the predicted risks in various subpopulations, especially
at low doses, because insufcient data are available for
building a completely reliable dose-response model (27).
We chose invasive listeriosis as our outcome and tested two
dose-response models, exponential and Weibull-Gamma, to
illustrate the variability in the outcome estimates from the
same dose. The predicted risk per serving and number of
listeriosis cases obtained with the two models, particularly
the Weibull-Gamma model, may appear inconsistent with
epidemiological data. This inconsistency has previously
been observed when comparing experimental data with data
from dose-response models for foodborne pathogens (22),
but this uncertainty in the prediction may be unavoidable for
low-probability risks and is always a trade-off with
exibility when choosing a dose-response model (60).
Because LM is an opportunistic pathogen affecting the
general and susceptible populations differently, it is
important to know who is consuming the potentially
contaminated product. Although we used an estimate that
20%of the consumers of a production lot of NRTE frozen
foods would belong to the susceptible subpopulation, the
actual percentage of the population that is susceptible may
be larger; thus, the likelihood of cases per lot would be even
higher. All uncertain (and therefore assumed) parameter
values were tested in scenario analysis, but the limitations
of current knowledge should be well understood and
considered when using the FFLLoRA tool for decision
making.
In conclusion, we developed a spreadsheet-based
decision support tool and user interface, FFLLoRA, for
assessment of the risk of listeriosis related to consumption
of NRTE frozen foods consumed without following proper
cooking instructions. Based on the results of the baseline
FFLLoRA scenario and given our model assumptions, low
levels of LM contamination in a production lot of frozen
vegetables would present a low risk for listeriosis if the
vegetables were prepared according to the package cooking
instructions. However, given changing consumer behaviors
regarding handling of frozen foods and the range in
estimates for the risk of listeriosis when servings were
assumed in the model to be consumed without cooking,
prevention and management of LM in frozen NRTE foods is
essential. Finished product testing, when properly designed
and interpreted, can be used to understand the potential
prevalence and concentration of contamination in a
production lot accepted for distribution. For manufacturers
and regulators conducting nished product testing, the
FFLLoRA may be useful when making product release and/
or recall decisions after frozen NRTE products test positive
for LM (or for certain LM concentrations when quantitative
testing is performed). Future work is needed to determine
the frequency and manner of such decision making under
uncertain scenarios to estimate listeriosis risk from frozen
vegetables at the national level. Although the risk pathway
modeled here did not consider how contamination of the
production lot occurs, the initial concentration of contam-
ination was the most inuential factor for the eventual LM
dose consumed, emphasizing that resources should be
focused on reducing the likelihood of LM contamination
of products during production. Special focus should be
directed toward quantifying and understanding consumer
practices associated with specic frozen NRTE products.
For frozen foods with low-level LM contamination and that
do not support the growth of LM, the consumer handling
phase of the risk assessment model was critical. Data on
actual handling and preparation practices of consumers are
required to properly estimate LM growth before consump-
tion and the risk per serving and to design adequate
formulations and consumer interventions.
ACKNOWLEDGMENTS
This work was supported by a grant from the Frozen Food
Foundation awarded to M. Wiedmann and partially supported by award
2016-67017-24421 from the USDA National Institute of Food and
Agriculture and the Schwartz Research Fund Award to R. Ivanek. Any
opinions, ndings, conclusions, or recommendations expressed in this
publication are those of the authors and do not necessarily reect the view
of the funding organizations.
SUPPLEMENTAL MATERIAL
Supplemental material associated with this article can be
found online at: https://doi.org/10.4315/0362-028X.JFP-19-092.s1
REFERENCES
1. Augustin, J. C., M. Ellouze, and L. Guillier. 2017. Microbial risk
assessment: integrating and quantifying the impacts of food
processing operations on food safety, chap. 30. In A. de S. SantAna
(ed.), Quantitative microbiology in food processing: modeling the
microbial ecology. Wiley, Hoboken, NJ.
2. Bemrah, N., M. Sanaa, M. H. Cassin, M. W. Grifths, and O. Cerf.
1998. Quantitative risk assessment of human listeriosis from
consumption of soft cheese made from raw milk. Prev. Vet. Med.
37:129145.
3. Buchanan, R. L., W. G. Damert, R. C. Whiting, and M. Van
Schothorst. 1997. Use of epidemiologic and food survey data to
estimate a purposefully conservative dose-response relationship for
Listeria monocytogenes levels and incidence of listeriosis. J. Food
Prot. 60:918922.
4. Centers for Disease Control and Prevention. 2018. Multistate
outbreak of listeriosis linked to Blue Bell Creameries products (nal
update). Centers for Disease Control and Prevention, Atlanta.
5. Centers for Disease Control and Prevention. 2018. Multistate
outbreak of listeriosis linked to frozen vegetables (nal update).
Centers for Disease Control and Prevention, Atlanta.
6. Chen, Y., E. Allard, A. Wooten, M. Hur, I. Sheth, A. Laasri, T. S.
Hammack, and D. Macarisin. 2016. Recovery and growth potential
of Listeria monocytogenes in temperature abused milkshakes
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2191
prepared from naturally contaminated ice cream linked to a listeriosis
outbreak. Front. Microbiol. 7:764. https://doi.org/10.3389/fmicb.
2016.00764
7. Chen, Y., L. S. Burall, D. Macarisin, R. Pouillot, E. Strain, A. J. De
Jesus, A. Laasri, H. Wang, L. Ali, A. Tatavarthy, G. Zhang, L. Hu, J.
Day, J. Kang, S. Sahu, D. Srinivasan, K. Klontz, M. Parish, P. S.
Evans, E. W. Brown, T. S. Hammack, D. L. Zink, and A. R. Datta.
2016. Prevalence and level of Listeria monocytogenes in ice cream
linked to a listeriosis outbreak in the United States. J. Food Prot.
79:18281832.
8. Chen, Y., W. H. Ross, V. N. Scott, and D. E. Gombas. 2003. Listeria
monocytogenes: low levels equal low risk. J. Food Prot. 66:570577.
9. Codex Alimentarius Commission. 1997. Recommended international
code of practice for the general principles of food hygiene. CAC/
RCP 1-1969, rev. 3. Codex Alimentarius Commission, Rome.
10. Codex Alimentarius Commission. 2007. Guidelines on the applica-
tion of general principles of food hygiene to the control of Listeria
monocytogenes in ready-to-eat foods. CAC/GL 61-2007. Codex
Alimentarius Commission, Rome.
11. Coote, P. J., C. D. Holyoak, and M. B. Cole. 1991. Thermal
inactivation of Listeria monocytogenes during a process simulating
temperatures achieved during microwave heating. J. Appl. Bacteriol.
70:489494.
12. Danyluk, M. D., and D. W. Schaffner. 2011. Quantitative assessment
of the microbial risk of leafy greens from farm to consumption:
preliminary framework, data, and risk estimates. J. Food Prot.
74:700708.
13. de Noordhout, C. M., B. Devleesschauwer, F. J. Angulo, G. Verbeke,
J. Haagsma, M. Kirk, A. Havelaar, and N. Speybroeck. 2014. The
global burden of listeriosis: a systematic review and meta-analysis.
Lancet Infect. Dis. 14:10731082.
14. European Centre for Disease Prevention and Control and European
Food Safety Authority. 2018. Multi-country outbreak of Listeria
monocytogenes serogroup IVb, multi-locus sequence type 6,
infections linked to frozen corn and possibly to other frozen
vegetables. First update. EFSA Support.Publ. https://doi.org/10.
2903/sp.efsa.2018.EN-1448
15. European Food Safety Authority, A. Allende, L. Barre, L. Jacxsens,
E. Liebana, W. Messens, E. Sarno, and M. T. da Silva Felicio. 2018.
Urgent scientic and technical assistance to provide recommenda-
tions for sampling and testing in the processing plants of frozen
vegetables aiming at detecting Listeria monocytogenes.EFSA
Support. Publ. 15. https://doi.org/10.2903/sp.efsa.2018.EN-1445
16. Falk, L. E., K. A. Fader, D. S. Cui, S. C. Totton, A. M. Fazil, A. M.
Lammerding, and B. A. Smith. 2016. Comparing listeriosis risks in
at-risk populations using a user-friendly quantitative microbial risk
assessment tool and epidemiological data. Epidemiol. Infect.
144:27432758.
17. Farber, J. M., W. H. Ross, and J. Harwig. 1996. Health risk
assessment of Listeria monocytogenes in Canada. Int. J. Food
Microbiol. 30:145156.
18. Gallagher, D., R. Pouillot, K. Hoelzer, J. Tang, S. B. Dennis, and J.
R. Kause. 2016. Listeria monocytogenes in retail delicatessens: an
interagency risk assessmentrisk mitigations. J. Food Prot.
79:10761088.
19. Heddleson, R. A., and S. Doores. 1994. Factors affecting microwave
heating of foods and microwave induced destruction of foodborne
pathogensa review. J. Food Prot. 57:10251037.
20. Hitchins, A. D. 1996. Assessment of alimentary exposure to Listeria
monocytogenes.Int. J. Food Microbiol. 30:7185.
21. Hoelzer, K., and R. Pouillot. 2013. Practical considerations for the
interpretation of microbial testing results based on small numbers of
samples. Foodborne Pathog.Dis. 10:907915.
22. Holcomb, D. L., M. A. Smith, G. O. Ware, Y.-C. Hung, R. E.
Brackett, and M. P. Doyle. 1999. Comparison of six dose-response
models for use with food-borne pathogens. Risk Anal. 19:10911100.
23. International Commission on Microbiological Specications for
Foods. 1994. Choice of sampling plan and criteria for Listeria
monocytogenes.Int. J. Food Microbiol. 22:8996.
24. International Commission on Microbiological Specications for
Foods. 1996. Characteristics of microbial pathogens, 1st ed. Blackie,
London.
25. International Commission on Microbiological Specications for
Foods. 2011. Use of data for assessing process control and product
acceptance. Springer, New York.
26. International Commission on Microbiological Specications for
Foods. 2016. Microbiological sampling plans: a tool to explore
ICMSF recommendations. Software. Available at: http://www.icmsf.
org/publications/software-downloads/.
27. International Commission on Microbiological Specications for
Foods. 2018. Microbiological testing in food safety management,
2nd ed. Springer, New York.
28. Ivanek, R., Y. T. Gröhn, M. T. Wells, A. J. Lembo, Jr., B. D. Sauders,
and M. Wiedmann. 2009. Modeling of spatially referenced
environmental and meteorological factors inuencing the probability
of Listeria species isolation from natural environments. Appl.
Environ. Microbiol. 75:58935909.
29. Jongenburger, I., H. M. W. den Besten, and M. H. Zwietering. 2015.
Statistical aspects of food safety sampling. Annu. Rev. Food Sci.
Technol. 6:479503.
30. Jongenburger, I., M. W. Reij, E. P. J. Boer, L. G. M. Gorris, and M.
H. Zwietering. 2011. Actual distribution of Cronobacter spp. in
industrial batches of powdered infant formula and consequences for
performance of sampling strategies. Int. J. Food Microbiol. 151:62
69.
31. Kataoka, A., H. Wang, P. H. Elliott, R. C. Whiting, and M. M.
Hayman. 2017. Growth of Listeria monocytogenes in thawed frozen
foods. J. Food Prot. 80:447453.
32. Latorre, A. A., A. K. Pradhan, J. A. S. Van Kessel, J. S. Karns, K. J.
Boor, D. H. Rice, K. J. Mangione, Y. T. Gröhn, and Y. H. Schukken.
2011. Quantitative risk assessment of listeriosis due to consumption
of raw milk. J. Food Prot. 74:12681281.
33. Lindqvist, R., and A. Westöö. 2000. Quantitative risk assessment for
Listeria monocytogenes in smoked or gravad salmon and rainbow
trout in Sweden. Int. J. Food Microbiol. 58:181196.
34. Lund, B. M., M. R. Knox, and M. B. Cole. 1989. Destruction of
Listeria monocytogenes during microwave cooking. Lancet 333:218.
35. Mataragas, M., M. H. Zwietering, P. N. Skandamis, and E. H.
Drosinos. 2010. Quantitative microbiological risk assessment as a
tool to obtain useful information for risk managersspecic
application to Listeria monocytogenes and ready-to-eat meat
products. Int. J. Food Microbiol. 141:S170S179.
36. Mena, C. 2004. Incidence of Listeria monocytogenes in different
food products commercialized in Portugal. Food Microbiol. 21:213
216.
37. Nauta, M. J. 2005. Microbiological risk assessment models for
partitioning and mixing during food handling. Int. J. Food Microbiol.
100:311322.
38. Nauta, M. J., W. F. Jacobs-Reitsma, and A. H. Havelaar. 2007. A risk
assessment model for Campylobacter in broiler meat. Risk Anal.
27:845861.
39. Pérez-Rodríguez, F., E. Carrasco, S. Bover-Cid, A. Jofré, and A.
Valero. 2017. Closing gaps for performing a risk assessment on
Listeria monocytogenes in ready-to-eat (RTE) foods: activity 2, a
quantitative risk characterization on L. monocytogenes in RTE foods;
starting from the retail stage. EFSA Support.Publ. https://doi.org/10.
2903/sp.efsa.2017.EN-1252
40. Pouillot, R., D. Gallagher, J. Tang, K. Hoelzer, J. Kause, and S. B.
Dennis. 2015. Listeria monocytogenes in retail delicatessens: an
interagency risk assessmentmodel and baseline results. J. Food
Prot. 78:134145.
41. Pouillot, R., V. Goulet, M. L. Delignette-Muller, A. Mahé, and M.
Cornu. 2009. Quantitative risk assessment of Listeria monocytogenes
in French cold-smoked salmon. II. Risk characterization. Risk Anal.
29:806819.
42. Pouillot, R., K. C. Klontz, Y. Chen, L. S. Burall, D. Macarisin, M.
Doyle, K. M. Bally, E. Strain, A. R. Datta, T. S. Hammack, and J. M.
Van Doren. 2016. Infectious dose of Listeria monocytogenes in
2192 ZOELLNER ET AL. J. Food Prot., Vol. 82, No. 12
outbreak linked to ice cream, United States, 2015. Emerg. Infect. Dis.
22:21132119.
43. Pouillot, R., M. B. Lubran, S. C. Cates, and S. Dennis. 2010.
Estimating parametric distributions of storage time and temperature
of ready-to-eat foods for U.S. households. J. Food Prot. 73:312321.
44. Pouillot, R., N. Miconnet, A.-L. Afchain, M. L. Delignette-Muller,
A. Beaufort, L. Rosso, J.-B. Denis, and M. Cornu. 2007. Quantitative
risk assessment of Listeria monocytogenes in French cold-smoked
salmon. I. Quantitative exposure assessment. Risk Anal. 27:683700.
45. Pradhan, A. K., R. Ivanek, Y. T. Gröhn, R. Bukowski, I. Geornaras, J.
N. Sofos, and M. Wiedmann. 2010. Quantitative risk assessment of
listeriosis-associated deaths due to Listeria monocytogenes contam-
ination of deli meats originating from manufacture and retail. J. Food
Prot. 73:620630.
46. Pradhan, A. K., R. Ivanek, Y. T. Gröhn, R. Bukowski, and M.
Wiedmann. 2011. Comparison of public health impact of Listeria
monocytogenes product-to-product and environment-to-product con-
tamination of deli meats at retail. J. Food Prot. 74:18601868.
47. Roberts, W., Jr. 2018. VegetablesUSMay 2018. Mintel Group,
London.
48. Salazar, J. K., V. M. Bathija, C. K. Carstens, S. S. Narula, A. Shazer,
D. Stewart, and M. L. Tortorello. 2018. Listeria monocytogenes
growth kinetics in milkshakes made from naturally and articially
contaminated ice cream. Front. Microbiol. 9:62. https://doi.org/10.
3389/fmicb.2018.00062
49. Shank, F. R., E. L. Elliot, I. K. Wachsmuth, and M. E. Losikoff.
1996. US position on Listeria monocytogenes in foods. Food Control
7:229234.
50. Therneau, T., B. Atkinson, and B. Ripley. 2018. Package rpart.
Recursive partitioning for classication, regression and survival
trees. Available at: https://cran.r-project.org/web/packages/rpart/
rpart.pdf.
51. Tiwari, U., E. Cummins, A. Valero, D. Walsh, M. Dalmasso, K.
Jordan, and G. Duffy. 2015. Farm to fork quantitative risk assessment
of Listeria monocytogenes contamination in raw and pasteurized
milk cheese in Ireland: Listeria monocytogenes contamination of
cheese. Risk Anal. 35:11401153.
52. Tromp, S. O., H. Rijgersberg, and E. Franz. 2010. Quantitative
microbial risk assessment for Escherichia coli O157:H7, Salmonella
enterica, and Listeria monocytogenes in Leafy green vegetables
consumed at salad bars, based on modeling supply chain logistics. J.
Food Prot. 73:18301840.
53. United Fresh Food Safety and Technology Council. 2018. Guidance
on environmental monitoring and control of Listeria for the fresh
produce industry. United Fresh Produce Association, Washington,
DC.
54. U.S. Food and Drug Administration. 2017. Control of Listeria
monocytogenes in ready-to-eat foods: guidance for industry. Draft
guidance. U.S. Food and Drug Administration, Silver Spring, MD.
55. U.S. Food and Drug Administration. 2018. Hazard analysis and risk-
based preventive controls for human food: draft guidance for
industry. Appendix 1. Potential hazards for foods and processes.
U.S. Food and Drug Administration, Silver Spring, MD.
56. U.S. Food and Drug Administration. 2019. 2018 Recalls, market
withdrawals & safety alerts. Available at: https://www.fda.gov/
safety/archive-recalls-market-withdrawals-safety-alerts/2018-recalls-
market-withdrawals-safety-alerts.
57. U.S. Food and Drug Administration and U.S. Department of
Agriculture. 2003. Quantitative assessment of relative risk to public
health from foodborne Listeria monocytogenes among selected
categories of ready-to-eat foods. U.S. Food and Drug Administration,
Silver Spring, MD.
58. U.S. Food and Drug Administration and U.S. Department of
Agriculture. 2013. Interagency risk assessment: Listeria monocyto-
genes in retail delicatessens. Technical report. U.S. Food and Drug
Administration, Silver Spring, MD.
59. van Schothorst, M., M. H. Zwietering, T. Ross, R. L. Buchanan, and
M. B. Cole. 2009. Relating microbiological criteria to food safety
objectives and performance objectives. Food Control 20:967979.
60. World Health Organization and Food and Agriculture Organization
of the United Nations. 2004. Risk assessment of Listeria monocy-
togenes in ready-to-eat foods. Technical report. World Health
Organization, Geneva.
61. Yang, H., A. Mokhtari, L.-A. Jaykus, R. A. Morales, S. C. Cates, and
P. Cowen. 2006. Consumer phase risk assessment for Listeria
monocytogenes in deli meats. Risk Anal. 26:89103.
62. Zoellner, C., R. Jennings, M. Wiedmann, and R. Ivanek. 2019.
EnABLe: an agent-based model to understand Listeria dynamics in
food processing facilities. Sci. Rep. 9. https://doi.org/10.1038/
s41598-018-36654-z
J. Food Prot., Vol. 82, No. 12 RISK ASSESSMENT FOR L. MONOCYTOGENES IN FROZEN FOOD 2193
... As a stochastic model that can simultaneously consider all possible combinations of model variables represented in a food system, Monte Carlo simulations can predict all possible outcomes associated with a defined problem (McNab, 1998). This modeling approach has been widely applied to predict spoilage in dairy products (Buehler et al., 2018a,b) and food safety risk in other food products (Pouillot et al., 2007;Pang et al., 2017;Zoellner et al., 2019). By simulating the growth, inactivation, transfer, or partition of microorganisms of interest over food processing or storage, this model can estimate the microbial concentrations in final food products and predict the associated risks. ...
Article
Late blowing defect (LBD) is an important spoilage issue in semi-hard cheese, with the outgrowth of Clostridium tyrobutyricum spores during cheese aging considered to be the primary cause. Although previous studies have explored the microbial and physicochemical factors influencing the defect, a risk assessment tool that allows for improved and rational management of LBD is lacking. The purpose of this study was to develop a predictive model to estimate the probability of LBD in Gouda cheese and evaluate different intervention strategies. The spore concentration distribution of butyric acid bacteria (BAB) in bulk tank milk was obtained from 8 dairy farms over 12 mo. The concentration of C. tyrobutyricum from raw milk to the end of aging was simulated based on Gouda brined for 2 d in saturated brine at 8°C and aged at 13°C. Predicted C. tyrobutyricum concentrations during aging and estimated concentration thresholds in cheese at onset of LBD were used to predict product loss due to LBD during a simulated 1-yr production. With the estimated concentration thresholds in cheese ranging from 4.36 to 4.46 log most probable number (MPN)/kg of cheese, the model predicted that 9.2% (±1.7%) of Gouda cheese showed LBD by d 60; cheeses predicted to show LBD at d 60 showed a mean pH of 5.39 and were produced with raw milk with a mean BAB spore count of 143 MPN/L. By d 90, 36.1% (±3.4%) of cheeses were predicted to show LBD, indicating that LBD typically manifests between d 60 and 90, which is consistent with observations from the literature and the cheese industry. Sensitivity analysis indicated that C. tyrobutyricum maximum growth rate as well as concentration threshold in cheese at onset of LBD are the most important variables, identifying key data needs for development of more accurate models. The implementation of microfiltration or bactofugation of raw milk (assumed to show 98% efficiency of spore removal) in our model prevented occurrence of LBD during the first 60 d of aging. Overall, our findings provide a framework for predicting the occurrence of LBD in Gouda as well as other cheeses and illustrate the value of developing digital tools for managing dairy product quality.
... Foods mostly associated with foodborne listeriosis include RTE products that: (i) support growth of L. monocytogenes; (ii) are consumed without receiving any antibacterial treatment (e.g., thermic treatment); and (iii) have long refrigerated shelf-life [60]. Despite frozen foods do not support the growth of L. monocytogenes, consumption of this type of food may contribute to the risk of listeriosis, mainly when it is added directly (e.g. to salads or smoothies) [61]. L. monocytogenes adaptation mechanisms to cold and their consequences are described in the next sections. ...
Article
Full-text available
Listeria monocytogenes is a saprophytic gram-positive bacterium, and an opportunistic foodborne pathogen that can produce listeriosis in humans and animals. It has evolved an exceptional ability to adapt to stress conditions encountered in different environments, resulting in a ubiquitous distribution. Because some food preservation methods and disinfection protocols in food-processing environments cannot efficiently prevent contaminations, L. monocytogenes constitutes a threat to human health and a challenge to food safety. In the host, Listeria colonizes the gastrointestinal tract, crosses the intestinal barrier, and disseminates through the blood to target organs. In immunocompromised individuals, the elderly, and pregnant women, the pathogen can cross the blood-brain and placental barriers, leading to neurolisteriosis and materno-fetal lister-iosis. Molecular and cell biology studies of infection have proven L. monocytogenes to be a versatile pathogen that deploys unique strategies to invade different cell types, survive and move inside the eukaryotic host cell, and spread from cell to cell. Here, we present the multi-faceted Listeria life cycle from a comprehensive perspective. We discuss genetic features of pathogenic Listeria species, analyze factors involved in food contamination, and review bacterial strategies to tolerate stresses encountered both during food processing and along the host's gastrointestinal tract. Then we dissect host-pathogen interactions underlying listerial pathogen-esis in mammals from a cell biology and systemic point of view. Finally, we summarize the epidemiology, pathophysiology, and clinical features of listeriosis in humans and animals. This work aims to gather information from different fields crucial for a comprehensive understanding of the pathogenesis of L. monocytogenes.
Article
Purpose Foodborne disease remains a significant public health threat and cause for concern. Despite numerous studies indicating that the domestic kitchen is a high-risk area for foodborne illness, consumers believe it is low-risk. The type of food being prepared in the kitchen has drastically changed, with an increased use of convenience products, perceived low-risk products. Little is known about consumers' behaviours with convenience meals. Therefore, it is essential to understand actual and perceived food safety behaviours in their use. Methodology A mixed-methods approach using in-home observations and semi-structured interviews was undertaken with participants from the UK and the Republic of Ireland ( N = 50). Observational data was summarized using descriptive statistics and an inductive thematic analysis was conducted to interpret the qualitative data. Findings In this study general food safety behaviours, such as handwashing, use-by date checking, as well as identifying obvious safety hazard were suboptimal. However, participants' convenience meal safety behaviours were as optimal as possible with the provided instructions. The qualitative data supported the participants' struggle with the provided on-packet instructions. They also displayed participants' uncertain perceptions around different products, for example if oven-products could be reheated multiple times, and varying levels of concern around meat and dairy products. Consumers urged for more detailed and clearer instructions with these products. Recommendations for producers are provided to enhance consumer experience with the use of the products as well as contributing towards ensuring consumer safety. Originality/value Mixed-methods approach investigating consumers' real and perceived food safety behaviours.
Article
The working population growth have created greater consumer demand for ready‐to‐eat (RTE) foods. Pasteurization is one of the most common preservation methods for commercial production of low‐acid RTE cold‐chain products. Proper selection of a pasteurization method plays an important role not only in ensuring microbial safety but also in maintaining food quality during storage. Better retention of flavor, color, appearance, and nutritional value of RTE products is one of the reasons for the food industry to adopt novel technologies such as high‐pressure processing (HPP) as a substitute or complementary technology for thermal pasteurization. HPP has been used industrially for the pasteurization of high‐acid RTE products. Yet, this method is not commonly used for pasteurization of low‐acid RTE food products, due primarily to the need of additional heating to thermally inactivate spores, coupled with relatively long treatment times resulting in high processing costs. Practical Application: Food companies would like to adopt novel technologies such as HPP instead of using conventional thermal processes, yet there is a lack of information on spoilage and the shelf‐life of pasteurized low‐acid RTE foods (by different novel pasteurization methods including HPP) in cold storage. This article provides an overview of the microbial concerns and related regulatory guidelines for the pasteurization of low‐acid RTE foods and summarizes the effects of HPP in terms of microbiology (both pathogens and spoilage microorganisms), quality, and shelf‐life on low‐acid RTE foods. This review also includes the most recent research articles regarding a comparison between HPP pasteurization and thermal pasteurization treatments and the limitations of HPP for low‐acid chilled RTE foods.
Article
Invasive listeriosis is a potentially fatal foodborne disease that according to this study may affect up to 32.9 % of the US population considered as increased risk and including people with underlying conditions and co-morbidities. Listeria monocytogenes has been scrutinized in research and surveillance programs worldwide in Ready-to-Eat (RTE) food commodities (RTE salads, deli meats, soft/semi-soft cheese, seafood) and frozen vegetables in the last 30 years with an estimated overall prevalence of 1.4–9.9 % worldwide (WD) and 0.5–3.8 % in the United States (US). Current L. monocytogenes control efforts have led to a prevalence reduction in the last 5 years of 4.9–62.9 % (WD) and 12.4–92.7 % (US). A quantitative risk assessment model was developed, estimating the probability of infection in the US susceptible population to be 10–10,000× higher than general population and the total number of estimated cases in the US was 1044 and 2089 cases by using the FAO/WHO and Pouillot dose-response models. Most cases were attributed to deli meats (>90 % of cases) followed by RTE salads (3.9–4.5 %), soft and semi-soft cheese and RTE seafood (0.5–1.0 %) and frozen vegetables (0.2–0.3 %). Cases attributed to the increased risk population corresponded to 96.6–98.0 % of the total cases with the highly susceptible population responsible for 46.9–80.1 % of the cases. Removing product lots with a concentration higher than 1 CFU/g reduced the prevalence of contamination by 15.7–88.3 % and number of cases by 55.9–100 %. Introducing lot-by-lot testing and defining allowable quantitative regulatory limits for low-risk RTE commodities may reduce the public health impact of L. monocytogenes and improve the availability of enumeration data.
Article
Full-text available
Although public health risk assessments for Listeria monocytogenes (Lm) have been published for various foods, firm-level decision making on interventions targeting Lm involves considerations of both public health and enterprise risks. Smoked seafood is a ready-to-eat product with a high incidence of Lm contamination and has been associated with several recalls. We used cold-smoked salmon as a model product to develop a decision support tool (the regulatory and recall risk [3R] model) to estimate (i) baseline regulatory and recall (RR) risks (i.e., overall risks of a lot sampled and found positive for Lm, e.g., by food regulatory agencies) due to Lm contamination and (ii) the RR risk reduction that can be achieved through interventions with underlying mechanisms such as reducing the prevalence and/or level of Lm and retarding or preventing Lm growth. Given that a set number of samples (e.g., 10) are tested for a given lot, the RR risk equals the likelihood of detecting Lm in at least one sample. Under the baseline scenario, which assumes a 4% Lm prevalence and no interventions, the median predicted RR risk for a given production lot was 0.333 (95% credible interval: 0.288, 0.384) when 10 25-g samples were tested. Nisin treatments, which reduce both the prevalence and initial level of Lm, reduced RR risks in a concentration-dependent manner to 0.109 (0.074, 0.146) with 5 ppm, 0.049 (0.024, 0.083) with 10 ppm, and 0.017 (0.007, 0.033) with 20 ppm. In general, more effective reduction in RR risks can be achieved by reducing Lm prevalence than by retarding Lm growth; the RR risk was reduced to 0.182 (0.153, 0.213) by a 50% prevalence reduction but to only 0.313 (0.268, 0.367) by bacteriostatic growth inhibitors. Sensitivity analysis indicated that prevalence and initial level of Lm and storage temperature have the greatest impact on predicting RR risks, suggesting that reliable data for these parameters will improve model performance. Highlights:
Article
The first step in quantitative microbial risk assessment (QMRA) is to determine the distribution of pathogen contamination among servings of the food in question at some point in the farm-to-table chain. In the present study, the distribution of Salmonella contamination among servings of chicken liver for use in the QMRA was determined at meal preparation. Salmonella prevalence (P), most probable number (MPN, N), and serotype for different serving sizes were determined by use of a combination of five methods: (i) whole sample enrichment; (ii) quantitative PCR; (iii) culture isolation; (iv) serotyping; and (v) Monte Carlo simulation. Epidemiological data also were used to convert serotype data to virulence (V) values for use in the QMRA. A Monte Carlo simulation model based in Excel and simulated with @Risk predicted Salmonella P, N, serotype, and V as a function of a serving size of one (58 g) to eight (464 g) chicken livers. Salmonella P of chicken livers was 72.5% (58 of 80) per 58 g. Four Salmonella serotypes were isolated from chicken livers: (i) Infantis (P = 28%, V = 4.5); (ii) Enteritidis (P = 15%, V = 5); (iii) Typhimurium (P = 15%, V = 4.8); and (iv) Kentucky (P = 15%, V = 0.8). Salmonella N was 1.76 log MPN/58 g (median) with a range of 0 to 4.67 log MPN/58 g, and the median Salmonella N was not affected (P > 0.05) by serotype. The model predicted a nonlinear increase (P ≤ 0.05) of Salmonella P from 72.5%/58 g to 100%/464 g, a minimum N of 0 log MPN/58 g to 1.28 log MPN/464 g, and a median N from 1.76 log MPN/58 g to 3.22 log MPN/464 g. Regardless of serving size, predicted maximum N was 4.74 log MPN per serving, mean V was 3.9 per serving, and total N was 6.65 log MPN per lot (10,000 chicken livers). The data acquired and modeled in this study address an important data gap in the QMRA for Salmonella and whole chicken liver. HIGHLIGHTS
Article
Full-text available
Detection of pathogens in food processing facilities by routine environmental monitoring (EM) is essential to reduce the risk of foodborne illness but is complicated by the complexity of equipment and environment surfaces. To optimize design of EM programs, we developed EnABLe (“Environmental monitoring with an Agent-Based Model of Listeria”), a detailed and customizable agent-based simulation of a built environment. EnABLe is presented here in a model system, tracing Listeria spp. (LS) (an indicator for conditions that allow the presence of the foodborne pathogen Listeria monocytogenes) on equipment and environment surfaces in a cold-smoked salmon facility. EnABLe was parameterized by existing literature and expert elicitation and validated with historical data. Simulations revealed different contamination dynamics and risks among equipment surfaces in terms of the presence, level and persistence of LS. Grouping of surfaces by their LS contamination dynamics identified connectivity and sanitary design as predictors of contamination, indicating that these features should be considered in the design of EM programs to detect LS. The EnABLe modeling approach is particularly timely for the frozen food industry, seeking science-based recommendations for EM, and may also be relevant to other complex environments where pathogen contamination presents risks for direct or indirect human exposure.
Article
Full-text available
This study assessed the growth of Listeria monocytogenes in milkshakes made using the process-contaminated ice cream associated with a listeriosis outbreak in comparison to milkshakes made with artificially contaminated ice cream. For all temperatures, growth kinetics including growth rates, lag phases, maximum populations, and population increases were determined for the naturally and artificially derived contaminants at 5, 10, 15, and 25°C storage for 144 h. The artificially inoculated L. monocytogenes presented lower growth rates and shorter lag phases than the naturally contaminated populations at all temperatures except for 5°C, where the reverse was observed. At 25°C, lag phases of the naturally and artificially contaminated L. monocytogenes were 11.6 and 7.8 h, respectively. The highest increase in population was observed for the artificially inoculated pathogen at 15°C after 96 h (6.16 log CFU/mL) of storage. Growth models for both contamination states in milkshakes were determined. In addition, this study evaluated the antimicrobial effectiveness of flavoring agents, including strawberry, chocolate and mint, on the growth of the pathogen in milkshakes during 10°C storage. All flavor additions resulted in decreased growth rates of L. monocytogenes for both contamination states. The addition of chocolate and mint flavoring also resulted in significantly longer lag phases for both contamination states. This study provides insight into the differences in growth between naturally and artificially contaminated L. monocytogenes in a food product.
Article
Full-text available
The growth characteristics of Listeria monocytogenes inoculated onto frozen foods (corn, green peas, crabmeat, and shrimp) and thawed by being stored at 4, 8, 12, and 20°C were investigated. The growth parameters, lag-phase duration (LPD) and exponential growth rate (EGR), were determined by using a two-phase linear growth model as a primary model and a square root model for EGR and a quadratic model for LPD as secondary models, based on the growth data. The EGR model predictions were compared with growth rates obtained from the USDA Pathogen Modeling Program, calculated with similar pH, salt percentage, and NaNO2 parameters, at all storage temperatures. The results showed that L. monocytogenes grew well in all food types, with the growth rate increasing with storage temperature. Predicted EGRs for all food types demonstrated the significance of storage temperature and similar growth rates among four food types. The predicted EGRs showed slightly slower rate compared with the values from the U.S. Department of Agriculture Pathogen Modeling Program. LPD could not be accurately predicted, possibly because there were not enough sampling points. These data established by using real food samples demonstrated that L. monocytogenes can initiate growth without a prolonged lag phase even at refrigeration temperature (4°C), and the predictive models derived from this study can be useful for developing proper handling guidelines for thawed frozen foods during production and storage.
Chapter
Full-text available
This chapter illustrates how unitary food processing operations and successive food operations quantitatively impact exposure assessment and individual and public health risks. It focuses on the use of risk-based metrics in food processing. Several examples are available in the literature to describe the effect of food processing on consumer exposure to a particular hazard by consumption of a contaminated food or to illustrate the impact of a succession of unitary food processing operations on individual and/or public health risks. Sensitivity analysis is used as a statistical tool to identify the most impacting steps on the final microbial concentration or risk. The safety of food processes relies on the application of good manufacturing and good hygiene practices and on a food management system such as the hazard analysis - critical control point (HACCP) program. Food safety operators are more and more aware of risk-based approaches to manage the safety of food production operations.
Article
Full-text available
The relationship between the number of ingested Listeria monocytogenes cells in food and the likelihood of developing listeriosis is not well understood. Data from an outbreak of listeriosis linked to milkshakes made from ice cream produced in 1 factory showed that contaminated products were distributed widely to the public without any reported cases, except for 4 cases of severe illness in persons who were highly susceptible. The ingestion of high doses of L. monocytogenes by these patients infected through milkshakes was unlikely if possible additional contamination associated with the preparation of the milkshake is ruled out. This outbreak illustrated that the vast majority of the population did not become ill after ingesting a low level of L. monocytogenes but raises the question of listeriosis cases in highly susceptible persons after distribution of low-level contaminated products that did not support the growth of this pathogen. © 2016, Centers for Disease Control and Prevention (CDC). All rights reserved.
Article
Full-text available
A most-probable-number (MPN) method was used to enumerate Listeria monocytogenes in 2,320 commercial ice cream scoops manufactured on a production line that was implicated in a 2015 listeriosis outbreak in the United States. The analyzed samples were collected from seven lots produced in November 2014, December 2014, January 2015, and March 2015. L. monocytogenes was detected in 99% (2,307 of 2,320) of the tested samples (lower limit of detection, 0.03 MPN/g), 92% of which were contaminated at <20 MPN/g. The levels of L. monocytogenes in these samples had a geometric mean per lot of 0.15 to 7.1 MPN/g. The prevalence and enumeration data from an unprecedented large number of naturally contaminated ice cream products linked to a listeriosis outbreak provided a unique data set for further understanding the risk associated with L. monocytogenes contamination for highly susceptible populations. © 2016, International Association for Food Protection. All rights reserved.
Article
Full-text available
Cross-contamination, improper holding temperatures, and insufficient sanitary practices are known retail practices that may lead to product contamination and growth of Listeria monocytogenes. However, the relative importance of control options to mitigate the risk of invasive listeriosis from ready-to-eat (RTE) products sliced or prepared at retail is not well understood. This study illustrates the utility of a quantitative risk assessment model described in a first article of this series (Pouillot, R., D. Gallagher, J. Tang, K. Hoelzer, J. Kause, and S. B. Dennis, J. Food Prot. 78:134–145, 2015) to evaluate the public health impact associated with changes in retail deli practices and interventions. Twenty-two mitigation scenarios were modeled and evaluated under six different baseline conditions. These scenarios were related to sanitation, worker behavior, use of growth inhibitors, cross-contamination, storage temperature control, and reduction of the level of L. monocytogenes on incoming RTE food products. The mean risk per serving of RTE products obtained under these scenarios was then compared with the risk estimated in the baseline condition. Some risk mitigations had a consistent impact on the predicted listeriosis risk in all baseline conditions (e.g. presence or absence of growth inhibitor), whereas others were greatly dependent on the initial baseline conditions or practices in the deli (e.g. preslicing of products). Overall, the control of the bacterial growth and the control of contamination at its source were major factors of listeriosis risk in these settings. Although control of cross-contamination and continued sanitation were also important, the decrease in the predicted risk was not amenable to a simple solution. Findings from these predictive scenario analyses are intended to encourage improvements to retail food safety practices and mitigation strategies to control L. monocytogenes in RTE foods more effectively and to demonstrate the utility of quantitative risk assessment models to inform risk management decisions.