Conference PaperPDF Available

Evaluation And Optimization Of Scrapie Surveillance In Great Britain: A Bayesian Framework

Authors:

Abstract

There are two main active surveillance sources in Great Britain: the fallen stock (FS) and the abattoir survey (AS). The FS is a volunteer scheme where farmers report dead sheep on farm. In constrast the AS is a random survey at the slaughterhouse. Del Río Vilas et al.(2008) estimated using capture-recapture methods the holding prevalence as 0.45%. Ortiz Pelaez et al. (2009) reported a 0.65% within-holding prevalence. There is evidence that scrapie prevalence has been reduced in Great Britain in the last years, however we consider it necessary to evaluate the power of detection of the current surveillance. Objective: Calculate the sensitivity of the surveillance system and determine an optimal sampling strategy combining both sources. We applied a Hierarchical Bayesian Mixture Model based on previous studies (Branscum et al, 2006). We combined both surveillance sources to study the factors involved in the reduction of the sensitivity at country, county and holding level. We modelled scenarios where we compared a targeted surveillance (stratification by county) and a more practical surveillance approach (stratification by holding size). We proved that the sensitivity of the surveillance could be improved if we sample more sheep within holdings rather than testing more holdings and a small number of sheep in each holding. We also reported the impact of changing several parameters in the model. Stratification by holding size was shown to be preferable to stratifying by county. The sensitivity of the current sampling strategy was established as less than 4.85%.
Evaluation And Optimization Of Scrapie Surveillance In Great Britain: A
Bayesian Framework
Vidal Diez; A1 Arnold; ME1 Del Río Vilas; V2
1 Centre for Epidemiology and Risk Analysis, Veterinary Laboratories Agency, United Kingdom
2 Department for Environment, Food and Rural Affairs, London, United Kingdom
ABSTRACT
There are two main active surveillance sources in Great Britain: the fallen stock (FS) and the abattoir survey
(AS). The FS is a volunteer scheme where farmers report dead sheep on farm. In constrast the AS is a random
survey at the slaughterhouse. Del Río Vilas et al.(2008) estimated using capture-recapture methods the holding
prevalence as 0.45%. Ortiz Pelaez et al. (2009) reported a 0.65% within-holding prevalence. There is evidence
that scrapie prevalence has been reduced in Great Britain in the last years, however we consider it necessary to
evaluate the power of detection of the current surveillance.
Objective: Calculate the sensitivity of the surveillance system and determine an optimal sampling strategy
combining both sources.
We applied a Hierarchical Bayesian Mixture Model based on previous studies (Branscum et al, 2006).
We combined both surveillance sources to study the factors involved in the reduction of the sensitivity at country,
county and holding level. We modelled scenarios where we compared a targeted surveillance (stratification by
county) and a more practical surveillance approach (stratification by holding size).
We proved that the sensitivity of the surveillance could be improved if we sample more sheep within holdings
rather than testing more holdings and a small number of sheep in each holding. We also reported the impact of
changing several parameters in the model. Stratification by holding size was shown to be preferable to stratifying
by county. The sensitivity of the current sampling strategy was established as less than 4.85%.
INTRODUCTION
The prevalence of classical scrapie in Great Britain has decreased since an EU-wide surveillance and Scrapie
control programme was created in 2002. The active surveillance system is based on two main sources: the
abattoir survey (AS) and the fallen stock (FS). DEFRA has sampled different number of sheep through each
stream every year. The FS used to detect more classical scrapie than the AS because infected sheep have high
probability of dying on farm. However atypical scrapie is mainly detected through the AS.
The evaluation of the sensitivity of active surveillance for any disease is essential in order to modify the system
to reflect the disease evolution. It becomes more important for diseases like TSE where testing is expensive and
the prevalence is very low.
Both surveys were designed to produce an estimate of the prevalence of scrapie in the sheep population [further
details of these two surveys are reported elsewhere (Elliott et al., 2005; Del Rio Vilas et al., 2005)]
Sample sizes are calculated based on the country’s adult sheep population and are sufficient to detect a
prevalence of 0.5% and 0.05% in the FS and AS respectively. The geographical distribution of the samples
follows a basic proportional allocation at the two points of sampling: the Animal Health Divisional Office (AHDO)
for the FS and a selection of slaughterhouses for the AS.. This narrow approach, however, seems wasteful
particularly when the application of control measures is done at the holding level (Ortiz-Pelaez and Del Rio Vilas,
2009).
Ebel et al. (2008) discussed several approaches which assess the effectiveness of surveillance systems
assuming a design prevalence below which a population can be considered disease free. Branscum et al. (2006)
used Bayesian Hierarchical Models (BHM) where he included non-infected models by using mixture models.
This approach provides great flexibility to analyse multiple levels and parameters within the surveillance systems
in order to find ways of improvement.
We aimed to study the efficacy of the sampling strategy and the accuracy of the current holding prevalence
estimates. Three aggregation levels were introduced into the model and other sampling strategies were also
investigated.
METHODOLOGY
Data
In order to represent the current surveillance system in Great Britain, we collected the following information:
The population and location of adult sheep in Great Britain from the GB census database of 2004.
Surveillance results from 2005 at holding (Del Rio Vilas et al, 2008) and within holding level (Ortiz
Pelaez et al, 2009)
European standard estimates of within-holding prevalence.
Number of collected adult sheep from the National Fallen Stock Scheme and Company (NFSCo).
Model
Three main stages make up the structure of the solution process:
On the first stage we generated the population and its level of infection at different aggregation levels. Then we
emulated the sampling strategy of both surveillance sources. Our samples were simulated with similar levels of
infection and sample sizes found in previous years. We saved the sample results.
The second stage consisted of implementing the BHM. The prior information was represented by distributions
which informed the level of infection in the population. The data came from the previously simulated sample
results. Two outputs were obtained from the model: the 10th percentiles of the distributions of the proportion of
truly infected holdings detected within the sample and the proportion of truly infected holdings within the
population.
The last step involved the repetition of stage one and two generating different populations. The final output will
be the 10th percentile of the distributions of the results across each population.
RESULTS
We investigated two main sampling strategies: 1) a simulation of a targeted surveillance scenario implemented
by a multi-stage sampling design with a stratification by county. 2) A simulation modeling a more practical
surveillance where we stratified by holding-size. In general the latter strategy produced better sensitivity.
A significant decrease in the sensitivity was found when we tested small numbers of sheep within a holding but
testing many holdings in comparison to testing fewer holdings but sampling more sheep within each holding.
The influence of the within-holding prevalence was presented comparing scenarios with the estimates of Ortiz-
Pelaez and Del Río Vilas (2009) and the suggested estimates by the European Union. We reported the impact
of the parameter which represents the probability that an infected sheep will die on farm.
The simulation of a scenario similar to the current situation produced a sensitivity estimate of 3.01% for the fallen
stock stream. The abattoir survey did not even reach 1% sensitivity.
Some scenarios studied the possibility of using the National Fallen Stock Scheme and Company as a
surveillance source.
CONCLUSIONS
We conclude that BHM models are very flexible and powerful for investigating issues in surveillance systems.
Optimal sampling designs can be found using these models.
We highlighted that the sensitivity was significantly affected by the small sample size within the holdings. A
sampling design based on stratification by holding size was found more efficient than a target surveillance
strategy. In general terms the sensitivity of the system is poor but the voluntary nature of the FS and the small
probability of finding classical scrapie through the AS make changes difficult. We propose to investigate other
possible sources like the National Fallen Stock Scheme and Company.
REFERENCES
Branscum A.J., Johnson W.O, Gardner,I.A.(2006) Sample size calculations for disease freedom and prevalence
estimation surveys. Statistics in Medicine 25;2678-2674
Del Río Vilas, V.J., Sayers, R., Sivam, K., Pfeiffer, D., Guitian, J., Wilesmith, J.W. (2005) A case study of
capture–recapture methodology using scrapie surveillance data in Great Britain. Prev Vet Med 67(4) 303-317
Del Rio Vilas, V.J., Bohning, D. (2008). Application of one-list capture-recapture models to scrapie surveillance
data in Great Britain. Prev Vet Med 85; 253-266.
Ebel E.D, Williams, M.S., Tomlinson, S.M (2008) Estimating herd prevalence of bovine brucellosis in 46 U.S.A.
states using slaughter surveillance. Prev Vet Med 85; 295-316
Elliott, H., Gubbins, S., Ryan, J., Ryder, S., Tongue, S.,Watkins, G.,Wilesmith, J., (2005). Prevalence of scrapie
in sheep in Great Britain estimated from abattoir surveys during 2002 and 2003. Vet. Rec. 157, 418–419.
Ortiz Pelaez, A., Del Rio Vilas, V.J. (2009). Within-holding prevalence of sheep classical scrapie in Great Britain.
BMC Vet Res 5:1
... A recent systematic review of evaluations of public and animal health surveillance systems revealed the absence of a universally accepted standard for evaluation of animal health surveillance (Drewe et al., 2012). Many different approaches have been applied without consistency or apparent agreement on what is an optimal protocol (Carpenter, 2002;St€ ark et al., 2006;United States Department of Agriculture, 2006;Moran and Fofana, 2007;Hadorn et al., 2008;Birch et al., 2009;Vidal Diez et al., 2009). Recently, the OASIS tool has been developed for the assessment of animal health and food safety surveillance systems (Hendrikx et al., 2011). ...
Article
Animal health surveillance programmes may change in response to altering requirements or perceived weaknesses but are seldom subjected to any formal evaluation to ensure that they provide valuable information in an efficient manner. The literature on the evaluation of animal health surveillance systems is sparse, and those that are published may be unstructured and therefore incomplete. To address this gap, we have developed SERVAL, a SuRveillance EVALuation framework, which is novel and aims to be generic and therefore suitable for the evaluation of any animal health surveillance system. The inclusion of socio-economic criteria ensures that economic evaluation is an integral part of this framework. SERVAL was developed with input from a technical workshop of international experts followed by a consultation process involving providers and users of surveillance and evaluation data. It has been applied to a range of case studies encompassing different surveillance and evaluation objectives. Here, we describe the development, structure and application of the SERVAL framework. We discuss users' experiences in applying SERVAL to evaluate animal health surveillance systems in Great Britain.
... Currently, there is no universally accepted standard for evaluation of animal health surveillance. Many different approaches have been applied [e.g. 2, [12][13][14][15][16] without consistency or apparent agreement on what is optimal. Evaluation of human health surveillance systems is more commonly practised and several generic guidelines exist for public health surveillance evaluation [17][18][19][20]. ...
Article
Full-text available
Disease surveillance programmes ought to be evaluated regularly to ensure they provide valuable information in an efficient manner. Evaluation of human and animal health surveillance programmes around the world is currently not standardized and therefore inconsistent. The aim of this systematic review was to review surveillance system attributes and the methods used for their assessment, together with the strengths and weaknesses of existing frameworks for evaluating surveillance in animal health, public health and allied disciplines. Information from 99 articles describing the evaluation of 101 surveillance systems was examined. A wide range of approaches for assessing 23 different system attributes was identified although most evaluations addressed only one or two attributes and comprehensive evaluations were uncommon. Surveillance objectives were often not stated in the articles reviewed and so the reasons for choosing certain attributes for assessment were not always apparent. This has the potential to introduce misleading results in surveillance evaluation. Due to the wide range of system attributes that may be assessed, methods should be explored which collapse these down into a small number of grouped characteristics by focusing on the relationships between attributes and their links to the objectives of the surveillance system and the evaluation. A generic and comprehensive evaluation framework could then be developed consisting of a limited number of common attributes together with several sets of secondary attributes which could be selected depending on the disease or range of diseases under surveillance and the purpose of the surveillance. Economic evaluation should be an integral part of the surveillance evaluation process. This would provide a significant benefit to decision-makers who often need to make choices based on limited or diminishing resources.
Article
Full-text available
Data from the Compulsory Scrapie Flocks Scheme (CSFS), part of the compulsory eradication measures for the control of scrapie in the EU, have been used to estimate the within-holding prevalence of classical scrapie in Great Britain (GB). Specifically data from one of the testing routes within the CSFS have been used; the initial cull (IC), whereby two options can be applied: the whole flock cull option by which the entire flock is depopulated, and the genotyping and cull of certain genotypes. Between April 2005 and September 2007, 25,316 suitable samples, submitted from 411 flocks in 213 scrapie-affected holdings in Great Britain, were tested for scrapie. The predicted within-holding prevalence for the initial cull was 0.65% (95% CI: 0.55-0.75). For the whole cull option was 0.47% (95% CI: 0.32-0.68) and for the genotype and cull or mixed option (both options applied in different flocks of the same holding), the predicted within-holding prevalence was 0.7% (95% CI: 0.6-0.83). There were no significant differences in the within-flock prevalence between countries (England, Scotland and Wales) or between CSFS holdings by the surveillance stream that detected the index case. The number of CSFS flocks on a holding did not affect the overall within-holding prevalence of classical scrapie. These estimates are important in the discussion of the epidemiological implications of the current EU testing programme of scrapie-affected flocks and to inform epidemiological and mathematical models. Furthermore, these estimates may provide baseline data to assist the design of future surveillance activities and control policies with the aim to increase their efficiency.
Article
We applied capture-recapture methodology (CRC) to data from three surveillance sources (statutory notification, abattoir survey (AS) and fallen stock (FS) survey) to estimate the number of holdings infected with scrapie in Great Britain and to assess the sensitivity of the surveillance network. Between January 1, 2002 and March 31, 2003, 144 holdings were identified by the three sources. Using CRC modelling techniques, we estimated a minimum lower bound for the total number of holdings infected as 642. A biologically plausible positive dependence between the statutory reporting and the fallen stock survey was found statistically significant. The sensitivity of the three sources combined was very low. The integration of the three overlapping sources provided a better understanding of the interactions within the surveillance network. However, the scarcity of the data and reduced overlapping among sources only allowed for very cautious inferences to be drawn about the true proportion of scrapie affected holdings in the national population. Future surveys and surveillance activities should be planned such that the resulting data can be used more effectively as part of CRC modelling approaches.
Article
We developed a Bayesian approach to sample size calculations for studies designed to estimate disease prevalence that uses a hierarchical model for estimating the proportion of infected clusters (cluster-level prevalence) within a country or region. The clusters may, for instance, be villages within a region, cities within a state, or herds within a country. Our model allows for clusters with zero prevalence and for variability in prevalences among infected clusters. Moreover, uncertainty about diagnostic test accuracy and within-cluster prevalences is accounted for in the model. A predictive approach is used to address the issue of sample size selection in human and animal health surveys. We present sample size calculations for surveys designed to substantiate freedom of a region from an infectious agent (disease freedom surveys) and for surveys designed to estimate cluster-level prevalence of an endemic disease (prevalence estimation surveys). In disease freedom surveys, for instance, assuming the cluster-level prevalence for a particular infectious agent in the region is greater than a maximum acceptable threshold, a sample size combination consisting of the number of clusters sampled and number of subjects sampled per cluster can be determined for which authorities conducting the survey detect this excessive cluster-level prevalence with high predictive probability. The method is straightforward to implement using the Splus/R library emBedBUGS together with WinBUGS.
Article
In this paper, we apply one-list capture-recapture models to estimate the number of scrapie-affected holdings in Great Britain. We applied this technique to the Compulsory Scrapie Flocks Scheme dataset where cases from all the surveillance sources monitoring the presence of scrapie in Great Britain, the abattoir survey, the fallen stock survey and the statutory reporting of clinical cases, are gathered. Consequently, the estimates of prevalence obtained from this scheme should be comprehensive and cover all the different presentations of the disease captured individually by the surveillance sources. Two estimators were applied under the one-list approach: the Zelterman estimator and Chao's lower bound estimator. Our results could only inform with confidence the scrapie-affected holding population with clinical disease; this moved around the figure of 350 holdings in Great Britain for the period under study, April 2005-April 2006. Our models allowed the stratification by surveillance source and the input of covariate information, holding size and country of origin. None of the covariates appear to inform the model significantly.
Article
Making valid inferences about herd prevalence from data collected at slaughter is difficult because the observed sample is dependent on the number of animals sampled from each herd, which varies with herd size and culling practices, and the probability of a positive test result, which depends on variable within-herd prevalence levels as well as test sensitivity and specificity. In this study, brucellosis herd prevalence among beef cow-calf operations is estimated from slaughter surveillance data using a method that combines process modeling with Bayesian inference. Inferences are made for two populations; the first population comprises cow-calf beef herds in a typical U.S. state. The second population represents all beef herds in a collection of 46 low-risk states. The Bayesian Monte Carlo method used in this study links process model inputs to observed surveillance results via Bayes Theorem. The surveillance evidence across multiple years is accumulated at a discounted rate based on the probability of introducing new infection into an area. The process model's inputs include herd size, culling rate per herd, within-herd prevalence, serologic test performance, and the probability of successfully investigating positive results. The surveillance results comprise the number of cows and bulls tested at slaughter and the number of affected herds detected each year. The results find at least 95% confidence that brucellosis herd prevalence among beef cow-calf herds is less than 0.014% (3 per 21,500 herds) and 0.00081% (5 per 6,15,770) after 5 years of slaughter surveillance (with no detections of affected herds) in a typical U.S. state and across 46 low-risk U.S. states, respectively. These results were based on conservative modeling assumptions, but sensitivity analysis suggests only slight changes in the results from changing the assumed process model input values. The most influential analytic input was the probability of introducing new infection into a putatively brucellosis-free state or group of states.
Sample size calculations for disease freedom and prevalence estimation surveys A case study of capture–recapture methodology using scrapie surveillance data in Great Britain Application of one-list capture-recapture models to scrapie surveillance data in Great Britain
  • A J Branscum
  • W Johnson
  • I A Gardner
  • Del
  • V J Vilas
  • R Sayers
  • K Sivam
  • D Pfeiffer
  • J Guitian
  • J W Wilesmith
Branscum A.J., Johnson W.O, Gardner,I.A.(2006) Sample size calculations for disease freedom and prevalence estimation surveys. Statistics in Medicine 25;2678-2674 Del Río Vilas, V.J., Sayers, R., Sivam, K., Pfeiffer, D., Guitian, J., Wilesmith, J.W. (2005) A case study of capture–recapture methodology using scrapie surveillance data in Great Britain. Prev Vet Med 67(4) 303-317 Del Rio Vilas, V.J., Bohning, D. (2008). Application of one-list capture-recapture models to scrapie surveillance data in Great Britain. Prev Vet Med 85; 253-266