[Show abstract][Hide abstract] ABSTRACT: Background:
To describe approaches used in systematic reviews of diagnostic test accuracy studies for assessing variability in estimates of accuracy between studies and to provide guidance in this area.
Meta-analyses of diagnostic test accuracy studies published between May and September 2012 were systematically identified. Information on how the variability in results was investigated was extracted.
Of the 53 meta-analyses included in the review, most (n=48; 91 %) presented variability in diagnostic accuracy estimates visually either through forest plots or ROC plots and the majority (n=40; 75 %) presented a test or statistical measure for the variability. Twenty-eight reviews (53 %) tested for variability beyond chance using Cochran's Q test and 31 (58 %) reviews quantified it with I(2). 7 reviews (13 %) presented between-study variance estimates (τ(2)) from random effects models and 3 of these presented a prediction interval or ellipse to facilitate interpretation. Half of all the meta-analyses specified what was considered a significant amount of variability (n=24; 49 %).
Approaches to assessing variability in estimates of accuracy varied widely between diagnostic test accuracy reviews and there is room for improvement. We provide initial guidance, complemented by an overview of the currently available approaches.
Full-text · Article · Dec 2016 · BMC Medical Research Methodology
[Show abstract][Hide abstract] ABSTRACT: Objectives:
Exposure to asbestos fibres increases the risk of mesothelioma and lung cancer. Although the vast majority of mesothelioma cases are caused by asbestos exposure, the number of asbestos-related lung cancers is less clear. This number cannot be determined directly as lung cancer causes are not clinically distinguishable but may be estimated using varying modelling methods.
We applied three different modelling methods to the Dutch population supplemented with uncertainty ranges (UR) due to uncertainty in model input values. The first method estimated asbestos-related lung cancer cases directly from observed and predicted mesothelioma cases in an age-period-cohort analysis. The second method used evidence on the fraction of lung cancer cases attributable (population attributable risk (PAR)) to asbestos exposure. The third method incorporated risk estimates and population exposure estimates to perform a life table analysis.
The three methods varied substantially in incorporated evidence. Moreover, the estimated number of asbestos-related lung cancer cases in the Netherlands between 2011 and 2030 depended crucially on the actual method applied, as the mesothelioma method predicts 17 500 expected cases (UR 7000-57 000), the PAR method predicts 12 150 cases (UR 6700-19 000), and the life table analysis predicts 6800 cases (UR 6800-33 850).
The three different methods described resulted in absolute estimates varying by a factor of ∼2.5. These results show that accurate estimation of the impact of asbestos exposure on the lung cancer burden remains a challenge.
No preview · Article · Feb 2016 · Occupational and environmental medicine
[Show abstract][Hide abstract] ABSTRACT: Pairwise meta-analysis is an established statistical tool for synthesizing evidence from multiple trials, but it is informative only about the relative efficacy of two specific interventions. The usefulness of pairwise meta-analysis is thus limited in real-life medical practice, where many competing interventions may be available for a certain condition and studies informing some of the pairwise comparisons may be lacking. This commonly encountered scenario has led to the development of network meta-analysis (NMA). In the last decade, several applications, methodological developments, and empirical studies in NMA have been published, and the area is thriving as its relevance to public health is increasingly recognized. This article presents a review of the relevant literature on NMA methodology aiming to pinpoint the developments that have appeared in the field.
Full-text · Article · Jan 2016 · Research Synthesis Methods
[Show abstract][Hide abstract] ABSTRACT: Objective: To identify risk indicators for referral during labor from community midwife to a gynecologist in a prospective cohort of women with a singleton term pregnancy, starting labor with a community midwife between 2000 and 2007, registered in the Dutch national perinatal registry.Main outcome measures: Referral from community midwife to a gynecologist during labor, because of fetal distress, failure to progress in second stage of labor, meconium stained amniotic fluid, failure to progress in first stage of labor, wish for pain relief, a combination of other less urgent reasons or no referral (reference).Results: A total of 241 595 (32%) were referred from community midwife to a gynecologist during labor, because of fetal distress (FD;5%), failure to progress in second stage of labor (FTP2;14%), meconium stained amniotic fluid (MSAF;24%), failure to progress in first stage of labor (FTP1;17%), wish for pain relief (WFPR;7%) or a combination of other less urgent reasons, for example, malpresentation (e.g. breech) or other nonspecified problems (OTHER;33%). The strongest overall risk indicators were gestational age (lower risk of referral because of FD, FTP2, MSAF, FTP1 and WFPR and a higher risk of referral because of OTHER at a gestational age between 37+0 and 37+6 weeks, and higher risks of referral for all reasons at a gestational age ≥41+0 when compared to a gestational age between 38 +0 and 40 +6 weeks and no referral), the intended place of delivery (higher risk of all types of referral compared to no referral when the intended place of delivery was either at a midwife-led birth center or a hospital instead of at home) and birth history (higher risk of all types of referral compared to no referral when women had a history of instrumental vaginal delivery or when they were nulliparous instead of being multiparous without a history of an instrument vaginal delivery). Risk indicators associated with specific reasons of referral were maternal age, ethnicity, degree of urbanization, social economic status, neonatal gender and birth weight.Conclusions: Among low-risk pregnant women, a referral during labor is associated with readily available risk indicators. These risk indicators may be used to increase referral risk awareness and to counsel women for the intended place to start labor.
No preview · Article · Nov 2015 · Journal of Maternal-Fetal and Neonatal Medicine
[Show abstract][Hide abstract] ABSTRACT: Objectives:
The objective of this study was to evaluate the performance of goodness-of-fit testing to detect relevant violations of the assumptions underlying the criticized 'standard' 2-class latent class model. Often used to obtain sensitivity and specificity estimates for diagnostic tests in the absence of a gold reference standard, this model relies on assuming that diagnostic test errors are independent. When this assumption is violated, accuracy estimates may be biased: goodness-of-fit testing is often used to evaluate the assumption and prevent bias.
Study design and setting:
We investigate the performance of goodness-of-fit testing by Monte Carlo simulation. The simulation scenarios are based on three empirical examples.
Goodness-of-fit tests lack power to detect relevant misfit of the standard 2-class latent class model at sample sizes that are typically found in empirical diagnostic studies. The goodness-of-fit tests that are based on asymptotic theory are not robust to the sparseness of data. A parametric bootstrap procedure improves the evaluation of goodness-of-fit in the case of sparse data.
Our simulation study suggests that relevant violation of the local independence assumption underlying the standard 2-class latent class model may remain undetected in empirical diagnostic studies, potentially leading to biased estimates of sensitivity and specificity.
No preview · Article · Nov 2015 · Journal of clinical epidemiology
[Show abstract][Hide abstract] ABSTRACT: Background:
Medical guidelines increasingly use risk stratification and implicitly assume that individuals classified in the same risk category form a homogeneous group, while individuals with similar, or even identical, predicted risks can still be very different. We evaluate a strategy to identify homogeneous subgroups typically comprising predicted risk categories to allow further tailoring of treatment allocation and illustrate this strategy empirically for cardiac surgery patients with high postoperative mortality risk.
Using a dataset of cardiac surgery patients (n=6517) we applied cluster analysis to identify homogenous subgroups of patients comprising the high postoperative mortality risk group (EuroSCORE≥15%). Cluster analyses were performed separately within younger (<75years) and older (≥75years) patients. Validity measures were calculated to evaluate quality and robustness of the identified subgroups.
Within younger patients two distinct and robust subgroups were identified, differing mainly in preoperative state and indication of recent myocardial infarction or unstable angina. In older patients, two distinct and robust subgroups were identified as well, differing mainly in preoperative state, presence of chronic pulmonary disease, previous cardiac surgery, neurological dysfunction disease and pulmonary hypertension.
We illustrated a feasible method to identify homogeneous subgroups of individuals typically comprising risk categories. This allows a single treatment strategy - optimal only on average, across all individuals in a risk category - to be replaced by subgroup-specific treatment strategies, bringing us another step closer to individualized care. Discussions on allocation of cardiac surgery patients to different interventions may benefit from focusing on such specific subgroups.
No preview · Article · Nov 2015 · International journal of cardiology
[Show abstract][Hide abstract] ABSTRACT: Objective To validate all diagnostic prediction models for ruling out pulmonary embolism that are easily applicable in primary care.
Design Systematic review followed by independent external validation study to assess transportability of retrieved models to primary care medicine.
Setting 300 general practices in the Netherlands.
Participants Individual patient dataset of 598 patients with suspected acute pulmonary embolism in primary care.
Main outcome measures Discriminative ability of all models retrieved by systematic literature search, assessed by calculation and comparison of C statistics. After stratification into groups with high and low probability of pulmonary embolism according to pre-specified model cut-offs combined with qualitative D-dimer test, sensitivity, specificity, efficiency (overall proportion of patients with low probability of pulmonary embolism), and failure rate (proportion of pulmonary embolism cases in group of patients with low probability) were calculated for all models.
Results Ten published prediction models for the diagnosis of pulmonary embolism were found. Five of these models could be validated in the primary care dataset: the original Wells, modified Wells, simplified Wells, revised Geneva, and simplified revised Geneva models. Discriminative ability was comparable for all models (range of C statistic 0.75-0.80). Sensitivity ranged from 88% (simplified revised Geneva) to 96% (simplified Wells) and specificity from 48% (revised Geneva) to 53% (simplified revised Geneva). Efficiency of all models was between 43% and 48%. Differences were observed between failure rates, especially between the simplified Wells and the simplified revised Geneva models (failure rates 1.2% (95% confidence interval 0.2% to 3.3%) and 3.1% (1.4% to 5.9%), respectively; absolute difference −1.98% (−3.33% to −0.74%)). Irrespective of the diagnostic prediction model used, three patients were incorrectly classified as having low probability of pulmonary embolism; pulmonary embolism was diagnosed only after referral to secondary care.
Conclusions Five diagnostic pulmonary embolism prediction models that are easily applicable in primary care were validated in this setting. Whereas efficiency was comparable for all rules, the Wells rules gave the best performance in terms of lower failure rates.
[Show abstract][Hide abstract] ABSTRACT: Prediction models are developed to aid health care providers in estimating the probability that a specific outcome or disease is present (diagnostic prediction models) or will occur in the future (prognostic prediction models), to inform their decision making. Prognostic models here also include models to predict treatment outcomes or responses; in the cancer literature often referred to as predictive models. Clinical prediction models have become abundant. Pathology measurement or results are frequently included as predictors in such prediction models, certainly in the cancer domain. Only when full information on all aspects of a prediction modeling study are clearly reported, risk of bias and potential usefulness of the prediction model can be adequately assessed. Many reviews have illustrated that the quality of reports on the development, validation, and/or adjusting (updating) of prediction models, is very poor. Hence, the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) initiative has developed a comprehensive and user-friendly checklist for the reporting of studies on, both diagnostic and prognostic, prediction models. The TRIPOD Statement intends to improve the transparency and completeness of reporting of studies that report solely on development, both development and validation, and solely on the validation (with or without updating) of diagnostic or prognostic, including predictive, models.
No preview · Article · Sep 2015 · Advances in anatomic pathology
[Show abstract][Hide abstract] ABSTRACT: Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI.
Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics.
57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances.
Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.
Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.
[Show abstract][Hide abstract] ABSTRACT: Healthcare provision is increasingly focused on the prediction of patients’ individual risk for developing a particular health outcome in planning further tests and treatments. There has been a steady increase in the development and publication of prognostic models for various maternal and fetal outcomes in obstetrics. We undertook a systematic review to give an overview of the current status of available prognostic models in obstetrics in the context of their potential advantages and the process of developing and validating models. Important aspects to consider when assessing a prognostic model are discussed and recommendations on how to proceed on this within the obstetric domain are given.
Full-text · Article · Jun 2015 · American journal of obstetrics and gynecology
[Show abstract][Hide abstract] ABSTRACT: BACKGROUND Valid comparison between hospitals for benchmarking or pay-for-performance incentives requires accurate correction for underlying disease severity (case-mix). However, existing models are either very simplistic or require extensive manual data collection. OBJECTIVE To develop a disease severity prediction model based solely on data routinely available in electronic health records for risk-adjustment in mechanically ventilated patients. DESIGN Retrospective cohort study. PARTICIPANTS Mechanically ventilated patients from a single tertiary medical center (2006-2012). METHODS Predictors were extracted from electronic data repositories (demographic characteristics, laboratory tests, medications, microbiology results, procedure codes, and comorbidities) and assessed for feasibility and generalizability of data collection. Models for in-hospital mortality of increasing complexity were built using logistic regression. Estimated disease severity from these models was linked to rates of ventilator-associated events. RESULTS A total of 20,028 patients were initiated on mechanical ventilation, of whom 3,027 deceased in hospital. For models of incremental complexity, area under the receiver operating characteristic curve ranged from 0.83 to 0.88. A simple model including demographic characteristics, type of intensive care unit, time to intubation, blood culture sampling, 8 common laboratory tests, and surgical status achieved an area under the receiver operating characteristic curve of 0.87 (95% CI, 0.86-0.88) with adequate calibration. The estimated disease severity was associated with occurrence of ventilator-associated events. CONCLUSIONS Accurate estimation of disease severity in ventilated patients using electronic, routine care data was feasible using simple models. These estimates may be useful for risk-adjustment in ventilated patients. Additional research is necessary to validate and refine these models.
No preview · Article · Apr 2015 · Infection Control and Hospital Epidemiology
[Show abstract][Hide abstract] ABSTRACT: Background
General practitioners (GP) can safely exclude pulmonary embolism (PE) using the Wells PE-rule combined with D-dimer testing.Objective
To compare the accuracy of a strategy using the Wells rule combined with either a qualitative point of care (POC) D-dimer test performed in primary care or a quantitative laboratory based D-dimer test.Methods
We used data from a prospective cohort study including 598 adults suspected of PE in primary care in the Netherlands. GPs scored the Wells rule and carried out a qualitative POC test. All patients were referred to hospital for reference testing. We obtained quantitative D-dimer-test results as performed in hospital laboratories. The primary outcome was the prevalence of venous thrombo-embolism in low-risk patients.ResultsPrevalence of PE was 12.2%. POC D-dimer-test results were available in 582 patients (97%). Quantitative test results were available in 401 patients (67%). We imputed results in 197 patients. The quantitative test and POC-test missed 1 (0.4%) and 4 patients (1.5%), respectively, with a negative strategy (Wells ≤4 points and D-dimer test negative)(p=0.20). The POC-test could exclude 23 more patients (4%)(p=0.05). The sensitivity and specificity of the Wells rule combined with a POC test was 94.5% and 51.0%, combined with a quantitative test 98.6% and 47.2%, respectively.Conclusions
Combined with the Wells PE-rule both tests are safe in excluding PE. The quantitative test seemed to be safer than the POC test, albeit not statistically significant. The specificity of the POC test was higher resulting in more patients in whom PE could be excluded.This article is protected by copyright. All rights reserved.
No preview · Article · Apr 2015 · Journal of Thrombosis and Haemostasis
[Show abstract][Hide abstract] ABSTRACT: Nagaan of een screening op COPD bij thuiswonende kwetsbare ouderen met klachten van kortademigheid of verminderd inspanningsvermogen zinvol zou kunnen zijn. Daartoe bepaalden we het medicatiegebruik, het aantal ziekenhuisopnames en de sterfte onder patiënten bij wie de screening leidde tot een eerste diagnose COPD.
Een panel van deskundigen bepaalde de diagnose op basis van alle screeningsgegevens, inclusief spirometrie. Follow-upgegevens verzamelden we via de deelnemende huisartsen.
De screening werd uitgevoerd bij 386 oudere huisartspatiënten met kortademigheid of inspanningstolerantie. Bij 84 (21,8%) patiënten leidde de screening tot een niet eerder gestelde diagnose COPD. Van deze 84 waren er 15 (17,9%) binnen zes maanden na de diagnose gestart met inhalatiemedicatie of ze hadden hun bestaande medicatie aangepast, en waren er 27 (32,1%) binnen twaalf maanden opgenomen in een ziekenhuis. In de groep bij wie de screening geen COPD had aangetoond, lag dit laatste percentage significant lager (22,9%). De mortaliteit was in beide groepen vergelijkbaar.
Door kwetsbare ouderen te screenen kunnen veel nieuwe gevallen van COPD worden ontdekt. De screening heeft echter weinig consequenties voor de daaropvolgende behandeling. Een mogelijke verklaring is dat patiënten die niet zelf met hun klachten naar de huisarts stappen, waarschijnlijk toch al minder gemotiveerd zijn voor behandeling.
Full-text · Article · Apr 2015 · Huisarts en wetenschap