Penny Whiting

• PhD
• Group Leader at University of Bristol and University Hospitals Bristol NHS Foundation Trust

BACKGROUND QUADAS, and more recently QUADAS-2, were developed to aid the evaluation of methodological quality within primary diagnostic accuracy studies. However, its current form, QUADAS-2 does not address the unique considerations raised by artificial intelligence (AI) centred diagnostic systems. The rapid progression of the AI diagnostics field...

Background Quality assessment of diagnostic accuracy studies (QUADAS), and more recently QUADAS-2, were developed to aid the evaluation of methodological quality within primary diagnostic accuracy studies. However, its current form, QUADAS-2 does not address the unique considerations raised by artificial intelligence (AI)–centered diagnostic system...

Introduction Standards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, w...

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update t...

The methods and results of systematic reviews should be reported in sufficient detail to allow users to assess the trustworthiness and applicability of the review findings. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed to facilitate transparent and complete reporting of systematic reviews an...

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update t...

For many users of the biomedical literature, abstracts may be the only source of information about a study. Hence, abstracts should allow readers to evaluate the objectives, key design features, and main results of the study. Several evaluations have shown deficiencies in the reporting of journal and conference abstracts across study designs and re...

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update t...

The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did, and what they found. Over the past decade, advances in systematic review methodology and terminology have necessitated an update t...

Assessment of risk of bias is regarded as an essential component of a systematic review on the effects of an intervention. The most commonly used tool for randomised trials is the Cochrane risk-of-bias tool. We updated the tool to respond to developments in understanding how bias arises in randomised trials, and to address user feedback on and limi...

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and progno...

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outco...

Objectives To evaluate the association between the quality of relationship between a person with dementia and their family carer and outcomes for the person with dementia. Design Systematic review. Eligibility criteria Cohort studies of people with clinically diagnosed dementia and their main carers. Exposures of interest were any elements of rel...

[This corrects the article DOI: 10.1186/s41512-016-0001-y.].

Background Over 150 000 cases of suspected transient ischaemic attack (TIA) are referred to outpatient clinics in England each year. The majority of referrals are made by GPS. Aim This study aimed to identify how many patients referred to a TIA clinic actually have TIA (that is, calculate the positive predictive value [PPV] of first-contact healthc...

Background: Over 150 000 cases of suspected transient ischaemic attack (TIA) are referred to outpatient clinics in England each year. The majority of referrals are made by GPs. Aim: This study aimed to identify how many patients referred to a TIA clinic actually have TIA (that is, calculate the positive predictive value [PPV] of first-contact he...

Background There is little evidence on the accuracy of psychosis relapse prediction models. Our objective was to undertake a systematic review of relapse prediction models in psychosis. Method We conducted a literature search including studies that developed and/or validated psychosis relapse prediction models, with or without external model valid...

PRISMA 2009 checklist. (DOC)

Systematic review protocol. (DOCX)

Objectives To summarise evidence on temporary discontinuation of medications to prevent acute kidney injury (AKI). Design Systematic review and meta-analysis of randomised and non-randomised studies. Participants Adults taking diuretics, ACE inhibitors (ACEI), angiotensin receptor blockers (ARB), direct renin inhibitors, non-steroidal anti-inflam...

Objectives To compare the validity of diagnosis of urinary tract infection (UTI) through urine culture between samples processed in routine health service laboratories and those processed in a research laboratory. Population and methods We conducted a prospective diagnostic cohort study in 4808 acutely ill children aged <5 years attending UK prima...

Joris de Groot, Christiana Naaktgeboren, Hans Reitsma, Carl Moons Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, 3584 CG Utrecht, Netherlands Correspondence: Joris de Groot (j.degroot-17@umcutrecht.nl) A major contributor to the rising problem of overdiagnosis, with the subsequent risk of overtreatment, is t...

Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their strengths and weaknesses. We developed ROBINS-I ("Risk Of Bias In Non-randomised Studies-of Interventions"), a new tool for evaluating risk of bias in...

Fig. S1 PRISMA flow diagram of included studies Table S1 Standardization of surgical interventions in the Rescue‐ASDH trial Table S2 Final descriptions of Rescue‐ASDH interventions for the trial protocol

Background: The complexity of surgical interventions has major implications for the design of RCTs. Trials need to consider how and whether to standardize interventions so that, if successful, they can be implemented in practice. Although guidance exists for standardizing non-pharmaceutical interventions in RCTs, their application to surgery is un...

Background It is not clear which young children presenting acutely unwell to primary care should be investigated for urinary tract infection (UTI) and whether or not dipstick testing should be used to inform antibiotic treatment. Objectives To develop algorithms to accurately identify pre-school children in whom urine should be obtained; assess wh...

Introduction Serious adverse outcomes for people with dementia include institutionalisation, hospitalisation, death, development of behavioural and psychiatric symptoms, and reduced quality of life. The quality of the relationship between the person with dementia and their informal/family carer is thought to affect the risk of these outcomes. Howev...

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research. Essential information is often not provided in study reports, impeding the identification, critical appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy studies, the Standards for Reporting of Diagnostic...

Background: Acute kidney injury (AKI) is common and often leads to significant morbidity and/or death. The development of AKI, or complications associated with it, may be due to use of certain medications in at-risk patients experiencing an intercurrent illness. Implicated drugs include diuretics, angiotensin-converting enzyme inhibitors/angiotens...

There is little evidence on how best to present diagnostic information to doctors and whether this makes any difference to clinical management. We undertook a randomised controlled trial to see if different data presentations altered clinicians' decision to further investigate or treat a patient with a fictitious disorder ("Green syndrome") and the...

Patients with substantive bleeding usually require transfusion and/or (re-)operation. Red blood cell (RBC) transfusion is independently associated with a greater risk of infection, morbidity, increased hospital stay and mortality. ROTEM (ROTEM(®) Delta, TEM International GmbH, Munich, Germany; www.rotem.de ), TEG (TEG(®) 5000 analyser, Haemonetics...

To evaluate whether clinicians differ in how they evaluate and interpret diagnostic test information. Systematic review. MEDLINE, EMBASE and PsycINFO from inception to September 2013; bibliographies of retrieved studies, experts and citation search of key included studies. Primary studies that provided information on the accuracy of any diagnostic...

Cannabis and cannabinoid drugs are widely used to treat disease or alleviate symptoms, but their efficacy for specific indications is not clear. To conduct a systematic review of the benefits and adverse events (AEs) of cannabinoids. Twenty-eight databases from inception to April 2015. Randomized clinical trials of cannabinoids for the following in...

Systematic reviews are generally considered to provide the most reliable form of evidence to guide decision makers. Here we describe ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies). ROBIS has been developed using rigorous methodology and is currently aimed at four broad categories of reviews m...

Surgical interventions are complex, with multiple components that require consideration in trial reporting. This review examines the reporting of details of surgical interventions in randomized clinical trials (RCTs) within the context of explanatory and pragmatic study designs. Systematic searches identified RCTs of surgical interventions publishe...

Early diagnosis of acute myocardial infarction (AMI) can ensure quick and effective treatment but only 20% of adults with emergency admissions for chest pain have an AMI. High-sensitivity cardiac troponin (hs-cTn) assays may allow rapid rule-out of AMI and avoidance of unnecessary hospital admissions and anxiety. Objective To assess the clinical e...

NHS expenditure has stagnated since the economic crisis of 2007, resulting in financial pressures. One response is for policy-makers to regulate use of existing health-care technologies and disinvest from inefficiently used health technologies. A key challenge to disinvestment is to identify existing health technologies with uncertain cost-effectiv...

Objectives: To assess cost-effectiveness of high sensitivity troponin (hs-cTn) assays for the management of adults presenting with acute chest pain at the emergency department. Methods: An economic model was constructed to estimate lifetime costs and QALYs of five hs-cTn strategies (differing according to manufacturer, timing of the test, number of...

Background: Bowel cancer is the third most common cancer in the UK. Most bowel cancers are initially treated with surgery, but around 17% spread to the liver. When this happens, sometimes the liver tumour can be treated surgically, or chemotherapy may be used to shrink the tumour to make surgery possible. Kirsten rat sarcoma viral oncogene (KRAS)...

Objectives: To investigate how the summary estimates in diagnostic test accuracy (DTA) systematic reviews are affected when searches are limited to MEDLINE. Study design and setting: A systematic search was performed to identify DTA reviews that had conducted exhaustive searches and included a meta-analysis. Primary studies included in selected...

Non-small cell lung cancer (NSCLC) is the most common form of lung cancer. Some epidermal growth factor receptor tyrosine kinase (EGFR-TK) mutations make tumours responsive to treatment with EGFR-TK inhibitors (EGFR-TKIs) but less responsive to treatment with standard chemotherapy. Patients with NSCLC are therefore tested for EGFR-TK tumour gene mu...

Background: Ivacaftor (Kalydeco®, Vertex Pharmaceuticals) is the first of a new class of drugs that target the underlying protein defect in cystic fibrosis (CF). It is aimed at patients with the G551D (glycine to aspartate change in nucleotide 1784 in exon 11) mutation; 5.7% of patients with CF in the UK have this mutation. Objectives: To review th...

https://sites.google.com/site/riskofbiastool/home/the-team

Background: Randomized trials assessing BCG vaccine protection against tuberculosis have widely varying results, for reasons that are not well understood. Methods: We examined associations of trial setting and design with BCG efficacy against pulmonary and miliary or meningeal tuberculosis by conducting a systematic review, meta-analyses, and me...

The recognition that surgical interventions are complex has major implications for the design of RCTs, including the need for methods to describe interventions in study protocols to allow delivery and fidelity to be accurately assessed. We report approaches to defining and describing surgical interventions and assessing intervention fidelity. Some...

Cook and colleagues highlight that descriptions of non-pharmacological interventions in randomised controlled trials are inadequate.1 Although reporting standards need to be improved, the level of information they suggest may not always be necessary. Surgical interventions are complex, comprising many components that are delivered with multiple co...

Objective: To classify the sources of bias and variation and to provide an updated summary of the evidence of the effects of each source of bias and variation. Study design and setting: We conducted a systematic review of studies of any design with the main objective of addressing bias or variation in the results of diagnostic accuracy studies....

Background: A systematic and extensive search for as many eligible studies as possible is essential in any systematic review. When searching for diagnostic test accuracy (DTA) studies in bibliographic databases, it is recommended that terms for disease (target condition) are combined with terms for the diagnostic test (index test). Researchers hav...

Background: Diagnostic selective nerve root block (SNRB) involves injection of local anaesthetic, sometimes in conjunction with corticosteroids, around spinal nerves. It is used to identify symptomatic nerve roots in patients with probable radicular pain that is not fully concordant with imaging findings. Objectives: (1) Determine the diagnostic...

Interpreting information on diagnostic accuracy is an area that health professionals struggle with. In this paper, we use the example of Mr Samways, a 45-year-old man with joint symptoms, to illustrate how to apply the results of a diagnostic accuracy study in clinical practice. We consider the various measures used to quantify diagnostic accuracy...

Background: Diagnostic and prognostic algorithms can help reduce clinical uncertainty. The selection of candidate symptoms and signs to be measured in case report forms (CRFs) for potential inclusion in diagnostic algorithms needs to be comprehensive, clearly formulated and relevant for end users. Objective: To investigate whether qualitative me...

Identification of causes of dementia soon after symptom onset is important, because appropriate treatment of some causes of dementia can slow or halt its progression or enable symptomatic treatment where appropriate. The accuracy of MRI and CT, and whether MRI is superior to CT, in detecting a vascular component to dementia in autopsy confirmed and...

Web Appendix: Included study details [26-64].

Objective (1) To examine the evidence for the effectiveness of differences in timing and type of speech and language therapy for children with cleft palate with or without a cleft lip and (2) to identify types of interventions assessed. Design Nine databases, including MEDLINE and EMBASE, were searched between inception and March 2011 to identify...

Reviewing and using diagnostic research for decision making involves complex issues about what are the exact diagnostic questions, how they should be studied, and to whom the results of such studies apply in real life. In this paper we aim to address some of the main issues concerning applicability of diagnostic research by looking at different dia...

Accurate and timely diagnosis of urinary tract infection (UTI) in young children presenting to primary care is important because appropriate treatment may alleviate suffering and help prevent long term sequelae such as renal scarring, poor renal growth, recurrent pyelonephritis, impaired glomerular function, hypertension, end stage renal disease, a...

In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk...

In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk...

Breast reconstruction after mastectomy for cancer requires accurate evaluation to inform evidence-based participatory decision making, but the standards of outcome reporting after breast reconstruction have not previously been considered. We used extensive searches to identify articles reporting surgical outcomes of breast reconstruction. We extrac...

To compare the performance of MEDLINE searches using index test(s) and target condition (subject searches) with the same searches combined with methodological filters for test accuracy studies. We derived a reference set of 506 test accuracy studies indexed on MEDLINE from seven systematic reviews that conducted extensive searches. We compared the...

Background: Early recognition and treatment of rheumatoid arthritis is important to prevent irreversible joint damage. Anti-citrullinated peptide antibodies (ACPA) have been suggested for early diagnosis. Purpose: To compare the accuracy of ACPA and rheumatoid factor in diagnosing rheumatoid arthritis in patients with early symptoms of the disea...

Early recognition and treatment of rheumatoid arthritis is important to prevent irreversible joint damage. Anti-citrullinated peptide antibodies (ACPA) have been suggested for early diagnosis. To compare the accuracy of ACPA and rheumatoid factor in diagnosing rheumatoid arthritis in patients with early symptoms of the disease. 10 medical databases...

To determine the diagnostic performance and cost-effectiveness of colour vision testing (CVT) to identify and monitor the progression of diabetic retinopathy (DR). Major electronic databases including MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature, and Cochrane Database of Systematic Reviews were searched from inception t...

Meta-analysis of diagnostic test accuracy presents many challenges. Even in the simplest case, when the data are summarized by a 2 × 2 table from each study, a statistically rigorous analysis requires hierarchical (multilevel) models that respect the binomial data structure, such as hierarchical logistic regression. We present a Stata package, meta...

Meta-analysis of studies of the accuracy of diagnostic tests currently uses a variety of methods. Statistically rigorous hierarchical models require expertise and sophisticated software. We assessed whether any of the simpler methods can in practice give adequately accurate and reliable results. We reviewed six methods for meta-analysis of diagnost...

Making an accurate diagnosis is essential to ensure that a patient receives appropriate treatment and correct information regarding their prognosis. Characteristics of diagnostic tests are quantified in test accuracy studies, but many such studies have methodological flaws. The HSRC evidence-based diagnosis programme has focused on methods for syst...

To estimate the yield from searching a range of bibliographic databases and additional sources to identify test accuracy studies for systematic reviews. We examined eight systematic reviews and their database searches: MEDLINE, EMBASE, BIOSIS, Science Citation Index, LILACS, Pascal, and CENTRAL. We used studies included in each systematic review as...

Graphical displays of results allow researchers to summarise and communicate the key findings of their study. Diagnostic information should be presented in an easily interpretable way, which conveys both test characteristics (diagnostic accuracy) and the potential for use in clinical practice (predictive value). We discuss the types of graphical di...

To consider the reasons and context for test ordering by doctors when faced with an undiagnosed complaint in primary or secondary care. We reviewed any study of any design that discussed factors that may affect a doctor's decision to order a test. Articles were located through searches of electronic databases, authors' files on diagnostic methodolo...

Studies of diagnostic accuracy require more sophisticated methods for their meta-analysis than studies of therapeutic interventions. A number of different, and apparently divergent, methods for meta-analysis of diagnostic studies have been proposed, including two alternative approaches that are statistically rigorous and allow for between-study var...

To determine the diagnostic accuracy of tests for detecting urinary tract infection (UTI) in children under 5 years of age and to evaluate the effectiveness of tests used to investigate further children with confirmed UTI. Also, to evaluate the effectiveness of following up children with UTI and the cost-effectiveness of diagnostic and imaging test...

To determine the accuracy of magnetic resonance imaging criteria for the early diagnosis of multiple sclerosis in patients with suspected disease. Systematic review. 12 electronic databases, citation searches, and reference lists of included studies. Review methods Studies on accuracy of diagnosis that compared magnetic resonance imaging, or diagno...

Background A quality assessment tool for diagnostic accuracy studies, named QUADAS, has recently been developed. Although QUADAS has been used in several systematic reviews, it has not been formally validated. The objective was to evaluate the validity and usefulness of QUADAS. Methods Three reviewers independently rated the quality of 30 studies...

To review the effectiveness and/or accuracy, cost-effectiveness, and predictive value of neuroimaging of the cerebral cortex to visualise the seizure focus in people with refractory epilepsy being considered for surgery. Electronic databases, Internet searches, hand searching and consultation with experts. A systematic review was undertaken accordi...

The use of systematic literature review to inform evidence based practice in diagnostics is rapidly expanding. Although the primary diagnostic literature is extensive, studies are often of low methodological quality or poorly reported. There has been no rigorously evaluated, evidence based tool to assess the methodological quality of diagnostic stu...

There is a lack of consensus regarding the use of quality scores in diagnostic systematic reviews. The objective of this study was to use different methods of weighting items included in a quality assessment tool for diagnostic accuracy studies (QUADAS) to produce an overall quality score, and to examine the effects of incorporating these into a sy...

Urinary tract infection (UTI) is one of the most common sources of infection in children under five. Prompt diagnosis and treatment is important to reduce the risk of renal scarring. Rapid, cost-effective, methods of UTI diagnosis are required as an alternative to culture. We conducted a systematic review to determine the diagnostic accuracy of rap...

Further investigation of confirmed UTI in children aims to prevent renal scarring and future complications. We conducted a systematic review to determine the most effective approach to the further investigation of confirmed urinary tract infection (UTI) in children under five years of age. 73 studies were included. Many studies had methodological l...