Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography

Primary Care Genetics, General Practice and Primary Care Research Unit, University of Cambridge and Public Health Genetics Unit, Strangeways Research Labs, Worts Causeway, Cambridge, UK.
International Journal of Epidemiology (Impact Factor: 9.2). 07/2007; 36(3):666-76. DOI: 10.1093/ije/dym018
Source: PubMed

ABSTRACT Assessing quality and susceptibility to bias is essential when interpreting primary research and conducting systematic reviews and meta-analyses. Tools for assessing quality in clinical trials are well-described but much less attention has been given to similar tools for observational epidemiological studies.
Tools were identified from a search of three electronic databases, bibliographies and an Internet search using Google. Two reviewers extracted data using a pre-piloted extraction form and strict inclusion criteria. Tool content was evaluated for domains potentially related to bias and was informed by the STROBE guidelines for reporting observational epidemiological studies.
A total of 86 tools were reviewed, comprising 41 simple checklists, 12 checklists with additional summary judgements and 33 scales. The number of items ranged from 3 to 36 (mean 13.7). One-third of tools were designed for single use in a specific review and one-third for critical appraisal. Half of the tools provided development details, although most were proposed for future use in other contexts. Most tools included items for selection methods (92%), measurement of study variables (86%), design-specific sources of bias (86%), control of confounding (78%) and use of statistics (78%); only 4% addressed conflict of interest. The distribution and weighting of domains across tools was variable and inconsistent.
A number of useful assessment tools have been identified by this report. Tools should be rigorously developed, evidence-based, valid, reliable and easy to use. There is a need to agree on critical elements for assessing susceptibility to bias in observational epidemiology and to develop appropriate evaluation tools.

Download full-text


Available from: Iain Tatt, Jul 24, 2014
    • "For our primary and separate meta-analyses, we performed subgroup analyses to assess the following sources of heterogeneity: type of IBD (UC or CD), bias control, type of data (published or unpublished), diagnostic criteria (intestinal culture or feces), age (adult or children). A value of 0% indicates no observed heterogeneity, and larger values show increasing heterogeneity [18]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Increased numbers of Escherichia coli and, furthermore, specific subtypes of E. coli, such as E. coli of the phylogenetic groups B2 and D have been found in the intestine of patients with inflammatory bowel disease (IBD). In this review, we wanted to evaluate the relationship between B2 and D E. coli intestinal colonization and IBD. A systematic review with meta-analyses. We included studies comparing colonization with B2 and D E. coli in IBD patients and in controls. Random-effects and fixed-effect meta-analyses were performed. We included 7 studies on 163 patients with IBD and 89 controls. Among IBD patients, 57 patients had ulcerative colitis (UC) and 95 Crohn's disease (CD). Random-effects meta-analysis showed that IBD patients were more likely to have B2 E. coli intestinal colonization compared with controls (odds ratio [OR]: 2.28; 95% confidence interval [CI]: 1.25-4.16). There was little between-study heterogeneity (I(2) = 0). The result was confirmed in subgroup analyses of patients with UC (OR: 3.58; 95% CI: 1.62-7.90), but not CD (OR: 1.94; 95% CI: 0.98-3.82). Intestinal colonization with phylogenetic group D E. coli was not found to be related to IBD, UC or CD. Our study reveals that intestinal colonization with phylogenetic group B2 E. coli is associated with UC. Due to the design, we are unable to determine if the colonization with B2 E. coli leads to the development of the disease or the disease increases the risk of colonization with B2 E. coli.
    Scandinavian Journal of Gastroenterology 04/2015; DOI:10.3109/00365521.2015.1028993 · 2.33 Impact Factor
  • Source
    • "Secondly, a well-designed study may be categorized as low quality because the authors failed to provide detail information in the publication. Finally, some items of the NOS such as representativeness of study cohort with respect to community and duration of follow-up do not belong to the risk of bias tools (Deeks et al. 2003; NAS, 2004; Sanderson et al. 2007). Thus, the interpretation of how well a study does on the NOS in our study should be done with caution. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Exposure to arsenic is one of the major global health problems affecting over 300 million people worldwide, but its effects on human reproduction are uncertain. We conducted a systematic review and meta-analysis to examine the association between arsenic and adverse pregnancy outcomes/infant mortality. We searched PubMed, Ovid MEDLINE (from 1946 to July 2013) and EMBASE (from 1988 to July 2013) databases; and the reference lists of reviews and relevant articles. Studies satisfying our a priori eligibility criteria were evaluated independently by two authors. Our systematic search yielded 888 articles from which 23 were included in the systematic review. Sixteen provided sufficient data for our quantitative analysis. Arsenic in ground water (≥ 50 µg/l) was associated with increased risk of spontaneous abortion (six studies: OR 1.98; 95% CI: 1.27, 3.10), stillbirth (nine studies: OR 1.77; 95% CI: 1.32, 2.36), moderate risk of neonatal mortality (five studies: OR 1.51; 95% CI: 1.28, 1.78) and infant mortality (seven studies: OR 1.35; 95% CI: 1.12, 1.62). Exposure to environmental arsenic was associated with a significant reduction in birth weight (four studies: β = -53.2 grams; 95% CI: -94.9, -11.4). There was paucity of evidence for low-to-moderate arsenic dose. Conclusions: Arsenic is associated with adverse pregnancy outcomes and infant mortality. The interpretation of the causal association is hampered by methodological challenges and limited studies on dose-response. Exposure to arsenic continues to be a major global health issue and we therefore advocate for high quality prospective studies that include individual level data to quantify the impact of arsenic on adverse pregnancy outcomes/infant mortality.
    Environmental Health Perspectives 01/2015; DOI:10.1289/ehp.1307894 · 7.98 Impact Factor
  • Source
    • "To ensure consistency in data extraction (Glasziou, Irwig, Bain, & Colditz, 2001), a data extraction template and a codebook were developed based on the STROBE (von Elm et al., 2008) and covered citation details, source of citation (e.g. CINAHL), study objectives, methods (selection of subjects, assessment, confounders and statistical analyses), results, conflict of interests and bias (Sanderson et al., 2007; von Elm et al., 2008). The template also included space for reviewers to make preliminary assessments of the information quality provided in the study (well covered, poor, adequate, not addressed, not reported or not applicable). "
Show more