John Ioannidis

Stanford University | SU · School of Medicine

Background: Screening for major depression with the Patient Health Questionnaire-9 (PHQ-9) can be done using a cutoff or the PHQ-9 diagnostic algorithm. Many primary studies publish results for only one approach, and previous meta-analyses of the algorithm approach included only a subset of primary studies that collected data and could have publish...

We thank Cohen for her letter. There is some inconsistency in the used terminology for risk factors. Investigators with different interests use “environmental” with different connotations or different specificity thresholds. Our umbrella review assessed any non‐genetic risk factors, in the broadest possible fashion. This included lifestyle, biobeha...

In 2014, one of us (JPAI) wrote a viewpoint article coining the term “stealth research” for touted biomedical innovation happening outside the peer‐reviewed literature in a confusing mix of “possibly brilliant ideas, aggressive corporate announcements, and mass media hype”. These reflections were prompted by Theranos, a medical diagnosis start‐up c...

In Reply The calculation of 12 years of life gained with the consumption of 12 hazelnuts daily (and similar calculations with other foods) was an argumentum ad absurdum.¹ The estimation combined several prevalent assumptions and common practices to demonstrate how absurd inferences can arise. These assumptions are that nutritional epidemiology–deri...

Guideline recommendations and health policy decisions rely on evidence from clinical and epidemiological studies. Adequate methodology and appropriate conclusions are essential to support healthcare and health policy decisions. An analysis of body mass index and mortality by the Global BMI Mortality Collaboration (GBMC) concluded that the associati...

The E-value was recently introduced on the basis of earlier work as “the minimum strength of association . . . that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment–outcome association, conditional on the measured covariates.” E-values have been proposed for wide applicat...

Despite modest effects from initiatives such as the Choosing Wisely campaign, unnecessary diagnostic imaging remains a substantial problem in the United States.¹⁻³ Significant between-country differences probably reflect largely wasted overuse. The United States occupies top usage ranks, with population rates of annual computed tomography (CT) scan...

Machine learning (ML), artificial intelligence (AI) and other modern statistical methods are providing new opportunities to operationalize previously untapped and rapidly growing sources of data for patient benefit. Whilst there is a lot of promising research currently being undertaken, the literature as a whole lacks: transparency; clear reporting...

Introduction The 30-item Geriatric Depression Scale (GDS-30) and the shorter GDS-15, GDS-5 and GDS-4 are recommended as depression screening tools for elderly individuals. Existing meta-analyses on the diagnostic accuracy of the GDS have not been able to conduct subgroup analyses, have included patients already identified as depressed who would not...

With expanding knowledge in tumor biology and biomarkers, oncology therapies are increasingly moving away from the “one-size-fits-all” rationale onto biomarker-driven therapies tailored according to patient-specific characteristics, most commonly the tumor's molecular profile. The advent of precision medicine in oncology has been accompanied by the...

Ever since 1971, when the U.S. President signed the “War on Cancer” [1] National Cancer Act, screening has been a hallmark in cancer control. The fundamental idea was that more cancers would be cured if they were detected and treated before symptoms arise. During the following decades, astronomic amounts of money and great hopes were invested to im...

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000–2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedi...

The original version of this article unfortunately contained a mistake. The affiliation “Department of Clinical Psychology and Psychotherapy, Babes-Bolyai University, Cluj-Napoca, Romania” was wrongly affiliated to John P. A. Ioannidis. The correct affiliations for John P. A. Ioannidis are 1, 3, 4, 5, 6. In addition the name of Ioana-Alinea Cristea...

Objective To compare the effect of exercise regimens and medications on systolic blood pressure (SBP). Data sources Medline (via PubMed) and the Cochrane Library. Eligibility criteria Randomised controlled trials (RCTs) of angiotensin-converting enzyme inhibitors (ACE-I), angiotensin-2 receptor blockers (ARBs), β-blockers, calcium channel blocker...

In Reply We agree with Dr Obstfeld and colleagues that harmonizing measurements across laboratory test platforms is critical to the success of personalized laboratory medicine. The efforts cited by the authors will contribute to reducing technical variation and measurement error across test platforms. However, the fact that large-scale data sets of...

In Reply We thank Weissman for bringing further discussion points in reference to our Viewpoint on improving disclosure of financial conflict of interests (COIs) in research on psychosocial interventions.¹ We regret the inadvertent ambiguity, but we had not suggested that the measurement of interpersonal difficulties was developed or marketed by We...

Registered reports present a substantial departure from traditional publishing models with the goal of enhancing the transparency and credibility of the scientific literature. We map the evolving universe of registered reports to assess their growth, implementation and shortcomings at journals across scientific disciplines.

(Reprinted with permission from Lancet 2018; 391:1357-66).

Advocates of precision medicine propose that individual patients require evidence tailored to their singular, unique, personal profile. According to this way of thinking, population‐based research, randomized clinical trials (RCTs) and evidence‐based medicine (EBM) are often seen as outdated tools for application of research evidence to individuals...

1 Background The objective of this study was to develop and validate a short form of the Patient Health Questionnaire‐9 (PHQ‐9), a self‐report questionnaire for assessing depressive symptomatology, using objective criteria. 2 Methods Responses on the PHQ‐9 were obtained from 7,850 English‐speaking participants enrolled in 20 primary diagnostic tes...

Background: The available evidence on the benefits and harms of novel drugs and therapeutic biologics at the time of approval is reported in publicly available documents provided by the US Food and Drug Administration (FDA). We aimed to create a comprehensive database providing the relevant information required to systematically analyze and assess...

Background To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria—quality, accessib...

To highlight uncertain norms in authorship, John P. A. Ioannidis, Richard Klavans and Kevin W. Boyack identified the most prolific scientists of recent years.

The vast majority of scientific articles published to-date have not been accompanied by concomitant publication of the underlying research data upon which they are based. This state of affairs precludes the routine re-use and re-analysis of research data, undermining the efficiency of the scientific enterprise, and compromising the credibility of c...

PurposeTo study the landscape of funding in intensive care research and assess whether the reported outcomes of industry-funded randomized controlled trials (RCTs) are more favorable. Methods We systematically assembled meta-analyses evaluating any type of intervention in the critical care setting and reporting the source of funding for each includ...

Pragmatic randomized clinical trials (pRCTs) are considered a highly valuable design, providing evidence for clinical decisions in real-world settings.¹ The conduct and setting of pRCTs mimic the usual practice of care while trying to maintain the internal validity of randomization,² and they are becoming increasingly common.³ Their perceived and d...

Abstract Background Depression is a prevalent and disabling mental disorder that frequently co-occurs with a wide range of chronic conditions. Evidence has suggested that depression could be associated with excess all-cause mortality across different settings and populations, although the causality of these associations remains unclear. Methods We...

In Reply The letters by Dr Ludwig and colleagues and Drs Bero and Grundy offer an opportunity to discuss potential trade-offs and clarify potential misunderstandings regarding our proposal.

A large majority of human nutrition research uses nonrandomized observational designs, but this has led to little reliable progress. This is mostly due to many epistemologic problems, the most important of which are as follows: difficulty detecting small (or even tiny) effect sizes reliably for nutritional risk factors and nutrition-related interve...

Background Multiple environmental factors have been implicated in obesity and multiple interventions, besides drugs and surgery, have been assessed in obese patients. Results are scattered across many studies and meta‐analyses and they often mix obese and overweight individuals. Materials and methods PubMed and Cochrane Database of Systematic Revi...

Objective To provide an overview of the evidence on prevalence and outcomes of incidental imaging findings. Design Umbrella review of systematic reviews. Data sources Searches of MEDLINE, EMBASE up to August 2017; screening of references in included papers. Eligibility criteria Criteria included systematic reviews and meta-analyses of observatio...

The development of depression may involve a complex interplay of environmental and genetic risk factors. PubMed and PsycInfo databases were searched from inception through August 3, 2017, to identify meta-analyses and Mendelian randomization (MR) studies of environmental risk factors associated with depression. For each eligible meta-analysis, we e...

Preventing psychosis in patients at clinical high risk may be a promising avenue for pre-emptively ameliorating outcomes of the most severe psychiatric disorder. However, information on how each preventive intervention fares against other currently available treatment options remains unavailable. The aim of the current study was to quantify the con...

Box plots with standard errors for the total significant P values by Journal-Year cohorts. Four outliers (> 30 P values), all from 2017, are not displayed. Median represented by a line on each bar. (TIF)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in Science 2017. (TIF)

Robustness analysis. (DOCX)

Descriptive statistics for the total number of countable P values across display items for each Journal-Year unit. (DOCX)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in Nature 1997. (TIF)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in PNAS 2017. (TIF)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in PNAS 1997. (TIF)

Robustness analysis. Proportion of significant P values and 95% confidence intervals by Journal-Year cohorts excluding 15 display items (all in 2017) with uncertainty about counting. (TIF)

Descriptive statistics for the total number of significant P values across display items for each Journal-Year unit. (DOCX)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in Science 1997. (TIF)

Proportion of significant P values and 95% confidence intervals across all display items with countable P values in Nature 2017. (TIF)

Extracted data for all screened articles. (XLSX)

In the context of a recent proposal to exclude research from consideration at the Environmental Protection Agency, John Ioannidis points out that "perceived perfection is not a characteristic of science, but of dogma" and envisions how governments can promote a standard of openness in science.

The definition of “normal” values for common laboratory tests often governs the diagnosis, treatment, and overall management of tested individuals. Some test results may depend on demographic traits of the tested population including age, race, and sex. Ideally, laboratory test results should be interpreted in reference to a population of “similar”...

Numerous functional magnetic resonance imaging (fMRI) studies have reported sex differences. To empirically evaluate for evidence of excessive significance bias in this literature, we searched for published fMRI studies of human brain to evaluate sex differences, regardless of the topic investigated, in Medline and Scopus over 10 years. We analyzed...

Introduction: Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and...

Assessment of researchers is necessary for decisions of hiring, promotion, and tenure. A burgeoning number of scientific leaders believe the current system of faculty incentives and rewards is misaligned with the needs of society and disconnected from the evidence about the causes of the reproducibility crisis and suboptimal quality of the scientif...

Background: Different diagnostic interviews are used as reference standards for major depression classification in research. Semi-structured interviews involve clinical judgement, whereas fully structured interviews are completely scripted. The Mini International Neuropsychiatric Interview (MINI), a brief fully structured interview, is also someti...

Importance Dietary modification remains key to successful weight loss. Yet, no one dietary strategy is consistently superior to others for the general population. Previous research suggests genotype or insulin-glucose dynamics may modify the effects of diets. Objective To determine the effect of a healthy low-fat (HLF) diet vs a healthy low-carboh...

Genomic and other related big data (Big Genomic Data, BGD for short) are ushering a new era of precision medicine. This overview discusses whether principles of evidence-based medicine (EBM) hold true for BGD and how they should be operationalized in the current era. Major EBM principles include the systematic identification, description and analys...

Objectives To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes. Design Survey of published RCTs. Setting PubMed/Medline. Eligibility criteria RCTs that had...

Supplementary information: e-Table 1: results of reanalyses for three ineligible studies published in The BMJ after the policy but submitted before

Objective: Better identification of at-risk groups could benefit HIV-1 care programmes. We systematically identified HIV-1 risk factors in two nationally representative cohorts of women in the Demographic and Health Surveys. Methods: We identified and replicated the association of 1415 social, economic, environmental, and behavioral factors with...

Objectives: Evaluate how often the European Medicines Agency (EMA) has authorized drugs based on non-randomized studies and whether there is an association between treatment effects and EMA preference for further testing in RCTs. Study design and setting: We reviewed all initial marketing authorizations in the EMA database on human medicines bet...

Background: Over the past 20 years, advances in genomic technology have enabled unparalleled access to the information contained within the human genome. However, the multiple genetic variants associated with various diseases typically account for only a small fraction of the disease risk. This may be due to the multifactorial nature of disease me...

Background: The multicomponent meningococcal serogroup B vaccine (4CMenB) has been licensed in more than 35 countries. However, uncertainties remain about the lowest number of doses required to induce satisfactory, persistent immune responses. We did a systematic review and meta-analysis to provide quantitative estimates for the immunogenicity, pe...

We aimed to assess which factors correlate with collaborative behavior and whether such behavior associates with scientific impact (citations and becoming a principal investigator). We used the R index which is defined for each author as log(Np)/log(I1), where I1 is the number of co-authors who appear in at least I1 papers written by that author an...

Nutrition research is among the most contentious fields of science. Although the totality of an individual’s diet has important effects on health, most nutrients and foods individually have ambiguously tiny (or nonexistent) effects.¹ Substantial reliance on observational data for which causal inference is notoriously difficult also limits the clari...

Objective To provide an overview of the breadth and validity of claimed associations between physical activity and risk of developing or dying from cancer. Design Umbrella review. Data sources We searched Medline, Embase, Cochrane Database and Web of Science. Eligibility criteria for selecting studies Systematic reviews about physical activity a...

Background: Off-label drug use is highly prevalent, but controversial and often discouraged assuming generally inferior medical effects associated with off-label use. Methods: We searched Medline, PubMed Health and the Cochrane Library up to May 2015 for systematic reviews including meta-analyses of randomized clinical trials (RCTs) comparing of...

Objective: To evaluate limitations of common statistical modeling approaches in deriving clinical prediction models and explore alternative strategies. Study design and setting: A previously published model predicted the likelihood of having a mutation at the time of diagnosis of colorectal cancer. This model was based on a cohort where 38 mutat...

Objective: Evaluate comparative harm-rates from medical interventions in pediatric randomized-trials (RCTs) from more-developed (MDCs) and less-developed countries (LDCs). Study design: Meta-epidemiologic empirical evaluation of Cochrane Database of Systematic Reviews (June 2014) meta-analyses reporting clinically-important harm-outcomes (severe...

Most human diseases have underlying genetic causes. To better understand the impact of genes on disease and its implications for medicine and public health, researchers have pursued methods for determining the sequences of individual genes, then all genes, and now complete human genomes. Massively parallel high-throughput sequencing technology, whe...

We investigate two critical dimensions of the credibility of empirical economics research: statistical power and bias. We survey 159 empirical economics literatures that draw upon 64,076 estimates of economic parameters reported in more than 6,700 empirical studies. Half of the research areas have nearly 90% of their results under-powered. The medi...

Background: Confounding bias is a most pervasive threat to validity of observational epidemiological research. Methods: We assessed whether authors of observational epidemiological studies consider confounding bias when interpreting the findings. We randomly selected 120 cohort or case-control studies published in 2011 and 2012 by the general me...

PurposeWe tested whether providing a genetic risk score (GRS) for coronary artery disease (CAD) would serve as a motivator to improve adherence to risk-reducing strategies.Methods We randomized 94 participants with at least moderate risk of CAD to receive standard-of-care with (N = 49) or without (N = 45) their GRS at a subsequent 3-month follow-up...

Individual patient data (IPD) meta-analyses are increasingly common in the literature. In the context of estimating the diagnostic accuracy of ordinal or semi-continuous scale tests, sensitivity and specificity are often reported for a given threshold or a small set of thresholds, and a meta-analysis is conducted via a bivariate approach to account...

Context: Therapeutic agents treating serious conditions are eligible for Food and Drug Administration (FDA) accelerated approval. The clinical evidence accrued on agents receiving accelerated approval has not been systematically evaluated. Our objective was to assess the timing and characteristics of available studies. Methods: We first identified...

Context: Therapeutic agents treating serious conditions are eligible for Food and Drug Administration (FDA) accelerated approval. The clinical evidence accrued on agents receiving accelerated approval has not been systematically evaluated. Our objective was to assess the timing and characteristics of available studies. Methods: We first identifi...

Individual patient data (IPD) meta-analyses are increasingly common in the literature. In the context of estimating the diagnostic accuracy of ordinal or semi-continuous scale tests, sensitivity and specificity are often reported for a given threshold or a small set of thresholds, and a meta-analysis is conducted via a bivariate approach to account...

In studies of diagnostic test accuracy, authors sometimes report results only for a range of cutoff points around data-driven "optimal" cutoffs. We assessed selective cutoff reporting in studies of the diagnostic accuracy of the Patient Health Questionnaire-9 (PHQ-9) depression screening tool. We compared conventional meta-analysis of published res...

Background: The large, expanding literature on biomarkers is characterized by almost ubiquitous significant results, with claims about the potential importance, but few of these discovered biomarkers are used in routine clinical care. Content: The pipeline of biomarker development includes several specific stages: discovery, validation, clinical...

Background Inclusion of vaccine herd-protection effects in cost-effectiveness analyses (CEAs) can impact the CEAs-conclusions. However, empirical epidemiologic data on the size of herd-protection effects from original studies are limited. Methods We performed a quantitative comparative analysis of the impact of herd-protection effects in CEAs for...

Objectives: To measure exclusion of elderly adults from randomized trials studying drug interventions for ischemic heart disease (IHD) and describe the characteristics of these trials. Design: Cross-sectional analysis. Setting: Interventional clinical trials studying a drug intervention for IHD that started in 2006 and after were identified in...

Science is not done in a vacuum - across fields of biomedicine, scientists have built on previous research and used data published in previous papers. A mainstay of scientific inquiry is the publication of one's research and recognition for this work is given in the form of citations and notoriety - ideally given in proportion to the quality of the...

Numerous studies have attempted to identify successful dietary strategies for weight loss, and many have focused on Low-Fat vs. Low-Carbohydrate comparisons. Despite relatively small between-group differences in weight loss found in most previous studies, researchers have consistently observed relatively large between-subject differences in weight...

The existing literature on the relationship between diet, body size, physical activity and prostate cancer risk was summarised by the World Cancer Research Fund Continuous Update Project (CUP). An evaluation of the robustness of this evidence is required to help inform public health policy. The robustness of this evidence was evaluated using severa...

Objective To evaluate the frequency, validity, and relevance of statistically significant (P<0.05) sex-treatment interactions in randomized controlled trials in Cochrane meta-analyses. Design Meta-epidemiological study. Data sources Cochrane Database of Systematic Reviews (CDSR) and PubMed. Eligibility criteria for study selection Reviews published...

Supplementary file: study protocol

Supplementary file: additional information about four statistically significant sex-treatment interactions

Supplementary file: characteristics of the reviews

Objective: To determine whether drugs used to treat diverse conditions cause insomnia symptoms and whether their prescription information is concordant with this evidence. Methods: We conducted a survey of meta-analyses (Cochrane Database of Systematic Reviews) and comparisons with package inserts compiled in the Physicians' Desk Reference (PDR)...

Appendix: Online supplemental data

Citation metrics are increasingly used to appraise published research. One challenge is whether and how to normalize these metrics to account for differences across scientific fields, age (year of publication), type of document, database coverage, and other factors. We discuss the pros and cons for normalizations using different approaches. Additio...