Calibration of credibility of agnostic genome-wide associations

Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece.
American Journal of Medical Genetics Part B Neuropsychiatric Genetics (Impact Factor: 3.27). 09/2008; 147B(6):964-72. DOI: 10.1002/ajmg.b.30721
Source: PubMed

ABSTRACT Genome-wide testing platforms are increasingly used to promote "agnostic" approaches to the discovery of gene variants associated with the risk of many common diseases and quantitative traits. The early track record of genome-wide association (GWA) studies suggests that some proposed associations are replicated quite consistently with large-scale subsequent evidence from multiple studies, others have a more inconsistent replication record, some have failed to be replicated by independent investigators and many more early proposed associations await further replication. An important question is how to calibrate the credibility of these postulated associations. A simple Bayesian method is applied here to achieve such calibration. The variability of the estimated credibility is examined under different assumptions. Empirical examples are drawn from existing GWA studies. It is demonstrated that the credibility of different proposed associations can cover a very wide range. The credibility of specific associations usually remains relatively robust when different plausible assumptions are made (within a reasonable range) for the prior odds of an association being true, or the magnitude of the anticipated effect size for genetic associations. Heterogeneity and bias assumptions can have a more major impact on the credibility estimates and thus they need very careful consideration in each case. Credibility calibration may be used in conjunction with qualitative criteria for the appraisal of the cumulative evidence that take into consideration the amount, consistency, and protection from bias in the data.

  • [Show abstract] [Hide abstract]
    ABSTRACT: Abstract In genetic association studies (GAS) as well as in genome-wide association studies (GWAS), the mode of inheritance (dominant, additive and recessive) is usually not known a priori. Assuming an incorrect mode of inheritance may lead to substantial loss of power, whereas on the other hand, testing all possible models may result in an increased type I error rate. The situation is even more complicated in the meta-analysis of GAS or GWAS, in which individual studies are synthesized to derive an overall estimate. Meta-analysis increases the power to detect weak genotype effects, but heterogeneity and incompatibility between the included studies complicate things further. In this review, we present a comprehensive summary of the statistical methods used for robust analysis and genetic model selection in GAS and GWAS. We then discuss the application of such methods in the context of meta-analysis. We describe the theoretical properties of the various methods and the foundations on which they are based. We also present the available software implementations of the described methods. Finally, since only few of the available robust methods have been applied in the meta-analysis setting, we present some simple extensions that allow robust meta-analysis of GAS and GWAS. Possible extensions and proposals for future work are also discussed.
    Statistical Applications in Genetics and Molecular Biology 04/2013; 12(3):1-24. DOI:10.1515/sagmb-2012-0016 · 1.52 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Distinguishing true from false positive findings is a major challenge in human genetic epidemiology. Several strategies have been devised to facilitate this, including the positive predictive value (PPV) and a set of epidemiological criteria, known as the "Venice" criteria. The PPV measures the probability of a true association, given a statistically significant finding, while the Venice criteria grade the credibility based on the amount of evidence, consistency of replication and protection from bias. A vast majority of journals use significance thresholds to identify the true positive findings. We studied the effect of p value thresholds on the PPV and used the PPV and Venice criteria to define usable thresholds of statistical significance. Theoretical and empirical analyses of data published on AlzGene show that at a nominal p value threshold of 0.05 most "positive" findings will turn out to be false if the prior probability of association is below 0.10 even if the statistical power of the study is higher than 0.80. However, in underpowered studies (0.25) with a low prior probability of 1 × 10(-3), a p value of 1 × 10(-5) yields a high PPV (>96 %). Here we have shown that the p value threshold of 1 × 10(-5) gives a very strong evidence of association in almost all studies. However, in the case of a very high prior probability of association (0.50) a p value threshold of 0.05 may be sufficient, while for studies with very low prior probability of association (1 × 10(-4); genome-wide association studies for instance) 1 × 10(-7) may serve as a useful threshold to declare significance.
    European Journal of Epidemiology 02/2013; 28(2). DOI:10.1007/s10654-012-9755-x · 5.15 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: It is known that statistically significant (positive) results are more likely to be published than non-significant (negative) results. However, it has been unclear whether any increasing prevalence of positive results is stronger in the "softer" disciplines (social sciences) than in the "harder" disciplines (physical sciences), and whether the prevalence of negative results is decreasing over time. Using Scopus, we searched the abstracts of papers published between 1990 and 2013, and measured longitudinal trends of multiple expressions of positive versus negative results, including p-values between 0.041 and 0.049 versus p-values between 0.051 and 0.059, textual reporting of "significant difference" versus "no significant difference," and the reporting of p < 0.05 versus p > 0.05. We found no support for a "hierarchy of sciences" with physical sciences at the top and social sciences at the bottom. However, we found large differences in reporting practices between disciplines, with p-values between 0.041 and 0.049 over 1990-2013 being 65.7 times more prevalent in the biological sciences than in the physical sciences. The p-values near the significance threshold of 0.05 on either side have both increased but with those p-values between 0.041 and 0.049 having increased to a greater extent (2013-to-1990 ratio of the percentage of papers = 10.3) than those between 0.051 and 0.059 (ratio = 3.6). Contradictorily, p < 0.05 has increased more slowly than p > 0.05 (ratios = 1.4 and 4.8, respectively), while the use of "significant difference" has shown only a modest increase compared to "no significant difference" (ratios = 1.5 and 1.1, respectively). We also compared reporting of significance in the United States, Asia, and Europe and found that the results are too inconsistent to draw conclusions on cross-cultural differences in significance reporting. We argue that the observed longitudinal trends are caused by negative factors, such as an increase of questionable research practices, but also by positive factors, such as an increase of quantitative research and structured reporting.
    01/2015; 3:e733. DOI:10.7717/peerj.733