Equivalence of ELISpot Assays Demonstrated between Major HIV Network Laboratories

University of Toronto, Canada
PLoS ONE (Impact Factor: 3.23). 12/2010; 5(12):e14330. DOI: 10.1371/journal.pone.0014330
Source: PubMed


The Comprehensive T Cell Vaccine Immune Monitoring Consortium (CTC-VIMC) was created to provide standardized immunogenicity monitoring services for HIV vaccine trials. The ex vivo interferon-gamma (IFN-γ) ELISpot is used extensively as a primary immunogenicity assay to assess T cell-based vaccine candidates in trials for infectious diseases and cancer. Two independent, GCLP-accredited central laboratories of CTC-VIMC routinely use their own standard operating procedures (SOPs) for ELISpot within two major networks of HIV vaccine trials. Studies are imperatively needed to assess the comparability of ELISpot measurements across laboratories to benefit optimal advancement of vaccine candidates.
We describe an equivalence study of the two independently qualified IFN-g ELISpot SOPs. The study design, data collection and subsequent analysis were managed by independent statisticians to avoid subjectivity. The equivalence of both response rates and positivity calls to a given stimulus was assessed based on pre-specified acceptance criteria derived from a separate pilot study.
Detection of positive responses was found to be equivalent between both laboratories. The 95% C.I. on the difference in response rates, for CMV (-1.5%, 1.5%) and CEF (-0.4%, 7.8%) responses, were both contained in the pre-specified equivalence margin of interval [-15%, 15%]. The lower bound of the 95% C.I. on the proportion of concordant positivity calls for CMV (97.2%) and CEF (89.5%) were both greater than the pre-specified margin of 70%. A third CTC-VIMC central laboratory already using one of the two SOPs also showed comparability when tested in a smaller sub-study.
The described study procedure provides a prototypical example for the comparison of bioanalytical methods in HIV vaccine and other disease fields. This study also provides valuable and unprecedented information for future vaccine candidate evaluations on the comparison and pooling of ELISpot results generated by the CTC-VIMC central core laboratories.

Download full-text


Available from: Aloysious Ssemaganda,
  • Source
    • "In the present study, in addition to IFNγ ELISpot and ICS assays, vaccine recipients immunized with recombinant Ad35-GRIN and Ad35-Env were assessed for their ability to inhibit a panel of HIV viruses in vitro, using the VIA [16]. The magnitude, breadth and specificity of insert specific T-cell responses were initially assessed using peptide pools corresponding to the insert-matched Gag, RT, Int, Nef and Env antigens, using a validated IFNγ ELISpot assay [25], [26]. Peptide matrix pools were subsequently designed and the ELISpot assay further qualified to allow the deconvolution of individual peptides within the responding antigen pools. "
    [Show abstract] [Hide abstract]
    ABSTRACT: A correlation between in vivo and in vitro virus control mediated by CD8+ T-cell populations has been demonstrated by CD8 T-cell-mediated inhibition of HIV-1 and SIV replication in vitro in peripheral blood mononuclear cells (PBMCs) from infected humans and non-human primates (NHPs), respectively. Here, the breadth and specificity of T-cell responses induced following vaccination with replication-defective adenovirus serotype 35 (Ad35) vectors containing a fusion protein of Gag, reverse transcriptase (RT), Integrase (Int) and Nef (Ad35-GRIN) and Env (Ad35-ENV), derived from HIV-1 subtype A isolates, was assessed in 25 individuals. The vaccine induced responses to a median of 4 epitopes per vaccinee. We correlated the CD8 responses to conserved vs. variable regions with the ability to inhibit a panel of 7 HIV-1 isolates representing multiple clades in a virus inhibition assay (VIA). The results indicate that targeting immunodominant responses to highly conserved regions of the HIV-1 proteome may result in an increased ability to inhibit multiple clades of HIV-1 in vitro. The data further validate the use of the VIA to screen and select future HIV vaccine candidates. Moreover, our data suggest that future T cell-focused vaccine design should aim to induce immunodominant responses to highly conserved regions of the virus.
    PLoS ONE 03/2014; 9(3):e90378. DOI:10.1371/journal.pone.0090378 · 3.23 Impact Factor
  • Source
    • "There have been earlier attempts at evaluating ELISpot proficiency panels and assay equivalence (Cox et al., 2005; Boaz et al., 2009; Gill et al., 2010), which focused primarily on positive or negative responses and concordance among labs. The EQAPOL ELISpot program does not have criteria for defining positive or negative responses, rather the goal is to assess how accurate to the consensus average and precise laboratories are using their own in-house assay given standard peptides and samples. "
    [Show abstract] [Hide abstract]
    ABSTRACT: In September 2011 Duke University was awarded a contract to develop the National Institutes of Health/National Institute of Allergy and Infectious Diseases (NIH/NIAID) External Quality Assurance Program Oversight Laboratory (EQAPOL). Through EQAPOL, proficiency testing programs are administered for Interferon-γ (IFN-γ) Enzyme-linked immunosorbent spot (ELISpot), Intracellular Cytokine Staining Flow Cytometry (ICS) and Luminex-based cytokine assays. One of the charges of the EQAPOL program was to apply statistical methods to determine overall site performance. We utilized various statistical methods for each program to find the most appropriate for assessing laboratory performance using the consensus average as the target value. Accuracy ranges were calculated based on Wald-type confidence intervals, exact Poisson confidence intervals, or via simulations. Given the nature of proficiency testing data, which has repeated measures within donor/sample made across several laboratories; the use of mixed effects models with alpha adjustments for multiple comparisons was also explored. Mixed effects models were found to be the most useful method to assess laboratory performance with respect to accuracy to the consensus. Model based approaches to the proficiency testing data in EQAPOL will continue to be utilized. Mixed effects models also provided a means of performing more complex analyses that would address secondary research questions regarding within and between laboratory variability as well as longitudinal analyses.
    Journal of immunological methods 01/2014; 409. DOI:10.1016/j.jim.2014.01.007 · 1.82 Impact Factor
  • Source
    • "Additionally, a threshold value (e.g. at least 11 SFU/106 PBMC in the experimental well) is also sometimes applied to provide a threshold of responsiveness that is considered to have biological significance. Similarly, an upper limit on the number of spots in the negative control well may be imposed, e.g. 10 in the case of T-SPOT and IAVI (International AIDS Vaccine Initiative)(Gill et al., 2010). These cut-offs and thresholds are often defined with reference to ELISpot responses in a known negative population and are therefore often referred to as empirical methods (Moodie et al., 2006). "
    [Show abstract] [Hide abstract]
    ABSTRACT: In the absence of replication of wells, empirical criteria for enzyme-linked immunospot (ELISpot) positivity use fixed differences or ratios between spot forming units (SFU) counts between test and control. We propose an alternative approach which first identifies the optimally variance-stabilizing transformation of the SFU counts, based on the Bland-Altman plot of the test and control wells. The second step is to derive a positivity threshold from the difference in between-plate distribution functions of the transformed test and control SFU counts. This method is illustrated using 1309 assay results from a cohort study of influenza in Vietnam in which some, but not all, of the peptide pools have clear tendencies for SFU counts to be higher in test than control wells.
    Journal of immunological methods 03/2013; 392(1-2). DOI:10.1016/j.jim.2013.02.014 · 1.82 Impact Factor
Show more