Joint Hypothesis Testing and Gatekeeping Procedures for Studies with Multiple Endpoints

Department of Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, USA.
Anesthesia and analgesia (Impact Factor: 3.42). 05/2012; 114(6):1304-17. DOI: 10.1213/ANE.0b013e3182504435
Source: PubMed

ABSTRACT A claim of superiority of one intervention over another often depends naturally on results from several outcomes of interest. For such studies the common practice of making conclusions about individual outcomes in isolation can be problematic. For example, an intervention might be shown to improve one outcome (e.g., pain score) but worsen another (e.g., opioid consumption), making interpretation difficult. We thus advocate joint hypothesis testing, in which the decision rule used to claim success of an intervention over its comparator with regard to the multiple outcomes are specified a priori, and the overall type I error is protected. Success might be claimed only if there is a significant improvement detected in all primary outcomes, or alternatively, in at least one of them. We focus more specifically on demonstrating superiority on at least one outcome and noninferiority (i.e., not worse) on the rest. We also advocate the more general "gatekeeping" procedures (both serial and parallel), in which primary and secondary hypotheses of interest are a priori organized into ordered sets, and testing does not proceed to the next set, i.e., through the "gate," unless the significance criteria for the previous sets are satisfied, thus protecting the overall type I error. We demonstrate methods using data from a randomized controlled trial assessing the effects of transdermal nicotine on pain and opioids after pelvic gynecological surgery. Joint hypothesis testing and gatekeeping procedures are shown to substantially improve the efficiency and interpretation of randomized and nonrandomized studies having multiple outcomes of interest.

1 Follower
  • [Show abstract] [Hide abstract]
    ABSTRACT: Background The contribution of regional anesthesia with thoracic paravertebral blockade to postoperative analgesia remains unclear. We compared the effect of a combination of paravertebral blockade and propofol general anesthesia (GA) with sevoflurane GA and opioid analgesia on postoperative pain and opioid use for patients undergoing breast cancer surgery. Methods Patients having breast cancer surgery were randomly assigned to paravertebral analgesia with propofol GA (PPA, n = 187) or sevoflurane GA with perioperative opioid analgesia (SOA, n = 199). The PPA and SOA groups were compared for opioid consumption and pain outcomes (on a 0-10 visual analogue scale [VAS]) at two hours postoperatively using superiority and inferiority statistics. We compared our results with previous publications in a meta-analysis. Results Compared with the SOA group, the PPA group experienced reduced median [interquartile range] pain VAS scores (1 [1,3] vs 2.5 [1,4], respectively; median difference −1.0; 99% confidence intervals [CI]: −1.5 to −0.5) and required less intraoperative fentanyl (50 [0, 125] µg vs 200 [100, 300] µg, respectively; median difference −100; 99% CI: −150 to −100) and less long-acting opioid (0 [0, 0] mg vs 3.0 [0, 12] mg, respectively, morphine equivalents; median difference −3; 99% CI: −4 to −2). Thus, non-inferiority was detected for all the above outcomes, and superiority tests for each outcome were highly significant in the expected directions (P vs SOA. Conclusion Our results were largely consistent with previous much smaller studies. Compared with sevoflurane GA with opioid analgesia, the combination of paravertebral analgesia with propofol GA provides an early clinical analgesic benefit in females having breast cancer surgery. This analysis is a substudy of an ongoing multicentre double-blinded randomized trial (, NCT00418457) of cancer recurrence.
    Canadian Journal of Anaesthesia 12/2014; 62(3). DOI:10.1007/s12630-014-0285-8 · 2.50 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Ultrasound guidance for continuous femoral perineural catheters may be supplemented by electrical stimulation through a needle or through a stimulating catheter. The authors tested the primary hypothesis that ultrasound guidance alone is noninferior on both postoperative pain scores and opioid requirement and superior on at least one of the two. Second, the authors compared all interventions on insertion time and incremental cost. Patients having knee arthroplasty with femoral nerve catheters were randomly assigned to catheter insertion guided by: (1) ultrasound alone (n = 147); (2) ultrasound and electrical stimulation through the needle (n = 152); or (3) ultrasound and electrical stimulation through both the needle and catheter (n = 138). Noninferiority between any two interventions was defined for pain as not more than 0.5 points worse on a 0 to 10 verbal response scale and for opioid consumption as not more than 25% greater than the mean. The stimulating needle group was significantly noninferior to the stimulating catheter group (difference [95% CI] in mean verbal response scale pain score [stimulating needle vs. stimulating catheter] of -0.16 [-0.61 to 0.29], P < 0.001; percentage difference in mean IV morphine equivalent dose of -5% [-25 to 21%], P = 0.002) and to ultrasound-only group (difference in mean verbal response scale pain score of -0.28 [-0.72 to 0.16], P < 0.001; percentage difference in mean IV morphine equivalent dose of -2% [-22 to 25%], P = 0.006). In addition, the use of ultrasound alone for femoral nerve catheter insertion was faster and cheaper than the other two methods. Ultrasound guidance alone without adding either stimulating needle or needle/catheter combination thus seems to be the best approach to femoral perineural catheters.
    Anesthesiology 04/2014; DOI:10.1097/ALN.0000000000000262 · 6.17 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: This prospective double-blinded, randomized controlled trial compared adductor canal block (ACB) with femoral nerve block (FNB) in patients undergoing total knee arthroplasty. The authors hypothesized that ACB, compared with FNB, would exhibit less quadriceps weakness and demonstrate noninferior pain score and opioid consumption at 6 to 8 h postanesthesia. Patients received an ACB or FNB as a component of a multimodal analgesic. Quadriceps strength, pain score, and opioid consumption were assessed on both legs preoperatively and at 6 to 8, 24, and 48 h postanesthesia administration. In a joint hypothesis test, noninferiority was first evaluated on the primary outcomes of strength, pain score, and opioid consumption at 6 to 8 h; superiority on each outcome at 6 to 8 h was then assessed only if noninferiority was established. Forty-six patients received ACB; 47 patients received FNB. At 6 to 8 h postanesthesia, ACB patients had significantly higher median dynamometer readings versus FNB patients (median [interquartile range], 6.1 kgf [3.5, 10.9] (ACB) vs. 0 kgf [0.0, 3.9] (FNB); P < 0.0001), but was not inferior to FNB with regard to Numeric Rating Scale pain scores (1.0 [0.0, 3.5] ACB vs. 0.0 [0.0, 1.0] FNB; P = 0.019), or to opioid consumption (32.2 [22.4, 47.5] ACB vs. 26.6 [19.6, 49.0]; P = 0.0115). At 24 and 48 h postanesthesia, there was no significant statistical difference in dynamometer results, pain scores, or opioid use between the two groups. At 6 to 8 h postanesthesia, the ACB, compared with the FNB, exhibited early relative sparing of quadriceps strength and was not inferior in both providing analgesia or opioid intake.
    Anesthesiology 01/2014; 120(3). DOI:10.1097/ALN.0000000000000119 · 6.17 Impact Factor