In meta-analysis 3RCTs, why does the Cochrane h'book ref 17.7 discourage using means difference?
In meta-analysis 3RCTs, unlucky random baseline(+SD) & end(+SD) PRO psychometric scores. Cochrane h'book ref 17.7 discourages using means difference? Non-significant result when outcome measure is used alone, significant benefit shown if the change in measure is used, due to unlucky randomisation (all three had worse baseline for treatment arm thus reducing the end score - despite significant improvement). H'book ref 16.1.3.2 advises imputing an SD for the change but worst case assumptions for correlation still just approximate the average of baseline/end SDs.
This issue also sent to Gotzsche and Glasziou, as benefit is 'obvious', but conclusion is negative!
Not sure if any of these thoughts cover what you're looking for:
There are (at least) 3 options for analysing the data: (1) just use outcomes at end of study, (2) use change from baseline, or (3) use outcomes at end, adjusting for baseline. The third option is more powerful and makes fewer assumptions.
In the context of meta-analysis, we want everything to be somehow on the same scale. So may be you want all estimates included to be estimates of the difference between the means adjusting for any baseline. Both (1) and (2) approximate to this in the long-run, so just use the one that's closest / that has the information presented.
Final thought is that if it really is "unlucky randomisation" then in the long-run, there'll be studies that are unlucky in the other direction. So maybe in the context of meta-analysis, this is just a bit of expected heterogeneity, and nothing to be scared of?
But these thoughts may not touch on what you were trying to ask. Maybe you could spell it out a bit more for me if I haven't grasped what you meant.
Concurr, Dr Greenwood. If you ruled the world then (3) ANCOVA would be mandated for all trials, and my dilemma disappears. (1) was used in the published review, and NS result (CI -6 to 1). However a -17 to -2 result from option (2) is pictured, even though imputing a worst-case SD of the difference as being ~1.4 times the highly correlated baseline & end SDs.
I think my problem is with the formula in the Cochrane handbook 16.1.3.2 being so much less conservative than the 1.4*SD [using SEdiff = sqrt (SEbase2 + SEend2)] where base & end variance are near same. Their result is an SD approximating zero - clearly a nonsense.
Randomised trials vary in methodological quality, and flaws in trial conduct can lead to biased estimation of the intervention effect. If a meta-analysis makes no allowance for methodological flaws, there's a danger that the results could be biased and over-precise, or that some lower-quality trials could be given too much influence in the meta-ana...
Gornall’s representation of the evidence on e-cigarettes is fairly one sided and negative.1 For example, he cites a “recent meta-analysis” on the effectiveness of e-cigarettes, which was posted on a blog and has not been peer-reviewed2 but fails to cite the Cochrane review on the same topic, which was published just three months earlier.3 While not...