James Carpenter’s research while affiliated with MRC Clinical Trials Unit at UCL and other places

Background Increasing rates of sexually transmitted infections (STIs) and antimicrobial resistance among young people underscore the urgent need for preventative interventions. Interventions should be evidence-based and tailored to the unique risks and needs associated with varying age, sex and sexual orientation. We used data from the Safetxt trial to explore whether young people’s age, sex and sexual orientation influence (1) their risk of STI reinfection and condom use and (2) the effect of the Safetxt intervention on STI reinfection and condom use. Methods We conducted exploratory secondary analyses of data from the Safetxt trial that evaluated a theory-based digital sexual health intervention tailored according to sex and sexual orientation. We recruited 6248 young people with STIs from 92 UK sexual health clinics and assessed outcomes after 1 year, including the cumulative incidence of STI reinfection and condom use at last sex. We used adjusted logistic regression and margins plots to visualise effect modification. Results There were differences in STI reinfection and condom use by age, sex and sexuality. Age was associated with STI reinfection (OR 0.90, 95% CI 0.87 to 0.94) with evidence for interaction between age and sexuality (p<0.001). Our findings suggest that the risk of STI reinfection decreases with age among young heterosexuals but increases among men-who-have-sex-with-men (MSM). Overall, MSM had the highest likelihood of reinfection (OR 3.53, 95% CI 2.66 to 4.68) despite being more likely to use condoms (OR 1.50, 95% CI 1.18 to 1.91). Among MSM, age modified the intervention effect on condom use at 1 year with highest benefits among participants aged 16–18, moderate to minor benefits among those aged 18–21 and no effect among participants aged 22–24 years. Conclusions Future digital health interventions tailored for diverse sexuality groups need to target young people early enough to have an impact on sexual behaviour. Specific novel interventions are needed for older MSM. Trial registration number ISRCTN64390461 .

Background Epidemiological and clinical studies often have missing data, frequently analysed using multiple imputation (MI). In general, MI estimates will be biased if data are missing not at random (MNAR). Bias due to data MNAR can be reduced by including other variables (“auxiliary variables”) in imputation models, in addition to those required for the substantive analysis. Common advice is to take an inclusive approach to auxiliary variable selection (i.e. include all variables thought to be predictive of missingness and/or the missing values). There are no clear guidelines about the impact of this strategy when data may be MNAR. Methods We explore the impact of including an auxiliary variable predictive of missingness but, in truth, unrelated to the partially observed variable, when data are MNAR. We quantify, algebraically and by simulation, the magnitude of the additional bias of the MI estimator for the exposure coefficient (fitting either a linear or logistic regression model), when the (continuous or binary) partially observed variable is either the analysis outcome or the exposure. Here, “additional bias” refers to the difference in magnitude of the MI estimator when the imputation model includes (i) the auxiliary variable and the other analysis model variables; (ii) just the other analysis model variables, noting that both will be biased due to data MNAR. We illustrate the extent of this additional bias by re-analysing data from a birth cohort study. Results The additional bias can be relatively large when the outcome is partially observed and missingness is caused by the outcome itself, and even larger if missingness is caused by both the outcome and the exposure (when either the outcome or exposure is partially observed). Conclusions When using MI, the naïve and commonly used strategy of including all available auxiliary variables should be avoided. We recommend including the variables most predictive of the partially observed variable as auxiliary variables, where these can be identified through consideration of the plausible casual diagrams and missingness mechanisms, as well as data exploration (noting that associations with the partially observed variable in the complete records may be distorted due to selection bias).

The use of digital health technologies to measure outcomes in clinical trials opens new opportunities as well as methodological challenges. Digital outcome measures may provide more sensitive and higher-frequency measurements but pose vital statistical challenges around how such outcomes should be defined and validated and how trials incorporating digital outcome measures should be designed and analysed. This article presents eight methodological questions, exploring issues such as the length of measurement period, choice of summary statistic and definition and handling of missing data as well as the potential for new estimands and new analyses to leverage the time series data from digital devices. The impact of key issues highlighted by the eight questions on a primary analysis of a trial are illustrated through a simulation study based on the 2019 Bellerophon INOPulse trial which had time spent in MVPA as a digital outcome measure. These eight questions present broad areas where methodological guidance is needed to enable wider uptake of digital outcome measures in trials.

Introduction/aims: Healthcare systems data (also known as real-world or routinely collected health data) could transform the conduct of clinical trials. Demonstrating integrity and provenance of these data is critical for clinical trials, to enable their use where appropriate and avoid duplication using scarce trial resources. Building on previous work, this proof-of-concept study used a data intelligence tool, the “Central Metastore,” to provide metadata and lineage information of nationally held data. Methods: The feasibility of NHS England’s Central Metastore to capture detailed records of the origins, processes, and methods that produce four datasets was assessed. These were England’s Hospital Episode Statistics (Admitted Patient Care, Outpatients, Critical Care) and the Civil Registration of Deaths (England and Wales). The process comprised: information gathering; information ingestion using the tool; and auto-generation of lineage diagrams/content to show data integrity. A guidance document to standardise this process was developed. Results/Discussion: The tool can ingest, store and display data provenance in sufficient detail to support trust and transparency in using these datasets for trials. The slowest step was information gathering from multiple sources, so consistency in record-keeping is essential.

Background Digital technologies, such as wearable devices and smartphone applications (apps), can enable the decentralisation of clinical trials by measuring endpoints in people’s chosen locations rather than in traditional clinical settings. Digital endpoints can allow high-frequency and sensitive measurements of health outcomes compared to visit-based endpoints which provide an episodic snapshot of a person’s health. However, there are underexplored challenges in this emerging space that require interdisciplinary and cross-sector collaboration. A multi-stakeholder Knowledge Exchange event was organised to facilitate conversations across silos within this research ecosystem. Methods A survey was sent to an initial list of stakeholders to identify potential discussion topics. Additional stakeholders were identified through iterative discussions on perspectives that needed representation. Co-design meetings with attendees were held to discuss the scope, format and ethos of the event. The event itself featured a cross-disciplinary selection of talks, a panel discussion, small-group discussions facilitated via a rolling seating plan and audience participation via Slido. A transcript was generated from the day, which, together with the output from Slido, provided a record of the day’s discussions. Finally, meetings were held following the event to identify the key challenges for digital endpoints which emerged and reflections and recommendations for dissemination. Results Several challenges for digital endpoints were identified in the following areas: patient adherence and acceptability; algorithms and software for devices; design, analysis and conduct of clinical trials with digital endpoints; the environmental impact of digital endpoints; and the need for ongoing ethical support. Learnings taken for next generation events include the need to include additional stakeholder perspectives, such as those of funders and regulators, and the need for additional resources and facilitation to allow patient and public contributors to engage meaningfully during the event. Conclusions The event emphasised the importance of consortium building and highlighted the critical role that collaborative, multi-disciplinary, and cross-sector efforts play in driving innovation in research design and strategic partnership building moving forward. This necessitates enhanced recognition by funders to support multi-stakeholder projects with patient involvement, standardised terminology, and the utilisation of open-source software.

The ICH E9(R1) Addendum (International Council for Harmonization 2019) suggests treatment‐policy as one of several strategies for addressing intercurrent events such as treatment withdrawal when defining an estimand. This strategy requires the monitoring of patients and collection of primary outcome data following termination of randomised treatment. However, when patients withdraw from a study early before completion this creates true missing data complicating the analysis. One possible way forward uses multiple imputation to replace the missing data based on a model for outcome on‐ and off‐treatment prior to study withdrawal, often referred to as retrieved dropout multiple imputation. This article introduces a novel approach to parameterising this imputation model so that those parameters which may be difficult to estimate have mildly informative Bayesian priors applied during the imputation stage. A core reference‐based model is combined with a retrieved dropout compliance model, using both on‐ and off‐treatment data, to form an extended model for the purposes of imputation. This alleviates the problem of specifying a complex set of analysis rules to accommodate situations where parameters which influence the estimated value are not estimable, or are poorly estimated leading to unrealistically large standard errors in the resulting analysis. We refer to this new approach as retrieved dropout reference‐base centred multiple imputation.

The use of Digital Health Technologies to measure endpoints in clinical trials opens new opportunities as well as methodological challenges. Digital endpoints may provide more sensitive and higher-frequency measurements of participants' health outcomes, but pose vital statistical challenges around how such endpoints should be defined and validated and how trials incorporating digital endpoints should be designed and analysed. This article presents eight methodological questions, exploring issues around decision making on key issues such as the length of measurement period, choice of summary statistic and definition and handling of missing data, as well as the potential for new estimands and new analyses to leverage the time series data from digital devices. A selection of the challenges are illustrated through a simulation study based on the 2019 Bellerophon INOPulse trial which used time spent in MVPA as a digital endpoint. These eight questions present broad areas where methodological guidance is needed to enable wider uptake of digital endpoints in trials.

Accurate frequentist performance of a method is desirable in confirmatory clinical trials, but is not sufficient on its own to justify the use of a missing data method. Reference‐based conditional mean imputation, with variance estimation justified solely by its frequentist performance, has the surprising and undesirable property that the estimated variance becomes smaller the greater the number of missing observations; as explained under jump‐to‐reference it effectively forces the true treatment effect to be exactly zero for patients with missing data.

Background Non‐inferiority trials comparing different active drugs are often subject to treatment non‐adherence. Intention‐to‐treat (ITT) and per‐protocol (PP) analyses have been advocated in such studies but are not guaranteed to be unbiased in the presence of differential non‐adherence. Methods The REMoxTB trial evaluated two 4‐month experimental regimens compared with a 6‐month control regimen for newly diagnosed drug‐susceptible TB. The primary endpoint was a composite unfavorable outcome of treatment failure or recurrence within 18 months post‐randomization. We conducted a simulation study based on REMoxTB to assess the performance of statistical methods for handling non‐adherence in non‐inferiority trials, including: ITT and PP analyses, adjustment for observed adherence, multiple imputation (MI) of outcomes, inverse‐probability‐of‐treatment weighting (IPTW), and a doubly‐robust (DR) estimator. Results When non‐adherence differed between trial arms, ITT, and PP analyses often resulted in non‐trivial bias in the estimated treatment effect, which consequently under‐ or over‐inflated the type I error rate. Adjustment for observed adherence led to similar issues, whereas the MI, IPTW and DR approaches were able to correct bias under most non‐adherence scenarios; they could not always eliminate bias entirely in the presence of unobserved confounding. The IPTW and DR methods were generally unbiased and maintained desired type I error rates and statistical power. Conclusions When non‐adherence differs between trial arms, ITT and PP analyses can produce biased estimates of efficacy, potentially leading to the acceptance of inferior treatments or efficacious regimens being missed. IPTW and the DR estimator are relatively straightforward methods to supplement ITT and PP approaches.