Project

Development and evaluation of a smartphone app ('Drink Less') for reducing excessive alcohol consumption

Goal: To develop and evaluate a theory- and evidence-based smartphone app with multiple intervention components to help excessive drinkers reduce their alcohol consumption.

Methods: Factorial Design

Updates
0 new
0
Recommendations
0 new
0
Followers
0 new
50
Reads
1 new
1819

Project log

Claire Garnett
added 2 research items
Background : A factorial experiment evaluating the Drink Less app found no clear evidence for main effects of enhanced versus minimal versions of five components but some evidence for an interaction effect. Bayes factors (BFs) showed the data to be insensitive. This study examined the use of BFs to update the evidence with further recruitment. Methods : A between-subject factorial experiment evaluated the main and two-way interaction effects of enhanced versus minimal version of five components of Drink Less. Participants were excessive drinkers, aged 18+, and living in the UK. After the required sample size was reached (n=672), additional data were collected for five months. Outcome measures were change in past week alcohol consumption and Alcohol Use Disorders Identification Test (AUDIT) score at one-month follow-up, amongst responders only (those who completed the questionnaire). BFs (with a half-normal distribution) were calculated (BF<0.33 indicate evidence for null hypothesis; 0.33<BF<3 indicate data are insensitive). Results : Of the sample of 2586, 342 (13.2%) responded to follow-up. Data were mainly insensitive but tended to support there being no large main effects of the enhanced version of individual components on consumption (0.22<BF<0.83) or AUDIT score (0.14<BF<0.98). Data no longer supported there being two-way interaction effects (0.31<BF<1.99). In an additional exploratory analysis, participants receiving four of the components averaged a numerically greater reduction in consumption than those not receiving any (21.6 versus 12.1 units), but the data were insensitive (BF=1.42). Conclusions : Data from extended recruitment in a factorial experiment evaluating components of Drink Less remained insensitive but tended towards individual and pairs of components not having a large effect. In an exploratory analysis, there was weak, anecdotal evidence for a synergistic effect of four components. In the event of uncertain results, calculating BFs can be used to update the strength of evidence of a dataset supplemented with extended recruitment.
Background : A factorial experiment evaluating the Drink Less app found no clear evidence for main effects of enhanced versus minimal versions of five components but some evidence for an interaction effect. Bayes factors (BFs) showed the data to be insensitive. This study examined the use of BFs to update the evidence with further recruitment. Methods : A between-subject factorial experiment evaluated the main and two-way interaction effects of enhanced versus minimal version of five components of Drink Less. Participants were excessive drinkers, aged 18+, and living in the UK. After the required sample size was reached (n=672), additional data were collected for five months. Outcome measures were change in past week alcohol consumption and Alcohol Use Disorders Identification Test (AUDIT) score at one-month follow-up, amongst responders only. BFs (with a half-normal distribution) were calculated for those for which we had outcome data (BF<0.33 indicate evidence for null hypothesis; 0.33<BF<3 indicate data are insensitive). Results : Of the sample of 2586, 342 (13.2%) responded to follow-up. Data were mainly insensitive but tended to support there being no large main effects of the enhanced version of individual components on consumption (0.22<BF<0.83) or AUDIT score (0.14<BF<0.98). Data no longer supported there being two-way interaction effects. In an unplanned comparison, participants receiving the four most promising components averaged a numerically greater reduction in consumption than those not receiving any (21.6 versus 12.1 units), but the data were insensitive (BF=1.42). Conclusions : Data from extended recruitment in a factorial experiment evaluating components of the Drink Less app remained insensitive but tended towards individual and pairs of components not having a large effect. There was weak evidence for a synergistic effect of four components. In the event of uncertain results, calculating BFs can be used to update the strength of evidence of a dataset supplemented with extended recruitment.
Claire Garnett
added a research item
Background Digital interventions for alcohol can help achieve reductions in hazardous and harmful alcohol consumption. The Drink Less app was developed using evidence and theory, and a factorial randomized controlled trial (RCT) suggested that 4 of its intervention modules may assist with drinking reduction. However, low engagement is an important barrier to effectiveness, and low response to follow up is a challenge for intervention evaluation. Research is needed to understand what factors influence users’ level of engagement, response to follow up, and extent of alcohol reduction. Objective This study aimed to investigate associations between user characteristics, engagement, response to follow up, and extent of alcohol reduction in an app-based intervention, Drink Less. Methods This study involved a secondary data analysis of a factorial RCT of the Drink Less app. Participants (N=672) were aged 18 years or older, lived in the United Kingdom, and had an Alcohol Use Disorders Identification Test score >7 (indicative of excessive drinking). Sociodemographic and drinking characteristics were assessed at baseline. Engagement was assessed in the first month of use (number of sessions, time on app, number of days used, and percentage of available screens viewed). Response to follow up and extent of alcohol reduction (change in past week consumption) were measured after 1 month. Associations were assessed using unadjusted and adjusted linear or logistic regression models. ResultsAge (all unstandardized regression coefficients [B] >.02, all P.18, all P
Claire Garnett
added 3 research items
A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.
This PhD research programme aimed to develop and evaluate a smartphone app to reduce excessive alcohol consumption and used the theoretical framework of the Behaviour Change Wheel to guide its development and evaluation. There are many different factors influencing alcohol consumption that can be targeted in an intervention to reduce excessive alcohol consumption. This thesis focuses on the cognitive and motivational factors affecting alcohol consumption. The thesis involves three stages: i) work informing intervention content to prioritise for inclusion; ii) the development of the app; and iii) evaluation of the app. The first stage involved four studies about who uses apps to reduce excessive alcohol use; how theory is currently used in existing digital alcohol interventions; people’s knowledge about how their drinking compares with others, and experts’ opinions on modules likely to be most effective in apps for reducing excessive alcohol consumption. Initial development and the first version of the app was based on pragmatic considerations as to how to deliver the intervention content, app developers’ opinion based on previous experience, previous delivery of similar intervention content, and frameworks for engagement and design. A person-based approach was taken in two usability studies conducted to inform further iterations and the final version. The app was evaluated using a factorial RCT to assess which intervention modules were most effective. The results of the trial relating to the cognitive and motivational factors suggest that the normative feedback and cognitive bias re-training modules may assist with drinking reduction and are worthy of including in an optimised app for further development and evaluation in a full-scale RCT.
Excessive alcohol consumption poses a serious problem for public health. Digital behavior change interventions have the potential to help users reduce their drinking. In accordance with Open Science principles, this paper describes the development of a smartphone app to help individuals who drink excessively to reduce their alcohol consumption. Following the UK Medical Research Council’s guidance and the Multiphase Optimization Strategy, development consisted of two phases: (i) selection of intervention components and (ii) design and development work to implement the chosen components into modules to be evaluated further for inclusion in the app. Phase 1 involved a scoping literature review, expert consensus study and content analysis of existing alcohol apps. Findings were integrated within a broad model of behavior change (Capability, Opportunity, Motivation-Behavior). Phase 2 involved a highly iterative process and used the “Person-Based” approach to promote engagement. From Phase 1, five intervention components were selected: (i) Normative Feedback, (ii) Cognitive Bias Re-training, (iii) Self-monitoring and Feedback, (iv) Action Planning, and (v) Identity Change. Phase 2 indicated that each of these components presented different challenges for implementation as app modules; all required multiple iterations and design changes to arrive at versions that would be suitable for inclusion in a subsequent evaluation study. The development of the Drink Less app involved a thorough process of component identification with a scoping literature review, expert consensus, and review of other apps. Translation of the components into app modules required a highly iterative process involving user testing and design modification.
Claire Garnett
added a research item
Our aim was to evaluate intervention components of an alcohol reduction app: Drink Less. Excessive drinkers (AUDIT>=8) were recruited to test enhanced versus minimal (reduced functionality) versions of five app modules in a 25 factorial trial. Modules were: Self-monitoring and Feedback, Action Planning, Identity Change, Normative Feedback, and Cognitive Bias Re-training. Outcome measures were: change in weekly alcohol consumption (primary); full AUDIT score, app usage, app usability (secondary). Main effects and two-way interactions were assessed by ANOVA using intention-to-treat. A total of 672 study participants were included. There were no significant main effects of the intervention modules on change in weekly alcohol consumption or AUDIT score. There were two-way interactions between enhanced Normative Feedback and Cognitive Bias Re-training on weekly alcohol consumption (F = 4.68, p = 0.03) and between enhanced Self-monitoring and Feedback and Action Planning on AUDIT score (F = 5.82, p = 0.02). Enhanced Self-monitoring and Feedback was used significantly more often and rated significantly more positively for helpfulness, satisfaction and recommendation to others than the minimal version. To conclude, in an evaluation of the Drink Less smartphone application, the combination of enhanced Normative Feedback and Cognitive Bias Re-training and enhanced Self-monitoring and Feedback and Action Planning yielded improvements in alcohol-related outcomes after 4-weeks.
Claire Garnett
added 2 research items
Background Digital behavior change interventions (DBCIs) appear to reduce alcohol consumption, but greater understanding is needed of their mechanisms of action. Purpose To describe the behavior change techniques (BCTs) used in DBCIs and examine whether individual BCTs, the inclusion of more BCTs or more Control Theory congruent BCTs is associated with increased effectiveness. Methods Forty-one randomized control trials were extracted from a Cochrane review of alcohol reduction DBCIs and coded for up to 93 BCTs using an established and reliable method. Random effects unadjusted and adjusted meta-regression models were performed to assess associations between BCTs and intervention effectiveness. Results Interventions used a mean of 9.1 BCTs (range 1–22), 23 different BCTs were used in four or more trials. Trials that used “Behavior substitution” (−95.112 grams per week [gpw], 95% CI: −162.90, −27.34), “Problem solving” (−45.92 gpw, 95% CI: −90.97, −0.87) and “Credible source” (−32.09 gpw, 95% CI: −60.64, −3.55) were significantly associated with greater alcohol reduction than trials without these BCTs. The “Behavior substitution” result should be treated as preliminary because it was reported in only four trials, three of which were conducted by the same research group. “Feedback” was used in 98% of trials (n = 41); other Control Theory congruent BCTs were used less frequently: for example, “Goal setting” 43% (n = 18) and “Self-monitoring” 29%, (n = 12). Conclusions “Behavior substitution,” “Problem solving,” and “Credible source” were associated with greater alcohol reduction. Many BCTs were used infrequently in DBCIs, including BCTs with evidence of effectiveness in other domains, such as “Self-monitoring” and “Goal setting.”
Background: Applying theory to the design and evaluation of interventions is likely to increase effectiveness and improve the evidence base from which future interventions are developed, though few interventions report this. Objective: The aim of this paper was to assess how digital interventions to reduce hazardous and harmful alcohol consumption report the use of theory in their development and evaluation, and whether reporting of theory use is associated with intervention effectiveness. Methods: Randomized controlled trials were extracted from a Cochrane review on digital interventions for reducing hazardous and harmful alcohol consumption. Reporting of theory use within these digital interventions was investigated using the theory coding scheme (TCS). Reported theory use was analyzed by frequency counts and descriptive statistics. Associations were analyzed with meta-regression models. Results: Of 41 trials involving 42 comparisons, half did not mention theory (50% [21/42]), and only 38% (16/42) used theory to select or develop the intervention techniques. Significant heterogeneity existed between studies in the effect of interventions on alcohol reduction (I2=77.6%, P<.001). No significant associations were detected between reporting of theory use and intervention effectiveness in unadjusted models, though the meta-regression was underpowered to detect modest associations. Conclusions: Digital interventions offer a unique opportunity to refine and develop new dynamic, temporally sensitive theories, yet none of the studies reported refining or developing theory. Clearer selection, application, and reporting of theory use is needed to accurately assess how useful theory is in this field and to advance the field of behavior change theories.
Claire Garnett
added a research item
Background: Excessive alcohol use contributes significantly to physical and psychological illness, injury and death, and a wide array of social harm in all age groups. A proven strategy for reducing excessive alcohol consumption levels is to offer a brief conversation-based intervention in primary care settings, but more recent technological innovations have enabled people to interact directly via computer, mobile device or smartphone with digital interventions designed to address problem alcohol consumption. Objectives: To assess the effectiveness and cost-effectiveness of digital interventions for reducing hazardous and harmful alcohol consumption, alcohol-related problems, or both, in people living in the community, specifically: (i) Are digital interventions more effective and cost-effective than no intervention (or minimal input) controls? (ii) Are digital interventions at least equally effective as face-to-face brief alcohol interventions? (iii) What are the effective component behaviour change techniques (BCTs) of such interventions and their mechanisms of action? (iv) What theories or models have been used in the development and/or evaluation of the intervention? Secondary objectives were (i) to assess whether outcomes differ between trials where the digital intervention targets participants attending health, social care, education or other community-based settings and those where it is offered remotely via the internet or mobile phone platforms; (ii) to specify interventions according to their mode of delivery (e.g. functionality features) and assess the impact of mode of delivery on outcomes. Search methods: We searched CENTRAL, MEDLINE, PsycINFO, CINAHL, ERIC, HTA and Web of Knowledge databases; ClinicalTrials.com and WHO ICTRP trials registers and relevant websites to April 2017. We also checked the reference lists of included trials and relevant systematic reviews. Selection criteria: We included randomised controlled trials (RCTs) that evaluated the effectiveness of digital interventions compared with no intervention or with face-to-face interventions for reducing hazardous or harmful alcohol consumption in people living in the community and reported a measure of alcohol consumption. Data collection and analysis: We used standard methodological procedures expected by The Cochrane Collaboration. Main results: We included 57 studies which randomised a total of 34,390 participants. The main sources of bias were from attrition and participant blinding (36% and 21% of studies respectively, high risk of bias). Forty one studies (42 comparisons, 19,241 participants) provided data for the primary meta-analysis, which demonstrated that participants using a digital intervention drank approximately 23 g alcohol weekly (95% CI 15 to 30) (about 3 UK units) less than participants who received no or minimal interventions at end of follow up (moderate-quality evidence).Fifteen studies (16 comparisons, 10,862 participants) demonstrated that participants who engaged with digital interventions had less than one drinking day per month fewer than no intervention controls (moderate-quality evidence), 15 studies (3587 participants) showed about one binge drinking session less per month in the intervention group compared to no intervention controls (moderate-quality evidence), and in 15 studies (9791 participants) intervention participants drank one unit per occasion less than no intervention control participants (moderate-quality evidence).Only five small studies (390 participants) compared digital and face-to-face interventions. There was no difference in alcohol consumption at end of follow up (MD 0.52 g/week, 95% CI -24.59 to 25.63; low-quality evidence). Thus, digital alcohol interventions produced broadly similar outcomes in these studies. No studies reported whether any adverse effects resulted from the interventions.A median of nine BCTs were used in experimental arms (range = 1 to 22). 'B' is an estimate of effect (MD in quantity of drinking, expressed in g/week) per unit increase in the BCT, and is a way to report whether individual BCTs are linked to the effect of the intervention. The BCTs of goal setting (B -43.94, 95% CI -78.59 to -9.30), problem solving (B -48.03, 95% CI -77.79 to -18.27), information about antecedents (B -74.20, 95% CI -117.72 to -30.68), behaviour substitution (B -123.71, 95% CI -184.63 to -62.80) and credible source (B -39.89, 95% CI -72.66 to -7.11) were significantly associated with reduced alcohol consumption in unadjusted models. In a multivariable model that included BCTs with B > 23 in the unadjusted model, the BCTs of behaviour substitution (B -95.12, 95% CI -162.90 to -27.34), problem solving (B -45.92, 95% CI -90.97 to -0.87), and credible source (B -32.09, 95% CI -60.64 to -3.55) were associated with reduced alcohol consumption.The most frequently mentioned theories or models in the included studies were Motivational Interviewing Theory (7/20), Transtheoretical Model (6/20) and Social Norms Theory (6/20). Over half of the interventions (n = 21, 51%) made no mention of theory. Only two studies used theory to select participants or tailor the intervention. There was no evidence of an association between reporting theory use and intervention effectiveness. Authors' conclusions: There is moderate-quality evidence that digital interventions may lower alcohol consumption, with an average reduction of up to three (UK) standard drinks per week compared to control participants. Substantial heterogeneity and risk of performance and publication bias may mean the reduction was lower. Low-quality evidence from fewer studies suggested there may be little or no difference in impact on alcohol consumption between digital and face-to-face interventions.The BCTs of behaviour substitution, problem solving and credible source were associated with the effectiveness of digital interventions to reduce alcohol consumption and warrant further investigation in an experimental context.Reporting of theory use was very limited and often unclear when present. Over half of the interventions made no reference to any theories. Limited reporting of theory use was unrelated to heterogeneity in intervention effectiveness.
Claire Garnett
added a research item
Background Interventions delivered by smartphone apps have the potential to help drinkers reduce their consumption of alcohol. To optimise engagement and reduce the high rates of attrition associated with the use of digital interventions it is necessary to ensure that an app’s design and functionality is appropriate for its intended purposes and target population. Aims To understand the user experience of an app to help people reduce their alcohol consumption. Method The app, Drink Less, contains a core module focusing on goal setting, supplemented by five additional modules: self-monitoring and feedback, identity change, cognitive bias re-training, action planning, and social comparison. Two studies were conducted, a ‘think aloud’ study performed with people using the app for the first time and a semi-structured interview study performed after users had had access to the app for at least 2 weeks. A thematic analysis of the ‘think aloud’ and interview transcripts was conducted by one coder and verified by a second. Results Twenty-four participants, half of whom were women and half from disadvantaged groups, took part in the two studies. Three main themes identified in the data were: ‘Feeling lost and unsure of what to do next’; ‘Make the app easy to use’; and ‘Make the app beneficial and rewarding to use’. These themes reflected participants’ need for (i) guidance, particularly when first using the app or when entering data; (ii) the data entry process to be simple and the navigation intuitive; (iii) neither the amount of text nor range of options to be overwhelming; (iv) the app to reward them for effort and progress; and (v) it to be clear how the app could help alcohol reduction goals be reached. Conclusion First time and experienced users want an alcohol reduction app to be easy, rewarding and beneficial to use. An easy-to-use app would reduce user burden, offer ongoing help and be aesthetically pleasing. A rewarding and beneficial app would provide positive reinforcement, give feedback about progress and demonstrate credibility. Users need help when first using the app and they need a compelling reason to continue using it.
David Crane
added a research item
This is the protocol for a review and there is no abstract. The objectives are as follows: The main objective is to assess the effectiveness and cost effectiveness of digital interventions for reducing hazardous and harmful alcohol consumption and/or alcohol-related problems in community-dwelling populations. We envisage two comparator groups: (1) no intervention (or minimal input) controls; and (2) another active intervention for delivering preventive advice or counselling to reduce hazardous or harmful alcohol consumption. Specifically, we will address two questions: (1) Are digital interventions superior to no intervention (or minimal input) controls? This question is important for individuals accessing interventions through their own motivation or interest. These individuals will be unlikely to experience active practitioner input and it is important to understand whether digital interventions are better than general material they might seek out on the internet or via mobile phone-based apps etc. (2) Are digital interventions at least equally effective as face-to-face brief alcohol interventions? Practitioner delivered brief interventions are generally accepted to be the best alternative in secondary preventive care in health, workplace, educational or community settings. However, time constraints can impede face-to-face delivery of such interventions and it is important to know whether digitally provided input can yield comparable effects to interventions delivered by trained practitioners. We will also identify the most effective component behaviour change techniques of such interventions and their mechanisms of action. Secondary objectives are as follows: 1.To assess whether outcomes differ between trials where the digital intervention targets participants attending health, social care, education or other community-based settings and those where it is offered remotely via the internet or mobile phone platforms; 2.To develop a taxonomy of interventions according to their mode of delivery (e.g. functionality features) and assess their impact on outcomes; 3.To identify theories or models that have been used in the development and/or evaluation of the intervention – this will inform intervention development work.
Claire Garnett
added a project goal
To develop and evaluate a theory- and evidence-based smartphone app with multiple intervention components to help excessive drinkers reduce their alcohol consumption.
 
Claire Garnett
added 3 research items
Mobile phone apps have the potential to reduce excessive alcohol consumption cost-effectively. Although hundreds of alcohol-related apps are available, there is little information about the behavior change techniques (BCTs) they contain, or the extent to which they are based on evidence or theory and how this relates to their popularity and user ratings. Our aim was to assess the proportion of popular alcohol-related apps available in the United Kingdom that focus on alcohol reduction, identify the BCTs they contain, and explore whether BCTs or the mention of theory or evidence is associated with app popularity and user ratings. We searched the iTunes and Google Play stores with the terms "alcohol" and "drink", and the first 800 results were classified into alcohol reduction, entertainment, or blood alcohol content measurement. Of those classified as alcohol reduction, all free apps and the top 10 paid apps were coded for BCTs and for reference to evidence or theory. Measures of popularity and user ratings were extracted. Of the 800 apps identified, 662 were unique. Of these, 13.7% (91/662) were classified as alcohol reduction (95% CI 11.3-16.6), 53.9% (357/662) entertainment (95% CI 50.1-57.7), 18.9% (125/662) blood alcohol content measurement (95% CI 16.1-22.0) and 13.4% (89/662) other (95% CI 11.1-16.3). The 51 free alcohol reduction apps and the top 10 paid apps contained a mean of 3.6 BCTs (SD 3.4), with approximately 12% (7/61) not including any BCTs. The BCTs used most often were "facilitate self-recording" (54%, 33/61), "provide information on consequences of excessive alcohol use and drinking cessation" (43%, 26/61), "provide feedback on performance" (41%, 25/61), "give options for additional and later support" (25%, 15/61) and "offer/direct towards appropriate written materials" (23%, 14/61). These apps also rarely included any of the 22 BCTs frequently used in other health behavior change interventions (mean 2.46, SD 2.06). Evidence was mentioned by 16.4% of apps, and theory was not mentioned by any app. Multivariable regression showed that apps including advice on environmental restructuring were associated with lower user ratings (Β=-46.61, P=.04, 95% CI -91.77 to -1.45) and that both the techniques of "advise on/facilitate the use of social support" (Β=2549.21, P=.04, 95% CI 96.75-5001.67) and the mention of evidence (Β=1376.74, P=.02, 95%, CI 208.62-2544.86) were associated with the popularity of the app. Only a minority of alcohol-related apps promoted health while the majority implicitly or explicitly promoted the use of alcohol. Alcohol-related apps that promoted health contained few BCTs and none referred to theory. The mention of evidence was associated with more popular apps, but popularity and user ratings were only weakly associated with the BCT content.
Background: Excessive alcohol consumption is a leading cause of death and morbidity worldwide and interventions to help people reduce their consumption are needed. Interventions delivered by smartphone apps have the potential to help harmful and hazardous drinkers reduce their consumption of alcohol. However, there has been little evaluation of the effectiveness of existing smartphone interventions. A systematic review, amongst other methodologies, identified promising modular content that could be delivered by an app: self-monitoring and feedback; action planning; normative feedback; cognitive bias re-training; and identity change. This protocol reports a factorial randomised controlled trial to assess the comparative potential of these five intervention modules to reduce excessive alcohol consumption. Methods: A between-subject factorial randomised controlled trial. Hazardous and harmful drinkers aged 18 or over who are making a serious attempt to reduce their drinking will be randomised to one of 32 (2(5)) experimental conditions after downloading the 'Drink Less' app. Participants complete baseline measures on downloading the app and are contacted after 1-month with a follow-up questionnaire. The primary outcome measure is change in past week consumption of alcohol. Secondary outcome measures are change in AUDIT score, app usage data and usability ratings for the app. A factorial between-subjects ANOVA will be conducted to assess main and interactive effects of the five intervention modules for the primary and secondary outcome measures. Discussion: This study will establish the extent to which the five intervention modules offered in this app can help reduce hazardous and harmful drinking. This is the first step in optimising and understanding what component parts of an app could help to reduce excessive alcohol consumption. The findings from this study will be used to inform the content of a future integrated treatment app and evaluated against a minimal control in a definitive randomised control trial with long-term outcomes. Trial registration: ISRCTN40104069 Date of registration: 10/2/2016.
Background Digital interventions to reduce excessive alcohol consumption have the potential to have a broader reach and be more cost-effective than traditional brief interventions. However, there is not yet strong evidence for their ability to engage users or their effectiveness. Objective This study aimed to identify the behavior change techniques (BCTs) and engagement strategies most worthy of further study by inclusion in a smartphone app to reduce alcohol consumption, using formal expert consensus methods. Methods The first phase of the study consisted of a Delphi exercise with three rounds. It was conducted with 7 international experts in the field of alcohol and/or behavior change. In the first round, experts identified BCTs most likely to be effective at reducing alcohol consumption and strategies most likely to engage users with an app; these were rated in the second round; and those rated as effective by at least four out of seven participants were ranked in the third round. The rankings were analyzed using Kendall’s W coefficient of concordance, which indicates consensus between participants. The second phase consisted of a new, independent group of experts (n=43) ranking the BCTs that were identified in the first phase. The correlation between the rankings of the two groups was assessed using Spearman’s rank correlation coefficient. ResultsTwelve BCTs were identified as likely to be effective. There was moderate agreement among the experts over their ranking (W=.465, χ211=35.8, P