Content uploaded by Xiaoxi Yan
Author content
All content in this area was uploaded by Xiaoxi Yan on Feb 08, 2022
Content may be subject to copyright.
Contemporary Clinical Trials Communications 26 (2022) 100899
Available online 1 February 2022
2451-8654/© 2022 Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
A pilot sequential multiple assignment randomized trial (SMART) protocol
for developing an adaptive coaching intervention around a mobile
application for athletes to improve carbohydrate periodization behavior
Xiaoxi Yan
a
,
b
,
*
, David M. Dunne
b
,
c
, Samuel G. Impey
b
,
d
, Brian Cunniffe
b
,
e
,
Carmen E. Lefevre
b
,
f
, Rodrigo Mazorra
b
, James P. Morton
c
, David Tod
c
, Graeme L. Close
c
,
Rebecca Murphy
c
, Bibhas Chakraborty
a
,
g
,
h
a
Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, 169857
b
Applied Behaviour Systems Ltd, London, N1 7GU, UK
c
Research Institute for Sport and Exercise Sciences, Liverpool John Moores University, Liverpool, L2 2QP, UK
d
Center for Exercise and Sports Science Research, School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, 6027, Australia
e
Institute of Sport, Exercise and Health (ISEH), London, W1T 7HA, UK
f
Centre for Behaviour Change, University College London, London, WC1E 6BT, UK
g
Department of Statistics and Data Science, National University of Singapore, Singapore, 119077
h
Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, 27708, United States
ARTICLE INFO
Keywords:
Mobile application
Athletes
Carbohydrate periodization
Behavioral sciences
Adaptive interventions
Sequential multiple assignment randomized
trial
ABSTRACT
Background: It has recently been identied that manipulating carbohydrate availability around exercise activity
can enhance training-induced metabolic adaptations. Despite this approach being accepted in the athletic pop-
ulations, athletes do not systematically follow the guidelines. Digital environments appear to allow nutritionists
to deliver this intervention at scale, reducing expensive human coaching time. Yet, digitally delivered dietary
behavior change interventions for athletes and the coaching strategy to support them are still novel concepts
within sports nutrition.
Methods/design: We aim to recruit 900 athletes across the UK. 500 athletes will be recruited to test the feasibility
of a novel menu planner mobile application with coaching for 6 weeks. 250 athletes with pre-existing nutritionist
support will also be recruited as control. We will then conduct a 4-week pilot sequential multiple assignment
randomized trial (SMART) with an additional 150 athletes. In the SMART, athletes will be given the application
and additional coaching according to their engagement responses. The primary outcomes are the mobile
application and coach uptake, retention, engagement, and success in attaining carbohydrate periodization
behavior. Secondary outcomes are changes in goal, weight, carbohydrate periodization self-efcacy, and beliefs
about consequences. Due to the high attrition nature of digital interventions, all quantitative analyses will be
carried out based on both the intention-to-treat and per-protocol principles.
Discussion: This study will be the rst to investigate improving carbohydrate periodization using a digital
approach and tailored coaching strategies under this context. Foundational evidence from this study will provide
insights into the feasibility of the digital approach.
1. Introduction
Sports nutrition has long been aware of the benets of carbohydrates
when it comes to exercise performance [1,2]. However, more recently it
has been identied that manipulating carbohydrate availability to
strategically undertake specic training sessions with higher – and
where appropriate lower – carbohydrate availability maintains meta-
bolic exibility, enhancing training-induced adaptations [3,4]. As a
result, a “fuel for the work required” theoretical framework was devel-
oped [5]. This framework postulates that the provision of nutrition, with
a focus on carbohydrate, should be tailored to the individual based on
the exercise they undertake, and the time available for recovery, to
optimize the desired training or performance response. This has since
* Corresponding author. Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, 169857.
E-mail address: xiaoxi.yan@u.duke.nus.edu (X. Yan).
Contents lists available at ScienceDirect
Contemporary Clinical Trials Communications
journal homepage: www.elsevier.com/locate/conctc
https://doi.org/10.1016/j.conctc.2022.100899
Received 13 March 2021; Received in revised form 23 December 2021; Accepted 30 January 2022
Contemporary Clinical Trials Communications 26 (2022) 100899
2
been recognized and accepted as an applied intervention strategy in the
eld of sports nutrition [5–8]. Yet, despite athletes knowing how to
periodize carbohydrate and energy intake, they do not systematically
follow current sports nutrition guidelines [9,10]. Heikura et al. [9]
report that athletes struggled to stick to this dietary periodization
behavior and highlight that the support required from a practitioner is
highly personalized, and as a result time-consuming.
This gap between knowledge and behavior is not new as athletes
continue to struggle to adhere despite knowing what and when to eat
[11,12]. This may be due to sports nutritionists’ lack of formal behavior
change training, or the time required to develop such an intervention
[13]. In addition to practitioner challenges, Bentley et al. [12] identied
a lack of food planning skills as a barrier to nutritional adherence in
athletes, linking it to lower levels of motivation. Notably, the authors
suggested that athletes need to develop not only the capability to plan,
but also be presented with repeated opportunities to practice planning.
Given practitioner’s time and resource constraints and the fact that food
choice is a dynamic, complex, and continually changing process, it ap-
pears technology may be well placed to offer a continuous and scalable
solution for both parties [14,15].
To date, technology-enabled interventions in nutrition are positively
associated with changes in diet [16–19]. It follows then, that digital
environments may provide sports nutritionists opportunities to deliver
behavior change interventions that can better support the needs of
athletes, as well as have the potential to save practitioner time. As a
result, we developed a theoretically driven digital menu planner mobile
application to enable athletes to plan their food intake in line with their
training and competition. However, to date, no research has explored
the effectiveness of such a digital tool in sports nutrition.
The rst objective of this study is to assess the feasibility of using the
custom-built, native menu planner mobile application supported by
nutrition coaching in improving carbohydrate periodization behavior.
The second objective is to assess the feasibility of strategically timing
human nutrition coaching around the mobile application to improve its
effectiveness. Secondary aims include assessing other related health and
behavioral outcomes such as weight, goal, change in carbohydrate
periodization self-efcacy, and belief about consequences. Exploratory
aims include assessing the impact of personality and need for autonomy
on application usage, and the potential cost-effectiveness of the
approach. The feasibility of the study is dened by the uptake of the
mobile application and nutrition coaching, participants’ retention and
engagement to the application, successful characterization of carbohy-
drate periodization behavior, and identication of suitable strategy (ies)
proposed for the second objective. The ndings from this study will help
to further rene the application and inform the next steps to the iden-
tied strategy (ies) (e.g., a conrmatory trial).
2. Methods
2.1. Research design
This study will be conducted in two phases, the observational and
pilot SMART phases. The ow process of this study is detailed in Fig. 1.
2.1.1. Observational phase
To assess the feasibility of the menu planner mobile application
paired with nutrition coaching, an observational study will be carried
out with participants using the application and usual nutrition coaching
(see interventions below) for 6 weeks. A separate group of participants
(with regular pre-existing nutritionist support) will be recruited as
control where no intervention will be given. The operational aspects,
including participation recruitment via gatekeepers, dissemination, and
uptake of the apps, communication channels will also be assessed as part
of feasibility. Operational procedures such as email communications and
mobile application dissemination (see Appendix Table A.1 for details)
will be rened as needed, and any bugs in the mobile application will
also be xed prior to the pilot SMART phase.
2.1.2. Pilot SMART phase
Subsequently, a 4-week pilot sequential multiple assignment ran-
domized trial (SMART) will be carried out on new participants to gain a
better understanding and develop possible adaptive coaching interven-
tion strategies around the mobile application. An adaptive intervention
strategy is a sequence of decision rules that specify when, how, and what
intervention to offer based on the present and past information. The
Abbreviations:
App: custom-built native menu planner mobile application
NC nutrition coaching
SMART sequential multiple assignment randomized trial
Fig. 1. General ow process of the whole study.
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
3
SMART is an efcient experimental approach suitable for developing
such adaptive intervention strategies, especially when there is not
enough a priori information on what sequence, when, and how the
intervention components within an adaptive intervention strategy
should be tailored [20,21]. In this study’s context, the SMART design is
used to explore when the human nutrition coaching should be assigned
and, when it can be stopped, given the mobile application. From here on,
we refer to the adaptive coaching intervention strategies in the SMART
design as “strategies”.
In this three-stage SMART design, as shown in Fig. 2, all participants
will receive the menu planner mobile application (App) at baseline.
Participants will be randomized at baseline to determine if they will
follow relaxed or stringent response criteria at two time points, the end
of week 1 and week 2, for re-randomization. Participants following the
relaxed response criteria (all the pathways starting with subgroup A in
Fig. 2), will be classied as responders if they engage with the App at
least once at the end of week 1 (i.e., e1≥1). Responders will continue
with the App only (subgroup G), while non-responders will be re-
randomized to either continue with the App only (subgroup C; note
subgroups G and C receives the same intervention) or App and nutrition
coaching (App +NC) (subgroup D) for one week. At the end of week 2,
participants will be re-classied again based on their weekly App
engagement (e2). If participants have e2≥2, they will receive App only
(subgroups G1, C1, and D1), else they will be re-randomized again to
either receive App (subgroups G2, C2, D2) or App +NC (subgroups G3,
C3, D3) at weeks 3–4. Participants following the stringent response
criteria (all the pathways starting with subgroup B) follow the same
structure, except participants will only be classied as responders when
e1≥2 and e2≥3.
It is important to highlight that although the participants may
receive the same intervention at a given time-point, the pathways
experienced by the participants are different. For example, for partici-
pants following the relaxed response criteria and receiving the App at
weeks 3–4, subgroup G1 consists of participants who were consistently
responders at both time-points (i.e., e1≥1 and e2≥2) and received the
App only in weeks 1 and 2 (see the example pathway highlighted in
Fig. 2); subgroup C1 consists of participants who were initially a non-
responder then a responder (i.e., e1=0 and e2≥2) but also received
the App only in weeks 1 and 2; subgroup D1 consists of participants who
were initially a non-responder then a responder (i.e., e1=0 and e2≥2)
and they received App +NC at week 2. By going through all the 18
possible design pathways in Fig. 2, data for a total of 16 embedded
strategies may be constructed. We present the corresponding mathe-
matical expressions of the 16 embedded strategies in Table 1. Note that
we did not simplify the mathematical expressions in Table 1, to better
correspond with Fig. 2. To further facilitate the comprehension of these
strategies, we take strategy 7 (also corresponding to the highlighted
strategy in Fig. 2) as an example. Strategy 7 can be described as “Use the
relaxed response criteria. First, give App in week 1, if the participant has
an engagement of ≥1, the participant is a responder and continues with
App, else give App +NC in week 2. If the participant has an engagement
of ≥2 in week 2, give App in weeks 3–4 regardless of the initial response
status. If the participant has an engagement of <2 in week 2, give App if
the participant was initially a responder, else give App +NC in weeks
3–4.” It is worth pointing out that the 16 strategies primarily explore
giving or not giving (App +NC) at different time-points and under
different response criteria, thereby exploring the second objective on
strategically timing human nutrition coaching around the application.
2.2. Description of the interventions
2.2.1. Menu planner mobile application (app)
The menu planner mobile application (App) is a custom-built native
application that is operable on either Android or iOS devices. The
custom-built app features were constructed using the Behavior Change
Wheel and Atkins and Michie’s six-step approach to dietary behavior
Fig. 2. The SMART design. The highlighted intervention pathway is an example of one of the 16 possible embedded strategies and the dashed arrow is one of the 18
possible pathways (C1-G3 and M1-M3) a participant may go through.
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
4
change intervention design [22,23]. The App centers around a native
menu planner targeting carbohydrate periodization behaviors in ath-
letes. A separate paper will describe the development process and the
behavioral change techniques implemented in this App. Fig. 3 shows the
key screens from the App.
In the App, the carbohydrate periodized menu planner is a weekly
Table 1
The 16 embedded strategies (in mathematical expressions) in the pilot SMART phase, where the relaxed response criteria have responders as (e1 ≥1 day/week and e2≥
2 days/week), and stringent response criteria as (e1 ≥2 days/week and e2≥3 days/week).
Embedded
Strategies
Response
Criteria
Intervention at week 1
(Stage 1)
Intervention at week 2
(stage 2)
Intervention at weeks 3–4 (Stage 3) Subgroups involve at
stages 1, 2 and 3
1 Relaxed App Appe1≥1Appe1=0 Appe1≥1(Appe2≥2Appe2<2)Appe1=0(Appe2≥2Appe2<2)A, {G, C}, {G1, G2, C1,
C2}
2 Appe1≥1(Appe2≥2(App +NC)e2<2)
Appe1=0(Appe2≥2Appe2<2)
A, {G, C}, {G1, G3, C1,
C2}
3 Appe1≥1(Appe2≥2Appe2<2)
Appe1=0(Appe2≥2(App +NC)e2<2)
A, {G, C}, {G1, G2, C1,
C3}
4 Appe1≥1(Appe2≥2(App +NC)e2<2)
Appe1=0(Appe2≥2(App +NC)e2<2)
A, {G, C}, {G1, G3, C1,
C3}
5 Appe1≥1(App +NC)e1=0 Appe1≥1(Appe2≥2Appe2<2)
(App +NC)e1=0(Appe2≥2Appe2<2)
A, {G, D}, {G1, G2, D1,
D2}
6 Appe1≥1(Appe2≥2(App +NC)e2<2)
(App +NC)e1=0(Appe2≥2Appe2<2)
A, {G, D}, {G1, G3, D1,
D2}
7 Appe1≥1(Appe2≥2Appe2<2)
(App +NC)e1=0(Appe2≥2(App +NC)e2<2)
A, {G, D}, {G1, G2, D1,
D3}
8 Appe1≥1(Appe2≥2(App +NC)e2<2)
(App +NC)e1=0(Appe2≥2(App +NC)e2<2)
A, {G, D}, {G1, G3, D1,
D3}
9 Stringent App Appe1≥2Appe1<2 Appe1≥2(Appe2≥3Appe2<3)Appe1<2(Appe2≥3Appe2<3)B, {M, E}, {M1, M2, E1,
E2}
10 Appe1≥2(Appe2≥3(App +NC)e2<3)
Appe1=0(Appe2≥3Appe2<3)
B, {M, E}, {M1, M3, E1,
E2}
11 Appe1≥2(Appe2≥3Appe2<3)
Appe1<2(Appe2≥3(App +NC)e2<3)
B, {M, E}, {M1, M2, E1,
E3}
12 Appe1≥2(Appe2≥3(App +NC)e2<3)
Appe1<2(Appe2≥3(App +NC)e2<3)
B, {M, E}, {M1, M3, E1,
E3}
13 Appe1≥2(App +NC)e1<2 Appe1≥2(Appe2≥3Appe2<3)
(App +NC)e1<2(Appe2≥3Appe2<3)
B, {M, F}, {M1, M2, F1,
F2}
14 Appe1≥2(Appe2≥3(App +NC)e2<3)
(App +NC)e1<2(Appe2≥3Appe2<3)
B, {M, F}, {M1, M3, F1,
F2}
15 Appe1≥2(Appe2≥3Appe2<3)
(App +NC)e1<2(Appe2≥3(App +NC)e2<3)
B, {M, F}, {M1, M2, F1,
F3}
16 Appe1≥2(Appe2≥3(App +NC)e2<3)
(App +NC)e1<2(Appe2≥3(App +NC)e2<3)
B, {M, F}, {M1, M3, F1,
F3}
Fig. 3. The carbohydrate periodized menu planner suggested energy intake (kcal) and recipes corresponding to a particular meal in the menu planner, and messaging
functions (from left to right) on the mobile application (App).
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
5
meal timetable that gives guidance pertaining to the appropriate amount
of carbohydrate and energy intake per meal according to their training
loads (see the left-most screen in Fig. 3). Each meal in the menu planner
is labeled as “low”, “medium” or “high” carbohydrate to help easier
visualization and understanding of carbohydrate periodization. The
carbohydrate recommendations in the menu planner follow the carbo-
hydrate periodization framework by Impey et al. [3,5]. Participants may
also click on the meals to view a list of recipe suggestions with matching
carbohydrate and energy intake in kcal (see the center screen in Fig. 3).
Participants will use the App to generate their personalized carbohy-
drate periodized menu planner by inputting their weekly training
schedules. They may also edit their plans at any time, and the weekly
menu planner will be updated instantaneously. We anticipate the par-
ticipants to input their training schedules once a week and view their
menu planners regularly (e.g., daily). Other features include daily
educational and motivational infographics, videos, and text articles to
further support the menu planner. See Appendix Fig. A.1 for more
details.
For the observational phase, the nutritionists will help participants
set up the App via webinars, phone calls, or text messages after the
participants received the App, as required within the 6 weeks. Email
messages will additionally be sent twice per week to remind participants
to use the application.
For the pilot SMART, upon receiving the invitation to download the
App, participants will be given one webinar on how to set up and use the
App. If they were unable to attend, participants will be sent the webinar
video afterward to watch on their own. All participants in the SMART
study will additionally receive three reminder push notications per
week via the App.
2.2.2. Nutrition coaching (NC)
Participants who will be randomized to receive nutrition coaching
(NC) will have the “Coaching” feature on the App (see the right-most
screen in Fig. 3 and Appendix Fig. A.1) activated, and a qualied
sports nutritionist will start a conversation with the participants within
two days of being assigned nutrition coaching. For the usual nutrition
coaching in the observational phase, coaches will send ad-hoc messages
and arrange for phone calls, where suitable, with the participants
throughout the 6-week study period. The coaches will decide the fre-
quency and type of communication based on their experience. For the
pilot SMART, coaches will only communicate with participants during
the periods when participants will be given NC as described in Fig. 2.
Coaches will be allowed to initiate conversations up to 3 times per week
and give ad-hoc calls where suitable according to their expertise judg-
ment only during the eligible periods. Although the coaches may vary
the messaging and call frequencies based on their expert judgment, the
consensus will be 2–3 times messaging per week and arrange one phone
call per participant (where eligible in the pilot SMART phase). In the
event the participant is not engaging with the coach, the coach may
attempt to call the participant up to two times. Coaches will also be
advised to focus solely on nutrition advice.
2.3. Determining design parameters for pilot SMART phase
2.3.1. Tailoring variable: engagement rate
As described in Section 2.1.2, the weekly engagement rates (e1 and
e2)will be used as the tailoring variable to determine the participants’
response status (i.e., whether they will be re-randomized in Fig. 2). The
engagement rate is dened as the number of days at least one App usage
log event was recorded in 7 days (i.e., week 1 day 1 to day 7 and week 2
day 1 to day 7). The types of events that constitute an App usage log
event are “login”, “logout”, “open”, “close”, “pause”, and “resume”.
Engagement is chosen as the tailoring variable as it is relatively objective
compared to other self-reported measures, is a type of passive data, and
thus requires no additional action from the participants. This mitigates
the risk of non-compliance due to missing intermediate data. Given that
participants have little external nutrition support, and the construction
of a menu plan that follows carbohydrate periodization requires sub-
stantial expertise, we made the reasonable assumption that the
engagement on the App is a good proxy to the participants’ adherence to
carbohydrate periodization behavior.
Since there is no a priori information on a suitable threshold for the
engagement rate, part of the SMART involves exploring different
thresholds. Yan et al. [24] has recently shown via simulations that there
exists an optimal threshold, which can give the best overall results. It is
found that for the healthy population, the engagement with health apps
ranges from twice a day to less than once a month, of which about 30%
engaged a few times a week or less [25]. Since the App was designed
such that the participants make weekly plans, the minimum reasonable
engagement rate is thus once a week. Therefore, we proposed the
threshold ranges to be from 1 to 3 days/week. We will explore two pairs
of thresholds (for two time-points), dening the pairs as relaxed and
stringent response criteria (relaxed: e1 ≥1 day and e2 ≥2 days; strin-
gent: e1 ≥2 days and e2 ≥3 days, where e1 and e2 are the engagement
rates at end of week 1 and week 2 respectively). Note that by exploring
the different thresholds, we are also exploring when (in terms of App
engagement performance) the human nutrition coaching should be
given.
2.3.2. Randomization time-points
The time-points (end of week 1 and week 2) and the intervals were
intentionally made relatively short and close to the baseline for many
reasons. Firstly, it is notorious that health apps have low engagement
and high dropout rates [25]. A large-scale study on app-usage data re-
ported that on average, at least 65% of the users dropped the app within
the rst week [26]. Another industry report by Liftoff and AppsFlyer
[27] found that the retention rates at day 1, day 7, and day 30 was
20.2%. 8.5% and 4% respectively for health and tness apps. Secondly,
based on experts’ consensus and literature [28], earlier nutrition
coaching may potentially be more benecial for the participants. Thus,
any changes to the intervention (App or App +NC) should be given
promptly. Following that the participants are expected to plan their
meals weekly, we dene the randomization time-points to be one week
apart.
2.4. Ethical aspects
The study has been registered at ClinicalTrials.gov (NCT04487015)
and ethical approval was obtained from Liverpool John Moores Uni-
versity Research Ethics Committee.
2.5. Participants and recruitment
The study will recruit a total of at least 900 participants over 18 years
of age for both the observational and pilot SMART phase. Participants
must be either elite or amateur athletes taking part in regular training
(three or more sessions a week). They must own a smartphone and be
willing to provide informed consent. Participants will not be eligible if
they report having or previously have had an eating disorder or suffered
from disordered eating. Participants will be recruited through direct
contacts with gatekeepers, nutritionists, or owners of various sports and
training organizations in the UK.
For the observational phase, we expect to recruit n =500 partici-
pants with little to no existing nutrition coaching support to use the
mobile application and n =250 participants with regular existing
nutrition coaching support. The existing coaching support status is
determined by whether the participants’ organizations have access to a
nutritionist at least one day a week at their sporting organization. For
the pilot SMART, we aim to recruit n =150 participants. The numbers
were estimated from practical recruitment potential. We aim to recruit
as many as possible primarily because of challenges in (1) non-
compliance and attrition in longitudinal eHealth studies and (2)
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
6
novelty of the approach in sports nutrition and athletes. We will aim to
maximize retention while gathering as much information as possible.
The sample size for SMART was also referenced from calculating the
precision-based sample size [24] and through simulations of the number
of resulting participants in each subgroup. As viewed in Fig. 2, there are
18 different possible design pathways for an individual and hence 18
subgroups. Assuming equal response rates at the end of weeks 1 and 2,
the sample size required for a binary outcome with 18% precision is 134.
The same sample size guarantees an average of 99.3% probability of
having at least 2 participants in the smallest subgroups [29]. The sample
size required drops to 75 if it is sufcient to assume an 82.4% average
probability. All simulations and sample size calculations were done
using R 3.6.1 [30]. A table of sample sizes and probabilities may be
found in Appendix Table A.2.
2.6. Randomization procedure
In the observational phase, access to the App is predetermined by
existing organizational nutrition support for each participant. In the
pilot SMART phase, randomization will occur in a 1:1 ratio using block
randomization with the size of 4 at all time points (baseline, end of
weeks 1 and 2). The randomization algorithm will be generated using R
3.6.1 [30]. The randomized allocation at baseline, to either stringent or
relaxed response criteria, will be based on the return order of the
baseline survey. Participants with engagement rates (e1 and e2)not
meeting their respective response criteria (i.e., the non-responders) will
then be re-randomized to either App or App +NC. The allocation orders
for the two time points will be based on the timestamp order of account
creation on the app. All participants eligible for NC will be assigned to
one of the six recruited qualied nutritionists by simple randomization,
subject to availability and any conict of interest pre-declared by the
coaches. All assignments will be independent of the nutritionists. Given
the nature of the intervention, coaches and participants cannot be
masked.
2.7. Data collection and outcome measures
Data will be primarily collected from online surveys, mobile appli-
cations, and documentation from nutritionists.
2.7.1. Participant characteristics
Participant characteristics such as age, gender, education level, or-
ganization will be collected via baseline surveys. Participants will also
self-report the number of nutrition apps currently using or used before.
The personality traits of the participants will be measured using the
30-item BFI–2S questionnaire [31] in the baseline survey. The ques-
tionnaire starts with “I am someone who … " and participants use a
5-point Likert scale (1 =strongly disagree; 5 =strongly agree) to rate
each item. The OCEAN (openness, conscientiousness, extraversion,
agreeableness, neuroticism) personality traits score may then be
calculated.
The need for autonomy and control will be measured by an adapted
version of the health causality orientation scale (HCOS) by Smit and Bol
[32]. Five nutrition-related scenarios will be presented to the partici-
pants (e.g., “You are considering making changes to your diet. How
likely are you to … ?“). In each scenario, participants rate two action
descriptors, representing an autonomy orientation (e.g., “Decide for
yourself which type of changes you would like to make.“) and a control
(nutritionist) orientation (e.g. “Find a nutritionist who will tell you what
to do.“), on a 7-point Likert scale (1: strongly disagree; 7 =strongly
agree). The adapted version changed phrases to be specic for nutrition
and removed the control (peers) orientation that is applicable in
nutrition.
The eHealth literacy will also be measured at baseline, by an adapted
version of the 8-item eHealth literacy scale (eHEALS) [33]. The state-
ments are modied to specically refer to nutrition such as “I know how
to use the Internet to answer my nutrition questions.”
2.7.2. Primary outcomes
The primary outcomes are the mobile application and coach uptake
rates, retention and engagement rates, and success in attaining carbo-
hydrate periodization behavior.
The uptake of the mobile application is assessed by whether the
participants’ created an account in the mobile application. Nutrition
coaching uptake is assessed by whether the participants had replied to
the coaches’ messages in the App or had a nutrition coaching call with
coaches during the study periods.
The retention rate on the app is assessed by the number of days until
the participant dropped the app (i.e., day starting from account creation
to the day of last usage log event detected). The follow-up period will be
at week 7 and week 5 for the observational phase and pilot SMART
phase, respectively. The overall app engagement is assessed by the
number of days at least one usage log event was detected during the
study periods.
Carbohydrate periodization behavior is assessed at baseline and
follow-up at week 7 for the observational phase and week 5 for pilot
SMART via a 3-day self-reported dietary periodization questionnaire
(see Fig. 4). The periodization questionnaire was adapted from the
previous work of Heikura et al. [9]. The questionnaire was modied to
remove all questions not deemed relevant to the current study following
an expert consensus. In the periodization questionnaire, six questions
around carbohydrate intake are presented sequentially, participants
may skip questions depending on their response to the previous ques-
tion. The rst four questions are multiple-choice questions (see Fig. 4)
on whether participants deliberately vary their amount of food/calories
intake. The subsequent two questions are on the type of food varied and
when they are varied. Depending on their responses, participants can be
ranked as “1-not periodizing at all”, “2-periodizing energy/kcal only”,
“3-periodizing both energy/kcal and carbohydrates’’. Only a score of
“3” is considered a successful carbohydrate periodization behavior. The
amended questionnaire was trialed with eight athletes (independent of
the study participants) who also completed a self-reported three-day
food and training diary. Six independent sports nutritionists each
assigned a carbohydrate periodization status to each athlete based on
their food and training diary data and these results were compared to the
questionnaire results to evaluate its reliability. Following this trial, the
expert panel made minor amendments to the questionnaire before its use
in this study.
The study is considered feasible if 70% of the participants created an
account in the mobile application, 20% of these participants continue to
use the application after 7 days and observe that more participants with
the mobile application intervention achieve successful carbohydrate
periodization behavior than those in the control group. It should be
noted in the literature [26,27,34,35], retention patterns are generally
complex, and retention rates vary greatly (e.g., from an industry average
of 11%–65% for the topmost popular applications).
2.7.3. Secondary outcomes
The self-reported weight, goal, carbohydrate periodization self-
efcacy, and belief about consequences will also be collected at base-
line and follow-up via surveys.
Carbohydrate periodization self-efcacy will be assessed using a 3-
item measure adapted from the Self-efcacy for Eating Behaviors
Scale [36] and Dieting Self-Efcacy Scale (DIET-SE) [37]. The frame-
work follows the DIET-SE, where statements starting with “How con-
dent are you to …” are presented, each representing a general
scenario-based factor [37], relapse resistant factor [36], and a plan-
ning behavior factor [36]. Participants rate each statement on a 5-point
Likert scale (1 =not condent at all; 5 =very condent).
The beliefs about consequences of carbohydrate periodization will be
measured on a 3-item scale adapted from Thrasher et al.‘s [38] research
on response efcacy beliefs. Statements aim to capture the participants’
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
7
beliefs about their perceived impact of dietary periodization behaviors
on health and performance consequences. All statements start with
“How much do you think that adjusting the amount of energy (calories)
and carbohydrate you eat, according to your weekly training &
competition demands, will benet your … ", before continuing to
describe three areas: body composition, health, and performance. Par-
ticipants will rate each statement on a 5-point Likert scale (1 =not at all;
5 =extremely).
2.7.4. Data generated during the intervention
Mobile application (App) generated data, including usage log events
(as described in Section 2.3.1), messages, log of recorded weights, and
menu planners created, will be recorded. All messages will be encrypted.
Phone call and voice message events will also be documented by the
nutritionists.
2.8. Statistical analysis
All statistical analyses will be performed using R 3.6.1 statistical
software [30]. For both the observational and the pilot SMART phase,
descriptive statistics will be used to describe the participants’ de-
mographic characteristics, baseline primary and secondary outcome
measures (mean and standard deviations for continuous variables, and
frequency and percentages for categorical variables). As attrition and
non-compliance are expected to be prominent, analyses will be pre-
sented using both the intention-to-treat and per-protocol approach,
where applicable.
2.8.1. Observational phase analysis
To address the feasibility objective in this phase, we will highlight
the barriers and challenges in delivering a mobile application inter-
vention in the sports nutrition eld, including presenting the uptake
rates of the application and nutritionists, engagement and usage level of
the application, and potential differences in app interactions among
different participant characteristics. The improvement from baseline to
follow-up and comparison between mobile application and control
groups for self-reported behavioral outcome measures will be compared
using Chi-square tests for binary outcomes, pairwise t-tests for combined
scale scores, and Mann-Whitney U tests for individual items in the scales.
Logistic regressions adjusting for baseline characteristics will also be
presented. Logistic and linear regressions will also be used to adjust for
baseline characteristics.
2.8.2. Pilot SMART phase analysis
The constructed strategies in this three-stage SMART describe the
different scenarios under which the App or App +NC may be given at
each time-point, given the responses to initial and subsequent in-
terventions under different response criteria (i.e., Table 1) in 4 weeks.
We will rst address the suitability of the relaxed and stringent response
criteria by reporting the response rates at each time point, and the
resulting number of participants in the 16 embedded strategies and 18
pathways taken. Depending on the resulting sizes in each pathway,
strategy components after the second time point (end of week 2) may not
be sufcient for hypothesis testing. Hence, the weighted means of the
self-reported behavioral outcome measures for each strategy will pri-
marily be compared empirically. We will also examine the uptake rates
of the NC and the change in engagement level when NC is given. We will
also consider the design as a two-stage SMART by disregarding the
intervention given after week 2, thereby collapsing the 16 strategies into
four general strategies (one: 1–4, two: 5–8, three: 9–12, four:13–16 in
Fig. 4. The carbohydrate periodization behavior classication is based on responses to questions in the periodization questionnaire at baseline and follow-
up surveys.
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
8
Table 1). The weighted means for the four general strategies will then be
compared. Further analyses using logistic and ordinal Q-learning models
[29] to identify the optimal decisions (whether to give NC) at each stage
will also be considered.
2.9. Novel aspects of the design
The benets of carbohydrate periodization are well established in
sports nutrition. However, adherence and adept knowledge to use the
framework poses practical challenges for both athletes and practitioners.
There is limited research in sports nutrition on how a digital environ-
ment could help improve carbohydrate periodization behaviors in ath-
letes. Hence, the intervention is novel in being the rst mobile
application to provide a personalized menu planner to athletes that
specically target improvements in carbohydrate periodization dietary
behaviors. The three-stage pilot SMART although not novel in concept,
is uncommon in practice. It is however tting in a digital environment
where decisions must be made at a higher frequency to reduce partici-
pant disengagement.
3. Results
The study started on September 14, 2020 and was completed on
January 4, 2021. We are currently in the process of consolidating and
cleaning the data.
4. Discussion
The ndings of this study will contribute to evidence regarding the
use of mobile applications to inuence athletes’ dietary behaviors in the
eld of sports nutrition. Specically, this research will provide insight
into the use of a menu planner mobile application and its impact on
dietary periodization behaviors in athletes. It will facilitate a greater
understanding of how technology may be used to support sports nutri-
tion service provision at scale, while at the same time providing a
personalized and continuous experience for athletes.
Additionally, the outcomes of the pilot SMART may be used to
inform practitioners on how best to structure the frequency and timing
of their communication with athletes when using mobile application
supportive solutions to their practice. These ndings may help optimize
the allocation of practitioner time as well as highlight scenarios where
the use of a mobile application alone may be sufcient to help athletes
self-manage their dietary periodization behaviors. This may help
improve the allocation of resources and potentially also the cost-
effectiveness of suggested solutions in practice.
Data generated from the exploratory aims of this research will pro-
vide insights into the psychological proles of elite athletes and the
potential impacts, if any, on mobile app uptake and usage. These au-
tonomy and personality-related ndings may provide sports nutrition
practitioners with useful insights into how best to collaborate with
athletes as part of their service provision. Conversely, the barriers and
challenges to mobile app uptake and usage identied in this research
may provide learnings and guidelines for future similar studies in sports
nutrition.
It is worth noting that there are multiple limitations to the current
study to highlight. Firstly, the App usage log events may not be the best
measure for app engagement (used to construct the tailoring variable in
the pilot SMART). In an ideal scenario, a better gauge of an effective App
engagement would be to look at their daily App usage duration (by using
the usage log events to calculate e.g., time from open to close), and the
user journeys on the App (clicks, time spent on each screen, etc).
However, this approach poses technical challenges to the current App.
There is currently no tracking functionality within the App to capture
these user journeys; it is a common issue (veried by initial testing of the
app) that log events may be missing at random or systematically, and
there may be random time lags between the actual and recorded events.
On the other hand, given how the App works (as described in Section
2.2.1), dening engagement as having at least one usage log event for
the day may be sufcient (e.g., viewing the recommended carbohydrate
levels for the day, see left-most of Fig. 3). Undeniably, there are non-
engagement-related events that are the result of participants directly
swiping the App away or accidently clicking on the App. However, such
cases are difcult to identify, unless complete and precise events data
are collected.
Secondly, given that the carbohydrate periodization questionnaire is
an adaptation of the dietary periodization questionnaire by Heikura
et al. [9], it could benet from more thorough validation.
The questionnaire could have beneted from trialing on a larger
group of athletes (currently tested on 8 athletes only) comparing their
three-day food and training diaries to the questionnaire results.
Thirdly, there is a high level of uncertainty as to the attrition and
non-compliance rates of participants, hence sample sizes cannot be
guaranteed. Finally, the mobile application may contain major bugs that
could affect usability or participant interest in the application thus
diluting its actual effect. Usage of the application may also be inuenced
by an athlete’s competition seasonality, e.g., in-season vs off-season.
Despite these limitations, the current study will provide sufcient
foundational evidence on delivering mobile application dietary
behavior interventions to athletes and selecting plausible nutrition
coaching strategies around the mobile application. Ongoing develop-
ment of the mobile application will also help overcome potential issues
on usability and functionality of the platform, thus potentially
improving its overall effectiveness in targeting changing dietary
behavior and other relevant outcomes.
Funding support
This work was supported by Applied Behaviour Systems Ltd. Ms. Yan
is also funded by Duke-NUS Medical School as part of her Ph.D. Dr.
Chakraborty would like to acknowledge support by Khoo Bridge Fund-
ing Award from the Duke-NUS Medical School (grant number: Duke-
NUS-KBrFA/2021/0040).
Declaration of interests
☐ The authors declare that they have no known competing nancial
interests or personal relationships that could have appeared to inuence
the work reported in this paper.
☒The authors declare the following nancial interests/personal re-
lationships which may be considered as potential competing interests:
The app was developed by Applied Behavior Systems Ltd. Ms Yan, Mr
Dunne, Dr Impey, Dr Lefevre, Dr Mazorra are co-founders of the com-
pany. Dr Cunniffe is also an advisor of the company. The rest of the
authors have no conicts of interest in the authorship or publication of
this study.
Declaration of competing interest
The app was developed by Applied Behavior Systems Ltd. Ms. Yan,
Mr. Dunne, Dr. Impey, Dr. Lefevre, Dr. Mazorra are co-founders of the
company. Dr. Cunniffe is also an advisor of the company. The rest of the
authors have no conicts of interest in the authorship or publication of
this study.
Acknowledgments
We are grateful to DataSpartan Ltd for providing ongoing technical
support for this project.
Appendix A. Supplementary data
Supplementary data to this article can be found online at https://doi.
X. Yan et al.
Contemporary Clinical Trials Communications 26 (2022) 100899
9
org/10.1016/j.conctc.2022.100899.
References
[1] J. Bergstr¨
om, et al., Diet, muscle glycogen and physical performance, Acta Physiol.
Scand. 71 (2-3) (1967) 140–150.
[2] L.J. van Loon, et al., The effects of increasing exercise intensity on muscle fuel
utilisation in humans, J. Physiol. 536 (1) (2001) 295–304.
[3] S.G. Impey, et al., Fuel for the work required: a practical approach to
amalgamating train-low paradigms for endurance athletes, Physiological Reports 4
(10) (2016) e12803.
[4] L.-A. Marquet, et al., Periodization of carbohydrate intake: short-term effect on
performance, Nutrients 8 (12) (2016) 755.
[5] S.G. Impey, et al., Fuel for the work required: a theoretical framework for
carbohydrate periodization and the glycogen threshold hypothesis, Sports Med. 48
(5) (2018) 1031–1048.
[6] T. Stellingwerff, J.P. Morton, L.M. Burke, A framework for periodized nutrition for
athletics, Int. J. Sport Nutr. Exerc. Metabol. 29 (2) (2019) 141–151.
[7] A.E. Jeukendrup, Periodized nutrition for athletes, Sports Med. 47 (1) (2017)
51–63.
[8] L.M. Burke, J.A. Hawley, Swifter, higher, stronger: what’s on the menu? Science
362 (6416) (2018) 781–787.
[9] I.A. Heikura, T. Stellingwerff, L.M. Burke, Self-reported periodization of nutrition
in elite female and male runners and race walkers, Front. Physiol. 9 (2018) 1732.
[10] I.A. Heikura, et al., Dietary microperiodization in elite female and male runners
and race walkers during a block of high intensity precompetition training, Int. J.
Sport Nutr. Exerc. Metabol. 27 (4) (2017) 297–304.
[11] S. Heaney, et al., Nutrition knowledge in athletes: a systematic review, Int. J. Sport
Nutr. Exerc. Metabol. 21 (3) (2011) 248–261.
[12] M.R. Bentley, et al., Athlete perspectives on the enablers and barriers to nutritional
adherence in high-performance sport, Psychol. Sport Exerc. 52 (2021) 101831.
[13] D.M. Dunne, et al., Performance Nutrition in the digital era–An exploratory study
into the use of social media by sports nutritionists, J. Sports Sci. 37 (21) (2019)
2467–2474.
[14] K.L. Birkenhead, G. Slater, A review of factors inuencing athletes’ food choices,
Sports Med. 45 (11) (2015) 1511–1522.
[15] K.J. Thomas Craig, et al., Systematic review of context-aware digital behavior
change interventions to improve health, Translational Behavioral Medicine 11 (5)
(2021) 1037–1048.
[16] B. Spring, et al., Multiple behavior changes in diet and activity: a randomized
controlled trial using mobile technology, Arch. Intern. Med. 172 (10) (2012)
789–796.
[17] M. Nour, J. Chen, M. Allman-Farinelli, Young adults’ engagement with a self-
monitoring app for vegetable intake and the impact of social media and
gamication: feasibility study, JMIR Formative Research 3 (2) (2019) e13324.
[18] Y.P. Duan, et al., Evaluation of a web-based intervention for multiple health
behavior changes in patients with coronary heart disease in home-based
rehabilitation: pilot randomized controlled trial, J. Med. Internet Res. 20 (11)
(2018) e12052.
[19] J.S. Naimark, Z. Madar, D.R. Shahar, The impact of a Web-based app (eBalance) in
promoting healthy lifestyles: randomized controlled trial, J. Med. Internet Res. 17
(3) (2015) e3682.
[20] S.A. Murphy, An experimental design for the development of adaptive treatment
strategies, Stat. Med. 24 (10) (2005) 1455–1481.
[21] P.W. Lavori, R. Dawson, Introduction to dynamic treatment strategies and
sequential multiple assignment randomization, Clin. Trials 11 (4) (2014) 393–399.
[22] S. Michie, M.M. Van Stralen, R. West, The behaviour change wheel: a new method
for characterising and designing behaviour change interventions, Implement. Sci. 6
(1) (2011) 1–12.
[23] L. Atkins, S. Michie, Changing eating behaviour: W hat can we learn from
behavioural science? Nutr. Bull. 38 (1) (2013) 30–35.
[24] X. Yan, et al., Sequential Multiple Assignment Randomized Trial (SMART) to
identify optimal sequences of telemedicine interventions for improving initiation
of insulin therapy: a simulation study, BMC Med. Res. Methodol. 21 (1) (2021)
1–11.
[25] R. Robbins, et al., Health app use among US mobile phone users: analysis of trends
by chronic disease status, JMIR mHealth and uHealth 5 (12) (2017) e7832.
[26] S. Sigg, et al., Exploiting usage to predict instantaneous app popularity: trend
lters and retention rates, ACM Trans. Web 13 (2) (2019) 1–25.
[27] Liftoff and AppsFlyer, Retention rate on day 1 and day 30 of mobile app installs
worldwide as of August 2020, by category. In Statista - the Statistics Portal
[Website] 2020 [cited 2021 October 25]; Available from: https://www.statista.
com/statistics/259329/ios-and-android-app-user-retention-rate/.
[28] J. Cuseo, Academic-support Strategies for Promoting Student Retention &
Achievement during the First-Year of College and beyond, 2015.
[29] I. Nahum-Shani, et al., Q-learning: a data analysis method for constructing
adaptive interventions, Psychol. Methods 17 (4) (2012) 478.
[30] R Core Team, R, A Language and Environment for Statistical Computing, R
Foundation for Statistical Computing, Vienna, Austria, 2020.
[31] B. Rammstedt, et al., Validation of the short and extra-short forms of the Big Five
Inventory-2 (BFI-2) and their German adaptations, Eur. J. Psychol. Assess. (2018).
[32] E.S. Smit, N. Bol, From self-reliers to expert-dependents: identifying classes based
on health-related need for autonomy and need for external control among mobile
users, Media Psychol. 23 (3) (2020) 391–414.
[33] C.D. Norman, H.A. Skinner, eHealth literacy: essential skills for consumer health in
a networked world, J. Med. Internet Res. 8 (2) (2006) e9.
[34] A. Pratap, et al., Indicators of retention in remote digital health studies: a cross-
study evaluation of 100,000 participants, NPJ digital medicine 3 (1) (2020) 1–10.
[35] B. Magazine, Spring 2016 mobile customer retention report [Website] 2016 [cited
2021 December 03]; Available from: https://www.braze.com/blog/app-customer-
retention-spring-2016-report/.
[36] J.F. Sallis, et al., The development of self-efcacy scales for healthrelated diet and
exercise behaviors, Health Educ. Res. 3 (3) (1988) 283–292.
[37] C. Stich, B. Kn¨
auper, A. Tint, A scenario-based dieting self-efcacy scale: the DIET-
SE, Assessment 16 (1) (2009) 16–30.
[38] J.F. Thrasher, et al., Inuences of self-efcacy, response efcacy, and reactance on
responses to cigarette health warnings: a longitudinal study of adult smokers in
Australia and Canada, Health Commun. 31 (12) (2016) 1517–1526.
X. Yan et al.
