Adaptive e-learning to improve dietary
behaviour: a systematic review and cost-
J Harris,1 L Felix,1 A Miners,2 E Murray,3 S Michie,4
E Ferguson,1 C Free,1 K Lock,2 J Landon5 and
1Faculty of Epidemiology and Population Health, London School of Hygiene
and Tropical Medicine, London, UK
2Faculty of Public Health and Policy, London School of Hygiene and Tropical
Medicine, London, UK
3Research Department of Primary Care and Population Health, University
College London, London, UK
4Research Department of Clinical, Educational & Health Psychology,
University College London, London, UK
5National Heart Forum, London, UK
Health Technology Assessment 2011; Vol. 15: No. 37
Health Technology Assessment
NIHR HTA programme
Adaptive e-learning to improve dietary behaviour
iiExecutive summary: Adaptive e-learning to improve dietary behaviour
The composition of habitual diets is associated with adverse or protective effects on health.
Consequently, UK public health policy strongly advocates dietary change for the improvement of
population health and emphasises the importance of individual empowerment to improve health.
A new and evolving area in the promotion of dietary behavioural change is ‘e-learning’, the use
of interactive electronic media to facilitate teaching and learning on a range of issues including
health. The high level of accessibility, combined with emerging advances in computer processing
power, data transmission and data storage, makes interactive e-learning a potentially powerful
and cost-effective medium for improving dietary behaviour. E-learning also has a number
of potential advantages compared with traditional approaches for the promotion of dietary
behaviour change, such as the possibility of tailoring to individual circumstances; translating
complex information through video, graphics and audio systems; and potential cost savings on
face-to-face interventions involving health-care practitioners. Some evidence that individualised,
tailored e-learning approaches are more effective than traditional non-tailored interventions has
given them a promising lead in health education.
The aims of this systematic review were to assess the effectiveness and cost-effectiveness of
adaptive e-learning for improving dietary behaviours. The specific objectives were to:
describe the range of e-learning technologies in use for promoting dietary
evaluate the effectiveness of interactive e-learning in terms of improvement in dietary
behaviour and clinical outcomes
analyse the e-learning interventions in order to determine the components contributing to
effects of e-learning interventions for dietary behaviour change
investigate potential explanations of dietary behaviour change and mechanisms of action
evaluate cost-effectiveness compared with current standard interventions, and estimate the
likely budget impact in England and Wales.
We included all randomised controlled trials (RCTs) of effectiveness of e-learning in adolescents
or adults to promote dietary behavioural change and all clinical conditions in which dietary
advice plays a major part in case management. Interventions were included if they were
interactive computer software programs that tailored output according to user input (including
interventions where users enter personal data, or make choices about information, that alter
pathways within programs to produce tailored material and feedback that is personally relevant).
Primary outcomes were measures of dietary behaviours, including estimated intakes or changes
in intake of energy, nutrients, dietary fibre, foods or food groups. Secondary outcome measures
iii Health Technology Assessment 2011; Vol. 15: No. 37 (Executive summary)
were clinical outcomes that would be expected to respond to changes in dietary behaviours (e.g.
anthropometric status and blood biochemistry). We also sought data on the costs of providing
the intervention, any unintended adverse consequences of the interventions, process outcomes
(e.g. usage) and data relating to potential cognitive and emotional mediators of dietary behaviour.
Searching, screening and data extraction
Eligible studies were identified by searches of 11 electronic bibliographic databases, trial registers
for ongoing and recently completed trials, inspection of the reference lists of all included studies
and previously published reviews, and by contact with authors of included studies. There were no
restrictions by language. Searches covered the period January 1990 to November 2009.
Two review authors independently examined the titles, abstracts and keywords of electronic
records according to the eligibility criteria above. The results of this initial screening were cross-
referenced between the two review authors and full-text records obtained for all potentially
relevant reports of trials. These potentially eligible trials went through a secondary screening by
each reviewer using a screening form based on the eligibility criteria for final inclusion in the
systematic review, with disagreements resolved by discussion with a third author.
Two review authors extracted relevant data into a Microsoft Access 2007 database (Microsoft
Corporation, Redmond, WA, USA) specifically designed for the review. Two measures of
methodological quality were used in the review: the Cochrane Collaboration’s risk of bias
assessment and the Effective Public Health Practice Project (EPHPP) quality assessment. The
dietary assessment tools and techniques used to estimate dietary behaviour were critically
examined in terms of quality.
Analysis of effectiveness
For studies that reported the same outcome, we pooled the results using a random effects model,
with weighted mean differences (WMDs), and calculated 95% confidence intervals (CIs) and
two-sided p-values for each outcome. When outcomes were assessed more than once during
follow-up, the final assessment was used in analysis. We assessed evidence for selection bias using
Egger’s test for small study effects. Heterogeneity among the trial results was assessed using both
a chi-squared test and the I2 statistic (we considered I2 > 50% to reflect ‘substantial heterogeneity’).
We conducted sensitivity analyses to investigate possible sources of heterogeneity including study
quality and sociodemographic factors that could act as effect modifiers. Causes of heterogeneity
and subgroup effects were assessed using random effects meta-analysis. This was implemented in
Stata (StataCorp LP, College Station, TX, USA) using the ‘metareg’ command and including trial
characteristics as covariates. All statistical analysis was conducted using Stata statistical software
The intention was to estimate cost-effectiveness in two ways: through a systematic literature
review and by building a de novo decision model to assess the cost-effectiveness of a ‘generic’
e-learning device compared with dietary advice delivered by a health-care professional. However,
no perfectly fitting published economic evaluations were identified; thus, the results from the
literature review were instead primarily used to inform the model design. The key assumption
within the model was that the interventions were designed to promote weight loss in already
obese people with a body mass index (BMI) of ≥ 30 kg/m2. The evaluation was performed from a
UK NHS cost perspective and outcomes were expressed as quality-adjusted life-years (QALYs).
The estimate of relative treatment effect was derived from the systematic review of effectiveness,
and the model was based on discrete event simulation techniques referred to as the ‘E-Learning
Economic Evaluation Model’ (E-LEEM).
iv Executive summary: Adaptive e-learning to improve dietary behaviour
A total of 36,379 titles were initially identified by the electronic searches, of which 2977 were
duplicates and were removed. The remaining 33,402 records were screened and 33,129 records
were excluded because they did not meet the inclusion criteria. Full-text reports were sought for
273 potentially eligible studies, of which 233 studies were excluded after inspection of the full
report. Three eligible studies were identified through searching reference lists of the included
studies, yielding a total of 43 studies for inclusion in the review.
Description of included studies
Of the 43 included studies, one was a crossover trial and two were cluster-randomised trials,
whereas the rest were parallel-group RCTs. The majority of interventions evaluated sought
to reduce fat intake (28 interventions) and/or to increase fruit and vegetable intake (21
interventions). Other interventions sought to increase fibre intake, reduce overall energy intake,
reduce or maintain weight, or reduce or maintain BMI.
All e-learning interventions were delivered in high-income countries: the majority were delivered
in the USA (29 interventions), the Netherlands (five studies) and Belgium (three studies).
Many interventions were offered over the internet or via a mobile device and without a specific
setting. Of those that were delivered in a specific setting, nine were designed to be delivered in
the home, eight in the workplace, three in community centres, four in schools/colleges and two
in supermarkets. Many study reports did not provide information on the intended intensity or
frequency of use; of those that did, intended intensity of interventions varied between a single
session to weekly interaction with the intervention over 1 year.
A majority of interventions involved the presentation of nutrition, health and lifestyle
information (31 studies), the entering of food consumption data (28 studies), and the
presentation of personalised feedback on food and nutrient consumption (21 studies). Other
interventions focused on the setting of goals and providing feedback on attainment of these goals
(15 studies). The most commonly used behavioural change techniques reported were goal setting
(behaviour) (14 studies); provide feedback on performance (14 studies); provide information
on consequences of behaviour in general (14 studies) or to the individual (11 studies); barrier
identification/problem solving (13 studies); prompt self-monitoring of behaviour (12 studies);
provide instruction on how to perform the behaviour (12 studies); prompt review of behavioural
goals (11 studies); and plan social support/social change (10 studies).
Of the 23 studies providing details of inclusion criteria, 11 offered the intervention only to those
with BMI > 25 kg/m2; 10 required participants to be generally healthy and free of diagnosed
disease; one trialled the intervention in patients with diabetes; and one required participants
to have at least one diagnosed risk factor for cardiovascular disease. Three studies included
adolescents aged < 18 years. Of studies targeting adults, the majority included participants with
an average age of 40–49 years. One study included adults with an average age > 60 years.
Twenty-two studies measured total fat intake and eight studies measured saturated fat intake
(either as grams consumed or as percentage of total energy consumed). Twenty-two studies
assessed servings of fruit and/or vegetables consumed per day. Other food and nutrient outcomes
measured included energy intake (nine studies) and fibre intake (five studies). In terms of
clinical outcomes, 11 studies measured BMI, 14 studies measured weight or weight loss, four
studies measured cholesterol, four studies assessed triglycerides and three studies assessed blood
pressure. Four studies concentrated on the avoidance of eating disorders and used scales of eating
disorder risk alongside measures of BMI and/or weight.
v Health Technology Assessment 2011; Vol. 15: No. 37 (Executive summary)
The Cochrane assessment was at ‘low risk of bias’ in 27 studies for methods of sequence
generation; 18 studies for methods of allocation concealment; 15 studies for methods of blinding;
22 studies for addressing incomplete outcome data; 11 studies for not demonstrating selective
outcome reporting; and 26 studies for providing a conflict of interest statement. The EPHPP
assessment was ‘strong’ in one study for selection bias criteria; 43 studies for study design; 36
studies for adjustment for confounders; five studies for criteria on blinding; 21 studies for data
collection methods; and 24 studies for attrition criteria. Thirteen studies were rated overall as
‘moderate’ and 30 studies were rated overall as ‘weak’; none was rated ‘strong’ overall.
Studies using 24-hour recall (two studies), or a combination of two or more dietary assessment
methods (11 studies), especially two independent dietary assessment methods (two studies),
had the strongest dietary assessment methods for measuring dietary change. Those studies that
used a prospective dietary assessment method (e.g. multiple day diet record) alone are valid for
measuring dietary change, as long as a compliance bias did not result in an overestimation of the
intervention effect and dietary practices were not simplified to reduce respondent burden. The
studies with the weakest dietary assessment methods were those using only shopping receipts
(two studies), a screening Food Frequency Questionnaire (three studies) or a one- or two-item
food group question (five studies). Most studies did not adequately describe their dietary
assessment techniques and seven questionnaires were not validated for the target population or
nutrient of interest.
Analysis of effectiveness
There was substantial heterogeneity in the estimates of effect of e-learning interventions on many
reported outcomes; however, there was relatively little evidence for heterogeneity in the estimates
of effect on total fat intake per day, total energy intake per day and BMI. When studies reporting
the same outcomes were pooled in a random effects meta-analysis, e-learning interventions
were associated with a WMD of 0.24 servings (95% CI 0.04 servings to 0.44 servings; p = 0.019)
of fruit and vegetables per day (heterogeneity p < 0.001 and I2 = 83%); WMD of –0.78 g (95% CI
–2.5 g to 0.95 g) of total fat consumed (heterogeneity p = 0.18 and I2 = 28%); WMD –0.24 g (95%
CI –1.44 g to 0.96 g; p = 0.7) of saturated fat intake (heterogeneity p = 0.001 and I2 = 78%); WMD
of –1.4% (95% CI –2.5% to –0.3%; p = 0.012) of total energy consumed from fat (heterogeneity
p < 0.001 and I2 = 77%); WMD of 1.45 g (95% CI –0.02 g to 2.92 g; p = 0.053) of dietary fibre per
day (heterogeneity p = 0.11 and I2 = 60%); WMD of 4 kcal (95% CI –85 kcal to 93 kcal; p = 0.93) of
daily energy intake (heterogeneity p = 0.33 and I2 = 13%); WMD of –0.1 kg/m2 (95% CI –0.7 kg/m2
to 0.4 kg/m2; p = 0.69) in BMI (heterogeneity p = 0.92 and I2 = 0%); and WMD of 0.6 kg (95% CI
–3.5 kg to 4.6 kg; p = 0.78) of weight (heterogeneity p < 0.001 and I2 = 86%).
The literature review did not reveal any published economic evaluations of e-learning devices
that were based purely on imparting nutritional advice. One published UK study that evaluated
a device in conjunction with physical exercise suggested that it was not cost-effective compared
with standard methods of providing advice. The base-case results from the E-LEEM model
suggested that the incremental cost-effectiveness ratio was approximately £60,000 per QALY. The
results were generally robust to most alternative assumptions, except the initial fixed cost of the
device. In the base case this cost was assumed to be £854 per person. When the cost was assumed
to be £0, the e-learning device was less costly and more effective than the alternative. Expected
value of perfect information (EVPI) analysis showed that although the individual-level EVPI was
arguably negligible, the population-level value was between £37M and £170M at a willingness to
pay of £20,000–30,000 per additional QALY.
viExecutive summary: Adaptive e-learning to improve dietary behaviour
Explanations of effectiveness
We analysed the data with a view to determining reasons for the observed variability in
effectiveness of e-learning interventions. We considered that potential reasons could include
mode of delivery
‘dose’ of intervention
Interventions trialled were so heterogeneous that no firm conclusions could be reached.
More mechanistic research is required to address these questions, including whether or not
adaptive e-learning tools are better suited for disease management than for health promotion;
understanding the psychological mechanisms of action of interventions; and whether or not there
is a relationship between ‘dose’ of the intervention and effectiveness. Our statistical assessment of
the evidence suggests that trials with lower methodological quality may overestimate effects.
Can e-learning interventions change dietary behaviour?
There are many factors affecting what foods people eat and why, and an intervention targeted
at individual behaviour change can address only a selection of these. E-learning has not yet
proved itself to be more effective or cost-effective than other behaviour change approaches at an
individual level for improving diet, or for preventing or reducing overweight or obesity. Nor is
there any research comparing e-learning approaches with population-level approaches to tackling
dietary change or reducing obesity.
Are e-learning interventions cost-effective to the NHS?
The results broadly suggest that the e-learning devices are not cost-effective at conventional levels
of £20,000–30,000 per QALY gained and the probability that they are cost-effective was > 25%
at these threshold levels, a result that is perhaps not unsurprising given the relatively modest
effects derived from the systematic review of effectiveness. However, the results were sensitive to
the assumptions regarding the initial fixed cost of the devices; lowering this value dramatically
increased the cost-effectiveness of the devices.
What is the potential population health impact of e-learning interventions?
Fruit and vegetables
Dietary recommendations suggest five servings of fruit and vegetables per person per day;
currently adults in the UK (aged 19–65 years) are eating on average 4.4 servings, so an increase of
a quarter of a serving would still not raise the average intake to meet the guidelines.
Dietary recommendations suggest that > 35% of calories consumed should come from total fat,
and > 11% of calories from saturated fats. Currently, UK adults consume an average of 34–36%
of energy from total fat and 12.8% of energy from saturated fat, so a reduction of 1% would not
facilitate achievement of guideline targets.
vii Health Technology Assessment 2011; Vol. 15: No. 37 (Executive summary)
Recommendations suggest an intake of 18 g of dietary fibre per day, with current intake in
UK adults at around 14 g. An increase of 1.5 g would therefore not facilitate achievement of
Dietary behaviours are likely to be heavily influenced by macro factors at the environmental,
organisational, population and sociocultural levels. These wider determinants of dietary
behaviour are unlikely to be changed by individually targeted interventions such as e-learning;
the results of this review seem to reflect this, with little evidence of effect found.
Implications for health care
The current clinical and economic evidence base suggests that e-learning devices designed to
promote dietary behaviour change will not produce clinically significant changes in dietary
behaviour and are at least as expensive as other individual behaviour change interventions.
Implications for research
Although the return on investment (in terms of the expected value of the reduction in decision
uncertainty) from further clinical trials of individual e-learning interventions is expected to be
high, we believe that further such clinical trials should not be undertaken until theoretically
informed work, which addresses the question of which characteristics of the target population,
target behaviour, content and delivery of the intervention are likely to lead to positive results,
The National Institute for Health Research Health Technology Assessment programme.
Harris J, Felix L, Miners A, Murray E, Michie S, Ferguson E, et al. Adaptive e-learning to improve
dietary behaviour: a systematic review and cost-effectiveness analysis. Health Technol Assess
NIHR Health Technology Assessment programme
The Health Technology Assessment (HTA) programme, part of the National Institute for Health Research (NIHR), was
set up in 1993. It produces high-quality research information on the effectiveness, costs and broader impact of health
technologies for those who use, manage and provide care in the NHS. ‘Health technologies’ are broadly defined as all
interventions used to promote health, prevent and treat disease, and improve rehabilitation and long-term care.
The research findings from the HTA programme directly influence decision-making bodies such as the National
Institute for Health and Clinical Excellence (NICE) and the National Screening Committee (NSC). HTA findings also
help to improve the quality of clinical practice in the NHS indirectly in that they form a key component of the ‘National
The HTA programme is needs led in that it fills gaps in the evidence needed by the NHS. There are three routes to the
start of projects.
First is the commissioned route. Suggestions for research are actively sought from people working in the NHS, from
the public and consumer groups and from professional bodies such as royal colleges and NHS trusts. These suggestions
are carefully prioritised by panels of independent experts (including NHS service users). The HTA programme then
commissions the research by competitive tender.
Second, the HTA programme provides grants for clinical trials for researchers who identify research questions. These
are assessed for importance to patients and the NHS, and scientific rigour.
Third, through its Technology Assessment Report (TAR) call-off contract, the HTA programme commissions
bespoke reports, principally for NICE, but also for other policy-makers. TARs bring together evidence on the value of
Some HTA research projects, including TARs, may take only months, others need several years. They can cost from as
little as £40,000 to over £1 million, and may involve synthesising existing evidence, undertaking a trial, or other research
collecting new data to answer a research problem.
The final reports from HTA projects are peer reviewed by a number of independent expert referees before publication in
the widely read journal series Health Technology Assessment.
Criteria for inclusion in the HTA journal series
Reports are published in the HTA journal series if (1) they have resulted from work for the HTA programme, and
(2) they are of a sufficiently high scientific quality as assessed by the referees and editors.
Reviews in Health Technology Assessment are termed ‘systematic’ when the account of the search, appraisal and
synthesis methods (to minimise biases and random errors) would, in theory, permit the replication of the review
The research reported in this issue of the journal was commissioned by the HTA programme as project number
08/57/02. The contractual start date was in April 2009. The draft report began editorial review in October 2010 and
was accepted for publication in January 2011. As the funder, by devising a commissioning brief, the HTA programme
specified the research question and study design. The authors have been wholly responsible for all data collection,
analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the
accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft
document. However, they do not accept liability for damages or losses arising from material published in this report.
The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the
Department of Health.
Editor-in-Chief:Professor Tom Walley CBE
Dr Martin Ashton-Key, Professor Aileen Clarke, Dr Tom Marshall, Professor John Powell,
Dr Rob Riemsma and Professor Ken Stein
Associate Editor:Dr Peter Davidson
ISSN 1366-5278 (Print)
ISSN 2046-4924 (Online)
ISSN 2046-4932 (DVD)
© Queen’s Printer and Controller of HMSO 2011. This work was produced by Harris et al. under the terms of a
commissioning contract issued by the Secretary of State for Health.
This journal is a member of and subscribes to the principles of the Committee on Publication Ethics (COPE) (http://www.
This journal may be freely reproduced for the purposes of private research and study and may be included in professional
journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising.
Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House,
University of Southampton Science Park, Southampton SO16 7NS, UK.
Published by Prepress Projects Ltd, Perth, Scotland (www.prepress-projects.co.uk), on behalf of NETSCC, HTA.
Printed on acid-free paper in the UK by the Charlesworth Group.