Systematic review and evaluation of web-accessible tools for management of diabetes and related cardiovascular risk factors by patients and healthcare providers

ArticleinJournal of the American Medical Informatics Association 19(4):514-22 · January 2012with81 Reads
Impact Factor: 3.50 · DOI: 10.1136/amiajnl-2011-000307 · Source: PubMed
Abstract

To identify and evaluate the effectiveness, clinical usefulness, sustainability, and usability of web-compatible diabetes-related tools. Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, world wide web. Studies were included if they described an electronic audiovisual tool used as a means to educate patients, care givers, or clinicians about diabetes management and assessed a psychological, behavioral, or clinical outcome. Study abstraction and evaluation for clinical usefulness, sustainability, and usability were performed by two independent reviewers. Of 12,616 citations and 1541 full-text articles reviewed, 57 studies met inclusion criteria. Forty studies used experimental designs (25 randomized controlled trials, one controlled clinical trial, 14 before-after studies), and 17 used observational designs. Methodological quality and ratings for clinical usefulness and sustainability were variable, and there was a high prevalence of usability errors. Tools showed moderate but inconsistent effects on a variety of psychological and clinical outcomes including HbA1c and weight. Meta-regression of adequately reported studies (12 studies, 2731 participants) demonstrated that, although the interventions studied resulted in positive outcomes, this was not moderated by clinical usefulness nor usability. This review is limited by the number of accessible tools, exclusion of tools for mobile devices, study quality, and the use of non-validated scales. Few tools were identified that met our criteria for effectiveness, usefulness, sustainability, and usability. Priority areas include identifying strategies to minimize website attrition and enabling patients and clinicians to make informed decisions about website choice by encouraging reporting of website quality indicators.

Figures

Full-text

Available from: Michael Scott Orr
Systematic review and evaluation of web-accessible
tools for management of diabetes and related
cardiovascular risk factors by patients and
healthcare providers
Catherine H Yu,
1,2
Robinder Bahniwal,
1
Andreas Laupacis,
1,2
Eman Leung,
1
Michael S Orr,
3
Sharon E Straus
1,2
ABSTRACT
Objective To identify and evaluate the effectiveness,
clinical usefulness, sustainability, and usability of
web-compatible diabetes-related tools.
Data sources Medline, EMBASE, CINAHL, Cochrane
Central Register of Controlled Trials, world wide web.
Study selection Studies were included if they described
an electronic audiovisual tool used as a means to
educate patients, care givers, or clinicians about
diabetes management and assessed a psychological,
behavioral, or clinical outcome.
Data extraction Study abstraction and evaluation for
clinical usefulness, sustainability, and usability were
performed by two independent reviewers.
Results Of 12616 citations and 1541 full-text articles
reviewed, 57 studies met inclusion criteria. Forty studies
used experimental designs (25 randomized controlled
trials, one controlled clinical trial, 14 beforee after
studies), and 17 used observational designs.
Methodological quality and ratings for clinical usefulness
and sustainability were variable, and there was a high
prevalence of usability errors. Tools showed moderate
but inconsistent effects on a variety of psychological and
clinical outcomes including HbA1c and weight. Meta-
regression of adequately reported studies (12 studies,
2731 participants) demonstrated that, although the
interventions studied resulted in positive outcomes, this
was not moderated by clinical usefulness nor usability.
Limitation This review is limited by the number of
accessible tools, exclusion of tools for mobile devices,
study quality, and the use of non-validated scales.
Conclusion Few tools were identified that met our
criteria for effectiveness, usefulness, sustainability, and
usability. Priority areas include identifying strategies to
minimize website attrition and enabling patients and
clinicians to make informed decisions about website
choice by encouraging reporting of website quality
indicators.
Diabetes mellitus affects 285 million people
worldwide and is a leading cause of death in most
high-income countries.
1
Clinical care gaps are
common in diabetes care. For example, in an
American population-based survey, only 62% of
patients with diabetes had low-density lipoprotein
cholesterol measured annually.
2
Reviews of inter-
ventions targeting patients and healthcare
providers to optimize diabetes care have shown
small effects on provider performance and patient
outcomes.
3e5
Given that consumers are increasingly using the
world wide web as a source of health information,
6
web-based tools offer potential for optimizing
quality of diabetes care. Use of web-based media
may improve knowledge, social support, behavior
change, and clinical outcomes.
78
However, existing
diabetes websites have wide variations in the
quality of evidence provided
9
and offer didactic
information at high reading levels with little
interactive technology, social support, or problem-
solving assistance.
10
Similarly, although healthcare
providers increasingly use online resources for
patient care, the volume, breadth, editorial quality,
and evidence-based methodology upon which they
were developed are highly variable.
11
The effectiveness of these tools in changing
clinical outcomes has been the subject of reviews in
other topic areas; for example, a systematic review
of consumer health informatics applications in
diverse topic areas, including breast cancer, found
that these applications improved clinical
outcomes.
12
Their effectiveness in a research setting
may not translate to effectiveness in clinical prac-
tice; factors that affect their adoption into clinical
practice include clinical usefulness, usability, and
sustainability.
13
Specically, a clinically useful tool,
dened as a tool that provides clinically useful
answers and is easy to use, access, and read,
14
may
differ in a research context; for example, while
a website on carbohydrate counting may be useful
in a research setting with a research dietician, it
may be less useful to the consumer trying to use it
alone in a real-life setting. Similarly, usability of the
tool (dened as the extent to which a product can
be used by specied users to complete tasks
successfully, in time, and with satisfaction in
a specied context
15
) may be underemphasized in
research studies,
16
where participants are routinely
oriented to and trained on the use of the tool.
Finally, sustainability, dened as the degree to
which an innovation continues to be used after
initial effort to secure adoption is completed,
17
is
a critical component in addressing the gap between
research and practice,
18
yet is often not addressed or
assessed.
19
Critical appraisal of web tools should
thus consider their effectiveness and their clinical
usefulness, usability, and sustainability. Previous
studies have not evaluated the validity, clinical
usefulness, usability, and sustainability of web-
compatible, diabetes-related tools for patients and
providers, which was the objective of this study.
<
Additional appendices are
published online only. To view
these files please visit the
journal online (www.jamia.bmj.
com/content/19/4.toc).
1
Keenan Research Centre, Li Ka
Shing Knowledge Institute of
St Michael’s Hospital, Toronto,
ON, Canada
2
Department of Medicine,
Faculty of Medicine, University
of Toronto, Toronto, ON, Canada
3
St Michael’s Hospital, Toronto,
ON, Canada
Correspondence to
Dr Catherine H Yu Keenan
Research Centre, Li Ka Shing
Knowledge Institute of St
Michael’s Hospital, 30 Bond St,
Toronto, ON M5B 1W8, Canada;
yuca@smh.ca
Received 9 April 2011
Accepted 27 November 2011
Published Online First
3 January 2012
This paper is freely available
online under the BMJ Journals
unlocked scheme, see http://
jamia.bmj.com/site/about/
unlocked.xhtml
514 J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307
Review
Page 1
Because of the importance of multifactorial vascular risk
reduction as well as comprehensive lifestyle modication in the
care of patients with diabetes,
20
we were interested in diabetes-
specic tools and tools for blood pressure, lipid, smoking,
obesity, nutrition, physical activity and weight management.
20
METHODS
Data sources and searches
Published literature search strategy
In consultation with an information scientist, Medline,
EMBASE, CINAHL, and the Cochrane Central Register of
Controlled Trials were searched from their earliest date to June 1,
2011. The following search terms were used: diabetes, hyper-
tension, smoking cessation, weight reduction, online, computer-
based, and internet. The complete search strategy is provided in
online appendices. Additional articles were identied through
review of reference lists of identied studies and discussions
with experts.
Grey literature search strategy
The world wide web was searched using the Google search
engine on June 14, 2009 with preselected phrases (online
appendices). We used these phrases to search websites of interest
that had been identied on the basis of expert knowledge.
21
Sixty web hits were captured for every phrase. The rst 30 hits
that met our denition of a web-compatible diabetes care tool
were retained.
Study inclusion and exclusion criteria
Studies were included if they evaluated a web-compatible
diabetes-related care tool, dened as an audiovisual tool that is
provided in electronic form to be used as a means to educate,
support, or advise patients, care givers, or healthcare providers,
and that addressed one of the following aspects of diabetes and
pre-diabetes management: glycemic control; cardiovascular risk
factor assessment and modication (including hypertension,
dyslipidaemia, and smoking cessation); nutrition, physical
activity, and weight management; self-management and
psychological issues; and complication screening and manage-
ment. Tools were included if they addressed a relevant topic area,
regardless of whether patients with diabetes were the target
population. Studies could have observational or experimental
designs and had to include at least one psychological, behavioral,
or clinical outcome.
Studies were excluded if they (1) did not include an evaluation
of the tool, (2) were in a language other than English or French,
or (3) evaluated a tool that (a) consisted of an electronic data-
base with no exportable stand-alone tool (such as an electronic
medical record), (b) had supplementary hardware or software
requirements that were not readily available to the average user,
or (c) required a real-time operator (such as a healthcare
provider).
Study selection
Titles and abstracts were screened for relevance by two inde-
pendent reviewers (CHY, SES; gure 1). Potentially eligible full
articles were then retrieved and reviewed independently by two
reviewers (CHY, SES) to determine whether they met inclusion
criteria. A third reviewer was available in cases of disagreement.
Determination of tool accessibility
Tools were evaluated for accessibility by attempting to access
the web-connected tool on a personal computer with standard
software. If the tools were not readily accessible online, two
attempts were made to contact the authors by email for addi-
tional information or access.
Data extraction and quality assessment
For each study that identied an accessible tool, two reviewers
independently extracted study characteristics using electronic
data extraction forms. We used a modication of the Cochrane
Effective Practice and Organization of Care Group data
abstraction form.
22
These forms characterized study design,
participants, tool description, study outcomes, and results.
Differences in data extraction were reconciled by consensus.
Tools identied from the grey literature search were catego-
rized by content and educational focus (online appendices). We
randomly selected ve websites from each of these categories
and reviewed them to determine if there was evidence of clinical
effectiveness. To assess these sites, we developed an instrument
based on a framework by Straus and Haynes
14
and tested its face
validity with relevant experts. This instrument contained 18
items and characterized the evidence base for the content and
effectiveness of the tool.
Tool evaluation
Clinical usefulness and sustainability
Each tool was independently reviewed for clinical usefulness and
sustainability by two members of a team of clinical experts
(CHY F Kim, H Halapy, and C West; see Acknowledgments).
Differences were reconciled by consensus. As there were no vali-
dated instruments to assess clinical usefulness or sustainability,
we developed instruments using a framework from the literature
and input from experts
14 23
; these instruments were determined
to have face validity by experienced clinicians and experts in
knowledge translation. The clinical usefulness instrument
contained ve items and assessed clinical relevance and ease of
access using a Likert scale with scores ranging from 0 to 5. A score
of 5 denoted clinically useful answers are available most of the
time, and are easily accessible and readable within a few minutes,
and a score of 0 denoted not useful clinically. The sustainability
instrument contained six items and assessed continued relevance
of the topic, potential barriers to sustainability, and engagement
of a group to keep the tool up to date. The instrument was
designed to exclude major barriers to sustainability.
Usability
Each tool was independently reviewed for usability by two
members of a team of human factors engineers (S Jovicic, A Xu,
H Takeshita, and F Wan; see Acknowledgments). The instru-
ment incorporated questions from three industry-standard
usability instruments (ISO 9241-110 Usability Heuristics,
24
Gerhardt-Powals Research-based Heuristic,
25
and Site Assess-
ment Tool
26
) and contained 27 items characterizing suitability
to users skill, ease of navigation, reduction in cognitive load,
and appearance. For each desirable usability characteristic, raters
scored yes, no,ornot applicable.
Data synthesis and analysis
Inter-rater reliability for data abstraction for clinical effective-
ness, clinical usefulness, sustainability, and usability were
calculated.
Owing to heterogeneity in study design, population, inter-
ventions, and outcomes, meta-analyses by intervention type or
outcome were not possible. However, we described study quality
and performed a descriptive analysis of studies with evidence
of impact on outcomes. In addition, we performed a meta-
regression of all eligible studies (irrespective of intervention
J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307 515
Review
Page 2
type or outcome) to assess whether clinical usefulness ratings
and usability ratings were moderators of the effectiveness of
these interventions. The meta-regression was performed in R
27
version 2.12.0 using the contributed package metafor
28
version
1.4-0. The Mad
29
version 0.8 package was used to convert
the treatment effects into a standardized treatment effect.
Hedges G
30
Studies were excluded if there were insufcient data
to determine a treatment effect or its variance. Studies without
a true control group were also excluded, as there is no way to
incorporate clinical usefulness or usability for both groups.
RESULTS
Published literature
Results of the literature search, study, and tool selection are
detailed in gure 1. While 393 studies and 219 unique tools were
identied, 127 tools were not accessible, and thus we were not
able to evaluate them. We evaluated the remaining 92 tools and
corresponding 57 studies.
Inter-rater reliability
Inter-rater reliability was moderate to good:
k
for data abstrac-
tion items for clinical effectiveness ranged from 0.66 to 0.72.
Weighted
k
values
31
for assessment of clinical usefulness and
sustainability were 0.50 (95% CI 0.36 to 0.64) and 0.77 (95% CI
0.63 to 0.87), respectively. Cohens
k
32
ranged from 0.50 to 0.65
for each component of the scale (ISO, 0.65; Gerhardt-Powals,
0.50; Site Assessment Tool, 0.55).
Description of studies
Study quality and type
Of 57 studies, 40 studies used experimental designs (25
randomized controlled trials (RCTs),
33e56
one controlled clinical
trial,
57
14 uncontrolled beforeeafter studies
39 47 58e68
), and 17
studies used observational designs (one caseecontrol trial,
69
seven cross-sectional studies,
70e79
nine cohort studies
76e78 80e85
).
One article consisted of four studies including two RCTs and
two uncontrolled beforeeafter studies.
47
One article consisted of
one RCT and one uncontrolled beforeeafter study.
39
Risk of bias
The methodological quality of all studies is described in online
appendix tables 1 and 2. Characteristics of the RCTs are
summarized in appendix table 1. Methodological quality was
variable; intention-to-treat analysis and description of loss to
Figure 1 Published literature search algorithm. PRISMA flow diagram outlining results of published literature search and tool identification process.
516 J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307
Review
Page 3
follow-up occurred in approximately half of the studies, and
calculation of statistical power, randomization, concealment of
allocation, and follow-up of more than 6 months were described
in the minority of studies.
Description of tools
Tool formats
Formats included static websites, decision aids,
34 49 52 56 63 64
interactive websites,
34e36 38e41 44e46 48 51 54e62 65e76 78 79 81 82 85
CD-ROM games or DVD,
33 42 43
and email feedback
programs.
34 37 47 52 53 77
Target audience
Five tools targeted patients with type 1 or 2 diabetes.
33 42 55 74 75 79
The remaining tools did not specically target patients with dia-
betes, but did address an aspect of comprehensive diabetes care in
overweight adults,
40 53e55 57
smoking adults,
35 47 49 51 56 59 60 70 71
76 77 80
depressed adults,
41 58 69
children,
43 62
smoking adoles-
cents,
48 50
and adolescents at risk of type 2 diabetes.
68
With
respect to informal care givers and healthcare professionals,
10 studies targeted healthcare providers, with six targeting
physicians,
54 65 72 73 82e84
three studies targeting nurses,
38 39
and
one study targeting public health professionals
61
; there were no
studies that targeted care givers.
Clinical usefulness, sustainability, and usability
Clinical usefulness, sustainability, and usability ratings are
summarized in table 1. The most common usability error (found
in 50% of tools) was not utilizing images to facilitate learning,
a feature that has been demonstrated to aid data interpretation
and improve recognition and recall.
25
Clinical effectiveness of tools
Patient outcomes, including knowledge, skill development,
behavior change, and psychological and clinical outcomes, were
examined in 17 studies.
37 40e53 55
Clinician knowledge and skill
were evaluated in three studies
38 39 54
(online appendix table 3).
Patient diabetes education tools
One study examined the effect of a multimedia general diabetes
education computer application for low-literacy patients to use
in clinic waiting rooms.
42
At 1 year, the intervention group had
a greater awareness of diabetes complications and a greater
reduction in HbA1c than the control group (online appendix
table 3).
Patient self-management and coping tools
Two studies examined the effect of a self-management website
or DVD in patients with poorly controlled diabetes.
33 55
In the
rst study,
55
the intervention group had a greater reduction in
HbA1c and systolic blood pressure, an increase in high-density
lipoprotein cholesterol, and reduction in triglycerides compared
with the control group at 12 months (online appendix table 3).
In addition, greater website use correlated with greater clinical
improvements: persistent website users had greater reduction in
HbA1c from baseline compared with intermittent users (1.9%
vs 1.2%, p¼0.051). Similarly, larger numbers of website data
uploads were associated with a larger decline in HbA1c (highest
Table 1 Summary of clinical usefulness, sustainability, and usability ratings
Summary of clinical usefulness ratings
Number of tools for which:
Had clinically useful
answers available:
Answers were easily accessible
within a few minutes References
Answers were not
easily accessible References
Most of the time 26 36 37 46 49 51 52 56 57 60 62
63 65 67 68 72 73 78 82 e 85
29 34 35 39e45 47 48 57 58 64
69 77 79 80
Some of the time 18 49 51 54 55 57 66 10 38 57 59 70 71 76 81
Number of tools References
Rarely 6 53 57 61
Not at all 3 57 74 75
Summary of sustainability ratings
Sustainability instrument item
Number of tools with response of:
Barriers identifiedYes References No References
Will this topic continue to be relevant? 32 34e78 80e83 85 0
Are there any potential barriers for
patients, care givers, the public, or
healthcare providers to using this tool?
84853e55 57 59 66 68
71 74e76
24 34e47 49e52 56e58
60e65 73 77e85
Length of time required, login
requirement, presence of advertising,
and site credibility
Can this tool be easily integrated into
existing practice and systems?
23 34e39 42 43 46 47 49e57
60 62e69 72e75 77e80 82e85
9 40414445485357e59
61 68 69 71 76 81
Use of country-specific language and
measurement units and cost
Can groups be easily engaged to facilitate
sustainability of this tool?
26 34 35 37e39 42 43 46 55e57
60e68 70 72e75 77e80 82e85
6 364041444557e59
69 71 76 81
Is there a leader responsible for making
modifications to this tool as new
knowledge is brought forward?
29 34 35 37e40 42e55 57 60e68
70 72e75 77e85
3 36415859697176
Summary of usability ratings
Number of usability errors Number of tools References
0 5 33 42 43 57
1e2 31 3436383941444547495154e59
61 64e66 68 69 71e77 80e85
3e535374649e52 57 62 63 67 68 70 79
6e10 21 35 40 48 53 55 57 60 68 78
J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307 517
Review
Page 4
tertile 2.1% vs lowest tertile 1.0%, p<0.02). The second
study, which compared in-person class-based programs with
DVD-based self-management programs, showed no differences
in self-management behavior or psychological or clinical
outcomes.
33
One study examined the effect of an interactive website
providing tailored advice on lifestyle modication and risk factor
screening, based on a questionnaire on family history and life-
style habits.
36
Compared with a control group who received
standardized messages, the intervention group improved their
diet and physical activity, although there were no changes in
smoking rates and assessment of blood glucose or blood pressure,
and a reduction in cholesterol screening.
Patient nutrition and physical activity tools
Seven studies examined the effect of nutrition and physical
activity websites
37 40 44 e 46 52 53
on waist circumference,
40
weight,
53
body mass index (BMI),
44
percentage body fat,
44
blood
pressure,
44
quality of life,
37
and coronary RR.
40
Measures of obesity (waist circumference, weight, BMI, and
percentage body fat) were signicantly improved,
40 44 53
as was
coronary RR
44
and self-assessed health status
37
(online
appendix table 3). In a study of a web-based physical activity
program,
40
waist circumference decreased signicantly in the
intervention group compared with the controls. Similarly, an
interactive personalized health promotion website was found to
result in a greater reduction in percentage body fat compared
with the control, although there was no signicant change in
BMI or blood pressure.
44
This study also found a signicant
reduction in coronary RR in the intervention group, compared
with the control group. A third study compared the effect of
a weight loss website in combination with human-email coun-
seling, computer-automated email counseling, or no coun-
seling
53
and found that, at 3 months, there was a signicantly
greater weight loss in the human-email group and computer-
automated email group than in the no counseling group.
However, at 6 months, only the human-email group retained
signicantly greater weight loss compared with the no coun-
seling group. Finally, self-assessed health status was signicantly
greater in the group using an email physical activity and diet
program than the control group.
37
One study examined the effect of an educational CD-ROM
game about nutrition and physical activity for children.
43
Whereas there was a greater reduction in BMI in girls (p¼0.04),
a greater increase in BMI was noted in boys (p¼0.04) 5 weeks
after the intervention.
Patient smoking cessation tools
Eight studies examined the effect of ve online smoking
prevention and cessation websites on clinical outcomes
(smoking initiation,
50
cigarette use,
48
1-day smoking
abstinence,
35
7-day smoking abstinence,
34 47 49
30-day point
prevalence
51 56
). Study quality was variable. There was incon-
sistent reporting of loss to follow-up, similarity of groups at
baseline, or adequacy of randomization (online appendix
table 3). It was unclear whether analysis was conducted by
intention-to-treat in the studies that reported positive
outcomes.
47 48
One study of an interactive, multimedia smoking prevention
and cessation curriculum demonstrated lower rate of smoking
initiation compared with use of a self-help booklet.
50
With
respect to smoking cessation, one study showed no change in
cigarette use at 6 months,
48
three studies demonstrated no
difference in quit rate (as measured by 1-day reported
abstinence,
35
7-day reported abstinence,
34 47 48
30-day point
prevalence
51 56
), and one study showed an improvement in quit
rate.
47
However, in an exploratory analysis by website utilization,
Rabius found that higher smoking-quit rates were associated
with the two more highly utilized websites compared with the
three less frequently utilized sites (12.5% vs 10.6%, p¼0.03).
51
Similarly, participants who visited a site more than ve times
were twice as likely to quit than participants who visited a site
less than ve times (20.0% vs 9.8%, p<0.001). In addition, higher
quit rates were found with more interactive, tailored sites
compared with the static control site (13% vs 10%, p¼0.04).
51
Clinician education tool
One study examined the effect of an online continuing educa-
tion seminar on physician knowledge of diabetes management.
54
Physicians recommendation of appropriate quality-of-care
measures was assessed immediately after the intervention using
a non-validated clinical vignette score and did not change, with
the exception of one process-of-care measure (ordering an eye
exam).
Clinician behavior change counseling tool
Two studies examined the effect of an interactive web-based
motivational interviewing educational program on teaching
effectiveness immediately after the intervention.
38 39
In both
studies, teaching effectiveness, as measured by qualitative
analysis and coding of written responses to counseling vignettes
and a multiple choice questionnaire, was higher in the inter-
vention group.
Role of interactivity
More interactive tools resulted in greater clinical improvement;
for example, Tate et al found that interaction in the form of
human- or computer-email counseling resulted in greater weight
loss than no counseling.
53
Similarly, Goran and Reynolds found
that an interactive multimedia CD-ROM game resulted in
greater reduction in BMI than a static educational CD-ROM.
43
This observation is seen also in patient smoking cessation
websites; Rabius et al found that interactive tailored smoking
cessation sites resulted in greater quit rates than a static site,
51
and Munoz et al found that individually timed educational
messages resulted in greater quit rates than the static smoking
guide alone.
47
A similar nding was seen in tools for healthcare
providers; Carpenter et al found that an interactive tutorial was
more effective in teaching motivational interviewing techniques
than reading material.
38 39
The role of tool interactivity on
continued website use is highlighted in the study of the inter-
active personalized health promotion website: Hurling et al
found that the interactive site resulted in a signicantly greater
percentage of participants logging in throughout the study
period, with less attrition than the static, control site.
45
Grey literature
Tool selection and evaluation are described in gure 2. Of the 360
websites reviewed, two
86 87
had been evaluated for clinical
effectiveness. Both evaluations had been identied in the pub-
lished literature search
47 48
and the tools described previously.
Clinical usefulness and usability as potential moderators of tool
effectiveness
Figure 3A,B shows the results of the meta-regression, with
clinical usefulness ratings and usability ratings as potential
moderators of tool effectiveness. Twelve studies were included,
comprising a total sample size of 2731 participants. There was
518 J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307
Review
Page 5
signicant heterogeneity, with
s
of 0.87 and 0.85, respectively.
While the standard meta-analysis demonstrated a signicant
positive effect on outcomes (standardized treatment effect,
Hedges G 0.64, 95% CI 0.15 to 1.13, p¼0.01), neither clinical
usefulness nor usability had a moderating effect on tool effec-
tiveness (regression coefcient 0.26 (95% CI 1.4 to 1.9, p¼0.76)
and 1.5 (95% CI 6.4 to 3.4, p¼0.55), respectively).
DISCUSSION
Although a large number of studies and tools were identied,
many tools were not accessible, and thus we were not able to
evaluate them. These tools would also not be accessible to
patients or healthcare providers; thus their exclusion does not
affect the applicability or relevance of our ndings. The 57
studies and tools identied were very different in terms of
participants, settings, and outcomes, which meant we could not
perform a meta-analysis by intervention or outcome. Although
there were a number of studies with positive results, these
results must be viewed with caution because of concerns about
the reported study designs. Half of the studies were preepost
designs or included a comparative group that was non-
randomized or not adequately randomized. Many studies would
have been strengthened through use of validated outcome
measures and longer-term follow-up of 1 year or more. With
respect to the tools, although the evidence base of the tools
content was high, only 25% had easily accessible, clinically
useful answers most of the time. Six percent of tools were free of
usability errors, but 60% had three or more errors in usability.
Common usability errors included limited use of images, icons,
and other visual elements to facilitate learning, and lack of
intuitiveness in navigation and expected next steps. These and
other usability errors can negatively affect users experience with
a tool and may lead them to stop using the tool. In one study,
ease of usability was one of the main determinants of an indi-
viduals satisfaction and willingness to engage with a website.
88
While a correlation between usability and tool effectiveness was
not demonstrated in this study, our meta-regression was limited
by the number of studies that adequately reported data, as well
as the heterogeneity in interventions, populations, and
outcomes. This high prevalence of usability errors is mirrored in
other reviews of usability of healthcare websites
89e91
and
highlights the need to ensure that websites provide useful and
usable formats and undergo usability testing before they are
launched.
Our review of the literature has identied areas for further
exploration. First, greater improvements in patient outcomes
were seen with greater use of the tool.
42 49 51 55
For example,
persistent website users had greater improvement in HbA1c
than intermittent users, and a larger number of website data
uploads was associated with a larger decline in HbA1c.
55
17 tools:
-Descriptive
studies
336 tools:
-No evidence identified
that tool was evaluated
-Not further evaluated
Tool
Identification
Tool
categorization
Tool
Selection
Tool
Eligibility
19 tools
1200 websites identified
through Google search
Categorized by tool content
and educational focus
360 tools reviewed for
evidence
24 tools:
-Evidence identified that tool
evaluated
5 tools excluded:
-Require real-time
operator (3)
-Electronic medical
record (2)
2 tools:
-Clinical effectiveness
studies
5 tools randomly selected
from each category
Already identified
in published
literature
Figure 2 Grey literature search algorithm. Modified PRISMA flow
diagram outlining results of grey literature search and tool identification
process.
Figure 3 (A) Modified forest plot demonstrating lack of moderating effect of clinical usefulness ratings on tool effectiveness. Squares with lines are
the observed treatment effects and CI. Grey diamonds show the predicted treatment effects based on the model. (B) Modified forest plot
demonstrating lack of moderating effect of usability ratings on tool effectiveness. Squares with lines are the observed treatment effects and CI. Grey
diamonds show the predicted treatment effects based on the model.
J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307 519
Review
Page 6
Caution should be used when interpreting this association, given
possible confounders that can result in reduced use such as
depression or lack of progress with respect to goals. A recent
systematic review found mixed results in the association
between adherence and outcomes; analysis was limited because
of heterogeneity of adherence and outcome measures, although
logins appeared to be associated with outcomes of physical
health inter ventions, while module completion appeared to be
associated with outcomes of psychological health interven-
tions.
92
In conjunction with the literature on website usage
attrition,
93e95
these ndings have implications for website
development and website evaluation. Second, this review
suggests a mechanism by which to minimize attrition and thus
maximize clinical improvement, through the use of interactivity
and feedback.
44 51 53
For example, Hurling et al found that an
interactive health promotion site resulted in a signicantly
greater percentage of participants logging in throughout the
study period, with less attrition, than the control static site.
44
Although greater interactivity may result in better outcomes, it
may require higher levels of health literacy, navigation skills, and
computer experience.
96 97
However, with careful user testing,
highly interactive applications can be designed to be user
friendly
98
and can have positive effects on user satisfaction,
effectiveness, efciency, and overall attitude toward the tool.
99
Other inter vention characteristics that enhance use include peer
or counselor support, email or phone contact, and updates
regarding the intervention website.
100
These ndings have implications for website developers,
researchers, patients, and clinicians. Web-based tool developers
must incorporate strategiesdsuch as optimization of website
usability and interactivitydto maximize frequency and persis-
tence of website use, and researchers must evaluate these
strategies and their impact on website usage and clinical
outcomes, as well as characteristics of users who are predisposed
to persistent website use. Given the degree of variability in
website quality, patients and clinicians should critically appraise
these resources for effectiveness, relevance, and usability before
selecting them for use. Given the burden of time and expertise
required to make these assessments, development of a trans-
parent recognized peer-review system to assess clinical effec-
tiveness, usefulness, sustainability, and usability of web-based
tools, as well as a requirement for standard reporting of these
characteristics by website developers, would enable both
patients and clinicians to make informed decisions in a timely
manner. Although website certication systems do exist, these
primarily address the evidence base of the website content rather
than website use, and do not address clinical usefulness or
usability. For example, Health On the Net Foundation (HON,
http://www.hon.ch/) is a non-prot organization that estab-
lished HONcode certication, an ethical standard aimed at
offering quality online health information. A review of
HONcode-accredited sites found that 87% were too difcult to
read for the average adult population.
101
In addition, this system
has not been universally adopted, with only 28% of diabetes
patient education sites being HonCODE-accredited.
102
This review is limited by the number of accessible tools,
exclusion of tools for mobile devices, the quality of the studies
identied, use of non-validated scales, and publication bias. In
addition, the broad interventions included, as well as the
number of study outcomes (clinical effectiveness, clinical
usefulness, sustainability, and usability), limited the ability to
synthesize data with a standard meta-analytical approach.
Mobile devices represent a highly accessible portal to health
information resources and thus have the potential to transform
healthcare delivery; however, assessing tools for mobile devices
was beyond the scope of this review. Although the rating scales
used were not formally validated, the items were derived from
the literature
14 23e26
and were assessed for face validity by
content experts. We chose to be inclusive when selecting inter-
ventions, given the multi-system involvement of the diabetic
disease process and the importance of comprehensive manage-
ment (including vascular risk modication) in the care of
the individual with diabetes. We also chose to assess the non-
traditional outcomes of clinical usefulness, sustainability, and
usability, as these are important predictors of knowledge use and
transfer.
14 88
We strove to reduce publication bias by including
a comprehensive search of the grey literature.
The strengths of this review include: an extensive literature
search that included the grey literature; the comprehensive
review of each tool for clinical effectiveness, usefulness,
sustainability, and usability; and the generalizability of ndings
regarding website use in health promotion and chronic disease
management. To our knowledge, although other reviews of
health informatics tools have addressed clinical outcomes
12 103
and usability
16
individually, no other systematic review of any
informatics intervention has considered all of these issues.
Web-based tools have the potential to improve health
outcomes and complement healthcare delivery, but their full
potential is hindered by limited knowledge about their effec-
tiveness, high prevalence of usability errors, and high attrition
rates. A development and research agenda should include:
developing strategies to reduce website attrition in order to
maximize clinical outcomes; standardizing website quality
indicators; and transparent reporting of these indicators in order
to allow patients and clinicians to make informed decisions
about website choice.
Acknowledgments We thank: Laure Perrier for carrying out the published and grey
literature searches; David Newton for information technology support; Julie Hukui,
Daniel Chong, Patrick Ong, and Kristin Tokarsky for reference management;
Mahmood Beheshti, Nathan McKibbon, Alex Rogovic, and Violetta Sochka for
website review; our clinician experts (Florence Kim, Henry Halapy, and Christine
West) for rating clinical usefulness and sustainability; and our human factors
engineers (Sasha Jovicic, Annie Xu, Harumi Takeshita, and Flora Wan) for rating
usability.
Funding This research was supported by the Ontario Ministry of Health and Long
Term Care, who had no role in study design, in the collection, analysis and
interpretation of data, in the writing of the report, or in the decision to submit the
article for publication. SES is supported by a Tier 1 Canada Research Chair.
Competing interests None.
Contributors CHY conceived the study, participated in its design, and drafted the
manuscript. RB participated in the coordination, performed data abstraction, and
drafted portions of the manuscript. EL and MSO performed data abstraction and
drafted portions of the manuscript. SES and AL participated in its design and revised
the manuscript critically for important intellectual content. All authors had access to
the data and read and approved the final manuscript.
Provenance and peer review Not commissioned; externally peer reviewed.
REFERENCES
1. Unwin N, Whiting D, Gan D, et al. IDF Diabetes Atlas. 4th edn. Internati onal
Diabetes Federation, 2009.
2. Saadine JB, Cadwell MS, Gregg EW, et al. Improvements in diabetes processes of
care and intermediate outcomes: United States, 1988e2002. Ann Intern Med
2006;144:465e74.
3. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom
and the United States: a framework for change. Milbank Q 2001;79:281e315.
4. Vermeire E, Wens J, Van Royen P, et al. Interventions for improving adherence to
treatment recommendations in people with type 2 diabetes mellitus. Cochrane
Database Sys Rev 2005;(5):CD003638.
5. Renders CM, Valk GD, Griffin SJ, et al. Interventions to improve the management
of diabetes mellitus in primary care, outpatient and community settings. Cochrane
Database Sys Rev 2001;(1):CD001481.
520 J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307
Review
Page 7
6. Statistics Canada: Canadian Internet Use Survey 2008. http://www.statcan.gc.ca/
cgi-bin/imdb/p2SV.pl?Function¼getSurvey&SDDS¼4432&lang¼en&db¼imdb&
adm¼8&dis¼2 (accessed 6 May 2009).
7. Jeste DV, Dunn LB, Folsom DP, et al. Multimedia educational aids for improving
consumer knowledge about illness management and treatment decisions: a review
of randomized controlled trials. J Psychiatr Res 2008;42:1e21.
8. Murray E, Burns J, See TS, et al. Interactive Health Communication Applications
for people with chronic disease. Cochrane Database Syst Rev 2004;(4):CD004274.
9. Seidman JJ, Steinwachs D, Rubin H. Design and testing of a tool for evaluating
the quality of diabetes consumer-information Web sites. J Med Internet Res
2003;5:e30.
10. Bull SS, Gaglio B, McKay HG, et al. Harnessing the potential of the internet to
promote chronic illness self-management diabetes as an example of how well we
are doing. Chronic Illn 2005;1:143e55.
11. Banzi R, Liberati A, Moschetti I, et al. A review of online evidence-based practice
point-of-care information summary providers. J Med Internet Res 2010;12:e26.
12. Gibbons MC, Wilson RF, Samal L, et al. Impact of consumer health informatics
applications. Evid Rep Technol Assess (Full Rep) 2009:1e546.
13. Mackert M, Kahlor L, Tyler D, et al. Designing e-health interventions for low-
health-literate culturally diverse parents: addressing the obesity epidemic. Telemed
J E Health 2009;15:672e7.
14. Straus S, Haynes R. Managing evidence-based knowledge: the need for reliable,
relevant and readable resources. CMAJ 2009;180:942e
5.
15. ISO/IEC. Ergonomic Requirements for Office Work with Visual Display Terminals
(VDT)s - Part 11 Guidance on Usability. ISO/IEC 9241-11 1998:(E).
16. Bock B, Graham AL, Sciamanna C, et al. Smoking cessation treatment on the
Internet: content, quality, and usability. Nicotine Tob Res 2004;6:207e19.
17. Rogers E. Diffusion of Innovations. 5th edn. New York: Free Press, 2005:429.
18. Mendel P, Meredith LS, Schoenbaum M, et al. Interventions in organizational and
community context: A framework for building evidence on dissemination and
implementation in health services research. Adm Policy Ment Health
2008;35:21e37.
19. Perry KJ, Hickson M, Thomas J. Factors enabling success in weight management
programmes: systemati c review and phenomenological approach. J Hum Nutr Diet
2011;24:301.
20. Gaede P, Lund-Andersen H, Parving H, et al. Effect of a multifactorial intervention
on mortality in type 2 diabetes. N Engl J Med 2008;358:580e91.
21. Canadian Agency for Drugs and Technologies in Health. Grey Matters: A
Practical Tool for Evidence-Based Searching. http://www.cadth.ca/indexphp/en/
cadth/products/grey-matters (accessed 6 May 2009).
22. Cochrane Effective Practice and Organisation of Care Group. EPOC
Resources for Review Authors. http://epoc.cochrane.org/epoc-resources-
review-authors
23. Davies B, Edwards N. The action cycle: sustaining knowledge use. In: Sharon S,
Jacqueline T, Ian DG, eds. Knowledge Translation in Health Care: Moving from
Evidence to Practice. West Sussex, UK: Blackwell Publishing Ltd, 2009.
24. Ergonomics of Human-System Interaction Part 110: Dialogue Principles. ISO
2006:9241:9110.
25. Gerhardt-Powals J. Cognitive engineering principles for enhancing human
computer performance. Int J Hum Comput Interact 1996;8:189e21.
26. U.S. Department of Health & Human Services: Site Assessment Tool.
Adapted from Research-Based Web Design & Usability Guidelines. 2009. http://
www.usability.gov/guidelines/ (accessed 13 Aug 2009).
27. R Development Core Team. A Language and Environment for Statistical
Computing. Vienna, Austria: R Foundation Statistical Computing, 2010, ISBN
3-900051-07-0.
28. Wolfgang V. Conducting meta-analyses in R with metafor package.
J Stat Softw
2010;36:1e48.
29. Del Re AC, Hoyt WT. Mad: Meta-Analysis with Mean Differences R package
version 08. 2010.
30. Cooper H, Hedges LV, Valentine JC. The Handbook of Research Synthesis and
Meta Analysis. New York, US: Russell Sage Foundation, 2009.
31. Fleiss JL, Cohen J, Everitt BS. Large-sample standard errors of kappa and
weighted kappa. Psychol Bull 1969;72:323e7.
32. Cohen J. A coefficient for agreement for nominal scales. Educ Psychol Meas
1960;20:37e46.
33. Glasgow RE, Edwards LL, Whitesides H, et al. Reach and effectiveness of
DVD and in-person diabetes self-management education. Chronic Illn
2009;5:243e9.
34. Mun
˜
oz RF, Barrera AZ, Delucchi K, et al. International Spanish/English Internet
smoking cessation trial yields 20% abstinence rates at 1 year. Nicotine Tob Res
2009;11:1025e34.
35. Etter J. Comparing computer-tailored, internet-based smoking cessation counseling
reports with generic, untailored reports: a randomized trial. J Health Commun
2009;14:646e57.
36. Ruffin MT 4th, Nease DE Jr, Sen A, et al. Effect of preventive messages tailored to
family history on health behaviors: The Family Healthware Impact Trial. Ann Fam
Med 2011;9:3e11.
37. Block G, Sternfeld B, Block CH, et al. Development of Alive! (A lifestyle intervention
via email), and its effects on health-related quality of life, presenteeism, and
other behavioral outcomes: Randomized controlled trial. J Med Internet Res
2008;10:e43.
38. Carpenter KM, Cohn LG, Glynn LH, et al
. Brief interventions for smoking cessation:
Using the internet to train healthcare providers. Int Electron J Health Educ
2008;11:1e12.
39. Carpenter KM, Watson JM, Raffety B, et al. Teaching brief interventions for
smoking cessation via an interactive computer-based tutorial. J Health Psychol
2003;8:149e60.
40. Carr LJ, Bartee RT, Dorozynski C, et al. Internet-delivered behavior change program
increases physical activity and improves cardiometabolic disease risk factors in
sedentary adults: Results of a randomized controlled trial. Prev Med
2008;46:431e8.
41. Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by
using the internet: Randomized controlled trial. BMJ 2004;328 :265e9.
42. Gerber BS, Brodsky IG, Lawless KA, et al. Implementation and evaluation of a low-
literacy diabetes education computer multimedia application. Diabetes Care
2005;28:1574e80.
43. Goran MI, Reynolds K. Interactive multimedia for promoting physical activity
(IMPACT) in children. Obes Res 2005;13:762e71.
44. Hurling R, Catt M, Boni MD, et al. Using internet and mobile phone technology to
deliver an automated physical activity program: Randomized controlled trial. J Med
Internet Res 2007;9:e7.
45. Hurling R, Fairley BW, Dias MB. Internet-based exercise intervention systems: Are
more interactive designs better? Psychol Health 2006;21:757e72.
46. Martinson BC, Crain AL, Sherwood NE, et al. Maintaining physical activity among
older adults: Six-month outcomes of the keep active Minnesota randomized
controlled trial. Prev Med 2008;46:111e19.
47. Munoz RF,
Lenert LL, Delucchi K, et al. Toward evidence-based internet
interventions: A Spanish/English web site for international smoking cessation trials.
Nicotine Tob Res 2006;8:77e87.
48. Norman CD, Maley O, Li X, et al. Using the internet to assist smoking prevention
and cessation in schools: a randomized, controlled trial. Health Psychol
2008;27:799e810.
49. Pike KJ, Rabius V, McAlister A, et al. American cancer society’s QuitLink:
randomized trial of internet assistance. Nicotine Tob Res 2007;9:415 e 20.
50. Prokhorov AV, Kelder SH, Shegog R, et al. Impact of a smoking prevention
interactive experience (ASPIRE), an interactive, multimedia smoking prevention and
cessation curriculum for culturally diverse high school students. Nicotine Tob Res
2008;10:1477e85.
51. Rabius V, Pike KJ, Wiatrek D, et al. Comparing internet assistance for smoking
cessation: 13-month follow-up of a six-arm randomized controlled trial. J Med
Internet Res 2008;10:e45.
52. Sternfeld B, Block C, Quesenberry CP, et al. Improving diet and physical activity
with alive: a worksite randomized trial. Am J Prev Med 2009;36:475e83.
53. Tate DF, Jackvony EH, Wing RR. A randomized trial comparing human e-mail
counseling, computer-automated tailored counseling, and no counseling in an
internet weight loss program. Arch Intern Med 2006;166:1620e5.
54. Weston CM, Sciamanna CN, Nash DB. Evaluating online continuing medical
education seminars: Evidence for improving clinical practices. Am J Med Qual
2008;23:475e83.
55. McMahon GT, Gomes HE, Hohne SH, et al. Web-based care management in
patients with poorly controlled diabetes. Diabetes Care 2005;28:1624e
9.
56. Graham AL, Cobb NK, Papandonatos GD, et al. A randomized trial of Internet and
telephone treatment for smoking cessation. Arch Intern Med 2011;171:46e53.
57. Woolf SH, Krist AH, Johnson RE, et al. A practice-sponsored web site to help
patients pursue healthy behaviors: an ACORN study. Ann Fam Med
2006;4:148e52.
58. Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy:
analysis of site usage and changes in depression and anxiety scores. J Med Internet
Res 2002;4:e3.
59. Graham AL, Cobb NK, Raymond L, et al. Effectiveness of an internet-based
worksite smoking cessation intervention at 12 months. J Occup Environ Med
2007;49:821e8.
60. Thieleke J, McMahon J, Meyer G, et al. An evaluation of the freedom from
smoking online cessation program among Wisconsin residents. WMJ
2005;104:41e4.
61. Sears KE, Cohen JE, Drope J. Comprehensive evaluation of an online tobacco
control continuing education course in Canada. J Contin Educ Health Prof
2008;28:41e4.
62. Bell JA, Patel B, Malasanos T. Knowledge improvement with web-based diabetes
education program: Brainfood. Diabetes Technol Ther 2006;8:444e8.
63. VanWormer JJ, Pronk NP, Boucher JL. Experience an alysis of a practice-based,
online pedometer program. Diabetes Spectrum 2006;19:197e200.
64. Lalonde L, O’Conner AM, Drake E, et al. Development and preliminary testing of
a patient decision aid to assist in pharmaceutical care in the prevention of
cardiovascular disease. Pharmacotherapy 2004;24:909e22.
65. Wells S, Furness S, Rafter N, et al. Integrated electronic decision support increases
cardiovascular disease risk assessment four fold in routine primary care practice.
Eur J Cardiovasc Prev Rehabil 2007;15:173e8.
66. Smith KE, Levine BA, Clement SC, et al. Impact of MyCareTeam for poorly
controlled diabetes mellitus. Diabetes Technol Ther 2004;6:828e35.
67. Speck RM, Hill RK, Pronk NP, et al. Assessment and outcomes of HealthPartners
10,000 Steps program in an academic work site. Health Promot Pract
2010;11:741e50.
J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307 521
Review
Page 8
68. Long JD, Armstron g ML, Amos E, et al. Pilot using world wide web to prevent
diabetes in adolescents. Clin Nurs Res 2006;15 :67e79.
69. Christensen H, Griffiths KM, Korten A, et al. A comparison of changes in anxiety
and depression symptoms of spontaneous users and trial participants of a cognitive
behavior therapy website. J Med Internet Res 2004;6:e46.
70. Balmford J, Borland R, Benda P. Patterns of use of an automated interactive
personalized coaching program for smoking cessation. J Med Internet Res 2008;10:
e54.
71. Cobb NK, Graham AL, Bock BC, et al. Initial evaluation of a real-world internet
smoking cessation system. Nicotine Tob Res 2005;7:207e16.
72. Riddell T, Lindsay G, Kenealy T, et al. The accuracy of ethnicity data in primary care
and its impact on cardiovascular risk assessment and management: Predict CVD-8.
N Z Med J 2008;121:40e8.
73. Riddell T, Kenealy T, Wells S, et al. Audit of health data captured routinely in
primary healthcare for the clinical decision support system predict (PREDICT CVD-4).
HCIRO 2008;12:15e21.
74. Zrebiec J. Internet communities: do they improve coping with diabetes? Diabetes
Educ 2005;31:825e8, 830e2, 834, 836.
75. Zrebiec JF, Jacobson A. What attracts patients with diabetes to an internet support
group? A 21 month longitudinal website study. Diabet Med 2001;18:154e8.
76. An LC, Schillo BA, Saul JE, et al. Utilization of smoking cessation information,
integrative and online community resources as predictors of abstinence: Cohort
study. J Med Internet Res 2008;10:e55.
77.
Lenert L, Munoz R, Perez JE, et al. Automated e-mail messaging as a tool for
improving quit rates in an internet smoking cessation intervention. J Am Med Inform
Assoc 2004;11:235e40.
78. Petersen R, Sill S, Lu C, et al. Effectiveness of employee internet-based weight
management program. J Occup Environ Med 2008;50:163e71.
79. Herrejon K, Hartke JL, Scherer J, et al. The creation and impact evaluation of “Your
guide to diet and diabetes,” an interactive web-based diabetes tutorial. Diabetes
Technol Ther 2009;11:171e9.
80. Lenert L, Munoz RF, Stoddard J, et al. Design and pilot evaluation of an internet
smoking cessation program. J Am Med Inform Assoc 2003;10:16e20.
81. Ware LJ, Hurling R, Bataveljic O, et al. Rates and determinants of uptake and use of
an internet physical activity and weight management program in office and
manufacturing work sites in England: Cohort study. J Med Internet Res 2008;10:e56.
82. Marshall RJ, Zhang Z, Broad JB, et al. Agreement between ethnicity recorded in
two New Zealand health databases: Effects of discordance on cardiovascular
outcome measures (PREDICT CVD3). Aust N Z J Public Health 2007;31:211e16.
83. Selak V, Wells S, Whittaker R, et al. Smoking status recording in GP electronic
records: The unrealised potential. Inform Prim Care 2006;14:235e41; discussion
242e5.
84. Whittaker R, Bramley D, Wells S, et al. Will a web-based cardiovascular disease
(CVD) risk assessment programme increase the assessment of CVD risk factors for
maori? N Z Med J 2006;119:U2077.
85. Sarna L, Bialous S, Wewers ME, et al
. Nurses trying to quit smoking using the
Internet. Nurs Outlook 2009;57:246e56.
86. The Stop Smoking Center. Welcome to the Stop Smoking Center 6.1!. http://
wwwstopsmokingcenternet/ (accessed 1 Mar 2010).
87. The American Cancer Society. Do You Need Help to Quit? 2008. http://
wwwcancerorg/docroot/PED/content/PED_10_13x_Smoking_Habits_Quizasp?
sitearea¼&level¼ (accessed 1 Mar 2010).
88. Fu L, Salvendy G. Th e contribution of apparent and inherent usability to a user’s
satisfaction in a searching and browsing task on the Web. Ergonomics
2002;45:415e24.
89. Seidman JJ, Steinwachs D, Rubin H. Design and testing of a tool for evaluating the
quality of diabetes consumer-information web sites. J Med Internet Res
2003;5:136e56.
90. Sutherland LA, Wildemuth B, Campbell MK, et al. Unraveling the web: an
evaluation of the content quality, usability, and readability of nutrition web sites. J
Nutr Educ Behav 2005;37:300 e 5.
91. Bock BC, Graham AL, Whiteley JA, et al. A review of web-assisted tobacco
interventions (WATIs). J Med Internet Res 2008;10 :e39.
92. Donkin L, Christensen H, Naismith SL, et al. A systematic review of the impact of
adherence on the effectivene ss of e-therapies. J Med Internet Res 2011;13:e52.
93. Gunther E. The law of attrition. J Med Internet Res 2005;7 :e11. http://
wwwjmirorg/2005/1/e11/v7e11
94. Christensen H, Mackinnon A. The law of attrition revisited. J Med Internet Res
2006;8:e21. http://wwwjmirorg/2006/3/e20/v8i3e20
95. Gunther E. The law of attrition revisited: Author’s Reply. J Med Internet Res
2006;8:e21. http://wwwjmirorg/2006/3/e20/v8i3e21
96. Morrel R. Older Adults, Health Information and the World-Wide Web. New Jersey,
US: Lawrence Erlbaum Associates, 2001.
97. Milne SAD, Carmichael A, Sloan D, et al. . Are guidelines enough? An introduction
to designing web sites accessible to older people. IBM Syst J 2005;43.
98. Boquete L, Rodrı
´
guez-Ascariz JM, Amo-Usanos C, et al. User-Friendly Cognitive
Training for the Elderly: A Technical Report. Telemed J E Health 2011;17:456e60.
99. Teo HH, Oh LB, Liu C, et al. An empirical study of the effects of interactivity on web
user attitude. Int J Hum Comput Stud 2003;58:281e305.
100. Brouwer W, Kroeze W, Crutzen dNJ R, et al. Which intervention characteristics
are related to more exposure to internet-delivered healthy lifestyle promotion
interventions? A systematic review. J Med Internet Res 2011;13:e2.
101. Kusec S, Brborovic O, Schillinger D. Diabetes websites accredited by the Health On
the Net Foundation Code of Conduct: Readable or Not? Stud Health Technol Inform
2003;95:655e60.
102. Thakurdesai PA, Kole PL, Pareek RP. Evaluation of the quality and contents of
diabetes mellitus patient education on Internet. Patient Educ Couns
2004;53:309e13.
103. Austin Boren S, Gunlock TL, Krishna S, et al. Computer-aided diabetes education:
a synthesis of randomized controlled trials. AMIA Annu Symp Proc 2006:51e5.
PAGE fraction trail=8.75
522 J Am Med Inform Assoc 2012;19:514e522. doi:10.1136/amiajnl-2011-000307
Review
Page 9
    • "One review researched the effectiveness of cell phone based interventions (Krishna, Boren, 2008). One review conducted an evaluation of web accessible tools for diabetes selfmanagement and related cardiovascular risk factors (Yu, Bahniwal et al. 2012). Finally, one review researched effectiveness of web based self-management education for cancer related chronic diseases (Kuijpers, Groen et al. 2013) The PUBMED search terms were (review[ "
    [Show description] [Hide description] DESCRIPTION: The development of a comparative effectiveness study is a task of the Diabetes Literacy Project´s work package (WP) 5 (led by Ludwig Boltzmann Institute Health Promotion Research LBIHPR). The primary research question is: Are some channels for diabetes Type II patient self-management interventions more effective than others? This working paper documents the first step in answering the primary research question: The development of an outcome framework for the Diabetes Literacy project. After an initial systematic metaliterature search in PUBMED and an assessment of the retrieved literature, WP5 research team concluded that existing studies and reviews could not provide the necessary information to consistently answer the question if some interventions are more effective than others. Different studies in the field use a plethora of psychological-, social-, behavioral-, clinical- and wellbeing-outcomes. This in fact reveals not only the field´s complexity but the many different perspectives on what may or may not be regarded as desirable outcomes of DSM interventions. In order to deal with the different kinds of DSM outcomes, WP5 research team (Ludwig Boltzmann Institute Health Promotion Research) initiated an in-depth investigation of outcomes used in the DSM literature, with the aim to develop a DSM outcome framework (DSMOF) for the Diabetes Literacy Project. This framework or map was a necessary condition for the Diabetes Literacy Project to develop rationales on how to deal, not only with questions about program comparability, but also for developing an empirical research instrument and study design.
    Full-text · Working Paper · Mar 2016
    • "For successful implementation of an innovation in PC, the new tool must be easily accessible, suit providers' day-to-day practice , and provide a certain level of comfort [55,56]. For example, when web-based tools were utilized for the management of diabetes and cardiovascular disease, the main factors that increased adoption in clinical practice were the usefulness, usability and sustainability of the in- strument [57]. Similarly, participants in our focus groups stated that they would recommend an e-DA tool only if it was user-friendly, easily accessible, free of cost, and guided patients to complete dietary assessments electronically with little or no assistance from a health provider. "
    [Show abstract] [Hide abstract] ABSTRACT: Dietary assessment can be challenging for many reasons, including the wide variety of foods, eating patterns and nutrients to be considered. In team-based primary care practice, various disciplines may be involved in assessing diet. Electronic-based dietary assessment (e-DA) instruments available now through mobile apps or websites can potentially facilitate dietary assessment. Providers views of facilitators and barriers related to e-DA instruments and their recommendations for improvement can inform the further development of these tools. The objective of this study was to explore provider perspectives on e-DA tools in mobile apps and websites. The exploratory sequential mixed methods design included interdisciplinary focus groups followed by a web-based survey sent to Family Health Teams throughout Ontario, Canada. Descriptive and bivariate analyses were completed. Focus group transcripts contributed to web-survey content, while interpretive themes added depth and context. 11 focus groups with 50 providers revealed varying perspectives on the use of e-DA for: 1) improving patients' eating habits; 2) improving the quality of dietary assessment; and, 3) integrating e-DA into the care process. In the web-survey 191 respondents from nine disciplines in 73 FHTs completed the survey. Dietitians reported greater use of e-DA than other providers (63% vs.19%; p = .000) respectively. There was strong interest among disciplines in the use of e-DA tools for the management of obesity, diabetes and heart disease, especially for patient self-monitoring. Barriers identified were: patients' lack of comfort with using technology, misinterpretation of e-DA results by patients, time and education for providers to interpret results, and time for providers to offer counselling. e-DA tools in mobile apps and websites may improve dietary counselling over time. Addressing the identified facilitators and barriers can potentially promote the uptake of e-DA into clinical practice.
    Full-text · Article · Dec 2015 · BMC Medical Informatics and Decision Making
    • "Reviews of existing diabetes websites showed that they presented didactic information of variable quality, they required advanced reading levels, and they followed a static, newspaper-format display, rather than harnessing the inherent advantages of websites, such as interactive technology , social support, and problem-solving assistance [11,13] . A systematic review of electronic diabetesrelated tools found that they had moderate but inconsistent effects on a variety of psychological and clinical outcomes, including HbA1c and weight; tools that were more interactive tools were associated with continued website use and greater clinical improvement [10]. In addition, greater website use was correlated with greater clinical improvements: regular website users had greater reductions in HbA1c compared with intermittent users. "
    [Show abstract] [Hide abstract] ABSTRACT: Background Management of diabetes mellitus is complex and involves controlling multiple risk factors that may lead to complications. Given that patients provide most of their own diabetes care, patient self-management training is an important strategy for improving quality of care. Web-based interventions have the potential to bridge gaps in diabetes self-care and self-management. The objective of this study was to determine the effect of a web-based patient self-management intervention on psychological (self-efficacy, quality of life, self-care) and clinical (blood pressure, cholesterol, glycemic control, weight) outcomes.Methods For this cohort study we used repeated-measures modelling and qualitative individual interviews. We invited patients with type 2 diabetes to use a self-management website and asked them to complete questionnaires assessing self-efficacy (primary outcome) every three weeks for nine months before and nine months after they received access to the website. We collected clinical outcomes at three-month intervals over the same period. We conducted in-depth interviews at study conclusion to explore acceptability, strengths and weaknesses, and mediators of use of the website. We analyzed the data using a qualitative descriptive approach and inductive thematic analysis.ResultsEighty-one participants (mean age 57.2 years, standard deviation 12) were included in the analysis. The self-efficacy score did not improve significantly more than expected after nine months (absolute change 0.12; 95% confidence interval ¿0.028, 0.263; p¿=¿0.11), nor did clinical outcomes. Website usage was limited (average 0.7 logins/month). Analysis of the interviews (n¿=¿21) revealed four themes:1) mediators of website use; 2) patterns of website use, including role of the blog in driving site traffic; 3) feedback on website; and 4) potential mechanisms for website effect.ConclusionsA self-management website for patients with type 2 diabetes did not improve self-efficacy. Website use was limited. Although its perceived reliability, availability of a blog and emailed reminders drew people to the website, participants¿ struggles with type 2 diabetes, competing priorities in their lives, and website accessibility were barriers to its use. Future interventions should aim to integrate the intervention seamlessly into the daily routine of end users such that it is not seen as yet another chore.
    Full-text · Article · Dec 2014 · BMC Medical Informatics and Decision Making
Show more

Similar publications

Discover more