Content uploaded by Liz Steed
Author content
All content in this area was uploaded by Liz Steed on Jun 24, 2015
Content may be subject to copyright.
DIABETES TECHNOLOGY & THERAPEUTICS
Volume 10, Number 2, 2008
© Mary Ann Liebert, Inc.
DOI: 10.1089/dia.2007.0255
Development and Piloting of an Acceptability
Questionnaire for Continuous Glucose
Monitoring Devices
L. STEED, Ph.D.,
1
D. COOKE, M.Sc.,
1
S.J. HUREL, Ph.D.,
2
and S.P. NEWMAN, D.Phil.
1
ABSTRACT
Background: User acceptability of new health technologies is important in determining their
widespread use and adoption. The aim of this current study was twofold: first, to investigate
the acceptability of two continuous glucose monitoring devices for people with diabetes; and
second, to develop a valid questionnaire measure to assess the acceptability of continuous glu-
cose monitoring devices.
Methods: Semi-structured interviews were conducted with six people with diabetes who had
previously used the GlucoWatch
®
Biographer (Animas Corp., West Chester, PA) or the CGMS
®
continuous glucose monitoring system (Medtronic MiniMed, Northridge, CA) in order to in-
crease understanding of the issues relating to acceptability of, and satisfaction with, the devices.
Interview transcripts were analyzed qualitatively using framework analysis. These analyses, to-
gether with consultation with researchers and health professionals in the field, provided the
foundation for development of a questionnaire measure that was piloted with 19 individuals.
Results: Six broad themes were elicited from the framework analysis: interference with daily
activities; reliability and accuracy of the devices; practicality and ease of use; improvements in
glycemic control; side effects; and self-consciousness and disclosure. Piloting of the question-
naire arising from this analysis demonstrated face validity. Further psychometric testing of the
questionnaire will be conducted as part of a randomized controlled trial evaluating the clinical
efficacy and cost-effectiveness of the CGMS and GlucoWatch G2 Biographer.
Conclusions: Ultimately it is the user’s preferences and his or her assessment of acceptability
that will determine uptake and use of continuous glucose monitoring devices. It is therefore es-
sential to consider and evaluate this alongside clinical efficacy and cost-effectiveness.
95
INTRODUCTION
D
IABETES MELLITUS
is associated with signifi-
cant morbidity, including both microvas-
cular and macrovascular complications. Im-
proved glycemic control has been shown to
significantly reduce the incidence of micro-
vascular complications.
1,2
A common aid in im-
proving glycemic control is frequent monitoring
of blood glucose levels, currently recommended
1
Centre for Behavioural & Social Sciences in Medicine, University College London; and
2
Department of Diabetes
& Endocrinology, UCLH Foundation Hospital, London, United Kingdom.
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the De-
partment of Health.
via capillary blood monitoring.
3
This approach
is limited as only snapshots of blood glucose lev-
els can be acquired,
4
and even testing seven
times daily may miss important episodes of
hypo- and hyperglycemia.
5
In addition, adher-
ence to capillary blood testing can be limited
with up to 67% of individuals reporting testing
less frequently than recommended.
6
This is in
part due to the pain and inconvenience associ-
ated with finger prick testing.
7
In response to
these issues new continuous glucose monitoring
(CGM) devices that are non- or minimally inva-
sive and that measure blood glucose over a pe-
riod of hours/days have been developed.
Two such CGM devices that have been used
clinically are the GlucoWatch
®
Biographer (An-
imas Corp., West Chester, PA) and the CGMS
®
CGM system (Medtronic MiniMed, Northridge,
CA). The GlucoWatch is slightly larger than a
watch and is typically worn on the forearm. It
can be worn for up to 15 h and provides pa-
tients with readings approximately every 20
min during this period. In contrast, the CGMS
is a Holter-style device, the size of a radio pager,
that is worn on the waist and connected via a
wire to a subcutaneous sensor. It is worn for 72
h, after which time results can be downloaded
for review. Both methods provide detailed pro-
files of glucose patterns that can be used by the
patient and his or her health care professionals
to adjust diabetes therapy and possibly lead to
improvements in glycemic control.
Evaluation of both devices has tended to fo-
cus on their clinical accuracy, efficacy, and cost-
effectiveness, although other issues such as the
user’s perspective or “acceptability” have also
been highlighted as important.
8,9
Acceptability
has been defined as an individual’s willingness
to use a device, which in turn depends on sev-
eral interrelated factors: the needs of the indi-
vidual, perceptions of safety and utility of the
device, and whether the person feels that use
of the device either supports or undermines his
or her sense of personal identity and control
over illness.
10
Ultimately it will be these issues
that determine uptake and adherence to new
technologies and their widespread adoption.
11
Although some studies have claimed to as-
sess acceptability of CGM devices, the methods
used are unclear, and findings tend to be lim-
ited to anecdotal or subjective reports on the
part of the researchers. For example, two stud-
ies stated that daily activities were not lim-
ited
12,13
but do not say how this was assessed.
Another study reported that the use of the
CGMS did not interfere with the care of chil-
dren with diabetes and was well accepted but
provided no information on how this was as-
sessed.
14
A further study stated that “patients
felt confident and satisfied with the CGMS” but
again provided no information on how confi-
dence and satisfaction were measured.
15
These
positive reports are to be contrasted with other
small-scale studies that have reported that the
devices produce skin reactions and are intru-
sive and difficult to calibrate.
16–18
One study has addressed the issue of the user
perspective in more detail.
9
In an evaluation of
the GlucoWatch G2 Biographer, children and
their parents completed a newly developed
questionnaire targeted at assessing satisfaction
with the device.
9
Unfortunately, the measure
was developed solely through health care pro-
fessional consultation and did not consult users
of the device. This is contrary to the recom-
mendation for the development of new psy-
chological measurement tools that states con-
sultation with the population of interest as well
as experts in the field is an important aspect of
questionnaire development.
19
Users are likely
to have a different perspective to professionals,
and hence incorporation of their opinion is crit-
ical to questionnaire development.
The aim of the current piece of work was first
to conduct a qualitative study (Study 1) with
individuals who had previously used one of
these two different CGM devices to understand
the issues related to acceptability of these de-
vices. Second, this information was used to de-
velop a questionnaire to assess acceptability of
CGM devices (ACGMD Questionnaire) ac-
cording to the recommendations of Todd and
Bradley
19
(Study 2).
STUDY 1
Methods
Study design. The qualitative study consisted
of semi-structured interviews with individuals
who had previously used one of two CGM de-
vices, with the aim of increasing understand-
ing of the acceptability of, and satisfaction with,
the devices among users.
STEED ET AL.96
Participants. All individuals who had used ei-
ther the GlucoWatch Biographer or CGMS de-
vice at a London teaching hospital (University
College London Hospital [UCLH]) in the past
18 months were eligible for participation in the
interview study unless they were currently un-
dergoing any psychiatric treatment or were un-
able to communicate in fluent English.
Recruitment and consent procedure. Individu-
als fulfilling the inclusion/exclusion criteria
were identified by the Consultant Endocrinol-
ogist. A letter and information sheet explain-
ing the purpose of the study and inviting them
to participate were then sent to eligible partic-
ipants. The following week they were con-
tacted to confirm interest in participation and
arrange an appointment for interview at their
convenience. The consent procedure included
confirmation that the information sheet had
been read, an opportunity to ask questions, and
re-emphasis that the interview would be tape-
recorded. All participants were assured of con-
fidentiality and anonymity and informed the
study was approved by the UCLH ethics com-
mittee. Written informed consent was subse-
quently obtained.
Interview format. The interviews were ex-
ploratory, and therefore a semi-structured for-
mat was used. The discussion was guided by
a topic guide, which was constructed from re-
ports in the literature and discussion with ex-
perts in the field. General topics addressed in-
cluded practical, social, and emotional impact
and concerns related to the devices; however,
there was scope for other issues related to the
devices to be raised, and the interviewer
probed all areas of importance to the intervie-
wee. No time restrictions were set for the in-
terviews.
Analysis. All interviews were tape-recorded
and transcribed. The transcripts were anon-
ymized. Framework analysis
20
was used to
identify themes relating to the acceptability of
the GlucoWatch and the CGMS. This qualita-
tive technique allows for the inclusion of a pri-
ori as well as emergent concepts, which is an
important consideration in relation to applied
research.
21
Results
Of eight eligible participants six (75%) con-
sented to take part in the study. Table 1 shows
the demographics of these participants, four of
whom had used the GlucoWatch and two the
CGMS.
Six broad themes were elicited through
analysis. These are described together with
examples of excerpts from the interview tran-
ACCEPTABILITY OF CGM DEVICES 97
T
ABLE
1. D
EMOGRAPHICS OF
P
ARTICIPANTS
U
NDERGOING
I
NDIVIDUAL
I
NTERVIEWS
Age Duration
Device Number (years) at of
last worn of times time of Type of diabetes
Device (months) worn interview Sex diabetes (years) Complications
1 Gluco Watch 18 15 times 56 F 1 45 Retinopathy,
in 2 peripheral
weeks neuropathy,
renal failure
2 CGMS 3 Once 43 M 1 8 None
3 CGMS 1 Once 45 F 1 42 None
4 Gluco Watch 18 Initially 55 M 2 18 Retinopathy,
a lot, neuropathy
less
since
5 Gluco Watch 18 15–20 35 F MODY 20 None
HNF1-
6 Gluco Watch 15 15–20 25 F 1 15 None
times in
2 weeks
MODY HNF-1
, maturity-onset diabetes of the young hepatocyte nuclear factor 1-
.
AU1
scripts that were categorized within each of
these themes. At the end of the analysis it was
felt that data saturation had been reached
with no further themes arising out of the anal-
ysis.
Interference with daily activities. This included
disruption with washing and sleep routines,
problems moving around, and variously cited
difficulties with traveling, work, shopping, and
eating out:
“It sort of slowed things down like dress-
ing, you had to take a bit more care”—Pa-
tient 2
“The only thing I found is on the tube,
because if you are standing and you are
being jammed and pushed that would
make it a problem”—Patient 3
Reliability and accuracy of the device. Gluco-
Watch participants reported that the device did
not always work in warm weather and that it
skipped readings. Some people also expressed
concerns about the accuracy of readings.
“If you are sweaty or have some problem
it will miss that reading”—Patient 5
“If you walked out into the cold, it
would go off on occasions”—Patient 4
Practicality and ease of use. This included the
inconvenience of still needing to do finger-
prick tests for calibration purposes when wear-
ing the device, time taken to learn how to use
the device and calibrate it, difficulty in calibra-
tion, and the inconvenience of alarms.
“It was extremely difficult to set up . . .
you had to have it turned on for 3 hours
beforehand, now as a working person that
meant I had to get up at 4 o’clock in the
morning to set it off, then to start it at 7
o’clock”—Patient 1
“I would have preferred one that vi-
brated discreetly, rather than bleeped all
over the place, because working at what I
do, it is usually a mixture of panic and
everybody rushing under their shirts to
look for their pagers”—Patient 4
Improvements in glycemic control. Some par-
ticipants reported that the devices filled in the
gaps in relation to the finger-prick readings.
Some stated that perceptions of control and
identification of hypoglycemia episodes were
improved.
“The feedback was good because we knew
the blood sugars dropped around 3–4
o’clock in the morning and it confirmed
that”—Patient 3
Side effects. These included dry skin, itchi-
ness, soreness, and tingling.
“ . . . you had to be quite careful about
the way you . . . it off the skin, otherwise
you would take the top layer with it”—Pa-
tient 4
“I did react to the strips . . . it went a bit
raw, but I am not sure whether I got little
scabs, but it was like that, it was itchy . . .
when I had a few on the same sort of area,
it looked like I had some horrible dis-
ease”—Patient 5
Self-consciousness and disclosure. This theme
encompassed concerns related to other people
knowing about their diabetes, having to ex-
plain what the device was, and worries about
appearance and what to wear.
“I didn’t really want to meet anyone to
have to explain what it was. . . . but I
avoided, I think, actually seeing anyone
where I might have to go into an expla-
nation”—Patient 5
Discussion
The analyses of the interview data high-
lighted the range of issues users consider im-
portant in assessing acceptability and satis-
faction with CGM devices. Although some
themes identified by participants are similar
to those reported by the DirecNet Study
Group,
9
other new areas have been identified
such as interference with daily activities. The
qualitative study provided the basis on which
to develop a measure of acceptability of CGM
devices.
STEED ET AL.98
STUDY 2
Methods
Questionnaire design. Drawing upon the
themes identified in Study 1, items were gen-
erated for the pilot questionnaire by one of the
authors (L.S.). These were divided into three
sections: (1) interference, (2) attitudes towards
the device (including reliability and accuracy,
practicality and ease of use, perceived benefits
in relation to glycemic control, and self-con-
sciousness), and (3) side effects. For each item
the response was a 5-point Likert scale, and
both positive and negatively framed questions
were incorporated. Within the interference and
side effects subscales individuals were asked to
state both whether a side effect occurred or the
extent to which the device interfered with par-
ticular activities as well as the extent to which
this was acceptable. This draws on related qual-
ity of life literature, which indicates that eval-
uations of disruptions to certain aspects of life
are only valid if individuals perceive that part
of their life to be important to them.
22,23
For ex-
ample, a patient’s social life may be affected
considerably by his or her diabetes, but this as-
pect of their life may not be important to that
individual. This principle was applied in the
current study such that respondents were
asked to rate both how much the device inter-
fered with a certain aspect of lifestyle, and then
they were asked to rate how acceptable this
was. People with diabetes may be willing to tol-
erate disruption to activities or particular side
effects if they feel they are benefiting from
wearing the device; hence it is important to as-
sess both of these aspects.
Following initial item generation, the ques-
tionnaire was discussed with experts in the
field, including endocrinologists, diabetes spe-
cialist nurses, statisticians, clinical trialists, and
health psychologists. Besides alterations to the
phrasing and formatting of the questionnaire,
on the basis of these discussions an amendment
was made to the section on interference. Ques-
tionnaire items assessing the extent that par-
ticular behaviors/activities were avoided or al-
tered as a result of wearing the devices were
also included. The questionnaire was then pi-
loted with the potential user group.
Participants. All individuals from two hospi-
tal diabetes clinic (UCLH and Bournemouth
Royal Hospital) who had used either the Glu-
coWatch or CGMS in the previous 18 months,
were sufficiently fluent in written English, and
were not currently undergoing any psychiatric
treatment were invited to participate in the pi-
loting of the questionnaire.
Recruitment and consent procedure. Individu-
als fulfilling the inclusion/exclusion criteria
were identified by the Consultant Endocrinol-
ogists at the two hospitals, and an invitation
letter, information sheet (including contact
number in case of queries), consent form, pilot
questionnaire, and prepaid envelope were
posted to them.
Results
Nineteen (95%) outpatient clinic attendees
from the two hospitals completed a copy of the
pilot questionnaire. Seven of these had used the
Glucowatch, and 12 had used the CGMS. Five
of these participants had already taken part in
the individual interviews. Two of these had di-
abetes-related complications (Table 1). Four-
teen were new to the study. One of these had
nephropathy. The majority of the respondents
had type 1 diabetes (89%) with a mean dura-
tion of diabetes of 18 years. The mean age of
the participants was 41 years with 53% of par-
ticipants female.
Comments from the user group included ref-
erence to phraseology, e.g., meaning of cali-
bration, advice to include “not applicable” op-
tions, and recommendations to clarify certain
statements, for example, “my normal bathing
routine” was changed to “my normal washing
routine” (e.g., bath/showering) as well as ad-
vice on formatting. These were the only areas
identified by the user group for amendment.
These changes were incorporated into the final
version of the questionnaire shown in sum-
mary in Table 2. Further psychometric testing
of the questionnaire is to be conducted as part
of a randomized controlled trial evaluating ef-
ficacy of the two devices. This will include an
analysis of the structure of the questionnaire to
consider the dimensions of acceptability that
are important to assess in the evaluation of
CGM devices.
ACCEPTABILITY OF CGM DEVICES 99
Discussion
Piloting of the questionnaire with patients
who had previously used CGM devices indi-
cated that the measure has face validity and has
the potential to assess an important aspect of
efficacy of these new technologies, alongside
formal evaluations of their clinical and cost-ef-
fectiveness.
GENERAL DISCUSSION
AND CONCLUSIONS
The need for more formal, systematic, and
patient-based evaluations of acceptability of
CGM devices is essential. People with diabetes
may only be accepting of the CGM devices if
perceptions of their utility, value, and benefits
outweigh any inconvenience, pain, and dis-
comfort. This aspect of acceptability has rarely
been assessed, and the development of the
questionnaire presented here attempts to pro-
vide a measure to address this. The method-
ological process of including the user group in
questionnaire development is a strength of the
current study and is in line with current rec-
ommendations.
19
This is in contrast to other
studies in the field, which have often used in-
validated approaches for assessing acceptabil-
ity.
12–18
The current study also included both
individuals with type 1 and type 2 diabetes.
This has rarely been the case in previous stud-
ies, with the majority of work conducted with
type 1 populations, and typically in children or
adolescents.
19
There is a need for assessments
of acceptability of CGM devices in the type 2
population with insulin-requiring diabetes,
and the current measure is designed with this
in mind.
Public acceptability of new technologies has
been conceptualized as being on a continuum,
with emphasis on fluidity and the fact that ac-
ceptability can change over time in response to
knowledge or new understanding.
24
Appro-
priate measurements of acceptability are nec-
essary in order to assess this. While the ques-
tionnaire was developed on older CGMS
devices, it does incorporate items on alarm fea-
tures and thus would be valuable in future
evaluations of real-time CGM devices. As has
already been argued, ultimately it is the user’s
preferences and his or her assessment of ac-
ceptability that will determine uptake and use
of CGM devices, which is why this needs to be
considered and evaluated alongside clinical ef-
ficacy and cost-effectiveness.
STEED ET AL.100
T
ABLE
2. S
UMMARY OF THE
ACGMD Q
UESTIONNAIRE
Example item(s) Response format
Section 1
Interference (9 items) (a) when wearing the monitor it 5-point Likert scale from not at
(a) interfered with my normal exercise all to completely
(a) routine
(b) I found this acceptable
Avoidance (8 items) I avoided wearing the monitor when . . . 3-point Likert scale from not at
exercising all to always
Section 2
Attitude to device I was not worried about the way I looked 5-point Likert scale from
(38 items) when wearing the monitor strongly disagree to strongly
I was confident that the monitor would agree
accurately record if I was going
hypoglycemic
I thought that generally the monitor was
impractical
I would recommend other people in a
similar situation to me to wear the
monitor
Section 3
Side effects (9 items) Did you experience . . . itching? Dichotomous scale (yes/no)
If yes, how acceptable was this to you? 5-point Likert scale from not at
all to completely
ACKNOWLEDGMENTS
This study was funded by the NIHR Health
Technology Assessment Programme.
REFERENCES
1. The Diabetes Control and Complications Trial Re-
search Group: The effect of intensive treatment of di-
abetes on the development and progression of long-
term complications of insulin-dependent diabetes
mellitus. N Engl J Med 1993;329:977–986.
2. UK Prospective Diabetes Study (UKPDS) Group: In-
tensive blood-glucose control with sulphonylureas or
insulin compared with conventional treatment and
risk of complications in patients with type 2 diabetes
(UKPDS 33). Lancet 1998;352:837–853.
3. American Diabetes Association Position Statement.
Standards of medical care in diabetes. Diabetes Care
2007;30(Suppl):S4–S41.
4. Bode BW, Sabbah H, Davidson PC: What’s ahead in
glucose monitoring? New techniques hold promise
for improved ease and accuracy. Postgrad Med
2001;109(4):41–49.
5. Bolinder J, HagstromToft E, Ungerstedt U, Arner P:
Self-monitoring of blood glucose in type I diabetic pa-
tients: comparison with continuous microdialysis
measurements of glucose in subcutaneous adipose tis-
sue during ordinary life conditions. Diabetes Care
1997;20:64–70.
6. Karter AJ, Ferrara A, Darbinian JA, Ackerson LM,
Selby JV: Self-monitoring of blood glucose: language
and financial barriers in a managed care population
with diabetes. Diabetes Care 2000;23:477–483.
7. Ginsberg BH: An overview of minimally invasive
technologies. Clin Chem 1992;38:1596–1600.
8. Heinemann L: Glucose monitoring: today and to-
morrow. Infusystems Int 2004;3:9–14.
9. Diabetes Research in Children Network (DirecNet)
Study Group: Youth and parent satisfaction with clin-
ical use of the GlucoWatch G2 Biographer in the man-
agement of pediatric type 1 diabetes. Diabetes Care
2005;28:1929–1935.
10. McCreadie C, Tinker A: The acceptability of assistive
technology to older people. Ageing Soc 2005;25:91–110.
11. Heinemann L: Clinical development of continuous
glucose monitoring systems: consideration for the op-
timal strategy. Diabetes Res Clin Pract 2006;74(Suppl
2):S82–S92.
12. Cauza E, Hanusch-Enserer U, Strasser B, Ludvik B,
Kostner K, Dunky A, Haber P: Continuous glucose
monitoring in diabetic long distance runners. Int J
Sports Med 2005;26:774–780.
13. Jungheim K, Wientjes KJ, Heinemann L, Lodwig V,
Koschinsky T, Schoonen AJ: Subcutaneous continu-
ous glucose monitoring: feasibility of a new micro-
dialysis-based glucose sensor system. Diabetes Care
2001;24:1696–1697.
14. Jeha GS, Karaviti LP, Anderson B, Smith EO, Don-
aldson S, McGirk TS, Haymond MW: Continuous glu-
cose monitoring and the reality of metabolic control
in preschool children with type 1 diabetes. Diabetes
Care 2004;27:2881–2886.
15. Chico A, Vidal-Rios P, Subira M, Novials A: The con-
tinuous glucose monitoring system is useful for de-
tecting unrecognized hypoglycemias in patients with
type 1 and type 2 diabetes but is not better than fre-
quent capillary glucose measurements for improving
metabolic control. Diabetes Care 2003;26:1153–1157.
16. Papworth A, Hurel SJ: Non-invasive GlucoWatch—the
future or big brother watching? Patient expectation, ex-
perience and results. Diabet Med 2001;18(Suppl 2):35.
17. Krimholtz MJ, Pickup JC: First clinical experiences
with the Glucowatch Biographer. Diabet Med 2001;
18(Suppl 2):115.
18. Lenzen H, Barrow BA, White S, Holman RR: The Glu-
cowatch Biographer—a useful home monitoring de-
vice for patients with type 1 diabetes. Diabet Med
2001;18(Suppl 2):115.
19. Todd C, Bradley C: Evaluating the design and devel-
opment of psychological scales. In: Bradley C, ed.
Handbook of Psychology and Diabetes: A Guide to
Psychological Measurement in Diabetes Research and
Practice. Reading, UK: Harwood, 1994:15–42.
20. Ritchie J, Spencer L: Qualitative data analysis for ap-
plied policy research. In: Bryman A, Burgess RG, eds.
Analyzing Qualitative Data. London: Routledge,
1994:173–194.
21. Lacey A, Luff D: Trent Focus for Research and De-
velopment in Primary Health Care: An Introduction
to Qualitative Data Analysis. Nottingham, UK: Trent
Focus, 1991.
22. McGee HM, O’Boyle CA, Hickey A: Assessing the
quality of life of the individual: the SEIQoL with a
health and with a gastroenterology unit population.
Psychol Med 1991;21;749–759.
23. Bradley C, Todd C, Gorton T, Symonds E, Martin A,
Plowright R: The development of an individualized
questionnaire measure of perceived impact of dia-
betes on quality of life: the ADDQoL. Qual Life Res
1999;8:79–91.
24. Wolfe AK, Bjornstad DJ, Russell M, Kirner ND: A
framework for analyzing dialogues over the accept-
ability of controversial technologies. Sci Technol Hu-
man Values 2002;27:134–159.
Address reprint requests to:
Prof. S.P. Newman, D.Phil.
Centre for Behavioural & Social
Sciences in Medicine
Royal Free & University College Medical School
Charles Bell House
67-73 Riding House Street
London, W1W 7EJ, UK
E-mail: s.newman@ucl.ac.uk
ACCEPTABILITY OF CGM DEVICES 101
STEED
AU1
All Latin phrases roman in this
journal.
