Evaluation of AllergiSense Smartphone Tools for Adrenaline Injection Training

Abstract

Anaphylaxis is an increasingly prevalent life-threatening allergic condition that requires people with anaphylaxis and their caregivers to be trained in the avoidance of allergen triggers and in the administration of adrenaline auto-injectors. The prompt and correct administration of auto-injectors in the event of an anaphylactic reaction is a significant challenge in the management of anaphylaxis. Unfortunately, many people do not know how to use auto-injectors and either fail to use them or fail to use them correctly. This is due in part to deficiencies in training and also to the lack of a system encouraging continuous practice with feedback. Assistive smartphone healthcare technologies have demonstrated potential to support the management of chronic conditions such as diabetes and cardiovascular disease, but there have been deficiencies in their evaluation and there has been a lack of application to anaphylaxis. This paper describes AllergiSense, a smartphone app and sensing system for anaphylaxis management, and presents the results of a randomized, controlled, pre-post evaluation of AllergiSense injection training and feedback tools with healthy participants. Participants whose training was supplemented with AllergiSense injection feedback achieved significantly better practiced injections with 90.5% performing correct injections compared to only 28.6% in the paper-only control group. In addition, the results provide insights into possible self-efficacy failings in traditional training and the benefits of embedding self-efficacy theory into the technology design process.

Full text

JBHI Ref: JBHI-00374-2015
1

Abstract--Anaphylaxis is an increasingly prevalent life-
threatening allergic condition that requires people with
anaphylaxis and their caregivers to be trained in the avoidance of
allergen triggers and in the administration of adrenaline auto-
injectors. The prompt and correct administration of auto-
injectors in the event of an anaphylactic reaction is a significant
challenge in the management of anaphylaxis. Unfortunately,
many people do not know how to use auto-injectors and either
fail to use them or fail to use them correctly. This is due in part to
deficiencies in training and also to the lack of a system
encouraging continuous practice with feedback. Assistive
smartphone healthcare technologies have demonstrated potential
to support the management of chronic conditions such as
diabetes and cardiovascular disease, but there have been
deficiencies in their evaluation and there has been a lack of
application to anaphylaxis. This paper describes AllergiSense, a
smartphone app and sensing system for anaphylaxis
management, and presents the results of a randomized,
controlled, pre-post evaluation of AllergiSense injection training
and feedback tools with healthy participants. Participants whose
training was supplemented with AllergiSense injection feedback
achieved significantly better practiced injections with 90.5%
performing correct injections compared to only 28.6% in the
paper-only control group. In addition, the results provide
insights into possible self-efficacy failings in traditional training
and the benefits of embedding self-efficacy theory into the
technology design process.
Index Terms--Assistive Technology, Pervasive Healthcare,
Anaphylaxis Management, Smartphone Wireless Sensing, Self-
Efficacy, Self-Management.
I. INTRODUCTION
NAPHYLAXIS is a serious allergic reaction that is rapid
in onset and can cause death [1, p. 392]. Its prevalence
has dramatically increased in recent years [2] with an
This paper was submitted for review on June 11th 2015. This research was
supported by the University of Birmingham UK, the Mexican Council of
Science and Technology (Conacyt) and the Mexican Secretariat of Public
Education (SEP). This study was funded by The Anaphylaxis Campaign UK's
Small Grant Scheme (04-13-LHM) and approved by the University of
Birmingham Ethics Committee (ERN_13-1496).
L. U. Hernandez-Munoz is with Birmingham City University at
Millennium Point, Curzon Street, Birmingham, B4 7XG. (Correspondence
e-mail: Luis.Hernandez-Munoz@bcu.ac.uk).
S. I. Woolley, T. Collins and L. Diwakar are with The University of
Birmingham. D. Luyt, G. Stiefel and K. Kirk are with the University Hospitals
of Leicester NHS Trust. N. Makwana is with Sandwell and West Birmingham
Hospitals NHS Trust. C. Melchior is with Heart of England NHS Foundation
Trust, T. C. Dawson is with the Worcestershire Acute Hospitals NHS Trust
and G. Wong and L. Diwakar are with the University Hospitals Birmingham
NHS Foundation Trust.
estimated lifetime prevalence of 0.05-2% [3]-[5].
Anaphylactic reactions can occur rapidly after ingestion,
inhalation or contact with an allergen that may be a food,
prescription drug, insect sting, or a substance such as latex [6].
Foods are the most common allergens for children,
adolescents and young adults while non-food allergens are
more common for older adults [7]. Children frequently
develop tolerance to milk, egg, soy and wheat allergens by
school age, however, allergies to nuts and shellfish are more
likely to be lifelong [8].
The first-line treatment for an anaphylactic reaction is the
immediate administration of adrenaline (epinephrine) given
via a pre-loaded Adrenaline Auto-Injector (AAI) into the outer
thigh and an ambulance must be called [9]. If symptoms do
not improve in 5-10 minutes a second injection is advised
[10], [11]. Correct use of the most commonly prescribed AAI
brands, EpiPen® and Jext®, requires the correct completion of
four steps: 1) safety cap removal, 2) delivery to the thigh, 3)
holding in place for 10 seconds and 4) massaging the injection
site for 10 seconds. Empty needleless AAI trainer devices are
available for the purpose of practicing injections.
The management of anaphylaxis requires allergen avoidance
and emergency preparedness [7], [9], [12], [13]. Allergen
avoidance includes the inspection of food ingredient labeling
[12], [14], for example, a chocolate bar may have
precautionary advisory labeling such as “may contain nuts”;
and awareness of contamination risks, for example, if food is
cut with a knife that has been in contact with an allergen.
Emergency preparedness includes knowing how to recognize
anaphylaxis symptoms, training in the use of AAIs [15] and
having an emergency allergy action plan [9], [10], [16], [17].
The contribution of this paper is three-fold: i) it presents
AllergiSense, a prototype smartphone app and sensing system
for emergency preparedness in anaphylaxis management; ii)
provides laboratory evidence, for an injection feedback tool,
of significantly improved practice injection skills; and iii)
provides proof-of-concept evidence to support a case for
future clinical trials implementing the technology with both
physicians and patients inside and outside the clinic.
AllergiSense design and evaluation was motivated by the
fact that the correct use of AAIs is significant in anaphylaxis
management [18] and because there are widely reported
failures in the provision of appropriate training and failures in
AAI injection procedure [19]-[29]. For example, Brown et al.
Evaluation of AllergiSense Smartphone Tools
for Adrenaline Injection Training
L. U. Hernandez-Munoz, Graduate Student Member, IEEE, S. I. Woolley, Senior Member, IEEE,
D. Luyt, G. Stiefel, K. Kirk, N. Makwana, C. Melchior, T. C. Dawson, G. Wong, T. Collins and
L. Diwakar
A

JBHI Ref: JBHI-00374-2015
2
[20] reported that only 15 out of 100 mothers could correctly
demonstrate AAI use despite a prior demonstration. Arkwright
and Farragher [30] found that 69% of the parents of food
allergic children attending a UK clinic were unable to use their
AAIs, did not have them available or did not know when to
administer them. In a randomized study with 343 previously-
trained Canadian school staff, Nguyen-Luu et al. [27] found
that only 26.3% of participants who had been fully informed at
recruitment about the AAI assessment could demonstrate
correct AAI use. And only 15.8% of the participants who were
not fully informed about the assessment could correctly
demonstrate AAI use. Physicians have also been shown to
lack AAI skills [18], [29]. For example, Mehr et al. [31] have
suggested that insufficient knowledge from prescribing
physicians was a contributing factor to the failure of parents
and children activating the device correctly. The authors
recruited 100 pediatric hospital physicians (including
residents, registrars and consultants), half of whom had
already prescribed AAIs. Only 2% of their demonstrated
injections were assessed as fully correct, improving to 41%
after they reviewed AAI instructions but still one in five self-
injected their own thumbs. Similarly, Arga et al. [18] found in
a study with 151 general pediatrics physicians, residents and
consultants that only thirty-five (23%) were able to
demonstrate correct AAI use, improving to 74% after training
and practice. Observing deficiencies on retesting six months
later, the authors [18] recommended repetition of education.
The consensus in the clinical literature is that training
should be improved and should ensure correct injection
techniques are used, and that training should be continuous,
monitored and assessed by allergy specialists so that skills are
refreshed and maintained [9], [19], [25], [26], [28], [32], [33].
Advances in pervasive and assistive health technology
research, evident in the expanding literature, have contributed
toward improved management of other chronic health
conditions such as diabetes, asthma, cardiovascular diseases
[34]-[36] and mental illness [37], but anaphylaxis has been
neglected [38]. A search of online app stores (Android and
Apple - August 2014) returned nine information-giving
smartphone anaphylaxis apps and services, most of which
were produced by support groups [39] and AAI manufacturers
[40], [41]. For example, apps with instructions for using a
manufacturer’s AAI, text reminder services about AAI expiry
dates or text alerts about allergen contamination in the food
supply chain. As with other healthcare apps, there is lack of
reported evaluation in the literature [42], [43]. In addition,
there are no systems or apps providing feedback on injection
performance or encouraging maintenance of AAI skills.
We aimed to investigate whether adrenaline injection
training using AllergiSense to supplement traditional paper
documents, may provide improved injection training skills and
better self-efficacy levels in comparison with adrenaline
injection training using paper documents alone. The following
section presents the design of AllergiSense and then section III
describes its implementation. Section IV explains how
AllergiSense injection feedback tools were evaluated and
section V presents the results of the evaluation. Finally section
VI provides a discussion of results and section VII outlines the
conclusions of this paper.
II. ALLERGISENSE DESIGN
A. Design based on self-efficacy theory
The AllergiSense design and its evaluation were grounded
in self-efficacy theory [44]. Self-efficacy theory is central to
social cognitive theory. It refers to one’s belief in one’s ability
and it is a major predictor of self-management outcomes and a
contributor to performance ibid.
Self-efficacy is modified by four information sources [44]:
Enactive experience - experiencing attainment through practice
and mastery. Vicarious experience - modeling others. Social
persuasion - encouragement or discouragement from others.
Physiological states - interpretation of one’s physiological
responses as indicators of personal competency.
Health-promotion interventions based on social and
behavioral science theories are more effective than those
without a theoretical base [45]. Though technology
evaluations may incorporate assessments of self-efficacy, the
majority of reported studies are not theoretically based on
such. In a review of mobile devices for healthcare and
behavioral change, Free et al. [35] observed only seven of
twenty-six (26.9%) behavioral change studies reported using
behavioral change theories to underpin their intervention.
B. Design methodology
The ambition of AllergiSense was to support anaphylaxis
self-management. The design and evaluation was informed by
technological prototyping [38], [46], participatory design [47]
and a multi-stage methodology enriched with embedded self-
efficacy sources.
The motivation for incorporating participatory design was
to evolve an improved design from a deeper understanding of
anaphylaxis management needs and from different
perspectives of users and stakeholders. The participatory
design process, shown in Fig. 1, comprised two workshop
events with expert clinical participants, caregivers for
individuals at risk of anaphylaxis, an adult with a history of
anaphylaxis, and system designers. Participants identified two
main anaphylaxis management contexts: emergency and
everyday life. They also identified specific management needs
including help to educate others, support for AAI use and AAI
Fig. 1. Participatory design methodology embedded with self-efficacy
components.

JBHI Ref: JBHI-00374-2015
3
management, and help with emergency situations. Participants
created paper interface prototypes of tools to support these
needs using the PICTIVE (Plastic Interface for Collaborative
Technology Initiatives through Video Exploration)
participatory design approach [47].
Fig. 2. AllergiSense mock-up prototype screenshots: a) Example of interfaces
choices presented to participants: Button vs icon menu styles; b) Examples of
participant suggestions (relocation of cancel button to avoid pressing it by
mistake; ticks and crosses over emoticons for injection feedback).
In addition to tools explicitly suggested in the participatory
design process, an AAI injection training tool was created as a
vehicle to increase self-efficacy: encouraging mastery (via
performance support) and providing persuasion (via
feedback). A simplified injection force-sensing tool had been
developed in earlier technology prototyping [38]. The tool was
enhanced and a new user interface was included in mock-ups
presented at the participatory design session 2 for participant
feedback. Interfaces for the full set of tools were mocked-up
using Balsamiq® software for higher fidelity user interface
prototyping. Fig. 2. shows a) an example of one of the
interface design choices presented to participants and b) shows
examples of participant preferences. As well as the bold and
strictly consistent use of function coloring (red for emergency
and green for everyday function), these examples demonstrate
ways in which AllergiSense design considerations varied from
those of a generic app. With effective and error-free use
identified as a top priority; clarity, simplicity and consistency
were essential to the interface design. For example, of the
choices shown in Fig. 2a, participants preferred the simple
button menu style with sympathetic coloring. In Fig. 2b,
participant annotations show a preference for the emergency
button (and cancel emergency button) at the top of the screen
which they said would improve visibility and better avoid
pressing it by mistake, and, as shown, ticks and crosses were
preferred for clarity in the injection training feedback tool.
III. ALLERGISENSE IMPLEMENTATION
A. AllergiSense mobile application tools
The AllergiSense design and the information content were
subject to clinical inspection prior to the production of the
final prototype used in the evaluation. AllergiSense was
implemented in an Android Smartphone. Example screenshots
are shown in Fig. 3. For everyday life, the AllergiSense tools
include a list of AAI expiry dates with reminders, videos about
anaphylaxis and symptoms, a step-by-step trainer tool
showing how to use an EpiPen® AAI, and the AAI trainer tool
to provide feedback on the correctness of sensed injection
steps. The AAI expiry date tool (Fig. 3d) requires users to
initially ‘register’ the serial number ID and expiry date of each
of their AAIs. The shelf life of EpiPen® and Jext® AAIs is 18
months. In part, this tool is equivalent to the alert services
provided by these manufacturers which send email or SMS
text messages at four months and two months prior to expiry
and again one day after expiry. But, as suggested by our
design participants, this AllergiSense tool has additional
functionality, for example, it stores the usual location of each
AAI and provides the number of days before each expires.
Green, yellow and red emoticons also summarize the AAI
expiry states, namely, "OK", "nearing expiry" and "expired",
respectively.
For emergency scenarios, AllergiSense tools include a
single-screen emergency 'what to do' list and step-by-step AAI
instructions. In addition, AllergiSense emergency messaging
tools can send text messages to predefined numbers
identifying the user’s GPS location, and emergency services
can be contacted with the touch of a button.
B. AllergiSense sensing unit
Fig. 4 shows the sensing unit mounted on an AAI trainer
device. It was encased in a slim plastic cover and comprised
an Arduino "Pro mini" microcontroller, a 3-axis
accelerometer, a push button sensor (to detect removal of the
safety cap), a Bluetooth™ transceiver and a coin cell battery.
The role of the sensing unit was to detect removal of the
safety cap and to collect acceleration data. The accelerometer
sensor unit was configured to sample X, Y and Z acceleration
channels at 70 Hz. This sampling rate was empirically selected
as sufficiently high for injection sensing fidelity and
sufficiently sustainable in terms of battery life. All data were
transmitted to the smartphone using a Serial Port Profile (SPP)
and used by the injection feedback training tool to determine
if the safety cap had been removed, if the injector was held the
right way around, if a 'swing and jab' was performed and if the
trainer was held in place for 10 seconds.

JBHI Ref: JBHI-00374-2015
4
a) b) c)
d) e) f)
g) h) i)
Fig. 3. AllergiSense screenshots: a) Initial screen; b) Emergency and everyday life menu buttons; c) Everyday life tools menu; d) AAI expiry dates list;
e) Information menu; f) Injection step-by-step instructions; g) Injection training questions; h) Injection feedback screen (provided after pressing the
button 'Get score' in 3g); i) Emergency tools.

JBHI Ref: JBHI-00374-2015
5
The removal of the safety cap was detected directly by the
sensor underneath the cap. Two action classes were used to
identify the injection steps: “swinging and jabbing” (when
delivering the injection) and “still” (when holding the trainer
injector in place). A third class, “moving”, was defined as any
other action. These classes were defined by accelerometer
training data from twelve correctly performed injections
provided by an allergy clinician. All the clinician’s injections
were performed with the right hand (the clinician’s dominant
hand). Six were performed while standing and six while
sitting.
Classification was performed using a J48 binary decision
tree using accelerometer data features that included the mean,
standard deviation, maximum and minimum values, difference
between the maximum and minimum values, and the average
distance from the mean. J48 is an open source Java
implementation of the C4.5 decision tree algorithm in the
WEKA data mining application. This algorithm was chosen
for ease of implementation and its robust performance in
testing. A correct injection is identified as a sequence of steps
in the following order: “moving”, “swinging and jabbing”,
“still” and “moving”. The results of testing with participant
data were: “swinging and jabbing” was classified with an
accuracy of 81%, precision of 96% and recall of 83%
(F-measure = 89%); “still” was classified with an accuracy of
81%, precision of 83% and recall of 91% (F-measure = 87%).
C. AllergiSense injection feedback tool
The AllergiSense injection feedback tool provides out-of-
six marks for practiced injections as depicted in Fig 3h. The
tool assesses injection site and massage time via two questions
with randomly located answers in pull-down menus (shown in
Fig. 3g), and the other four assessments (cap removed, injector
the right way around, swing and jab, and held in place for 10
seconds) are assessed automatically via the data
communicated from the sensing unit.
If users provide incorrect responses or perform erroneous
actions these are marked as incorrect as depicted in Fig. 3h,
the out-of-six score is deducted accordingly and informative
recommendations are provided in a subsequent screen if the
user presses the 'Recommendations' button. The
recommendation explains how to improve a specific step of
the injection and encourages another injection training
attempt.
IV. ALLERGISENSE EVALUATION
The AllergiSense adrenaline injection training tool was
evaluated with a three-arm, pre-post (two-week), randomized
controlled study with sixty-three healthy participants recruited
from the University of Birmingham, UK.
The main hypothesis of this evaluation was that using
AllergiSense (in addition to traditional training using
information leaflets) would enhance adrenaline injection
training skills compared to traditional instruction using
information leaflets alone. The primary aim of the evaluation
was an assessment of the effect of different training materials
on practiced adrenaline injection skills. The secondary aim
was to evaluate participants’ self-reported AAI self-efficacy,
workload, system usability, system usefulness, ease-of-use and
attitudes towards its use.
The training provided was clinically approved and the
procedure overseen by an expert clinical collaborator.
A. Statistics
A Shapiro-Wilk test was used to test if results were
samples of a normally distributed population (Significance
level = 0.05) [48]. Parametric t-tests and ANOVA test were
used on normally distributed results; Friedman's Rank and
Mann-Whitney (U) tests for results not normally distributed
and chi-squared test (χ2) for comparing frequencies of data.
The statistical tests were undertaken using SPSS® version 20.
B. Participants
Sixty-three student and staff participants aged between 18-
60 were recruited via email invitation from the University of
Birmingham, UK. All participants reported carrying and using
mobile phones. Participants were block randomized into three
groups of twenty-one participants. The groups comprised
participants with broadly equivalent smartphone experience in
terms of smartphone usage and number of apps used, and with
similar average age and gender balance, and all participants
were right-handed. Individuals known to be at risk of
anaphylaxis and their caregivers were excluded from the study
(their recruitment would have required extensive National
Health Service ethical permissions; future approval for testing
of new technology with patients would be more likely in the
event of positive outcomes from testing with healthy
participants).
C. Assessment of performance and administered
questionnaires
The assessment of AAI performance was based on the
four-step marking scheme used in other studies [19], [29],
which, in turn, were based on the steps recommended by the
EpiPen® AAI manufacturer [49] which are:
1. “Remove the blue safety cap.
2. ’Swing and jab’ the orange tip of the AAI trainer
against the outer thigh until it 'clicks'.
3. Hold firmly against the thigh for 10 seconds.
4. Remove the auto-injector from the thigh. The orange
tip will extend to cover the needle and massage the
injection area for 10 seconds.”
Fig. 4. AllergiSense sensing unit mounted on an EpiPen® AAI trainer
device.

JBHI Ref: JBHI-00374-2015
6
The AllergiSense system separates step 2 into two by
i) sensing “swing and jab” and ii) explicitly asking the user to
select the correct injection site from a randomly ordered list.
In addition, AllergiSense senses for the injector being held the
right way around. This means that while AllergiSense assesses
the four step injection performance, it reports on-screen out of
six rather than out of four (Fig. 3h).
The performance of all participants’ adrenaline injections
was evaluated via video observation of the four recommended
steps. An inter-rated test with an independent researcher was
carried out with a random sample of injections
(Cohen’s Kappa > 0.8). Injection step differences were
discussed with, and verified by, the independent researcher
using the recorded video and sensor data from the
AllergiSense sensing unit.
All the three groups used the same AllergiSense sensing
unit depicted in Fig. 4. All sensor data for all participants in all
groups was logged and recorded during the experimental
sessions. Data from the paper-only group and the AllergiSense
without feedback group were recorded via HyperTerminal for
research records. While the data of the AllergiSense group
were recorded in the smartphone. Participants in the
AllergiSense group were the only people that received
feedback about their training injections.
Workload and self-reported usability. NASA TLX [50] and
System Usability Scale (SUS) [51] questionnaires were used
for evaluation of workload and self-reported usability,
respectively. NASA TLX quantifies workload component
levels of mental, physical and temporal demands. The SUS
questionnaire provides a measure of perceived usability,
covering aspects of acceptance, need for support, training and
system complexity [52], [53].
Self-efficacy. A self-efficacy questionnaire for adrenaline
injection was created using eleven-point (0: Not at all
confident – 10: Totally confident) scale responses as
recommended by Bandura [54]. The questionnaire comprised
statements relevant to the use of AAIs in training and
emergencies for participants to rate. For example, “I am
confident that I can correctly use an auto-injector trainer in a
practice session.”, “I am confident I can apply the correct
force when injecting”, “I am confident I can identify the
correct injection site”, “I am confident I can correctly use an
auto-injector in an allergic emergency” and “I am confident
that I would inject correctly in an emergency even if I was
very anxious”. The selection and phrasing of the questions
was first reviewed by allergy clinical collaborators and
assessed by eighteen allergy specialists.
Usefulness, ease-of-use and attitudes towards use. Self-
reported measures of usefulness, ease of use and willingness
regarding use were collected from technology acceptance
questionnaires [55].
D. Materials
Subsequent to clinically approved training (i.e., allergy
specialist’s videos about anaphylaxis and EpiPen® use),
participants were randomly assigned to one of the three
following groups.
Paper (traditional care information with paper leaflets
documentation). Participants in all groups received a paper
copy of the EpiPen® AAI instruction leaflet (the instructions
for use provided in the EpiPen® AAI patient information).
This document provides information about injector use and
step-by-step pictures for each of the four injection steps.
Participants in the paper-only (control) group received only
this information. Participants in the other groups had this
material supplemented with AllergiSense materials as
described below.
AllergiSense without feedback. Participants in this group
received the AllergiSense smartphone system without the
injection practice feedback functionality, i.e., AllergiSense
without the out-of-six injection practice feedback. Thus
participants with AllergiSense without feedback were
provided with the paper instructions (the same as the control
paper group) supplemented with smartphone video (an
instructional Epipen® AAI video produced by the
manufacturer and available online on the EpiPen® AAI
website) and an AAI step-by step instruction tool (text and
pictures as per paper steps depicted in Fig. 3f).
AllergiSense. This was the complete AllergiSense
smartphone system using the sensing unit connected to the
AllergiSense smartphone and providing out-of-six injection
feedback. Thus, participants in this group were provided paper
instructions (the same as the control paper group)
supplemented with smartphone AAI step-by-step instructions
(Fig. 3f) and an AAI usage video (the same as the
AllergiSense without feedback) and the mark out-of-six
injection feedback (Fig. 3h).
E. Experimental procedure
The experiment comprised two sessions, two weeks apart.
In session one, participants were randomly allocated to one of
the three groups. All participants received the same clinically
approved training with videos of an allergy specialist using an
EpiPen® AAI trainer. Participants were then asked to
demonstrate an injection of adrenaline with the trainer device
(Demonstration 1), and were then provided with one of three
different training materials described earlier: paper-only,
AllergiSense without feedback or AllergiSense. Participants
were then required to practice three injections using their
allocated training materials before completing a demonstration
injection (Demonstration 2). In session two, two weeks later,
participants were recalled to demonstrate their injection skills
(Demonstration 3) then practice three injections using their
allocated training material before completing a final
demonstration injection (Demonstration 4).

JBHI Ref: JBHI-00374-2015
7
Only the participants in the AllergiSense group received
feedback on their injection performance (from the injection
feedback training tool - Fig. 3h). All other participants
received no feedback on their injections until they were
provided with an account during the experimental debrief at
the end of session two. None of the participants reported extra
training between sessions.
V. RESULTS
Table I shows the number of participants in each group that
correctly completed the four injection steps. Only 28.6% of
the paper-only group correctly completed all four injection
steps in their final demonstration vs. 66.7% for AllergiSense
without feedback and 90.5% for AllergiSense. Although more
people in the AllergiSense group performed all steps correctly
after the initial training (i.e., in Demonstration 1), there were
no significant differences between groups: 5 vs 4 (p = 0.707),
5 vs 7 (p = 0.495) and 4 vs 7 (p = 0.242). Similarly, after
training in session 1 (i.e., in Demonstration 2) although more
people in the AllergiSense group correctly completed all the
steps, there were no significant differences between the
groups: 5 vs 9 (p = 0.19), 5 vs 10 (p = 0.107) and 9 vs 10
(p = 0.757). However, after training in session 2 (i.e., in
Demonstration 4) significantly more people in the
AllergiSense and AllergiSense without feedback groups
completed the four steps correctly compared to the control
(paper-only) group: 6 vs 19 (p < 0.001) and 6 vs 14
(p = 0.013) respectively, while the difference between
AllergiSense without feedback and AllergiSense showed a
trend towards significance: 14 vs 19 (p = 0.060). The
AllergiSense group improved significantly after training in
session 2, from 9 to 19 of 21 participants injecting without
error (p = 0.013), and the AllergiSense without feedback
group showed a trend towards significance: from 8 to 14 of 21
participants injecting without error (p = 0.064).
TABLE I
PRIMARY OUTCOME: NUMBER OF PEOPLE CORRECTLY COMPLETING THE FOUR
INJECTION STEPS
Group
Session 1
Session 2
Demonstration
1
(after
watching
clinical video)
Demonstration
2
(after training)
Demonstration
3
(after two
weeks)
Demonstration
4
(after
training)
1.Paper-only
5
5
8
6
(23.8 %)
(23.8 %)
(38.1 %)
(28.6 %)
2.AllergiSense
without
feedback
4
9
8
14
(19.0 %)
(42.9 %)
(38.1 %)
(66.7 %)
3.AllergiSense
7
10
9
19
(33.3 %)
(47.6 %)
(42.9 %)
(90.5 %)
In contrast, the paper-only group actually deteriorated in
session 2: from 8 to 6 of 21 participants injecting correctly,
and across the four demonstrations there was no significant
change in this group’s injection ability despite the training
opportunities (p > 0.05).
For the two AllergiSense groups the number of errors made
decreased with training. The total number of injection errors
from all four demonstrations of the three groups was 225
(from a theoretical maximum of 1008 errors = 63[participants]
× 4[possible errors] × 4[demonstrations]). Only 3.1% of all
errors involved a failure to remove the safety cap and all of
these occurred in Demonstration 1. Not massaging the
injection site for 10 seconds comprised 52.9% of all errors, not
injecting with sufficient force comprised 24.9% and not
holding the AAI trainer in place for 10 seconds comprised the
remaining 19.1% of all errors
In Demonstration 2 more participants in the paper-only
group injected with sufficient force in comparison with the
AllergiSense groups, but the difference was not significant
(p > 0.05). However, at the end of the study (Demonstration 4)
both AllergiSense groups made significantly less errors in this
step than the paper-only group. More participants in both
AllergiSense groups held the AAI trainer in place for 10
seconds in all four of their demonstrations, compared with the
paper-only group. After training with their allocated material,
in Demonstration 2 and Demonstration 4, more people in both
AllergiSense groups massaged the injection site for 10
seconds and made significantly less errors in this step at the
end of the two-week study in comparison with the paper-only
group. TABLE II
SECONDARY OUTCOMES: SELF-EFFICACY, USEFULNESS, EASE-OF-USE,
ATTITUDES TOWARDS USE, SYSTEM USABILITY AND WORKLOAD
Group 1
Paper-only
Group 2
AllergiSense
without
feedback
Group 3
AllergiSense
Self-efficacy after
Demonstration 1
(session 1)
Average
score
7.5
7.6
7.1
Standard
deviation
1.4
1.2
1.4
Self-efficacy after
Demonstration 2
(session 1)
Average
score
8.5
8.6
8.5
Standard
deviation
1.2
1.0
0.9
Self-efficacy after
Demonstration 3
(session 2)
Average
score
8.5
8.7
8.4
Standard
deviation
1.4
1.1
1.0
Usefulness
Average
score
5.1
6.1
6.2
Standard
deviation
1.5
0.6
1.0
Ease-of-use
Average
score
4.7
6.2
6.0
Standard
deviation
1.5
0.6
1.0
Attitudes towards
use
Average
score
5.1
6.1
6.1
Standard
deviation
1.1
0.6
0.7
System Usability
Scale (SUS)
Average
score
68.5
86.3
82.7
Standard
deviation
14.5
9.0
15.9
Workload
(NASA TLX)
Average
score
34.1
31.9
31.5
Standard
deviation
17.1
13.9
12.2
Table II shows the questionnaire results for self-efficacy,
usefulness, ease-of-use, attitudes towards use, system usability
and workload. Self-efficacy differences within groups were
seen after training with their allocated material in session 1
(From Demonstration 1 to Demonstration 2). The self-efficacy
of the paper-only group increased from 7.5 to 8.5 (p < 0.001),
the AllergiSense without feedback group increased from 7.6 to
8.6 (p < 0.001) and the AllergiSense group increased from 7.1
to 8.5 (p < 0.001). Self-efficacy remained high for the three

JBHI Ref: JBHI-00374-2015
8
groups for two weeks, and no significant differences were
found between the three groups (p > 0.05).
After using their allocated material in session 1 (after
Demonstration 2), participants in the AllergiSense and
AllergiSense without feedback groups reported significantly
higher average scores for the usefulness, the ease-of use and in
the willingness to use their training materials compared to the
paper-only group as follows. Usefulness: 5.1 (paper) vs 6.1
(AllergiSense without feedback) (p = 0.012); 5.1 (paper) vs
6.1 (AllergiSense) (p = 0.001); Ease-of-use: 4.7 (paper) vs 6.2
(AllergiSense without feedback) (p = 0.001); 4.7 (paper) vs
6.0 (AllergiSense) (p = 0.005); Willingness towards use: 5.1
(paper) vs 6.1 (AllergiSense without feedback) (p < 0.001) and
5.1 (paper) vs 6.1 (AllergiSense) (p < 0.001).
In addition, both AllergiSense groups reported significantly
higher system usability scores (SUS), after Demonstration 2,
than the paper-only group: 68.5 (paper) vs 86.3 (AllergiSense
without feedback) (p < 0.001); 68.5 (paper) vs 82.7
(AllergiSense) (p = 0.001). While the workload, reported after
Demonstration 4, was not significantly different between
groups (p = 0.991).
VI. DISCUSSION OF RESULTS
While the results of small studies should, necessarily, be
interpreted cautiously, the results presented here provide a
measure of evidence toward the hypothesis that smartphone
tools supplementing traditional instruction paper leaflets could
improve adrenaline injection training skills.
The improved results for AllergiSense could be a
consequence of improved training from the explicit and
purposeful reinforcement of self-efficacy via mastery,
vicarious and social experiences embedded within videos,
step-by-step instructions and visual feedback. Where, in
contrast, paper instructions provide only limited modeling
opportunities from text and pictures. There was no significant
improvement in the performance of the paper-only group
throughout the study.
The results appear to support other reports in the literature
[19]-[29] regarding the inadequacy of the current approach to
adrenaline injection education (i.e., expert explanation and
AAI demonstration). Current instruction, where the use of
AAIs is just demonstrated does not include provision for
feedback, nor encouragement nor support of continuous
practice. This was observed after Demonstration 1 (after the
clinically-approved training) when, at best, only one third of
people in the three groups could correctly complete all four
steps of the injection (23.8%, 19% and 33.3% for control,
AllergiSense without feedback and AllergiSense,
respectively). These very low results concur with other
extremely poor findings reported in the literature.
One interesting and unexpected result was the significant
increase in self-efficacy in the paper-only group after first use
of their material for training (after Demonstration 2). This
increase was less than the increase for the AllergiSense groups
but not significantly so. The paper-only group retained their
increased self-efficacy throughout the study despite the lack of
any significant improvement in their performance. This was
exemplified at the end of session 2 by one paper-only
participant who had made no correct injection demonstrations
at all, but expressed surprise for each when informed of the
results. Bandura [56] has reported that improved self-efficacy
in the absence of improved performance indicates a problem
in the system. Perhaps then, the experiment revealed
something of the problem with the current system, i.e., that in
the absence of monitoring and feedback people have elevated
self-efficacy based on incorrect assumptions about their
mastery skills. This could have several consequences, not least
the lack of motivation for continuous practice.
Secondary outcome results showed that participants
reported no significant differences in workload for the three
different training materials. Interestingly, compared to the
paper-only group both AllergiSense groups scored
significantly better for usefulness and ease-of-use of their
materials and also reported significantly more willingness
towards use. Additionally, average self-reported usability
scores (SUS) for AllergiSense were very positive. The
paper-only participants reported, according to Bangor et al.'s
adjective scale [57, p. 592], a marginally acceptable SUS
score of 68.45 (between OK and good), while the SUS score
for AllergiSense without feedback was 86.31 and was 82.74
for AllergiSense (both between good and excellent).
Results showed that adrenaline injection self-efficacy
improved after the first training session and then was not
significantly different two weeks later. Perhaps if participants
had been recalled six weeks or six months later these self-
efficacy results might be substantially different. Further work
involving longer-term studies is recommended to investigate
how self-efficacy and adrenaline injection skills attenuate over
time and how these are impacted by the training materials
used.
This research was limited to short-term evaluations with
healthy participants. Thus, in every aspect of the work
presented here there is scope for further contribution. Children
are most affected by anaphylaxis and the most common
allergen, peanuts, is not generally outgrown. This new
generation will need support in the management of their
anaphylaxis. We hope that the results presented here will
encourage further technology research and development in
support of anaphylaxis management. For example, further
work is also needed to populate solutions with content and
define tools aimed at supporting symptom recognition and
allergen avoidance. Further work could also consider the
issues of responsibility for the support and maintenance of the
technology and the information contained within it. In
addition, further work is needed for the creation and validation
of self-efficacy questionnaires for anaphylaxis management
and adrenaline injection and, importantly, much further work
is needed for evaluation of tools in longitudinal studies with
patients in and outside the clinic.

JBHI Ref: JBHI-00374-2015
9
VII. CONCLUSIONS
This paper provided experimental evidence supporting the
potential of smartphone tools and wireless sensors to
significantly improve AAI training skills, usefully supplement
traditional care paper information leaflets and positively
influence injection training performance and user’s self-
efficacy. The study was limited to a randomized, controlled,
pre-post intervention with healthy participants simulating
adrenaline injections with an AAI trainer, but still the results
provided valuable insights and proof-of-concept evidence to
support a case for future clinical trials implementing the
technology with both physicians and patients.
It was noted that participants in the control group, trained
with traditional care paper information leaflets alone, did not
improve their AAI performance and made persistent errors in
administration of the AAI throughout training practice.
Notwithstanding the poor AAI skills exhibited by the paper-
only group, the results revealed that their levels of self-
efficacy increased, despite being wholly incompatible with
their actual AAI skills. Whilst it is important to be cautious in
the interpretation of these data given the limited participant
numbers, this finding was interesting and unexpected. It may
provide an insight into deficits in AAI use. Incorrect
assumptions behind inappropriately elevated self-efficacy
could be a consequence of the lack of AAI training monitoring
and feedback and suggests that it is difficult to identify one’s
own errors and assess one’s own competence. This could have
several consequences, not least complacency regarding AAI
training and a lack of motivation for continuous practice.
The injection sensing implemented in AllergiSense
performed robustly throughout all evaluations presented here.
However, in a subsequent qualitative evaluation study in
which AllergiSense was provided to expert allergy physicians
and nurses, further improvements were identified. For
example, the expansion of the training data to include different
injection scenarios, such as injecting while lying down.
Improved sensing in realistic scenarios could also be useful in
prototyping new “smart” AAI designs with emergency AAI
sensing capability. Further research in support of anaphylaxis
management may have positive implications since people with
anaphylaxis and their caregivers are motivated more than most
to learn how to use AAIs, and carry smartphones because they
may need to make emergency calls, and so the technology
platform needed for an assistive healthcare solution is already
available.
VIII. ACKNOWLEDGMENT
The authors gratefully acknowledge the institutions that
funded and supported this research, the expertise and support
of the Anaphylaxis Campaign UK, and the clinicians and staff
at hospital trusts: University Hospitals of Leicester NHS Trust,
University Hospitals Birmingham NHS Foundation Trust,
Heart of England NHS Foundation Trust, Worcestershire
Acute Hospitals NHS Trust and Sandwell & West
Birmingham Hospitals NHS Trust.
IX. REFERENCES
[1] H. A. Sampson, A. Munoz-Furlong, R. L. Campbell, N. F. Adkinson, S.
A. Bock, A. Branum, S. G. A. Brown, C. A. Camargo, R. Cydulka, S.
J. Galli, J. Gidudu, R. S. Gruchalla, A. D. Harlor, D. L. Hepner, L. M.
Lewis, P. L. Lieberman, D. D. Metcalfe, R. O’Connor, A. Muraro, A.
Rudman, C. Schmitt, D. Scherrer, E. R. Simons, S. Thomas, J. P. Wood
and W. W. Decker, “Second Symposium on the Definition and
Management of Anaphylaxis: Summary report - Second National
Institute of Allergy and Infectious Disease/Food Allergy and
Anaphylaxis Network Symposium”, Annals of Emergency Medicine,
vol. 47, no. 4, pp. 373–380, 2006.
[2] F. E. R. Simons and H. A. Sampson, “Anaphylaxis epidemic: fact or
fiction?”, Journal of Allergy and Clinical Immunology, vol. 122, no. 6,
pp. 1166–8, Dec. 2008.
[3] P. Lieberman, C. A. Camargo, K. Bohlke, H. Jick, R. L. Miller, A.
Sheikh, and E. R. Simons, “Epidemiology of anaphylaxis: findings of
the American College of Allergy, Asthma and Immunology
Epidemiology of Anaphylaxis Working Group”, Annals of Allergy,
Asthma and Immunology, vol. 97, no. 5, pp. 596–602, 2006.
[4] P. Lieberman, “Epidemiology of anaphylaxis”, Current Opinion in
Allergy and Clinical Immunology, vol. 8, no. 4, pp. 316–320, 2008.
[5] F. E. R. Simons, "World Allergy Organization survey on global
availability of essentials for the assessment and management of
anaphylaxis by allergy-immunology specialists in health care
settings", Annals of Allergy, Asthma & Immunology vol. 104, no. 5,
pp. 405–412.2010.
[6] C. Editors, P. Lieberman, S. F. Kemp, J. Oppenheimer, J. N. Fink, P. A.
Greenberger, K. Dennis, J. Li, A. L. Sheffer, R. Solensky, and B. L.
Wolf, “The diagnosis and management of anaphylaxis: an updated
practice parameter”, Journal of Allergy and Clinical Immunology, vol.
115, no. 3 Suppl 2, pp. S483–523, Mar. 2005.
[7] F. E. R. Simons, L. R. F. Ardusso, M. B. Bilò, V. Dimov, M. Ebisawa,
Y. M. El-Gamal, D. K. Ledford, R. F. Lockey, J. Ring, M. Sanchez-
Borges, G. E. Senna, A. Sheikh, B. Y. Thong, and M. Worm, “2012
Update: World Allergy Organization Guidelines for the assessment and
management of anaphylaxis”, World Allergy Organization, vol. 12, no.
4, pp. 13–37, Aug. 2011.
[8] K. J. Allen, D. J. Hill, and R. G. Heine, “Food allergy i n childhood”,
Medical Journal of Australia, vol. 185, no. 7, pp. 394–400, 2006.
[9] NICE, “Anaphylaxis. Assessment to confirm an anaphylactic episode
and the decision to refer after emergency treatment for a suspected
anaphylactic episode”, 2011.
[10] Anaphylaxis101 (Mylan), “Anaphylaxis101 app,” October, 2012.
[Online]. Available:
https://play.google.com/store/apps/details?id=com.aurasma.skinned.ana
phylaxis101&hl=en_GB. [Accessed: 14-Jun-2014].
[11] A. Muraro, G. Roberts, A. Clark, P. A. Eigenmann, S. Halken, G. Lack,
A. Moneret-Vautrin, B. Niggemann, and F. Rancé, “The management
of anaphylaxis in childhood: position paper of the European academy
of allergology and clinical immunology”, Allergy, vol. 62, no. 8, pp.
857–71, Aug. 2007.
[12] S. Walker and a Sheikh, “Managing a naphylaxis: effective emergency
and long-term care are necessary”, Clinical and Experimental Allergy,
vol. 33, no. 8, pp. 1015–8, Aug. 2003.
[13] J. Wang, “Management of the patient with multiple food allergies”
Current Allergy and Asthma Reports, vol. 10, no. 4, pp. 271–277, 2010.
[14] T. Umasunthar, J. Leonardi-Bee, M. Hodes, P. J. Turner, C. Gore, P.
Habibi, J. O. Warner, and R. J. Boyle, “Incidence of fatal food
anaphylaxis in people with food allergy: a systematic review and meta-
analysis”, Clinical and Experimental Allergy, vol. 43, no. 12, pp. 1333–
41, Dec. 2013.
[15] F. E. R. Simons, “Anaphylaxis: Recent advances in assessment and
treatment”, The Journal of Allergy and Clinical Immunology, vol. 124,
no. 4, Elsevier Ltd, pp. 625–36; quiz 637–8, Oct-2009.
[16] American Academy of Allergy, Asthma & Immunology, “AAAAI,
emergency action plan” 2013. [Online]. Available:
https://www.aaaai.org/Aaaai/media/MediaLibrary/PDF
Documents/Libraries/Anaphylaxis-Emergency-Action-Plan.pdf.
[Accessed: 01-Dec-2013].
[17] BSACI, “BSACI Allergy Action Plan”, 2014. [Online]. Available:
http://www.bsaci.org/about/download-paediatric-allergy-action-
plans?EID=33016898&CID=5385780. [Accessed: 20-Jun-2014].
[18] P. Lieberman, R. A. Nicklas, J. Oppenheimer, S. F. Kemp, D. M. Lang,
D. I. Bernstein, J. a Bernstein, a W. Burks, A. M. Feldweg, J. N. Fink,

JBHI Ref: JBHI-00374-2015
10
P. a Greenberger, D. B. K. Golden, J. M. James, D. K. Ledford, A. L.
Sheffer, J. Blessing-Moore, L. Cox, D. a Khan, D. Lang, J. M. Portnoy,
C. Randolph, D. E. Schuller, S. L. Spector, S. Tilles, and D. Wallace,
“The diagnosis and management of anaphylaxis practice parameter:
2010 update”, Journal of Allergy and Clinical Immunology, vol. 126,
no. 3, pp. 477–80.e1–42, Sep. 2010.
[19] M. Arga, A. Bakirtas, F. Catal, O. Derinoz, K. Harmanci, C. H. Razi, S.
Ergöcen, M. S. Demirsoy, and I. Turktas, “Training of trainers on
epinephrine autoinjector use”, Pediatric Allergy and Immunology, pp.
1–4, Feb. 2011.
[20] J. Brown, D. Tuthill, M. Alfaham, and E. Spear, “A randomized
maternal evaluation of epinephrine autoinjection devices”, Pediatric
Allergy and Immunology, vol. 24, no. 2, pp. 173–7, Mar. 2013.
[21] S. K. Carlisle, P. A. Vargas, S. Noone, P. Steele, S. H. Sicherer, A.
Wesley-Burks, and S. M. Jones, “Food allergy education for school
nurses: a needs assessment survey by the consortium of food allergy
research”, Journal of School Nursing, vol. 26, no. 5, pp. 360–367, 2010.
[22] L. Diwakar, J. Heslegrave, A. G. Richter, A. P. Huissoon, and M. T.
Krishna, “Self-injectable adrenaline devices: is training necessary?”,
Journal of Investigational Allergology & Clinical Immunology, vol. 20,
no. 5, pp. 446–453, 2010.
[23] M. Gallagher, A. Worth, S. Cunningham-Burley, and A. Sheikh,
“Epinephrine auto-injector use in adolescents at risk of anaphylaxis: a
qualitative study in Scotland, UK”, Clinical & Experimental Allergy,
vol. 41, no. 6, pp. 869–877, 2011.
[24] H. M. Luckhurst, D. Tuthill, J. Brown, E. Spear, and J. Pitcher,
“Anapen, E piPen and Jext auto-injectors; assessment of successful use
after current training pa ckage”, Archives of Disease.in Childhood, vol.
98, no. Suppl 1, pp. A42–A43, Jun. 2013.
[25] C. Macadam, J. Barnett, G. Roberts, G. Stiefel, R. King, M. Erlewyn-
Lajeunesse, J.A. Holloway, and J. S. Lucas, “What factors affect the
carriage of epinephrine auto-injectors by teenagers?”, Clinical and
Translational Allergy, vol. 2, no. 1, p. 3, Jan. 2012.
[26] S. Mehr, M. Robinson, and M. Tang, “Doctor - how do I use my
EpiPen?”, Pediatric Allergy and Immunology, vol. 18, no. 5, pp. 448–
452, 2007.
[27] N. U. Nguyen-Luu, L. Cicutto, L. Soller, L. Joseph, S. Waserman, Y.
St-pierre, and A. Clarke, “Management of anaphylaxis in schools :
Evaluation of an epinephrine auto-injector ( EpiPen W ) use by school
personnel and comparison of two approaches of soliciting
participation” Allergy, Asthma and Clinical Immunology pp. 1–8,
2012.
[28] L. Noimark, J. Wales, G. Du Toit, C. Pastacaldi, D. Haddad, J.
Gardner, W. Hyer, G. Vance, C. Townshend, M. Alfaham, P. D.
Arkwright, R. Rao, S. Kapoor, a Summerfield, J. O. Warner, and G.
Roberts, “The use of adrenaline autoinjectors by children and
teenagers”, Clinical & Experimental Allergy, vol. 42, no. 2, pp. 284–
92, Feb. 2012.
[29] S. H. Sicherer, J. A. Forman, and S. A. Noone, “Use Assessment of
Self-Administered Epinephrine among Food-Allergic Children and
Pediatricians”, Pediatrics, vol. 105, no. 2, pp. 359–362, 2000.
[30] P. D. Arkwright and A. J. Farragher, "Factors determining the ability of
parents to effectively administer intramuscular adrenaline to food
allergic children". Pediatric allergy and immunology, vol. 17, no. 3, pp.
227-229, 2006.
[31] S. Mehr, M. Robinson, T. M. Doctor, S. Mehr, and M. Robinson,
“Doctor – How do I use my EpiPen ?”, Pediatric Allergy and
Immunology, no. 6, pp. 448–452, 2007.
[32] B.M. Patel, P. J. Ba nsal, and M. C. Tobin, “Management of anaphylaxis
in child care centers: evaluation 6 and 12 months after an intervention
program”, Annals of Allergy, Asthma & Immunology, vol. 97, no. 6,
pp. 813–5, Dec. 2006.
[33] A. J. Frew, “What are the ‘ideal’ features of an adrenaline (epinephrine)
auto-injector in the treatment of anaphylaxis?” Allergy, vol. 66, no. 1,
pp. 15–24, Jan. 2011.
[34] B. S. Fjeldsoe, A. L. Marshall, and Y. D. Miller, “Behavior change
interventions delivered by mobile telephone short-message service”,
American Journal of Preventive Medicine, vol. 36, no. 2, pp. 165–173,
Feb. 2009.
[35] C. Free, G. Phillips, L. Galli, L. Watson, L. Felix, P. Edwards, V. Patel,
and A. Haines, “The effectiveness of mobile-health technology-based
health behaviour change or disease management interventions for
health care consumers: a systematic review”, PLoS Med., vol. 10, no. 1,
p. e1001362, Jan. 2013.
[36] J. M. Belisario, “Smartphone and tablet self-management apps for
asthma”, The Cochrane Library, no. 11, 2013.
[37] G. Agnes, A. Muaremi, V. Osmani, G. Bahle, S. Ohler, G. Troster, O.
Mayora, C. Haring, P. Lukowicz. "Smartphone-based recognition of
States and state changes in bipolar disorder patients." IEEE Journal of
Biomedical and Health Informatics, 19, no. 1 (2015): 140-148.
[38] L. U. Hernandez-Munoz and S. I. Woolley, “Mobile Phone Tools with
Ambient Intelligence for the Management of Life-Threatening
Allergies,” in Human Aspects in Ambient Intelligence, vol. 8, T. Bosse,
D. J. Cook, M. Neerincx, and F. Sadri, Eds. Paris: Atlantis Press
(SpringeLinkr), 2013, pp. 153–172.
[39] Coventry University, “Anaphylaxis,” 2013. [Online]. Available:
https://itunes.apple.com/gb/app/anaphylaxis/id583861393?mt=8.
[Accessed: 15-Jun-2014].
[40] Jext (ALK-Abelló), “Jext app,” June, 2013. [Online]. Available:
https://play.google.com/store/apps/details?id=com.androidapp.alkabello
.jext_uk&hl=en_GB. [Accessed: 14-Jun-2014].
[41] Auvi-Q (Sanofi-Aventis U.S. LLC), “Auvi-Q app,” February, 2013.
[Online]. Available:
https://play.google.com/store/apps/details?id=com.sanofiaventis.AuviQ
Companion. [Accessed: 14-Jun-2014].
[42] B. J. Visser, D. A. Korevaar, and T. Nolan, “Mobile medical apps:
dangers and potential solutions", Journal of Telemedicine and Telecare,
pp. 1–2, Mar. 2013.
[43] M. McCartney, “How do we know whether medical apps work?”, BMJ,
no. 346, 2013.
[44] A. Bandura, “Self-efficacy: Toward a unifying theory of behavioral
change”, Psychological Review, vol. 84, pp. 191–215, 1977.
[45] K. Glanz and D. B. Bishop, “The role of behavioral science theory in
development and implementation of public health interventions”,
Annual Review of Public Health, vol. 31, pp. 399–418, Jan. 2010.
[46] L. U. Hernandez-Munoz, S. I. Woolley and L. Diwakar, “Pilot
Evaluation of Smartphone Technology for Adrenaline Injection
Training”, in The British Society for Allergy and Clinical Immunology
–BSCAI- Annual Meeting, 2014.
[47] M. J. Muller, “PICTIVE - An Exploration in Participatory Design”, in
Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, 1991, pp. 225–231.
[48] A. Field, Discovering statistics using SPSS for Windows, London:
SAGE Publications Ltd, 2000.
[49] Meda, “Epipen,” 2014. [Online]. Available: http://www.epipen.co.uk.
[Accessed: 15-Jan-2014].
[50] NASA, “NASA TLX: Task Load Index,” NASA. 01-Dec-2003.
[51] J. Brooke, “System Usability Scale,” Usabilitynet.org, 01-Dec-2006.
[Online]. Available:
www.usabilitynet.org/trump/documents/Suschapt.doc. [Accessed: 01-
Jun-2006].
[52] M. Jones and G. Marsden, Mobile Interaction Design, London: John
Wiley & Sons, Ltd, 2006.
[53] J. Preece, Y. Rogers, and H. Sharp, Interaction design, beyond human-
computer interaction, New York: Wiley & Sons, Inc., 2002.
[54] A. Bandura, “Guide for constructing self-efficacy scales”, in Self-
Efficacy Beliefs of Adolescents, 2006, pp. 307–337.
[55] F. D. Davis, “Perceived Usefulness, Per ceived Ease of Use, and User
Acceptance of Information Technology,” Management Information
Systems, vol. 13, no. 3, pp. 319–340, 1989.
[56] A. Bandura, “On the Functional Properties of Perceived Self-Efficacy
Revisited”, Journal of Management, vol. 38, no. 1, pp. 9–44, Sep. 2012.
[57] A. Bangor, P. T. Kortum, and J. T. Miller, “An Empirical Evaluation of
the System Usability Scale”, International Journal of Human Computer
Interaction, vol. 24, no. 6, pp. 574–594, Jul. 2008.

JBHI Ref: JBHI-00374-2015
11
X. BIOGRAPHIES
Luis U. Hernandez-Munoz is an IEEE Graduate
member and obtained his PhD from the School of
Electronic, Electric and Systems Engineering at the
University of Birmingham, UK. His PhD research was
related to pervasive computing for anaphylaxis
management. He also has a BSc in Electrical and
Electronics Engineering (hons) from the National
Autonomous University of Mexico. His current
research involves assistive technologies for healthcare,
m-health, human-centered design, technology
innovation, internet-of-things and studies of usability, effectiveness and
acceptance of technology.
Sandra I. Woolley is an IEEE Senior member and a
lecturer in the School of Electronic, Electrical and
Systems Engineering at The University of
Birmingham, UK. She trained as a graduate engineer
with Lucas Aerospace, UK and received a PhD degree
in Electronic Engineering from The University of
Manchester before working as a researcher at the
National Institute of Standards and Technology
(NIST), Maryland, U.S.A. Her current research
interests include aspects of e-health and, in particular, applications in
rehabilitation and assistive technology.
David Luyt is a Consultant Pediatrician at the
University Hospitals of Leicester. He completed his
undergraduate medical and postgraduate pediatric
studies at the University of the Witwatersrand in
Johannesburg, South Africa. After qualifying as a
pediatrician he undertook a research fellowship in
Leicester from which he obtained a Doctor of Medicine
from the University of Leicester. He is the lead
consultant in the Children’s Allergy Service in Leicester
since 1996 and a lead allergy clinician in the UK.
Gary Stiefel is a Consultant in Pediatric Allergy at the
University Hospitals of Leicester NHS Trust. He
gained his MBChB & BMedSci degree at the
University of Birmingham. He gained specialist
training in pediatric allergy at St Mary Hospital,
London and University Hospital Southampton
Foundation Trust. He completed an Allergy MSc at
Imperial College London and now regularly lectures on
the food allergy module for the Allergy MSc at
University of Southampton.
Kerrie Kirk is a Children’s Allergy Specialist Nurse
at Leicester Royal Infirmary. She has been qualified as
a Registered Nurse since 1985 and as a Children’s
Nurse since 1988. She has worked in Birmingham
Children’s Hospital in infant cardiology and Leicester
mostly on a respiratory ward followed by a medical
Day Care ward prior to working with children with
allergies which she has done for 12 years. She is a
Specialist Nurse for children with allergies. She has
been involved with research looking at the egg allergy
and the flu vaccine in children for the last 2 years and is the Immunisation and
Vaccination trainer for the Children’s Hospital at University Hospitals of
Leicester
Nick Makwana graduated from the University of
Birmingham Medical School and completed his
pediatric training within the West Midlands. He has
been a registrar at Birmingham Heartlands Hospital.
He went on to work at Alder Hey Hospital in Liverpool
and gained an MD from the Department of Clinical
Infection, Microbiology and Immunology from the
University of Liverpool. He is currently a consultant at
Sandwell and West Birmingham NHS Trust (SWBH)
and runs the pediatric allergy service cross site. He is one of the authors of
the Management of Cows Milk Protein Allergy Guideline for BSACI and is
the coordinator of the Midlands Pediatric Allergy Group. He trains doctors on
the recognition and emergency management of children with life threatening
conditions. He is an Honorary Clinical Lecturer in Pediatrics at the University
of Birmingham and he is the Vice Chair on the examinations committee for
the Royal College of Pediatrics and Child Health (RCPCH).
Tom C Dawson is a pediatrician with an interest in
allergy based in Worcestershire. He completed his
medical training at University College London and
trained in pediatrics in the West Midlands. He has an
MSc in Allergy from Imperial College and has
previously been involved in vaccines research and
surveillance studies into toxic shock syndrome and
pollen food syndrome. He is currently studying pollen
induced asthma with colleagues at the National Pollen
Unit.
Cathryn Melchior is an Allergy Nurse Specialist at
the Heart of England NHS Foundation Trust, which is
one of the largest allergy centers in the UK, providing a
comprehensive service for the diagnosis and
management of all allergic conditions. Cathryn
qualified in nursing in 2002 and worked in General
Medicine and Critical Care before working in A&E in
major trauma centers in Nottingham and Birmingham.
She joined the Heart of England allergy team in 2009 and has a particular
interest in drug allergy.
Gabriel Wong is a Wellcome Trust funded Clinical
Research Fellow in Clinical Immunology at the
University of Birmingham. He has substantial clinical
experience in managing patients with allergic diseases
and has published in the field of summation anaphylaxis
and wheat-dependent exercise induced anaphylaxis.
Tim Collins is an academic from the School of
Electronic, Electrical and Systems Engineering at The
University of Birmingham. His PhD was in the field of
active sonar and he supervises research in applied
signal processing for applications such as audio, digital
heritage and medical devices.
Lavanya Diwakar is a consultant immunologist
employed at the University Hospital, Birmingham. She
is currently undertaking a PhD in Health Economics at
the University of Birmingham, where she is looking at
ways to incorporate end user preferences into planning
allergy services for the West Midlands and will
estimate the costs and effectiveness of various possible
pathways for delivery of these services.