Content uploaded by Lisa Connally
Author content
All content in this area was uploaded by Lisa Connally on Feb 18, 2021
Content may be subject to copyright.
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
Journal of Clinical and Translational Research 2020; 5(3): 96-101
Journal of Clinical and Translational Research
Journal homepage: http://www.jctres.com/en/home
ORIGINAL RESEARCH
Development and evaluaon of an augmented reality educaon program
for pediatric research
Alan R. Tait1,2*, Lisa Connally2, Aalap Doshi2, Anita Johnson2, Abbey Skrzpek2, Mashala Grimes1, Asif Becher1,
Jae Eun Choi3, Monica Weber1
Department of 1Anesthesiology, 2Michigan Institute for Clinical and Health Research, Michigan Medicine, and 3ALTality Inc., Ann Arbor, MI, USA
ARTICLE INFO
Article history:
Received: December 15, 2019
Revised: February 24, 2020
Accepted: February 24, 2020
Published online: February 29, 2020
Keywords:
augmented reality
pediatrics
research
ABSTRACT
Background: Children often have limited understanding of clinical research and what they might
expect from participating in a clinical study. Studies, however, suggest that multimedia delivery of
medical and research information may promote greater understanding and engagement compared with
standard written approaches.
Aim: This study was designed to examine the eects of a novel interactive augmented reality (AR)
program on children’s understanding of clinical research.
Methods: Children (ages 7-13 years) were randomized to receive the basic information about clinical
research using either a printed storybook (control) or the same storybook enhanced using a video
see-through AR iPad program (AR) with embedded interactive quizzes. Children were interviewed
to assess their understanding of the material before (pre-test) and after (post-test) receiving either of
the randomized interventions. Both parents and children completed short surveys to measure their
perceptions of the information delivery.
Results: Ninety-one parent/child dyads were included in the analysis. There were no dierences
between the control and AR children’s pre-test understanding of the research information. However,
both groups demonstrated signicant and similar improvements in post-test understanding. Parents of
children in the AR group found the information to be of higher quality and greater clarity compared
with the control group, and 91.7% of children in the AR group found the inclusion of interactive
quizzes to be helpful. Both parents and children found the AR program very easy to use and 85.0 %
and 71.2%, respectively, indicated that if recruited for a future study that they would prefer information
delivered using some type of iPad AR program together with a discussion with the researcher.
Conclusions: Results demonstrated the importance of providing children and parents with information
in an easy to read and visually compelling manner. Although both groups demonstrated improved
understanding, children and their parents preferred the AR program and reported a preference for
receiving information using computer-based technology. Given the seemingly insurmountable
challenge of keeping children and families engaged in health research related information exchange,
the use of AR would appear to provide a novel and eective vehicle for enhancing children’s and
parents assimilation and understanding of research (and medical) information and as a potential tool
to optimize the informed consent and assent processes.
Relevance for patients: This study reinforces the importance in providing information to research
participants and patients in an easy-to-read and visually salient manner. Although the AR program used
in this study did not result in an increased level of understanding, AR was deemed the preferred method
of information delivery. It is hoped that the results of this study will serve as a platform for future studies.
*Corresponding author:
Alan. R. Tait
Department of Anesthesiology, Michigan Medicine, 1500 E. Medical Center Drive, Ann Arbor, MI 48109, United States
Email: atait@umich.edu
Tait et al. | Journal of Clinical and Translational Research 2020; 5(3): 96-101 97
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
1. Introduction
Unlike virtual reality which immerses the user in an entirely
articial environment, augmented reality (AR) allows the virtual
and real worlds to coexist and interact in real-time in a manner that
promotes user engagement and active participation in learning.
In this process, visually salient and contextually relevant digital
information can be infused into the real environment [1]. While
early AR applications focused primarily on gaming, AR has gained
considerable traction for use in the military, navigation, advertising,
education, and medicine. By superimposing virtual anatomic details
on mannequins or real patients AR has been successfully used in
medicine to teaching complex surgical and nursing techniques.
Recently, AR applications have been used in our institution to help
children understand their therapies and to serve as a distraction
technique when undergoing minor painful surgical and medical
procedures, for example, blood draws, and dressing changes.
The previous studies have shown that parents and children
often have diculty understanding both the child’s role in clinical
research (what it is and what it might entail), research concepts,
and specic details about clinical protocols [2-6]. In response to
this, there have been a number of studies showing that interactive
multimodal educational programs can improve children’s and
adults’ understanding of complex research and health information
over traditional paper consent forms and written educational
materials [7-10]. Furthermore, many parents and children have
expressed a preference for computer-based multimodal formats for
the presentation of such information. Although the reasons for this
are likely multifactorial, it is believed that multimodal approaches
work because they provide greater visual saliency, promote
active participation in learning, and for many, reduce cognitive
burden by message simplication [11-13]. Multimodal interactive
programs appear to be particularly benecial for children and
for adults with low literacy and numeracy abilities. Given the
increasing use of digital media and the natural facility of children
to interact with computer-based multimodal messaging including
virtual and AR technology, we believed that AR might provide
an opportunity to help children better understand important
clinical research concepts and their roles as potential research
participants. Therefore, this study was designed to develop and
evaluate an interactive AR program for clinical research education
for children and parents.
2. Materials and Methods
This study was deemed exempt by our Institutional Review
Board.
2.1. Content design and development
Content for the AR program was drawn from the extant
literature, expert opinion, and federal guidelines for research
involving children (45CFR46 Subpart D) [14]. We were
particularly interested in younger children and early adolescents
who are often required to provide assent for research. Based on this
information, the investigators identied and reached consensus on
common themes and elements deemed most critical for inclusion
into the AR program. Based on this process, the following items
were deemed necessary for inclusion into the program:
1. What is clinical research and why is it important for children?
2. What types of children might be asked to participate in
clinical research (e.g., sick and healthy).
3. A description of who is typically involved in the decisions
regarding participation in pediatric research and an
understanding that participation is voluntary and can be
withdrawn at any time without penalty.
4. What children might expect from participating in a clinical
research study (i.e., what a clinical study might entail including
types of procedures, time commitment, and potential burden).
5. The importance of knowing the risks and benets (direct and
indirect) of participation.
6. The importance of condentiality.
7. A series of game-related interactive exercises and quizzes to
promote engagement in the material and to establish a sense
of understanding.
The AR technology program included a storybook with generic
(i.e., not study-specic) information about children’s involvement
in research. The storybook introduced a gender-neutral cartoon
character named “Remy” who becomes interested in the idea of
participating in a research study after seeing a recruitment poster
in a pediatrician’s oce (Figure 1). Participants were able to select
a Remy avatar as either an astronaut, explorer, or superhero each
with accompanying avatar-specic background eects. Although
the storybook alone provided all the basic information, when
scanned with an iPad, video see-through technology initiated the
overlay of 3D graphics and sound onto the storybook allowing
Remy to “come to life” with action and speech. The AR program
also included embedded interactive quizzes to evaluate children’s
real-time understanding of the information.
The AR program was developed on a proprietary platform
which leverages an Amazon Web Services back end, Vuforia for
Figure 1. When using the AR iPad app. specic content in the storybook
“comes to life” in 3-D.
98 Tait et al. | Journal of Clinical and Translational Research 2020; 5(3): 96-101
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
computer vision, and the Unity gaming engine. These factors
make the application generally compatible for all mobile devices
and AR headsets. The “assets” (images, 3D models, sounds, etc.)
are called from the cloud when the application is rst launched on
the device and then presented when the device camera recognizes
the target images in the booklet. The program was built for
scalability and has been load tested to allow for hundreds of
thousands of simultaneous cloud requests. In practice, this would
allow a multitude of users and institutions to use the program at
the same time.
Prototypes of content, characters, and voice-overs were
evaluated and modied in an iterative process involving both
children and adults (experts and non-experts). During this
usability phase, participants were asked to verbalize their thoughts
(“think aloud”) as they navigated through the program. Responses
were written down verbatim and used to qualitatively assess the
participants’ perceptions (likes and dislikes) of the program.
Feedback from this usability testing phase was subsequently used
to adjust and rene the content prior to formal evaluation.
2.2 Product evaluation
Child (7-13 years.) and parent dyads attending any one of
our several outpatient facilities at our Children’s hospital were
recruited consecutively. Baseline demographic characteristics
were obtained including age and gender of the child, parental
education and role (i.e., Mom, Dad, or other), and race/ethnicity.
Child participants were rst given a short pre-test to elicit their
baseline understanding of eight core elements of clinical research
(i.e., what is research, potential risks, direct benets, indirect
benets, voluntariness, ability, and consequences of withdrawal,
and who decides about participation) using a semi-structured,
face-to-face interview. The responses to each question were
written down verbatim by trained research assistants who were
allowed to clarify questions and prompt the participants for
additional information but were unable to oer any specic details.
The children’s understanding of each individual core element
question was scored using a 0-2 scale where 0=no understanding,
1=partial understanding or poverty of content, and 2=complete
understanding. Individual item scores were subsequently
combined to provide an overall score of understanding (range
0-16 where 16=complete understanding). This scoring system
was based on the Deaconess Informed Consent Comprehension
Test [15] and has been described previously [4,16]. Understanding
was scored by individuals with no vested interest in the study and
accuracy was validated by two independent researcher assistants.
Children were randomized (computer-generated) to receive
information about research using either the storybook alone or the
same storybook used in conjunction with the iPad AR program.
Trained research assistants were available at all times to answer
any questions and help children navigate through the program, if
needed.
Once the subjects had either read the storybook or used the
AR program, the child participants were again interviewed and
scored to determine their “new” understanding of the information
provided (post-test). The items in the pre-and immediate post-test
interview were identical.
A survey related to the children’s perceptions of the clarity,
amount of information, perceived eectiveness of the message,
and their overall satisfaction with the information (i.e. AR/
storybook vs. storybook alone) using 0-10 visual analog scales
(where 10=high) was conducted along with the immediate post-
test interview.
Parents were able to watch the program with or independently
from their child but were instructed not to discuss the content or
presentation of information until after the data collection was
complete. Although parents were not tested for their understanding
of the material per se, they were asked to complete the same survey
of perceptions as did the children. At the end, parents and children
were shown both the control and AR information and asked how
they would prefer to receive the research information if asked to
participate in a study in the future.
2.3. Statistical design and sample size
Sample size determination was based on data from a previous
study that showed that children’s post-test understanding of
clinical research concepts following exposure to a non-AR
multimedia program for clinical trials was 11.65±4.1 (0-18 scale)
compared with 8.85±4.1 for children who received standard
text information [10]. Based on these data, we determined that
we would require 45/group (parents=90 and child=90) to detect
a dierence in understanding of at least that size (β=20%, two-
tailed) between the AR and control information.
Statistical analyses were performed using SPSS© (IBM Corp,
New York, v 21.0) software. Data are described as means (±SD)
and medians and were analyzed using statistics for parametric
(t tests) and non-parametric data as appropriate (e.g., Chi-square,
Mann–Whitney U, and Wilcoxon Signed-Rank tests). Statistical
signicance was accepted at the 5% level (P<0.05).
3. Results
A total of 97 parent-child dyads were approached for
participation in this study. Of these six were excluded due to
incomplete data. Ninety-one child/parent dyads were thus included
in the analysis (Control=46, AR=45). There were no dierences
between the control and AR groups in terms of demographics
(Table 1).
There were no signicant dierences in baseline understanding
between the two groups; both groups demonstrating poor
understanding of research concepts (Table 2). There were
however signicant improvements in understanding following
administration of both the control and AR interventions. These
improvements were similar between the groups although
understanding of the ability to say “no” to participation in research
was signicantly better in the AR group compared with the control
group. Children’s understanding of the information improved
with the age of the child. Older children (10-13 years based on
median split) had signicantly greater overall understanding
of the information compared with children aged 7-9 years
Tait et al. | Journal of Clinical and Translational Research 2020; 5(3): 96-101 99
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
(11.39±2.21 vs. 9.44±2.77 out of 16, respectively, P<0.001). This
was consistent for both the control and AR groups.
Both groups found the information equally helpful and clear
although children in the AR group thought that there was too much
information compared with the control group (Table 3). However,
when split by age (7-9 vs. 10-13 years), it was found that children
in the younger group were more likely to report that the amount
of information in the AR program was “too much” compared with
the older children (61.9% vs. 15.8%, P<0.005).
Parents, on the other hand, perceived the AR information to
be of signicantly higher quality and clarity compared with the
control information and were more likely to believe that the
amount of information provided was “just right” (Table 4). Parents
in the AR group were also signicantly more likely to report a
likelihood of allowing their child to participate in any future study
if information was presented in AR format.
Parents were very satised with all aspects of the program
including the interactivity and graphics (Table 5). Overall,
children in the AR group found the AR program extremely easy to
use (8.68±2.01 out of 10, where 10: extremely easy). The ability
of children to correctly answer the embedded games/quizzes in
the AR program on the rst attempt ranged from 44.2 to 97.7%
(Average=76.3%). Only four children (8.3%) believed that the
games/quizzes were “NOT helpful.”
Table 1. Demographics.
Control (
n
=46) AR (
n
=45)
Child’s age (years) 9.50±1.89 9.58±1.86
Child’s sex (F/M) 46.7/53.3 52.5/47.5
Parent role (mother/father) 75.6/24.4 87.2/12.8
Parent education
Grade school
High school only
Trade school/some college
College
Graduate school
3 (6.7)
10 (22.2)
16 (35.6)
12 (26.7)
4 (8.9)
1 (2.6)
9 (23.1)
11 (28.2)
16 (41.0)
2 (5.1)
Family race/ethnicity
White
African American
Hispanic
Native American
Asian
Other
29 (64.4)
6 (13.1)
2 (4.4)
0 (0.0)
0 (0.0)
8 (17.8)
25 (64.1)
2 (5.1)
4 (10.3)
2 (5.1)
2 (5.1)
5 (12.8)
AR: Augmented reality
Table 2. Children’s Pre- and Post-Test Understanding of Research
Concepts by Group.
Concept Pre-test
Con
Post-test
Con
Pre-test
AR
Post-test AR
Research 0.61±0.80 1.30±0.84* 0.51±0.73 1.38±0.76†
Risks 0.33±0.47 0.43±0.65 0.29±0.46 0.33±0.48
Direct Benets 0.69±0.81 1.15±0.89* 0.69±0.73 1.19±0.77†
Indirect Benets 0.87±0.86 1.40±0.75* 0.78±0.88 1.38±0.79†
Choice 1.63±0.77 1.78±0.64 1.47±0.87 1.86±0.52†
Say “No” 0.65±0.92 1.13±0.99* 0.71±0.95 1.63±0.77††
Withdraw 0.93±0.98 1.62±0.78* 1.11±0.99 1.85±0.53†
Who decides
Total Understandinga
0.83±0.57
6.54±2.55
1.13±0.46*
9.93±2.60*
0.88±0.44
6.52±2.72
1.12±0.57†
10.71±2.80†
Con: Control group: AR: Augmented Reality group. Data are mean ± SD based on scores
of 0-2 where 2: Complete Understanding. aTotal understanding based on scores of 0-16.
*
P
<0.05 versus Control Pre-test value; †
P
<0.05 versus AR Pre-test value. ‡
P
<0.05 verusu
Control Post-test value
Table 3. Children’s perceptions of information delivery.
Control AR
P
value
Likelihood of participation in future
research based on presentation
8.02 ± 2.54 7.65 ± 2.71 0.517
Helpfulness of information
Not at all
Somewhat helpful
Extremely helpful
4 (8.9)
32 (71.1)
9 (20.0)
2 (5.1)
23 (59.0)
14 (35.9)
0.245
Amount of information
Too little
Just right
Too much
4 (8.9)
37 (82.2)
4 (8.9)
1 (2.5)
23 (57.5)
16 (40.0)
0.002
Clarity of information
Not clear
Fairly clear
Extremely clear
5 (11.1)
20 (44.4)
20 (44.4)
4 (10.0)
9 (22.5)
27 (67.5)
0.080
AR: Augmented reality
Table 4. Parent’s perceptions of information delivery.
Control AR P value
Likelihood of participation in future
research based on presentation
7.62±1.54 9.02±0.95 0.000
Quality of information 7.78±1.66 8.85±1.23 0.001
Ability to follow information 8.82±1.13 9.15±1.05 0.172
Helpfulness of information
Not at all
Somewhat helpful
Extremely helpful
4 (8.9)
30 (66.7)
11 (24.4)
1 (2.5)
24 (60.0)
15 (37.5)
0.247
Amount of information
Too little
Just right
Too much
17 (37.8)
28 (62.2)
0 (0.0)
3 (7.5)
34 (85.0)
3 (7.5)
0.001
Clarity of information
Not clear
Fairly clear
Extremely clear
1 (2.2)
26 (57.8)
18 (40.0)
0 (0.0)
12 (31.6)
26 (68.4)
0.029
AR: Augmented reality. *
P
<0.05 vs Control. Data are mean±SD and
n
(%)
Table 5. Parents’ satisfaction with the augmented reality program.
Satisfaction*
iPad program easy to use 8.61±1.70 (9.0)
Graphics 8.80±1.47 (9.0)
Interactivity 9.39±1.02 (10.0)
Narration 9.22±1.06 (10.0)
Sound eects 9.09±1.04 (9.0)
Quizzes/games 9.17±1.32 (10.0)
Overall satisfaction 9.20±1.14 (9.5)
*Satisfaction scores on scale of 0-10 where 10: Extremely Satised. Data are mean±SD
(median)
100 Tait et al. | Journal of Clinical and Translational Research 2020; 5(3): 96-101
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
Open-ended comments from both parents and children in
the AR group were overall very positive. Comments included
“helped child to learn that kids/adults can participate in
studies;” “interactive, entertaining, helped to understand more;”
“characters and reading it to you;” “like the games;” “(like) the
games but hard to use book and iPad at times;” “reading it out
loud;” and “listening to it/the kid’s voice.” At the end of the study,
parents and children were asked how they preferred to receive
information about research. Table 6 shows that both parents and
children reported that they would prefer information using some
type of AR program like the one used in this study together with
a discussion with the researcher. This nding was consistent for
both younger and older children.
4. Discussion
Results from this study showed that information provided
in an easy to read and visually salient manner can help children
understand research concepts. The previous studies have shown that
information delivered to both children and adults using computer-
based multimedia results in greater participant and patient
understanding of research and medical information, compared with
standard written information [7-9]. This study was novel in that it
expanded on the previous computer-based messaging to include
AR. The observation that there were no dierences in children’s
understanding between the control and AR groups in this study may
have, in part, been a reection of the enhanced presentation of the
information in the control booklet. Previously, we have shown that
relatively simple improvements in the formatting of a standard paper
consent document for a pediatric study (e.g., 8th grade reading level,
use of cartoons and pictographs, color, and bulleting) signicantly
improves child and parent understanding of the information [16,17].
However, despite the fact that no signicant dierences in
understanding were observed between groups (other than the
concept of being able to say “no”), the AR delivery of information
was better received by parents and children and both reported
that AR would be the preferred method of information delivery
for participation in any future study. Interestingly, parents also
reported that they would be more likely to consider participation
in a future study for their child if presented with information using
an AR medium. While participants are likely to be more obliging
within the connes of a research study, keeping children and their
parents engaged in potential health related information exchange
in the real world are a signicant challenge. In this environment,
where there may be competing demands from clinical personnel
and attention is eeting, participants’ preferences for a medium
play an important role in getting (and keeping) them engaged with
information at hand. With this in mind, the children’s and parents’
preference for the rich, layered world of AR over the control booklet
in this study is noteworthy. Almost all the children (91.7%) found
the interactive games/quizzes to be helpful which supports the
previous research showing that the use of exercises with corrected
feedback can facilitate information retention [18,19]. Use of these
interactive exercises is important in establishing a sense of real-time
understanding of the information and can be used by investigators
to ensure that participants understand the information at the time
decisions are being made. It is thought that these interactive games
facilitate understanding and provide a sense of fun by promoting
active participation in learning rather than passive retention [20,21].
The potential limitations of this study are recognized. First, this
study describes a single prototype intervention at one institution
and, as such, may not be generalizable to all other institutions or
populations. Further, we limited our sample to children between
the ages of 7 and 13 years. This group was chosen because children
of this age are typically asked to provide assent to participate in
clinical research but often have diculty with written information.
The content of the AR program was thus designed to be appropriate
to this age group. Older adolescents were therefore excluded
believing that they would perhaps consider the presentation “too
young” for them. We have used the term understanding throughout
rather than recall. While the terms are often used interchangeably,
we believe that by asking the children to describe the information
in their own words provides some level of understanding.
This study reinforces the importance of presenting research
information to parents and children in an easy to read and
visually compelling manner. Although the AR and control groups
performed equally well in promoting children’s understanding
of research information, results suggest that both children and
parents preferred the interactive and immersive nature of the AR
technology over the more traditional written format. These results
therefore support the use of enhanced information delivery to
children and parents and highlight the potential promise of AR as
a future technology for enhancing the assent and consent process.
Disclosures
Jae Eun Choi is the Chief Research Ocer for ALTality, Inc.
which developed the AR program. Ms. Choi provided intellectual
input for the project but had no involvement in subject recruitment,
data collection, analysis, or interpretation of the data. None of
the other investigators have any nancial, commercial, or other
interests in ALTality, Inc.
Table 6. Children’s and parents’ preferences for information delivery.
7-9 years* 10-13 years* All children Parents
Written information only 3 (9.4) 2 (7.4) 5 (8.5) 0 (0.0)
Verbal only (from investigator) 3 (9.4) 0 (0.0) 3 (5.1) 0 (0.0)
Written and verbal 0 (0.0) 4 (14.8) 4 (6.8) 7 (8.8)
iPad AR only 4 (12.5) 1 (3.7) 5 (8.5) 5 (6.3)
iPad AR and verbal 22 (68.8) 20 (74.1) 42 (71.2) 68 (85.0)
AR: Augmented Reality. *Age based on median split
Tait et al. | Journal of Clinical and Translational Research 2020; 5(3): 96-101 101
Distributed under creative commons license 4.0 DOI: http://dx.doi.org/10.18053/jctres.05.202003.002
References
[1] Bower M, Howe C, McCredie N, Robinson A, Grover D.
Augmented Reality in Education-cases, Places and
Potentials. Educ Media Int 2014;51:1-15.
[2] Bareld R, Church C. Informed Consent in Pediatric
Clinical Trials. Curr Opin Pediatr 2005;17:20-4.
[3] Simon C, Simino L, Kodish E, Burant C. Comparison
of the Informed Consent Process for Randomized Clinical
Trials in Pediatric and Adult Oncology. J Clin Oncol
2004;22:2708-17.
[4] Tait AR, Voepel-Lewis T, Malviya S. Do They Understand?
(Part I): Parental Consent for Children Participating in
Clinical Anesthesia and Surgery Research. Anesthesiology
2003;98:603-8.
[5] Tait AR, Voepel-Lewis T, Malviya S. Do They Understand?
(Part II): Assent of Children Participating in Clinical
Anesthesia and Surgery Research. Anesthesiology
2003;98:609-14.
[6] Featherstone K, Donovan J. Why Don’t They Just Tell Me
Straight, Why Allocate it? The Struggle to Make Sense of
Participating in a Randomised Controlled Trial. Soc Sci
Med 2002;55:709-19.
[7] Tait AR, Voepel-Lewis T, McGonegal M, Levine R.
Evaluation of a Prototype Interactive Consent Program
for Pediatric Clinical Trials: A Pilot Study. J Am Med Inf
Assoc 2012;19:e43-5.
[8] Tait AR, Voepel-Lewis T, Moscucci M, Brennan-Martinez C,
Levine R. Patient Comprehension of an Interactive, Computer-
based Information Program for Cardiac Catheterization. Arch
Intern Med 2009;169:1907-14.
[9] Hermann M. 3-Dimensional Computer Animation-a
New Medium for Supporting Patient Education Before
Surgery. Acceptance and Assessment of Patients Based
on Prospective Randomized Study-picture Versus Text
[German]. Chirug 2002;73:500-7.
[10] Tait AR, Voepel-Lewis T, Levine R. Using Digital
Multimedia to Improve Parents’ and Children’s
Understanding of Clinical Trials. Arch Dis Child
2016;100:589-93.
[11] Ally B, Gold C, Budson A. The Picture Superiority Eect
in Patients with Alzheimer’s Disease and Mild Cognitive
Impairment. Neuropsychologica 2009;47:595-8.
[12] Cherry K, Park D, Frieske D, Smith A. Verbal and Pictorial
Elaborations Enhance Memory in Younger and Older
Adults. Aging Neuropsychol Cogn 1996;3:15-29.
[13] Hockley W. The Picture Superiority Eect in Associatve
Recognition. Memory Cogn 2008;36:1351-9.
[14] Additional Protections for Children Involved as Subjects in
Research. Fed Regist 1983;43:786.
[15] Miller C, O’Donnell D, Searight H, Barbarash R. The
Deaconess Informed Consent Comprehension Test:
An Assessment Tool for Clinical Research Subjects.
Pharmacotherapy 1996;16:872-8.
[16] Tait AR, Voepel-Lewis T, Malviya S, Philipson S. Improving
the Readability and Processability of a Pediatric Informed
Consent Document: Eects on Parents’ Understanding.
Arch Pediatr Adolesc Med 2005;159:347-52.
[17] Tait AR, Voepel-Lewis T, Malviya S. Presenting Research
Information to Children: A Tale of Two Methods. Anesth
Analg 2007;105:358-64.
[18] Festinger D, Dugosh K, Croft J, Arabia P, Marlowe D.
Corrected feedback: A Procedure to Enhance Recall of
Informed Consent to Research Among Substance Abusing
Oenders. Ethics Behav 2010;20:387-99.
[19] Kang H, McDermott B, Roediger H. Test Format and
Corrective Feedback Modify the Eect of Testing on Long-
term Retention. Eur J Cogn Psychol 2007;19:528-58.
[20] Schrand T. Tapping Into Active Learning and Multiple
Intelligences with Interactive Multimedia: A Low-
threshold Classroom Approach. Coll Teach 2008;56:78-84.
[21] Zhang D. Interactive Multimedia-based E-learning: A Study
of Eectiveness. Am J Distance Learn 2005;19:149-62.
