Article

Readability and Visuals in Medical Research Information Forms for Children and Adolescents

Abstract and Figures

Children are often-overlooked receivers of medical information, and little research addresses their information needs. However, young children are capable of understanding medical concepts, and they express the desire to be informed. This study addresses the quality of medical research information forms for children in the Netherlands, by assessing text readability and the role of visuals. Children's reading books, nonfiction books, and textbooks were used as comparison. Seven focus groups were conducted to identify children's preferences and needs for text and supporting visuals. We argue that the use of visuals is a powerful, but neglected, tool to improve medical information for minors.
Content may be subject to copyright.
Science Communication
2015, Vol. 37(1) 89 –117
© 2014 SAGE Publications
Reprints and permissions:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1075547014558942
scx.sagepub.com
Article
Readability and Visuals
in Medical Research
Information Forms for
Children and Adolescents
Petronella Grootens-Wiegers1,
Martine C. De Vries2, Tessa E. Vossen1,
and Jos M. Van den Broek1
Abstract
Children are often-overlooked receivers of medical information, and little
research addresses their information needs. However, young children are
capable of understanding medical concepts, and they express the desire to be
informed. This study addresses the quality of medical research information
forms for children in the Netherlands, by assessing text readability and the
role of visuals. Children’s reading books, nonfiction books, and textbooks
were used as comparison. Seven focus groups were conducted to identify
children’s preferences and needs for text and supporting visuals. We argue
that the use of visuals is a powerful, but neglected, tool to improve medical
information for minors.
Keywords
informed consent, medical research information form, child, visuals,
readability
1Leiden University, Leiden, Netherlands
2Leiden University Medical Centre, Leiden, Netherlands
Corresponding Author:
Petronella Grootens-Wiegers, Department of Science Communication & Society, Leiden
University, Sylviusweg 72, 2333 BE, Leiden, Netherlands.
Email: p.grootens@umail.leidenuniv.nl
558942SCXXXX10.1177/1075547014558942Science CommunicationGrootens-Wiegers et al.
research-article2014
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
90 Science Communication 37(1)
Introduction
Children and adolescents are often-overlooked receivers of medical informa-
tion, and little research addresses their information needs. However, young
children are already capable of understanding medical concepts (Lewis,
Lewis, & Ifekwunigue, 1978; Redding, 1993). Indeed, young children
express the desire to be informed and to be involved in decision making in
medical situations to some extent (Baker et al., 2013; Geller, Tambor,
Bernhardt, Fraser, & Wissow, 2003; Swartling, Hansson, Ludvigsson, &
Nordgren, 2011; Van Der Pal et al., 2010).
The right for minors to be involved in decision making is described in
Article 12 of the United Nations Convention on the Rights of the Child,
which states that “children shall be provided with the opportunity to be heard
in any judicial or administrative proceeding affecting the child directly”
(UNICEF, 1989). This statement is not specifically aimed at medical situa-
tions but nevertheless has implications for the role of minors in medical deci-
sion making.
As currently 45% to 60% of medications prescribed to children are “off-
label” (i.e., not officially approved for use in this specific group of patients or
for a specific medical indication), the World Health Organization (WHO;
Kaplan et al., 2013) recommends including more children in research to iden-
tify optimal treatments and prescription doses specifically for children. This
implies that in the future more children in hospitals are likely to be asked to
participate in medical scientific research. Children and adolescents do not
have the same rights as adults in deciding about research participation,
because their decision-making capacity is still developing. However, this
developing capacity to understand and oversee medical information also
requires an ethical response of increasing information provision and involve-
ment in decisions with age (John, Hope, Savulescu, Stein, & Pollard, 2008).
A number of guidelines state the right of minors to receive suitable medical
information. The WHO (1981) describes that “the researcher should provide
the child with information appropriate for his or her level of development to
obtain the child’s voluntary cooperation.” The Second Directive 2001/20/EC
by the European Parliament and the Council of the European Union, states, “A
clinical trial on minors may be undertaken only if . . . the minor has received
information according to its capacity of understanding . . .” (European Union,
2001, p. 38). In addition, the Guidelines from the Ethics Working Group of the
Confederation of European Specialists in Paediatrics argue,
The information (oral and written) to be provided to the (potential) child-
participant should be in conformity with the capacity of the child to understand
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 91
and should be adapted to assist the child at arriving independently at a decision.
In particular, the content, language, and mode of communicating the information
should be adapted to the child’s capacity to understanding and decision. (Gill,
2003, p. 457)
In spite of these guidelines, there is little research on the quality of medi-
cal information for children and adolescents, and there are no evidence-based
insights in how health communication can be optimally adapted for this tar-
get group. Therefore, we aim to assess the comprehensiveness of medical
information material for minors, and we seek to explore how visuals could
address the problem of incomprehensible medical texts for children.
Readability Gap
When a patient is asked to participate in a clinical trial, a medical research
information form is provided, explaining topics such as the aim, procedures,
and risks and benefits of research participation. These forms are subject to
legal rules stating which information should be in the document in order to
obtain ethical informed consent for participation. Readability analyses of
adult research information forms indicate without exception a large gap
between the required reading level to understand the information and the
actual reading ability of research participants (Kass, Chaisson, Taylor, &
Lohse, 2011; Souza et al., 2013; Sudore et al., 2006; Terranova et al., 2012).
Poor readability of information can lead to poor understanding and unin-
formed consent.
Based on the poor readability of adult information material, a similar read-
ability gap can be expected in medical information for minors. This is espe-
cially likely since children have a lower reading level than adults and require
material with an even better readability in order to reach comprehension.
Various studies assessed minors’ comprehension of research information,
which could serve as an indicator of the appropriateness of the information
process. As children’s capacity to understand and process information devel-
ops over time, comprehension is strongly related with age. From the age of 9,
children become capable of understanding research concepts (Berto, Peroni,
Milleri, & Spagnolo, 2000; Chappuy, Doz, Blanche, Gentet, & Treluyer,
2008; Franck & Winter, 2004; Ogloff & Otto, 1991; Ondrusek, Abramovitch,
Pencharz, & Koren, 1998; Paasche-Orlow, Taylor, & Brancati, 2003; Raich,
Plomer, & Coyne, 2001; Sanders, Federico, Klass, Abrams, & Dreyer, 2009;
Terranova et al., 2012). However, how well children of a certain age can
understand research concepts is also determined by the readability of the
information material (Barnett, Harrison, Newman, Bentley, & Cummins,
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
92 Science Communication 37(1)
2005). Research demonstrates that children and adolescents often do not
understand essential aspects of the research in which they are participating,
such as aim of the study, possible risks, and right to withdraw (Burke,
Abramovitch, & Zlotkin, 2005; Ondrusek et al., 1998; Swartling et al., 2011;
Tait, Voepel-Lewis, & Malviya, 2003, 2007). Moreover, some children do not
comprehend that their participation in research is not a standard treatment
(Barrett, 2005; Chappuy et al., 2008; Unguru, Sill, & Kamani, 2010). The
minors in these studies indicated that they did not feel informed about
research or did not understand what the doctor told them. This is no surprise,
as doctors sometimes deliberately choose to communicate with parents,
rather than directly with the child itself (De Vries, Wit, Engberts, Kaspers, &
van Leeuwen, 2010). It is thus apparent that children and adolescents are
often not as informed as laws and regulations require.
Nevertheless, little evidence exists that addresses minors’ information
needs and preferences for medical information. A recent systematic literature
review yielded only three studies that specifically discussed readability of
research information for children (Grootens-Wiegers, De Vries, & Van Den
Broek, 2014). One study assessed readability of pediatric consent forms in
the United States and showed an average reading level comparable to a uni-
versity reading level (Tarnowski, Allen, Mayhall, & Kelly, 1990). A study in
France compared research information for children to nonmedical texts for
children, such as novels (Menoni et al., 2011). The readability of research
information was considerably poorer than the readability of nonmedical
texts. Also, the nonmedical material contained a high number of supporting
illustrations, whereas only 14% of the medical text did. In a third study, infor-
mation material was developed together with children aged 6 to 12 years old,
resulting in a readability twice as high as the readability of medical texts in
the American study (Ford, Sankey, & Crisp, 2007). This evidence seems to
support the hypothesis of a readability gap in medical information for
children.
Readability, Comprehension, and Visuals
The readability score of a text is generally determined by the length of words
and sentences. However, readability is not equivalent to understanding. There
are other factors that also influence understanding, such as use of medical
terminology, the use of complex concepts (e.g., randomization), length of the
total document (Kass et al., 2011), font, layout of the document (e.g., use of
bullet points and text boxes; Tait et al., 2007), and visual support (Mayer,
2001). Improving only the readability of medical information will thus not
guarantee increased comprehension. Moreover, it has been argued that it is
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 93
practically impossible to write medical information at the recommended
readability levels, because of the complicated content of the information
(Hochhauser, 2007). In order to improve children’s comprehension of medi-
cal information, it is thus imperative to look beyond the readability itself, to
reading comprehension.
Reading comprehension involves not only the ability to understand the
words in the text, as defined by readability and vocabulary, but also the abil-
ity to process the input and create a mental model of the information (Barnes,
Raghubar, Faulkner, & Denton, 2014; Mayer, 2001). The cognitive theory of
multimedia learning from Mayer (2001) describes how input is processed,
divided in three assumptions. First, the dual channel assumption describes
the use of dual channels for visual/pictorial and auditory/verbal information
processing. The second assumption is the limited capacity assumption, indi-
cating that both channels can process only a limited amount of input. When
there is too much input in either of the channels, this cognitive overload will
hinder the processing of the information (Mayer & Moreno, 2003).
Third, the active processing assumption explains how taking up and learn-
ing information requires the active selecting of information and the subse-
quent organizing and integrating of that information into a mental model.
This processing involves three activities: essential processing, incidental pro-
cessing, and representational holding (Mayer & Moreno, 2003). Essential
processing is making sense of all relevant input, whereas incidental process-
ing is taking up nonessential information, such as the radio in the background
while reading a text. Representational holding is keeping the information in
the working memory and forming a mental model (Mayer & Moreno, 2003;
Pike, Barnes, & Barron, 2010). Such a mental model is created by selecting
relevant information from the text, updating the model while proceeding, and
suppressing irrelevant information (Pike et al., 2010). The newly encountered
information is integrated with prior knowledge and experiences in the mental
model (Barnes et al., 2014). The ability to update a mental model, based on
inferences from the text, is essential to reading comprehension (Carlson,
Seipel, & McMaster, 2014). Children are already capable of making infer-
ences from text, but this ability improves with age (Barnes et al., 2014). In
addition, the use of working memory, necessary for representational holding,
improves significantly during development, leading to better reading com-
prehension (Pike et al., 2010). Based on this model, comprehension of com-
plicated (e.g., medical) tests can be improved by the use of visuals. When
information addresses both channels (i.e., visuals and verbal), more informa-
tion can be taken up, reducing the risk for a cognitive overload and increasing
comprehension (Mayer & Moreno, 2003), and better mental representations
of the provided information can be made by the receiver. Visuals can help to
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
94 Science Communication 37(1)
create mental models in working memory, even if the visual does not contain
any new information next to the text (Brookshire, Scharff, & Moses, 2002;
Glenberg & Langston, 1992; Pike et al., 2010). Consistent with an immature
and inefficient working memory in developing children, it has been found
that younger children in particular rely on visuals for reading comprehension
(Pike et al., 2010). Although the effect of visuals reduces in older children,
they still show an influence on older children’s ability to make inferences and
thereby create a mental model (Brookshire et al., 2002; Pike et al., 2010).
In medical information, the visual channel is often completely neglected,
and information is provided only in conversations and text documents.
Although almost all books for children contain many images, a study on med-
ical texts for minors found that only 14% of the texts contained images
(Menoni et al., 2011). Exploiting the visual channel could therefore be a
promising approach to increase understanding when the limits of improved
readability of the text itself are reached.
Abundant research has indicated the value of visuals to support written
information. Patients prefer visuals-based information, and adding pictures to
a text increases the probability that a text will actually be read (Delp & Jones,
1996; Katz, Kripalani, & Weiss, 2006; Mansoor & Dowse, 2003; Michielutte,
Bahnson, Digman, & Schroeder, 1992). In an extensive overview of the influ-
ence of visuals in health communication, Houts, Doak, Doak, and Loscalzo
(2006) describe the pictorial superiority effect: Text with accompanying pic-
tures is remembered better compared to only written or spoken text. When a
patient receives verbal health information with an accompanying visual and
later views the same picture, it helps him or her to remember the information,
a process that is called cued recall. Participants also score significantly higher
on comprehension when they receive a text accompanied with pictures
(Austin, Matlack, Dunn, Kesler, & Brown, 1995; Mansoor & Dowse, 2003;
Carney & Levin, 2002; Houts, Witmer, Egeth, Loscalzo, & Zabora, 2001).
Not only can visuals improve recall, comprehension, and adherence, but they
also improve satisfaction with the information material (Katz et al., 2006).
Effective Visuals
Not just any visual will support comprehension; the effect strongly depends
on the quality and placement of the visual itself (Katz et al., 2006). Visuals
need to be placed close to the text to which they refer, and captions should be
provided in order to explain the picture and indicate the most important sec-
tions of the image (Carney & Levin, 2002; Fillippatou & Pumfrey, 1996).
Readance and Moore (1981) concluded that line drawings facilitated the
understanding of a text more than drawings with shading or photographs, and
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 95
the effect of pictures was greater when they were in color. Simple drawings
are better for comprehension than detailed pictures, as too many details can
be distracting or create a cognitive overload (Houts et al., 2006; Mayer &
Moreno, 2003). People prefer pictures in health information texts that refer to
their own culture and that include people they can identify with (Dowse &
Ehlers, 2001; Hosey & Stracqualursi, 1990). These culturally sensitive pic-
tures influence how much information is absorbed and the way in which
readers perceive the information (Dowse & Ehlers, 2001; Roter, Rudd,
Keogh, & Robinson, 2006). Research indicates that children prefer realistic
pictures (Houts et al., 2006). So, to clearly convey a message to children with
pictures, it is preferred that the pictures connect to the perception of the child
(e.g., using pictures or drawings of children of the same age and ethnicity).
Moll (1986) reported that medical messages for adults accompanied by car-
toon drawings scored highest on comprehension. In a study among 14-
year-olds, a comic book on disease information was shown to successfully
improve understanding of the material (Gillies, Stork, & Bretman, 1990).
Based on the literature, Houts et al. (2006) have proposed seven recom-
mendations for using pictures in health education: (1) Pictures should be used
in health communication as literature has shown its effectiveness; (2) visuals
should be simple; (3) textual information should be clear and simple; (4)
guidance for picture interpretation should be provided, for example, by cap-
tions and picture-text proximity; (5) pictures should be sensitive to the cul-
ture of the target group; (6) health professionals should be actively consulted
when creating pictures; and (7) the effects of pictures need to be evaluated in
research.
Aim of This Study
Our aim was twofold. First, based on the readability gap between the average
adult reading level and the readability of adult consent material, we hypoth-
esized that a similar gap could be present in pediatric medical information
material. We aimed to assess whether this gap is indeed present in research
information forms for children and adolescents.
Second, given the extensive literature suggesting that visual communica-
tion enhances the readability of text and the evidence regarding cognitive
development in children, we aimed to identify to what extent the use of visual
communication enhances children’s understanding of health information. In
addition, the potential of visuals in medical tests was investigated, by con-
sulting children to gain more insight in their (visual) information needs and
preferences. The study is situated in the Netherlands, where children from the
age of 12 are legally allowed to co-decide on research participation together
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
96 Science Communication 37(1)
with their parents. The analyzed documents are thus of legal status, and there-
fore it is essential that the information is adapted to the level of children and
adolescents, as is stated in the various rules and regulations of the United
Nations, WHO, and others.
Method
Readability Analysis
A readability analysis was performed on research information forms, in order
to stimulate research and discussion in this field and to propose a preferred
reading level for pediatric documents.
Collection of Research Information Forms. Medical research information forms
were collected from two Dutch academic hospitals. Together, these hospitals
covered 24% of Dutch pediatric studies in 2012 (Dutch Central Committee
on Research Involving Human Subjects [CCMO], 2012). Forms were
obtained via the institutional review board (IRB) of one hospital (the most
recent forms in their database) and via individual researchers from the other
hospital (from active studies). All analyzed forms were approved by the
IRBs.
Determination of Readability. Readability was determined using four different
instruments: Flesch Reading Ease (henceforth Flesch score), Flesch-Kincaid
Grade Level, Gunning fog index, and the Flesch-Douma Grade Level. All
instruments can be used to calculate the reading ease of a text, based on the
number of words per sentence and the number of syllables per word (Douma,
1960; Flesch, 1948; Gunning, 1968). The Flesch-Douma formula is adapted
from the Flesch score, accounting for the generally longer words in the Dutch
language. As the different formulas can provide significantly different results,
because of differences in measurements methods, it is important to use mul-
tiple formulas to supplement each other (Klingbeil, Speece, & Schubiner,
1995).
The Flesch score and Flesch-Douma formula result in a number between
0 and 100 for reading ease; a score of 100 indicates a text is very easy to read,
60 to 70 indicates the text is good to read for the average adult, and <60 varies
between hard and very hard to read. Flesch-Kincaid and Gunning fog scores
indicate the years of reading education required to understand the text.
Calculation with the latter formulas results in nonrounded numbers, for
example, 8.53, which indicates that one needs about 8½ years of reading
education. The interpretation of readability and reading level scores is indi-
cated in Table 1.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 97
The use of these readability formulas is simplistic and limited: The formu-
las only measure length of words and sentences but do not incorporate other
factors influencing readability, such as whether words used are common
words, length of the total document, and font. However, in spite of their
shortcomings, we were unable to find any studies that used a method other
than these instruments to determine readability of medical information so far
(Grootens-Wiegers et al., 2014).
The readability analysis was performed using the online tool provided by
the Language and Translation Technology Team of University College Gent
(van Oosten, Tanghe, & Hoste, 2010). Per consent form, three samples were
analyzed of the first 100 words from three paragraphs in the document: goal
of the study, procedures, and benefits/disadvantages of participation. If a
paragraph was shorter than 100 words, the last sentences of the previous
paragraph were included.
Other Variables. The total length of the document (in words and pages), and
number of pictures in the document were measured as well. The year of the
study and age of the target group were also documented.
Table 1. Interpretation of Readability and Reading Level Scores.
Score/level Interpretation
Flesch Reading Ease and Flesch-Douma Reading Ease (score)
0-30 Very hard to read (academic journals)
30-50 Hard to read
50-60 A bit hard to read
60-70 Good to read for the average adult
70-80 Fairly easy to read
80-90 Easy to read
90-100 Very easy to read
Flesch-Kincaid Grade Level and Gunning fog index (level)
Number of
years reading
education
Reading education generally starts at age 6. The score +
6 indicates the age at which children should be able to
read the text
4 10 years old
5 11 years old
6 12 years old
7 13 years old
8 14 years old
9 15 years old
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
98 Science Communication 37(1)
Material for Comparison. As no golden standard for readability of pediatric
consent documents exists, other texts for children were analyzed as compari-
son and to propose a standard for medical information. Children’s novels
were chosen for this comparison, since these books consist of plain text and
therefore compare well to the forms. Books were selected based on top 10
bestsellers of children’s and young adults’ books of bol.com, a large web
shop. Three random pages were selected from each book, and the first 100
words on the page were analyzed.
In order to assess the role of pictures in children’s books (nonmedical),
textbooks and nonfiction books were also analyzed. Five children’s textbooks
were collected from three local elementary schools. The textbooks were used
for 12-year-olds, corresponding with the eighth grade of Dutch elementary
education, and covered the subjects nature, geography, and history.
Six nonfiction books for children were selected by age category (i.e., from
the age of 12) from a local library, and one nonfiction book was provided by
a cooperating school. The books were selected to cover similar topics as the
textbooks, namely, nature and geography.
For textbooks and nonfiction books, the first 100 words of the pages 10, 20,
and 30 were analyzed (excluding captions of images). If there were fewer than
100 words on the page, additional words from the next page were used.
For nonfiction books that were shorter than 20 pages (indicated with an
asterisk * in a later table), pages 5, 10, and 15 were analyzed. When pages
had a deviant layout, such as assignments or only pictures, the text on the
following page was analyzed.
The picture-text ratio of children’s school textbooks and nonfiction books
was also analyzed for pages 10/11 and 20/21. The total number of words and
the number of images on the two pages were counted. Subsequently, a quan-
tification of the number of images per 100 words of written text (excluding
captions of images) was made. If the nonfiction book was shorter than 20
pages, pages 5/6 and 10/11 of the book were used. When pages had a deviant
layout, the next pages were analyzed.
Qualitative Focus Groups
In-depth focus groups were performed with children in the Dutch eighth
grade in seven groups of 10 to 12 children. A total of 77 children (age 11-12
years) from three different elementary schools in the area of Haarlem (the
Netherlands) participated in the focus groups.
Ten questions about the attractiveness of text and pictures in a research
information form in textbooks and in nonfiction books were discussed. Topics
of the discussion were understandability of the text, the most and least
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 99
attractive book, characteristics and layout of a text, the pictures, the ideal
nonfiction book and information form, amount and type of images, and dif-
ficult words.
A section of a representative pediatric research information form was dis-
cussed with the children. The selected sample consisted of two paragraphs
about the aim and procedure of a research study on a gene variation. The
section had a length of 178 words and a Flesch score of 55.43, which is fairly
difficult to read for an adult. In the text, the effect of the gene on hormone
secretion was discussed, as well as the procedure of coming to the hospital
for drawing blood and subsequent testing.
After the children read this document, the same text—however adapted by
the authors to contain images—was shown to them again. Three images were
added: a photograph of drawing blood, an image of a DNA structure, and a
time table for the procedure. Comprehension and preference was discussed.
The children were also asked to discuss the positive and negative aspects
of text and picture use in the textbooks and nonfiction books. The last three
focus groups also read a text from one of the nonfiction books on the topic of
insects (readability Flesch score 27.94). This book was selected since chil-
dren seemed to understand the book even though the reading score was very
low. Therefore, we aimed to investigate whether they found this text more or
less understandable than the information form.
Results
Readability Analysis
Hospital 1. Eleven pediatric research information forms from the first hospi-
tal were analyzed. The forms were written between 2007 and 2012 and were
directed at children and adolescents 12 to 17 years old. The length of the
forms varied from 628 to 3,790 words, or 3 to 9 pages, with an average of
1,990 words or 5.4 pages. None of the forms contained any illustration, lead-
ing to a picture-text ratio of 0 per 100 words.
The average readability scores per document are shown in Table 2. The
average Flesch-Douma was 63.58, Flesch score 49.54, average Flesch-
Kincaid Grade Level was 9.76, and Gunning fog 13.16. Documents with the
highest or lowest score on one scale do not necessarily have the highest or
lowest score on another scale, due to the different calculation methods of the
instruments.
Hospital 2. Eleven forms from between 2008 and 2013 from the second hos-
pital were analyzed. Forms were directed at ages 12 to 18, and length of the
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
100 Science Communication 37(1)
forms varied from 2 to 11 pages and from 512 words to 3,370 words, with an
average of 4.8 pages or 1,503 words. Three of the forms contained illustra-
tions: One form had a picture with every paragraph (11 in total). The average
picture-text ratio was 0.11 per 100 words.
Average readability scores were Flesch-Douma 63.89, Flesch score 49.88,
Flesch-Kincaid 9.96, and Gunning fog 13.39. The scores per document can
be found in Table 2.
Reading Books. Ten novels were analyzed for readability. The average readabil-
ity was Flesch-Douma Reading Ease 83.80, Flesch score 71.75, Flesch-Kincaid
Table 2. Readability of Medical Research Information Forms From Hospital 1 and
Hospital 2.
Year Pages Words Pictures
Flesch-
Douma Flesch
Flesch-
Kincaid
Gunning
fog
Hospital 1
2013 8 3,790 0 64.34 50.37 9.36 12.60
2012 4 1,460 0 49.31 33.88 12.56 16.61
2012 8 2,829 0 57.89 43.31 11.15 15.16
2012 9 3,226 0 57.16 42.49 10.72 14.68
2012 4 1,325 0 59.04 44.56 10.90 13.69
2012 3 628 0 56.80 42.11 11.43 14.03
2012 6 2,406 0 64.96 51.06 8.87 11.26
2009 3 1,374 0 77.72 65.08 7.90 11.23
2009 4 1,305 0 78.52 65.90 5.73 9.24
2009 4 1,067 0 80.06 67.63 6.56 9.47
2007 6 2,476 0 53.60 38.60 12.19 16.83
Average 5.4 1,990 0 63.58 49.54 9.76 13.16
Hospital 2
2013 2 512 0 70.79 57.43 7.40 10.84
2013 4 1,034 0 66.09 52.30 9.66 13.11
2012 4 1,601 0 69.26 55.78 8.99 12.78
2012 2 644 0 59.35 44.91 11.04 15.51
2012 4 1,215 0 71.20 57.90 8.56 11.48
2010 4 1,376 11 67.33 53.67 9.49 12.83
2010 11 3,058 3 41.81 25.64 13.56 16.54
2010 6 1,883 6 80.77 68.41 6.45 10.31
2008 9 3,370 0 44.48 28.60 14.24 17.34
2008 3 597 0 80.79 68.45 6.97 9.97
2008 4 1,245 0 50.87 35.62 13.19 16.57
Average 4.8 1,503 1.82 63.89 49.88 9.96 13.39
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 101
Grade Level 6.54 and Gunning fog 9.06. The scores and target age-groups per
book can be found in Table 3.
Textbooks. Five children’s school textbooks from the eighth grade were ana-
lyzed. Average readability scores were Flesch-Douma Reading Ease 80.76,
Flesch score 68.37, Flesch-Kincaid Grade Level 5.87 and Gunning fog 8.51.
The picture-text ratio was 2.83 per 100 words. The scores of the books and
their editors can be found in Table 4.
Nonfiction Books. Seven children’s nonfiction books suited for children aged
12 and older were analyzed for readability. Average Flesch-Douma Reading
Ease was 63.51, Flesch score 49.45, Flesch-Kincaid Grade Level 9.15 and
Gunning fog 11.82. The picture-text ratio was 1.83 per 100 words. The scores
of the books can be found in Table 5.
Comparison. Scores of the forms for both hospitals taken together indicated
an average Flesch score of 49.71, Flesch-Douma Reading Ease 63.73, Flesch-
Kincaid Grade Level 9.86, and Gunning fog 13.28. Average readability and
reading level scores for each of the texts are indicated in box plots in Figure 1.
Table 3. Readability of Reading Books.
Title Author
Target
age-group,
years
Flesch-
Douma Flesch
Flesch-
Kincaid
Gunning
fog
Hoe overleef ik mijn vader?
(en hij mij!) [How Do I
Survive My Dad? (and He
Me!)]
Oomen, F. 10-12 92.90 81.72 4.19 7.59
Sproetenliefde [Freckle Love] Stoffels, M. 10-12 91.91 80.65 4.80 6.90
De Hongerspelen [The
Hunger Games]
Collins, S. 13-15 78.88 66.36 7.93 10.35
Hasta la Vista Visser, J. 15-18 80.75 68.42 7.41 10.11
Het leven van een loser [Diary
of a Wimpy Kid]
Kinney, J. 10-12 80.35 68.00 8.18 11.11
Broederband / 3 De Jagers
[Brotherband / 3 The
Hunters]
Flanagan, J. 10-12 80.15 67.74 7.29 9.25
Spijt! [Regret!] Slee, C. 13-15 95.34 84.41 4.15 6.31
Gone–licht [Gone–light] Grant, M. 15-18 76.63 63.87 7.37 9.43
Promille Vreeswijk, H. 15-18 73.71 60.66 7.95 10.69
Inwijding [Divergent 1] Roth, V. 15-18 87.34 75.64 6.15 8.91
Average 83.80 71.75 6.54 9.06
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
102 Science Communication 37(1)
Table 4. Readability and Picture-Text Ratio of Nonfiction Books.
Title Author
Target
age-group
Flesch-
Douma Flesch
Flesch-
Kincaid
Gunning
fog
Picture-
text ratio
Ooggetuigen–Weera
[Eyewitnesses–weather]
Cosgrove, B. 12 55.19 40.32 11.19 14.46 1.76
Ooggetuigen–Dierena
[Eyewitnesses–animals]
Jackson, T. 12 62.15 47.96 9.43 12.65 1.18
Insiders–Insecten en
spinnena [Insects and
spiders]
Tait, N. 12 51.60 36.36 11.07 13.63 1.34
Insiders–Extreem weera
[Extreme weather]
Mogil, H. M. 12 53.41 38.34 10.53 12.86 1.90
Doc–De toendrab [The
tundra]
Kolfschoten,
M. van
11-12 79.55 67.05 6.08 7.85 0.43
Doc–Ziek in je hoofdb
[Ill in the head]
Jansen, W. 11-12 79.12 66.60 6.81 9.43 0.26
Animal planet–Super
zintuigena [Super senses]
Burdon, A. 11-12 63.55 49.49 8.94 11.86 5.96
Average 63.51 49.45 9.15 11.82 1.83
a.Translated into Dutch from the original (English) version.
b.Books with less than 20 pages.
Table 5. Readability Analysis and Picture-Text Ratio of Textbooks.
Title Publisher
Target
age-group,
years
Flesch-
Douma Flesch
Flesch-
Kincaid
Gunning
Fog
Picture-
text
ratio
Wijzer door de
tijd, Geschiedenis
[History]
Wolters
Noordhoff
11-12 87.20 75.46 4.77 6.65 2.38
Naut, Natuur
en Techniek
[Science and
technology]
Malmberg 11-12 79.45 66.95 6.29 8.56 2.26
Hier en daar,
Aardrijkskunde
[Geography]
Malmberg 11-12 82.78 70.60 5.73 8.03 4.22
De blauwe planeet,
Aardrijkskunde
[Geography]
Thieme
Meulenhoff
11-12 76.54 63.74 6.49 10.28 4.03
Leefwereld, Natuur
en Techniek
[Science and
technology]
Wolters
Noordhoff
11-12 77.81 65.12 6.05 9.03 1.25
Average 80.76 68.37 5.87 8.51 2.83
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 103
Note that the Flesch-Douma is adapted from the Flesch Reading Ease with a
constant, and therefore shows an equal spread.
Prerequisites for an analysis of variance (ANOVA) were tested with a
Fligner-Killeen test and a Bartlett test for homogeneity of variances. An
ANOVA was performed using the software R (R Development Core Team,
2011) for each of the readability scores to compare the five types of texts.
The ANOVA indicated a significant difference between the groups
(p < .01).
A post hoc test was performed with Tukey’s honestly significant differ-
ence test, to identify significant differences between certain text types; the
results are indicated in Figure 2. The forms scored significantly lower on
EducationNon-fictionNovel Form 1Form 2
6
8
10
12
14
16
Gunning Fog Test
30
40
50
60
70
80
Flesch Reading Ease Test
Education Non-fictionNovel Form 1 Form 2
EducationNon-fiction Nove l Form 1Form 2
4
6
8
10
12
14 Flesch-Kincaid Test
40
50
60
70
80
90
Type of text
Flesch Douma Test
Education Non-fictionNovel Form 1 Form 2
Test scores
Figure 1. Box plots indicating the reading level (left boxes) and readability (right
boxes) of education and nonfiction books, novels, and research information forms
for children.
Note: Reading level values indicate number of reading years required to read the text;
readability values under 65 are difficult to read for the average adult. Research information
forms are harder to read than education books and novels but not nonfiction books.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
104 Science Communication 37(1)
readability than novels (p < .05 for Flesch-Kincaid, p < .01 for Flesch, Flesch-
Douma, and Gunning fog) and textbooks (p < .05 for all readability scales).
Novels had a significantly better readability than nonfiction books according
to the Flesch Reading Ease (p < .01) and the Flesch-Douma tests (p < .01) but
not according to the Flesch-Kincaid Grade Level and the Gunning fog tests
(p > .1). Nonfiction books show a trend of poorer readability than textbooks;
however, these differences were not significant. Novels and textbooks had a
similar readability; also information forms and nonfiction books were
comparable.
.057 .98 .033 .038
.003 1.00 1.00
.0007 .0009
1.00
Edu
Nov
Edu Nov
FRE
.057 .98 .033 .038
.003 1.00 1.00
.0007 .0009
1.00
Edu
Nov
Edu Nov
FD
.081 .98 .013 .008
.11 .97 .93
.010 .006
1.00
Edu
Nov
Edu Nov
FK
.12 .99 .005 .003
.13 .75 .62
.002 .0010
1.00
Edu
Nov
Form
1
Form
1
Form
2
Form
2
Form
1
Form
1
Form
2
Form
2
Edu Non-
fict Nov
GF Non-
fict
Non-
fict
Non-
fict
Non-
fict
Non-
fict
Non-
fict
Form
1
Form
1
Form
1
Form
1
Non-
fict
Form
2
Form
2
Form
2
Form
2
Figure 2. Tukey test results.
Note: GF = Gunning fog; FRE = Flesch Reading Ease; FK = Flesch-Kincaid Reading Level; FD =
Flesch Douma Reading Ease. The p values indicated per readability instrument (GF, FRE,
FK, FD); p values below .05 indicate a significant difference in readability between two types
of text. These figures confirm the difference in reading level between research information
forms and novels and education materials for all four instruments. In addition, readability
scores but not reading levels are significantly different between forms and nonfiction books.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 105
Qualitative Focus Groups
Children’s Opinions on the Research Information Forms in Relation to the Text-
books and Nonfiction Books. Children thought the text from the research infor-
mation form was hard to read and boring: “You are distracted easily when the
text is so boring.” They indicated that the abbreviated gene name was espe-
cially difficult. None of the children could give us the definition of a gene.
One child thought it was “something in your blood” and another thought it
was a disease. On being asked what would happen if you had a deviant form
of the gene, one boy answered, “I think you die.” Another boy mentioned he
found the text to be “negative” and “sinister.” Only three children knew how
to interpret the general intention of the research described: “They found
something in your DNA and they are going to investigate it.” Some children
were confused: “I don’t understand this, because they think that with some
people such a deviant gene can change the hormones in the body. But, does
the heart change into a lung then, for example?”
All children could understand the method: taking a blood test. In all focus
groups, children mentioned they would prefer a better explanation of the
meaning of a gene. “You can see this was written by an adult”; “If this is sup-
posed to be for children, I would make it easier.”
When asked, the children in the focus group said they would not want to
participate in the research, because they needed a better explanation. After we
told them that a research information form is usually a few pages long, instead
of the nine-sentence sample, they thought no one would read it.
I think four pages, a child that’s going to read that will never finish it. That’s
more for the parents.
When presented with the adapted form with accompanying pictures, some
children indicated they could understand it better, but still quite a few did not
fully understand the text. Some children indeed showed an improved under-
standing but there were still errors in their explanations, such as: “Well, that’s
your DNA and the genes come into your blood or something.”
In general, almost all children thought the text looked better with pictures
(“more attractive”), although two children preferred a text without visuals.
Children in different focus groups independently stated that they would pre-
fer a drawing of the blood test over a photograph, because it was “less scary”;
“You want to know if you’re going to be okay. That if they take a blood test,
you will be healthy again”
They had a strong opinion on the pictures used and a lot of suggestions:
“More pictures!”; “More color!”; “Arrows or lines to the pictures with explana-
tions”; “I’d like it to be a cartoon”; “I’d like the font of the text to be bigger.”
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
106 Science Communication 37(1)
One of the children suggested the information form should be written by a
child. We asked whether they would like an example of another child partici-
pating in the research, and they were enthusiastic: “When I went to surgery
there was a booklet about that too. In it was written what was going to happen
to you. I really liked that.”
The text from a nonfiction book about insects, which had a remarkably
low readability (Flesch score 27.94, comparable to academic journals), was
considered by all participants easier to read than the form: “It’s not very hard
to read.”; “A lot of words in it we have learned already.” The text from the
nonfiction book looked more attractive, according to the children: “You see
the pictures and the text and you feel eager to read it because you know what
the picture is about”; “It’s more exciting because there are more things to
see”; “And then you’d like to know more about it.”
Children’s Opinions on the Layout of Textbooks and Nonfiction Books. We showed
the children all the textbooks and nonfiction books, and the majority pre-
ferred a nonfiction book about the weather with spectacular computer-gener-
ated three-dimensional (3D) images and a nonfiction book about animals,
from Animal Planet. On being asked why they liked those particular books,
they answered: “A lot of pictures”; “And not so much text”; “There’s a lot to
see even before you read the text”; “The whole page looks colorful!”; “It
looks really cool”; “You just want to read it now.”
Other books were least favorite, because the pictures were too small and
looked “old-fashioned” and “gross!” and the colors were boring. “These are
just pictures of clouds. I can look outside if I want to see clouds!”; “Those
pictures have nothing to do with the text!”
Many children expressed a preference for photographs over drawings,
because they are more realistic. Still, some of them also said that the image
of the blood test should be a drawing, to make it less real and less scary. Also,
computer-generated 3D images evoked enthusiastic reactions from the chil-
dren, because they were exciting” and “it looks like you’re actually there
because the picture is so big!” In one focus group, the children specifically
stated they liked exciting pictures. Many children mentioned they wanted a
text to have an element of fun, or a joke, and children liked it when cartoons
were used in the books.
The majority of the children preferred the text to be divided into smaller
sections, or boxes, spread onto the page, because “it makes the text look
shorter.” Only a few stated a preference for a continuous text, because it
was “less distracting.” Almost all children wanted difficult words to be
explained, preferably on the same page. On being asked what they did when
they encountered a difficult word, they replied they would skip it, read it
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 107
again, ask someone (“my mum”), or look it up on the Internet or in a
dictionary.
We asked the children whether they would prefer a difficult word to be
explained by a picture, and they responded, Depends on the picture.”
But still with a little bit of explanation, because a picture alone is just . . . Look,
if that text is not there . . . then you still don’t know what it is exactly. You need
to explain it.
In all focus groups, children expressed that they still needed a written
explanation of the picture in order to help them understand what it was they
were looking at.
We asked what the children would think of a page containing a lot of pic-
tures. Almost all children answered they would find that distracting.
“Normally you look once and you’re done, and now you’ll have to look a
hundred times!”; “You want to read it but instead you only look at the pic-
tures”; “And then I don’t know where I left off reading! Then you don’t know
which picture belongs to which piece of text.”
Some children—of whom one indicated “hating” reading—would rather
see lots of pictures. When asking whether they would prefer images in color
or black and white, all children preferred color. One child also said that black-
and-white pictures would be better than no pictures at all.
Some children expressed disappointment if a text would not contain any pic-
tures: “If you see the cover of such a large book, it’s very colorful. Then it looks
nice. And then when you look inside there’s no color!”; “Or no pictures at all!”
That’s why I always look inside the book before reading.
Discussion
In this study, the current quality of medical research information forms for
children and adolescents was analyzed. In addition, children were consulted
about their preferences and needs for text and visuals in medical information.
To our knowledge, this is one of the first studies in which the quality of medi-
cal research information forms is examined and discussed in consultation
with children.
Readability Analyses
The information forms were very lengthy, with a maximum of 11 pages or
3,790 words. Some studies have shown that if a document is longer than
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
108 Science Communication 37(1)
1,000 words, people will merely skim the text (Rugege-Hakiza et al., 2003;
Sharp, 2004). In our sample, 18 of the 22 documents exceeded a length of
1,000 words, which leads to the likelihood that children would not read the
full document. This will affect their understanding and ability to consent or
dissent to a study.
Only 5 of the 22 documents had a Flesch score higher than 58, which is the
minimum level of readability for adult forms, as indicated by the CCMO
(2008). The Flesch-Kincaid Grade Level average was 9.86 and Gunning fog
13.28 years of reading education required to read the documents. Even the
most forgiving instrument, the Flesch-Kincaid, indicates that documents
require much more reading experience than can be expected from 12-
year-olds, the age at which children in the Netherlands are formally asked to
consent to research (in addition to parental consent).
A significant difference between the readability of information forms and
novels and textbooks was found. Although Tukey’s honestly significant dif-
ference test is not very powerful (i.e., does not generate significant differ-
ences as easily as other tests), we detected significant differences with p <
.001 between the forms and novels, indicating large differences between the
two. The novels were significantly easier to read than nonfiction books, and
a nonsignificant trend was found for a lower readability of nonfiction books
as compared to textbooks.
The nonfiction books were of the same readability as the forms, even
though they contained similar topics as the (easier to read) textbooks.
Remarkably, there is little spreading in the data for textbooks, even though
the sample size is small (n = 5). Information forms and nonfiction books
show a lot of spreading, while these had higher sample size than the text-
books. A possible explanation for this difference is that school textbooks are
bound to regulations more than nonfiction books and possibly are revised
more extensively by writers as well as teachers. The lack of specific regula-
tions for nonfiction books and research information forms leaves room for
interpretation of the reader’s capacities by the writer.
Visuals and Comprehension
Only 3 information forms out of 22 used visuals to clarify the information in
the document. In one form, medical scanning procedures were explained next
to photographs of the patient’s position on the apparatus. In another, a visual
vaccination schedule was supplied to illustrate the study procedure. The third
document had a picture next to each new paragraph, and an illustrated over-
leaf, as in a brochure. All of the other documents consisted of plain text, and
the resulting overall average picture-text ratio of the forms was very low.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 109
All (97%) children except for 2 voiced a preference for visuals in books
and texts, to create an image of what they were reading. The children consid-
ered the nonfiction books easier to read than the forms, even though both had
a comparable readability. We hypothesize that this difference is explained by
the presence of visuals in the nonfiction books, decreasing the cognitive load
of the text and supporting the creation of mental models required for reading
comprehension (Mayer, 2001). Indeed, when visuals were added to the infor-
mation form, it was better understood by most children than the original text-
only sample. Children rated nonfiction books with a high picture-text ratio
more positively than books with a low picture-text ratio. It is noteworthy that
children themselves suggested that there should be just as much images as
text or more. Possibly, children unconsciously estimate the picture-text ratio
of a text before they start reading.
In the book that was designated the most favorite by the majority of the
children, the images contained a lot of information, portraying processes,
structured by arrows and captions. The amount of text on the page was low,
which resulted in a high picture-text ratio.
A combination of an attractive layout, pictures, and explanations might
make the text look understandable for a child. If children feel that they are
unable to understand a text, they might become discouraged and, as a conse-
quence, indeed understand less of the text. But when a text layout gives the
reader the feeling that it is readable, regardless of its actual readability, it
might have a positive influence of the comprehension level and stimulate
children to read and make an effort to understand the information.
Children’s Preferences for Visuals
Children made it clear that visuals should be informative, rather than only
decorative. Almost all children expressed a need for guidance by captions
next to illustrations. In addition, there was a need for clear and concise expla-
nations of terms and difficult words, within the text itself, or in a small box
on the same page. It is important to children that visuals are attractive in
color, size, and content. The children generally preferred realistic images,
such as photographs or 3D images. Some children preferred drawings, while
others thought drawings were childish. Surprisingly, almost all children liked
funny cartoons, which are essentially drawings as well. The funny aspect of
cartoons was appealing to them, because they would prefer a text to be not
too serious. Naturally, research information is of a serious nature, and the use
of funny cartoons and “exciting” pictures might not be appropriate. However,
the idea to use cartoons to explain medical information is not new and can be
very effective, if used in a considerate way (M. J. Green & Myers, 2010).
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
110 Science Communication 37(1)
The children indicated they would consider reading a text of which
they found the topic not very interesting, if it had attractive features. On
the other hand, if a text did not contain any visuals, they would likely not
read it.
Children were very enthusiastic about improving the information forms
with their own ideas. After consulting the children about the form, one boy
actually said: “I’m just thinking by myself right now that I would like to
know more about DNA and stuff, it seems quite interesting to me.”
Children aged 12 are likely capable of coping with difficult words or pro-
cesses if these are explained to them well. An effective explanation raises
interest, such as in the quotation above, which helps a child be motivated to
keep reading the rest of the text.
Limitations
In addition to readability and format of information forms, other factors play
a role in the quality of these forms, such as selection of the content or use of
fonts and colors. In addition, emotions toward the information content might
differ between medical information and books and might influence under-
standing. These aspects were not included in our analysis.
The context of the focus groups was different from the situation where a
child in the hospital receives medical research information, in that the chil-
dren only read an excerpt of the form and had no personal relation or emo-
tional connection to the information. Further research should address these
factors.
The instruments used for readability measure length of words and sen-
tences, but differences might exist in length of words between languages. In
addition, length of words or sentences is not the only factor influencing read-
ability, as long everyday words might be much easier to read than short medi-
cal terms. Thus, the readability analysis should not be considered to give an
absolute judgment on readability but rather used as a rough indication of the
current standard of pediatric consent forms.
In addition, a possible bias might have occurred in the selection of the
forms. Forms were obtained via the IRB and via individual researchers.
Selection therefore was partly dependent on the willingness of researchers to
offer their forms for analysis.
Conclusion and Recommendations
As children and adolescents have a growing influence in the consent process,
they need to understand the information provided. Our readability analysis on
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 111
22 pediatric forms from two out of the six large medical centers in the
Netherlands demonstrates a very low readability for the majority of the
forms. Some of the analyzed forms were very long, and only three forms did
contain visuals, which could greatly support understanding of the informa-
tion (Mayer, 2001).
To improve reading comprehension, information material and consent
documents should be written with the average reading level of the target
group in mind, using plain language (J. B. Green, Duncan, Barnes, &
Oberklaid, 2003; Houts et al., 2006; Lorenzen, Melby, & Earles, 2008;
Terranova et al., 2012). Guidelines on writing research information and other
medical information material are necessary. In the Netherlands, where the
current study was performed, the CCMO (2008) provided a writing guide for
consent documents. However, these guidelines mention only the preferred
Flesch Reading Ease for adults (58-65) but do not mention pediatric docu-
ments. The same is true for the Second Evaluation of Research Involving
Human Subjects Act, which contains advice to write clear texts for adults, but
no word is provided on the pediatric situation (Stukart et al., 2012).
Therefore, we suggest that pediatric material should be written at Grade
Level 6 or 7 or a corresponding Flesch Reading Ease of 80. This level indi-
cates that texts are readable for someone who has received 6 or 7 years of
reading education, which applies to children of 12 or 13 years old.
We are aware that changing length of words and syllables as measured by
these indexes is not the only factor to improve understanding. Also, at times
it can be impossible to approach this high readability when explaining medi-
cal information. And even when a suitable readability is met, the medical
terminology might discourage the reader and reduce comprehension.
Therefore, we strongly encourage the use of visuals in research information
forms. There is a strong theoretical framework for the supporting effect of
visuals in reading comprehension, even more so in younger children with a
relatively inefficient working memory capacity and higher risk for cognitive
overload (Glenberg & Langston, 1992; Pike et al., 2010). Our study has dem-
onstrated that even text with a readability that is theoretically too low can be
understood by children when sufficient visual support is provided. Visuals
can motivate children to pick up a text and keep reading to the end, and are a
powerful tool to increase their comprehension of the medical information.
The forms studied were directed at the age range of 12 to 18, in which chil-
dren develop into adolescents and preferences change. Therefore, further
studies are needed to investigate visual use and preferences among adoles-
cents ages 13 to 17. Future research should address the effectiveness of visu-
als in medical information (both research and treatment) in a nonsimulated
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
112 Science Communication 37(1)
setting, with the entire form instead of an excerpt, tested in the hospital with
prospective research or treatment participants.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research,
authorship, and/or publication of this article: This work was supported by The
Netherlands Organisation for Health Research and Development (Project No.
113203016).
References
Austin, P. E., Matlack, R., II, Dunn, K. A., Kesler, C., & Brown, C. K. (1995).
Discharge instructions: Do illustrations help our patients understand them?
Annals of Emergency Medicine, 25, 317-320.
Baker, J. N., Leek, A. C., Salas, H. S., Drotar, D., Noll, R., Rheingold, S. R., &
Kodish, E. D. (2013). Suggestions from adolescents, young adults, and parents
for improving informed consent in phase 1 pediatric oncology trials. Cancer,
119, 4154-4161. doi:10.1002/cncr.28335
Barnes, M. A., Raghubar, K. P., Faulkner, H., & Denton, C. A. (2014). The construc-
tion of visual-spatial situation models in children’s reading and their relation to
reading comprehension. Journal of Experimental Child Psychology, 119, 101-111.
Barnett, K., Harrison, C., Newman, F., Bentley, C., & Cummins, C. (2005). A ran-
domised study of the impact of different styles of patient information leaflets for
randomised controlled trials on children’s understanding. Archives of Disease in
Childhood, 90, 364-366. doi:10.1136/adc.2003.034041
Barrett, R. (2005). Quality of informed consent: Measuring understanding among
participants in oncology clinical trials. Oncology Nursing Forum, 32, 751-755.
doi:10.1188/05.ONF.751-755
Berto, D., Peroni, M., Milleri, S., & Spagnolo, A. G. (2000). Evaluation of the read-
ability of information sheets for healthy volunteers in phase-I trials. European
Journal of Clinical Pharmacology, 56, 371-374.
Brookshire, J., Scharff, L. F. V., & Moses, L. E. (2002). The influence of illustra-
tions on children’s book preferences and comprehension. Reading Psychology,
23, 323-339.
Burke, T. M., Abramovitch, R., & Zlotkin, S. (2005). Children’s understanding of
the risks and benefits associated with research. Journal of Medical Ethics, 31,
715-720.
Carlson, S. E., Seipel, B., & McMaster, K. (2014). Development of a new reading
comprehension assessment: Identifying comprehension differences among read-
ers. Learning and Individual Differences, 32, 40-53.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 113
Carney, R. N., & Levin, J. R. (2002). Pictorial illustrations still improve students’
learning from text. Educational Psychology Review, 14, 5-26.
CCMO. (2008). Schrijfwijzer informatiebrief proefpersoon [Writing guide for
research participant information]. Retrieved from http://www.ccmo.nl/attach-
ments/files/schrijfwijzer-versie-3-11-08-1.pdf
CCMO. (2012). Jaarverslag 2012 [CCMO yearly report 2012]. Retrieved from http://
www.ccmo.nl/attachments/files/ccmo-jaarverslag-2012.pdf
Chappuy, H., Doz, F., Blanche, S., Gentet, J. C., & Treluyer, J. M. (2008). Children’s
views on their involvement in clinical research. Pediatric Blood & Cancer, 50,
1043-1046. doi:10.1002/pbc.21359
Delp, C., & Jones, J. (1996). Communicating information to patients: The use of
cartoon illustrations to improve comprehension of instructions. Academic
Emergency Medicine, 3, 264-270.
De Vries, M. C., Wit, J. M., Engberts, D. P., Kaspers, G. J., & van Leeuwen, E.
(2010). Pediatric oncologists’ attitudes towards involving adolescents in deci-
sion-making concerning research participation. Pediatric Blood & Cancer, 55,
123-128. doi:10.1002/pbc.22510
Douma, W. H. (1960). De leesbaarheid van landbouwbladen: een onderzoek naar
en een toepassing van leesbaarheidsformules Bulletin No. 17 [The readability
of agricultural journals: A study to and application of readability formulas].
Wageningen, Netherlands: Landbouwhogeschool.
Dowse, R., & Ehlers, M. S. (2001). The evaluation of pharmaceutical pictograms in
a low-literate South African population. Patient Education and Counseling, 45,
87-99.
European Union. (2001). Directive 2001/20/EC of the European Parliament and of
the Council of 4 April 2001 on the approximation of the laws, regulations and
administrative provisions of the member states relating to the implementation of
good clinical practice in the conduct of clinical trials on medicinal products for
human use. Official Journal of the European Communities, 44, 34-44. Retrieved
from http://www.eortc.be/services/doc/clinical-eu-directive-04-april-01.pdf
Fillippatou, D., & Pumfrey, P. D. (1996). Pictures, titles, reading accuracy and read-
ing comprehension: A research review (1973–95). Educational Research, 38,
259-291.
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32,
221-233.
Ford, K., Sankey, J., & Crisp, J. (2007). Development of children’s assent docu-
ments using a child-centred approach. Journal of Child Health Care, 11, 19-28.
doi:10.1177/1367493507073058
Franck, L., & Winter, I. (2004). Research participant information sheets are difficult
to read. Bulletin of Medical Ethics, March, 13-16.
Geller, G., Tambor, E. S., Bernhardt, B. A., Fraser, G., & Wissow, L. S. (2003).
Informed consent for enrolling minors in genetic susceptibility research: A quali-
tative study of at-risk children’s and parents’ views about children’s role in deci-
sion-making. Journal of Adolescent Health, 32, 260-271.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
114 Science Communication 37(1)
Gill, D. (2003). Guidelines for informed consent in biomedical research involving
paediatric populations as research participants. European Journal of Pediatrics,
162, 455-458.
Gillies, P. A., Stork, A., & Bretman, M. (1990). Streetwize UK: A controlled trial of
an AIDS education comic. Health Education Research, 5, 27-33.
Glenberg, A. M., & Langston, W. E. (1992). Comprehension of illustrated text:
Pictures help to build mental models. Journal of Memory and Language, 31,
129-151.
Green, J. B., Duncan, R. E., Barnes, G. L., & Oberklaid, F. (2003). Putting the
“informed” into “consent”: A matter of plain language. Journal of Paediatrics
and Child Health, 39, 700-703.
Green, M. J., & Myers, K. R. (2010). Graphic medicine: Use of comics in medical
education and patient care. British Medical Journal, 340, c863. doi:10.1136/bmj.
c863
Grootens-Wiegers, P., De Vries, M. C., & Van Den Broek, J. M. (2014). Research
information for minors: suitable formats and readability: A systematic review.
Journal of Paediatrics and Child Health. Advance online publication.
Gunning, R. (1968). The fog index after twenty years. Journal of Business
Communication, 6, 3-13.
Hochhauser, M. (2007). Consent forms–No easy read. Retrieved from http://
www.appliedclinicaltrialsonline.com/appliedclinicaltrials/author/authorInfo.
jsp?id=5034
Hosey, G. M., & Stracqualursi, F. (1990). Designing and evaluating diabetes educa-
tion materials for American Indians. The Diabetes Educator, 16, 407-408.
Houts, P. S., Doak, C. C., Doak, L. G., & Loscalzo, M. J. (2006). The role of pictures
in improving health communication: A review of research on attention, compre-
hension, recall, and adherence. Patient Education and Counseling, 61, 173-190.
doi: 10.1016/j.pec.2005.05.004
Houts, P. S., Witmer, J. T., Egeth, H. E., Loscalzo, M. J., & Zabora, J. R. (2001).
Using pictographs to enhance recall of spoken medical instructions II. Patient
Education and Counseling, 43, 231-242.
John, T., Hope, T., Savulescu, J., Stein, A., & Pollard, A. J. (2008). Children’s con-
sent and paediatric research: Is it appropriate for healthy children to be the deci-
sion-makers in clinical research? Archives of Disease in Childhood, 93, 379-383.
doi:10.1136/adc.2007.118299
Kaplan, W., Wirtz, J., Mantel-Teeuwisse, A., Stolk, P., Duthey, B., & Laing, R.
(2013). Priority medicines for Europe and the world: 2013 update. Geneva,
Switzerland: World Health Organization.
Kass, N. E., Chaisson, L., Taylor, H. A., & Lohse, J. (2011). Length and complex-
ity of US and international HIV consent forms from federal HIV network trials.
Journal of General Internal Medicine, 26, 1324-1328. doi:10.1007/s11606-011-
1778-6
Katz, M. G., Kripalani, S., & Weiss, B. D. (2006). Use of pictorial aids in medica-
tion instructions: A review of the literature. American Journal of Health-System
Pharmacy, 63, 2391-2397. doi:10.2146/ajhp060162
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 115
Klingbeil, C., Speece, M. W., & Schubiner, H. (1995). Readability of pediatric patient
education materials: Current perspectives on an old problem. Clinical Pediatrics,
34, 96-102.
Lewis, C. E., Lewis, M. A., & Ifekwunigue, M. (1978). Informed consent by chil-
dren and participation in an influenza vaccine trial. American Journal of Public
Health, 68, 1079-1082.
Lorenzen, B., Melby, C. E., & Earles, B. (2008). Using principles of health literacy to
enhance the informed consent process. AORN Journal, 88, 23-29. doi:10.1016/j.
aorn.2008.03.001
Mansoor, L. E., & Dowse, R. (2003). Effect of pictograms on readability of patient
information materials. Annals of Pharmacotherapy, 37, 1003-1009.
Mayer, R. E. (2001). Multimedia learning. New York, NY: Cambridge University
Press.
Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia
learning. Educational Psychology, 38, 43-52.
Menoni, V., Lucas, N., Leforestier, J. F., Doz, F., Chatellier, G., Jacqz-Aigain, E.,
. . .Chappuy, H. (2011). Readability of the written study information in pediatric
research in France. PLoS One, 6(4), e18484. doi:10.1371/journal.pone.0018484
Michielutte, R., Bahnson, J., Digman, M. B., & Schroeder, E. (1992). The use of
illustrations and narrative text style to improve readability of a health education
brochure. Journal of Cancer Education, 7, 251-260.
Moll, J. M. (1986). Doctor–patient communication in rheumatology: Studies of visual
and verbal perception using educational booklets and other graphic material.
Annals of the Rheumatic Diseases, 45, 198-209.
Ogloff, J. R., & Otto, R. K. (1991). Are research participants truly informed?
Readability of informed consent forms used in research. Ethics & Behavior, 1,
239-252. doi:10.1207/s15327019eb0104_2
Ondrusek, N., Abramovitch, R., Pencharz, P., & Koren, G. (1998). Empirical exami-
nation of the ability of children to consent to clinical research. Journal of Medical
Ethics, 24, 158-165.
Paasche-Orlow, M. K., Taylor, H. A., & Brancati, F. L. (2003). Readability standards
for informed-consent forms as compared with actual readability. New England
Journal of Medicine, 348, 721-726. doi:10.1056/NEJMsa021212
Pike, M. M., Barnes, M. A., & Barron, R. W. (2010). The role of illustrations in chil-
dren’s inferential comprehension. Journal of Experimental Child Psychology,
105, 243-255.
R Development Core Team. (2011). R: A language and environment for statistical
computing [Computer software]. Vienna, Austria: R Foundation for Statistical
Computing.
Raich, P. C., Plomer, K. D., & Coyne, C. A. (2001). Literacy, comprehension, and
informed consent in clinical research. Cancer Investigation, 19, 437-445.
Readance, J. E., & Moore, D. W. (1981). A meta-analytic review of the effect of
adjunct pictures on reading comprehension. Psychology in the Schools, 18,
218-224.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
116 Science Communication 37(1)
Redding, R. E. (1993). Children’s competence to provide informed consent for mental
health treatment. Washington and Lee Law Review, 50, 695-753.
Roter, D. L., Rudd, R. E., Keogh, J., & Robinson, B. (2006). Worker produced health
education material for the construction trades, 1986-87. International Quarterly
of Community Health Education, 27, 231-243. doi:10.2190/IQ.27.3.d
Rugege-Hakiza, S. E., Glynn, S. A., Hutching, S. T., Bethel, J., Nass, C. C., McEntire,
R. L., . . .Schreiber, G. B. (2003). Do blood donors read and understand screening
educational materials? Transfusion, 43, 1075-1083.
Sanders, L. M., Federico, S., Klass, P., Abrams, M. A., & Dreyer, B. (2009). Literacy
and child health: A systematic review. Archives of Pediatrics & Adolescent
Medicine, 163, 131-140. doi:10.1001/archpediatrics.2008.539
Sharp, S. M. (2004). Consent documents for oncology trials: Does anybody read these
things? American Journal of Clinical Oncology, 27, 570-575.
Souza, M. K., Jacob, C. E., Gama-Rodrigues, J., Zilberstein, B., Cecconello, I., &
Habr-Gama, A. (2013). The written informed consent form (WICF): Factors that
interfere with acceptance. Arquivos Brasileiros de Cirurgia Digestiva, 26, 200-
205.
Stukart, M., Olsthoorn-Heim, E., van de Vathorst, S., van der Heide, A., Tromp, K.,
& de Klerk, C. (2012). Tweede evaluatie Wet medisch-wetenschappelijk onder-
zoek met mensen [Second evaluation of Research Involving Human Subjects
Act] (Reeks evaluatie wetgeving, Vol. 30). Retrieved from http://www.zonmw.
nl/uploads/tx_vipublicaties/evaluatie_wmo_webversie_17x24_-_nieuw.pdf
Sudore, R. L., Landefeld, C. S., Williams, B. A., Barnes, D. E., Lindquist, K., &
Schillinger, D. (2006). Use of a modified informed consent process among vul-
nerable patients: A descriptive study. Journal of General Internal Medicine, 21,
867-873. doi:10.1111/j.1525-1497.2006.00535.x
Swartling, U., Hansson, M. G., Ludvigsson, J., & Nordgren, A. (2011). “My parents
decide if I can. I decide if I want to”: Children’s views on participation in medical
research. Journal of Empirical Research on Human Research Ethics, 6(4), 68-75.
Tait, A. R., Voepel-Lewis, T., & Malviya, S. (2003). Do they understand? (Part
II): Assent of children participating in clinical anesthesia and surgery research.
Anesthesiology, 98, 609-614.
Tait, A. R., Voepel-Lewis, T., & Malviya, S. (2007). Presenting research informa-
tion to children: A tale of two methods. Anesthesia & Analgesia, 105, 358-364.
doi:10.1213/01.ane.0000270326.44507.11
Tarnowski, K. J., Allen, D. M., Mayhall, C., & Kelly, P. A. (1990). Readability of
pediatric biomedical research informed consent forms. Pediatrics, 85, 58-62.
Terranova, G., Ferro, M., Carpeggiani, C., Recchia, V., Braga, L., Semelka, R. C., &
Picano, E. (2012). Low quality and lack of clarity of current informed consent
forms in cardiology: How to improve them. JACC Cardiovascular Imaging, 5,
649-655. doi:10.1016/j.jcmg.2012.03.007
Unguru, Y., Sill, A. M., & Kamani, N. (2010). The experiences of children enrolled in
pediatric oncology research: Implications for assent. Pediatrics, 125, e876-883.
doi: 10.1542/peds.2008-3429
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
Grootens-Wiegers et al. 117
UNICEF. (1989). Convention on the Rights of the Child, In Treaty series (Vol. 1577,
p. 3). Retrieved from https://treaties.un.org/doc/Publication/UNTS/Volume%20
1577/v1577.pdf
Van Der Pal, S., Sozanska, B., Madden, D., Kosmeda, A., Debinska, A., Danielewicz,
H., . . . Detmar, S. (2010). Opinions of children about participation in medical
genetic research. Public Health Genomics, 14, 271-278.
van Oosten, P., Tanghe, D., & Hoste, V. (2010). Towards an improved methodology
for automated readability prediction. In Proceedings of the seventh conference
on International Language Resources and Evaluation (LREC’10) (pp. 775-782).
Valletta, Malta: ELRA.
World Health Organization. (1981). The WHO/CIOMS guidelines, proposed inter-
national guidelines for biomedical research involving human subjects. Geneva,
Switzerland: Author.
Author Biographies
Petronella Grootens-Wiegers, MSc, is a research associate in the Department of
Science Communication & Society, Leiden University, The Netherlands. She is cur-
rently conducting her PhD study, for which she develops and studies targeted informed
consent material for children.
Martine C. De Vries, MD, PhD, is pediatrician and medical ethicist at the Leiden
University Medical Centre. Her research interests include research ethics and child
participation in decision making. She chairs the Committee on Ethics and Health Law
of the Dutch Paediatric Association.
Tessa E. Vossen, BSc, is an MSc student in biology, currently conducting an intern-
ship at the Department of Science Communication & Society, Leiden University, The
Netherlands. She is interested in research on science communication for children and
health communication in ethnic minorities.
Jos M. Van den Broek, PhD, earned a Knight Science Journalism Fellowship at
MIT, Cambridge, in 1989-1990. After having been a science communicator for over
20 years he now is a professor in biomedical science communication at Leiden
University, with a keen interest in visual communication and health communication
for children as well as low-literate individuals.
at Universiteit Leiden \ LUMC on March 31, 2015scx.sagepub.comDownloaded from
... One of the few studies focused on medical communication for children assessed the readability and use of visuals in medical research informational consent forms in the Netherlands [Grootens-Wiegers et al., 2015]. This study found low readability scores for consent forms and noted only three of the 22 forms used visuals [Grootens-Wiegers et al., 2015]. ...
... One of the few studies focused on medical communication for children assessed the readability and use of visuals in medical research informational consent forms in the Netherlands [Grootens-Wiegers et al., 2015]. This study found low readability scores for consent forms and noted only three of the 22 forms used visuals [Grootens-Wiegers et al., 2015]. The researchers also noted that children preferred forms with visuals, especially realistic images and funny cartoons, as well as forms that included informative, clear text or captions [Grootens-Wiegers et al., 2015]. ...
... This study found low readability scores for consent forms and noted only three of the 22 forms used visuals [Grootens-Wiegers et al., 2015]. The researchers also noted that children preferred forms with visuals, especially realistic images and funny cartoons, as well as forms that included informative, clear text or captions [Grootens-Wiegers et al., 2015]. The researchers noted, "there is little research on the quality of medical information for children and adolescents, and there are no evidence-based insights in how health communication can be optimally adapted for this target group" [Grootens-Wiegers et al., 2015, p. 91]. ...
Article
Full-text available
YouTube videos offer a potentially useful vehicle for the communication of science, health, and medical information about COVID-19 to children. Findings from this research showed that primary characters appearing in children's educational YouTube videos about COVID-19 were most often adults, with about an equal number of men and women and few characters from diverse racial and ethnic backgrounds. Primary characters frequently demonstrated and modeled protective health measures. Adult expert characters (medical professionals and scientists) appeared to some extent in these videos. Directive discourse frames appeared most frequently, followed by the informative and persuasive discourse frames when communicating scientific and health information. Changes in the use of informative, directive, and persuasive frames before and after the U.S. Centers for Disease Control (CDC) announced guidelines on how to communicate about COVID-19 with children are explored.
... Enhanced communication between doctor and patient can also be established with the help of visualization. Moreover, the ease for understanding and with the memorizing/recalling ability of information visually relayed would be more effective, entertaining and engaging for patients to collaborate and take part in the decision making process, especially with the pediatric population and their family members [26], [27]. Visualizing Uncertainty Injected through Machine Learning: In decision-making, the ability to observe the nature or the degree of uncertainty injected by the design nature of the machine learning algorithm and the mechanics of its implementation is also essential at figuring the extent at which our results can be relied upon. ...
Preprint
Full-text available
With the advent of sophisticated machine learning (ML) techniques and the promising results they yield, especially in medical applications, where they have been investigated for different tasks to enhance the decision-making process. Since visualization is such an effective tool for human comprehension, memorization, and judgment, we have presented a first-of-its-kind estimation approach we refer to as Visualized Learning for Machine Learning (VL4ML) that not only can serve to assist physicians and clinicians in making reasoned medical decisions, but it also allows to appreciate the uncertainty visualization, which could raise incertitude in making the appropriate classification or prediction. For the proof of concept, and to demonstrate the generalized nature of this visualized estimation approach, five different case studies are examined for different types of tasks including classification, regression, and longitudinal prediction. A survey analysis with more than 100 individuals is also conducted to assess users' feedback on this visualized estimation method. The experiments and the survey demonstrate the practical merits of the VL4ML that include: (1) appreciating visually clinical/medical estimations; (2) getting closer to the patients' preferences; (3) improving doctor-patient communication, and (4) visualizing the uncertainty introduced through the black box effect of the deployed ML algorithm. All the source codes are shared via a GitHub repository.
... The used tool is not the most accurate or modern but it is used and can provide accurate results. 25 Also, in Flanders the use of ICF is not regulated by law and can make it difficult to create a consensus on them. Brochures are more used, 18 which should be taken into account for further researches. ...
Article
Informed consent forms have been useful in clinical practice and they constitute a part of the shared decision making in the informed consent process. They provide information to patients about clinical procedures and techniques. They also act as a remainder of the information discussed after the medical interview. Sometimes these documents are not readable to everybody. Belgian law specifies that all information that patients receive has to be proportionate verbally, but written information is also handled. The present research analyzes the readability of the Flemish informed consent forms located in the webs of all General Hospitals using a simple random sample of 75 informed consent forms. By using the Douma tool, which bases its analysis in the length of words and sentences, the readability mean of the sample was 46, level “Difficult”. The 59% of them had a difficult level. The 11% were normal. It is a fact, then, that the 59% of the informed consent forms evaluated in this study are not suitable for everybody in Flanders, especially those people with low literacy. There were some researches made in other countries that agreed with these results. Written clinical information was poorly written so the informed consent forms were not working helping patients to recall information nor helping patients to become a part in the shared decision making about their health. The use of readability formulas represented a simple way to discriminate those informed consent forms that had normal readability scores from those that should be adapted.
... The earliest approach is the use of simple readability formulas such as the Flesch-Kincaid formula (Kincaid et al., 1975) or the Dale-Chall readability formula (Chall and Dale, 1995); see DuBay (2006) for an overview. These formulas are still widely used in non-linguistic studies (Esfahani et al., 2016;Grootens-Wiegers et al., 2015) and in information retrieval systems (cf. Section 2.1). ...
Conference Paper
Full-text available
We present KANSAS, a search engine designed to retrieve reading materials for functional illiterates and learners of German as a Second Language. The system allows teachers to refine their searches for teaching material by selecting appropriate readability levels and (de)prioritizing linguistic constructions. In addition to this linguistically-informed query result ranking, the system provides visual input enhancement for the selected linguistic constructions. Our system combines state-of-the-art Natural Language Processing (NLP) with lightweight algorithms for the identification of relevant linguistic constructions. We have evaluated the system in two pilot studies in terms of the identification of linguistic constructions and the identification of readability levels. Both pilots achieved highly promising results and are being followed by full-fledged performance studies and usability tests.
... Automatic readability assessment matches texts to readers with a certain literacy skill such that they can fulfill a predefined reading goal or task such as extracting information from a text. Early work on readability assessment started with readability formulas (Kincaid et al., 1975;Chall and Dale, 1995) which are still used in some studies (Grootens-Wiegers et al., 2015;Esfahani et al., 2016) despite having been widely criticized for being too simplistic and unreliable Benjamin, 2012). In answer to this criticism, more advanced methods supporting broader linguistic modeling using Natural Language Processing (NLP) were established. ...
Conference Paper
Full-text available
Reading material that is of interest and at the right level for learners is an essential component of effective language education. The web has long been identified as a valuable source of reading material due to the abundance and variability of materials it offers and its broad range of attractive and current topics. Yet, the web as source of reading material can be problematic in low literacy contexts. We present ongoing work on a hybrid approach to text retrieval that combines the strengths of web search with retrieval from a high-quality, curated corpus resource. Our system, KANSAS Suche 2.0, supports retrieval and reranking based on criteria relevant for language learning in three different search modes: unrestricted web search, filtered web search, and corpus search. We demonstrate their complementary strengths and weaknesses with regard to coverage, readability, and suitability of the retrieved material for adult literacy and basic education. We show that their combination results in a very versatile and suitable text retrieval approach for education in the language arts.
... In one study in the Netherlands Grootens-Wiegers et al. (2015) examined readability and use of text and visuals in medical research information forms for children, and adolescents. In the Netherlands, children from the age of 12 are legally allowed to co-decide on research participation together with their parents. ...
Book
Full-text available
Traditionally the concept of “literacy” was restricted to the ability to read, write and use arithmetic. In a multicultural world with fast technological advances people in all societies need abilities and skills to manage many kinds of systems for communication and information provide in imag-es, symbols, and texts. We all have to learn to interpret visual messages accurately and to create such messages. Interpretation and creation in visual literacy can be said to parallel read-ing and writing in print literacy.
... Furthermore, adolescents have expectations regarding the readability of documents directed to them. Grootens-Wiegers, De Vries, Vossen, and Van den Broek (2015) conducted focus groups with 77 adolescents (11-and 12-year-olds) to discuss their perceptions of a pediatric research information form that had a Flesch reading ease score of 55.43 (i.e., fairly difficult to read). Participants found the form hard to read and expressed the need for a better explanation of the meaning of scientific concepts. ...
Article
The authors examined adolescents’ detection of features that affect the quality of web information. In experiment 1, participants (12–16 years old) rated the goodness/usefulness of four web‐like documents for a simulated study assignment. Each document came with an issue that potentially undermined its quality. Two documents had source‐related issues (i.e., noncompetent author, outdated), and the other two documents had content‐related issues (i.e., topic mismatch, poor readability). Most students failed to notice the issues, including topic mismatch. The participants also produced inconsistent evaluations of topic match, readability, author competence, and currency. In experiment 2, students were prompted to assess each criterion separately. The participants distinguished poorer from better documents in relation to each criterion, except for author competence. The authors discuss these results in light of previous research on adolescents’ evaluation behavior, propose further avenues for reading research, and articulate recommendations for educational practice.
Article
This article tests a novel method for starting focus groups quickly while ensuring that respondents understand the topic of interest for the study. To kick start the focus groups, a cartoon style illustration was developed. The cartoons depicted various food service experience from start to finish. Respondents first task was to indicate when it was natural for them to give feedback on the food service experience. The results show that the cartoons allowed respondents to quickly understand the topic, which in turn gave them confidence to contribute with relevant information from the start. In addition, respondents used the framework of the illustrations to keep the discussion focused on the topic throughout. We generally find that it is getting harder to find people who are willing to commit to 2hour focus groups after work. For us to find participants who are willing to take part, we are having to be much more time conscious, this method allows for shorter groups without losing valuable information. The method has a potential when it comes to testing food products as well as in the development of dishes for gastronomic experiences. It may also be a good tool to use in sensory focus groups. The use of comics to start off the focus group worked well in the context of a non-sensitive topic, in this case food service experiences. Further research could explore using this method for more sensitive topics.
Article
There is global acceptance that individuals should be allowed to decide whether or not to take part in research studies, and to do so after being informed about the nature of the research and the risk that might attach to participation. The process of providing detailed information before seeking consent (formalized by signatures) in advance of undertaking research procedures may not be possible in some circumstances, and sometimes an amended approach may be adopted. The use of opt-out consent has been recognized as a valid and ethical means of recruiting participants to studies particularly with large samples and where the risk to participants is small. However, it is sometimes misunderstood and can be a problematic factor in being accepted by research ethics committees and governing authorities. This may be due partly to differing expectations of the amount of information and support offered, together with the nature of the process that is adopted to ensure that a decision has been made rather than consent simply being assumed. In accordance with ongoing discussions with young people, and following consultation with parents, an opt-out consent strategy including varied means of providing information was employed in a large study of 44,501 cases of children attending emergency or urgent care departments. The study was conducted over more than 12 months in dissimilar emergency departments and an urgent care unit, and was designed to support better decision-making in pediatric emergency departments about whether children need to be admitted to hospital or can be discharged home safely. Robust analysis of the factors that exerted the greatest impact on predicting the need to admit or the safety of discharging children led to a revised version of an existing tool. In this article, we review approaches to consent in research, the nature and impact of opt-out consent, the factors that made this an effective strategy for this study, but also more recent concerns which may make opt-out consent no longer acceptable.
Article
Full-text available
Research conducted primarily during the 1970s and 1980s supported the assertion that carefully constructed text illustrations generally enhance learners' performance on a variety of text-dependent cognitive outcomes. Research conducted throughout the 1990s still strongly supports that assertion. The more recent research has extended pictures-in-text conclusions to alternative media and technological formats and has begun to explore more systematically the “whys,” “whens,” and “for whoms” of picture facilitation, in addition to the “whethers” and “how muchs.” Consideration is given here to both more and less conventional types of textbook illustration, with several “tenets for teachers” provided in relation to each type.
Article
Full-text available
The written informed consent form (WICF) provides information that must be written in simple, easily understood language, highlighting voluntary participation safeguards, risks, possible benefits, and procedures. Currently, the possibility that research subjects do not fully understand the text of the WICF or their rights as participants, despite having signed the WICF and agreed to participate in the study, has been a point of discussion. To evaluate the readability of the WICFs, as well as to correlate research subject acceptance of the WICF with demographic status, social factors, risk-benefit relationship, and education level. The study involved 793 patients treated in public or private hospitals and asked to give informed consent for their inclusion. Were reviewed patient medical charts in order to obtain demographic and social data, and was used the Flesch Reading Ease and the Flesch-Kincaid Readability Indices to evaluate the reading level of the WICF texts. Acceptance was higher (99.7%) among patients treated in public health care facilities and among patients (99.73%) who participated in protocols involving lower risk. Although acceptance was not influenced by education level, 462 patients (58.26%) had eight or less years of schooling. The obtained readability index ranged from 9.9 to 12 on the Flesch-Kincaid test, and from 33.1 to 51.3 on the Flesch Reading Ease test. The WICFs had high degree of reading difficulty. Although patient acceptance was not found to be related to demographic or social factors, it was found to be influenced by the risk-benefit relationship.
Article
Full-text available
First, we propose a theory of multimedia learning based on the assumptions that humans possess separate systems for processing pictorial and verbal material (dual-channel assumption), each channel is limited in the amount of material that can be processed at one time (limited-capacity assumption), and meaningful learning involves cognitive processing including building con- nections between pictorial and verbal representations (active-processing assumption). Second, based on the cognitive theory of multimedia learning, we examine the concept of cognitive over- load in which the learner's intended cognitive processing exceeds the learner's available cogni- tive capacity. Third, we examine five overload scenarios. For each overload scenario, we offer one or two theory-based suggestions for reducing cognitive load, and we summarize our re- search results aimed at testing the effectiveness of each suggestion. Overall, our analysis shows that cognitive load is a central consideration in the design of multimedia instruction.
As children age, their capacity to consent or dissent to research participation increases. Numerous regulations and guidelines require that children should receive information 'according to their capacity of understanding'. In order to gain more insight in the quality of patient information forms for minors, a systematic literature search was performed. Two aspects of quality will be analysed in this paper: the effect of format on understanding and the readability of text in the documents. A systematic search was executed in PubMed, Embase and PsycINFO. Seventeen papers on format were included. Interventions testing information formats indicate that improvement is possible, but outcome measurement varied per study and no apparently successful intervention was repeated. Only three readability papers were found, all indicating a readability gap between patient information forms and children's actual reading level. The results indicate an urgent need for further research on how to adequately inform minors about clinical trials.
Article
Readers construct mental models of situations described by text to comprehend what they read, updating these situation models based on explicitly described and inferred information about causal, temporal, and spatial relations. Fluent adult readers update their situation models while reading narrative text based in part on spatial location information that is consistent with the perspective of the protagonist. The current study investigated whether children update spatial situation models in a similar way, whether there are age-related changes in children's formation of spatial situation models during reading, and whether measures of the ability to construct and update spatial situation models are predictive of reading comprehension. Typically developing children from 9 to 16years of age (N=81) were familiarized with a physical model of a marketplace. Then the model was covered, and children read stories that described the movement of a protagonist through the marketplace and were administered items requiring memory for both explicitly stated and inferred information about the character's movements. Accuracy of responses and response times were evaluated. Results indicated that (a) location and object information during reading appeared to be activated and updated not simply from explicit text-based information but from a mental model of the real-world situation described by the text; (b) this pattern showed no age-related differences; and (c) the ability to update the situation model of the text based on inferred information, but not explicitly stated information, was uniquely predictive of reading comprehension after accounting for word decoding.
Article
Informed consent for a pediatric oncology phase 1 trial is a delicate process, and is made more complex by the difficulty of the information and the requirement for parental consent, and patient assent when applicable. This analysis identifies suggestions for improving the informed consent process received from parents and adolescent and young adult patients (aged 14 years-21 years) who had the option of participating in a phase 1 pediatric oncology trial. A total of 57 parents and 20 patients completed interviews as part of a multisite, prospective, descriptive study. These transcribed interviews were studied using established content analysis methods. Parent and patient responses contained 220 suggestions and 54 suggestions, respectively. A total of 21 unique suggestions for improvement emerged in 3 main themes: 1) provision of more information; 2) structure and presentation of the informed consent process, and 3) suggestions for physicians conducting the process. Common suggestions included providing more specific information about the trial, allowing more time for decision-making, and using different methods to deliver information. Participants involved in the informed consent process for a phase 1 trial provided specific recommendations to research teams to enhance the process. Physician/investigators should be informed of these recommendations and develop and test interventions incorporating them. Cancer 2013. © 2013 American Cancer Society.
Article
For which readers and texts are different types of Pictures and/or Titles ‘worth a thousand words'? Pictures and Titles are extensively used in reading materials under the assumption that they enhance individuals’ motivation for reading, and that they also facilitate reading. This paper presents findings from studies into the effects of Pictures and/or Titles on the reading and comprehension of continuous printed text.The transfer‐appropriate processing hypothesis encourages the specification of conditions under which Pictures and/or Titles enhance or interfere with children's text‐processing skills. Such studies contribute to an evolving theory of the cognitive conditions required for meaningful learning from materials comprising continuous text, Pictures and Titles.Four groups of researches are summarized. The first considers the effects of Pictures on Reading Comprehension (17 studies). The second summarizes work on effects of Pictures on both Reading Accuracy and Reading Comprehension (three studies). Turning to Titles, their effects on comprehension are presented (five studies). The fourth section summarizes studies into the effects of both Pictures and Titles on Reading Comprehension (four studies). Finally, one study that simultaneously examines the effects of both Pictures and Titles in relation to Reading Accuracy and Reading Comprehension is presented.Three main points emerge from the Picture and Title research reviewed. The first is that Pictures and Titles are not uniformly effective in all prose‐reading situations. The second is that not all types of Pictures and Titles are equally effective for children with differing reading abilities. Both of these points are, in part, a consequence of methodological variations between researches, coupled with their respective limitations. Despite such limitations, an appreciation of such work suggests that teachers and textbook producers could increase slightly the likelihood of Pictures and Titles being effective in facilitating Reading Accuracy and Reading Comprehension for particular groups of readers. The third point raised by this survey is the challenge inherent in the first two. Can Pictures and/or Titles that systematically and differentially affect the accessibility of running text be constructed for children with different reading attainments and learning styles working with text of various genres?
Article
This study evaluated the impact of an AIDS education comic on the knowledge, attitudes and behavioural intentions of 14-year-old school pupils. In a controlled trial, 122 experimental pupils matched with 162 control pupils completed a confidential questionnaire prior to the educational intervention and again 2 weeks later. No differences between groups were recorded at pre-testing. Significantly higher mean levels of knowledge were recorded at post-comic testing in the experimental compared with the control group. Knowledge was gained in key areas covering how HIV is transmitted and the way in which condoms could be used to reduce the risk. The comic also had a significant impact on attitudes and beliefs but little short-term effect on behavioural intentions.