Content uploaded by Matthew Motta
Author content
All content in this area was uploaded by Matthew Motta on Apr 04, 2020
Content may be subject to copyright.
Reducing the Administrative Demands of the Science
Curiosity Scale (SCS): A Validation Study
Matt Motta∗
Postdoctoral Fellow
Annenberg Public Policy Center
University of Pennsylvania
matthew.motta@appc.upenn.edu
Dan Chapman
Postdoctoral Fellow
Annenberg Public Policy Center
University of Pennsylvania
daniel.chapman@appc.upenn.edu
Kathryn Haglin
Postdoctoral Fellow
Annenberg Public Policy Center
University of Pennsylvania
matthew.motta@appc.upenn.edu
Dan Kahan
Professor
Yale Law School
Yale University
dan.kahan@yale.edu
June 22, 2019
Science curious people – those who enjoy consuming science-related information – are less
likely to hold politically polarized views about contentious science. Consequently, science curios-
ity is of great interest to scholars across the social sciences. However, measuring science curiosity
via the science curiosity scale (SCS) is time intensive; potentially impeding its widespread us-
age. We present two new methods for reducing SCS administration time. One method presents
respondents with a randomly selected subset of items (“Random Subset Method; RS”). The
other asks all respondents a core set of just four items (“Reduced-Form Method; RF”). In three
nationally representative surveys, we assess the construct, convergent, and predictive validity
of these alternatives. We find both versions to be well validated.
1
Author Biographies:
Matt Motta is a Science of Science Communication postdoctoral fellow at the Annenebrg
Public Policy Center at the University of Pennsylvania, and is based at the Yale Law School.
Dan Chapman is a Science of Science Communication postdoctoral fellow at the Annenebrg
Public Policy Center at the University of Pennsylvania, and is based at the Yale Law School.
Kathryn Haglin is a Science of Science Communication postdoctoral fellow at the Annene-
brg Public Policy Center at the University of Pennsylvania, and is based at the Yale Law School.
Dan Kahan is the Elizabeth K. Dollard Professor of law at Yale Law School.
2
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 1
Science curious people – those who enjoy consuming science-related information – are
less likely to hold politically polarized views about contentious science. Consequently,
science curiosity is of great interest to scholars across the social sciences. However, mea-
suring science curiosity via the science curiosity scale (SCS) is time intensive; potentially
impeding its widespread usage. We present two new methods for reducing SCS adminis-
tration time. One method presents respondents with a randomly selected subset of items
(“Random Subset Method;” RS). The other asks all respondents a core set of just four
items (“Reduced-Form Method;” RF). In three nationally representative surveys, we as-
sess the construct, convergent, and predictive validity of these alternatives. We find both
versions to be well validated.
Word Count: 6,401
Keywords: science curiosity, science opinion, motivated reasoning, item measurement,
questionnaire design
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 2
Science curiosity has been heralded as a promising, if not elusive, predictor of public
comprehension, interest, and engagement with science (Kahan et al., 2017). Conceived of
as a tendency to consume scientific information for the sake of personal interest and plea-
sure, science curiosity may indeed be central to the investigation of public understanding
of science across a wide range of topics. Furthermore, science curiosity may help amelio-
rate the impact of partisan motivated reasoning on attitudes toward polarized scientific
issues such as climate change and childhood vaccinations (Kahan et al. 2017). In spite of
this, concentrated research on science curiosity and its implications for public engagement
is surprisingly limited. Therefore, increased investigation of science curiosity may yield
valuable insights for public science literacy while also identifying means of harnessing such
curiosity to improve scientifically-literate decision making.
One of the perennial barriers in the study of science curiosity is a lack of reliable and
valid approaches to quantification; measures must be capable of quantifying individuals
science curiosity while avoiding conflation with other forms of curiosity or inviting so-
cial desirability biases among respondents (Kahan et al., 2017). To complicate matters
more, the extent to which science curiosity should be conceived of as a temporary psy-
chological state versus a stable trait which varies across individuals has been in dispute
(e.g., Loewenstein, 1994). Kahan and colleagues (2017) developed a survey instrument
to examine the extent to which science curiosity can be reliably measured as a trait and
whether such a measure validly predicts attitudes and behaviors one might expect a sci-
ence curious individual to exhibit (Kahan et al., 2017). Masked as part of a broader
consumer marketing survey, this measure combines both self-reported scientific interests
as well as measures of science curious behavior (e.g., reading science-themed books in ones
free time) to create an index of curiosity. While a promising measurement strategy, the
instrument takes considerable time to administer, placing a burden both on respondents
and on scholars seeking to examine science curiosity in connection to other constructs.
In the research presented here, we aim to address the aforementioned dilemmas in
the measurement of science curiosity. We report on a psychometric evaluation of three
different versions of the science curiosity measure developed by Kahan et al. (2017). This
evaluation includes the original curiosity scale as well as two new ‘short-form‘ measures.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 3
One short-form contains a subset of items selected for administration by the authors
(i.e., the ‘Reduced-Form Method; RF‘) while the other randomly presented respondents
with a unique subset of the curiosity items (i.e., the ‘Random Subset Method; RS‘). We
compare the content, convergent, and criterion validity of the science curiosity measures
in two nationally representative samples (N‘s = 2,500 and 3,000), followed by another
representative study (N = 2,500) in which respondents were randomly assigned to receive
the original scale or one of the two aforementioned short-form versions.
Ultimately, our hope is to ‘democratize‘ the study of science curiosity by crafting
several valid instruments with different administration lengths but equitable psychometric
properties. Such an approach should enable scholars to expand their study of science
curiosity without placing undue burden on survey administration. Further, we hope this
demonstration of systematic scale evaluation can serve as an example to motivate more
rigorous scale development in scholarship on the science of science communication moving
forward (Flake, Pek, and Hehman, 2017; Kahan, 2015).
Of course, we recognize that building SCS measures with fewer items necessarily poses
an important tradeoff; lower levels of measurement precision (see: Ansolabhere, Rodden,
& Snyder 2008). The introduction of random measurement error could weaken the rela-
tionship between science curiosity and science-related attitudes or behaviors of interest,
especially at extreme values of SCS (e.g., the relationship between SCS and pursuing
graduate education in STEM; a behavior we might expect only the most curious to do).
Consequently, though these three scales may have similar psychometric properties with
respect to validation, we urge researchers to choose an administration method that fits
not only their budget, but the nature of the relationships they hope to study.
Defining Science Curiosity
The study of curiosity, and science curiosity more specifically, has been a feature of psy-
chological and educational research for a century. In Talks to Teachers on Psychology:
and to Students on Some of Lifes Ideals (1899), William James discusses the important
role of curiosity in psychological functioning, defining curiosity as “the impulse towards
better cognition”, reflecting the desire to better understand what one does not already
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 4
know about the world. Similarly, Berlyne (1954) frames the investigation of curiosity and
knowledge acquisition as intending to address the question of “why, out of the infinite
range of knowable items in the universe, certain pieces of knowledge are more ardently
sought and more readily retained than others” (p. 180). From this perspective, curios-
ity is conceived of as an means of motivating information seeking behaviors which are
intended to bolster knowledge acquisition.
Loewenstein (1994) and some others (e.g., Kidd and Hayden, 2015) operationalize
curiosity primarily as a motivated state of information seeking induced by situational
context, rather than a dispositional trait of individuals (Loewenstein, 1994). That is, cer-
tain topics, issues, or experiences may induce a state of curiosity in individuals, but there
may not be a general underlying psychological trait of curiosity. There is not unanimity
on this position, however. Berlyne (1954), for instance, views curiosity as potentially both
a trait of individuals and a state of arousal. And, Kahan et al. (2017) argue that, at
least in the more specific case of curiosity about science, there is a dispositional trait of
curiosity that is variable across individuals and can be reliably observed and measured.
As such, Kahan and colleagues define science curiosity as “a general disposition, variable
in intensity across persons, that reflects the motivation to seek out and consume scientific
information for personal pleasure.” (Kahan et al., 2017, p. 180). For our purposes, we
retain this definition as our working framework.
Science Curiosity & Public Opinion
Disputes of definition aside, the potential utility of science curiosity for promoting posi-
tive public engagement with scientific material seems self-evident. One might reasonably
expect science curiosity to engender a variety of desirable attitudes and behaviors, such as
open-mindedness and information seeking about scientific topics, more careful consider-
ation of scientific evidence, and better evidence-informed decision making. For example,
Kahan et al. (2017) found that those scoring higher on their measure of science curiosity
spent more time on average watching a documentary specifically about science. Such
conclusions are not foregone, however, and more research is needed to firmly establish
the nature of science curiosity as well as its potential benefits (or drawbacks) for public
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 5
engagement with science.
Another interesting finding observed by Kahan et al. (2017) is that science curiosity
may not be amenable to the same type of motivated reasoning biases as other intellectual
and psychological characteristics, such as science intelligence (e.g., Kahan et al., 2012). A
common finding in the study of public opinion on contentious social risks such as climate
change and childhood vaccination is that such opinions are heavily polarized (Egan &
Mullin, 2017; Kahan, 2013). Rather than buffering the negative impacts of partisan
ideology on such risks, certain cognitive reasoning capacities, such as science intelligence,
are at times associated with more, rather than less, polarized attitudes (Kahan et al., 2012;
Flynn, Nyhan, & Reifler, 2017). That is, highly informed partisans tend to be the most
capable of arguing against counter-attitudinal information, exhibiting greater polarized
attitudes than less-informed individuals. However, as science curiosity is defined in part
as a desire to seek out scientific content for the sake of general interest or pleasure rather
than to achieve a particular goal, we might reasonably anticipate that science curiosity
may be immune to the polarization observed when studying science intelligence.
Importantly, our claim is not that science curiosity will itself lead to a particular
conclusion on a scientific subject (e.g., greater belief in anthropogenic climate change).
Rather, curiosity may promote a more even-handed evaluation of evidence, such that
those higher in curiosity should not be considerably more polarized than those low in
curiosity. This premise comes from recent findings from Kahan et al. (2017), in which
polarization as a product of science curiosity was considerably weaker in comparison to
that observed for science intelligence.
Measuring Science Curiosity
Given the promise of science curiosity for improving decision making, it is vital to have
rigorously validated measures of science curiosity for scholars to use. Arguably no extant
measures have achieved this standard. Chief among the problems with curiosity measures
are their self-report question style and the potential for social desirability biases. As
Kahan et al. (2017) point out, asking respondents to answer questions such as “I am
curious about the world in which we live” (Fraser, 1978) not only gives away the purpose
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 6
of the measure to respondents, but also invites them to respond with more affirmative
“pro-curiosity” answers. To address this issue with current measures, Kahan et al. (2017)
crafted a 12-item measure of science curiosity.
The 12-item science curiosity measure is embedded within a larger battery of mea-
sures masked as a social marketing survey. Participants are asked a series of self-report
attitudinal measures, past behavior questions, and a behavioral measure. The attitudinal
measures, for example, ask respondents to indicate the things such as their degree of gen-
eral interest in science, politics, history, and other topics. The measures of past behavior
ask respondents to report whether they have engaged in a variety of behaviors in the past
year, such as reading a science (or politics, history, etc.) book, attending public lectures,
etc. Finally, the behavioral measure asks respondents to select to read a news article from
one of four categories, including science, entertainment, politics, and sports.
This scale was initially validated in Kahan et al. (2017) and we refer the reader to this
paper for more details on the validation process and findings. Using item response theory,
factor analysis, and correlation analysis for validation, the science curiosity scale was found
to be psychometrically reliable and valid. The unidimensional scale is roughly normally
distributed in the broader U.S. population, and is moderately positively correlated with a
measure of science intelligence (ordinary science intelligence; Kahan, 2017). Furthermore,
Kahan et al. (2017) found that those higher on the science curiosity scale spent more time
viewing a science documentary, and exhibited other characteristics of content engagement.
In the research described here, we employ the same version of the science curiosity measure
as in Kahan et al. (2017), although we modified the behavioral measure to include more
up-to-date stories on each topic.
The Present Research
While the science curiosity scale created by Kahan et al. (2017) is promising, it is quite
long to administer due to the number of items involved and the masking as a social
marketing survey. The goal of this research was replicate and extend the scale validation
work described in Kahan et al. (2017), particularly with the goal of creating several
short-form measures. We used two techniques to craft short form measures, which are
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 7
described in more detail in the results that follow. We report first on analyses of two
large existing datasets containing the science curiosity measure as well as the measure of
science intelligence. We then report on the findings of a new survey in which respondents
were randomly assigned to answer one of three versions of the science curiosity scale. In
this third set of analyses, we also explore the association between the curiosity measure,
science intelligence, and self-reported consumption of science-related media from a variety
of media platforms. Across all analyses, we find highly similar psychometric properties
for each science curiosity measure.
Study 1. SCS Administration Reduction with Existing
Data
Purpose
The purpose of Study 1 is to assess the content, convergent, and criterion (“predictive”)
validity of the Random Subset (RS) and Reduced-Form (RF) versions of SCS, using
existing survey data. The major benefit of this approach is that respondents’ actual SCS
scores are already known. This enables us to directly estimate convergent validity, by
correlating the original SCS with the results produced from the RS and RF methods.
Data
Data for this study come from two large, nationally representative surveys of U.S. adults.
The first is a survey of 2,500 adults, recruited by YouGov in September 2015, and the
second is a survey of 3,000 adults recruited in January 2016. The primary purpose of both
studies was to assess how Americans’ levels of science curiosity and ordinary science intel-
ligence (OSI, see Kahan et al. 2017) influence the types of scientific media they are willing
to consume, and the way in which they engage with that media. Detailed information
items used to produce these scales can be found in the Supplementary Materials.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 8
Procedure
To simulate exposing respondents to a random subset of science curiosity items (i.e., the
RS method), we first set a random subset (33%) of each respondents’ answers to the
original SCS to be missing data. Then, relying on insights from Item Response Theory
(IRT) we construct a measure of science curiosity based on the available, non-missing
data. The general idea here is that respondents’ placement on an unobserved science
curiosity continuum is determined only by their responses items not set to missing.
We determined respondents’ placement on a latent science curiosity continuum using
hybrid item response theory (IRT) modeling. Scores on dichotomous choice questions
(i.e., the previously mentioned behavioral items) were assessed using a two-parameter
logistic function (2PL), and scores on ordinal choice questions (i.e., the self-reported
items) were measured using a graded response model (GRM). Although this procedure
may seem somewhat abstract, we will walk through how to interpret results from these
models shortly, when discussing content validity.
To simulate the RF method, we selected just four items (the same for all respondents)
to serve as candidates for a reduced SCS. We selected the following items based on the
results of IRT models available in the Supplemental Materials. In keeping with the scale’s
original developmental goals (Kahan et al., 2016, 2017), we included both self-reported
interests and retrospective reports of past behavior in this reduced collection.
First, we selected respondents’ self-reported interest in science, because those who
exhibit low interest in science are highly likely to earn a low score on the science curiosity
scale. In IRT parlance, we would say that this item has a comparatively high discrimi-
nation parameter – i.e., it is good at discerning who falls where on the scale – and a low
difficulty parameter – i.e., many people are express high levels of interest.
Next, we selected a measure of whether or not respondents reported attending a science
lecture in the past year. We chose this item because people who have taken this action are
highly likely to earn a top score on the science curiosity scale. This item has a moderately
high discrimination parameter and a high difficulty parameter; meaning that most people
have not attended a science lecture, but that those who do are quite likely to score near
the top of the science curiosity distribution.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 9
Finally, we selected two “middle of the road” measures that were more or less reflective
of the typical items included in the original SCS. We included a measure of whether or
not respondents reported reading a book about science in the previous year as well as
their interest in news about technology. These items were generally good at discerning
who would fall where on the scale (discrimination), and extreme scores on each one were
not especially common (difficulty).
Results
Content Validity
We begin this analysis by assessing the content validity of the RS and RF versions of
SCS. Content validity refers to the extent to which a measure appears to reflect a specific
substantive domain, as called for by prior theorizing (Carmines & Zeller 1979). For
example, for the RS and RF scales to be considered “content valid,” we should expect
that people who score highly on each measure are more likely to consume – and enjoy
consuming – information related to science than those scoring lowly on the measure.
To assess whether or not each SCS measure is content valid, the top row of Figures
1 and 2 present item and category characteristic curves (known as “ICCs” and “CCCs,”
respectively) resulting from the previously mentioned hybrid IRT models used to produce
the RS, RF, and Original SCS. These curves tell us the probability that a person scoring at
the maximum value of each individual item used to build SCS (y-axis) earns a particular
score on a latent science curiosity continuum (x-axis). We would consider each SCS
measure to be content valid if, for example, people who report that they strongly enjoy
following news about scientific issues have a high probability of scoring highly on the
latent SCS, and a low probability of earning a low score. Figure 1 presents results from
the 2015 survey, and Figure 2 presents results from the 2016 survey.
Figures 1 and 2 demonstrate that both the short-form and original versions of SCS
have strong content validity. For example, focusing first on the leftmost pane in Figure 1,
it appears to be the case that high scores on each item tend to be associated with a low
probability of earning a low score on the latent science curiosity continuum, and a high
probability of scoring highly on it. Of course, there is some heterogeneity in this pattern.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 10
Some items – like respondents’ self-reported interest in scientific issues – produce very
“steep” s-shaped curves – indicating that the item is highly discriminating.
Others, however, others have curves that more closely resemble the letter J – such as
whether or not respondents read a science book in the past year. These items may be too
“difficult” (i.e., too few people actually do them) to discriminate much among those low
in science curiosity, but are comparatively better at differentiating between those scoring
highly on the latent continuum. We observe an identical pattern in Figure 2. Overall,
this provides strong evidence in favor the idea that the original SCS is content valid.
Turning now to the second and third panels from left in each figure, the results suggest
that the I/CCCs associated with the two modified versions of SCS correspond highly with
those presented in the first panel. Even when setting one third of items to missing (RS,
panel two), or reducing the scale to just four items (RF, panel three) low scores on each
item tend to be associated with a low probability of scoring highly on the modified SCS,
which high scores tend to be associated with a high probability of doing so.
Of course, we note that this result is unsurprising in the case of the RF measure, as
the RF items were selected in part based on their ability to distinguish people high and
low in science curiosity from one another. Nevertheless, it is encouraging that I/CCC
analysis presented for this “ideal case” strongly resemble the original measure, and the
RS alternative.
The second row of Figures 1 and 2 further probes the content validity of each SCS
administration method. Here, we plot “test information curves” for each scale, which
we re-expressed as scale reliabilities (Demars 2010; Kahan et al., 2017). The height
(i.e., peakedness) of each curve provides a sense of measurement precision (i.e., internal
consistency amongst the items used to create the scale) at different values of science
curiosity (on the x-axis).If each version of SCS is content valid, we should expect to see
high levels of reliability (i.e., in excess of 0.70) across the bulk of the science curiosity
distribution; i.e., all those respondents falling within one (theta equal to roughly -1 or
+1) or two standard deviations (theta equal to roughly -2 or +2) of the mean.
Figures 1 and 2 again provide strong evidence in favor of content validation. Across the
majority of the science curiosity distribution (which can be found just below the second
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 11
row; more on this shortly) reliabilities are consistently above 0.70 for the Original and
RS approaches. Reliability tends to drop off in at extremely low or high values of science
curiosity, where we have few observations.
We also note a decline in reliability using the RF method, which tends to produce
reliability scores in the 0.60 - 0.80 range, across the bulk of the science curiosity distribu-
tion. On some level, this is unsurprising. Since the scale features just a small fraction of
the items used to build the other two scales, and therefore prone to random measurement
error and a loss in precision (Ansolabehere, Rodden, & Snyder 2008). However, as we
note later on, this drop-off in measurement precision is a legitimate trade off to consider
when using the RF version of SCS. Consequently, we caution that the RF method may be
less optimal for making inferences about the relationship between science curiosity and
science-related attitudes and behaviors, particularly for those who score very high (or
very low) on SCS.
In addition to this analysis, prior theorizing posits that science curiosity should be
fairly normally distributed throughout the population (e.g., Kahan et al., 2017). If the
original and alternative SCS are content valid, we should expect the measures resulting
from the hybrid IRT procedures described earlier to be distributed roughly normally
throughout the population. The bottom rows of Figures 1 and 2 test this possibility by
presenting histograms of the original and two modified versions of SCS.
The results show that the average (i.e., modal) respondent on each distribution tends
to receive a score of 0 on the scale. Follow up analyses in the 2015 survey suggest that all
three measures are similarly distributed about a mean of just about zero (MOrig. = 0.00
[SD = 0.99], M2RS = 0.00 [SD = 0.92], M3RF = 0.00 [SD = 0.85]), with high degrees of
normality (skewOrig. = , skewRS = -0.09, skewRF = -0.02; noting that a normal distribution
has a skew of 0). We again find mean centering at zero (MOrig. = , M2RS = 0.00 [SD =
0.84], M3RF = 0.00 [SD = 0.85]) and high degrees of normality (skewOrig. = 0.00 [SD =
0.99], skewRS = -0.08, skewRF = -0.04), in the 2016 study.
Figure 1; Figure 2
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 12
Convergent Validity Assessment
Next, we assess the convergent validity of each alternative measure of science curiosity.
Convergent validity refers to the extent to which a measure is associated with other
measures of the same, or closely related, constructs (Carmines & Zeller 1979). Recall
that a key advantage of Study 1 is that we have the ability to estimate covergent validity
directly. Because respondents’ scores on the original science curiosity are known a priori,
the correlation between each modified measure and the original scale provides a clear
indication of convergent validity.
Figures 3 and 4 plot each respondents’ score on the original SCS (y-axis) against their
scores on the RS (x-axis in the left panel) and RF (x-axis on the right panel) versions.
In both Figures 3 and 4, we find that – in both cases – the results point to a high degree
of correspondence between the modified versions of SCS and the original scale. Even
when responses to one third of science curiosity items (left pane) are set to missing, the
correlation between the two scales is very high (r2015 = 0.96, r2016 = 0.96). The correlation
drops somewhat when evaluation the four item version of SCS, and the scatterplot is
somewhat “noisier.” Still, the correlation between the two is robust; in excess of 0.90 (2015
= 0.92, r2016 = 0.92).
These results suggest that respondents’ placement on each alternative measure of
science curiosity is highly associated with their placement on the original scale. Conse-
quently, we think that we have strong claims to convergent validity in our assessment of
these administration burden reduction techniques.
Criterion (“Predictive”) Validity Assessment
Finally, we test whether or not each alternative measure of SCS has high levels of criterion
or predictive validity. A measure with strong criterion validity should be highly correlated
with attitudes and/or behaviors with which we would expect scores on that measure to
be associated. For example, a measure of respondents interest in baseball should be
correlated with the likelihood that they attend a baseball game, or watch the World
Series on television.
We assess criterion validity by correlating each version of the SCS with the following
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 13
two factors. First, we probe the correlation between SCS and a a dichotomous measure
of respondents’ science information seeking behavior – i.e., whether or not they chose to
read about science-related news over sports, entertainment, and finance; see Kahan et al.,
2017). If the SCS is indeed measuring people’s enjoyment of consuming science-related
information, then we should expect that more science curious people are more likely to
choose to read the story about science.
Second, we correlate SCS with respondents’ levels of ordinary science intelligence: a
measure of respondents’ knowledge of basic scientific facts and more-abstract scientific
reasoning skills (OSI; Kahan 2016). Although OSI and SCS are conceptually distinct
– e.g., one can be knowledgeable about basic scientific facts without being interested
in science (and vice versa) – we might nevertheless expect that curiosity increases the
likelihood that individuals will be exposed to basic facts about science and the workings
of the scientific method.
It is important to note that we are interested not only in the correlation between the
original SCS scale and each of these factors, but the extent to which these correlations
hold across alternate measurement strategies. Similar correlations across measures would
suggest that the predictive power of the SCS scale is not diluted by administering fewer
questions.
The original scale is moderately and positively correlated with OSI in both studies
(2015 r= 0.29; 2016 r= 0.26). These correlations are virtually identical across measure-
ment strategies, varying by no more than r = 0.04 from the original estimates (for RS,
r2015 = 0.27, r2016 = 0.25; for RF, r2015 = 0.31, r2016 = 0.30). In fact, the RF method
tends to produce slightly stronger correlations with each of these factors. Although we
want to avoid reading too much into small changes across measurement strategies, we
think this could be due to the fact that the RF method purposely selected candidate
items based on their strong ability to reflect an underlying science curiosity construct in
the item response theory analyses.
Figure 3
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 14
Discussion
The results presented here suggest that both the RS and RF for administering the science
curiosity scale are well validated. Both alternative scales appear to be measuring the
same content as the original scale (content validity), are highly correlated with the original
scale (convergent validity), and are associated with engagement with scientific information
(criterion validity). Because the RF method is both well-validated and highly economical
(i.e., it involves the fewest questions and administration time), we think that this may be
a particularly attractive option for survey researchers hoping to study science curiosity
in the future. However, we note that the introduction of random measurement error may
make RF less well-suited for drawing conclusions about the relationship between science
curiosity and science-related attitudes and behaviors for those who earn extreme scores
on SCS.
However, we caution that Study 1 cannot overcome issues related to observational
equivalence. While we can simulate reducing the number of questions administered to
respondents (i.e., by setting them to “missing” in our data), we can never go back in time
and not administer those items. This means that if the presence of items set to missing –
but, in reality, were actually asked to respondents – influences their scores on those items
not set to missing, we do not have the ability to observe it. Consequently, in Study 2, we
aim to overcome this inferential hurdle by randomly assigning respondents to take “from
scratch” either the RS, RF, or Original SCS scale.
Study 2. Split Ballot Study: YouGov 2019 (N = 2,500)
Purpose
The purpose of Study 2 is to determine whether or not the pattern of results observed
in Study 1 replicates when respondents are randomly assigned to complete one of three
versions of SCS (RS, RF, and Original). Recall that an important limitation of Study 1
was that all SCS questions were asked to all respondents. While we were able to simulate
what responses might look like if certain questions were not administered, Study 1 could
not account for the possibility that responses on the items used to create each scale were
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 15
shaped by items we treated as not-administered (but that were, in reality, administered).
Study 2 overcomes these obstacles via random assignment to administration modes
that vary the number of questions to which respondents are exposed. Consequently, this
study offers an even more rigorous assessment of content and predictive validity than
Study 1. However, one important limitation of Study 2 is that we cannot leverage the
data to get a sense of convergent validation, as we do not know what respondents scores
on the original scale would be on the original scale if they were assigned to the RS or RF
condition.
Data
Data for Study 2 come from a survey of 2,500 American adults administered from January
19 - February 7, 2019, via YouGov. YouGov originally interviewed 2,701 individuals to
take the study from their large, online, opt-in panel, and “matched down” those responses
on the basis of known national benchmarks on gender, age, race, and education. To ac-
count for potential remaining deviations between the sample and the population, YouGov
then weighted these responses on the basis of age, gender, race, educational attainment,
and region, and 2016 presidential vote choice.
Procedure
Study 2 overcomes the inferential obstacles posed in Study 1 by randomly assigning re-
spondents to take one of three versions of SCS. We again determined respondents’ place-
ments on latent science curiosity continnua using hybrid item response theory modeling
(refer to Study 1 for more information). The original version of SCS was administered
identically to that in Study 1 (see also: Kahan et al., 2017), and was shown to exactly
33% of respondents.
Next, to administer the RS version of the scale, YouGov programmed the survey such
that one third of the original SCS items were not shown to each respondent. As was the
case in Study 1, the items not administered varied by respondent; meaning that, in all
likelihood, no two respondents assigned to the RS condition saw the exact same subset of
SCS items. Again, exactly 33% of our sample was assigned to complete the RS version of
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 16
SCS.
Finally, we administered the RF version of the scale by presenting respondents with
the same subset of self-reported items used in Study 1 (see our discussion above for a
rationale for why we picked each one). As a reminder, we selected items pertaining to
respondents’ interest in science, whether or not they attended a science lecture in the
past year, interest in news about technology, and whether or not they read a book about
science in the past year.
Results
Content Validation
We begin our analysis by assessing the content validity of each SCS administration
method. Figure 4 is analogous to Figures 1 and 2 in Study 1. The top three panels
plot item/category characteristic curves from each hybrid IRT model used to determine
respondents’ science curiosity scores. The bottom three panels are histograms that display
the distribution of each variable (with weighted means and standard deviations printed
above each one).
As was the case in Study 1, Figure 4 suggests that both the original and two modified
versions of SCS have strong content validity. Across administration modes, and for every
item used to build SCS, people who indicate higher levels of interest in (and enjoyment of)
consuming scientific information have high probability of earning a high science curiosity
score, and a low probability of earning a low score.
Of course, as was also the case in Study 1, these items vary in their difficulty, and
discriminatory power. For example, across all three modes, self-reported interest in science
produces sharp S-shaped curves. This indicates that people who indicate high levels of
interest in science are very likely to earn high scores on SCS, and very unlikely to earn
low scores.
Figure 4
However, attending a science lecture – a more “difficult” action that few Americans do
in a typical year – produces more of a J-shaped curve. Across all three models, attending
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 17
a lecture virtually guarantees a high science curiosity score. However, not attending a
lecture does not necessarily suggest that a person is likely to earn a low score on SCS.
The second row of Figure 4 again plots the test information curves – again, expressed
as reliabilities – for each version of SCS. Again, indicative that each measure is content
valid, we note high levels of reliability across the bulk of the SCS distribution for the
Original and RS methods, with drop offs in precision at extreme values on each end of
the scale. And, again, we note a drop off in test reliability – hovering between the 0.60 -
0.80 range across the bulk of the distribution – when using the RF method. While the RF
method is economical and otherwise well-validated, the possibility of increased random
measurement error is an trade off to consider when using this method; especially if one
hopes to study attitudes or behaviors at very high and/or very low scores on SCS.
Finally, indicative of strong content validity, the bottom three panels of Figure 4
suggest that all three measures of science curiosity are distributed similarly, and roughly
normally, throughout the mass public. All three distributions produce a mean of about
0, and standard deviations of about 1. Skewness levels were smaller than +/-0.10 in all
three cases – again indicative of high degrees of normality.
Predictive Validation
Finally, to assess the predictive validity of each measure, we correlated each version of
SCS with respondents’ levels of ordinary science intelligence (measured and administered
identically to how we did so in Study 1). The first row of Figure 5 presents scatterplots
of the relationship between each measure of SCS and OSI, with fitted regression lines
(dashed red lines) and locally weighted regression lines (solid black lines).
As we observed in Study 1, all three SCS measures are positively correlated with
OSI. The original scale produced the highest correlation (r= 0.28), followed by the RF
(r= 0.24) and RS (r= 0.20) measures. The size of these correlations are substantively
similar to those observed in both samples featured in Study 1 (which ranged from r= 0.25
to r= 0.31). This provides solid evidence in favor of predictive validation.
Interestingly, and perhaps surprisingly given the distributional similarities of the RS
and original scales, it was the RF method that most closely recovered the relationship
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 18
between the original SCS scale and OSI. The locally weighted regression line in the middle
panel offers some potential insights as to why. Although the locally weighted and line of
best fit track together closely in the first and third panel (although the third is, as we might
expect, a bit “noisier”), the lowess line deviates from monotonicity (and, correspondingly,
the line of best fit) at high levels of science curiosity in the middle panel. This attenuates
the slope of the line of best fit. Because few earn the highest possible scores on SCS,
and only one third of our sample was administered the RS version of SCS, we think it is
most likely that this result is simply due to random chance. Still, we reiterate that the
differences between measures are fairly minor, and nevertheless provide strong evidence
in favor of predictive validity.
Additionally, Study 2 contained a ten item measure of respondents’ openness to new
experiences from the Big Five Inventory – 2 (Soto & John 2017; see the Supplemental
Materials for full question wording information). Recall that, as theorized earlier, highly
curious individuals should be more likely to entertain novel ideas, as well as information
which runs contrary to their previous beliefs. Consequently, we might expect SCS to be
correlated with this personality trait.
As anticipated, we find that SCS is moderately and positively correlated with openness
across the original (r = 0.46), RS (r = 0.47), and RF (r = 0.44) measurement strategies. In
all three cases, the line of best fit closely matches the locally weighted regression line. This
provides additional evidence in favor of predictive validity across measurement strategies.
To further highlight the predictive validity of each version of the SCS, we also admin-
istered a battery of items pertaining to respondents’ use of various media platforms (e.g.,
public television, Netflix/Hulu, podcasts), and the extent to which they watch science-
related content (versus other types of content) on each one. Respondents were first asked
whether or not they consume any information on each platform, and then asked about
how regularly they consume different type of programming on each one (Never, Less than
monthly, Monthly, Weekly, or Daily). A full list of items can be found in the Supplemental
Materials.
Because the SCS is designed to tap respondents’ enjoyment of consuming scientific
information, we should expect increased levels of science curiosity are associated with
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 19
a considerable increase in the likelihood of consuming science-related programming on
each platform. Moreover, if all three versions of SCS are equally well-validated, we would
expect that the strength of this relationship is similar across all three SCS administration
approaches.
To assess the effect of science curiosity on science-related media consumption, we used
ordered logistic regression to model seven indicators of how frequently people consume
science related content. The models account for several other factors that could potentially
explain consumption habits, including ordinary science intelligence (see: Kahan 2015),
educational attainment, political ideology, and a host of demographic controls. Daily and
weekly viewers were collapsed into a single category, in order to account for the fact that
some platforms (e.g., science podcasts) produce science related content more frequently
than others (e.g., websites devoted to science). Figures and full model output can be
found in the Supplementary Materials.
Finally, we recognize that some readers may be interested in further investigating the
social, political, and demographic correlates of the SCS, and assessing how they com-
pare across administration modes. To take stock of this, we built three multivariate
regression models regressing SCS on respondents’ political ideology, religiosity, educa-
tional attainment, and a variety of other factors. The results can be found in Table S5 in
the Supplementary Materials. There we find that openness to new experiences is a con-
sistently strong predictor of increases SCS across models. We note only minor differences
attributable to social, political, and other demographic factors (consistent with previous
research: see Kahan et al., 2016). Perhaps unsurprisingly, we note a minor drop off in
variance explained (R2) for the RS and RF models; from about 31% using the original
scale to about 27% for each of the alternative versions.
Figure 5
We find that the predicted probability of being a daily or weekly viewer rises sharply
– roughly exponentially – as scores on the SCS move from low to high. Most importantly,
for our purposes in this paper, we find that the sharp s-shaped predicted probability
curves are highly similar in magnitude across administration modes. Although the RF
version of SCS produces error bands that are somewhat larger in size, and relationships
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 20
that appear more monotonic than they do exponential, it is nevertheless the case that SCS
is associated with sharp increases in science content consumption across administration
modes. This offers further validation that the science curiosity scale is, in fact, measuring
what it purports to measure, and that the predictive validity of the SCS is unencumbered
by even dramatically reducing the number of items used to construct the scale.
Discussion
Overall, the results from Study 2 strongly corroborate those from Study 1. We again
find that both the RS and RF administration procedures produce measures of science
curiosity that appear to be well validated. In terms of content validity, we again find
that (1) increases in each factor thought to make people more science curious do in fact
predict higher placement on the SCS, and (2) that the RS and RF administration modes
produce science curiosity distributions that mirror that of the original scale. In terms of
criterion (predictive) validity, we again find that all three versions of SCS are moderately
and positively correlated with OSI.
Although Study 2 is not well suited to make claims about convergent validity (i.e.,
because we cannot correlate scores on the original SCS with the alternate scales presented
here), it offers a critical advantage over Study 1. Unlike Study 1 – which simulated
assignment to either the RS or RF version of SCS – Study 2 actually administers each
one to a random subset of respondents. This means that Study 2 eliminates the possibility
that being exposed to, or responding to, questions treated as missing for the purpose of
simulation confounds our assessment of content and/or predictive validity.
Bearing the strengths and weaknesses of each design in mind, and noting that we
observe a similar pattern of results across each one, we are confident that both the RS
and RF administration modes of the SCS produce valid measures of science curiosity.
General Discussion
Science curiosity is a promising area of study for many academic fields, serving as a
predictor of science engagement and a mitigator of partisan motivated reasoning about
science. This paper seeks to ‘democratize’ the study of science curiosity by developing
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 21
and testing instruments of different lengths but equitable psychometric properties. This
would allow researchers to study science curiosity without the burden of including such a
long instrument on their already space-limited surveys.
In this collection of studies, we evaluated the properties of two short-form SCS mea-
sures: a subset of items selected for administration by the authors (i.e., the Reduced-Form
Method; RF) and randomly presenting respondents with a unique subset of the curios-
ity items (i.e., the Random Subset Method; RS). Across three nationally representative
studies, we find that both RF and RS are generally well validated. Both methods appear
to measure the same underlying concept and correlate highly with the original science
curiosity scale. Each of our studies has strengths and weaknesses. Study 1 was very
well suited for testing convergent validity; however, assessment of content and predictive
validity could be confounded in the simulation process. Study 2 removed those potential
confounds by administering either the RS or RF version of the measure to a random
subset of study respondents.
While both short-form measures are well validated, choosing which one to use in a
study, or if one should use the original scale instead, depends on the goals of and resources
available to the researcher. The RF approach certainly offers researchers an economical
option for measuring science curiosity, especially if survey space is limited. However, we
also know that RF comes with a trade off- a noisier measure of the SCS. The RS approach
reduces some of this noise, but is more complex to analyze and program, given the need
to randomize the items shown across respondents.
Both short-form versions of the SCS also pose potential inferential issues, depending
upon the interests of the researcher. Given the shortened nature of the scales, both RS
and especially RF offer a somewhat less precise sense of where individuals fall on the
science curiosity distribution, particularly at the highest or lowest values on the scale.
If one’s theory suggests that there may be some finely graded differences among those
with the lowest and highest levels of science curiosity, we cannot be confident that the
short-form versions are able to provide as complete a picture of those potential differences
as the original SCS.
For example, it could be the case that science curiosity “turns off” motivated reasoning
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 22
with respect to highly politicized aspects of scientific consensus (e.g., climate change) at
only very high values of science curiosity. If future research were to demonstrate that this
is true, we would then want to pursue a measurement strategy that provides high levels
of precision at the upper end of the SCS distribution. Future research should identify
whether or not thresholds like this exist, as this information could prove useful in deter-
mining which measurement strategies are appropriate, given ones’ research objectives.
And this leads us to our final, and most general point. Ultimately, which form of the
SCS a researcher should choose to use depends on what the goals of their research are and
the role science curiosity plays in their theoretical expectations. If the theory suggests
differences in the tails of the science curiosity distribution, using a longer version of the
SCS is likely preferable. If these potential variations are not central to the theory, or
if survey space is at a premium, then the short-form versions provide a viable alterna-
tive. While RS and RF provide less information and ability to discriminate across the
science curiosity spectrum, they nonetheless offer researchers the ability to measure this
disposition using a cost-effective method with limited sacrifices in validity.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 23
References
Ansolabehere, S., Rodden, J., & Snyder, J. M. (2008). The strength of issues: Using
multiple measures to gauge preference stability, ideological constraint, and issue voting.
American Political Science Review, 102(2), 215-232.
Berlyne, D. E. (1954). A theory of human curiosity. British Journal of Psychology.
General Section, 45(3), 180-191.
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment (Vol. 17).
Sage publications.
DeMars, C. (2010). Item response theory. Oxford University Press.
Flake, J. K., Pek, J., & Hehman, E. (2017). Construct validation in social and per-
sonality research: current practice and recommendations. Social Psychological and Per-
sonality, 8(4), 370-378.
Flynn, D. J., Nyhan, B., & Reifler, J. (2017). The nature and origins of misperceptions:
understanding false and unsupported beliefs about politics. Political Psychology, 38(51),
127-150.
Fraser, B. J. (1978). Development of a test of science-related attitudes. Science
Education, 62, 509-515.
James W. (1899). Talks to Teachers on Psychology: And to Students on Some of Life
Ideals. New York: Henry Holt & Company; 1899.
Kahan, D. M., Peters, E., Wittlin, M., Slovic, P., Ouellette, L. L., Braman, D., &
Mandel, G. (2012). The polarizing impact of science literacy and numeracy on perceived
climate change risks. Nature climate change, 2(10), 732.
Kahan, D. M. (2013). A risky science communication environment for vaccines. Sci-
ence, 342, 53-54.
Kahan, D. (2015). What is the science of science communication? Journal of Science
Communication, 14, 1-12.
Kahan, D. M. (2017). Ordinary science intelligence: A science-comprehension measure
for study of risk and science communication, with notes on evolution and climate change.
Journal of Risk Research, 20(8), 995-1016.
Kahan, D. M., Landrum, A., Carpenter, K., Helft, L., & Jamieson, K. H. (2017).
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 24
Science curiosity and political information processing. Advances in Political Psychology,
38(1), 179-199.
Kidd, C., & Hayden, B. Y. (2015). The psychology and neuroscience of curiosity.
Neuron, 88(3), 449-460.
Loewenstein, G. (1994). The psychology of curiosity: a review and reinterpretation.
Psychological Bulletin, 116, 75-98.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 25
0
.5
1
Probability
-4 -2 0 2 4
Theta
Original
0
.5
1
Probability
-4 -2 0 2 4
Theta
RS Version
0
.5
1
Probability
-4 -2 0 2 4
Theta
RF Version
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
Original
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RS
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RF
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.99
Original
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.91
RS
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.82
RF
Attended a Science Lecture This Year
Read a Science Book This Year
Interest in Science
Follow Technology News
Chose Science Article (Behavioral Task)
Visited Sci Museum This Year
Interest in Technology
Interest in Nature
Conversations about Science
Conversations about Technology
Follow Science News
Share Science Content on Social Media
Figure 1. Content Validity Assessment (YouGov 2015). Top three panels present item/category characteristic curves from the hybrid IRT models used to construct each measure of science curiosity.
The middle three panels present test information function curves across the majority of the distribution (+/- 2 standard deviations; see the bottom panel for additional information about the mean
and SD of latent science curiosity [“theta” in row one; “SCS” in row three]). Full IRT models are available in the supplemental materials. The bottom three panels are histograms of the science
curiosity scales resulting from each hybrid IRT model.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 26
0
.5
1
Probability
-4 -2 0 2 4
Theta
Original
0
.5
1
Probability
-4 -2 0 2 4
Theta
RS Version
0
.5
1
Probability
-4 -2 0 2 4
Theta
RF Version
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
Original
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RS
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RF
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.99
Original
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.92
RS
0 5 10 15 20
Percent
-4 -2 0 2 4
SCS
M = 0.00
SD = 0.91
RF
Attended a Science Lecture This Year
Read a Science Book This Year
Interest in Science
Follow Technology News
Chose Science Article (Behavioral Task)
Visited Sci Museum This Year
Interest in Technology
Interest in Nature
Conversations about Science
Conversations about Technology
Follow Science News
Share Science Content on Social Media
Figure 2. Content Validity Assessment (YouGov 2016). Top three panels present item/category characteristic curves from the hybrid IRT models used to construct each measure of science curiosity.
The middle three panels present test information function curves across the majority of the distribution (+/- 2 standard deviations; see the bottom panel for additional information about the mean
and SD of latent science curiosity [“theta” in row one; “SCS” in row three]). Full IRT models are available in the supplemental materials. The bottom three panels are histograms of the science
curiosity scales resulting from each hybrid IRT model.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 27
-4 -2 0 2 4
Original SCS Scale
-4 -2 0 2 4
RS Version
r = 0.96
-4 -2 0 2 4
Original SCS Scale
-4 -2 0 2 4
RF Version
r = 0.91
2015
-4 -2 0 2 4
Original SCS Scale
-4 -2 0 2 4
RS Version
r = 0.96
-4 -2 0 2 4
Original SCS Scale
-4 -2 0 2 4
RF Version
r = 0.92
2016
Figure 3. Convergent Validity Assessment (YouGov 2015). Scatter plots compare respondents’ scores on the original SCS to their scores on those resulting from each alternative measurement
strategy.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 28
0
.5
1
Probability
-4 -2 0 2 4
Theta
Original
0
.5
1
Probability
-4 -2 0 2 4
Theta
RS Version
0
.5
1
Probability
-4 -2 0 2 4
Theta
RF Version
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
Original
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RS
.4 .6 .8 1
Reliability
-2 SD -1SD Mean +1SD +2SD
Science Curiosity
RF
0 5 10 15 20 25
% of Ballot
-4 -2 0 2 4
Sci. Curiosity
M = 0.00, SD = 0.93
Original
0 5 10 15 20 25
% of Ballot
-4 -2 0 2 4
Sci. Curiosity
M = 0.00, SD = 0.91
RS
0 5 10 15 20 25
% of Ballot
-4 -2 0 2 4
Sci. Curiosity
M = 0.00, SD = 0.83
RF
Attended a Science Lecture This Year
Read a Science Book This Year
Interest in Science
Follow Technology News
Chose Science Article (Behavioral Task)
Visited Sci Museum This Year
Interest in Technology
Interest in Nature
Conversations about Science
Conversations about Technology
Follow Science News
Share Science Content on Social Media
Figure 4. Content Validity Assessment (YouGov 2019). Top three panels present item/category characteristic curves from the hybrid IRT models used to construct each measure of science curiosity.
The middle three panels present test information function curves across the majority of the distribution (+/- 2 standard deviations; see the bottom panel for additional information about the mean
and SD of latent science curiosity [“theta” in row one; “SCS” in row three]). Full IRT models are available in the supplemental materials. The bottom three panels are histograms of the science
curiosity scales resulting from each hybrid IRT model.
REDUCING THE ADMINISTRATIVE DEMANDS OF THE SCIENCE
CURIOSITY SCALE: A VALIDATION STUDY 29
-4 -2 0 2
Ordinary Sci. Intelligence
-4 -2 0 2
Science Curiosity
r = 0.28
Original
-4 -2 0 2
Ordinary Sci. Intelligence
-4 -2 0 2
Science Curiosity
r = 0.20
RS
-4 -2 0 2
Ordinary Sci. Intelligence
-4 -2 0 2
Science Curiosity
r = 0.24
RF
-4 -2 0 2
Openness
-4 -2 0 2
Science Curiosity
r = 0.46
Original
-4 -2 0 2
Openness
-4 -2 0 2
Science Curiosity
r = 0.47
RS
-4 -2 0 2
Openness
-4 -2 0 2
Science Curiosity
r = 0.44
RF
Figure 5. Predictive Validity Assessment (YouGov 2019).