MethodPDF Available

Abstract

We normed one hundred trivia claims, each with a true and false version, by proportion of people who judged them to be true. Twenty claims were chosen for each of the following five categories: Science, geography, sports, animals, and food. This paper outlines the method of claim creation and includes all claims and truth rating data. Data was collected in November, 2016. Citation: Jalbert, M., Newman, E., & Schwarz, N. (2019). Trivia claim norming: Methods report and data. ResearchGate. doi: 10.6084/m9.figshare.9975602
Running&head:&TRIVIA&CLAIM&NORMING&
1&
Trivia Claim Norming: Methods Report and Data
Madeline Jalbert, Eryn Newman, & Norbert Schwarz
University of Southern California
Citation:
Jalbert, M., Newman, E., & Schwarz, N. (2019). Trivia claim norming: Methods report and data.
ResearchGate. doi: 10.6084/m9.figshare.9975602
TRIVIA&CLAIM&NORMING&
2
Psychological investigations often use trivia claims or questions as stimuli. In perhaps the
most well-known set of claims, Nelson and Narens (1980) created a set of 300 general
knowledge questions which have been cited over 300 times on Google Scholar. These claims
were recently updated and expanded (Tauber, Dunlosky, Rawson, Rhodes, & Sitzman, 2013) and
a subset has been adapted to Spanish (Duñabeitia et al., 2016). These claims have been used as
stimuli in studies of memory and source monitoring (Shimamura & Squire, 1987), judgments of
truth (Fenn, Newman, Pezdek, & Garry, 2013; Newman et al., 2015), feelings of knowing (e.g.,
Metcalfe, 1986), tip of the tongue phenomena (Burke, MacKay, Worthley, & Wade, 1991), and
metamnemonic judgments (Benjamin, Bjork, & Schwartz, 1998).
More recently, these claims, or variations of these claims, have been used as stimuli for a
number of studies that recruit participants online from the platform Mechanical Turk (MTurk;
e.g., Birnbaum, Kornell, Bjork, & Bjork, 2013; Chua & Bliss-Moreau, 2016, Fenn et al., 2013;
Kornell, 2015, Newman et al., 2015), a website where users participate in surveys in exchange
for monetary compensation. However, the repeated use of these trivia claims on MTurk may
pose a problem because a small subset of participants complete a large percentage of the MTurk
studies. For example, Paolacci and Chandler (2014) found that 41% of their tasks were
completed by just 10% of their worker pool. Thus, participants may encounter the same stimuli
multiple times. Seeing the same stimuli repeatedly may have deleterious effects on research
outcomes, as participants may learn the answers through repeated exposure. Additionally, the
mere exposure to materials can increase their liking (e.g., Zajonc, 1968) and perceived truth
(e.g., Hasher, Goldstein, & Toppino, 1977).
Nelson and Narens (1980) originally asked participants to recall the answers to free-
response questions. However, some researchers who used Nelson and Narens’ (1980) materials
TRIVIA&CLAIM&NORMING&
3
ask participants for statement recognition or perceived truth, which benefits from the use of
normed true and false statements.
Hence, the present materials serve two purposes. First, they provide a new set of trivia
claims that has not yet been widely circulated on MTurk. Second, the materials include true and
false versions of the same claims to allow for their use in studies requiring statements of
differing veracity. They were originally created and used in a series of truth effect experiments
(Jalbert, Newman, & Schwarz, 2018), but will likely prove useful for other experimental
investigations.
Materials
One hundred trivia claims, each with a true and false version, were compiled. The true
version of each claim was taken from the online Encyclopedia Britannica Website
(https://www.britannica.com/). Twenty claims were chosen for each of the following five
categories: Science, geography, sports, animals, and food. Each claim was taken from a different
article.
Once the true claims were chosen, one word from the claim was altered in order to make
the claim false. Each false claim was checked to ensure they were not actually true. The false
claims were altered from the true claims using the following procedure.
(1) If there was a way to easily create an opposite statement by changing a single word,
this was done first. Some examples are presented below.
TRIVIA&CLAIM&NORMING&
4
(2) If this was not possible, one of the object words was changed to something in a
similar category that could be true.
Original True Statement
Altered False Statement
The banana plant is a large herb
The grape plant is a large herb
Pluto is part of the Kuiper belt
Neptune is part of the Kuiper belt
The sun constitutes more than 99 percent of the
entire mass of the solar system
The planets constitute more than 99 percent of
the entire mass of the solar system
Procedure
We recruited one hundred participants from MTurk to take an online survey,
administered through Qualtrics. Due to how participants are sent from Mechanical Turk to
Qualtrics, ninety-nine participants completed the survey. Participants were told the experiment
was expected to take 20 to 30 minutes and received a payment of $.80. Fifty-three (53.5%) were
male, and 46 (46.5%) were female. Only MTurk workers located within the United States and
with a HIT approval rate (%) greater than or equal to 95% were eligible to participate.
Participants were randomly assigned to one of two counterbalance conditions. In each
condition, participants were presented with 100 claims, half true and half false. In one condition,
participants saw the true version of the first 50 claims and the false version of the second 50
claims for each category. Participants in the other condition saw the false version of the first 50
claims and true version of the second 50 claims. All claims were presented in random order.
In the instructions, participants were told that they would view a series of 100 trivia
statements. They were also told that some of the statements would be true, and some of the
Original True Claim
Altered False Claim
In almost all human populations of newborns,
there is a slight excess of males
In almost all human populations of newborns,
there is a slight excess of females
Snakes lack movable eyelids
Snakes have movable eyelids
Ham is the rear leg of a hog
Ham is the front leg of a hog
TRIVIA&CLAIM&NORMING&
5
statements would be false. For each statement, they were asked to answer the question, “Is this
trivia statement true or false?”. Participants were instructed to make their best guess if they did
not know the answer to a question.
Results
A list of trivia claims with the mean proportion of participants rating each one true are
presented below. The proportion of participants rating a claim as true ranged from .120 to .922,
with a mean of .562 (SD = .164). The mean proportion rated true for true claims was .620 (SD =
.157), with a range of .250 to .923, while the mean proportion rated for false claims was .504 (SD
= .150) with a range of .118 to .843.
TRIVIA&CLAIM&NORMING&
6
Norming Results
True Version of Claims
Claim
Number
True Claims
N
Mean
proportion
rated true
Std.
Deviation
Lower
95%
CI
Upper
95%
CI
1
The electron is the
lightest charged particle
found in nature
51
0.7451
0.44014
0.6213
0.8689
2
In chemistry, a mass
spectrometer is used to
separate substances into
its constituent parts
according to mass
51
0.6863
0.46862
0.5545
0.8181
3
Steel is the most widely
used material for
building the worlds
infrastructure
51
0.7255
0.45071
0.5987
0.8523
4
Sublimation refers to a
substance changing
states from solid to
vapor
51
0.549
0.50254
0.4077
0.6904
5
Xylem is the water-
transporting tissue in
plants
51
0.5686
0.5002
0.4279
0.7093
6
Binary fission is a kind
of asexual reproduction
51
0.3333
0.4761
0.1994
0.4672
7
Many of the genes in
baker's yeast are also
present in humans
51
0.7059
0.46018
0.5765
0.8353
8
Glia cells function
primarily to support
neurons.
51
0.5882
0.49705
0.4484
0.728
9
Amino acids are the
essential building
blocks of all proteins
51
0.902
0.30033
0.8175
0.9864
10
Night blindness is a
symptom of vitamin A
deficiency
51
0.7059
0.46018
0.5765
0.8353
11
The Earth is composed
predominantly of
igneous rock
48
0.6667
0.47639
0.5283
0.805
12
Pluto is part of the
Kuiper belt
48
0.5208
0.50485
0.3742
0.6674
13
The sun constitutes
48
0.6042
0.4942
0.4607
0.7477
TRIVIA&CLAIM&NORMING&
7
more than 99 percent of
the entire mass of the
solar system
14
Normal color vision is
known as trichromacy
48
0.5625
0.50133
0.4169
0.7081
15
In almost all human
populations of
newborns, there is a
slight excess of males
48
0.4583
0.50353
0.3121
0.6045
16
Levels of the metal
iridium are higher in
meteorites than on
Earth
48
0.8125
0.39444
0.698
0.927
17
Terminal velocity is the
steady speed achieved
by an object freely
falling
48
0.875
0.33422
0.778
0.972
18
Exothermic reactions
release chemical energy
48
0.6667
0.47639
0.5283
0.805
19
Plasma is the
noncellular portion of
blood
48
0.8125
0.39444
0.698
0.927
20
Glucagon is a hormone
secreted by the pancreas
48
0.5833
0.49822
0.4387
0.728
21
The highest waterfall in
the world is in
Venezuela
51
0.6863
0.46862
0.5545
0.8181
22
Vesuvius is an active
volcano in Italy
51
0.5882
0.49705
0.4484
0.728
23
The Dead Sea is the
lowest body of water on
the surface of Earth
51
0.7451
0.44014
0.6213
0.8689
24
The monetary unit in
India is the rupee
51
0.9216
0.27152
0.8452
0.9979
25
Verkhoyansk is a town
in northeastern Russia
51
0.7843
0.41539
0.6675
0.9011
26
Canada is the second
largest country in the
world in area
51
0.6078
0.49309
0.4692
0.7465
27
Greenland is a part of
the Kingdom of
Denmark
51
0.4118
0.49705
0.272
0.5516
28
The Arctic Ocean is
centered approximately
on the North Pole
51
0.6471
0.48264
0.5113
0.7828
29
All of the mountain
51
0.2745
0.45071
0.1477
0.4013
TRIVIA&CLAIM&NORMING&
8
regions of Antarctica
have been visited
30
The Pyrenees
Mountains form a high
wall between France
and Spain
51
0.7255
0.45071
0.5987
0.8523
31
Taboga Island is in
Panama
48
0.4583
0.50353
0.3121
0.6045
32
The Zulu are the single
largest ethnic group in
South Africa
48
0.7083
0.45934
0.575
0.8417
33
Australias capital is
Canberra
48
0.25
0.43759
0.1229
0.3771
34
Alexander the Great
conquered the region of
Egypt
48
0.8333
0.37662
0.724
0.9427
35
The Nile river flows
northward
48
0.6875
0.46842
0.5515
0.8235
36
Lake Baikal is the
world's largest
freshwater lake by
volume
48
0.5208
0.50485
0.3742
0.6674
37
Asia has the highest
average elevation of the
continents
48
0.7708
0.42474
0.6475
0.8942
38
Algeria is a
predominantly Muslim
country
48
0.625
0.48925
0.4829
0.7671
39
Finland is the most
densely forested
country in Europe
48
0.7292
0.44909
0.5988
0.8596
40
French is an official
language of
Madagascar
48
0.5833
0.49822
0.4387
0.728
41
Mogul skiing is the
navigation of large
bumps on the ski slope
51
0.6863
0.46862
0.5545
0.8181
42
Birling is the sport of
logrolling
51
0.6078
0.49309
0.4692
0.7465
43
Dart boards are
commonly made of
sisal
51
0.5686
0.5002
0.4279
0.7093
44
Bandy is a game similar
to ice hockey
51
0.3725
0.48829
0.2352
0.5099
45
Handball has been
traced back to the baths
51
0.6667
0.4761
0.5328
0.8006
TRIVIA&CLAIM&NORMING&
9
of Rome
46
Competitive badminton
is usually played
indoors
51
0.6863
0.46862
0.5545
0.8181
47
Rugby is played with an
oval ball
51
0.6471
0.48264
0.5113
0.7828
48
The steeplechase, in
athletics, is a footrace
over an obstacle course
51
0.7059
0.46018
0.5765
0.8353
49
Swimming is the first
event in a triathlon
51
0.3333
0.4761
0.1994
0.4672
50
The Boston Marathon is
the worlds oldest
annual marathon
51
0.7059
0.46018
0.5765
0.8353
51
Tenpins is the most
widely played variation
of bowling
48
0.7708
0.42474
0.6475
0.8942
52
Volleyball was
originally called
mintonette
48
0.3542
0.48332
0.2138
0.4945
53
The stationary ball in
lawn bowls is called a
jack
48
0.4375
0.50133
0.2919
0.5831
54
Rhythmic gymnastics is
practiced only by
women
48
0.3333
0.47639
0.195
0.4717
55
The stones used in
curling are concave on
the bottom
48
0.6042
0.4942
0.4607
0.7477
56
The longbow was
invented before the
crossbow
48
0.8958
0.30871
0.8062
0.9855
57
Snowboarding is
believed to have
originated in America
48
0.6875
0.46842
0.5515
0.8235
58
Fly-fishing is the oldest
method of recreational
fishing
48
0.5833
0.49822
0.4387
0.728
59
The invention of roller
skates has been
traditionally credited to
a Belgian
48
0.6458
0.48332
0.5055
0.7862
60
In foxhunting, the
hunter usually wears a
red shirt
48
0.5208
0.50485
0.3742
0.6674
TRIVIA&CLAIM&NORMING&
10
61
Gnu is another name for
wildebeest
51
0.6275
0.48829
0.4901
0.7648
62
The whale shark
subsists on plankton
51
0.6667
0.4761
0.5328
0.8006
63
Most sea turtles are
carnivorous
51
0.4118
0.49705
0.272
0.5516
64
Deer do not have gall
bladders
51
0.3333
0.4761
0.1994
0.4672
65
Moose may dive
underwater while
feeding
51
0.3922
0.49309
0.2535
0.5308
66
Sheep are a type of
ruminant mammal
51
0.7647
0.4284
0.6442
0.8852
67
Domesticated goats are
descended from the
pasang
51
0.4902
0.50488
0.3482
0.6322
68
The mouth of a sea
urchin is on its
underside
51
0.8235
0.38501
0.7152
0.9318
69
Some salmon use the
Earths magnetic field
to navigate
51
0.8039
0.40098
0.6911
0.9167
70
Most eels are solitary
fish
51
0.6275
0.48829
0.4901
0.7648
71
Walruses use their tusks
primarily for mating
48
0.3958
0.4942
0.2523
0.5393
72
A koala's pouch opens
rearward
48
0.2917
0.45934
0.1583
0.425
73
Both sexes of lions are
polygamous
48
0.4583
0.50353
0.3121
0.6045
74
The flamingo's pink
color comes from
carotenoid pigments in
its food
48
0.5625
0.50133
0.4169
0.7081
75
Guinea pigs belong to
the cavy family
48
0.5417
0.50353
0.3955
0.6879
76
The otter belongs to the
weasel family
48
0.8333
0.37662
0.724
0.9427
77
A panda's rear paws
point inwards
48
0.6458
0.48332
0.5055
0.7862
78
Female turkeys
generally weigh half as
much as males
48
0.5208
0.50485
0.3742
0.6674
79
Snakes lack movable
eyelids
48
0.5833
0.49822
0.4387
0.728
TRIVIA&CLAIM&NORMING&
11
80
Giraffes have excellent
eyesight
48
0.6667
0.47639
0.5283
0.805
81
Dough is boiled in the
process of making
bagels
51
0.7059
0.46018
0.5765
0.8353
82
Most limes have more
acid than lemons
51
0.6275
0.48829
0.4901
0.7648
83
The banana plant is a
large herb
51
0.3333
0.4761
0.1994
0.4672
84
Corn was first
domesticated by native
peoples in Mexico
51
0.7843
0.41539
0.6675
0.9011
85
California produces
most of the world's
almonds
51
0.7647
0.4284
0.6442
0.8852
86
Beef is the most
popular meat in the
United States
51
0.7451
0.44014
0.6213
0.8689
87
Ham is the rear leg of a
hog
51
0.7059
0.46018
0.5765
0.8353
88
The Colchester is a
popular type of oyster
51
0.4118
0.49705
0.272
0.5516
89
Mead is an alcoholic
beverage fermented
from honey
51
0.7255
0.45071
0.5987
0.8523
90
Kava is a beverage
made from the root of
the pepper plant
51
0.4902
0.50488
0.3482
0.6322
91
Mayonnaise is usually
made with raw egg
yolks
48
0.8542
0.35667
0.7506
0.9577
92
Tomatoes are a good
source of the
phytochemical lycopene
48
0.8333
0.37662
0.724
0.9427
93
The lima bean is also
known as the sieva bean
48
0.5417
0.50353
0.3955
0.6879
94
Cabbages are in the
mustard family
48
0.5417
0.50353
0.3955
0.6879
95
Couscous is a dish from
Africa
48
0.5833
0.49822
0.4387
0.728
96
Sherbert has more sugar
than ice cream
48
0.6667
0.47639
0.5283
0.805
97
The final extraction of
molasses from sugar
crystals produces
blackstrap molasses
48
0.8125
0.39444
0.698
0.927
TRIVIA&CLAIM&NORMING&
12
98
Sticky toffee pudding is
a classic British dessert
48
0.8542
0.35667
0.7506
0.9577
99
Rhubarb leaves contain
toxins
48
0.625
0.48925
0.4829
0.7671
100
Halvah is a confection
made of sesame seeds
48
0.5417
0.50353
0.3955
0.6879
False version of claims
Claim
Number
False Claims
N
Mean
proportion
rated true
Std.
Deviation
Lower
95%
CI
Upper
95%
CI
1
The electron is the
heaviest charged particle
found in nature
48
0.5
0.50529
0.3533
0.6467
2
In chemistry, a mass
spectrometer is used to
separate substances into
its constituent parts
according to color
48
0.625
0.48925
0.4829
0.7671
3
Concrete is the most
widely used material for
building the world’s
infrastructure
48
0.8333
0.37662
0.724
0.9427
4
Deposition refers to a
substance changing
states from solid to
vapor
48
0.4792
0.50485
0.3326
0.6258
5
Phloem is the water-
transporting tissue in
plants
48
0.5625
0.50133
0.4169
0.7081
6
Binary fission is a kind
of sexual reproduction
48
0.2292
0.42474
0.1058
0.3525
7
None of the genes in
baker's yeast are also
present in humans
48
0.5
0.50529
0.3533
0.6467
8
Dendritic cells function
primarily to support
neurons.
48
0.5833
0.49822
0.4387
0.728
9
Amino acids are the
essential building blocks
of all carbohydrates
48
0.6667
0.47639
0.5283
0.805
10
Night blindness is a
symptom of vitamin D
deficiency
48
0.5417
0.50353
0.3955
0.6879
TRIVIA&CLAIM&NORMING&
13
11
The Earth is composed
predominantly of
metamorphic rock
51
0.8039
0.40098
0.6911
0.9167
12
Neptune is part of the
Kuiper belt
51
0.4706
0.5041
0.3288
0.6124
13
The planets constitute
more than 99 percent of
the entire mass of the
solar system
51
0.2745
0.45071
0.1477
0.4013
14
Normal color vision is
known as dichromacy
51
0.549
0.50254
0.4077
0.6904
15
In almost all human
populations of
newborns, there is a
slight excess of females
51
0.7059
0.46018
0.5765
0.8353
16
Levels of the metal
iridium are lower in
meteorites than on Earth
51
0.451
0.50254
0.3096
0.5923
17
Orbital velocity is the
steady speed achieved
by an object freely
falling
51
0.3922
0.49309
0.2535
0.5308
18
Endothermic reactions
release chemical energy
51
0.6471
0.48264
0.5113
0.7828
19
Plasma is the cellular
portion of blood
51
0.6863
0.46862
0.5545
0.8181
20
Glucagon is a hormone
secreted by the thyroid
51
0.5098
0.50488
0.3678
0.6518
21
The highest waterfall in
the world is in Argentina
48
0.5417
0.50353
0.3955
0.6879
22
Vesuvius is an active
volcano in France
48
0.3333
0.47639
0.195
0.4717
23
The Caspian Sea is the
lowest body of water on
the surface of Earth
48
0.625
0.48925
0.4829
0.7671
24
The monetary unit in
Afghanistan is the rupee
48
0.4792
0.50485
0.3326
0.6258
25
Verkhoyansk is a town
in northeastern Ukraine
48
0.7708
0.42474
0.6475
0.8942
26
China is the second
largest country in the
world in area
48
0.5417
0.50353
0.3955
0.6879
27
Greenland is a part of
the Kingdom of Sweden
48
0.3125
0.46842
0.1765
0.4485
28
The Arctic Ocean is
centered approximately
48
0.4583
0.50353
0.3121
0.6045
TRIVIA&CLAIM&NORMING&
14
on the South Pole
29
None of the mountain
regions of Antarctica
have been visited
48
0.3125
0.46842
0.1765
0.4485
30
The Carpathian
Mountains form a high
wall between France and
Spain
48
0.5208
0.50485
0.3742
0.6674
31
Taboga Island is in
Nicaragua
51
0.5098
0.50488
0.3678
0.6518
32
The Swazi are the single
largest ethnic group in
South Africa
51
0.5882
0.49705
0.4484
0.728
33
Australia’s capital is
Wellington
51
0.2745
0.45071
0.1477
0.4013
34
Alexander the Great
conquered the region of
Turkey
51
0.7451
0.44014
0.6213
0.8689
35
The Nile river flows
southward
51
0.5098
0.50488
0.3678
0.6518
36
Lake Malawi is the
world's largest
freshwater lake by
volume
51
0.6078
0.49309
0.4692
0.7465
37
Europe has the highest
average elevation of the
continents
51
0.4902
0.50488
0.3482
0.6322
38
Algeria is a
predominantly Christian
country
51
0.4902
0.50488
0.3482
0.6322
39
Finland is the least
densely forested country
in Europe
51
0.451
0.50254
0.3096
0.5923
40
Spanish is an official
language of Madagascar
51
0.3529
0.48264
0.2172
0.4887
41
Slalom skiing is the
navigation of large
bumps on the ski slope
48
0.5833
0.49822
0.4387
0.728
42
Birling is the sport of
woodchopping
48
0.5833
0.49822
0.4387
0.728
43
Chess boards are
commonly made of sisal
48
0.2292
0.42474
0.1058
0.3525
44
Kabaddi is a game
similar to ice hockey
48
0.3333
0.47639
0.195
0.4717
45
Tennis has been traced
back to the baths of
48
0.625
0.48925
0.4829
0.7671
TRIVIA&CLAIM&NORMING&
15
Rome
46
Competitive badminton
is usually played
outdoors
48
0.5208
0.50485
0.3742
0.6674
47
Cricket is played with an
oval ball
48
0.2708
0.44909
0.1404
0.4012
48
The heptathlon, in
athletics, is a footrace
over an obstacle course
48
0.5833
0.49822
0.4387
0.728
49
Biking is the first event
in a triathlon
48
0.5625
0.50133
0.4169
0.7081
50
The Chicago Marathon
is the world’s oldest
annual marathon
48
0.4167
0.49822
0.272
0.5613
51
Candlepins is the most
widely played variation
of bowling
51
0.6275
0.48829
0.4901
0.7648
52
Golf was originally
called mintonette
51
0.2353
0.4284
0.1148
0.3558
53
The stationary ball in
lawn bowls is called a
queen
51
0.4902
0.50488
0.3482
0.6322
54
Trampoline gymnastics
is practiced only by
women
51
0.1176
0.3254
0.0261
0.2092
55
The stones used in
curling are convex on
the bottom
51
0.6275
0.48829
0.4901
0.7648
56
The longbow was
invented after the
crossbow
51
0.3922
0.49309
0.2535
0.5308
57
Snowboarding is
believed to have
originated in Europe
51
0.5294
0.5041
0.3876
0.6712
58
Fly-fishing is the newest
method of recreational
fishing
51
0.4314
0.5002
0.2907
0.5721
59
The invention of roller
skates has been
traditionally credited to a
Russian
51
0.4902
0.50488
0.3482
0.6322
60
In foxhunting, the hunter
usually wears a purple
shirt
51
0.1961
0.40098
0.0833
0.3089
61
Gnu is another name for
antelope
48
0.4792
0.50485
0.3326
0.6258
TRIVIA&CLAIM&NORMING&
16
62
The white shark subsists
on plankton
48
0.3333
0.47639
0.195
0.4717
63
Most sea turtles are
herbivorous
48
0.7708
0.42474
0.6475
0.8942
64
Mice do not have gall
bladders
48
0.3333
0.47639
0.195
0.4717
65
Moose may dive
underwater while mating
48
0.4167
0.49822
0.272
0.5613
66
Sheep are a type of
tylopod mammal
48
0.4375
0.50133
0.2919
0.5831
67
Domesticated hogs are
descended from the
pasang
48
0.4375
0.50133
0.2919
0.5831
68
The mouth of a sea
urchin is on its top
48
0.5833
0.49822
0.4387
0.728
69
Some salmon use the
Earth’s electrical field
to navigate
48
0.6875
0.46842
0.5515
0.8235
70
Most eels are social fish
48
0.3958
0.4942
0.2523
0.5393
71
Walruses use their tusks
primarily for feeding
51
0.2941
0.46018
0.1647
0.4235
72
A koala's pouch opens
frontward
51
0.7255
0.45071
0.5987
0.8523
73
Both sexes of lions are
monogamous
51
0.3137
0.46862
0.1819
0.4455
74
The flamingo's pink
color comes from
anthocyanin pigments in
its food
51
0.7255
0.45071
0.5987
0.8523
75
Guinea pigs belong to
the muridae family
51
0.5098
0.50488
0.3678
0.6518
76
The otter belongs to the
squirrel family
51
0.4314
0.5002
0.2907
0.5721
77
A panda's rear paws
point outwards
51
0.4902
0.50488
0.3482
0.6322
78
Female turkeys generally
weigh twice as much as
males
51
0.549
0.50254
0.4077
0.6904
79
Snakes have movable
eyelids
51
0.4118
0.49705
0.272
0.5516
80
Giraffes have terrible
eyesight
51
0.5098
0.50488
0.3678
0.6518
81
Dough is boiled in the
process of making
croissants
48
0.5208
0.50485
0.3742
0.6674
TRIVIA&CLAIM&NORMING&
17
82
Most limes have less
acid than lemons
48
0.75
0.43759
0.6229
0.8771
83
The grape plant is a
large herb
48
0.3542
0.48332
0.2138
0.4945
84
Corn was first
domesticated by native
peoples in Argentina
48
0.4167
0.49822
0.272
0.5613
85
Spain produces most of
the world's almonds
48
0.5
0.50529
0.3533
0.6467
86
Chicken is the most
popular meat in the
United States
48
0.75
0.43759
0.6229
0.8771
87
Ham is the front leg of a
hog
48
0.3125
0.46842
0.1765
0.4485
88
The Colchester is a
popular type of clam
48
0.5833
0.49822
0.4387
0.728
89
Kvass is an alcoholic
beverage fermented from
honey
48
0.4167
0.49822
0.272
0.5613
90
Mate is a beverage made
from the root of the
pepper plant
48
0.4583
0.50353
0.3121
0.6045
91
Mayonnaise is usually
made with raw egg
whites
51
0.6275
0.48829
0.4901
0.7648
92
Carrots are a good
source of the
phytochemical lycopene
51
0.8431
0.36729
0.7398
0.9464
93
The navy bean is also
known as the sieva bean
51
0.4902
0.50488
0.3482
0.6322
94
Cabbages are in the
parsley family
51
0.3725
0.48829
0.2352
0.5099
95
Couscous is a dish from
Asia
51
0.549
0.50254
0.4077
0.6904
96
Sherbert has less sugar
than ice cream
51
0.6275
0.48829
0.4901
0.7648
97
The first extraction of
molasses from sugar
crystals produces
blackstrap molasses
51
0.6667
0.4761
0.5328
0.8006
98
Sticky toffee pudding is
a classic Polish dessert
51
0.451
0.50254
0.3096
0.5923
99
Beet leaves contain
toxins
51
0.6078
0.49309
0.4692
0.7465
100
Halvah is a confection
made of sunflower seeds
51
0.5294
0.5041
0.3876
0.6712
TRIVIA&CLAIM&NORMING&
18
References
Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998). The mismeasure of memory: when
retrieval fluency is misleading as a metamnemonic index. Journal of Experimental
Psychology: General, 127(1), 55-68.
Birnbaum, M. S., Kornell, N., Bjork, E. L., & Bjork, R. A. (2013). Why interleaving enhances
inductive learning: The roles of discrimination and retrieval. Memory and
Cognition, 41(3), 392-402.
Burke, D. M., MacKay, D. G., Worthley, J. S., & Wade, E. (1991). On the tip of the tongue:
What causes word finding failures in young and older adults?. Journal of Memory and
Language, 30(5), 542-579.
Chua, E. F., & Bliss-Moreau, E. (2016). Knowing your heart and your mind: The relationships
between metamemory and interoception. Consciousness and Cognition, 45, 146-158.
Duñabeitia, J. A., Griffin, K. L., Martín, J. L., Oliva, M., Sámano, M. L., & Ivaz, L. (2016). The
Spanish General Knowledge Norms. Frontiers in Psychology, 7, 1888.
Fenn, E., Newman, E. J., Pezdek, K., & Garry, M. (2013). The effect of nonprobative
photographs on truthiness persists over time. Acta psychologica, 144(1), 207-211.
Hasher, L., Goldstein, D., & Toppino, T. (1977). Frequency and the conference of referential
validity. Journal of verbal learning and verbal behavior, 16(1), 107-112.
Jalbert, M., Newman, E., & Schwarz, N. (2017, November). Only half of what Ill tell you is
true: How experimental procedures lead to an underestimation of the truth effect in the
real world. Poster presented at the Psychonomic Society Annual Convention. Vancouver,
Canada.
TRIVIA&CLAIM&NORMING&
19
Kornell, N. (2015). If it is stored in my memory I will surely retrieve it: anatomy of a
metacognitive belief. Metacognition and Learning, 10(2), 279-292.
Metcalfe, J. (1986). Feeling of knowing in memory and problem solving. Journal of
Experimental Psychology: Learning, Memory, and Cognition, 12(2), 288-294.
Nelson, T. O., & Narens, L. (1980). Norms of 300 general-information questions: Accuracy of
recall, latency of recall, and feeling-of-knowing ratings. Journal of Verbal Learning and
Verbal Behavior, 19(3), 338-368.
Newman, E. J., Garry, M., Unkelbach, C., Bernstein, D. M., Lindsay, D. S., & Nash, R. A.
(2015). Truthiness and falsiness of trivia claims depend on judgmental contexts. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 41(5), 1337-1348.
Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a
participant pool. Current Directions in Psychological Science, 23(3), 184-188.
Shimamura, A. P., & Squire, L. R. (1987). A neuropsychological study of fact memory and
source amnesia. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 13(3), 464-473.
Tauber, S. K., Dunlosky, J., Rawson, K. A., Rhodes, M. G., & Sitzman, D. M. (2013). General
knowledge norms: Updated and expanded from the Nelson and Narens (1980)
norms. Behavior Research Methods, 45(4), 1115-1143.
Zajonc, R.B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social
Psychology: Monograph Supplement, 9, 1-27.
&
... Yet, in spite of all this, general knowledge is still highly valued by individuals, and is usually regarded as one of the hallmarks of what we consider to be a "cultured person". Thus, it comes as no surprise that psychologists have remained interested in the acquisition and retainment of this type of data over the years (e.g., Bäckman and Lipinska, 1993;Murayama and Kuhbandner, 2011;Coane and Umanath, 2019) and that much effort has been invested into creating and updating scales as a means to measure a person's general knowledge (Duñabeitia et al., 2016;Nelson and Narens, 1980;Jalbert, Newman and Schwarz, 2019;Tauber, Dunlosky, Rawson, Rhodes and Sitzman, 2013;Martín-Luengo, Zinchenko, Alekseeva and Shtyrov, 2020). ...
... However, while this made the database applicable to a Spanish population, it also meant the set was reduced from 300 questions to only 132. As pointed out by Jalbert et al. (2019), only having access to such a limited number of questions poses a threat to the viability of the database if used repeatedly, as it increases the likelihood that a given person has already been exposed to the same question, hampering the validity of any results that might be found. To address these issues, we present a new set of 1364 general knowledge questions that have been selected to cover a wide variety of seemingly country-unspecific topics, and that have been normed using a Spanish population. ...
... It also allows us to include multipleword answers, widening the type of questions that can be included. Moreover, as noted by Jalbert et al. (2019), several studies examining statement recognition and perceived truth have adapted the Nelson and Narens (1980) questions to form true ("Photosynthesis is the name of the process by which plants make their food") and false ("Chemosynthesis is the name by which plants make their food") statements (e.g., Brashier at al., 2017;Fazio et al., 2015;Newman, Garry, Bernstein, Kantner and Lindsay, 2012). The incorrect answers used as part of the false statements do not necessarily follow any kind of norming between different studies, hampering the cross-validity of any possible findings. ...
Article
Full-text available
General knowledge questionnaires have been ubiquitously used to study a wide variety of phenomena, such as illusory truth, error correction and tip-ofthe-tongue situations. However, their normings are highly restricted to the territory and the time period they in which they were obtained. This requires that new normings are obtained for each new territory in which they be used. Here, we present a new set of 1364 general knowledge questions normed for a Spanish population. The questions span a total of 37 different fields of knowledge and an extensive range of difficulty levels. They are formulated in a multiple-choice format, and pick rates for the correct answer as well as for the three incorrect response options are provided. We hope that a database of such size and flexibility will prove to be a useful research tool for the Spanish community.
... Yet, while much effort has been put into creating scales to measure an individual's general knowledge level-the most prominent ones being Nelson and Narens (1980) and Tauber et al. (2013), but see also Duñabeitia et al. (2016), Jalbert et al. (2019) and Martín-Luengo et al. (2020)-most of the research in which they have been employed has not focused on such constructs per se. Instead, these scales have mostly been utilized as a source of questions and statements in experiments studying phenomena such as illusory truth (Fazio et al., 2015), metacognition (Jackson & Greene, 2014;Weinstein & Roedinger, 2010) and error correction (Sitzman et al., 2014;Sitzman et al., 2015), as the wide range of topics they cover acts as an easy way to control for the influence of the content of the items. ...
Article
Full-text available
Studies on sociodemographic data and crystallized intelligence have often struggled to recruit enough participants to achieve sufficient validity. However, the advent of the internet now allows this problem to be solved through the creation of megastudies. Yet, this methodology so far has only been used in studies on vocabulary size, while general knowledge, another key component of crystallized intelligence, remains unexamined. In the present study, regression models were used to examine the impact of sociodemographic variables—gender, age, years of study and socioeconomic status—on general knowledge scores. The sample comprised 48,234 participants, each of whom answered 60 general knowledge questions, their data being fully available online. Men were found to score higher than women in general knowledge. Years of study and socioeconomic status acted as strong and weak positive predictors, respectively. Age acted as a strong positive predictor until the age of 50, where it became progressively detrimental. These results are discussed relative to other studies on crystallized intelligence, highlighting the need to study each of its components individually.
Article
Full-text available
Information is judged as more true when it has been seen or heard repeatedly than when it is new. This illusory truth effect has important consequences in the real world, where we are repeatedly exposed to information of unknown veracity. While false information in natural contexts rarely comes with a warning label, false information in truth effect experiments often does. Commonly used experimental procedures alert participants to potential falsehoods at exposure through instructional warnings. Three experiments show that the size of the truth effect is over twice as large when such warnings are avoided. The influence of pre-exposure warnings on the size of the truth effect persists even after a delay of three to six days. These findings demonstrate that common experimental procedures invite a systematic underestimation of illusory truth effects. They also highlight that simple warnings can curb the impact of repetition on judgments of truth.
Article
Ease of processing-cognitive fluency-is a central input in assessments of truth, but little is known about individual differences in susceptibility to fluency-based biases in truth assessment. Focusing on two paradigms-truthiness and the illusory truth effect-we consider the role of Need for Cognition (NFC), an individual difference variable capturing one's preference for elaborative thought. Across five experiments, we replicated basic truthiness and illusory truth effects. We found very little evidence that NFC moderates truthiness. However, we found some evidence that (without an experimental warning), people high on NFC may be more susceptible to the illusory truth effect. This may reflect that elaborative thought increases the fluency with which encoded statements are processed after a delay (thus increasing the illusory truth effect). Future research may fruitfully test whether the influence of NFC and other individual difference measures depends on whether people are making immediate or delayed truth judgments.
Article
Full-text available
Retrieval failures—moments when a memory will not come to mind—are a universal human experience. Yet many laypeople believe human memory is a reliable storage system in which a stored memory should be accessible. I predicted that people would see retrieval failures as aberrations and predict that fewer retrieval failures would happen in the future. After responding to a set of trivia questions, participants were asked whether they would do better, about the same, or worse if they were given a different, but equally difficult, set of questions to answer. The majority of participants said they would do about the same, but more participants said they would do better next time than said they would do worse, although these participants did not actually do better. This finding was especially pronounced when participants were given feedback, suggesting that hindsight bias—the feeling, which emerges when an answer is presented, that one knew it all along—contributed to participants’ belief that they had underperformed on the first set of questions. The finding that metacognitive judgments were influenced by beliefs stands out in a literature full of studies in which beliefs fail to influence judgments.
Article
Full-text available
Mechanical Turk (MTurk), an online labor market created by Amazon, has recently become popular among social scientists as a source of survey and experimental data. The workers who populate this market have been assessed on dimensions that are universally relevant to understanding whether, why, and when they should be recruited as research participants. We discuss the characteristics of MTurk as a participant pool for psychology and other social sciences, highlighting the traits of the MTurk samples, why people become MTurk workers and research participants, and how data quality on MTurk compares to that from other pools and depends on controllable and uncontrollable factors.
Article
Full-text available
When people rapidly judge the truth of claims presented with or without related but nonprobative photos, the photos tend to inflate the subjective truth of those claims-a "truthiness" effect (Newman et al., 2012). For example, people more often judged the claim "Macadamia nuts are in the same evolutionary family as peaches" to be true when the claim appeared with a photo of a bowl of macadamia nuts than when it appeared alone. We report several replications of that effect and 3 qualitatively new findings: (a) in a within-subjects design, when people judged claims paired with a mix of related, unrelated, or no photos, related photos produced truthiness but unrelated photos had no significant effect relative to no photos; (b) in a mixed design, when people judged claims paired with related (or unrelated) and no photos, related photos produced truthiness and unrelated photos produced "falseness;" and (c) in a fully between design, when people judged claims paired with either related, unrelated, or no photos, neither truthiness nor falsiness occurred. Our results suggest that photos influence people's judgments when a discrepancy arises in the expected ease of processing, and also support a mechanism in which-against a backdrop of an expected standard-related photos help people generate pseudoevidence to support claims. (PsycINFO Database Record (c) 2015 APA, all rights reserved).
Article
Full-text available
The Nelson and Narens (Journal of Verbal Learning and Verbal Behavior 19:338-368, 1980) general knowledge norms have been valuable to researchers in many fields. However, much has changed over the 32 years since the 1980 norms. For example, in 1980, most people knew the answer to the question "What is the name of the Lone Ranger's Indian sidekick?" (answer: Tonto), whereas in 2012, few people know this answer. Thus, we updated the 1980 norms and expanded them by providing new measures. In particular, we report two new metacognitive measures (confidence judgments and peer judgments) and provide a detailed report of commission errors. Each of these measures will be valuable to researchers, and together they are likely to facilitate future research in a number of fields, such as research investigating memory illusions, metamemory processes, and error correction. The presence of substantial generational shifts from 1980 to 2012 necessitates the use of updated norms.
Article
Full-text available
Kornell and Bjork (Psychological Science 19:585-592, 2008) found that interleaving exemplars of different categories enhanced inductive learning of the concepts based on those exemplars. They hypothesized that the benefit of mixing exemplars from different categories is that doing so highlights differences between the categories. Kang and Pashler (Applied Cognitive Psychology 26:97-103, 2012) obtained results consistent with this discriminative-contrast hypothesis: Interleaving enhanced inductive learning, but temporal spacing, which does not highlight category differences, did not. We further tested the discriminative-contrast hypothesis by examining the effects of interleaving and spacing, as well as their combined effects. In three experiments, using photographs of butterflies and birds as the stimuli, temporal spacing was harmful when it interrupted the juxtaposition of interleaved categories, even when total spacing was held constant, supporting the discriminative-contrast hypothesis. Temporal spacing also had value, however, when it did not interrupt discrimination processing.
Article
Humans experience a unified self that integrates our mental lives and physical bodies, but many studies focus on isolated domains of self-knowledge. We tested the hypothesis that knowledge of one’s mind and body are related by examining metamemory and interoception. We evaluated two dimensions of metamemory and interoception: subjective beliefs and the accuracy of those beliefs compared to objective criteria. We first demonstrated, in two studies, that metamemory beliefs were positively correlated with interoceptive beliefs, and this was not due to domain-general confidence. Finally, we showed that individuals with better metamemory accuracy also had better interoceptive accuracy. Taken together, these findings suggest a common mechanism subserving knowledge of our cognitive and bodily states.
Article
When making rapid judgments about the truth of a claim, related nonprobative information leads people to believe the claim-an effect called "truthiness" (Newman, Garry, Bernstein, Kantner, & Lindsay, 2012). For instance, within a matter of seconds, subjects judge the claim "The Mona Lisa has no eyebrows," to be true more often when it appears with a photograph of the Mona Lisa viewed at a distance by a person. But does truthiness persist longer than a few seconds? To determine if truthiness "sticks," we asked people to judge if each trivia claim in a series was true. Half of the claims appeared with nonprobative photos; the rest appeared alone. In a second session 48h later, people returned and made the same judgments about the same statements, but this time, all claims appeared without photos. We found that truthiness "stuck." The magnitude of the effect of photos on subjective feelings of truth was consistent over time. These results fit with those from cognitive and educational psychology, as well as with the related idea that photos make relevant information more available and familiar-and therefore feel more true-even after a delay.