ArticlePDF Available

Abstract

The feeling of clarity can be dangerously seductive. It is the feeling associated with understanding things. And we use that feeling, in the rough-and-tumble of daily life, as a signal that we have investigated a matter sufficiently. The sense of clarity functions as a thought-terminating heuristic. In that case, our use of clarity creates significant cognitive vulnerability, which hostile forces can try to exploit. If an epistemic manipulator can imbue a belief system with an exaggerated sense of clarity, then they can induce us to terminate our inquiries too early-before we spot the flaws in the system. How might the sense of clarity be faked? Let's first consider the object of imitation: genuine understanding. Genuine understanding grants cognitive facility. When we understand something, we categorize its aspects more easily; we see more connections between its disparate elements; we can generate new explanations; and we can communicate our understanding. In order to encourage us to accept a system of thought, then, an epistemic manipulator will want the system to provide its users with an exaggerated sensation of cognitive facility. The system should provide its users with the feeling that they can easily and powerfully create categori-zations, generate explanations, and communicate their understanding. And manipulators have a significant advantage in imbuing their systems with a pleasurable sense of clarity, since they are freed from the burdens of accuracy and reliability. I offer two case studies of seductively clear systems: conspiracy theories; and the standardized, quantified value systems of bureaucracies.
1
The$seductions$of$clarity$
C.#Thi#Nguyen#
University#of#Utah#
c.thi.nguyen@utah.edu#
#
#
(This#is#a#pre-print.#Please#cite#the#final#version,#
forthcoming#in#the#!"#$%&'()*+*,*-&".&/0+%")"10#&2,11%-3-(*))#
#
#
#
The$feeling$of$clarity$can$be$dangerously$seductive.$It$is$the$feeling$associated$with$understanding$
things.$And$we$use$that$feeling,$in$the$rough-and-tumble$of$daily$life,$as$a$signal$that$we$have$
investigated$a$matter$sufficiently.$The$sense$of$clarity$functions$as$a$thought-terminating$heuris-
tic.$In$that$case,$our$use$of$clarity$creates$significant$cognitive$vulnerability,$which$hostile$forces$
can$try$to$exploit.$If$an$ epistemic$ manipulator$ can$ imbue$ a$ belief$ system$with$an$exaggerated$
sense$of$clarity,$then$they$can$induce$us$to$terminate$our$inquiries$too$early$$before$we$spot$the$
flaws$in$the$system.$How$might$the$sense$of$clarity$be$faked?$Let’s$first$consider$the$object$of$imi-
tation:$genuine$understanding.$Genuine$understanding$grants$cognitive$facility.$When$we$under-
stand$something,$we$categorize$its$aspects$more$easily;$we$see$more$connections$between$its$dis-
parate$elements;$we$can$generate$new$explanations;$and$we$can$communicate$our$understand-
ing.$In$order$to$encourage$us$to$accept$a$system$of$thought,$then,$an$epistemic$manipulator$will$
want$the$system$to$provide$its$users$with$an$exaggerated$sensation$of$cognitive$facility.$The$sys-
tem$should$provide$its$users$with$the$feeling$that$they$can$easily$and$powerfully$create$categori-
zations,$generate$explanations,$and$communicate$their$understanding.$And$manipulators$have$a$
significant$advantage$in$imbuing$their$systems$with$a$pleasurable$sense$of$clarity,$since$they$are$
freed$from$the$burdens$of$accuracy$and$reliability.$I$offer$two$case$studies$of$seductively$clear$
systems:$conspiracy$theories;$and$the$standardized,$quantified$value$systems$of$bureaucracies.$
#
#
#
Here#is#a#worrying#possibility:#there#is#a#significant#gap#between#our#feeling#that#some-
thing#is#clear#and#our#actually#understanding#it.#The#sense#of#clarity#can#be#a#marker#of#cog-
nitive#success,#but#it#can#also#be#seductive.#Oversimplifications#slip#easily#into#our#minds#and#
connive#themselves#into#our#deliberative#processes.#
In#that#case,#the#sense#of#clarity#might#be#intentionally#exaggerated#for#exploitative#ends.#
2
Outside#forces,#with#an#interest#in#manipulating#our#beliefs#and#actions,#can#make#use#of#clar-
ity’s#appeal.#Seduction,#after# all,# often# involves#a#seducer.#Romantic# seduction,# in# its#more#
malicious#form,#involves#manipulating#the#appearances#of#intimacy#and#romance#in#order#to#
subvert#the#aims#of#the#seduced.#There#is#an#analogous#form#of#cognitive#seduction,#where#
hostile#forces#play#with#the#signals#and#appearances#of#clarity#in#order#to#lead#our#thinking#
astray.#
The#sense#of#clarity#is#a#potent#focal#point#for#manipulation#because#of#its#crucial#role#in#
managing# our# cognitive# resources.# After# all,# we# only# have# so# much# mental# energy# to# go#
around;#we#need#to#prioritize#our#inquiries.#In#particular,#we#need#some#way#to#estimate#that#
we’ve#probably#thought#enough#on#some#matter#for#the#moment#—#that#it’s#probably#safe#to#
move#on#to#more#pressing#matters,#even#if#we#haven’t#gotten#to#the#absolute#rock#bottom#of#
the#matter.#Our#sense#of#clarity,#and#its#absence,#plays#a#key#role#in#our#cognitive#self-regula-
tion.#A#sense#of#confusion#is#a#signal#that#we#need#to#think#more.#But#when#things#feel#clear#
to#us,#we# are# satisfied.#A#sense#of#clarity#is#a#signal#that# we# have,#for#the#moment,#thought#
enough.#It#is#an#imperfect#signal,#but#it#is#one#we#often#actually#use#in#the#quick-and-dirty#of#
everyday#practical#deliberation.#This#shows#why,#say,#manipulative#interests#might#be#par-
ticularly#interested#in#aping#clarity.#If#the#sense#of#clarity#is#a#thought-terminator,#then#suc-
cessful#imitations#of#clarity#will#be#quite#powerful.#If#somebody#else#can#stimulate#our#sense#
of#clarity,#then#they#can#gain#control#of#a#particular#cognitive#blind#spot.#They#can#hide#their#
machinations#behind#a#veil#of#apparent#clarity.#
Here’s#another#way#to#put#it:#the#moment#when#we#come#to#understand#often#has#a#par-
ticular#feel#to#it# —# what# some#philosophers#have# called# the# “a-ha!”#moment.#The#moment#
when#we#come#to#understand,#says#Alison#Gopnik,#is#something#like#an#intellectual#orgasm#
3
(Gopnik,#1998).#And,#as#John#Kvanvig#suggests,#it#is#our#internal#sense#of#understanding#—#
our#sense# of# “a-ha!”# and# “Eureka!”# —# that# provides# a#sense# of# closure# to# an# investigation#
(Kvanvig,#2011,#88).# The# “a-ha”#feeling#is#both#pleasurable#and#indicates# that#a#matter#has#
been#investigated# enough.# If,#then,# hostile# forces#can# learn# to#simulate# that# “a-ha”# feeling,#
then#they#will#have#a#very#powerful#weapon#for#epistemic#manipulation.#
I#offer#two#sustained#case#studies#of#cognitive#subversion#through#the#seductions#of#clar-
ity.#First,#I#will#look#at#the#sorts#of#belief#systems#often#promulgated#by#moral#and#political#
echo#chambers,#which#offer#simplistic#pictures#of#a#world#full#of#hostile#forces#and#conspiracy#
theories.#Such#belief#systems#can#create#an#exaggerated#sense#of#clarity,#in#which#every#event#
can#be#easily#explained#and#every#action#easily#categorized.#Second,#I#will#look#at#the#seduc-
tive#clarity#of#quantification.#I#borrow#my#use#of#“seduction”#from#Sally#Engle#Merry’s#40-&
2-5,6*+"()&".&7,$(*+.+6$*+"(#(2016),#a#study#into#how#global#institutions#deploy#metrics#and#
indicators#in#the#service#of#political#influence.#Merry#focuses#on#the#generation#of#indicators#
and#metrics#on#the#global#stage,#such#as#the#Human#Development#Index,#which#attempts#to#
sum#up#the#quality#of#life#across#each#country’s#entire#citizenship#in#a#single,#numerical#score.#
The#HDI#then#compiles#these#scores#to#offer#a#single#apparently#authoritative#ranking#of#all#
countries# by# their# quality# of# life.# Such# systems# of# quantification# can# offer# an# exaggerated#
sense#of#clarity#without#an#accompanying#amount#of#understanding#or#knowledge.#Their#cog-
nitive#appeal#can#outstrip#their#cognitive#value.#
It#is#striking#how#quantified# presentations# of&value# seem# to#have#a#profound# cognitive#
stickiness.# The# motivational# draw# of# quantified# values# has# been# well-documented# across#
many# terrains# (Porter# 1996;# Merry# 2016;# Espeland# and# Sauder# 2016).# This# motivational#
4
power#is#why#so#many#companies#and#governments#have#become#interested#in#the#technol-
ogies# of# gamification.# Gamification# attempts# to# incorporate# the# mechanics# of# games# —#
points,#experience#points,#and#leveling#up#—#into#non-game#activities,#in#order#to#transform#
apparently#“boring”#activity#as#work#and#education#into#something#more#engaging,#compel-
ling,#and#addictive#(McGonigal#2011;#Walz#et#al#2015;#Lupton#2016).#I#am#worried,#however,#
that#gamification# might# increase#motivation,# but#only# at# the# cost# of#changing# our# goals#in#
problematic#ways.#After#all,#step#counts#are#not#the#same#as#health,#and#citation#rates#are#not#
the#same#as#wisdom#(Nguyen#2020,#189-215;#forthcoming).#The#seductions#of#clarity#are,#I#
believe,#one#important#mechanism#through#which#gamification#works.#
Let#me#be#clear:#the#present#inquiry#is#not#a#study#in#ideal#rationality,#nor#is#it#a#study#of#
epistemic#vice# and# carelessness.#It# is#a# study# in# the# vulnerabilities#of# limited,# constrained#
cognitive#agents,#and#how#environmental#features#might#exploit#those#vulnerabilities.#It#is#a#
foray#into#what#we#might#call#0")*+%-&-1+)*-3"%"8#.#Hostile#epistemology#includes#the#inten-
tional#efforts# of# epistemic# manipulators,# working# to# exploit# those# vulnerabilities# for# their#
own#ends.#We# might#call#the#study#of#these#intentional# epistemic#hostilities#6"39$*&-1+)*-:
3"%"8#;#Hostile#epistemology#also#includes#the#study#of#environmental#features#which#pre-
sent#a#danger# to#those#vulnerabilities,#made#without#hostile# epistemic#intent.#Hostile#envi-
ronments,#after#all,#don’t#always#arise#from#hostile#intent.#Hostile#environments#include#in-
tentionally#placed#minefields,#but#also#crumbling#ruins,#the#deep#sea,#and#Mars.#An#epistem-
ically#hostile#environment#contains#features# which,# whether# by# accident,# evolution,#or#de-
sign,#attack#our#vulnerabilities.##
I#will#focus#for#the#early#parts#of#this#paper#on#cases#of#combat#epistemology.#I#think#this#
is#the#easiest#place#to#see#how#certain#sorts#of#systems#have#a#hostile#epistemic#function.#The#
5
cases# of# intentionally# manufactured# hostile# environments# will# then# help# us# to# recognize#
cases#of# the# unintentional# formation# of# hostile# epistemic#environments.# Hostile# epistemic#
environments#can#arise#from#entirely#well-intentioned,#and#even#successful,#pursuits#of#other#
purposes.#A#culinarily#extraordinary#pastry#shop#also#presents#an#environment#hostile#to#my#
attempts#at#healthy#eating.#In#many#bureaucratic#cases,#as#we#will#see,#systems#of#quantifica-
tion#often#arise#for#very#good#reason:#to#efficiently#manage#large#and#complex#institutional#
data-sets,#or# to#increase# accountability# (Scott# 1998;# Perrow# 2014).# But# these# very# design#
features#also# make# them# into# epistemically# hostile# environment.# Because# of# the# magnetic#
motivational#pull#of#quantification,#the#very#features#which#render#them# good# for# efficient#
administration#also#functions#to#imbue#them#with#seductive#clarity.
1
#
Other#recent#inquiries#into#hostile#epistemology#include#discussions#of#epistemic#injus-
tice,#propaganda,#echo#chambers,#fake#news,#and#more#(Fricker,#2007;#Medina#2012;#Dotson#
2014;#Stanley,#2016;#Rini#2017;#Nguyen#2018b).#Importantly,#the#study#of#hostile#epistemol-
ogy#is#distinct#from#the#study#of#epistemic#vice.#The#study#of#the#epistemic#vices#—#such#as#
closed-mindedness,#gullibility,#active#ignorance,#and#cynicism#—#is#a#study#of#epistemically#
problematic#character# traits.# It# is# the# study# of#failings# in#the# epistemic# agents# themselves#
(Sullivan#and#Tuana#2007;#Proctor#and#Schiebinger#2008;#Cassam#2016;#Battaly#2018).#Hos-
tile#epistemology,#on#the#other#hand,#is#the#study#of#how#external#features#might#subvert#the#
1
!I!am!influenced!here!by!A.W.!Eaton’s!discussion!of!artifact!function,!which!draws!on!and!develops!Ruth!Milli-
kan’s!notion!of!function!(Millikan!1984,!Eaton!2020).!Eaton!argues!that!the!intent!of!an!artifact’s!designer!does!
not!determine!that!artifact’s!function.!She!suggests!a!more!evolutionary!model:!An!artifact!may!be!unintention-
ally!imbued!with!trait,!but!insofar!as!that!trait!is!selectively!reproduced!in!future!artifacts,!then!its!effect!is!part!
of!those!artifacts’!function.!So,!if!a!bureaucracy!generates!a!quantified!metric!for!accounting!purposes,!but!that!
quantified!metric!survives! and!is!reproduced!in!further! bureaucratic!systems!because!of!its! seductive!effect,!
then!the!seductiveness!is!part!of!those!systems’!function.!
6
efforts#of#epistemic#agents.#Of#course,#vice#and#hostility#are#often#entangled.#Hostile#environ-
ments#press#on#our#vices#and#make#it#easier#for#us#to#fall#more#deeply#into#them.#But#vice#and#
hostility#represent#two#different#potential#loci#of#responsibility#for#epistemic#failure.#
This#all#might#just#seem#like#common#sense.#Of#course#people#are#drawn#to#oversimplifi-
cations;#what’s#new#in#that?#But#there#are#important#questions#here,#about#why#we’re#drawn#
to#oversimplification#and#how#culpable#we#are#for#giving#in#to#it.#Importantly,#many#theorists#
treat#our#interest#in#oversimplification#as#straightforwardly#irrational.#In#the#psychological#
and#social#sciences,#the#appeal#of#oversimplification#is#usually#explained#as#a#mistake#which#
can#be#understood# in#terms#of#individual#psychological#tendencies,#such#as# motivated#rea-
soning#or#the#undue#influence#of#the#emotions.#We#accept#oversimplifications,#it#is#thought,#
because#they#make#us#feel#smug,#they#comfort#us,#or#they#reinforce#our#sense#of#tribal#identity#
(Kahan#and#Braman#2006;#Sunstein#2017).#Similarly,#many#philosophical#accounts#treat#our#
susceptibility# to# oversimplification# as# a# problem#arising# wholly# from# an# individual’s# own#
personal#failures# of# character#–# from# their#epistemic# vices.# Quassim#Cassam,# for# example,#
tells#the#story#of#Oliver#the#conspiracy#theorist,#who#believes#that#9/11#was#an#inside#job.#
Says#Cassam,#there#isn’t#a#good#rational#explanation#for#Oliver’s#beliefs.#The#best#explanation#
is#a#failure#of#intellectual#character.#Oliver,#says#Cassam,#is#gullible#and#cynical;#he#lacks#dis-
cernment#(162-3).#
I#will#present#a#picture#that#is#far#more#sympathetic#to#the#seduced.#It#is#a#picture#in#which#
exaggerated# clarity# plays# upon# specific# structural# weaknesses# in# our# cognition.# As# cogni-
tively#limited#beings,#we#need#to#rely#on#various#heuristics,#signals,#and#short-cuts#to#manage#
the#cognitive#barrage.#But#these#strategies#also#leave#us#vulnerable#to#exploitation.#Seductive#
7
clarity#takes#advantage#of#our# cognitive# vulnerabilities,# which#arise,#in#turn,#from#our#per-
fectly#reasonable#attempts#to#cope# with# the# world# using# our#severely#limited#cognitive#re-
sources.#And,# certainly,# the#pull# of# seductive#clarity# will# be# worse# if#we# give# in#to# various#
epistemic#vices.#And,#certainly,#once#we#realize#all#this,#we#will#want#to#act#more#vigorously#
to#secure#the#vulnerable#backdoors#to#our#cognition.#The#general#point,#however,#is#that#giv-
ing#into# the#seductions#of#clarity#isn’t# just#some#brute#error,#or#the#result#of#sheer# laziness#
and#epistemic#negligence.#Rather,#it#is#driven,#in#significant#degree,#by#systems#and#environ-
ments#which#function#to#exploit#the#cognitive#vulnerabilities#generated#by#the#coping#strate-
gies#of#cognitively#finite#beings.#
#
#
Clarity(as(thought-terminator(
#
I#have#been#speaking#loosely#so#far;#let#me#now#stipulate#some#terminology.#On#the#one#
hand,#there#are#epistemically#positive#states:#knowledge,#understanding,#and#the#like.#On#the#
other#hand,#there#are#the#phenomenal#states#that#are#connected#to#those#epistemically#posi-
tive#state.#These# are# the#-<1-=+-(6-)& of# being#in# an# epistemically#positive# state# —#like# the#
sense#of#understanding,#the#feeling#of#clarity.#Loosely:#understanding#is#our#successful#grasp#
of#parts#of#the#world#and#their#relationships,#and#the#sense#of#clarity#is#the#phenomenal#state#
associated#with# understanding.# For#brevity’s# sake,# let#me# use# the#terms# “clarity”#and# “the#
sense#of#clarity”#interchangeably,#to#refer#to#the#phenomenal#experience#associated#with#un-
derstanding.#I#do#not#mean#to#be#using#“clarity”#in#the#Cartesian#sense,#where#it#is#a#perfect#
guarantee#of#knowledge.#Clarity,#in#my#usage,#is#merely#an#impression#of#a#certain#kind#of#
8
cognitive#success#—#what#J.D.#Trout#has#called#the#)-()-#of#understanding#(Trout#2002).#Clar-
ity#may#often# accompany#genuine#understanding,#but#it#is#by# no#means#a#perfect#indicator#
that#we# do,# in# fact,# genuinely# understand.#So#external# forces#can# exploit#the# gap#between#
genuine#understanding#and#the#feeling#of#understanding#–#that#sense#of#clarity.##
There#are#two#general#strategies#for#epistemic#manipulation.#There#is#-1+)*-3+6&+(*+3+5$:
*+"(:#the#strategy#of#trying#to#get#an#epistemic#agent#to#accept#something#by#making#them#
afraid#or#uncomfortable#to#think#otherwise.#There#is#also#-1+)*-3+6&)-5,6*+"(:#the#strategy#of#
manipulating#positive# cognitive#signals#to#get#an#epistemic#agent#to#accept#something.#The#
manipulation#of#clarity#is#a#form#of#epistemic#seduction.#It#is#the#attempt#to#use#our#own#cog-
nitive#processes#against#us,#whispering#pleasantly#all#the#while.#
How#might#clarity#seduce?#There#are#many#potential#pathways.#For#one#thing,#clarity#se-
duces#because# it#is#pleasurable.#But# for#the#remainder#of#this#discussion,#I’ll#focus#another,#
even#more#dangerous#feature:#that#the#sense#of#clarity#can#bring#us#to#end#our#inquiries#into#
a#topic#too#early.#This#possibility#arises#because#of#the#profoundly#quick-and-dirty#nature#of#
daily#decision-making.#We#are#finite#beings#with#limited#cognitive#resources.
2
#In#daily#life,#we#
need#to#figure#out#what#to#do:#where#to#spend#our#money,#who#to#vote#for,#which#candidate#
to#back.#We#face#a#constant#barrage#of#potentially#relevant#information,#evidence,#and#argu-
ment#—#far#more#than#we#could#assess#in#any#conclusive#manner.#So#we#need#to#figure#out#
the#best#way#to#allocate#our#cognitive#resources#while#leaving#most#of#our#investigations#un-
finished,#in#some#cosmic#sense.#
2
!Two!particularly!relevant!discussions!on!cognitive!limitation!and!epistemology!are!(Wimsatt!2007;!Dallman!
2017).!
9
When#practically#reasoning#about#the#messy#complexities#of#the#real#world,#we#are#un-
likely#to#arrive#at#any#conclusive#ground-floor,#where#we#can#know#with#any#certainty#that#
we’re#done.
3
#So,#for#everyday#practical#deliberation,#we#need#some#method#for#determining#
that#we’ve#thought#enough.$
4
#And#that#basis#often#needs#to#be#fast#and#loose,#to#cope#with#the#
fast#and#loose#manner#of#everyday#practical#deliberation.#We#need#some#basis#for#estimating#
that#our#understanding#is#probably#good#enough,#so#that#we#can#make#a#decision#and#move#
on.#We#need#something#like#a#heuristic#for#terminating#thought.#
Here,#then,#is#the#ruling#supposition#for#my#inquiry:#the#sense#of#clarity#is#one#of#the#sig-
nals#we#typically#use#to#allocate#our#cognitive#resources.#(I#do#not#claim#that#it#is#the#only#
signal,#though# I# do# claim#it# is# a#significant# one.)# We#often# use# our# sense#of# confusion# as#a#
3
!As!Elijah!Millgram! puts! it,! practical!reasoning! doesn’t! result! in! settled! arguments!to!finalized! conclusions.!
Practical!reasoning!produces!only!tentative!conclusions.!Practical!conclusions!are!always!open!to!defeat!from!
unexpected!angles,!and!new!forms!of!defeat!may!always!surprise!us!(Millgram!1997).!The!closest!we!can!get!to!
conclusiveness!is!to!think!that!a!certain!piece!of!practical!reasoning!seems!good!enough,!so!far!as!we!can!tell.!
And!even!if! you! reject! Millgram’s! view!and!believe!that!there!were!firm! practical! conclusions! that!we!might!
eventually!reach!—!surely,!finding!such!firm!conclusions!is!well!beyond!the!reach!of!most!human-scale!practical!
deliberation!in!everyday!circumstances.!
4
!Very!little!has!been!written!on!how!we!decide!to!end!our!inquiries!in!practical!deliberation.!And!much!of!that!
work!has!focused,!not!on!fast-and-loose!daily!heuristics!for!terminating!inquiry,!but!on!when!we!can!conclu-
sively!terminate!inquiry.!See,! for! example,! Alan! Millar!and!Kvanvig’s!debate!about!whether! we! need! merely!
need!knowledge!to!conclusively!terminate!inquiry,!or!whether!we!need!to!reflectively!know!that!we!know!in!
order!to!terminate!inquiry!(Millar!2011;!Kvanvig,!2011).!Trout!himself!argues!that!the!“sense!of!understanding”!
—!that!“a-ha”!feeling!!!is!not!of!particular!use!in!the!sciences!because!it!is!quite!vulnerable!to!cognitive!biases!
and!other!corrupting!psychological!influences.!In!Trout’s!terms,!the!mere!sense!of!understanding!doesn’t!grant!
us!what!we!really!want!in!science,!which!is!good!explanations.!We!have!other!ways!of!recognizing!good!expla-
nations,!far!more!accurate!than!mere!internal!feelings.!We!know!we!have!a!good!scientific!explanation!when!
our!scientific!model!makes!good!predictions.!We!should,!says!Trout,!therefore!largely!ignore!the!various!inter-
nal!signals!of!understanding,!which!will!simply!lead!us!astray.!We!should,!instead,!remain!firmly! fixed!on!the!
evidence!that!our!scientific!model!provides!good!explanations,!which!are!measured!in!the!usual!scientific!meth-
ods:!prediction,!testing,!and!the!like!(Trout!2002,!2017).!Notice,!however,!that!this!sort!of!approach!imagines!
the!relevant!epistemic!agents!to!be!cognitively!ideal!beings!with!essentially!unlimited!resources.!It!then!asks!
how!such! cognitive!beings!!should! go!about!getting!things! right!once!and!for!all.!And!that!might!be!the!right!
idealization!for!thinking! about! how! we! should!pursue!long-term!epistemic!projects! as! parts! of! intergenera-
tional!communities,!as!we!do!in!philosophy!and!science.!But!things!look!very!different!for!cognitively!limited!
beings!in!the!quick-and-dirty!of!day-to-day!decision-making.!Sometimes!we!might!be!able!to!adopt!some!meth-
odology!with!a!pre-established!threshold!for!terminating!thought.!Consider,!for!example,!the!cognitive!strategy!
of!satisficing:!taking!the! first! solution! which! crosses!some!pre-established!minimal!threshold! (Simon! 1956).!
But!what!do!we!do!when!we!aren’t!satisficing?!In!many!cases,!our!investigations!are!more!open-ended,!without!
any!sort!of!pre-established!minimal!threshold.!For!those!sorts!of!investigations,!we!need!some!heuristic!basis!
for!attentional!management.!
10
signal# that# we# need# to# keep# investigating,# and# our# sense# of# clarity# as# a# signal# that# we’ve#
thought#enough.
5
#Our#sense#of#clarity#is#a#signal#that#we#can#terminate#an#investigation.#When#
a#system#of#thought#seems#clear#to#us,#then#we#have#a#heuristic#reason#to#stop#inquiring#into#
it.
6
#
I’m#not#claiming#that#this#heuristic#is#a#necessary#part#of#all# practical# reasoning#–#only#
that#the#heuristic#is#currently#under#common#usage.#After#all,#heuristics#are#usually#contin-
gent#tendencies#and#not#necessary#parts#of#our#cognitive#architecture.#In#fact,#some#research#
suggests#that#we#can#slowly#change#the#heuristics#we#use#(Reber#and#Unkelbach#2010).#
Here’s#my#plan.#First,#we’ll#start#to#think#about#how#powerful#it#would#be#if#this#supposi-
tion#were#true,#and#there#were#such#a#pleasurable#and#thought-terminating#heuristic.#I’ll#look#
at#some#evidence#from#the#empirical#literature#on#cognitive#heuristics#that#supports#some-
thing#in#the#vicinity#of#my#supposition.#I’ll#show#how#the#supposition,#which#concerns#how#
we#use#our#feeling#of#understanding,#emerges#from#a#recent#discussion#in#the#philosophy#of#
science#about#the#nature# of# genuine#understanding.# Then,# I’ll#use#the# supposition# to# think#
about#what#sorts#systems#and#environments#might#successfully#exploit#the#sense#of#clarity.#
I’ll#dig#into#some#historical#and# sociological# literature# on# echo# chambers# and#on#the#social#
5
!My!discussion!here! heavily!borrows!structural!features!from! Elijah!Millgram’s!discussion!of!the! function!of!
boredom!and!interest!in!practical!reason!and!agency.!Millgram!argues!that!a!sense!of!interest!is!our!signal!that!
our!values!are!good!ones!for!us!to!have,!and!a!sense!of!boredom!is!our!signal!that!our!values!are!bad!for!us!to!
have,!so!we!should!change!them!(Millgram!2004).!
6
!As!far!as!I!know,!Justin!Dallman!offers!the!only!contemporary!account!of!how!our!cognitive!limitations!force!
us!to!manage!our!efforts!of!inquiry.!The!best!procedure!to!cope!with!cognitive!limitation,!he!says,!is!to!set!up!a!
priority!queue.! We! assign! priority! levels!to!our!various! outstanding! investigations,! and!then! we! proceed!in!
order!from!highest!priority!to!lowest!(Dallman!2017).!But!what!basis!do!we!have!for!assigning!priority!levels?!
To!put!my!suggestion!into!Dallman’s!terms,!we!need!some!heuristic!for!quickly!estimating!priorities,!and!our!
sense!of!clarity!functions!as!a!heuristic!basis!assigning!a!low!priority!to!its!investigation.!A!sense!of!clarity!can!
thus!terminate!a! line! of!inquiry!—!not! conclusively,! but! by! lowering! its! priority!below!the!barrage!of!other,!
more!pressing!matters.!
11
effect#of#simplistic#quantification.#The#supposition#will#turn#out#to#provide#a#unifying#expla-
nation#for#many#of#the#documented#effects#of#echo#chambers#and#quantification.#My# argu-
ment#in#favor#of#the#supposition,#then,#will#be#that#it#provides#a#unifying#explanation#for#var-
ious# observations# from# cognitive# science,# sociology,# and# history,# while#integrating#neatly#
with#a#standard#account#of#the#nature#of#understanding.#But#this#mode#of#argumentation#can#
only#render#the#supposition#a#plausible#hypothesis;#more#empirical#investigation#is#certainly#
called#for.#
#
#
Clarity(as(vulnerability(
#
Suppose,#then,#that#the#sense#of#clarity#plays#a#crucial#role#in#the#regulation#of#our#cogni-
tive#resources,#functioning#as#a#signal#that#we#can#safely#terminate#a#particular#line#of#inquiry.#
Obviously,#the#sense# of# clarity# can#come# apart# from# actual#full#understanding.
7
#It#must,# in#
order#for#it#to#play#a#heuristic#role#in#quick-and-dirty#daily#deliberation.
8
#In#order#to#know#
that#we#fully#understood#something,#we#would#need#to#conduct#an#exhaustive#and#thorough#
investigation.#The#sense#of#clarity#is#far#more#accessible#to#us,#so#we#can#use#it#to#make#rough#
estimates#about#whether#we’ve#inquired#enough.##
If#a#hostile#force#could#ape#such#clarity,#then#they#would#have#a#potent#tool#for#getting#us#
to#accept#their#preferred#systems#of#thought.#This#is#because#false#clarity#would#provide#an#
7
!For!an!in-depth!discussion!of!this!point,!see!Trout’s!discussion!of!the!gap!between!the!sense!of!understanding!
in!science,!and!actually!possessing!a!genuine!understanding!(Trout!2002).!There!is!a!useful!further!discussion!
in!(Grimm!2012,!106-109),!which!defends!Trout’s!claims!against!Linda!Zagzebksi’s!claim!that!we!always!know!
when!we!understand!(Zagzebski!2001,!247).!See!also!(Strevens!2013).!
8
!I!am!drawing!here!from!the!cognitive!science!literature!on!heuristics.!Key!relevant!moments!in!that!literature!
include!(Gigerenzer!and!Goldstein!1996;!Kahneman!2013).!
12
excellent#cover#for#intellectual#malfeasance.#A#sense#of#clarity#could#bring#us#to#terminate#our#
inquiry#into#something#before#we#could#discover#its#flaws.#It#would#be#something#like#an#in-
visibility#cloak#—# one#that#works#by#manipulating#our# attention.#Our#attention,#after#all,# is#
narrow.#We#barely#notice#what‘s#outside#the#focused#spotlight#of#our#attention.#We#can#make#
something#effectively#disappear#simply#by#directing#their#attention#elsewhere.
9
#One#way#to#
make# something# cognitively# invisible,# then,# is# by# making# it# signal# unimportance.# The# spy#
novelist#John#Le#Carre#—#who#had#actually#worked#in#British#intelligence#—#describes,#in#his#
novel#4+(>-=&4$+%"=&2"%5+-=&21#,#what#a#genuinely#effective#spy#looks#like.#They#aren’t#dashing#
and#handsome,#like#some#James# Bond# figure.# An# effective# spy#presents#as#entirely#normal,#
bland,#and#dull.#They#can#disappear#because#they#have#learned#to#magnify#the#signals#of#bor-
ingness.#Similarly,#the#techniques#of#stage#magic#involve#attentional#misdirection.#Stage#ma-
gicians#learn#to#signal#boringness#with#the#active#hand#while#directing#signals#of#interesting-
ness#elsewhere,#in#order#to#control#their#audience’s#attention.#The#sense#of#clarity#can#work#
in#an# analogous#strategy#of#attentional#misdirection.# An#epistemic#manipulator#who#wants#
us#to#accept#some#system#of#thought#should#imbue#that#system#with#a#sense#of#clarity,#so#that#
cognitive#resources#will# be# less#likely# to# be# directed#towards# it.# The#strategy#will# be# even#
more#effective#if#they#simultaneously#imbue#some#other#target#with#a#sense#of#confusion.#The#
confusing#object#seizes#our#attention#by#signaling#that#we#need#to#investigate#it,#which#makes#
it#easier#for#the#clear-seeming#system#to#recede#into#the#shadows.#The#manipulator#can#thus#
9
!The!locus!of!the!modern!discussion!of!this!sort!of!attentional!blindness!is!in!Christopher!Chabris!and!Daniel!
Simons’s!influential!experiments,!including,!famously,!an!experiment!where!half!of!the!study!subjects!failed!to!
notice!a!person!in! a!gorilla!suit!walking!across!a!room,!and!pounding!their!chest,!when!the!subjects!were! in-
structed!to!perform!a!relatively!simple!counting!task!(Chabris!and!Simons!2011).!
13
gain#control#of#their#target’s#attention#by#manipulating# their#targets’#priority#queue#for# in-
vestigation.##
Thus,#hostile#forces#can#manipulate#the#cognitive#architecture#of#resource-management#
in#order#to#bypass#the#safeguards#provided#by#the#various#processes#of#cognitive#inquiry.#In#
the#movies,#the#crooks#are#always#hacking#the#system#which#controls#the#security#cameras.#
Epistemic#criminals#will#want#to#hack#the#cognitive#equivalent.##
#
#
Ease(and(fluency(
(
The#experience#of#clarity#is#complex#and#its#phenomenal#markers#many.#Let’s#start#with#
a#case#study#in#one#small#and#simple#aspect#of#clarity#—#one#which#has#been#relatively#well-
studied#in#the#psychological#sciences.#Consider#the#experience#of#cognitive#ease#—#the#rela-
tive#degree#to#which#it#is#easy#to#think#about#something.#In#the#literature#on#cognitive#heu-
ristics,#cognitive#ease#is#part#of#the#study#of#“cognitive#fluency”,#which#is#the#“subjective#ex-
perience#of#ease#or#difficulty#with#which#we#are#able#to#process#information”#(Oppenheimer#
2008,#237).#Research#has#demonstrated#that#we#do,#in#fact,#often#use#fluency#as#a#cognitive#
heuristic.#If# we# comprehend# an# idea# easily,# we# will# be#more# likely#to# accept# it.# # Cognitive#
difficulty,#on#the#other#hand,#makes#it#more#likely#that#we#will#reject#an#idea.#This#heuristic#is#
not#entirely#unreasonable:#we#often#experience#cognitive#ease#in#a#domain#precisely#because#
we#have# a#lot#of#experience#with#it.#Cognitive#ease#often#correlates#with# experience,#which#
correlates#with#skill#and#accuracy.#But,#obviously,#ease#is#separable#from#accuracy.#Studies#
have#demonstrated#that#one’s#mere#familiarity#with#an#idea#makes#one#more#likely#to#accept#
14
it.#Familiarity#creates#a#sense#of#cognitive#ease,#but#without#the#need#for#any#relevant#skill#or#
expertise.#Studies#have#also#shown#that#we#are#more#likely#to#believe#something#written#in#a#
more#legible#font.#Legibility#leads#to#easier#processing,#which#leads#to#readier#acceptance.#In#
other#words:#we#are#using#our#cognitive#ease#with#some#proposition#or#domain#as#a#heuristic#
for#our#accuracy#with#that#proposition#or#domain.#Rolf#Reber#and#Christian#Unkelbach#have#
argued#that#fluency#heuristics#are,#in#fact,#often#quite#useful.#Through#a#Bayesian#analysis,#
they#conclude# that#fluency#is#a#good#heuristic#when#the#user’s#environment#contains#more#
true#propositions#than#false#ones#—#and#the#better#the#ratio#of#true#to#false#propositions#in#
their#environment,#the#better#the#fluency#heuristic#will#work#(Reber#and#Unkelbach#2010).##
But#that#heuristic#can#be#gamed.
10
#
Suppose#that#the#usual#fluency#heuristic#is#in#place.#How#might#it#be#exploited?#To#game#
the# fluency# heuristic,# a# manipulator# would# want# to# offer# their# targets# ideas# expressed# in#
some#familiar#manner,#by#using#well-worn#patterns#of#thought#and#forms#of#expression.#This#
exploitative#methodology#should#be#quite#familiar:#it#explains#the#rhetorical#power#of#cliched#
slogans#and#Internet#memes.#
Suppose# that# the# world# has# many# such# epistemic# manipulators# in# it,# and# has# become#
chock#full#of#misleading#ideas#that#have#been#engineered#to#seem#familiar.#Our#best#strategy#
to#avoid#manipulation#would#be#to#update#our#heuristics#to#close#off#this#cognitive#backdoor.#
As#Reber#and#Unkelbach#showed,# we# are# capable# of#changing#and#updating#our# heuristics#
when# we# received# evidence#that# they# have# lead# us# astray.# The#manipulators,#then,# would#
want#to#mask#from#us#any#evidence#that#our#use#of#the#fluency#heuristic#was#leading#us#astray.#
10
!Trout!makes!a!similar!point!about!fluency!and!the!sense!of!understanding!(Trout!2017),!although!his!concern!
is!largely!with!attacking!other!accounts!of!understanding,!and!not!providing!a!full!picture!of!exploitation.!I!take!
myself!to!be!filling!in!the!details!of!his!suggestion.!
15
This#is,#however,#easier#to#do#in#some#domains#than#others.#Some#epistemic# domains#have#
obvious#litmus#tests.#It# is# easy#to#check#for#mistaken#reasoning#in#them#because# successes#
and#failures#are#obvious#to#any#onlooker.#For#example,#we#can#tell#that#our#theory#of#bridge-
building#has#gone#wrong#corrupted#if#our#new#bridges#keep#falling#down.#But#other#epistemic#
domains#have#no#such#easy#litmus#tests#—#like#the#moral#and#aesthetic#domains.#If#one’s#rea-
soning#has#been#systematically#subverted#in#such#a#subtle#domain,#there#is#no#obvious#error#
result#that#could#function#as#a#check.
11
#So#if#manipulators#wanted#to#gain#control#via#the#flu-
ency#heuristic,#one#good#strategy#would#be#to#perform#their#fluency-manipulations#over,#say,#
claims#about#morality# and# value.#Alternatively,#they#may#want#to#devote#their# fluency-ma-
nipulations#to#complex#and#diffuse#social#phenomena#or#more#esoteric#scientific#phenome-
non.#Some#empirical#claims#cannot#be#straightforwardly#checked#by#the#layperson,#such#as#
scientific#arguments#for#climate#change#or#sociological#claims#how#oppression#perpetuates.#
If#the#manipulators’#targets#have#been#given#a#seductively#clear#explanation#which#dismisses,#
say,#sociologists#and#climate#change#scientists#as#corrupt,#those#explanations#will#be# quite#
hard#to#dislodge.#Most# targets# will# be#unable#to#see#that#they#have#been# led# astray,#and#so#
won’t#update#their#heuristics#(Nguyen#2018b;#Nguyen#2018c).##
#
#
Aping(understanding(
#
Perhaps#it#seems#implausible#to#you#that#somebody#would#terminate#a#really#important#
inquiry#just#because#of#fluency.#There#is,#however,#another#much#more#sophisticated#form#of#
11
!For!an!extensive!discussion!of!litmus!tests!and!expert-vetting,!see!Nguyen!(2018a).!!
16
epistemic# seduction# which# will# more# plausibly# trigger# the# thought-terminating# function.#
Hostile#epistemic#manipulators# can#try#to#imitate,#not#just# ease,#but#a#full#.--%+(8#of#under-
standing.#They#can#present#the#phenomena#associated#with#a#positive#and#rich#experience#of#
clarity.###
In#order#to#see#how#one#might#fake# the#feeling#of#understanding,#let’s# start#by#thinking#
about#the#nature#of#genuine#understanding.#For#that,#let’s#turn#to#a#recent#discussion#of#the#
nature#of#understanding#in#the#philosophy#of#science.#According#to#a#recent#strand#of#think-
ing,#knowledge#isn’t#actually#the#primary#goal#of#much#of#our#epistemic#efforts.#Knowledge#is#
usually#conceived#of#as#something#like#the#possession#of#true#facts.#Having#knowledge,#by#the#
usual#accounts,#doesn’t#require#any#particular#integration#of#those#facts.#But#many#of#our#in-
tellectual# efforts# are# aimed# at# getting# something# more# than# just# knowing# some# disparate#
facts.#We#aim#at#something#more#holistic:#understanding.#The#precise#nature#of#understand-
ing#is#still#under#some#debate,#but#we#can#extract#some#common#and#largely#uncontroversial#
ideas.
12
#First,#when#we#understand#something,#we#not#only#possess#a#lot#of#independent#facts,#
but# we# see# how# those# facts# connect.# Understanding# is# of# a# system;# it# involves# grasping# a#
structure#and#not#just#independent#nodes.#Second,#when#we#understand#something,#we#pos-
sess#some#internal#model# or# account# of# it#which#we#can# use# to# make#predictions,#conduct#
12
!Much!of!the!debate!in!that!literature!has!turned!on!what!is!constitutive!of!understanding,!and!what!is!merely!
typically!associated!with!understanding.!For!example,!according!to!Steven!Grimm!and!Henk!de!Regt,!the!skill!
of!practical!application!is!partially!constitutive!of!understanding!(Grimm!2006;!de!Regt!2009;!Wilkenfeld!2013,!
2017).!Michael!Strevens,!on! the!other!hand,!denies!this! constitutive! relationship;! skill! typically! follows!from!
understanding,!but!isn’t!constitutive!of!it!(Strevens!2013).!Note!that!we!don’t!need!to!resolve!debates!like!this!
for!the!current!inquiry.!Since!we’re!interested!what!signs!are!associated!with!understanding,!we!don’t!really!
need!to!distinguish!carefully!between!what!is!constitutive!of!understanding,!and!what!follows!from!it.!Finally,!
Kareem!Khalifa!has!argument!that!these!accounts!of!understanding!can!be!reduced!to!the!idea!of!knowing!an!
explanation!(Khalifa!2012).!My!account!here!should!be!compatible!with!Khalifa’s!view!–!though,!in!his!language,!
I!would!be!talking!about!faking!the!feel!of!knowing!an!explanation.!!
17
further#investigations,#and#categorize#new#phenomena.
13
#
That#is#an#account#of#what#it#means#to#actually#0$?-&an#understanding.#So#what#are#the#
experiential#phenomena#$))"6+$*-5&@+*0#understanding?#What#does#it#.--%#like#to#understand#
something?#There#are#several#distinct#phenomena#to#consider#here.#First,#there#are#the#ex-
periences#associated#with#6"3+(8&*"&,(5-=)*$(5.#As#Catherine#Elgin#puts#it,#when#we#come#
to# understand,# our# way# of# looking# at# things# suddenly# shifts# to# accommodate# new# infor-
mation.#Understanding,#she#says,#“comes#not#through#passively#absorbing#new#information,#
but#through#incorporating#it#into#a#system#of#thought#that#is#not,#as#it#stands,#quite#ready#to#
receive#it”#(Elgin# 2002,#14).#When#we#come#to# understand,#our#system#of#thought#changes#
and# pieces# of# information# that# we# could# not# accommodate# before# suddenly# find# a# place.#
Kvanvig#offers#a#similar#account:#to#understand,#he#says,#is#to#grasp#a#coherence#relationship.#
It#is#to#be#aware#of#how#the#information#fits#together#(Kvanvig#2003,#202).#The#experience#of#
coming#to#understand,#then,#involves#an#experience#of#grasping#a#new#and#improved#coher-
ence.#Let#us#call#this#the#phenomenon#of#6"8(+*+?-&-1+10$(#.#And,#as#Gopnick#points#out,#cog-
nitive#epiphanies#are#incredibly#pleasurable.##
Next,#there#are#phenomena#associated#with#0$?+(8&$(&,(5-=)*$(5+(8.#Understanding#in-
volves#a#certain#facility#with#the#terrain.#As#Kvanvig#puts#it,#
…To!have!mastered!such!explanatory!relationships!is!valuable!not!only!because!it!involves!the!finding!
of!new!truths!but!also!because!finding!such!relationships!organizes!and!systematizes!our!thinking!on!
a!subject!matter!in!a!way!beyond!the!mere!addition!of!more!true!beliefs!or!even!justified!true!beliefs.!
Such!organization!is!pragmatically!useful!because!it!allows!us!to!reason!from!one!bit!of!information!to!
13
!This!discussion!constitutes! a! fast-growing! literature.! I! am! particularly! influenced! by! Catherine! Elgin’s! ac-
count,!Stephen!Grimm’s! useful! survey,!and! Michael!Strevens’!and!Michael!Patrick!Lynch’s!discussions!(Elgin!
2002,!2017;!Grimm!2012,!Strevens!2013;!Lynch!2018).!
18
another!related!information!that!is!useful!as!a!basis!for!action,!where!unorganized!thinking!provides!
no!such!basis!for!inference.!Moreover,!such!organized!elements!of!thought!provide!intrinsically!satis-
fying!closure! to! the!process!of!inquiry,! yielding!a!sense!or! feeling! of!completeness!to!our! grasp!of!a!
particular!subject!matter.!(202)!
When#we#understand#a#cognitive#terrain,#we#can#move#between#its#nodes#more#quickly#
and#easily.#We#can#use#our#understanding#to#easily#and#powerfully#generate#relevant#expla-
nations.#And#if#our#understanding#is#fecund,#these#new#explanations#will#serve#to#create#even#
more#useful#connections.# And,# as# Michael#Strevens#says,# having# an# understanding#also#in-
volves#having#the#capacity#to#communicate#that#understanding#—#to#explain#to#how#the#con-
nections#work#(Strevens#2013).#Let’s#call#all#these#the#phenomena#of#6"8(+*+?-&.$6+%+*#;
14
#And,#
at#least# in#my#own#experience,#the#pleasure#of#clarity#lies#not#only#in#Gopnick’s#moment#of#
coming#to# understand,# but#also# in# the#continuing# joys#of# apparent#facility# and# intellectual#
power.#It#feels# incredibly# good#to# be# able# to#swiftly# explain# complex#phenomena.# It# is# the#
pleasure#of#engaging#our#skills#and#capacities#to#powerful#effect.
15
#
Let’s#enter#into#the#mindset#of#the#hostile#epistemic#manipulator.#Our#goal#is# to#seduce#
with#apparent#clarity#—# to# game# other#people’s#cognitive#processes#and#heuristics#so#that#
they#will#accept#our#preferred#system#of#thought.#We’ll#want#to#engineer#that#system,#then,#
to#create#the#feeling#of#cognitive#epiphany.#We’ll#want#to#maximize,#for#our#system’s#adopters,#
the#sense#that#unexplained#information#is#sliding#into#place,#the#feeling#of#newfound#coher-
ency.#So#we’ll#want#to#give#the#system#easy-to-apply#categorizations#which#are#readily#con-
nected#into#a# coherent# network.# And,#once#that# system# has# been#adopted,#we’ll# want# it# to#
create# the# feeling# of# cognitive# facility.# We’ll# want# to# engineer# it# so# that,# once# somebody#
14
!I!owe!my!framing!to!Laura!Callahan’s!(2018,!442)!useful!discussion!of!understanding.!
15
!For!more!on!the!aesthetic!pleasure!of!one’s!own!skillful!action,!see!Nguyen!(2020,!101-120).!
19
adopts#the#system,#thinking#in#its#terrain#will#seem#distinctly#easier#and#more#effective#than#
before.#We’ll# want#it# to# give# adopters# a# heightened# sensation#of# forming#connections# and#
moving#easily#between#them.#We’ll#want#it#to# create#the#impression#of#explanatory# power,#
quickly#and#easily#explaining#any#new#phenomena#that#come#up.#And#we#would#want#to#do#
all#that#while#simultaneously#masking#its#epistemic#faults.##
This#might#seem#like#an#overwhelmingly#difficult#task#for#the#aspiring#manipulator.#We#
manipulators,#however,#have#some#very#significant#advantages.#First,#we#don’t#need#to#suc-
cessfully#imitate#understanding#all#the#way#down.#We#simply#need#for#our#system#to#trigger#
the# clarity# heuristic# early# enough,# before# its# adopters# stumble# across# any# of# the# flaws.# If#
you’re#building#a#Potemkin#village,#you#don’t#need#to#actually#build#any#actual#houses.#You#
just#need#to#build#the#facades#—#so#long#as#those#facades#convince#people#not#to#try#and#enter#
the#buildings.# We#manipulators,#then,#can#hide#our#system’s#weakness#and#inferior#perfor-
mance#behind#a#veil#of#apparent#clarity.
16
#
But#our#most#significant#advantage#is#that#we#are#unburdened#by#the#constraints#of#truth#
in#engineering#our#extra-tasty#system#of#thought.#Epistemically#sincere#systems#—#that#is,#
systems#of#thought#generated#for#the#sake#of#real#knowledge#and#genuine#understanding#—#
are#heavily#constrained#by#their#allegiance#to#getting#things#right.
17
#We#manipulators#are#un-
bound#by#any#such#obligations.#We#are#free#to#tweak#our#system#to#maximize#its#appealing#
clarity.#This#is#similar,#in#a#way,#to#how#unhealthy#restaurants#are#free#to#appeal#more#directly#
to#our#sense#of#deliciousness,#because#they#are#freed#from#considerations#of#health.#(Or,#at#
16
!This!strategy!exploits!a! cognitive! error! of!over-weighting!early!evidence.!For!a!discussion!of!why! this! is! a!
cognitive!error,!see!Kelly!(2008).!For!an!application!of!that!discussion!to!conspiracy!theories!and!echo!cham-
bers,!see!Nguyen!(2018b).!
17
!Elgin!(2017)!defends!the!use!of!idealizations!and!non-truths!as!parts!of!the!models!that!help!us!to!understand.!
However,!the!choice!of!models!is!still!driven!by!an!orientation!towards!getting!the!world!right,!in!a!more!holistic!
way.!
20
least,#that’s#how#my#mother#saw#it.)#We#manipulators,#then,#can#optimize#our#system#to#offer#
the#sense#of#easily#made#connection#and#explanations.#We#can#build#a#cartoon#of#understand-
ing.#And#that#cartoon#will#have#a#competitive#advantage#in#the#cognitive#marketplace.#It#can#
be#engineered#for#the#sake#of#pleasure,#and#it#will#carry#with#it#a#signal#that#inquiry#is#finished,#
and#that#we#should#look#elsewhere.##
#
(
Two(systems(of(cognitive(seduction(
#
Let’s#look#at#two#case#studies#of#the#seductions#of#clarity:#echo#chambers#and#institutional#
quantification.#The#first#case#study#of#echo#chambers#will#strike#many,#I#suspect,#as#a#plausi-
ble#and#familiar#case#of#the#seductions#of#clarity.#The#discussion#of#quantification#may#prove#
more#surprising.#And#I#hope#that#the#differences#between#these#two#case#studies#will#help#us#
to#hone#in#on#the#phenomenon’s#more#general#qualities.#
Let’s#start#with#echo#chambers.#Most#social#scientists#and#journalists#use#the#terms#“echo#
chamber”#and#“epistemic#bubble”#synonymously.#But,#as#I’ve#argued,#if#we#look#at#the#original#
sources#of#these#terms,#we#find#two#very#different#phenomena.#An#epistemic#bubble#is#a#so-
cial#phenomenon#of#simple#omission.#It’s#bad#connections#in#your#information#network#–#like#
if#all#your#friends#on#Facebook#share#your#politics,#and#you#simply#never#run#across#the#ar-
guments#presented#by#the#other#side.#An#echo#chamber,#on#the#other#hand,#is#a#social#struc-
ture#which#discredits#all#outsiders.#When#you#are#in#a#bubble,#you#don’t&0-$=&the#other#side.#
When#you’re#in#an#echo#chamber,#you#don’t#*=,)*#the#other#side.#Echo#chambers#don’t#cut#off#
21
lines#of# communication#from#the#outside#world;#rather,#they#isolate#their#members#by#ma-
nipulating#their#members’#trust#(Nguyen#2018b).#
What#matters#for#the#present#study#is#the#particular#content#of#the#systems#of#thought#
which#echo#chambers#use#to#manipulate#trust.#I’m#drawing#here#on#Kathleen#Jamieson#and#
Joseph#Cappella’s#empirical#analysis#of#the#echo#chamber#around#Rush#Limbaugh#and#the#Fox#
News#ecosystem#(Jamieson#and#Cappella#2010).#According#to#Jamieson#and#Cappella,#Rush#
Limbaugh#offers#a#world-view#with#some#very#distinctive#features.#First,#Limbaugh#presents#
a#world#of#sharply#divided#forces#locked#in#a#life-or-death#struggle.#There#are#no#onlookers#
or#reasonable#moderates.#Either#you’re#a#Limbaugh#follower#—#and#so#on#the#side#of#right#—#
or#you#are#one#of#the#malevolent#forces#out#to#undermine#the#side#of#right.#Limbaugh#then#
offers#an#explanatory#system#in#which#most#moral#and#political#action#can#be#understood#in#
terms#of#that#all-consuming#struggle.#Disagreement#with#Limbaugh’s#world#view#can#be#read-
ily#explained#as#the#product#of#some#organized,#malevolent#action#to#block#the#side#of#right.#
Most#importantly,#for#our#present#purposes,#the#undermining#function#and#the#explanatory#
function#are#often#accomplished#with#the#help#of#conspiracy#theories,#which#provide#a#ready#
explanation#for#disagreement#from#outsiders.#The#liberal#media#is#in#the#grip#of#a#nefarious#
network#of#elites,#as#are#universities,#and#the#academic#sciences.#These#conspiracy#theories#
offer#to#explain#complex#features#of#the#world#in#terms#of#a#single#coherent#narrative.#
This#is#an#obvious#deployment#of#the#seductions#of#clarity.#First,#Limbaugh’s#world-view#
offers#the#sensations#of#epiphany.#Once#his#world-view#is#accepted,#difficult-to-categorize#ac-
tions#suddenly#become#easily#categorized.#Previously#hard-to-explain#facts#—#like#the#exist-
ence#of#substantive#moral#disagreement#between#apparently#sincere#people#—#suddenly#be-
come#easily#explicable# in# terms#of#a#secret#war#between#good#and# evil.# Second,#the#world-
22
view# offers# the# sensations# of# cognitive# facility.# The# conspiracy# theory# offers# a# ready# and#
neatly#unified#explanation#for#all#sorts#of#behavior.#And#those#explanations#are#easy#to#create.#
The#world#suddenly# becomes# more#intellectually#manageable.# This# is#particularly#vivid# in#
some#of#communities#around#the#wilder#conspiracy#theories.#CNN#recently#conducted#some#
quite#telling# interviews#with# some# members# of# the# fast-growing# community# of# Flat# Earth#
conspiracy#theorists.#Many#theorists#describe#the#satisfactions#of#being#a#Flat#Earth#theorist#
in#in#terms# of#cognitive#facility.#As#Flat#Earth#theorist# and# filmmaker#Mark#Sargent#puts#it,#
"You#feel#like#you've#got#a#better#handle#on#life#and#the#universe.#It's#now#more#manageable.”#
And#Flat# Earth# theorist# David#Weiss# says,# “When#you# find# out#the# Earth# is# flat…#then# you#
become#empowered”#(Picheta#2019).##
Furthermore,#well-designed#echo# chambers# typically# have#systems#of# belief# which#can#
reinterpret#incoming#evidence#in#order#to#avoid#refutation.#For#example,#many#echo#cham-
bers# include# sweeping# scientific# claims,# such# as# denying# the# existence# of# climate# change.#
Echo#chamber#members#may#have#adopted#belief#systems#with#the#help#of#the#clarity#heuris-
tic.#But,#one#might#think,#heuristics#are#defeasible#—#and#contrary#scientific#evidence#should#
surely#bring#members#to#abandon#their#settled#acceptance#of#their#belief#system.#However,#a#
clever#echo#chamber#can#preemptively#defuse#such#contrary#evidence.#A#well-designed#echo#
chamber#can#include,#in#its#belief#system,#a#conspiracy#theory#about#how#the#media#and#the#
institutions#of#science#were#entirely#corrupt#and#in#the#grip#of#a#vast#malicious#conspiracy.#
This#explanation#performs#a#kind#of#intellectual#judo.#As#Endre#Begby#(2020)#points#out,#such#
a#belief#system#transforms#apparently#contrary#evidence#into#confirmations#of#the#belief#sys-
tem#—#a#process#which#he#calls#“evidential#pre-emption”.#If#Limbaugh#predicts#that#the#lib-
23
eral#media#will#accuse#him#of#falsifying#information,#then#when#his#followers#hear#such#accu-
sations#from#the#liberal#media,#they#will#have#reason#to#increase#their#trust#in#Limbaugh#—#
since#his#predictions#have#been#fulfilled!#But#notice#that#there#is#a#secondary#effect,#beyond#
the#simple#confirmation#Begby#describes#—#an#effect#that#arises#from#the#seductions#of#clar-
ity.#The#belief#system#makes#it#easy#to#create#an#explanation#for#incoming#contrary#evidence#
and#to#provide#explanations#that#unify#and#connect#that#event#with#many#others.#This#pro-
vides#an#experience#of#cognitive#facility#—#which#should#trigger#the#clarity#heuristic.#This#is#
an#extremely#well-designed#epistemic#trap,#in#which#contrary#evidence#triggers#two#different#
defense#mechanisms.#First,#the#conspiracy#theory#preemptively#predicts#the#presence#of#con-
trary#evidence,#and# so# confirms# itself#in# the# process# of#dismissing#that# contrary# evidence.#
Second,#the#ease#with#which#the#conspiracy#theory#performs#that#prediction#and#dismissal#is#
an#experience#of#cognitive#facility#—#which#creates#the#sense#of#clarity,#which,#in#turn,#trig-
gers#the#thought-terminating#heuristic.##
Such#defensive#conspiracy#theories# are# an# obvious#case#of#the#seductive,# manipulative#
use#of#clarity.#Let’s#now#turn#to#a#less#obvious#case.#Consider#the#appeal#of#quantified#systems.#
Consider,#especially,#the#way#in#which#large-scale#institutions#try#to#reduce#complex,#value-
laden#qualities#to#simple#metrics#and#measures.#In#4=,)*&+(&A,39-=)B&a#history#of#the#culture#
of#quantification,# Theodore#Porter#notes#that#quantified#systems#are#powerfully#attractive.#
This#is#why,#he#says,#politicians#and#bureaucrats#love#to#cite#the#authority#of#quantified#sys-
tems#of#analysis.#Numbers,#he#says,#smell#of#science.#They#have#the#ring#of#objectivity,#and#so#
they#will#be#used#in#inappropriate#circumstances#in#attempts#to#gain#political#control#(Porter#
1996,#8).#I#think#Porter#is#entirely#right#about#the#credibility#advantage#of#numbers#and#their#
scientific#feel#—#but#I#don’t#think#this#is#the#whole#story.#The#details#of#his#study#offer#us#the#
24
opportunity#to#build#a#second#account#of#the#appeal#of#numbers,#alongside#his#credibility#ac-
count,#in#terms#of#the#seductions#of#clarity.##
There#are,#says#Porter,# qualitative# ways# of# knowing#and#quantitative#ways#of#knowing.#
Porter#is#not#here#making#the#crude#claim#that#quantitative#ways#of#knowing#are#inherently#
bad.#Rather,# he# is# interested# in# the# relative# advantages#and# disadvantages# of# each# way# of#
knowing.#Qualitative#ways#of#knowing,#he#says,#are#typically#nuanced,#sensitive,#and#rich#in#
contextual#detail,#but#they#are#not#portable#or#aggregatable.#When#we#transition#to#from#qual-
itative#to#quantitative#ways#of#knowing,# we# strip# out# much#of#the#nuance#and#many#of#the#
contextual#details.# In# return# for# this# loss# of# informational# richness,#we# get# to# express# our#
knowledge# in# neat# packages:# in# the# form# of# numbers,# whose# meanings# are# portable,# and#
which#can#be#easily#aggregated#with#other#numerical#results.#This#can#be#very#valuable.#Ob-
viously,#quantification#is#vital#for#modern#science.#And#there#are#many#administrative#func-
tions#which#quantification#makes#far#more#efficient.#But,#says#Porter,#contemporary#culture#
seems#to#have#lost#sight#of#the#distinctive#value#of#qualitative#ways#of#knowing.#We#tend#to#
reach#for#quantitative#ways#of#knowing#compulsively,#even#when#they#aren’t#most#appropri-
ate#for#the#task#at#hand.#
In#40-&2-5,6*+"()&".&7,$(*+.+6$*+"(,#Merry#applies#Porter’s#analysis# to#the#recent#rise#of#
quantified# metrics# in# international# governance.# She# is# interested# in# +(5+6$*"=)# —# simple,#
quantified#representations#of#complex#global#phenomena.#One#indicator#is#the#UN’s#Human#
Development#Index,#which#gives#countries#a#single#score#for#their#performance#in#supporting#
the#quality#of#life#of#their#citizens.#Another#indicator#is#the#US#State#Department’s#4=$..+6>+(8&
+(&/-=)"()&!-1"=*),#which#gives#countries#a#score#on#their#performance#in#reducing#sex#traf-
25
ficking.#Indicators# present# themselves# in# the# form# of# a# single,# easy-to-use,# easy-to-under-
stand#numerical# score.# These#indicators,# she# says,#hide# the# complexity#and# subjectivity# of#
their#manufacture.# And#that# concealment# is# much# of# the# point.# # Their# power,# says# Merry,#
comes#in#significant#part#from#their#appearance#of#unambiguity.#And#once#these#indicators#
have#been#manufactured,#they#invariably#become#central#in#various#governments’#and#poli-
ticians’#decision-making# processes.# The#very# qualities# which# make# them# so# powerful#also#
make#them#blunt#instruments,#missing#in#much#subtlety#and#detail.#But,#says#Merry,#they#are#
incredibly#hard#to#dislodge#from#the#minds#of#the#public#and#of#policy-makers#(Merry#2016,#
1-43,#112-60).#
Why#are#quantifications#so#sticky?#The#seductions#of#clarity#offer#an#explanation.#Quanti-
fied#systems#are,#by#design,#highly#usable#and#easily#manipulable.#They#provide#a#powerful#
experience#of#cognitive#facility.#It#is#much#easier#to#5"&*0+(8)#with#grades#and#rubrics#than#it#
is#with#qualitative#descriptions.# We# can# offer#justifications#(“I#averaged#it#according#to#the#
syllabus’#directives”;#“I#applied#the#rubric”).#We#can#generate#graphs#and#quantified#summar-
ies.#And# the# sense# of# facility# is# even# stronger# in# large-scale# institutions,# where# the# use# of#
numbers#has# been#stringently# regularized.#Because# of#the# portability# of# numbers# and# the#
constancy# and# enforced# regularity# of# typical# institutional# deliberation# procedures,# inside#
such#institutions,#it#is#vastly#easier#to#use#numbers#to#produce#powerful#and#effective#com-
munications.#And#they#are#communications#in#terms#which#we#know#will#be#,(5-=)*""5#and#
$6*-5&,1"(&—#because#the#meanings#and#uses#of#these#institutional#terms#has#been#so#ag-
gressively#regularized.#
In#a# university#for#which#I# once#worked,#all#departments# had#to#produce#yearly# assess-
ment#data#which#was#supposed#to#demonstrate,#in#quantitative#form,#the#quality#of#education#
26
that#our#students#had#received.#Our#assessments#results#had#to#be#coded#according#to#certain#
institutionally# specified# Educational# Learning# Outcomes# (ELOs).# So,# the# fact# that# our# stu-
dents#scored#well#this#year#in#their#critical#thinking#multiple#choice#tests#gets#coded#and#en-
tered#into#the#system.#Those#scores#now#support#our#claim#that#a#particular#class#succeeds#
in#supporting#certain#university-wide#learning:#the#Critical#Thinking#ELO,#the#Writing#Skills#
ELO,#the#Moral#Reflection#ELO#and#the#Mathematical#Reasoning#ELO.#And#the#data#for#each#
particular#class,#in#turn,#is#used#to#support#the#claim#that#our#department#as#a#whole#supports#
the#university-wide#learning#outcomes.#And#that#claim,#in#turn,#is#used#as#support#the#claim#
that#the# University# is# succeeding# in# its# mission,# and# achieving# its# stated# Core# Values:# like#
Communication,#Community,#and#Engagement.# And# the# way#in#which# class,# departmental,#
and#university#ELO’s# link# up#are# coded# explicitly# into#our# databasing# system,#so# that# new#
data#can#travel#automatically#up#the#chain.#When#I#enter#the#latest#batch#of#scores#from#my#
students,#it#produces#an#immediate#effect#into#the#system:#all#the#reported#ELOs#up#the#chain#
will#change.#And#this#is#possible#precisely#because#the#data#I’ve#entered#has#been#rendered#
portable#and#because#our#outcomes#reporting#system#has#been#set#up#to#automatically#take#
advantage#of#that#portability.#
Notice#that#all#this#gives#me#the#experience#of#an#enormous#amount#of#apparently#effec-
tive#cognitive#and#communicative#activity.#I#have#a#sense#of#grasping#connections.#I#can#see#
exactly#how#my#class’s#ELOs#support#my#department’s#ELOs,#which#in#turn#support#my#col-
lege’s#ELOs,#which# in# turn#support#the# university’s# ELOs#and,#in# turn,# the#University#Core#
Values.#And#my#grasp# of# this#system#can#give#me#a#certain#sense# of#cognitive#facility.#I#can#
easily#generate#explanations#of#course#content# and# generate# evidence# of# teaching#success.#
And#I# can# know#that# they# will# be# understood,#since# they# have# been# expressed#in#the# pre-
27
prepared,#standardized,#and#explicitly#interconnected#language#of#the#institution.#I#know#that#
my#justifications#will#be#incorporated#into#larger#institutional#aggregates,#because#my#justi-
fications#occur#in#those#intentionally#stabilized#terms.#And#I#know#that#when#I#give#justifica-
tions#in#those#designated#terms,#they#will#usually#generate#pre-specified#sorts#of#actions#—#
ones#which#I#can#usually#predict#with#some#success.#A#stabilized,#explicit#system#of#quantified#
and#systemized#institutional#value#is#designed#so#that#its#users#can#make#themselves#easily#
understood#and#their#pronouncements#quickly#integrated#into#institutional#systems#of#infor-
mation# processing# and# decision-making.# In# short,# by# using# the# provided# terms# of# institu-
tional#discourse#inside#the#institution,#my#speech#and#thinking#will#seem#clear,#precisely#be-
cause#they#fit#so#well#into#a#pre-established#network#of#communication#and#justification.#That#
pre-engineered#fit#creates#a#sense#of#cognitive#facility,#with#all#its#associated#pleasures.#And#
the# ring# of# clarity# can#trigger# the# thought-terminating# heuristic# in# others# who# have# also#
bought#into#the#provided#system#of#institutional#discourse#–#ending#inquiry#into#the#appar-
ently#clear#claim.##
Of#course,#I’ll#have#genuine#cognitive#facility#if#my#various#mental#efforts#actually# track#
real#elements#in# the# world# and#process#them# in# some# epistemically#valuable# way.# And,# as#
Charles#Perrow#and#Paul#Du#Gay#have#argued,#bureaucracies#certainly#need#regular#methods#
and#quantified#systems#in#order#to#function#and#to#administrate#fairly#(Du#Gay#2000;#Perrow#
2014).#The#worry,# though,# is#that#we#might#set#up#systems#that#are# useful# for#certain#very#
specific#data-collection#and#managerial#function#—##but#that#can#also#exert#a#magnetic#pull#
on#our#thinking#in#nearby#domains.#For#example:# GPAs#and#citations#rates#might# be#useful#
for#certain#particular#tasks#of#bureaucratic#administration.#But,#because#they#are#so#seduc-
tive,#students# and# scholars# may# start#using# them#as# the# primary# lens# through#which# they#
28
evaluate#their#own#education#and#output.
18
#And#surely#GPAs#are#not#perfect#indicators#of#a#
good#education,#and#citations#rates#are#not#perfect#indicators#of#good#scholarship.#A#particu-
lar#quantification#can# get# an# excess#grip#on#our#reasoning,#even#in#contexts#when#it#is# less#
appropriate,#by# presenting# an# appealing# sense# of# clarity.# And# we# will# fail# to# investigate#
whether#this#quantified#metric#is#the#most#appropriate#form#of#evaluation#to#use,#precisely#
because#its#clarity#terminates#our#investigations#into#its#appropriateness.#
So#far,#we’ve#been#concentrating#on#systems#of#thought#whose#contents#themselves#are#
seductively#clear.#But#the#seductions#of#clarity#can#also#affect#our#judgments#of#the#expertise#
and#authority#of#the#sources#of#those#contents.#The#seductions#of#clarity#can#get#us#to#accept#
a#system#by#making#its#users#and#authors#seem#more#credible#or#expert,#precisely#because#
they#seem#more#clear.#Recall#that#one#of#standard#signals#of#expertise#is#communicative#fa-
cility.# Non-experts# trust# purported# experts# when# those# experts# are# able# to# communicate#
their#understanding#—#when#the#purported#experts#can#explain#to#their#audiences#the#con-
nections#between#nodes,#generate#justifications,#and#the#like.#But#consider#what#happens#to#
the#appearance#of#communicative#facility#inside#a#bureaucratized#system#of#educational#as-
sessment.#Those#users#willing#to#express#themselves#in#the#designated#terms#of#that#system#
have#a#considerable#advantage#in#displaying#communicative#facility.#They#can#easily#generate#
justifications.#They#can#easily#make#their#reasons#and#requests#understood#and#acted#upon#
in#institutional#settings.#They# will# seem# clear# because#their#communication#will#be#readily#
taken#up#and#acted#upon.#Their#apparent#facility#will#seem#especially#impressive#to#outsiders,#
who#are#out#of#contact#with#the#subtler#values#involved#with#education.#This#is,#obviously,#a#
18
!I!offer!a!fuller!discussion!of!how!simplified!and!quantified!systems!of!value!can!give!their!adopters!the!game-
like!pleasures!of!value!clarity!in!Nguyen!(2020).!!
29
form#of#epistemic#injustice#(Fricker#2007).#Here,#it#is#a#form#of#epistemic#injustice#which#gives#
a#significant#credibility#advantage#to#anybody#willing#to#speak#in#the#terms#provided#by#bu-
reaucracies# and# institutions,# which# provide# regularized# systems# of# justification# and# lan-
guages#of#evaluation.#And#since#the#ability#to#create#and#disseminate#such#systems#is#usually#
held#by#those#already#in#power,#the#bureaucratization#of#language#will#typically#serve#to#am-
plify#power#differentials#by#granting#more#credibility#to#those#who#accept#those#bureaucratic#
terms#of#discourse.#
To#put#it#in#Kristie#Dotson’s#terms,#epistemic#oppression#occurs#when#agents#are#denied#
the#opportunity#to#use#shared#epistemic#resources#to#participate#in# knowledge#production#
(Dotson#2014).#Bureaucratic#and#institutionalized#language#can#enable#a#particular#kind#of#
epistemic#oppression.# Ideas# that# can# be#easily# expressed#in# the# institutional# language# are#
readily#entered#into#the#shared#knowledge#base.#But#the#standardization#of#language#puts#a#
special# oppressive# power# in# the# hands# of# whoever# creates# the#standardization.# Once# the#
standardization#is#in#place#and#widely#accepted,#anybody#who#uses#it#will#5-3"()*=$*-&6"8:
(+*+?-&.$6+%+*##and#5-3"()*=$*-&6"33,(+6$*+?-&.$6+%+*#;&They#will#)--3&6%-$=#precisely#because#
they#are#using#language#for#which#a#system#of#reception#has#been#pre-prepared.##
The#sense#of#clarity#is#a#terminator#for#inquiry,#and#ideas#expressed#in#that#regularized#
institutional#language#will#bear#that#sense#of#clarity.#So#ideas#expressed#in#that#language#are#
more#likely#to#be#accepted#without#question.#Information#that#isn’t#placed#into#institutional#
language,#on#the#other#hand,#will#tend#to#be#disappear.#Such#recalcitrant#expressions#will#be#
less#likely#to#be#accepted,#transmitted#and#remembered#within#the#system.#At#the#very#least,#
since#they#seem#confusing#rather#than#clear,#those#recalcitrant#expressions#will#be#subject#to#
constant#questioning#and#inquiry,# rather# than# quickly#accepted.#In#a# standardized# system,#
30
non-standardized#information# will# be#subject# to# incredible# friction.# This# creates# a# further#
competitive# disadvantage.# By# the# very# fact# that# such# information# transmits# slowly# and#
poorly,#the#information#and#its#authors#will#seem#to#have#less#communicative#facility#and#so#
seem#less#credible.#Those#whose#ideas#don’t#fit#comfortably#into#the#regularized#institutional#
language#are#at#a#significant#disadvantage#in#participating#in#the#production#and#dissemina-
tion#of#knowledge.##
#
#
Nuance(and(closure(
(
The#point#here#is#not#to#claim#that#quantified#systems#and#conspiracy#theories#are#always#
bad.#Science#and#bureaucracy#need#quantification,#and#we#certainly#should#accept#conspiracy#
theories#when#there#are#actually#conspiracies.
19
#The#point#is,#rather,#that#these#sorts#of#ideas#
and#methodologies#are#among#the#choicest#tools#for#epistemic#subversion.#A#ruthless#epis-
temic# manipulator,# freed# from# the# constraints# of# genuine# inquiry,# can# re-formulate# these#
sorts#of#systems#to#maximize#their#potential#for#seductiveness.#
And#this#also#offers#us#insights#into#unintentional#cognitive#seduction.#Bureaucracies#and#
institutions#have#very#good#reason#to#develop#internally#consistent#and#quantified#systems#
of#evaluation.#Such#systems#make#the#administration#of#complex#organizations#possible.#But#
insofar#as#such#systems#share#a#significant#number#of#the#traits#and#effects#as#those#systems#
19
!There!is!a! very! useful!discussion!of!the!occasional!usefulness! of! conspiracy! theories!in!(Coady!2012,!110-
137;!Dentith!2018,!2019).!
31
made#for#intentional#manipulation#—#and#especially#insofar#as#such#systems#perpetuate#be-
cause#of#their#seductive#effects#—#then#such#systems#also#function#as#seductively#clear.#
This#suggests#another#reason#to#resist#the#seductions#of#clarity.#Sometimes,#we#need#to#
dwell#in#unclear#systems#of#thought#because#we#have#not#yet#earned#the#right#to#clarity.#In#
her#study#of#metaphors,#Elizabeth#Camp#(2006)#suggests#that#metaphors#are#most#appropri-
ate#when#we#are#still#in#the#process#of#coming#to#understand.#Metaphors#are#unclear#by#de-
sign.#They#are,# says# Camp,#a#special#way#of#pointing#to# the# world.#We#define#simple#nouns#
through#simpler# forms# of#pointing.# “Red”# we#define# as# looking# like# *0$*.#Metaphors# let# us#
point#with#a#rough,#waving#gesture.#
The#reason#we#might#want#to#do#so,# says#Camp,#is#that#such#pointing# lets#us#access#the#
richness#of#the#world#in#our#talk.#When#I#say,#“I#don’t#understand#what’s#going#on#with#Robert#
very#much,#but#his#neurosis#seems#a#lot#like#Liza’s,”#I’m#not#using#some#well-defined#abstract#
predicate#to#describe#Robert.#I#am#pointing#to#Liza#and#to#all#the#rich#features#of#reality#that#
are#bound#up#with#her.#I#am#saying#that#I#don’t#know#what#it#is#about#Liza#that#matters,#ex-
actly,#but#it’s#something#over#there,#where#“there”#is#a#gesture#in#the#direction#of#all#the#rich-
ness#of#Liza’s#actual#self.#And#this#sort#of#vague#gesture#is#especially#useful,#says#Camp,#when#
we#are#trying#to#grapple#with#things#we#do#not#yet#adequately#understand.#With#metaphors,#
she#says,#we#are#gesturing#vaguely#at#part#the#worlds.#
Intentionally#and#openly#vague#forms#of#communication#are#very#important.#They#remind#
us#that#our#thinking#—#our#concepts,#our#inquiries,#our#understanding#—#is#not#yet#finished.#
Clarity#is#compelling,#but#signals#us#to#end#our#inquiries.#Seductively#clear#systems#mask#the#
fact#that#we#should,#in#fact,#be#confused,#and#should#be#pressing#on#with#our#inquiries.#They#
32
present#themselves#as#finalized.#On#the#other#hand,#metaphors#and#their#kin#wear#their#un-
finishedness#plainly#on#their#faces.#They#are#hard#to#use,#and#that#difficulty#reminds#us#that#
there#is#more#work#to#be#done.#They#leave#the#basement#door#open,#so#we#know#there#is#more#
to#explore#down#there.#When#clarity#seduces,#it#can#prevent#us#from#pushing#on,#from#finding#
and#dwelling#on#our#confusions.#Seductive#clarity#presents#us#with#a#false#floor#for#our#inves-
tigations#into#the#world.#
How#do#we#resist#the#seductions#of#clarity?#One#possible#defensive#strategy#is#to#develop#
new#counter-heuristics,#designed#to#sniff#out#the#seductive#manipulation#of#our#original#heu-
ristics.#Here’s#a#rough#analogy:#a#certain#kind#of#culinary#yumminess#was#once#a#decent#heu-
ristic#for#nutritious#eating.#But#our#nutritive#environment#changed,#especially#when#various#
corporate#forces#figured#out#our#heuristics#and#tendencies#and#started#to#aggressively#game#
them.#In# response,#we#have#had#to#adapt#our#heuristics.#We#have#needed#to#become#suspi-
cious#of# *""& 3,60& #,33+(-)).# Many# of# us# have# #already# trained# ourselves# to# notice# when#
things#are#just#a#little#too#delicious.#The#crunchy,#sweet,#salty#stuff#that#hits#us#just#so#—#we#
have#learned#to#taste#in#them#the#engineer’s#manipulative#touch.#We#have#developed#an#in-
tuitive#feel#for#designed#craveability.#This#is#a# counter-heuristic,# designed# to# trigger#in#re-
sponse#to#signals#that#outside#forces#are#trying#to#manipulate#our#more#primitive#heuristics.#
Sweetness,#crunchiness,#saltiness#—#our#counter-heuristic#makes#as#immediately#suspicious#
when#we#find#these#in#plenty.#
In#fighting#the#seductions#of#clarity,#we#need#to#develop#new#counter-heuristics#in#a#sim-
ilar#key.#The#sense#of#clarity#is#something#like#cognitive#sugar.#Once#upon#a#time,#using#our#
sense#of#clarity#as#a#signal#to#terminate#our#inquiries#might#have#been#a#good#and#useful#heu-
ristic.#But# now# we# live# in#an# environment# where# we# are#surrounded# by# seductive# clarity,#
33
much#of#it#designed#to#exploit#our#heuristics.#We#now#need#to#train#ourselves#to#become#sus-
picious#of#ideas#and#systems#that#go#down#just#a#little#too#sweetly#—#that#are#pleasurable#and#
effortless#and# explain# everything# so# wonderfully.# Systems# of# thought# that# feel# too# clear#
should#make#us#step#up#our#investigative#efforts#instead#of#ending#them.#We#need#to#learn#to#
recognize,#by#feel,#the#seductions#of#clarity.
20
#
#
#
#
Bibliography,
(
Battaly,#Heather.#2018.#“Closed-Mindedness#and#Dogmatism.”#C1+)*-3-#15#(3):#261282.#
Begby,# Endre.# 2020.# “Evidential# Preemption.”# /0+%")"10#& $(5& /0-("3-("%"8+6$%& !-)-$=60;#
https://doi.org/10.1111/phpr.12654#
Callahan,#Laura#Frances.#2018.#“Moral#Testimony:#A#Re-Conceived#Understanding#Explana-
tion.”#/0+%")"10+6$%&7,$=*-=%##68#(272):#437459.#
Camp,#Elisabeth.# 2006.#“Metaphor#and#That# Certain#‘Je#Ne#Sais# Quoi.’”#/0+%")"10+6$%&2*,5+-)#
129#(1):#125.#
Cassam,#Quassim.#2016.#“Vice#Epistemology.”#40-&D"(+)*#99#(2):#159180.#
Chabris,#Christopher,#and#Daniel#Simons.#2011.#40-&'(?+)+9%-&E"=+%%$F&G"@&H,=&'(*,+*+"()&I-:
6-+?-&J).#Reprint#edition.#Harmony.#
20
!I’d!like!to!thank,!for!all!their!help!with!this!paper,!Andrew!Buskell,!Josh!DiPaolo,!A.W.!Eaton,!Caitlin!Dolan,!Jon!
Ellis,!Melinda!Fagan,!Keren!Gorodeisky,!Arata! Hamakawa,!Rob! Hopkins,!Jenny!Judge,!Samantha!Matherne,!Jay!
Miller,!Stephanie!Patridge,!Antonia!Peacocke,!Geoff!Pynn,!Nick!Riggle,!David!Spurrett,!Madelaine!Ransom,!Jonah!
Schupbach,!Tim!Sundell,!and!Matt!Strohl.!!!
34
Coady,#David.#2012.#K0$*&*"&L-%+-?-&A"@F&M11%#+(8&C1+)*-3"%"8#&*"&N"(*-31"=$=#&')),-).#Ho-
boken:#Wiley-Blackwell.#
Dallmann,#Justin.#2017.#“When#Obstinacy#Is#a#Better#Policy.”#/0+%")"10-=)O&'31=+(*#17.#
Dentith,#Matthew#R.#X.#2018.#“The#Problem#of#Conspiracism.”#M=8,3-(*$#3#(2):#327343.#
Dotson,#Kristie.#2014.#“Conceptualizing#Epistemic#Oppression.”# 2"6+$%& C1+)*-3"%"8##28#(2):#
11538.#
Eaton,#A.#W.#2020.#“Artifacts#and#Their#Functions.”#In#H<."=5&G$(59"">&".&G+)*"=#&$(5&D$*-=+$%&
N,%*,=-.# Edited# by# Ivan# Gaskell# and# Sarah# Anne# Carter.# Oxford:# Oxford# University#
Press.#
Elgin,#Catherine#Z.#2002.#“Creation#as#Reconfiguration:#Art#in#the#Advancement#of#Science.”#
'(*-=($*+"($%&2*,5+-)&+(&*0-&/0+%")"10#&".&26+-(6-#16#(1):#1325.#
Espeland,#Wendy#Nelson#and#Michael#Sauder.#2016.#C(8+(-)&".&M(<+-*#F&M6$5-3+6&!$(>+(8)B&
!-1,*$*+"(B&$(5&M66",(*$9+%+*#.#New#York:#Russell#Sage#Foundation.#
Fricker,#Miranda.#2007.#C1+)*-3+6&'(P,)*+6-F&/"@-=&$(5&*0-&C*0+6)&".&Q("@+(8.#Clarendon#Press.#
Gay,#Dr#Paul#du.#2000.#'(&/=$+)-&".&L,=-$,6=$6#F&K-9-=&:&H=8$(+R$*+"(&:&C*0+6).#London:#SAGE#
Publications#Ltd.#
Gigerenzer,#Gerd,#and#Daniel#G.#Goldstein.#1996.#“Reasoning#the#Fast#and#Frugal#Way:#Models#
of#Bounded#Rationality.”#/)#60"%"8+6$%&!-?+-@#103#(4):#65069.#
Gopnik,#Alison.#1998.#“Explanation#as#Orgasm.”#D+(5)&$(5&D$60+(-)#8#(1):#101118.#
Grimm,#Stephen#R.#2006.#“Is#Understanding#a#Species#of#Knowledge?”#L=+*+)0&S",=($%&."=&*0-&
/0+%")"10#&".&26+-(6-#57#(3):#515535.#
———.#2012.#“The#Value#of#Understanding.”#/0+%")"10#&N"31$))#7#(2):#103117.#
Jamieson,#Kathleen#Hall,#and#Joseph#Cappella.#2010.#C60"&N0$39-=F&!,)0&T+39$,80&$(5&*0-&
35
N"()-=?$*+?-&D-5+$&C)*$9%+)03-(*.#Oxford:#Oxford#University#Press.#
Kahan,#Dan#M.,#and#Donald#Braman.#2006.#“Cultural#Cognition#and#Public#Policy.”#U$%-&T$@&V&
/"%+6#&!-?+-@#24:#14770.#
Kahneman,#Daniel.#2013.#40+(>+(8B&W$)*&$(5&2%"@.#1st#edition.#New#York:#Farrar,#Straus#and#
Giroux.#
Kelly,#Thomas.#2008.#“Disagreement,#Dogmatism,#and#Belief#Polarization.”#S",=($%&".&/0+%")":
10##105#(10):#611633.#
Khalifa,#Kareem.#2012.#“Inaugurating#Understanding#or#Repackaging#Explanation?”#/0+%")":
10#&".&26+-(6-#79#(1):#1537.#
Kva nvig, # Jo na than # L. # 20 03 .# 40-&X$%,-&".&Q("@%-58-&$(5& *0-& /,=),+*&".&J(5-=)*$(5+(8.#Cam-
bridge:#Cambridge#University#Press.#
———.#2011.#“IIJonathan#L.#Kvanvig:#Millar#on#the#Value#of#Knowledge.”#M=+)*"*-%+$(&2"6+:
-*#&2,11%-3-(*$=#&X"%,3-#85#(1):#8399.#
Lupton,#Deborah.#2016.#40-&7,$(*+.+-5&2-%..#Cambridge:#Polity.#
Lynch,#Michael.# 2018.#“Understanding# and# Coming#to# Understand.”#In#D$>+(8&2-()-& ".& *0-&
K"=%5F&A-@&C))$#)&"(&*0-&/0+%")"10#&".&J(5-=)*$(5+(8,#edited#by#Stephen#Grimm,#194–
208.#Oxford:#Oxford#University#Press.#
McGonigal,# Jane.# 2011.# !-$%+*#& ')& L=">-(F& K0#& E$3-)& D$>-& J)& L-**-=& $(5& G"@& 40-#& N$(&
N0$(8-&*0-&K"=%5.#New#York:#Penguin#Books.#
Medina,#Jose.#2012.#40-&C1+)*-3"%"8#&".&!-)+)*$(6-F&E-(5-=&$(5&!$6+$%&H11=-))+"(B&C1+)*-3+6&
'(P,)*+6-B&$(5&!-)+)*$(*&'3$8+($*+"().#Oxford,#New#York:#Oxford#University#Press.#
Merry,#Sally#Engle.#2016.#40-&2-5,6*+"()&".&7,$(*+.+6$*+"(F&D-$),=+(8&G,3$(&!+80*)B&E-(5-=&
X+"%-(6-B&$(5&2-<&4=$..+6>+(8.#Chicago:#University#of#Chicago#Press.#
36
Millar,#Alan.#2011.#“Why#Knowledge#Matters.”#M=+)*"*-%+$(&2"6+-*#&2,11%-3-(*$=#&X"%,3-#85#
(1):#6381.#
Millgram,#Elijah.#1997.#/=$6*+6$%&'(5,6*+"(.#Cambridge,#Mass.:#Harvard#University#Press.#
———.#2004.#“On#Being#Bored#Out#of#Your#Mind.”#/="6--5+(8)&".&*0-&M=+)*"*-%+$(&2"6+-*##104#
(2):#163184.#
Millikan,#Ruth#Garrett.#1984.#“Language,#Thought#and#Other#Biological#Categories:#New#Foun-
dations#for#Realism.”#/0+%")"10#&".&26+-(6-#52#(3):#477478.#
Nguyen,#C.#Thi.#2018a.#“Cognitive#Islands#and#Runaway#Echo#Chambers:#Problems#for#Epis-
temic# Dependence# on# Experts.”# 2#(*0-)-.# https://doi.org/10.1007/s11229-018-
1692-0.#
———.# 2018b.# “Echo# Chambers# and# Epistemic# Bubbles.”# C1+)*-3-.#
https://doi.org/10.1017/epi.2018.32#
———.#2018c.#“Expertise#and#the#Fragmentation#of#Intellectual#Autonomy.”#/0+%")"10+6$%&'(:
Y,+=+-)&6#(2):#107-124.#
———.#2020.#E$3-)F&M8-(6#&$)&M=*.#New#York:#Oxford#University#Press.#
———.#Forthcoming.#“How#Twitter#gamifies#communication.”.#M11%+-5&C1+)*-3"%"8#,#ed.#Jen-
nifer#Lackey.#New#York:#Oxford#University#Press.#
Oppenheimer,# Daniel# M.# 2008.# “The# Secret# Life#of#Fluency.”#4=-(5)&+(&N"8(+*+?-&26+-(6-)#12#
(6):#23741.#
Perrow,# Cha rles.# 2014.# N"3 1%-<& H=8$(+R$*+"()F&M& N=+*+6$%& C))$#.#Third# edition.# Brattleboro,#
Vermont:#Echo#Point#Books#&#Media.#
Porter,#Theodore.#1996.#4=,)*&+(&A,39-=).#Princeton:#Princeton#University#Press.#
Proctor,#Robert,#and#Londa#L.#Schiebinger.# 2008.# M8("*"%"8#F&40-&D$>+(8&$(5&J(3$>+(8&".&
37
'8("=$(6-.#Stanford#University#Press.#
Reber,#Rolf,#and#Christian#Unkelbach.#2010.#“The#Epistemic#Status#of#Processing#Fluency#as#
Source#for#Judgments#of#Truth.”#!-?+-@&".&/0+%")"10#&$(5&/)#60"%"8##1#(4):#563581.#
Regt,#Henk#W.#de.#2009.#“The#Epistemic#Value#of#Understanding.”#/0+%")"10#&".&26+-(6-#76#(5):#
585597.#
Rini,#Regina.#2017.#“Fake#News#and#Partisan#Epistemology.”#Q-((-5#&'()*+*,*-&".&C*0+6)&S",=:
($%#27#(S2):#4364.#h#
Scott,#James#C.#1998.#2--+(8&T+>-&$&2*$*-F&G"@&N-=*$+(&260-3-)&*"&'31="?-&*0-&G,3$(&N"(5+:
*+"(&G$?-&W$+%-5.#New#Haven:#Yale#University#Press.#
Simon,#Herbert.#1956.#“Rational#Choice#and#the#Structure#of#the#Environment.”#/)#60"%"8+6$%&
!-?+-@#63#(2):#12938.#
Stanley,#Jason.#2016.#G"@&/="1$8$(5$&K"=>).#Princeton:#Princeton#University#Press.#
Strevens,#Michael.#2013.#“No#Understanding#Without#Explanation.”#2*,5+-)&+(&G+)*"=#&$(5&/0+:
%")"10#&".&26+-(6-&/$=*&M#44#(3):#510515.#
Sullivan,# Shannon,# and# Nancy# Tuana.# 2007.# !$6-& $(5& C1+)*-3"%"8+-)& ".& '8("=$(6-.# SUNY#
Press.#
Sunstein,#Cass.#2017.#Z!-1,9%+6.#Princeton:#Princeton#University#Press.#
Trout,#J.#D.#2002.#“Scientific#Explanation# and#the#Sense#of#Understanding.”#/0+%")"10#&".&26+:
-(6-#69#(2):#212233.#
———.#2017.#“Understanding#and#Fluency”.#In#D$>+(8&2-()-&".&*0-&K"=%5F&A-@&C))$#)&"(&*0-&
/0+%")"10#& ".& J(5-=)*$(5+(8.# Edited# by# Stephen# Grimm.# Oxford:# Oxford# University#
Press.#
Wilkenfeld,#Daniel#A.#2013.#“Understanding#as#Representation#Manipulability.”#2#(*0-)-#190#
38
(6):#9971016.#
———.#2017.#“MUDdy#Understanding.”#2#(*0-)-#194#(4):#12731293.#
Wimsatt,#William#C.#2007.#!-:C(8+(--=+(8&/0+%")"10#&."=&T+3+*-5&L-+(8)F&/+-6-@+)-&M11="<+:
3$*+"()&*"&!-$%+*#.#Cambridge,#Mass:#Harvard#University#Press.#
Zagzebski,#Linda.#2001.#“Recovering#Understanding.”#In#Q("@%-58-B&4=,*0B&$(5&I,*#F&C))$#)&
"(&C1+)*-3+6&S,)*+.+6$*+"(B&!-)1"()+9+%+*#B&$(5&X+=*,-,#edited#by#M.#Steup.#Oxford:#Oxford#
University#Press.#
Zimmerman,#Eric,#Ian# Bogost,# Conor#Linehan,#Ben#Kirman,#Bryan#Roche,#Mark#Pesce,#Scott#
Rigby,#et#al.#2015.#40-&E$3-.,%&K"=%5F&M11="$60-)B&')),-)B&M11%+6$*+"().#Edited#by#Stef-
fen#P.#Walz#and#Sebastian#Deterding.#Cambridge,#Mass.:#The#MIT#Press.#
#
#
#
#
#
... Fernandes, 2021;Guttman & Lev, 2021;Krause et al., 2020;Lu et al., 2021). Interventions that help to counter the natural tendency to prefer simple, clear explanations and messages of unwarranted certainty (Nguyen, 2021b) appear critical during a pandemic. ...
Article
Pandemics are characterized by extreme uncertainty and effective communication is critical to help the public manage this uncertainty. We summarized scholarship on public communication related to uncertainty during the COVID-19 pandemic, synthesizing insights published from January 2020–February 2022. We reviewed key findings and arguments from 39 empirical papers and 21 nonempirical papers with a particular focus on the theories and concepts that underpinned them. While this work shed light on elements of effective communication, conceptual and operational treatments of uncertainty varied considerably, and less than half of papers used a theory to analyze or discuss uncertainty-related communication. In all, this review highlights opportunities to strengthen our field’s theoretical and practical contributions to public communication in uncertain situations.
... public (Nguyen, 2021). For example, Tetlock (2005) and Tetlock and Gardner (2016) argues that "hedgehog experts" who stick to a handful of Big Ideas that allow them to explain everything are more likely to be favored by the media, even if their predictions tend to be worse. ...
Article
Full-text available
Some people publicly pretend to be experts while not being ones. They are pseudoexperts, and their presence seems to be ubiquitous in the current cultural landscape. This manuscript explores the nature and mechanisms of pseudoexpertise. We first provide a conceptual analysis of pseudoexperts based on prototypical cases of pseudoexpertise and recent philosophical work on the concept of expertise. This allows us to propose a definition that captures real-world cases of pseudoexpertise, distinguishes it from related but different concepts such as pseudoscience, and highlights what is wrong with pseudoexpertise. Next, based on this conceptual analysis, we propose a framework for further research on pseudoexpertise, built on relevant empirical and theoretical approaches to cultural cognition. We provide exploratory answers to three questions: why is there pseudoexpertise at all; how can pseudoexperts be successful despite not being experts; and what becomes of pseudoexperts in the long run. Together, these conceptual and theoretical approaches to pseudoexpertise draw a preliminary framework from which to approach the very troubling problem posed by persons usurping the capacities and reputations of genuine experts.
Article
Many explanations have a distinctive, positive phenomenology: receiving or generating these explanations feels satisfying. Accordingly, we might expect this feeling of explanatory satisfaction to reinforce and motivate inquiry. Across five studies, we investigate how explanatory satisfaction plays this role: by motivating and reinforcing inquiry quite generally (“brute motivation” account), or by selectively guiding inquiry to support useful learning about the target of explanation (“aligned motivation” account). In Studies 1–2, we find that satisfaction with an explanation is related to several measures of perceived useful learning, and that greater satisfaction in turn predicts stronger curiosity about questions related to the explanation. However, in Studies 2–4, we find only tenuous evidence that satisfaction is related to actual learning, measured objectively through multiple-choice or free recall tests. In Study 4, we additionally show that perceptions of learning fully explain one seemingly specious feature of explanatory preferences studied in prior research: the preference for uninformative “reductive” explanations. Finally, in Study 5, we find that perceived learning is (at least in part) causally responsible for feelings of satisfaction. Together, these results point to what we call the “imperfectly aligned motivation” account: explanatory satisfaction selectively motivates inquiry towards learning explanatory information, but primarily through fallible perceptions of learning. Thus, satisfaction is likely to guide individuals towards lines of inquiry that support perceptions of learning, whether or not individuals actually are learning.
Article
Full-text available
In her BBC Reith Lectures on Trust, Onora O’Neill offers a short, but biting, criticism of transparency. People think that trust and transparency go together but in reality, says O'Neill, they are deeply opposed. Transparency forces people to conceal their actual reasons for action and invent different ones for public consumption. Transparency forces deception. I work out the details of her argument and worsen her conclusion. I focus on public transparency – that is, transparency to the public over expert domains. I offer two versions of the criticism. First, the epistemic intrusion argument: The drive to transparency forces experts to explain their reasoning to non‐experts. But expert reasons are, by their nature, often inaccessible to non‐experts. So the demand for transparency can pressure experts to act only in those ways for which they can offer public justification. Second, the intimate reasons argument: In many cases of practical deliberation, the relevant reasons are intimate to a community and not easily explicable to those who lack a particular shared background. The demand for transparency, then, pressures community members to abandon the special understanding and sensitivity that arises from their particular experiences. Transparency, it turns out, is a form of surveillance. By forcing reasoning into the explicit and public sphere, transparency roots out corruption — but it also inhibits the full application of expert skill, sensitivity, and subtle shared understandings. The difficulty here arises from the basic fact that human knowledge vastly outstrips any individual’s capacities. We all depend on experts, which makes us vulnerable to their biases and corruption. But if we try to wholly secure our trust — if we leash groups of experts to pursuing only the goals and taking only the actions that can be justified to the non‐expert public — then we will undermine their expertise. We need both trust and transparency, but they are in essential tension. This is a deep practical dilemma; it admits of no neat resolution, but only painful compromise.
Book
Full-text available
Games occupy a unique and valuable place in our lives. Game designers do not simply create worlds; they design temporary selves. Game designers set what our motivations are in the game and what our abilities will be. Thus: games are the art form of agency. By working in the artistic medium of agency, games can offer a distinctive aesthetic value. They support aesthetic experiences of deciding and doing. And the fact that we play games shows something remarkable about us. Our agency is more fluid than we might have thought. In playing a game, we take on temporary ends; we submerge ourselves temporarily in an alternate agency. Games turn out to be a vessel for communicating different modes of agency, for writing them down and storing them. Games create an archive of agencies. And playing games is how we familiarize ourselves with different modes of agency, which helps us develop our capacity to fluidly change our own style of agency.
Article
Full-text available
In The Great Endarkenment, Elijah Millgram argues that the hyper-specialization of expert domains has led to an intellectual crisis. Each field of human knowledge has its own specialized jargon, knowledge, and form of reasoning, and each is mutually incomprehensible to the next. Furthermore, says Millgram, modern scientific practical arguments are draped across many fields. Thus, there is no person in a position to assess the success of such a practical argument for themselves. This arrangement virtually guarantees that mistakes will accrue whenever we engage in cross-field practical reasoning. Furthermore, Millgram argues, hyper-specialization makes intellectual autonomy extremely difficult. Our only hope is to provide better translations between the fields, in order to achieve intellectual transparency. I argue against Millgram's pessimistic conclusion about intellectual autonomy, and against his suggested solution of translation. Instead, I take his analysis to reveal that there are actually several very distinct forms intellectual autonomy that are significantly in tension. One familiar kind is direct autonomy, where we seek to understand arguments and reasons for ourselves. Another kind is delegational autonomy, where we seek to find others to invest with our intellectual trust when we cannot understand. A third is management autonomy, where we seek to encapsulate fields, in order to manage their overall structure and connectivity. Intellectual transparency will help us achieve direct autonomy, but many intellectual circumstances require that we exercise delegational and management autonomy. However, these latter forms of autonomy require us to give up on transparency.
Article
Full-text available
Discussion of the phenomena of post-truth and fake news often implicates the closed epistemic networks of social media. The recent conversation has, however, blurred two distinct social epistemic phenomena. An epistemic bubble is a social epistemic structure in which other relevant voices have been left out, perhaps accidentally. An echo chamber is a social epistemic structure from which other relevant voices have been actively excluded and discredited. Members of epistemic bubbles lack exposure to relevant information and arguments. Members of echo chambers, on the other hand, have been brought to systematically distrust all outside sources. In epistemic bubbles, other voices are not heard; in echo chambers, other voices are actively undermined. It is crucial to keep these phenomena distinct. First, echo chambers can explain the post-truth phenomena in a way that epistemic bubbles cannot. Second, each type of structure requires a distinct intervention. Mere exposure to evidence can shatter an epistemic bubble, but may actually reinforce an echo chamber. Finally, echo chambers are much harder to escape. Once in their grip, an agent may act with epistemic virtue, but social context will pervert those actions. Escape from an echo chamber may require a radical rebooting of one's belief system.
Book
Epistemology standardly holds that there can be no epistemically good reason to accept a known falsehood or to accept a mode of justification that is not truth-conducive. Such a stance cannot accommodate science, for science unabashedly relies on models, idealizations, and thought experiments which are known not to be true. We ought not assume that the inaccuracy of such devices is a sign of their inadequacy. When effective, they are felicitous falsehoods that exemplify features they share with the phenomena they bear on. Inasmuch as works of art also deploy felicitous falsehoods, they too advance understanding. True Enough develops a holistic epistemology that focuses on the understanding of broad ranges of phenomena rather than on knowledge of individual facts. Epistemic acceptability on this account is not a matter of truth or truth-conduciveness, but of what would be reflectively endorsed by members of an idealized epistemic community – a quasi-Kantian realm of epistemic ends.
Article
As a general rule, whenever a hearer is justified in forming the belief that p on the basis of a speaker’s testimony, she will also be justified in assuming that the speaker has formed her belief appropriately in light of a relevantly large and representative sample of the evidence that bears on p. In simpler terms, a justification for taking someone’s testimony entails a justification for trusting her assessment of the evidence. This introduces the possibility of what I will call “evidential preemption.” Evidential preemption occurs when a speaker, in addition to offering testimony that p, also warns the hearer of the likelihood that she will subsequently be confronted with apparently contrary evidence: this is done, however, not so as to encourage the hearer to temper her confidence in p in anticipation of that evidence, but rather to suggest that the (apparently) contrary evidence is in fact misleading evidence or evidence that has already been taken into account. Either way, the speaker is signalling to the hearer that the subsequent disclosure of this evidence will not require her to significantly revise her belief that p. Such preemption can effectively inoculate an audience against future contrary evidence, and thereby creates an opening for a form of exploitative manipulation that I will call “epistemic grooming.” Nonetheless, I argue, not all uses of evidential preemption are nefarious; it can also serve as an important tool for guiding epistemically limited agents though complex evidential scenarios.
Chapter
Many philosophers take understanding to be a distinctive kind of knowledge that involves grasping dependency relations; moreover, they hold it to be particularly valuable. This chapter aims to investigate and to address two well-known puzzles that arise from this conception: (1) the nature of understanding itself-in particular, the nature of “grasping” (2) the source of understanding’s distinctive value. The chapter argues that we can shed light on both puzzles by recognizing, first, the importance of the distinction between the act of coming to understand and the state of understanding; and, second, that coming to understand is a creative act.