PresentationPDF Available

Subversive Learning Analytics

Presentation

Subversive Learning Analytics

Learning Analytics Research Network
www.nyu.edu/learn-analytics
Subversive
Learning Analytics
Alyssa Friend Wise
Juan Pablo Sarmiento
Maurice Boothe Jr.
@NYU_LEARN
@alywise
@jp_sarmiento
@boothemjr
Lets rethink education
If we were to start from scratch…
Are the educational systems we have the
ones we would want?
Subversive Learning Analytics arises
from the premise that data should be used as a
transformative force for change, rather than to perpetuate an
often-problematic status quo, and uses a critical stance to
reveal and challenge the power structures and inequities in
education.
This might be a good metaphor for
education right now. This is
functionalist architecture, easy to
build, easy to replicate in all places
and circumstances, scalable and
comparatively inexpensive.
But if we go back to our better vision
of education, it might not look like
this. This has some challenges; it is
not adjusting to the environment, it
feels imposed and even a bit
inhuman.
What we have been doing to a
degree with Learning Analytics is to
improve on this status quo...
Creating tools through measuring,
collect and optimize processes
through data.
But if ultimately if what we want our
education to be something like this
-organic architecture, that is
designed to be in a dialogue with its
environment, with the particular
space where it is located...
IS GRADUAL IMPROVEMENT THE ANSWER?
Is our aim to perpetuate
or transform
the status quo of education?
Inventing the LA
we need
Use of LA for emancipation
& democratization
Improving the LA
we have
Eg. detecting and reducing
bias in predictive models
Prinsloo, Slade, Selwyn,
Gasevic, Ferguson, etc..
Two sets of problems
Looking within LA
There are multiple scholarly
traditions wrestling with
these issues for quite some
time
[the next slide is an interactive slide - you can click on the authors to access resources]
RUHA BENJAMIN
SASHA COSTANZA-CHOCK
TENDAYI ACHIUME
EVGENY MOROZOV
YESHIMABEIT MILNER
MEREDITH BROUSSARD
SAFIYA NOBLE
Design Justice
United Nations Report on
Racism and Technology
Big Data Challenges
Data for
Black Lives
JACK SCHNEIDER
New measures of
educational quality
What does it mean
to be subversive?
Designers & Researchers Can
Ask Disruptive Questions
Erode Stubborn Myths
Innovate Radical Design Strategies
ASK
DISRUPTIVE
QUESTIONS
Drawing on Costanza-Chock (2018, 2020)
we offer 5 sets of questions about
people, power and their intersection in
the creation and use of learning analytics
1
Who are the decision makers in
Learning Analytics?
Who is not at the table? Who gets the ultimate say?
Are these the same people who will be influenced?
1
Costanza-Chock (2020)
What student populations do we have in mind?
Are there minoritized groups we are not explicitly thinking about?
Who besides students may benefit or profit from the analytics?
Who will be impacted by the
Learning Analytics?
2
Costanza-Chock (2020)
3
What are the identities and positionalities of the players at the table?
How may our privilege be impacting or creating blind spots for our decisions?
What are our positionalities
related to Learning Analytics?
Costanza-Chock (2020)
What other entities are exercising power on the process?
What structures give them that power and what are their values?
What tradeoffs are we willing to make?
What larger structures are at play
in Learning Analytics?
Costanza-Chock (2020); Morozov (2013); Benjamin (2019)
4
What are our values about education, data, or being human?
What assumptions about the social order are embedded into our process or tools?
What value systems are embedded
in Learning Analytics?
5
Costanza-Chock (2020); Benjamin (2019)
Asking disruptive questions
helps reveal the many ways in
which learning analytics work
can be influenced by existing
systems and structures
ERODE
STUBBORN
MYTHS
*which we all know aren’t really true
but still operate as if they were
*
The Objectivity Myth
Data
arent found, but generated
come with different embedded values
necessarily reflect institutional politics
dont speak for themselves
may not always tell the whole story
cant unproblematically predict the future
Prinsloo & Slade (2017); Noble (2018); Broussard (2019); Selwyn (2020); Wise (2020)
And yet looking at
LAK’20...
When the Objectivity Myth slips into our
work, we may start to equate data with
reality neglecting the complexity and
nuance of the world that led to the data
in the first place.
Benjamin (2019)
The Perfect Process Myth
c.f. Kitto, Shum & Gibson, A. (2018); Achiume (2020)
The deceptive notion that
all problems are solvable
by enough data and a good process
Morozov (2013)
When the Perfect Process Myth slips into
our work, we may over structure
something that requires flexibility and
fluidity, losing valuable aspects of the
human experience.
INNOVATE
RADICAL
DESIGN
STRATEGIES
Predicted Final Biology Course Grade
(default is Asian and Male) (N=5688)
Empty
Boxes
Model 1 Model 2 Model 3
Black (N=462) -0.538*** -0.131*** ?
Hispanic (N=887) -0.352*** -0.115*** ?
Int’l (N=434) 0.147*** 0.027*** ?
White (N=1372) 0.048*** -0.056***?
Female (N=3603) -0.087*** -0.120*** ?
cumGPA \ 1.289*** ?
Access to AP Bio in HS \ \ ?
Work / Family Responsibilities \ \ ?
Understanding of Office Hours \ \ ?
Representation in Curriculum \ \ ?
What data is missing?
How can we keep these
unknown factors in
mind?
Do we have ways to
represent the absence
of data as well as its
presence?
Listened to when speak in the classroom
A Different
Baseline
How do we move away from
a deficit mentality?
How do we acknowledge
layers of privilege in the
“average”?
Predicted Final Biology Course Grade
Have a quiet space to study at home
Went to a HS w/ bio labs and field trips
Always knew you would go to college
Have instructors who feel supported by school
BASELINE
YOU
Stevens (2019)
Grades
Counter-
Narrative
Metrics
Retention
Do we have a full measure of
academic success and school
quality?
How can we assess, model and
optimize for what different
communities value?
Counter-
Narrative
Metrics
Grades
Capacity to Change
the World
Professional
Identity
Sense of
Belonging
Academic
Challenge
Student-Teacher
Relationships
Student Safety &
Institution Trust
Engagement with the
Community
Gagnon & Schneider (2019)
Retention
Do we have a full measure of
academic success and school
quality?
How can we assess, model and
optimize for what different
communities value?
Thank you
Lets start the conversation
@alywise
@jp_sarmiento
@boothemjr
Learning Analytics Research Network
www.nyu.edu/learn-analytics
@NYU_LEARN
Best short paper nominee LAK’21
Subversive Learning Analytics
https://dl.acm.org/doi/10.1145/3448139.3448210
Learning Analytics Research Network
www.nyu.edu/learn-analytics
@NYU_LEARN
ResearchGate has not been able to resolve any citations for this publication.
ResearchGate has not been able to resolve any references for this publication.