Engaging Students as Co-Designers of Learning Analytics

Abstract

As Learning Analytics (LA) moves from theory into practice, researchers have called for increased participation of stakeholders in design processes. The implementation of such methods, however, still requires attention and specification. In this report, we share strategies and insights from a co-design process that involved university students in the development of a LA tool. We describe the participatory design workshops and highlight three strategies for engaging students in the co-design of learning analytics tools.

Full text

Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
Engaging Students as Co-Designers of Learning Analytics
Juan Pablo Sarmiento Fabio Campos Alyssa Wise
New York University New York University New York University
jp.sarmiento@nyu.edu fabioc@nyu.edu alyssa.wise@nyu.edu
ABSTRACT: As Learning Analytics (LA) moves from theory into practice, researchers have called
for increased participation of stakeholders in design processes. The implementation of such
methods, however, still requires attention and specification. In this report, we share strategies
and insights from a co-design process that involved university students in the development of
a LA tool. We describe the participatory design workshops and highlight three strategies for
engaging students in the co-design of learning analytics tools.
Keywords: participatory design, student-facing dashboards, higher education.
1. INTRODUCTION
While the use of Learning Analytics (LA) in higher education institutions is increasing, not many of
these tools are intended for students’ direct use (Bodily and Verbert, 2017). When such student-facing
tools are created, research shows that there can be misalignment between designers’ intentions,
students’ perceptions, and institutional limitations, with learners often distrusting the tools as a result
(de Quincey et. al., 2019). This may be because the majority of such tools are developed without the
direct involvement of students (Bodily and Verbert, 2017). In response, researchers have called for
increased participation of stakeholders in the design processes of these tools (Ahn, Campos, Hays &
DiGiacomo, 2019; Buckingham Shum, Ferguson & Martinez-Maldonado, 2019).
At New York University we have recently embarked on such a participatory design effort, developing
a learning analytics tool with and for students. This report describes a series of co-design workshops
held as part of the participatory process, highlighting key strategies for engaging students in co-
designing LA tools for their own tools.
2. INSTITUTIONAL BACKGROUND
New York University prides itself on being an innovator in higher education. With over 50,000 students
in both undergraduate and graduate programs, it has begun developing multiple LA initiatives to
support its diverse and growing student body. This includes the creation of a LA faculty service as part
of the university-wide instructional technology offerings, and the creation of an organization devoted
to LA research, NYU-LEARN research network. These two entities have been collaborating on a variety
of projects, and in November 2017 embarked on a joint project for the research and design of a
student-facing learning analytics tool, with the intent of eventual roll out to the entire student body.
3. PROCESS FRAMING AND PARTICIPATORY DESIGN WORKSHOPS
Our team, composed of members from both the LA service in IT and the research network, was tasked
with leading the process of developing a student-facing LA tool which would address challenges for
student learning at our university. The team formed a steering committee, composed of stakeholders
from NYU-LEARN, Faculty of Arts and Science and student advisors, which was responsible for setting

Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
the major boundaries of the project and participant sampling strategies. The committee decided to
target students with diverse backgrounds, focusing on first-generation participants as “extreme
users,” whose needs are often underserved when designing for an “average” user (Pullin & Newell,
2007).
Our design process utilized elements of Human-Centered Design, bringing stakeholders in not only as
sources of information but as co-designers. The aim was to understand their experiences, needs and
points of view, while taking into account learning challenges previously identified by the university.
Prior to the co-design sessions, we conducted in-depth interviews with 13 students, three faculty and
six advisors to surface the specific needs and challenges faced by students which LA could potentially
address. These were used to develop user profiles ("personas") that outlined salient problems for
students’ learning experiences. Students were then invited to take part in co-design sessions to
develop LA solutions that tackled these user personas’ challenges.
Participants were recruited through both an open call and individual emails which targeted a list of
106 first-generation students identified by advisors as potentially interested in design. Recruitment
was challenging due to time constraints of students at the end of term. Although more than 20
signalled interest, only 10 were able to participate. Most participants came back for multiple design
workshops and many indicated explicitly that they were happy with the experience. Students who
participated said they did so because of an interest in design and/or big data.
We conducted three 5-hour design workshops, in which a total of 10 freshman, junior, and graduate
students (between 4 and 7 in each session) learned about design and LA and worked alongside
researchers and a professional designer. The workshops consisted of: (1) Ice breakers; (2) Short lecture
on Design Thinking and Human-Centered Design; (3) Collaborative fleshing out of student personas
through empathy mapping; (4) Collaborative mapping of data types and representations relevant to
the personas; (5) Individual ideation of solutions through brain-writing; (6) Selection of promising
solutions; (7) Collaborative ideation and paper prototyping; (8) Iteration and development of new
prototypes (Fig. 1). Facilitators and a designer worked side by side with participants in these activities,
iterating between sketching designs, sharing them, providing feedback and redesigning.
Figure 1: Structure of co-design workshops
To respond to varying levels of data literacy among participants, part 4 of the session was devoted to
mapping, alongside students, the types of data the university could collect that might be relevant to
the challenges participants identified. Similarly, as a starting point for an ideation brainwriting session
(where participants wrote long lists of ideas for solutions), students were presented with a set of cards
(Fig. 2), which had (a) inferences that could potentially be made based on the available data (for
example “which readings have you opened?”), and (b) action phrases which suggested potential uses
for such information (such as “compare”; “help you understand”). Participants randomly took a couple
of the cards from each group and put them together, to stimulate lateral thinking and simultaneously

Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
ground their ideas in the available data. They would then share their favorite ideas and iterate on
these ideas in teams.
The resulting designs ranged broadly, from a tool to aid students in finding “study buddies” to systems
where users who took a given course could share information with new students. Students’ solutions
were often similar to those described in the literature (like tools for monitoring course progress or
using peer activity as a motivator) and sometimes pushed in directions new to the field (like leveraging
social networks and emotion as central elements of design).
Participants also brought their identities and interests into the discussion. For instance, one student
advocated for platforms which took into account variables such as wellbeing and mental health, after
having themselves been diagnosed with depression. Some ideas were mixed with others in playful
ways, often using metaphors such as a “hive” in which students visualized competing deadlines and
responsibilities from multiple courses, merged with a timeline, and a system for monitoring peer
resource activity (Fig. 2). Students also identified qualitative input from other students (such as advice,
tips or warnings) as a valuable source of information for how to successfully navigate personal and
academic challenges. While LA typically concerns itself with the presentation of data for insights, this
suggests that systems that pair analysis with the communication of know-how information through a
human network can have special value for students.
Figure 2: Ideation cards, student sketches and first prototype of the Hive interface.
4. KEY STRATEGIES FOR ENGAGING STUDENTS
Below we highlight three key strategies we found useful in the participatory design workshops:
Explicitly Address Power Dynamics. Power imbalances, such as those between students and
researchers or designers, can hinder participatory processes (Dollinger, 2018). Developing trust
between participants can be a way to overcome this problem. Throughout the workshops, facilitators
reminded participants that it was safe to share opinions and to be provocative and encouraged
students to challenge facilitators’ views and assumptions. The strategy seemed to succeed, in that
participants scrutinized the very objective of the session. They pointed out that questions of happiness
and meaning may be more relevant to students than academics, inciting a discussion about the role
of a university in students’ lives.
Keep the Problem Space Flexible. Traditional design thinking advocates for a clear problem framing,
but when working with students we saw value in allowing the room to go back to problematizing the
question and refrained from limiting students' ideation even while hunting for solutions. When
students pointed to unexpected problem spaces, such as happiness vs. academics, or the role of family
and motivation, these were honored. We believe that this played a role in them framing and

Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20)
Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)
integrating solutions in novel ways, such as tools which used interactions between student’s academic
and emotional data to provide suggestions.
Use Vulnerability to Develop Psychological Safety. Psychological safety can be an important
precondition to creativity (Hunter et. al., 2007), though one hard to achieve with strangers. As
facilitators, we used sharing vulnerability as a means to create a safe space, through ice breaker
exercises where participants worked together, engaged in physical contact and were silly in front of
each other. Likewise, participants were encouraged to share personal experiences and stories. Success
of the strategy is evident in video recordings of the workshops, which show participants shifting from
being hesitant to contributing to laughing and enjoying working together. The environment is likely to
have played a role in students being increasingly comfortable with sharing unconventional paths to a
solution, such as a tool where achievement was monitored through the “health” of a virtual pet (like
a Tamagotchi). Several of these more unconventional solutions emphasized affective and playful
design, an area that has yet not been central in LA tool creation.
5. CONCLUSION AND NEXT STEPS
As some in the field have argued, having students actively participate in the process of co-creating LA
tools can have a positive impact on ownership, understanding and use (de Quincey et. al., 2019;
Dollinger, 2018; Buckingham Shum et al, 2019). Our experience with participatory design suggests that
addressing issues of power imbalance and developing an environment conducive to creative thinking
has the potential to uncover innovative designs with the potential to improve students’ experience
and learning. At the time of this writing, we are preparing to test this hypothesis through user
experience trials of prototypes developed from these workshops; to understand if the process indeed
produces student-facing LA tools whose purpose is valued by students and which are seen as aligning
with their authentic needs.
REFERENCES
Ahn, J., Campos, F., Hays, M., & DiGiacomo, D. (2019). Designing in Context: Reaching beyond Usability
in Learning Analytics Dashboard Design. Journal of Learning Analytics, 6(2), 70-85.
Bodily, R., & Verbert, K. (2017). Trends and issues in student-facing learning analytics reporting
systems research. In Proceedings of the Seventh International Learning Analytics & Knowledge
Conference (pp. 309-318).
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-Centred Learning
Analytics. Journal of Learning Analytics, 6(2), 1-9.
Dollinger, M. (2018). Technology for the scalability of co-creation with students. Open Oceans:
Learning without borders, 346-350.
Hunter, S. T., Bedell, K. E., & Mumford, M. D. (2007). Climate for creativity: A quantitative review.
Creativity research journal, 19(1), 69-90.
Pullin, G., & Newell, A. (2007, July). Focussing on extra-ordinary users. In Proceedings International
Conference on Universal Access in Human-Computer Interaction (pp. 253-262).
de Quincey, E., Briggs, C., Kyriacou, T., & Waller, R. (2019, March). Student Centred Design of a
Learning Analytics System. In Proceedings of the 9th International Conference on Learning
Analytics & Knowledge (pp. 353-362).