Simon Buckingham Shum

University of Technology Sydney | UTS · Connected Intelligence Centre

Advances in generative artificial intelligence (GenAI) have created uncertainties and tensions in higher education, particularly concerning learning, equity and quality. Despite emerging empirical research, much current policy is based on assumptions about how and why students are using GenAI. This Pecha Kucha reports on 20 online focus groups invo...

Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to suppor...

Dashboards are increasingly used in education to provide teachers and students with insights into learning. Yet, existing dashboards are often criticised for their failure to provide the contextual information or explanations necessary to help students interpret these data. Data Storytelling (DS) is emerging as an alternative way to communicate ins...

It is widely documented that higher education institutional responses to the COVID-19 pandemic accelerated not only the adoption of educational technologies, but also associated socio-technical controversies. Critically, while these cloud-based platforms are capturing huge datasets, and generating new kinds of learning analytics, there are few stro...

Human‐Centred Learning Analytics (HCLA) has emerged in the last 5 years as an active sub‐topic within Learning Analytics, drawing primarily on the theories and methods of Human‐Computer Interaction (HCI). HCLA researchers and practitioners are adopting and adapting HCI theories/methods to meet the challenge of meaningfully engaging educational stak...

Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges com...

To promote cross-community dialogue on matters of significance within the field of learning analytics (LA), we as editors-in-chief of the Journal of Learning Analytics (JLA) have introduced a section for papers that are open to peer commentary. An invitation to submit proposals for commentaries on the paper was released, and 12 of these proposals w...

It is well-established that a student’s sense of belonging is associated with successful transition into higher education, along with a raft of positive outcomes including enhanced learning, well-being, and attainment for all students. The importance of belonging was further heightened by the Covid-19 pandemic, as the increased shift to online lear...

Effective learning depends on effective feedback, which in turn requires a set of skills, dispositions and practices on the part of both students and teachers which have been termed feedback literacy. A previously published teacher feedback literacy competency framework has identified what is needed by teachers to implement feedback well. While thi...

An extraordinary amount of data is becoming available in educational settings, collected from a wide range of Educational Technology tools and services. This creates opportunities for using methods from Artificial Intelligence and Learning Analytics (LA) to improve learning and the environments in which it occurs. And yet, analytics results produce...

A burgeoning literature, policy landscape, and set of practice resources and guidelines have emerged around ethical educational technology development and implementation, particularly in the context of data or artificial intelligence informed tools. However, while these resources provide valuable tools to support navigation of the ethical landscape...

In this chapter, I describe and reflect on the last 8 years at an Australian public university, inventing, piloting and evaluating Learning Analytics tools, specifically focused on data-driven personalised feedback, leading in some cases to integration with the institution’s learning technology ecosystem, and accompanied by staff training and suppo...

Students’ sense of belonging is associated with successful transition into higher education and a range of positive outcomes including enhanced learning, well-being, and demonstrated achievement. The COVID-19 pandemic underscored the importance of belonging as the shift to online learning highlighted the challenges of supporting and monitoring stud...

Collocated teamwork remains a pervasive practice across all professional sectors. Even though live observations and video analysis have been utilized for understanding embodied interaction of team members, these approaches are impractical for scaling up the provision of feedback that can promote developing high-performance teamwork skills. Enrichin...

Central to imagining the future of technology-enhanced professional learning is the question of how data are gathered, analyzed, and fed back to stakeholders. The field of learning analytics (LA) has emerged over the last decade at the intersection of data science, learning sciences, human-centered and instructional design, and organizational chang...

There are emerging concerns about the Fairness, Accountability, Transparency, and Ethics (FATE) of educational interventions supported by the use of Artificial Intelligence (AI) algorithms. One of the emerging methods for increasing trust in AI systems is to use eXplainable AI (XAI), which promotes the use of methods that produce transparent explan...

The proliferation of AI in many aspects of human life—from personal leisure, to collaborative professional work, to global policy decisions—poses a sharp question about how to prepare people for an interconnected, fast-changing world which is increasingly becoming saturated with technological devices and agentic machines. What kinds of capabilities...

There is growing interest in creating learning analytics feedback interfaces that support students directly. While dashboards and other visualizations are proliferating, the evidence is that many fail to provide meaningful insights that help students reflect productively. The contribution of this paper is qualitative and quantitative evidence from...

Teachers’ spatial behaviours in the classroom can strongly influence students’ engagement, motivation and other behaviours that shape their learning. However, classroom teaching behaviour is ephemeral, and has largely remained opaque to computational analysis. Inspired by the notion of Spatial Pedagogy, this paper presents a system called ‘Moodoo’...

Teachers’ spatial behaviours in the classroom can strongly influence students’ engagement, motivation and other behaviours that shape their learning. However, classroom teaching behav-iour is ephemeral, and has largely remained opaque to computational analysis. This paper presents a library called ‘Moodoo’ that can serve to automatically model how...

Effective teamwork is critical to improve patient outcomes in healthcare. However, achieving this capabilityrequires that pre-service nurses develop the spatial abilities they will require in their clinical placements, suchas: learning when to remain close to the patient and to other team members; positioning themselves correctlyat the right time;...

Using data to generate a deeper understanding of collaborative learning is not new, but automatically analyzing log data has enabled new means of identifying key indicators of effective collaboration and teamwork that can be used to predict outcomes and personalize feedback. Collaboration analytics is emerging as a new term to refer to computationa...

Objective. To explore pharmacy students' perceptions of a novel web application tool (AcaWriter) implemented in a Master of Pharmacy curriculum to support reflective thinking in scientific research.Methods. A qualitative research design involving a 50-minute focus group (n=12) was used. The focus group session was audio-taped, transcribed verbatim,...

The formative assessment of written reflection provides opportunities for students to improve their practice in an iterative manner using reflective writing. However, manual formative assessment of written reflection is time consuming and subjective. While progress has been made in deploying writing analytics tools to provide automated, formative f...

Writing analytics has emerged as a sub-field of learning analytics, with applications including the provision of formative feedback to students in developing their writing capacities. Rhetorical markers in writing have become a key feature in this feedback, with a number of tools being developed across research and teaching contexts. However, there...

Current writing support tools tend to focus on assessing final or intermediate products, rather than the writing process. However, sensing technologies, such as keystroke logging , can enable provision of automated feedback during, and on aspects of, the writing process. Despite this potential , little is known about the critical indicators that ca...

This paper presents Moodoo, a system that models how teachers make use of classroom spaces by automatically analysing indoor positioning traces. We illustrate the potential of the system through an authentic study aimed at enabling the characterisation of teachers’ instructional behaviours in the classroom. Data were analysed from seven teachers de...

Lay Description What is already known about this topic The term “Classroom Proxemics” refers to how teachers and students use the classroom space, and its impact on learning. A large number of teachers, particularly in higher education, receive no pedagogical training or feedback on classroom proxemics. Little is known about how to create interfac...

Human-centred software design gives all stakeholders an active voice in the design of the systems that they are expected to use. However, this is not yet commonplace in Learning Analytics (LA). Co-design techniques from other domains therefore have much to offer to LA, in principle, but there are few detailed accounts of exactly how such sessions u...

Automatic tracking of activity and location in the classroom is becoming increasingly feasible and inexpensive. However, although there is a growing interest in creating classrooms embedded with tracking capabilities using computer vision and wearables, more work is still needed to understand teachers' perceived opportunities and concerns about usi...

Failing to understand the perspectives of educators, and the constraints under which they work, is a hallmark of many educational technology innovations' failure to achieve usage in authentic contexts, and sustained adoption. Learning Analytics (LA) is no exception, and there are increasingly recognised policy and implementation challenges in highe...

Learning Analytics (LA) systems can offer new insights into learners’ behaviours through analysis of multiple data streams. There remains however a dearth of research about how LA interfaces can enable effective communication of educationally meaningful insights to teachers and learners. This highlights the need for a participatory, horizontal co-d...

This paper seeks to contribute to the emerging field of Quantitative Ethnography (QE) by demonstrating its utility to solve a complex challenge in Learning Analytics: the provision of timely feedback to collocated teams and their coaches. We define two requirements that extend the QE concept in order to operationalise it such a design process, name...

The design of effective learning analytics extends beyond sound technical and pedagogical principles. If these analytics are to be adopted and used successfully to support learning and teaching, their design process needs to take into account a range of human factors, including why and how they will be used. In this editorial, we introduce principl...

Reflective writing is widely acknowledged to be one of the most effective learning activities for promoting students’ self-reflection and critical thinking. However, manually assessing and giving feedback on reflective writing is time consuming, and known to be challenging for educators. There is little work investigating the potential of automated...

The title is provocative: Quantitative Ethnography. Those two words are rarely uttered in the same breath except in conflict! For the former to be an adjective to the latter is surely ridiculous. Yet the “quant/qual” gulf is precisely what David Williamson Shaffer seeks to bridge. His message: people, we’re all on the same side, and I’m not asking...

This project brought together learning analytics experts from across Australia to explore key ethical issues relating to the development and use of learning analytics in higher education. The result of these discussions was a discussion paper that provides an outline of seven ethical principles as well as practical considerations associated with th...

A major promise of learning analytics is that through the collection of large amounts of data we can derive insights from authentic learning environments, and impact many learners at scale. However, the context in which the learning occurs is important for educational innovations to impact student learning. In particular, for student-facing learnin...

Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members' embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly...

Purpose – Many of today’s information and technology systems and environments facilitate inquiry, learning, consciousness-raising and knowledge-building. Such platforms include e-learning systems which have learning, education and/or training as explicit goals or objectives. They also include search engines, social media platforms, video-sharing pl...

The learning analytics community has matured significantly over the past few years as a middle space where technology and pedagogy combine to support learning experiences. To continue to grow and connect these perspectives, research needs to move beyond the level of basic support actions. This means exploring the use of data to prove richer forms o...

In this paper we present a three-phase process for crafting Learner-Data Journey maps and using them as communication tools to involve other stakeholders in the co-design of a data-intensive educational tool. The three phases in this process are i) scaffolding groups of learners to collaboratively co-create a Learner-Data Journey based on their own...

From a human-centred computing perspective, supporting the interpretation of educational dashboards and visualizations by the people intended to use them exposes critical design challenges that may often be trivialized. Empirical evidence already shows that “usable” visualizations are not necessarily effective from an educational perspective. Since...

In this chapter, we outline a project which traces its source of inspiration back to the grand visions of Vannevar Bush (scholarly trails of linked concepts), Doug Engelbart (highly interactive intellectual tools, particularly for argumentation), and Ted Nelson (large scale internet publishing with recognised intellectual property). In essence, we...

Providing immediate, effective feedback on team and individual performance in healthcare simulations is a challenging task for educators, such is their complexity. Focusing on emergency procedures on patient manikins, our prior work has demonstrated the feasibility of using multimodal data capture and analysis to generate visualisations of student...

Text revision is regarded as an important process in improving written products. To study the process of revision activity from authentic classroom contexts, this paper introduces a novel visualization method called Revision Graph to aid detailed analysis of the writing process. This opens up the possibility of exploring the stages in students’ rev...

Objective. To assess pharmacy students' perceptions of the benefits and utility of a novel online reflective-writing tool. Methods. After completing a required Academic Writing Analytics (AWA) workshop, Master of Pharmacy students submitted a reflective writing assignment in the AWA web application. A six-item survey was administered to students pr...

Data science is now impacting the education sector, with a growing number of commercial products and research prototypes providing learning dashboards. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that `usable' visuali...

Learning Analytics (LA) sits at the confluence of many contributing disciplines, which brings the risk of hidden assumptions inherited from those fields. Here, we consider a hidden assumption derived from computer science, namely, that improving computational accuracy in classification is always a worthy goal. We demonstrate that this assumption is...

Academic writing is a key skill required for higher education students, which is often challenging to learn. A promising approach to help students develop this skill is the use of automated tools that provide formative feedback on writing. However, such tools are not widely adopted by students unless useful for their discipline-related writing, and...

Data science is now impacting the educational sector, with a growing number of commercial products and research prototypes providing learning dashboards as feedback for both educators and students. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical...

Awarded the Teaching Innovation Prize- The Poster describes the utilization of a novel online tool, AWA-Academic Writing Analytics, which is currently trialled as a research pilot in the first year Masters of Pharmacy Curriculum at the University of Technology Sydney (UTS). AWA provides a student guide to constructive, formative feedback on drafts...

In many situations, it remains critical for team members to develop strategies to effectively use the space and tools available to complete demanding tasks. However, despite the availability of sensors and analytics for instrumenting physical space, relatively little progress has been made in modelling the embodied dimensions of co-located teamwork...