Content uploaded by James Richard Birt
Author content
All content in this area was uploaded by James Richard Birt on Jan 10, 2019
Content may be subject to copyright.
Bond University
Research Repository
Glide: Towards an intelligent kinesthetic learning space for teaching dance
De Byl, Penny Baillie; Birt, James R.; Khan, Muqeem
Published in:
13th Middle Eastern Simulation and Modelling Conference
Published: 01/01/2012
Document Version:
Publisher's PDF, also known as Version of record
Link to publication in Bond University research repository.
Recommended citation(APA):
De Byl, P., Birt, J., & Khan, M. (2012). Glide: Towards an intelligent kinesthetic learning space for teaching
dance. In M. Al-Akaidi (Ed.), 13th Middle Eastern Simulation and Modelling Conference : MESM 2012
Proceedings (pp. 81-82). EUROSIS.
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
For more information, or if you believe that this document breaches copyright, please contact the Bond University research repository
coordinator.
Download date: 10 Jan 2019
GLIDE:
TOWARDS AN INTELLIGENT KINESTHETIC LEARNING SPACE FOR
TEACHING DANCE
Penny de Byl and James Birt
Bond University, Australia
E-mail: pdebyl@bond.edu.au, jbirt@bond.edu.au
Muqeem Khan
Northwestern University in Qatar
muqeem@usa.net
KEYWORDS
Learning, Teaching, Motion-Capture, Dance, Artificial
Intelligence, XBox Kinect
ABSTRACT
technology; and 3) perpetuate the notion of serious games in
the realm of kinesthetic learning space.
GLIDE: A Guided Learning and Immersive Dance
Environment
In this paper we present the overview for a proposed
artificially intelligent learning environment which instructs
dance. The Guided Learning and Immersive Dance
Environment (GLIDE) will teach dance via a virtual
instructor that senses student movement in the real world via
a Microsoft Kinect. As a student attempts to mimic the
movements of the instructor the system will extract skeleton
movement and joint rotations to evaluate the dance
performance. The analysis of the student's movements will
be fed into the system's artificial intelligence which will
provide real-time feedback and customised targeted
instruction to assist in bettering the student's performance.
This technology has far reaching application from tradition
dance instruction to preservation and dissemination of
intangible cultural heritage.
I INTRODUCTION
Dance can be learned from verbal description, spatial guides
or instructor imitation. Verbal description works well if the
dance isn't too complicated, however as the movements
become more complex and increase in speed, words are too
deliberate, general and linear to embody the complexity of
precise movement. The spatial guide method used in old-
fashioned dancing, which forms the basis of numerous
computer games such as Dance Dance Revolution, instructs
players through tasks of hitting a sequence of spatial targets
in rhythmic time. This provides a greater kinesthetic
learning experience than verbal description alone. In this
paper we outline the proposed GLIDE system being
developed by Bond University in Australia, Northwestern
University in Qatar and Hanze University, The Netherlands.
Through it we will evaluate motion-sensing technologies,
developed for use in contemporary digital computer games,
through the design of an intelligent kinesthetic learning
space. This project aims to produce the following outcomes
1) create a proof-of-concept intelligent kinesthetic learning
space, 2) evaluate and explore knowledge transfer
opportunities afforded by kinesthetic peripheral games
In the hierarchy of visual and kinesthetic experiences with
GLIDE, the cognitive subprocesses play the leading role.
The peripheral and focused visions constantly scan the
application on the screen for learning cues and feedbacks.
This transection of information from user's interaction
interacts simultaneously with short term or working/active
memories, depending on the user's choices and sudden
emotional or structural changes from digital elements
presented by GLIDE application. Research shows that human
have limited capacity for coding, storage and retrieval of
information from the screen (Lang, 2006). The limited
capacity model categorizes television viewers as information
processor. Viewers process the information on the screen
with parallel cognitive sub-process of coding, storage and
retrieval of messages that they have exposed.
The main difference in our approach to designing GLIDE
with respect to existing systems is in determining the optimal
configuration between the artificial intelligence (AI)
techniques used to provide student feedback and the
accuracy with which motion is being captured and analysed.
In short, we endeavour to determine a point in the design at
which learning effectiveness does not improve with more
complex processing and feedback.
GLIDE will include the projection of a virtual dance
instructor character programmed with real world dance
routines captured from expert dancers and a dance area for
students (as illustrated in Figure 1). Students entering the
space will receive dance instruction from the character. The
motion-sensing devices will track the students' movements
and the character will give them feedback on their
performance.
© EUROSIS-ETI
Student foUov/s end
jneiiipli to mimic ilic
iasliuciof.
IntelBgcll softwaieconipiircs
insinKIO' moveiw'ol with sHidoiil movanicnl
1 adapts insiructionfor cwrecling .wfl pioo'eniraj
lessen.
Iiuuudor teachiis
by eximpla.
Inlurmi cameras
Captllt« .in<l tKOKl
student mowiiiiMil.
based on the research team's access to a Hip Hop dance
instmctor and a large cohort of secondary school children.
Pose Evaluation
Figure 2: GLIDE illustrating the instructor on the left, the
student's avatar and an accuracy display highlighting the
joints of the student which are not in line with those of the
instructor.
Pose evaluation identifies in real time or in post processing
how a human body and/or individual limbs and joints are
configured in a given scene. There are many techniques
revealed in contemporary literature for determining the
accuracy of human pose capture (Moeslund & Granum,
2001). These techniques can be coarsely grouped into two
classes, 2D image sequencing and 3D data point analysis
(Raptis, Kirovski, & Hoppe, 2011). Methods using 2D image
sequences focus on extracting image and pattern features of
the movement in space and time. Examples of these features
include contours to identify similar motions between the
recorded movement and the user (Aaron, James, & IEEE
Computer, 2001), and matching systems against pre-recorded
poses or templates of human actions such as walking, waving
and running (Schuldt, Laptev, & Caputo, 2004). The use of
3D information provides many benefits over the pure 2D
image based approaches. The data does not suffer from
sensitivity such as vantage point, scale and light changes
(Raptis, et al., 2011). The 3D approach allows tracking of
individual parts of the human body enabling temporal
analysis of human body dynamics or simply dance movement
in time.
The method used for determining pose accuracy in our
project will depend on the level of detail required to achieve
the optimal learning situation. Using the provided pose
evaluation examples as a baseline we propose to experiment
using: 1) distinct time intervals and 3D joints, 2) musical
beat intervals and 3D joints, 3) realtime 3D joints, 4)
approximate 3D joints in real time with simple Gaussian
analysis, 5) approximate 3D joints in real time with inverse
kinematics and Euclidean distance The goal is to generate a
series of algorithms from these pose evaluations that range in
complexity and can adapt to the user.
Proposed Efficacy and Affordance
The current prototype, shown in Figure 2, is implemented
with the Unity 3D game engine. It has been developed to
begin evaluation on the first configuration of visual accuracy
feedback and the use of Euclidian distance to determine
player pose accuracy. Although the image illustrates and
Australian aboriginal dancer the design will change to
accommodate our first study which will evaluate the use of
GLIDE in teaching Hip Hop. This decision is primarily
SUMMARY
GLIDE offers a well crafted kinesthetic learning space with
playfulness that can easily foster the active, purposeful and
entertaining user interaction with observational learning
opportunities. With the marketplace entry of the Nintendo
Wii, PlayStation Move and Microsoft Kinect motion capture
technology is now available to everyone. While there are a
plethora of serious applications of this technology being
examined, there is little research into its efficacy with respect
to teaching dance. This project represents a cutting-edge and
necessary examination of motion-sensing technologies and
their potential for kinesthetic knowledge transfer to a wider
international community. While the project outlined herein
will examine the design parameters for a truly effective
virtual dance tutor, the applications stemming from such are
further reaching. For example, the preservation and teaching
of cultural dances is an important focal area in ICH research.
The use of these technologies not only provides tremendous
showcasing opportunities for dance and related kinesthetic
domains but also guarantees spontaneous, undirected
learning experiences for people of all ages (Tanenbaum &
Bizzocchi, 2009).
REFERENCES
Lang, A. (2006). Using the limited capacity model of motivated
mediated message processing to design effective cancer
communication messages. Journal of Communication, 56, S57—
S80.
Aaron, F. B., James, W. D., & IEEE Computer, S. (2001). The
Recognition of Human Movement Using Temporal Templates.
Moeslund, T. B., & Granum, E. (2001). A Survey of Computer
Vision-Based Human Motion Capture. Computer Vision and
Image Understanding, 81(3), 231-268. doi:
10.1006/cviu.2000.0897
Raptis, M., Kirovski, D., & Hoppe, H. (2011). Real-time
classification of dance gestures from skeleton animation.
Tanenbaum, J., & Bizzocchi, J. (2009). Rock Band: a case study in
the design of embodied interface experience. Proceedings of the
2009 ACM SIGGRAPH Symposium on Video Games (pp.
127-134).
© EUROSIS-ETI
