Conference PaperPDF Available

Usability Evaluation of Kinect-Based System for Ballet Movements

Authors:

Abstract and Figures

Since the 1800s, ballet education is influenced by the use of mirrors. The aim of this study is to evaluate a Kinect-based system called Super Mirror, to discover if it has an impact on the usability in ballet instruction. Ballet students were evaluated on eight ballet movements (plié, élevé, grand plié, batte- ment tendu (front, side and back), passé and développé) to measure the Super Mirror’s impact. The results show a potential usage in ballet education but improvements of Super Mirror are needed to comply with the standardized subject-matter expert’s criteria.
Content may be subject to copyright.
Usability Evaluation of Kinect-Based System for Ballet
Movements
Milka Trajkova and Mexhid Ferati
South East European University, Skopje, Macedonia
{mt19992, m.ferati@seeu.edu.mk}
Abstract. Since the 1800s, ballet education is influenced by the use of mirrors.
The aim of this study is to evaluate a Kinect-based system called Super Mirror,
to discover if it has an impact on the usability in ballet instruction. Ballet stu-
dents were evaluated on eight ballet movements (plié, élevé, grand plié, batte-
ment tendu (front, side and back), passé and développé) to measure the Super
Mirror’s impact. The results show a potential usage in ballet education but im-
provements of Super Mirror are needed to comply with the standardized sub-
ject-matter expert’s criteria.
Keywords: usability, Kinect, ballet, user evaluation
1 Introduction
Judith A. Gray, a pioneer for dance technology once said, Dance the oldest art, is
today but a young science” [6]. Ballet instruction roots in the studio and is composed
of three components: barre, specialized flooring and mirrors. Today, its traditional
approach is still in use; the learning environment of mirror use in ballet likely began
sometime in the eighteenth century, although historically the genesis is not clearly
documented [5]. Mirrors thus become central to a dancer’s ballet education. The psy-
chology of a dancer is built around it as it is taught around it. The mirror becomes the
source for a dancer on how others view them and the portrayal of success of their
technique [11]. A dancer’s perception of themselves is partially bound to the exist-
ence of mirrors in traditional dance environments. It contributes a physical self-
evaluation, behavior regulation, and competition in dancers [7]. Its traditional ap-
proach has been the subject of a new methodology suggested by Marquardt et al ti-
tled, Super Mirror [9]. It is a system developed through the use of Kinect-based tech-
nology that “combines the functionality of studio mirrors and prescriptive images to
provide the user with instructional feedback in real-time” [9].
While the Super Mirror and the similar YouMove system by Anderson et al devel-
op a comprehensive evaluative methodology [2], the focus was not on benchmarking
the systems to the expertise of ballet teachers. A need remains for a method to evalu-
ate the quality of feedback received from the Super Mirror, a reference system, to the
expertise of a subject matter, a ballet teacher, a control. Testing the quality of feed-
back will indicate a level of effectiveness of the system and ensure potential efficien-
cy as a learning tool for ballet. The purpose of this study focuses on the effectiveness
and efficiency of the feedback received from the Super Mirror on pre-professional
ballet dancers. Can a system prove to be as accurate as a ballet teacher in assessing
the quality of dancers’ movements?
2 Related Work
Studying the effects of mirrors in dancers’ perceptions of themselves with regard to
their performance and as a guide to self-correction is not new. Studies by Radell et al
[12] have suggested that the use of a mirror in a ballet classroom may negatively af-
fect the skill acquisition of the dancer and ultimately impact their performance, which
has contradicted the results from Dearborn et al [3]. The first study [12] has conclud-
ed that while 85.7% of dancers remarked that the use of mirrors has influenced their
understanding of the concepts taught, satisfaction with overall appearance decreased
for high performing dancers in a mirrored class [11]. Green has expressed, in a cri-
tique of traditional dance instruction, “the constant focus on an externalized view of
the body, as reflected in the mirror, objectifies the dancer’s body and requires students
to strive to achieve a specific ‘look’ while being ‘corrected’ so the students perform
‘proper’ dance technique” [7]. While the mirror provides immediate visual feedback
in real-time, it also may result in a false perception of a dancer’s weaknesses. The
consciousness of thoughts contributed by the mirror may welcome detrimental effects
in the overall well-being of the dancer and hinder the development of their technique.
In order to combat the given negative effects of mirror in ballet instruction, re-
searchers have turned to technology to help aide teachers and students alike as a guide
to self-correction. This study [2] compared the YouMove system to traditional video-
based instruction methods and has discovered that learning increased using the sys-
tem. Another study [4] reported the effects of real-time virtual reality (VR) feedback
on motor skills and explored the ability to focus the learner to key features of a to-be
learned action. Similarly, [16] has identified that video analyses support a basis for
rank-specific supplemental training in ballet companies. Video analyses help, “teach-
ers…tailor their classes to the appropriate intensity and can create combinations…that
can replicate the demands of specific roles” [16]. Further studies [8] have revealed
that students considered streaming video as effective for carrying out self-evaluation.
Other studies have suggested computer animations benefit dancers with experience
and are at least as effective for learners without dance experience in contrast to video
[13].
3 Research Methodology
A controlled study was conducted using pre-professional ballet dancers to compare
the Super Mirror’s assessment of movements, an embedded reference system, to the
evaluation of a ballet teacher, the control. A total of 5 ballet students from the State
Ballet School, Ilija-Nikolovski Luj from Skopje, Macedonia were tested. The pre-
professional students were between the ages of 16 to 18 with an average of 8.8 years
of ballet education. Each student has class 5 times a week with each class lasting for
an hour and 30 minutes, excluding rehearsals for performances. Eight movements
provided by the Super Mirror: plié, élevé, grand plié, battement tendu (front, side and
back), passé and développé were assessed. Each movement has an embedded refer-
ence model, a predefined movement template that measured the correct matches or
“hits” as referred by the terminology reported in [9] by comparing the set of thresh-
olds of the x, y and z rotational values of the left upper leg, right upper leg, left lower
leg and right lower leg. The only interaction the Super Mirror had was motion-
capture. This was performed by joint skeleton tracking through a Kinect camera, and
the transfer of input from the camera to the processor [was] mediated by the Synapse
application [14]. The specific interfaces developed for [the] system use the Tryplex
toolkit [15], a set of open source macro patches for Quartz Composer” [9].
Intentionally, each dancer began with alteration, either with or without the Super
Mirror to nullify any possible effects of the dancer to accustom to the system. This
will avoid the ability to receive a higher number of hits without achieving higher
performance techniques. Additionally, to remove any influence caused by the Super
Mirror, the teacher was positioned where she was unable to view the screen where the
Super Mirror was projected. Differentiated from the YouMove design, the experiment
added a control, a ballet teacher, for the purposed of testing the reference system, the
Super Mirror.
3.1 Procedure
Testing the dancers involved setting up the Super Mirror and having the system dis-
played on a 37” LCD screen in a wide room to most closely resemble a ballet studio.
A pre-test questionnaire was first distributed to all of the participants to capture cer-
tain demographics such as age, how long they have been dancing ballet for, how
many hours per week do they dance, etc. The dancers were tested in an ascending
order by grade in order to keep a clear and logical flow. One by one, the dancers were
tested on the eight movements. Each movement was performed enface (to the front)
according to a number of times predetermined from the ballet teacher. This number
was due to the artistic nature of ballet. It was necessary and essential to mimic the
Fig. 1. Experiment procedure for a single participant
number of times each movement was performed as in a typical ballet class to evoke as
closely as possible its natural environment. All the movements, plié, grand plié,
battement tendu (front, side and back), passé and développé except for élevé, were
performed 4 times. Élevé was performed 8 times. Figure 1 explains the procedure of
testing one participant.
Each test comprised of eight movements. Each movement was conducted in two
sequences (S1 and S2). Each sequence comprised of two parts, P1 and P2 that includ-
ed with and without (W/O) the Super Mirror. Between each sequence, a one-minute
break was given to allow for rest. The teacher evaluated each part and assessed the
student’s performance on a scale of one to ten based on a set of criteria specific to
each movement. During the test, when the part with the Super Mirror was included,
the dancer’s accuracy of performed movement was measured against the embedded
reference model in the system. The dancers’ successful performance was registered
by the system as a number of matches or “hits” as offered by the terminology used in
[9]. The roles of Kinect and the teacher were complementary. The teacher assessed
the technique elements that were not tested by the system to determine its effective-
ness. After all the students were tested, a System Usability Scale (SUS) was adminis-
tered to both the students and the teacher, which also included open-ended questions.
A short discussion was held to gather feedback about the system. Figure 2 shows the
dancers using the Super Mirror during the experiment.
Fig. 2. Dancers performing a plié (left) and passé (right) using the Super Mirror
4 Results and Discussion
Data from previous studies involving the testing of the YouMove system [2] reveal
the effectiveness of using such systems when compared to traditional video-based
instruction methods. In our study, the focus was the comparison of such a system to
the knowledge of a domain specific expert. The results concentrated on three specific
movements, plié, élevé and tendu front. The other five movements, grand plié, tendu
side, tendu back, passé and développé to the side, were not included because the Su-
per Mirror results were non-conclusive. This was most likely due to the inability to
adapt the reference model of the movements to the height of the dancers.
Further investigation is needed to accurately calibrate the reference template to the
specificity of each dancer. Adequately, the possibility for comparison between the
Super Mirror and the teacher was impossible. The following figures represent aggre-
gate results of the three movements. The aggregation of the scores of each movement
was based on the grade level of the student (x-axis), the average teacher score (left y-
axis) and the average Super Mirror hit (right y-axis). The teacher score was graded on
a scale of 1-10, while the Super Mirror score was a ratio given as a percentage of the
successful hits vs. the predetermined number of times a movement was performed.
Figure 3 represents the assessment of the teacher score vs. the Super Mirror score in
the movement of a plié. A plié is the bending of the knee or knees [1].
The teacher’s score indicated a gradual level of increase with the experience level
of the student. According to the subject-matter expert, the teacher scored the dancers
on the following set of criteria for a plié: do heels lift from the floor, is weight distrib-
uted equally between both feet, are legs turned out from the hips, are shoulders back,
is stomach in, is back straight, is torso strong, are ribs in, are hands soft, are arms
synchronized with legs, and is bottom tucked in. As students move from year to year,
Fig. 3. Plié score
their technique improves. Therefore the teacher’s score was greater. However, the
same trend is not seen with the Super Mirror. The results from the Super Mirror
showed higher results compared to the teacher score. The Super Mirror hits com-
pared only the angles of knee and hip joints with prerecorded angle widths [9]. This
indicated that the Super Mirror score complexity is much lower than the teacher’s.
The Super Mirror did not carry the ability to view the dancer’s technique as a whole
therefore accounting for inaccurate results. Figure 4 represents the assessment of an
élevé, a rise on demi-pointe [1]. According to the subject-matter expert, an élevé has
the same complexity as a plié.
The teacher assessed the dancers on the following set of criteria for an élevé: are
legs turned out, is weight distributed between both feet, are knees straight, are shoul-
ders back, is stomach in, is back straight, is torso aligned, are ribs in, and are heels
turned out. The teacher’s score also increased with the student’s experience. The same
explanation determined the score as a plié. As the students’ advance, their technique
improves; therefore the teacher’s score increases. Contrary to a plié, the Super Mirror
score is lower than the teacher’s score. Our explanation was that the Super Mirror
system does not detect the angles as well as the plié. Figure 5 represents the assess-
ment of a tendu front. A battement tendu is an extension of the leg [1].
According to the subject-matter expert, a tendu is the most complex out of the
three movements. The teacher assessed the dancers on the following set of criteria
similar to the other two movements: are legs turned out, is weight of the body on the
left foot, is tendu begun with a turned-out heel, are shoulders back, is stomach in, and
is back straight. The teacher’s scores follow the same pattern as the previous two
movements i.e., increases with the level of complexity. We found a similar reaction
where the Super Mirror system was not able to successfully detect the angles. The
system scores show certain anomalies in the second year students. Our explanation
was that the second year students were capable to negatively accustom to the faults of
Fig. 4. Élevé score
the system, although the teacher’s scores were the lowest as expected. Most of the
students recognized the system’s inability to recognize a correct battement tendu. One
student said, “I observed that I got a “hit” if I did some of the movements [battement
tendu front] in a specific way that was inconsistent to ballet principles”.
SUS presented a mean score of 57 for the students and a score of 42.5 for the
teacher which indicated a below average result. A score above 68 would be consid-
ered as above average as presented by Usability.gov [17].
5 Conclusion
The Super Mirror is only capable to assess the partial complexity of the movements’
i.e., the angle of joints to be within certain limits. The teacher score shows a pattern
that is proportional to the level of the student’s experience, while the Super Mirror
shows opposite scoring compared to the teacher when the complexity of movement
increases. Nevertheless, the Super Mirror shows a consistency of scores among the
students for the same movement regardless of the opposition of the teacher that gives
an opportunity to calibrate the system to match the teacher’s scores.
With the know-how of our previous experience in the world of ballet and the
teacher’s input, the following improvements on the Super Mirror are envisioned.
First, the Super Mirror reference model needs to have a fast tuning capability. In other
words, there is a need for an easier capability to calibrate the system for each individ-
ual dancer through a simple user interface. Further, the measured parameters (the
angles of the joints and hips) should be expanded in the direction of the assessment
criteria of the subject-matter expert. Examples of the criteria include detecting if the
weight is distributed equally between both feet or one, if the legs are turned out, and if
the arms are synchronized to the legs. The ballet professor indicated that, “[The Super
Mirror] may be useful if perfected and simplified for use in class. It cannot evaluate
physical predispositions for a classical ballet dancer and other important factors, such
as musicality and dance ability.” At the end of this stage of development, a user inter-
face designed based on Nielsen’s 10 Usability Heuristics would be beneficial to im-
prove the interaction between the dancer and the system. This would allow users to be
able to manipulate the system’s parameters and consequently increase their learning.
The system should always present a visibility of its status by giving familiar terminol-
ogy to the user rather than using system terms. Moreover, the system should provide
more user control, consistency, error prevention, recognition rather than recall, flexi-
bility, adequate error messages, and finally help and documentation [10].
Although initial testing of Super Mirror was not highly conclusive to test the effec-
tiveness and efficiency of the feedback, these types of tools open the door to integrate
a Kinect technology to ballet. Even more, it shows promise. With an adequate im-
provement of the system and a user-controlled capability to calibrate the parameters, a
Kinect-based system has the potential to become a useful tool to students, teachers,
and professionals. In a more advanced stage of development, the level of usability of
the Super Mirror will further increase, if there is a measurement of the speed of
movements, a correlation between the speed of movements and music, a correlation
between the movement of head, arms, and feet, and the measurement of posture and
balance. The most exciting part is the possibility of making a technical and even artis-
tic assessment of the whole performance that could potentially benefit ballet competi-
tions, and remove the bias of subject matter experts. The integration of video stream-
ing, multiple networked Kinect sensors and cloud technology as a one system would
move ballet from “Dance the oldest art, is today but a young scienceto the needs of
21st century ballet.
Acknowledgements. Authors would like to thank Joao Beira and Sebastian Kox for
offering the Super Mirror system for testing and providing technical help. We thank
Prof. Snezana Filipovska, Ph.D. and Prof. Slagjana Spasenovska, M.A. for the in-
sightful input regarding the fundamentals of ballet, through which many of the ideas
in this paper were developed and shaped. We would also like to acknowledge the
students from the State Ballet School, Ilija-Nikolovski Luj from Skopje, Macedonia
who participated in the study.
6 References
1. American Ballet Theatre: American Ballet Theatre Ballet Dictionary,
http://www.abt.org/education/dictionary/index.html
2. Anderson, F., Grossman, T., Matejka, J., Fitzmaurice, G.W.: YouMove: Enhancing
Movement Training with an augmented reality mirror. In: User Interface Software and
Technology (UIST), pp. 311-320. ACM (2013)
3. Dearborn, K., Ross, R.: Dance Learning & the Mirror: Comparison Study of Dance Phrase
Learning With and Without Mirrors: J. Dance. Educ. 6(4), 109-115 (2006)
4. Eaves, D. L., Breslin, G., van Schaik, P., Robinson, E., Spears, I.R.: The Short-Term Ef-
fects of Real-Time Virtual Reality Feedback on Motor Learning in Dance. Presence: Tele-
operators and Virtual Environments. 20(1), 62-77 (2011).
5. Foster, S.L.: Dancing Bodies. Meaning in Motion: New Cultural Studies of Dance. Jane
Desmond, Durham, Duke University Press, pp. 235-257 (1997)
6. Gray, J. A.: The Dance Teacher A Computerized Behavioral Profile: J. Phys. Res. Educ.
Dance. 54(9), 34-35 (1983)
7. Green, J.: Somatic Authority and the Myth of the Ideal Body in Dance Education: Dance.
Res. J. 31(2), 80-100 (1999)
8. Leijen, Ä., Lam, I., Wildschut, L., Robert-Jan, S., Admiraal, W.: Streaming Video to En-
hance Students’ Reflection in Dance Education: Compu. Educ. 52(1), 169-176 (2009)
9. Marquardt, Z., Beira, J., Em, N., Paiva, I., Kox, S: Super Mirror: A Kinect Interface for
Ballet Dancers. In: CHI ’12 Extended Abstracts on Human Factors in Computing Systems,
pp. 1619-1624. ACM (2012)
10. Nielsen Norman Group, http://www.nngroup.com/articles/ten-
usability-heuristics/
11. Radell, S.A.: Mirrors in Dance Class: Help or Hindrance?. In International Association for
Dance Medicine & Science. (2013)
12. Radell, S.A., Adame, D.D., Cole, S.P.: Effect of Teaching With Mirrors on Ballet Dance
Performance: Percept. Motor. Skill. 97, 960-964 (2003)
13. Sukel, K.E., Catrambone, R., Essa, I., Brostow, G.: Presenting Movement in a Computer-
Based Dance Tutor: Int. J. Hum-Comput. Int. 15(3), 433-452 (2003)
14. Synapse - Synapse for Kinect,
http://synapsekinect.tumblr.com/post/6610177302/syn apse
15. Tryplex - the toolkit for collaborative design innovation - Google Project Hosting,
http://code.google.com/p/tryplex/
16. Twitchett, E., Angioi, M., Koutedakis, Y., Wyon, M.: Video Analysis of Classical Ballet
Performance: J. Dance. Med. Sci. 13(4), 124-128 (2009)
17. U.S. Dept. of Health and Human Services. The Research-Based Web Design & Usability
Guidelines, Enlarged/Expanded edition. Washington: U.S. Government Printing Office,
(2006)
... In this work, we focus on the advantages and limitations of the augmented mirror setting, using depth cameras such as Kinect [10,21,8,1,9,11,18,12] or regular cameras [13,19]. In addition, although the goal is different, dance learning experiences design can benefit from existing guidelines in movement based games [14]. ...
... We argue that there is not one solution to fit all and that the possible answers are highly related not only to the devices used but also to the movement domain, dance genre and particular learning objectives. Comparing to a regular mirror, the setting of the Augmented Mirror presents several advantages, such as providing feedback on what can be enhanced in terms of technique or body posture, as if seen from a different spatial perspective [10,21] or different time [13]. Similar solutions have been applied for capturing and evaluating effectively the posture of piano performers [16]. ...
... The augmented mirror paradigm using Kinect for evaluating students performance can be used in dance in two ways: one is to compare the overall performance and closeness of positions and motions in relation to a stored ideal performance [1,9]. The other is to define particular rules and patterns focusing on specific aspects e.g, calculate the posture deviation through defining e.g torso misalignment or rotation to the pelvis rotation and compare with the ideal range [10,21]. Although most of the systems that use the first approach provide specific feedback on body parts, this might not be very accurate due to differences between human bodies, and learning objectives that are aligned with the dance system of teaching and practicing. ...
... It is true that some dance practices heavily rely on the use of the mirror both for supporting orientation and for self-correcting posture and motion. In these cases, the metaphor of the "augmented-mirror" and the use of Kinect [29,30] have shown positive results regarding both usability and effectiveness in self-practice [31,32,33]. Mirror, however, as well as, video and Kinect, unless combined with more than one devices, provide only one 2dimensional perspective of the body and movement. ...
... The mirror, as well as, video and Kinect, unless combined with more than one devices, provide only one 2-dimensional perspective of the body and movement. In these cases, e.g., for ballet, where usually a more traditional teaching approach with the mirror is applied in the physical world, the metaphor of the "augmented mirror" and the use of Kinect [29,30] have shown positive results regarding both usability and effectiveness in self-practice [31,32,33]. In such cases, the students looking at themselves and their posture and correcting it is the objective. ...
Preprint
Full-text available
The last decades the development of whole-body interaction technologies, as well as XR (Extended Reality) technologies , including Augmented, Mixed and Virtual Reality , created a strong potential for embodied and immer-sive experiences to support learning and the use of notation while moving. In our ongoing work, we explore this potential on the user case of familiarizing dance experts and amateurs with movement notation in general and Laban-otation in particular. By applying methodologies of user-centered design, including co-design workshops with notation and dance experts, interviews, focus-groups, questionnaires supporting the iterative design of our prototype, we focus on how we can meaningfully transfer the concepts related to space from notation to full body interaction instructions. So far we have developed two prototypes following two paradigms: a. the augmented mirror paradigm using Kinect and b. the immersive paradigm using HTC-Vive, that we have used as technology probes to interact with dance experts in the context of our co-design work. We reflect on this experimentation and we document the emerging challenges of transferring a symbolic language that is meant to be transmitted through paper, into spatial semantic queues. We discuss the challenges that arise between the gaps of symbolically referring to space, within a rich conceptual framework, such as Laban Movement Analysis (LMA), experiencing space kinaesthetically, and transferring these into a digital experience, always within the limitations of the current technologies.
... So far, many motion-learning systems have been developed to help novice learners learn motion and have been applied in various fields (e.g., ballet [20,23,61], factory operations [7], sports training assistance [15], Tai-Chi Chuan exercises [12,23,36] and physical rehabilitation [3,10,56,57]). Considerable efforts have been made to provide lightweight, intuitive, and effective systems for users. ...
Article
Full-text available
Conventional motion tutorials rely mainly on a predefined motion and vision-based feedback that normally limits the application scenario and requires professional devices. In this paper, we propose VoLearn, a cross-modal system that provides operability for user-defined motion learning. The system supports the ability to import a desired motion from RGB video and animates the motion in a 3D virtual environment. We built an interface to operate on the input motion, such as controlling the speed, and the amplitude of limbs for the respective directions. With exporting of virtual rotation data, a user can employ a daily device (i.e., smartphone) as a wearable device to train and practice the desired motion according to comprehensive auditory feedback, which is able to provide both temporal and amplitude assessment. The user study demonstrated that the system helps reduce the amplitude and time errors of motion learning. The developed motion-learning system maintains the characteristics of high user accessibility, flexibility, and ubiquity in its application.
... While not for sports training, there have also been prior investigations of using motion tracking and visualizing virtual avatars for motion training for those interested in ballet dancing [19,35] and playing a stringed instrument [22]. In comparison, our system uses a 'mirror display' that allows users to watch their own body reflected in the mirror. ...
Article
Full-text available
This paper describes the user experience (UX) design for a smart-mirror-based personalized training system which aims to help people live a healthy life. A number of researchers and companies have developed fitness systems that use a virtual coach which shows the user with actions they should perform. However such systems can be difficult to accurately follow the virtual guide’s motions and there are also limitations in the feedback provided to inform users of their correct body posture. This is because most systems are designed for users to simply watch and follow a character’s motions (poses) from a third person perspective. In our smart mirror-based system, users are able to follow the exercise-postures of a virtual professional trainer shown in a first person viewpoint and receive coaching through a real-time motion correction. This is based on a predefined database of the trainer’s postures gained from motion-capture technology, and it is personalized to the user’s body 3D model acquired through an instant one-time scanning process. In this paper, we report on the UX design of our system, mainly focusing on understandable visualization, intuitive interaction, attractive information representation and easily acceptable user scenarios. Through a series of user studies, we analyze and discuss user friendliness, information comprehension, and user satisfaction as they relate to our design. In addition, we also assess the similarity and effectiveness of the proposed system compared to traditional personalized training (PT) at a gym. Based on the implications, we discuss future research directions for improving the user experience of the smart-mirror-based PT system.
... Anderson et al. discuss design guidelines for movement training systems based on previous literature which include [1]: reducing visual complexity [51], motivating the user [3], adapting guidance as the user learns [42], providing summary feedback [59], and allowing users to progress at their own pace [60]. They also propose to leverage domain knowledge in the authoring system, however our approach contrasts systems designed for specifc physical activities, examples of which include ballet [49], tai chi [19], and weight-lifting [29]. ...
Conference Paper
Full-text available
Videos are a convenient platform to begin, maintain, or improve a fitness program or physical activity. Traditional video systems allow users to manipulate videos through specific user interface actions such as button clicks or mouse drags but have no model of what the user is doing and are unable to adapt in useful ways. We present adaptive video playback, which seamlessly synchronizes video playback with the user's movements, building upon the principle of direct manipulation video navigation. We implement adaptive video playback in Reactive Video, a vision-based system which supports users learning or practicing a physical skill. The use of pre-existing videos removes the need to create bespoke content or specially authored videos, and the system can provide real-time guidance and feedback to better support users when learning new movements. Adaptive video playback using a discrete Bayes and particle filter are evaluated on a data set collected of participants performing tai chi and radio exercises. Results show that both approaches can accurately adapt to the user's movements, however reversing playback can be problematic.
... The average score of the systems evaluated with SUS is 68. We also compare our results with the Super Mirror [19]; a study [20] proposed the evaluation of this system, and its SUS results presented a score of 57 for students and 42.5 for evaluated teachers. As it can be seen in Table II, the BalletVR outperforms Super Mirror on SUS. ...
... Therefore, the system doesn't provide an overall result for the student's performance, but rather it characterizes every specific move as correct or incorrect. The usability of Super Mirror has been reported by Trajkova & Ferati [70]. Whatever dance toolbox [7] provides corrective feedback as well, since there are specific tasks that the student should complete in order to move to the next one. ...
Article
Full-text available
Motion Capture and whole-body interaction technologies have been experimentally proven to contribute to the enhancement of dance learning and to the investigation of bodily knowledge, innovating at the same time the practice of dance. Designing and implementing a dance interactive learning system with the aim to achieve effective, enjoyable, and meaningful educational experiences is, however, a highly demanding interdisciplinary and complex problem. In this work, we examine the interactive dance training systems that are described in the recent bibliography, proposing a framework of the most important design parameters, which we present along with particular examples of implementations. We discuss the way that the different phases of a common workflow are designed and implemented in these systems, examining aspects such as the visualization of feedback to the learner, the movement qualities involved, the technological approaches used, as well as the general context of use and learning approaches. Our aim is to identify common patterns and areas that require further research and development toward creating more effective and meaningful digital dance learning tools.
Book
Full-text available
The INTERACT Conferences are an important platform for researchers and practitioners in the field of human-computer interaction (HCI) to showcase their work. They are organised biennially by the International Federation for Information Processing Technical Committee on Human–Computer Interaction (IFIP TC13), a committee of 30 member national societies and 9 Working Groups. The 17th IFIP TC13 International Conference on Human-Computer Interaction (INTERACT 2019) took place during 2-6 September 2019 in Paphos, Cyprus. The conference was held at the Coral Beach Hotel Resort, and was co-sponsored by the Cyprus University of Technology and Tallinn University, in cooperation with ACM and ACM SIGCHI. With an emphasis on inclusiveness, these conferences work to lower the barriers that prevent people in developing countries from participating in conferences. As a multidisciplinary field, HCI requires interaction and discussion among diverse people with different interests and backgrounds. This volume contains the Adjunct Proceedings to the 17th INTERACT Conference, and comprises a series of papers from the workshops. It follows the INTERACT Conference tradition of the publication of adjunct proceedings by a University Press which has a connection to the conference itself. This tradition has been established to enhance the outreach and reputation of the University Press chosen. For INTERACT 2019, both the Conference Program Chair, Dr Fernando Loizides, and the Adjunct Proceedings Chair of the conference, Dr Usashi Chatterjee, work at Cardiff University which is the home of Cardiff University Press.
Article
In recent years, the Kinect-based systems that enable users to be trained without the participation of teachers have been widely used in the field of physical education. In this paper, we propose a novel technique that helps the Kinect-based training system to select the subsequential training material for the users according to their realtime performance. An algorithm based on the Hidden Markov Model is demonstrated to generate the customized training pathes(training curriculums) for each individual. We present an edutainment gaming system for children in order to illustrate the feasibility of the training method. A user study of 10 children participants is conducted and the results show that the proposed technique enhances the effect of physical training significantly.
Article
Full-text available
Recently, the field of somatics has provided dance scholarship with a growing body of literature. Research has been conducted in the areas of dance science and education. Dance medicine and somatic education scholars have been able to help dance teachers find ways of using the body effectively in technique classes. For example, Glenna Batson (1990, 1993) and Sylvie Fortin (1993, 1995) have investigated the role of somatics in the improvement of technical dance skills. Further, Fortin (1995) has investigated learning and teaching theory as applied to somatics and dance pedagogy. As a somaticist and educator, I acknowledge and appreciate the impressive work conducted by these researchers and educators. However, my current work moves somatics into another direction. I am interested in looking at somatic theory and practice through a sociocultural lens. I am particularly interested in investigating how the body is shaped by society and the dance world, in which performers constantly strive for perfection.
Article
Full-text available
We propose the Super Mirror, a Kinect-based system that combines the functionality of studio mirrors and prescriptive images to provide the user with instructional feedback in real-time. In this study, we developed a working prototype of this system, which records ballet movements (also called positions and poses), captures live motion, and shows the difference between the two.
Article
Full-text available
This paper presents an evaluation case study that describes the experiences of 15 students and 2 teachers using a video-based learning environment, DiViDU, to facilitate students’ daily reflection activities in a composition course and a ballet course. To support dance students’ reflection processes streaming video was applied as follows: video editing and viewing for facilitating students in describing their practice; writing online self-assessments about the experiences captured on video to support students in evaluating their practice; online peer-feedback activities concerning the recorded practice for facilitating students in learning from multiple perspectives. In the composition course eight students reflected on their choreographic work, which was performed by their peer students. In the ballet course seven students reflected on themselves practicing the ballet technique. Data about the streaming video facilitation were collected after the completion of the reflection assignments using semi-structured interviews. The results revealed that students in both courses considered steaming video as effective for carrying out self-evaluations. The usefulness of video and online peer-feedback for other reflection processes differed among the courses in students’ view. The teachers considered streaming video generally useful for all the reflection processes of their students; however they also indicated some shortcomings.
Article
Full-text available
This article addresses how to present movement information to learners as part of a larger project on developing a nonconventional computational system that teaches ballet. The requirements of such a system are first described, and then discoveries regarding the first requirement, presenting movement to a user, are discussed. Background research regarding how people learn movement, hypotheses concerning presenting movement with computer animation versus videotape, and an experiment testing those hypotheses are presented. The experiment required individuals to perform movements after viewing them in one of the formats. Each participant viewed a movement sequence multiple times and then was evaluated on his or her performance of that movement by two expert judges. Animations resulted in higher performance ratings for individuals with some previous dance experience. Format did not affect performance for other learners. This result implies that domain knowledge interacts with presentation format in learning ballet. These results will influence the design and implementation of a computer-based dance tutor under development, and they point to several interesting research directions, including exploring the effects of multimodal sensory presentations and prior knowledge in learning movement.
Article
Full-text available
Does virtual reality (VR) represent a useful platform for teaching real-world motor skills? In domains such as sport and dance, this question has not yet been fully explored. The aim of this study was to determine the effects of two variations of real-time VR feedback on the learning of a complex dance movement. Novice participants (n == 30) attempted to learn the action by both observing a video of an expert's movement demonstration and physically practicing under one of three conditions. These conditions were: full feedback (FULL-FB), which presented learners with real-time VR feedback on the difference between 12 of their joint center locations and the expert's movement during learning; reduced feedback (REDUCED-FB), which provided feedback on only four distal joint center locations (end-effectors); and no feedback (NO-FB), which presented no real-time VR feedback during learning. Participants' kinematic data were gathered before, immediately after, and 24 hr after a motor learning session. Movement error was calculated as the difference in the range of movement at specific joints between each learner's movement and the expert's demonstrated movement. Principal component analysis was also used to examine dimensional change across time. The results showed that the REDUCED-FB condition provided an advantage in motor learning over the other conditions: it achieved a significantly greater reduction in error across five separate error measures. These findings indicate that VR can be used to provide a useful platform for teaching real-world motor skills, and that this may be achieved by its ability to direct the learner's attention to the key anatomical features of a to-be-learned action.
Article
Full-text available
Video analysis of classical ballet to date has been largely limited to examining the artistic elements of choreography. The aim this study was to employ a method of video analysis to describe the physiological demands of classical ballet performance and to examine differences between artists, soloists, and principal dancers. Forty-eight performances [male = 24, female = 24; artists (corps de ballet) = 16, soloists = 16, principals = 16] were analyzed in four fields: work intensity, body movement, partner work, and number of transitory movements occurring per minute. Statistical analysis revealed significant differences between ranks in two intensity bands: time at rest (p < 0.05) and time performing at moderate intensity (p < 0.05), with soloists and principals resting for 75.2 +/- 15.1% and 53 +/- 24.1% of the total performance, respectively (p < 0.05). Principals also spent a significantly greater percentage of time at moderate intensity than both soloists and artists (p < 0.05). Significant differences between males and females (p < 0.05) were seen in the number of lifting and supporting movements performed. It was concluded that classical ballet is an intermittent form of exercise, utilizing both aerobic and anaerobic energy systems, a finding that supports previous studies. The demands of the performances analyzed varied according to role. Therefore, it was also concluded that video analysis can help provide a basis for rank-specific supplemental training.
Conference Paper
YouMove is a novel system that allows users to record and learn physical movement sequences. The recording system is designed to be simple, allowing anyone to create and share training content. The training system uses recorded data to train the user using a large-scale augmented reality mirror. The system trains the user through a series of stages that gradually reduce the user's reliance on guidance and feedback. This paper discusses the design and implementation of YouMove and its interactive mirror. We also present a user study in which YouMove was shown to improve learning and short-term retention by a factor of 2 compared to a traditional video demonstration.
Article
In the dance studio, the mirror can play a large role in the dancer's learning process. Research on learning and memory shows that reducing the amount of feedback during training enhances long-term motor skill retention and that more externally focused attention may aid performance. Research testing the effectiveness of training with a mirror as a source of external feedback is scarce. This study involved 22 college dance students between the ages of 17 to 22 who had studied dance for an average of 13 years. The purpose of the research was to examine how learning and performing dance combinations in mirrored and non-mirrored situations affected student's ability to remember and replicate movement sequences one week later. The testing occurred over two weeks during which time subjects learned and performed the given movement phrases and completed a questionnaire. Collected data included electronic scoring of footwork accuracy and video analysis and scoring by five dance teachers. Results showed that initial learning in a mirrored setting produced better performances in the retest one week later.
Article
This study assessed the effect of mirrors on the dance performance of beginning college ballet students in the classroom setting, using an evaluation methodology developed for this study. 13 women enrolled in one ballet class were taught using mirrors, 14 women in a second beginning ballet class were taught without mirrors. Both classes were taught by the same instructor. All students were videotaped performing the same adagio and grand allegro phrase during Weeks 5 and 14 of the 14-wk. term. At the end of the semester two ballet teachers viewed the videotapes for both classes. One evaluator was the instructor, and the other was a blind reviewer who had no knowledge of which was the mirror and nonmirror class during the evaluation process. They were instructed to choose a score for each dancer with anchors of 1: low skill and 5: high skill for both the adagio and allegro phrases. High interrater reliability was noted for both the adagio and allegro phrase data. In the nonmirror class, there was a significant increase in adagio scores, but no significant increases in adagio and allegro scores for the mirror class. These results suggest that the use of the mirror in a ballet classroom may negatively affect skill acquisition of the dancer.