Conference Paper

Towards a Pragmatic and Theory-Driven Framework for Multimodal Collaboration Feedback

Conference Paper

Towards a Pragmatic and Theory-Driven Framework for Multimodal Collaboration Feedback

If you want to read the PDF, try requesting it from the authors.

Abstract

This paper proposes an overarching framework for automated collaboration feedback that bridges theory and tool as well as technology and pedagogy. This pragmatic and theory-driven framework guides our thinking by outlining the components involved in converting theoretical collaboration constructs into features that can be automatically extracted and then converted into actionable feedback. Focusing on the pedagogical components of the framework, the constructs are validated by mapping them onto a selection of multi-disciplinary collaboration frameworks. The resulting behavioral indicators are then applied to measure collaboration in a sample scenario and those measurements are then used to exemplify how feedback analytics could be calculated. The paper concludes with a discussion on how those analytics could be converted into feedback for students and the next steps needed to advance the technological part of the framework.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Collaboration is one of the important 21st-century skills. It can take place in remote or co-located settings. Co-located collaboration (CC) is a very complex process that involves subtle human interactions that can be described with indicators like eye gaze, speaking time, pitch, and social skills from different modalities. With the advent of sensors, multimodal learning analytics has gained momentum to detect CC quality. Indicators (or low-level events) can be used to detect CC quality with the help of measurable markers (i.e., indexes composed of one or more indicators) which give the high-level collaboration process definition. However, this understanding is incomplete without considering the scenarios (such as problem solving or meetings) of CC. The scenario of CC affects the set of indicators considered: for instance, in collaborative programming, grabbing the mouse from the partner is an indicator of collaboration; whereas in collaborative meetings, eye gaze, and audio level are indicators of collaboration. This can be a result of the differing goals and fundamental parameters (such as group behavior, interaction, or composition) in each scenario. In this review, we present our work on profiles of indicators on the basis of a scenario-driven prioritization, the parameters in different CC scenarios are mapped onto the indicators and the available indexes. This defines the conceptual model to support the design of a CC quality detection and prediction system.
Conference Paper
Full-text available
Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members' embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly affordable to instrument work spaces to study teamwork and groupwork. The possibility of visualising key aspects of a collaboration has huge potential for both academic and professional learning, but a frontline challenge is the enrichment of quantitative data streams with the qualitative insights needed to make sense of them. In response, we introduce the concept of collaboration translucence, an approach to make visible selected features of group activity. This is grounded both theoretically (in the physical, epistemic, social and affective dimensions of group activity), and contextually (using domain-specific concepts). We illustrate the approach from the automated analysis of healthcare simulations to train nurses, generating four visual proxies that fuse multimodal data into higher order patterns.
Conference Paper
Full-text available
The Presentation Trainer is a multimodal tool designed to support the practice of public speaking skills, by giving the user real-time feedback about different aspects of her nonverbal communication. It tracks the user's voice and body to interpret her current performance. Based on this performance the Presentation Trainer selects the type of intervention that will be presented as feedback to the user. This feedback mechanism has been designed taking in consideration the results from previous studies that show how difficult it is for learners to perceive and correctly interpret real-time feedback while practicing their speeches. In this paper we present the user experience evaluation of participants who used the Presentation Trainer to practice for an elevator pitch, showing that the feedback provided by the Presentation Trainer has a significant influence on learning.
Conference Paper
Full-text available
Recent research has established the potential for virtual characters to estab- lish rapport with humans through simple contingent nonverbal behaviors. We hy- pothesized that the contingency, not just the frequency of positive feedback is crucial when it comes to creating rapport. The primary goal in this study was evaluative: can an agent generate behavior that engenders feelings of rapport in human speakers and how does this compare to human generated feedback? A secondary goal was to an- swer the question: Is contingency (as opposed to frequency) of agent feedback crucial when it comes to creating feelings of rapport? Results suggest that contingency mat- ters when it comes to creating rapport and that agent generated behavior was as good as human listeners in creating rapport. A "virtual human listener" condition per- formed worse than other conditions.
Article
Full-text available
The analysis of the process of collaboration is a central topic in current CSCL research. However, defining process characteristics relevant for collaboration quality and developing instruments capable of assessing these characteristics are no trivial tasks. In the assessment method presented in this paper, nine qualitatively defined dimensions of collaboration are rated quantitatively: sustaining mutual understanding, dialogue manage- ment, information pooling, reaching consensus, task division, time management, technical coordination, reciprocal interaction, and individual task orientation. The data basis for the development of these dimensions was taken from a study in which students of psychology and medicine collaborated on a complex patient case via a desktop-videoconferencing system. A qualitative content analysis was performed on a sample of transcribed collaboration dialogue. The insights from this analysis were then integrated with theoretical considerations about the roles of communication, joint information processing, coordina- tion, interpersonal relationship, and motivation in the collaboration process. The resulting rating scheme was applied to process data from a new sample of 40 collaborating dyads. Based on positive findings on inter-rater reliability, consistency, and validity from this evaluation, we argue that the new method can be recommended for use in different areas of CSCL.
Article
Full-text available
Our research aims to identify children’s communicative strategies when faced with the task of solving a geometric puzzle in CSCL contexts. We investigated how to identify and trace distributed cognition in problem-solving interactions based on discursive cohesion to objects, participants, and prior discursive content, and geometric and cooperative concepts. We report on the development of a method of coding and representation of verbal and gestural content for multimodal interactional data and initial application of this framework to a microethnographic case study of two small groups of 7 and 8-year-old learners solving tangram manipulatives in physical and virtual desktop settings. We characterize the establishment of shared reference points as “coreferences” which cohere on object, para, and meta-levels through both gesture and speech. Our analysis foregrounds how participants establish common referential ground to facilitate collaborative problem solving with either computer-supported or physical puzzles. Using multimodal analysis and a theoretical framework we developed to study interactional dynamics, we identified patterns of focus, dominance, and coalition formation as they relate to coreferentiality on multiple levels. Initial findings indicate increased communication and cohesion to higher-level principles in the virtual tangram puzzle-solving setting. This work contributes to available models of multimodal analysis of distributed cognition using current manipulative technologies for early childhood mathematics education.
Article
Full-text available
Social interdependence theory is a classic example of the interaction among theory, research, and practice. The premise of the theory is that the way in which goals are structured determines how individuals interact, which in turn creates outcomes. Over 750 research studies have been conducted in the past 11 decades on the relative merits of cooperative, competitive, and individualistic efforts and the conditions under which each is appropriate. These studies have validated, modified, refined, and extended the theory. Social interdependence theory has been widely applied, especially in education. The applications have resulted in revisions of the theory and the generation of considerable new research.
Conference Paper
Many pedagogical models in the field of learning analytics are implicit and do not overtly direct learner behavior. While this allows flexibility of use, this could also result in misaligned practice, and there are calls for more explicit pedagogical models in learning analytics. This paper presents an explicit pedagogical model, the Team and Self Diagnostic Learning (TSDL) framework, in the context of collaborative inquiry tasks. Key informing theories include experiential learning, collaborative learning, and the learning analytics process model. The framework was trialed through a teamwork competency awareness program for 14 year old students. A total of 272 students participated in the program. This paper foregrounds students' and teachers' evaluative accounts of the program. Findings reveal positive perceptions of the stages of the TSDL framework, despite identified challenges, which points to its potential usefulness for teaching and learning. The TSDL framework aims to provide theoretical clarity of the learning process, and foster alignment between learning analytics and the learning design. The current work provides trial outcomes of a teamwork competency awareness program that used dispositional analytics, and further efforts are underway to develop the discourse layer of the analytic engine. Future work will also be dedicated to application and refinement of the framework for other contexts and participants, both learners and teachers alike.
The need to collaborate in the fields of business, health care, and technology is not a new concept. Educational institutions, however, have been slow to recognize the need to teach collaboration skills (Brownell & Walther-Thomas, 20026. Brownell , M. T. and Walther-Thomas , C. 2002. An interview with Dr. Marilyn Friend. Intervention in School and Clinic, 37: 223–228. View all references; Jackson, 200415. Jackson, S. (2004, November 4). Leading the modern college or university: Engaging higher education in social change. Presented at the 2004 Summit for Women of Color Administrators and Faculty American Council on Education, Brown University, Providence, RI http://www.rpi.edu/president/speeches/ps110504-ace.html (Accessed: 16 January 2006). View all references). This article focuses on the development of the Collaboration Self-Assessment Tool (CSAT). The identification and practice of specific collaboration skills needed in a student teaching experience are emphasized. The article uses a case-study approach to describe how the tool can be used to enhance collaboration between a cooperating teacher and a teacher candidate, paving the way for a more successful and meaningful student-teaching experience.
Collaborative Learning Analytics. International Handbook of Computer-Supported Collaborative Learning
  • Alyssa Friend Wise Simon Knight
  • Simon Buckingham Shum
Bridging the Gap Between Theory and Tool: A Pragmatic Framework for Multimodal Collaboration Feedback
  • Maurice Boothe
  • Collin Yu
  • Xavier Ochoa
Maurice Boothe, Jr., Collin Yu, and Xavier Ochoa. 2021. Bridging the Gap Between Theory and Tool: A Pragmatic Framework for Multimodal Collaboration Feedback. In Companion Proceedings 11th International Conference on Learning Analytics & Knowledge (LAK '21). Society for Learning Analytics Research, Irvine, CA, 398-401. https://www.solaresearch.org/wp-content/uploads/2021/ 04/LAK21_CompanionProceedings.pdf
Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving
  • Oecd
OECD. 2017. PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. In PISA 2015 Assessment and Analytical Framework. OECD Publishing, Paris, France, 131-188. https://www.oecd-ilibrary.org/education/pisa-2015-assessment-and-analytical-framework/pisa-2015-collaborative-problemsolving-framework_9789264281820-8-en
International Handbook of Computer-Supported Collaborative Learning
  • Analytics Collaborative Learning
Collaborative Learning Analytics. International Handbook of Computer-Supported Collaborative Learning ([n. d.]), 19.