Conference PaperPDF Available

A Tale of Two Nurses: Studying Groupwork in Nurse Training by Analyzing Taskwork Roles, Social Interactions, and Self-Efficacy



Modern healthcare requires the coordination of a team of professionals with complementary skillsets. To help facilitate teamwork, healthcare professionals, such as nurses, undergo rigorous training of their clinical skills in team settings. In this paper, we analyze a mixed-reality, simulation-based training exercise involving three nurses in a hospital room. We perform multimodal interaction analysis to contrast strategies used in two cases where the patient expressed doubts about their medical care. By analyzing these strategies and comparing them to the student nurses' self-reflections, we show connections among the nurses' clinical roles, their self-efficacy, and their teamwork.
A Tale of Two Nurses: Studying Groupwork in Nurse Training by
Analyzing Taskwork Roles, Social Interactions, and Self-Efficacy
Caleb Vatral, Clayton Cohn, Eduardo Davalos, Gautam Biswas
{caleb.m.vatral, clayton.a.cohn, eduardo.davalos.anaya, gautam.biswas}
Institute for Software Integrated Systems, Vanderbilt University
Madison Lee, Daniel Levin
{madison.j.lee, daniel.t.levin}
Peabody College, Vanderbilt University
Eric Hall, Jo Ellen Holt
{eric.s.hall, jo.e.holt}
School of Nursing, Vanderbilt University
Abstract: Modern healthcare requires the coordination of a team of professionals with comple-
mentary skillsets. To help facilitate teamwork, healthcare professionals, such as nurses, undergo
rigorous training of their clinical skills in team settings. In this paper, we analyze a mixed-
reality, simulation-based training exercise involving three nurses in a hospital room. We per-
form multimodal interaction analysis to contrast strategies used in two cases where the patient
expressed doubts about their medical care. By analyzing these strategies and comparing them
to the student nurses’ self-reflections, we show connections among the nurses’ clinical roles,
their self-efficacy, and their teamwork.
In modern healthcare, diverse teams of individuals work closely to provide optimal patient care. This complex
orchestration requires each team member to employ strong individual and teamwork skills. Therefore, effective
training of such skills is critical to ensure success in complex healthcare environments. Often, skill training takes
place in mixed-reality simulation-based training, where the nurses can practice their clinical and teamwork skills
in realistic environments that are safe and repeatable. This allows trainees to develop a strong sense of self-effi-
cacy in teamwork and self-confidence in their clinical skills that can be transferred to real clinical settings and
improve patient outcomes (Hustad et al., 2019). In this work, we use a cognitive ethnographic approach to conduct
a case study of three nurses training in a manikin-based simulation. By contrasting two situations where the stu-
dent nurses address their patient’s doubts regarding the medical care being administered, we examine how the
nurses’ interactions with the patient relate to their roles and self-efficacy.
Three nurses worked on a training exercise in a simulated hospital room with standard medical equipment and a
high-fidelity manikin as the patient. The simulation utilized a predefined script, but a trained instructor guided the
simulation by observing the nurses’ actions. The nurses were tasked with evaluating the patient and performing
the prescribed medical care to alleviate the conditions of an adult patient who had persistent coughing, difficulty
breathing, and pleural pain. The goal was to administer two prescribed medications to relieve the patient’s symp-
toms. All the student nurses who participated in the study provided their informed consent to collect video, audio,
and eye tracking data, and the study was approved by the university Institutional Review Board. From the multi-
modal data collected, we derived two additional records by applying machine learning algorithms to the raw
audio: (1) a simple text transcription of dialogue using the Otter automated transcription software (with manual
correction of errors); and (2) an emotion recognition deep learning transformer model to the audio, which pro-
duced a 3-dimensional vector of the nurses’ arousal, dominance, and valence for each utterance (Wagner et al.,
2022). In this paper, we focus on the dominance value, defined as the degree of control exerted by the nurses.
This concept is closely related to the speaker’s confidence.
Using the multimodal data records, our research team conducted interaction analysis over six sessions
(Hall & Stevens, 2015). The first sessions focused on basic interpretation of nurses’ tasks and roles, guided by a
cognitive task analysis of the nursing domain (Vatral et al., 2022). The next sessions focused on justifying the
nurses’ actions, using the video, audio, and textual transcriptions. We also used eye-tracking data to help disam-
biguate the nurses’ focus and attention, especially when they were not speaking. The final sessions focused on
analysis of the nurses’ emotions and self-efficacy, using their speech and eye gaze to identify instances of social
referencing, doubt, and questioning. We used automated speech emotion to add further evidence to the nurses’
feelings (Wagner et al., 2022). Our analysis focused on two contrasting clips from the complete simulation.
To compare and validate the insights generated by multimodal interaction analysis, we used data col-
lected from a post-simulation guided student reflection designed to promote metacognitive awareness. We first
presented students with their own egocentric eye-tracking footage from the simulation. Students then rewatched
this footage while being asked to segment the simulation into meaningful event units (Zacks & Swallow, 2007).
After event segments were marked, students were shown each marked event and responded to six reflection ques-
tions based on that event. Of particular interest to this paper were three questions: (1) the degree to which they
worked individually versus as a team during each segment; (2) their level of self-efficacy during each segment;
(3) self-reported confidence for the simulation overall.
Initial Role Assignment
During the pre-briefing before the training exercise, the instructor assigned high-level roles to the three partici-
pating nurses. Two were assigned the role of nurses, hereafter referred to as nurse 1 and nurse 2. The third student
was assigned the role of care partner, i.e., a type of unlicensed professional who can assist with a restricted subset
of clinical tasks. Soon after the scenario began and the students introduced themselves to the patient, the students
self-assigned themselves tasks based on their assigned roles and understanding of the situation provided by the
instructor in the pre-briefing. Nurse 1 began to focus on patient assessment (information gathering) and immediate
stabilization tasks. This included taking vital signs, adjusting the patient's bed, and starting to administer oxygen.
Nurse 2 began to focus on intervention treatment by retrieving the patient's medications. The care partner assisted
nurse 1 by retrieving equipment for oxygen delivery and reading aloud relevant information from the patient's
chart. From these initial self-assigned tasks, clear differences emerged between nurse 1 and nurse 2. While
nurse 1 focused very directly on the patient, nurse 2 was narrowly focused on administering medications, and did
not interact with the patient at all other than introducing herself at the patient's request. In addition, nurse 1 was
team-oriented from the outset of the simulation, often asking the care partner for assistance. Nurse 2 worked more
independently, leaving the bedside to go retrieve the medication, and then attempting to administer the medication
on her own. This attempt at administering medication (i.e., performing an intervention) with little communication
with the patient or the other nurses resulted in the patient expressing doubt about nurse 2’s qualifications to ad-
minister the chosen medication using a nebulizer. This turned out to be a catalyst for a change in nurse 2's behavior
and she became more team-oriented, as we will describe in the next section.
Addressing Patient Doubt
Instance 1
After nurse 2 retrieved the nebulizer from the pharmacy, she attempted to administer it to the patient. However,
the patient questioned nurse 2’s qualifications to perform the task. Figure 1 (left) shows the transcript of this
encounter; nurse 2 and the patient went back and forth disagreeing on nurse 2’s qualifications to administer the
nebulizer. Eventually, nurse 2 relented and called for a respiratory therapist to administer the nebulizer.
Nurse 2 was not team-oriented, and she attempted to perform the procedure by herself with very little
communication with her team and the patient. Early in the disagreement with the patient, nurse 2 used I-language
to communicate. It is not until nurse 2 began to doubt herself that she switched to using team-oriented we-lan-
guage. Initially there was a rise in nurse 2’s speech dominance, then she began to doubt herself and asked nurse
1 for confirmation (“Right?”). At this point, her dominance suddenly drops and continues to decrease until the
end of the segment. This clearly shows that nurse 2 intended to handle her tasks by herself without much commu-
nication with the other nurses and the patient. When challenged by the patient, nurse 2's confidence wavered, and
she switched to a more team-oriented approach. As this switch occurred, the care partner de-escalated the situation
by interjecting that they should call for assistance. The change to a team-oriented approach continued, and nurse
2 often contributed by helping her teammates when they were unsure of themselves. She made more explicit
suggestions for role assignment among the team members.
Instance 2
Later in this scenario, the nurses needed to administer a second medication intravenously. After computing the
dosage, nurse 1 prepared to inject the medication into the IV tubing, but the patient again expressed doubt, won-
dering what the medication was for. A transcript of the interaction is shown in Figure 1 (right).
In this case, nurse 1’s approach differed from nurse 2’s. Nurse 1 was quick to admit that she was unsure
about the medication, and immediately asked her team if they knew. This was consistent with the behavior nurse
1 had exhibited throughout the scenario, often asking her teammates for information or help. Her measured speech
dominance was also consistent with this team-oriented strategy. Nurse 1 began this interaction with low domi-
nance, unsure about the medication and how to answer the patient's question. Her dominance scores increased
when her team suggested calling the pharmacist or looking up the information. This team-oriented approach raised
nurse 1’s confidence temporarily; but her dominance decreased again after she looked up and read verbatim the
description of the medication from an electronic search. This combined with her low dominance at the end of the
situation suggests that she was still unsure about the medication. This low confidence prompted nurse 2 to interject
with the correct interpretation of the medication’s use, resolving the patient’s doubts.
Comparing Multimodal Interpretations to Self-Report
We compared the interpretation of student behaviors from the previous section with the students post-simulation
reflections. Since these reflection ratings and interpretations were given by the students themselves, they represent
a way of comparing and validating our analysis of the nurses’ behaviors against their own judgments.
The results from post-simulation reflections match well with the analysis previously presented. During
her reflection, nurse 2 recognized the same individual orientation in her taskwork behavior that we interpreted,
stating, “I tend to get focused in on get[ting] tasky things, such as medication administration, done before worry-
ing about the patient interactions. This early individual focus was again confirmed by nurse 2’s reported low
team-orientation score, rating herself a mean of 2.0/5. In contrast, our analysis of nurse 1 suggested she took on a
more team-oriented focus early in the simulation. However, nurse 1’s reported team-orientation score was lower
than what our analysis inferred, rating herself a mean of 2.0/5 during this early part of the simulation. This dis-
crepancy between our analysis and her self-reported scores could be explained by nurse 1’s lack of self-confi-
dence. She rated herself a 2.5/5 for overall confidence during the simulation, indicating uncertainty in her abilities,
and may have been unsure of her performance as a team member during this segment. This presents the oppor-
tunity to provide more reassuring and positive feedback in a debriefing to make the nurse feel more confident and
continue to build on her teamwork processes.
Instance 1
Instance 2
(Removes oxygen tube and begins
to cover PATIENT face with a
nebulizer mask)
(Begins to hook up the medication syringe
into the patient’s IV tubing.)
Are you a respiratory therapist?
What’s that for?
No, I’m not. I’m a nurse though.
Umm. (Looks up at NURSE 2 and CARE
PARTNER.) Do you guys know exactly
what this is for?
You know how to give nebulizer
Well, I don’t want you to give it to me if you
don’t know what it’s for! What if its arsenic?
Yes. (Long pause. NURSE 2 looks
up a NURSE 1.) Right?
We can call the pharmacist.
Yeah, let me call the pharmacist. (Walks
away from the bedside.)
Yes, I do.
You can’t just look it up?
The last time I did one they called
the respiratory therapist.
Yeah, I could just look it up. Let me do that.
We can call them if that makes
you feel more comfortable.
Well, are you licensed to give neb-
ulizer medications?
It’s a steroid that can treat inflammation, se-
vere allergies, flares of chronic illnesses.
No, we’re not. Do you want to
So, this is an anti-inflammatory that should
help your lungs.
We can call.
Yeah, we’ll call.
Figure 1. Transcript of the two instances of patient doubt during the simulation.
Next, the simulation reflections also supported our analyses of doubt in instance 1. Our analysis shows
that nurse 2 entered this scenario with the intention of handling the situation herself. After being challenged by
the patient, her confidence wavered, and she adopted more team-oriented behavior. During this instance, nurse 2
self-reported working as mostly individual, rather than as part of a team (2/5 on a Likert scale). However, after
this segment, we saw a consistent increase in her team-orientation rating with a moderate positive correlation with
time (𝑟ℎ𝑜 = 0.46). Consistent with our analysis, nurse 2 changed her behavior to a teamwork orientation.
Finally, the simulation reflections were also supportive of our analysis of doubt in instance 2. Our results
showed that nurse 1 entered this instance with low self-efficacy, and she relied on her teammates to resolve the
situation. This is consistent with her reflection, where she reported working Mostly as a team during this seg-
ment (4/5 on a Likert scale). In addition, her low confidence in this scenario is consistent with both her reported
self-efficacy during this segment at 3.3/5 and overall confidence at 2.5/5. Finally, our inference that nurse 1 relied
on her team because of her low self-efficacy was also confirmed by her reflection, where we saw a moderate
negative correlation between nurse 1’s self-efficacy and team-orientation (𝑟ℎ𝑜 = −0.54). In fact, across all three
nurses we saw this same pattern of higher reliance on the team when self-efficacy was low (𝑟ℎ𝑜 = −0.33).
Overall, a major conclusion from these results for nursing instruction is that students’ self-efficacy and teamwork
are inversely correlated. At first glance, this is reasonable, especially for novice learners. We might expect that
novices rely on one another when they are unsure of themselves. However, there are two primary reasons that this
represents an important takeaway for nursing instructors. First, it is important for instructors to consider that self-
reported teamwork increased when the nurses were unsure of themselves. When students lacked confidence, other
students stepped in post hoc to support the role in question. A better approach may involve more metacognitive
processes, such as planning and assessing the situation jointly before taking on tasks. This involves a complex
series of supports that includes effective communication, coordination of tasks, and helping one another in main-
taining their confidence levels. When instructors see such naïve teamwork, it may be important to point out to
students that they are missing other key components of effective teamwork. Second, it may be important for
instructors to inform students that they need to work as a team even when their self-efficacy is high. When the
students were confident in themselves, they tended to work far more independently. This was particularly evident
in nurse 2, who rarely communicated with her team or the patient when her self-efficacy was high. It is important
for nurses to be confident in their abilities and what they can offer to the care team while also communicating
effectively with both the patient and the rest of the care team.
In this paper, we presented a case-study using multimodal data and interaction analysis to generate insights about
the connections between student nurses role playing, self-efficacy, and teamwork during training. We contrasted
the strategies adopted by two student nurses when faced with situations of patient doubt, which lowered the self-
efficacy of both nurses. The nurses turned to their team for assistance, but in different ways. We also highlighted
implications for instructor debriefing based on these results. This work had some limitations due to its exploratory
nature and the use of a partial case analysis of small segments. In future work, we will generalize our approach
by applying similar methods to additional learners and varied training scenarios. Future work will also focus on
developing more machine learning-based automated analysis methods. It is our hope that with continued research
and development, we can provide tools to help support healthcare team training more broadly.
Hall, R., & Stevens, R. (2015). Interaction analysis approaches to knowledge in use. In Knowledge and interaction
(pp. 88-124). Routledge.
Hustad, J., Johannesen, B., Fossum, M., Hovland, O. (2019). Nursing students’ transfer of learning outcomes from
simulation-based training to clinical practice: a focus-group study. BMC Nursing, 18:53.
Vatral, C., Biswas, G., Cohn, C., Davalos, E., & Mohammed, N. (2022). Using the DiCoT framework for inte-
grated multimodal analysis in mixed-reality training environments. Frontiers in artificial intelligence, 5,
941825. doi: 10.3389/frai.2022.941825
Wagner, J., Triantafyllopoulos, A., Wierstorf, H., Schmitt, M., Eyben, F., & Schuller, B. W. (2022). Dawn of the
transformer era in speech emotion recognition: closing the valence gap. arXiv:2203.07378.
Zacks, J. M., & Swallow, K. M. (2007). Event segmentation. Current directions in psychological science, 16(2),
... Students entered the room and performed routine evaluations of the manikin patient, and then performed relevant prescribed treatments based on their evaluation. For more details on the simulation environment, see [7]. All students provided their informed consent to collect video and audio data as they performed their training activities, and some students volunteered to wear Tobii 3 eye-tracking glasses. ...
... For each of the students' event segments, we computed these 27 features from the observed data. These initial features were selected in a somewhat post-hoc fashion, partially based on previous work with similar nursing student data [7], and partially based on the features which were easily available from the sensor systems. Because of this post-hoc strategy, not all of these features may be relevant to the prediction of students' self-confidence, so further refinement of the feature set through feature selection processes was necessary. ...
... So, we utilized a proxy target variable instead. Utilizing the relationship between teamwork and self-confidence [7], we built the mixed-effects model with students' self-rated teamwork in each segment as the target variable and measured the fixed effects between each of the features and students' self-rated teamwork. ...
Full-text available
Simulation-based experiential learning environments used in nurse training programs offer numerous advantages, including the opportunity for students to increase their self-confidence through deliberate repeated practice in a safe and controlled environment. However, measuring and monitoring students’ self-confidence is challenging due to its subjective nature. In this work, we show that students’ self-confidence can be predicted using multimodal data collected from the training environment. By extracting features from student eye gaze and speech patterns and combining them as inputs into a single regression model, we show that students’ self-rated confidence can be predicted with high accuracy. Such predictive models may be utilized as part of a larger assessment framework designed to give instructors additional tools to support and improve student learning and patient outcomes.KeywordsExperiential LearningSimulation-based TrainingMultimodal Learning Analytics (MMLA)Self ConfidenceMachine Learning
Full-text available
Recent advances in transformer-based architectures have shown promise in several machine learning tasks. In the audio domain, such architectures have been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream performance, and have shown limited attention to generalisation , robustness , fairness , and efficiency . The present contribution conducts a thorough analysis of these aspects on several pre-trained variants of wav2vec 2.0 and HuBERT that we fine-tuned on the dimensions arousal, dominance, and valence of MSP-Podcast, while additionally using IEMOCAP and MOSI to test cross-corpus generalisation. To the best of our knowledge, we obtain the top performance for valence prediction without use of explicit linguistic information, with a concordance correlation coefficient (CCC) of. 638 on MSP-Podcast. Our investigations reveal that transformer-based architectures are more robust compared to a CNN-based baseline and fair with respect to gender groups, but not towards individual speakers. Finally, we show that their success on valence is based on implicit linguistic information, which explains why they perform on-par with recent multimodal approaches that explicitly utilise textual information. To make our findings reproducible, we release the best performing model to the community.
Full-text available
Simulation-based training (SBT) programs are commonly employed by organizations to train individuals and teams for effective workplace cognitive and psychomotor skills in a broad range of applications. Distributed cognition has become a popular cognitive framework for the design and evaluation of these SBT environments, with structured methodologies such as Distributed Cognition for Teamwork (DiCoT) used for analysis. However, the analysis and evaluations generated by such distributed cognition frameworks require extensive domain-knowledge and manual coding and interpretation, and the analysis is primarily qualitative. In this work, we propose and develop the application of multimodal learning analysis techniques to SBT scenarios. Using these analysis methods, we can use the rich multimodal data collected in SBT environments to generate more automated interpretations of trainee performance that supplement and extend traditional DiCoT analysis. To demonstrate the use of these methods, we present a case study of nurses training in a mixed-reality manikin-based (MRMB) training environment. We show how the combined analysis of the video, speech, and eye-tracking data collected as the nurses train in the MRMB environment supports and enhances traditional qualitative DiCoT analysis. By applying such quantitative data-driven analysis methods, we can better analyze trainee activities online in SBT and MRMB environments. With continued development, these analysis methods could be used to provide targeted feedback to learners, a detailed review of training performance to the instructors, and data-driven evidence for improving the environment to simulation designers.
Full-text available
Background: Simulation-based training is used to develop nursing students' clinical performance in assessing and managing situations in clinical placements. The use of simulation-based training has increased and become an integrated part of nursing education. The aim of this study was to explore nursing students' experiences of simulation-based training and how the students perceived the transfer of learning to clinical practice. Methods: Eight focus group interviews were conducted with a total of 32 s- and third-year nursing students who participated in a simulation-based training organized as preparation for clinical placement. The transcribed interviews were analysed with thematic analysis. Results: Three major themes emerged from the focus group interviews; first, the simulation-based training promoted self-confidence; second, understanding from simulation-based training improved clinical skills and judgements in clinical practice; and third, simulation-based training emphasised the importance of communication and team collaboration. Conclusions: This study revealed students' transfer of learning outcomes from simulation-based training to clinical practice. The students' experiences of the simulation-based training remain as enduring and conscious learning outcomes throughout their completion of clinical practice. The organisation of simulation-based training and its implementation in the curriculum are crucial for the learning outcomes and for students' experiences of the transfer of knowledge to clinical practice.
One way to understand something is to break it up into parts. New research indicates that segmenting ongoing activity into meaningful events is a core component of ongoing perception, with consequences for memory and learning. Behavioral and neuroimaging data suggest that event segmentation is automatic and that people spontaneously segment activity into hierarchically organized parts and sub-parts. This segmentation depends on the bottom-up processing of sensory features such as movement, and on the top-down processing of conceptual features such as actors' goals. How people segment activity affects what they remember later; as a result, those who identify appropriate event boundaries during perception tend to remember more and learn more proficiently.
Interaction analysis approaches to knowledge in use
  • R Hall
  • R Stevens
Hall, R., & Stevens, R. (2015). Interaction analysis approaches to knowledge in use. In Knowledge and interaction (pp. 88-124). Routledge.