Conference PaperPDF Available

An Approach for Training Decision Making Competences in a Multi-Modal Online Environment

Authors:

Abstract

In this paper, we describe an approach for training decision making competences in emergency situations with the help of a multi-modal online environment. Decision making is an essential aspect of emergency management and a successful response to an emergency situation highly depends on whether decisions are being taken in an effective and timely manner. For this reason effective training programs are needed, in order to guarantee that emergency managers respond adequately to disasters. This paper addresses these needs by outlining and structuring the competences required for making meaningful decisions and by presenting an approach on how these competences can be trained in an online environment.
Ogata, H. et al. (Eds.) (2015). Proceedings of the 23rd International Conference on Computers in Education.
China: Asia-Pacific Society for Computers in Education
An Approach for Training Decision Making
Competences in a Multi-Modal Online
Environment
Alexander NUSSBAUMERa*, Christina M. STEINERa, Nora MCCARTHYb, Simon DWANEb,
Karen NEVILLEc, Sheila O'RIORDANc, & Dietrich ALBERTa,d
aKnowlege Technologies Institute, Graz University of Technology, Austria
bASSERT for Health Centre, University College Cork, Ireland
cBusiness Information Systems, University College Cork, Ireland
dDepartment of Psychology, University of Graz, Austria
*alexander.nussbaumer@tugraz.at
Abstract: In this paper, we describe an approach for training decision making competences in
emergency situations with the help of a multi-modal online environment. Decision making is an
essential aspect of emergency management and a successful response to an emergency situation
highly depends on whether decisions are being taken in an effective and timely manner. For this
reason effective training programs are needed, in order to guarantee that emergency managers
respond adequately to disasters. This paper addresses these needs by outlining and structuring
the competences required for making meaningful decisions and by presenting an approach on
how these competences can be trained in an online environment.
Keywords: Decision making competences, Competence-based Knowledge Space Theory,
personalization, online training
1. Introduction
Decision making is an essential component of emergency management, and a successful response to an
emergency situation highly depends on whether decisions are being taken in an effective and timely
manner. Decision making in emergencies encompasses a broad range of individual decisions, which
may occur before, during, or after an event, from declaring a state of emergency, issuing an evacuation,
to specific operational decisions. Acute emergency situations represent decision environments that are
dynamic and complex. This is due to the large number and interdependence of variables involved,
including: dynamism, uncertainty and intransparency of the situation, information overload or lack of
information, time pressure, risk, plurality of goals, and the involvement of multiple players (St.Pierre et
al., 2008).
For effective emergency decision making, domain-specific knowledge on emergencies and
emergency management is needed, as well as knowledge and competence on a meta-level
(metacognition, decision making). While domain-specific knowledge typically refers to the operations
during emergencies and the resources needed for these operations, competences on the meta-level refers
to decision making in general, targeting the logical structure of decisions and their consequences. In this
paper we address the training of these meta-level decision making competences.
2. Decision Making Competences in Emergencies
Decision making constitutes an area of critical thinking skills that is considered essential for students
and workers in the 21st century. Decision making competence is a multidimensional construct
including a range of key skills (e.g. Finucane & Gullion, 2007). In the context of our psychological
framework (Steiner et al., 2015) a set of decision making competences have been defined focusing on
the steps of decision making:
97
competences related to the forethought phase: e.g. identify the problem, defining the goal, and
collecting information
competences related to the decision phase: e.g. generating and evaluating alternatives, selecting
and implementing the best alternative
competences related to the reflection phase: e.g. evaluating results, attribution of causes,
monitoring progress, adapting reaction
For a systematic approach to model decision making competences, we use Competence-based
Knowledge Space Theory (Heller et al., 2006), a psychological set-theoretic framework for
representing domain and personal knowledge and competences. This theoretical approach establishes a
structure on a knowledge domain by capturing the inherent structural dependencies (prerequisites)
between the items or competences of a domain. The established knowledge or competence structures
may be used as a reference point for realising meaningful sequences of learning, for efficient
assessment procedures, and for personalizing learning. The decision making competence metrics allow
for a useful assessment method as each can be scored (Gallagher, 2012).
3. Multi-Modal Training Environment
Multi-modal learning has been shown to be more effective than more traditional, single-modal
approaches of learning. Kolb & Kolb (2005) describe an experimental learning cycle that integrates
abstract concepts and concrete experiences as two dialectically related modes of transforming
knowledge. Based on that, we present an approach to train decision making competences in two ways.
First, an adaptive and personalized online course is used to train theoretical and conceptual knowledge
of decision making competences. Second simulated environments are used to apply and train the
application of decision making competences. Both of these environments employ deliberate practice
and proficiency based progression. Deliberate practice involves the provision of immediate feedback on
the performance of a task or activity for the purpose of skill improvement (Ericsson, 2003). Proficiency
Based Progression emphasizes the stepwise approach required for progression from a novice to an
expert level of performance (Dreyfus et al., 1986, Anderson & Krathwohl 2001).
3.1 Web-based learning service Compod
Compod is a service that supports the adaptive and personalized learning based on competence
structures (Nussbaumer et al., 2015). One of the core components of Compod is the domain model that
includes competences, their prerequisite structures, learning objects, and assessment items. The
competence structure is graphically depicted, so that the trainee can choose individual competences to
be learned next. Learning objects and assessment items related to selected competences are presented to
the trainee.
In order to train decision making competences, we have modelled a decision making competence
structure and added learning objects and assessment items for their training. The trainee can step
through this structure, visit learning objects and undergo assessment items for selected competences.
Each time a learning object is visited or an assessment item is solved correctly, the related competence
is marked (with different colours). In this way, the learning progress is graphically represented.
Traversing through the whole competence structure, the complete domain of decision making can be
covered.
3.2 Online simulator
Emergency management training is often done in virtual reality simulators. Such simulators are able to
provide a realistic environment with emergency procedures, respective infrastructure, safety
environment, fire and smoke propagation, and other components available in a real incident (Louka, &
Balducelli, 2001). In order to use these environments for training, scenarios are created, where
emergency managers have to respond to incidents and make respective decisions.
98
In our approach these scenarios are treated in a similar way to assessment items. The decisions to
be made in these scenarios are related to the decision making competences as described above. Each
time a trainee undergoes a scenario and makes a decision, the actions are analysed regarding the quality
and correctness of the decisions taken and any related decision making competences. Then the
information on the decision making competences are sent to the Compod service, where the new
information is used to update the personal profile of the trainee. This information is then graphically
shown in the decision making competence structure (Figure 1). This allows the trainee to go back to the
theoretical knowledge of certain decision making competences, and to assess whether they failed in
taking certain decisions correctly. The overall goal is to complete the learning objects and assessment
items, as well as the scenarios for all decision making competences, which triggers live updates in the
Compod system.
4. Conclusion and Outlook
This paper presents an approach to train decision making competences needed for emergencies in two
different ways. The Web-based system Compod allows for learning the theoretical knowledge of
decision making competences and the online simulator allows for training in emergency scenarios.
While the Compod system and the decision making competence structure have already been developed,
the online simulator is not yet available. Next steps and future work include the selection of a virtual
reality application that enables training in emergency scenarios. These scenarios will be assigned with
decision making competences from our domain model and a mechanism that analyse if the correct or
wrong decisions have been taken. This information is then sent to Compod, where user information on
available decision making competences from assessment items and scenarios are stored and brought
together. It is the amalgamation of the decision making task analysis and metric development, web
based and simulator learning with deliberate practice and proficiency based progression that converts
the multi modal environment from an educational experience to a training programme that allows
optimal learning to occur.
References
Anderson, L.W. & Krathwohl, D.R. (Eds.) (2001). A taxonomy for learning, teaching, and assessing: A revision
of Bloom’s taxonomy of educational objectives. New York: Longman.
Dreyfus, H., Dreyfus, S., & Athanasiou, T. (1986). Five steps from novice to expert. In H. Dreyfus & S. Dreyfus
(Eds.), Mind over machine: The power of human intuition and expertise in the era of the computer (pp.
16-51). New York: Blackwell Publishers
Ericsson, K. A. (2008). Deliberate practice and acquisition of expert performance: a general overview. Academic
Emergency Medicine, 15, 988-994.
Gallagher, A. G. (2012). Metric-based simulation training to proficiency in medical education: what it is
and how to do it. Ulster Medical Journal, 81, 107-113.
Finucane, M.L. and Gullion, C.M. (2010), Developing a tool for measuring decision-making competence of older
adults, Psychol. Aging, 25, 271-288.
Heller, J., Steiner, C., Hockemeyer, C. and Albert, D. (2006), Competence-based knowledge structures for
personalised learning, International Journal on E-Learning, 5, 75-88
Kolb, A.Y., & Kolb, D.A. (2005). The Kolb Learning Style Inventory Version 3.1: 2005 Technical
Specifications. Haygroup: Experience Based Learning Systems Inc.
Louka, M. N., & Balducelli, C. (2001). Virtual reality tools for emergency operation support and training.
Proceedings of the International Emergency Management Society (TIEMS), Oslo.
Nussbaumer, A., Hillemann, E.-C., Gütl, C., & Albert, D. (in press). A competence-based service for supporting
self-regulated learning in virtual environments. Journal of Learning Analytics.
Steiner, C. M., Nussbaumer, A., & Albert, D. (in press). A Psychological Framework Modelling Decision Making
in Emergencies. In Proceedings of the 9th European Conference on IS Management and Evaluation (ECIME
2015). Bristol, UK
St.Pierre, M., Hofinger, G. and Buerschaper, C. (2008), Crisis management in acute care settings. Human factors
and team psychology in a high stakes environment, Berlin: Springer.
99
... Albert, Hockemeyer, Kickmeier-Rust, Nussbaumer, & Steiner, 2012;Conlan, O'Keeffe, Hampson, & Heller, 2006). (Nussbaumer et al., 2015) used Competence-based Knowledge Space Theory to provide a systematic approach to model decision-making competences, adapting the above mentioned reviews of decision-making, and drawing comparisons between the self-regulation process, which includes critical thinking, metacognitive regulation, communication and teamwork and the decision-making process. The resulting decision-making skill set provides a good prescriptive model for training in the decision-making process, combining both the knowledge structure that defines decision-making and also the cognitive processes involved. ...
... As decision-making needs a situation in which to be applied, the creation an emergency scenario is needed for assessment and improvement. Specifically targeting this required level of granularity, the skillset developed by Steiner and (Nussbaumer et al., 2015) will be utilised, with content developed for a learning course. The initial training will concentrate on learning the key component skills of the decision-making skill set through a theory course developed by (Nussbaumer et al., 2015); they will then be practiced through various scenarios, with metrics developed in consultation with experts, to encourage the development of fluidity and automaticity to allow movement from the novice stage of skill acquisition. ...
... Specifically targeting this required level of granularity, the skillset developed by Steiner and (Nussbaumer et al., 2015) will be utilised, with content developed for a learning course. The initial training will concentrate on learning the key component skills of the decision-making skill set through a theory course developed by (Nussbaumer et al., 2015); they will then be practiced through various scenarios, with metrics developed in consultation with experts, to encourage the development of fluidity and automaticity to allow movement from the novice stage of skill acquisition. The driver and learning goal of training is the decision-making skill set, with the scenarios supporting the training. ...
Article
Full-text available
Errors in decision-making worldwide highlight the need for training in decision-making. The unpredictability and complexity of emergencies makes training in every possible emergency impossible. Rather than training in specific examples of major emergency events, training in a decision-making skill set will provide a method of response that will be transferable to all emergencies. Various scenarios will support the training as a decision needs a context for application. The resulting educational tool will focus on emergency services at the strategic and tactical levels in the response stage of an emergency. The continual engagement of stakeholder should result in a purpose-built training course. Design science research approach will be utilised, investigating connections between theories of cognitive load and expert performance. Key aspects of the developed training course will include the concepts of metrics, deliberate practice and proficiency based progression, to ensure an appropriate training programme rather than a mere educational experience.
Article
Full-text available
This paper presents a conceptual approach and a Web-based service that aim at supporting self-regulated learning in virtual environments. The conceptual approach consists of four components, which are (a) a self-regulated learning model for supporting a learner-centred learning process, (b) a psychological model for facilitating competence-based personalisation and knowledge assessment, (c) an open learner model approach for visual interaction and feedback, and (d) a learning analytics approach for capturing relevant learner information required by the other components. The Web-based service provides a technical implementation of the conceptual approach, as well as a linkage to existing virtual environments used for learning purposes. The approach and service have been evaluated in user studies in university courses on computer science to demonstrate the usefulness of the overall approach and to get an understanding of some limitations.
Conference Paper
Full-text available
This paper discusses the current state of the art of virtual reality (VR) technology within the context emergency operation support and training. Recent widely reported fatal road and railway tunnel accidents (e.g. Mont Blanc, Kaprun) have highlighted the problems that arise when evacuating people to safety in the event of a fire in a confined space with limited exit possibilities. Similar problems arise in the evacuation of aircraft and high-speed trains or subways. In order to improve preparedness plans, it is necessary to develop rescue scenarios to ascertain the fire safety and to train rescue personnel by simulation of fire emergencies. The integration of VR technology with traditional geographical information systems and tools for managing emergency procedures is particularly interesting from both a decision support and training perspective. A VR interface could either replace a conventional 2D interface or complement it, depending on user requirements. This paper proposes VR methodologies as the only feasible alternative to full-scale fire tests inside tunnels. In a computer simulation-based virtual environment, all data about infrastructure, safety equipment, ventilation resources, fire and smoke propagation, and vehicle/passengers involvement could be managed by a single database. Fire and smoke simulation tools as well as computerised emergency procedures, acting on the virtual data, could also be integrated in such a system and utilised for training support by the emergency operators.
Book
This work begins with the basic principles of human behaviour and decision making and then partitions into three sections where the individual, the team, and the organisational influences within the health-care system are discussed in greater depth.
Article
High profile error cases and reduced work hours have forced medicine to consider new approaches to training. Simulation-based learning for the acquisition and maintenance of skills has a growing role to play. Considerable advances have been made during the last 20 years on how simulation should be used optimally. Simulation is also more than a technology learning experience for supplanting the traditional approach of repeated practice. Research has shown that simulation works best when it is integrated into a curriculum. Learning is optimal when trainees receive metric-based feedback on their performance. Metrics should unambiguously characterize important aspects of procedure or skill performance. They are developed from a task analysis of the procedure or skills to be learned. The outcome of the task analysis should also shape how the simulation looks and behaves. Metric-based performance characterization can be used to establish a benchmark (i.e., a level of proficiency) which trainees must demonstrate before training progression. This approach ensures a more homogeneous skill-set in graduating trainees and can be applied to any level of training. Prospective, randomized and blinded clinical studies have shown that trainees who acquired their skills to a level of proficiency on a simulator in the skills laboratory perform significantly better in vivo in comparison to their traditionally trained colleagues. The Food and Drug Administration in the USA and the Department of Health in the UK have candidly indicated that they see an emergent and fundamental role for simulation-based training. Although a simulation-based approach to medical education and training may be conceptually and intellectually appealing it represents a paradigm shift in how doctors are educated and trained.
Article
The authors evaluated the reliability and validity of a tool for measuring older adults' decision-making competence (DMC). A sample of 205 younger adults (25-45 years), 208 young-older adults (65-74 years), and 198 old-older adults (75-97 years) made judgments and decisions related to health, finance, and nutrition. Reliable indices of comprehension, dimension weighting, and cognitive reflection were developed. Comparison of the performance of old-older and young-older adults was possible in this study, unlike previous research. As hypothesized, old-older adults performed more poorly than young-older adults; both groups of older adults performed more poorly than younger adults. Hierarchical regression analyses showed that a large amount of variance in decision performance across age groups (including mean trends) could be accounted for by social variables, health measures, basic cognitive skills, attitudinal measures, and numeracy. Structural equation modeling revealed significant pathways from 3 exogenous latent factors (crystallized intelligence, other cognitive abilities, and age) to the endogenous DMC latent factor. Further research is needed to validate the meaning of performance on these tasks for real-life decision making.
Five steps from novice to expert Mind over machine: The power of human intuition and expertise in the era of the computer (pp. 16-51) Deliberate practice and acquisition of expert performance: a general overview
  • H Dreyfus
  • S Dreyfus
  • T K A Athanasiou
Dreyfus, H., Dreyfus, S., & Athanasiou, T. (1986). Five steps from novice to expert. In H. Dreyfus & S. Dreyfus (Eds.), Mind over machine: The power of human intuition and expertise in the era of the computer (pp. 16-51). New York: Blackwell Publishers Ericsson, K. A. (2008). Deliberate practice and acquisition of expert performance: a general overview. Academic Emergency Medicine, 15, 988-994.
A Psychological Framework Modelling Decision Making in Emergencies
  • C M Steiner
  • A Nussbaumer
  • D Albert
Steiner, C. M., Nussbaumer, A., & Albert, D. (in press). A Psychological Framework Modelling Decision Making in Emergencies. In Proceedings of the 9th European Conference on IS Management and Evaluation (ECIME 2015). Bristol, UK