Conference PaperPDF Available

How to Train Employees, Identify Task-Relevant Human Factors, and Improve Software Systems with Business Simulation Games

Abstract and Figures

In today's globalized world, cost pressure, the demand for higher production efficiency, and growing product diversity lead to increasing complexity of manufacturing and business processes. Insufficiently understood human and social factors further increase this complexity. Game-based learning environments and business simulation games can reduce this complexity by identifying and understanding the contributing human factor. Business simulation games can be used (1) to identify and quantify human as well as social factors influencing the effectivity and efficiency, (2) to assess the aptitude of prospective employees and identify suitable training interventions, (3) to improve management and production software, and (4) to present effective, efficient, and entertaining training environments. By allowing employees to investigate cause-and-effect relationships of simulated manufacturing and business environments, they can test and understand the consequences of their actions in safe environments. In this paper, we report a practical case of a business simulation game for conveying quality management strategies. The development of the game is presented along the definition of learning objectives, the underlying System Dynamics model, and the design of the user interface. The evaluation of the game reveals that human factors relate to the simulation's metrics. Finally, we give guidelines to design and develop game-based simulation and training environments.
Content may be subject to copyright.
International Conference on Competitive Manufacturing
How to Train Employees, Identify Task-Relevant Human Factors,
and Improve Software Systems with Business Simulation Games
P. Brauner1, M. Ziefle1
1Chair for Communication Science, Human-Computer Interaction Center
RWTH Aachen University, Germany
In today’s globalized world, cost pressure, the demand for higher production efficiency, and
growing product diversity lead to increasing complexity of manufacturing and business processes.
Insufficiently understood human and social factors further increase this complexity. Game-based
learning environments and business simulation games can reduce this complexity by identifying
and understanding the contributing human factor. Business simulation games can be used (1) to
identify and quantify human as well as social factors influencing the effectivity and efficiency, (2) to
assess the aptitude of prospective employees and identify suitable training interventions, (3) to
improve management and production software, and (4) to present effective, efficient, and
entertaining training environments. By allowing employees to investigate cause-and-effect
relationships of simulated manufacturing and business environments, they can test and
understand the consequences of their actions in safe environments. In this paper, we report a
practical case of a business simulation game for conveying quality management strategies. The
development of the game is presented along the definition of learning objectives, the underlying
System Dynamics model, and the design of the user interface. The evaluation of the game reveals
that human factors relate to the simulation’s metrics. Finally, we give guidelines to design and
develop game-based simulation and training environments.
Game-Based Learning, Simulation-Based Learning, Serious Games, Vocational Training, Human
Factors, Usability, Usability in Production Engineering
Today’s manufacturing companies are facing
profound changes due to increasing globalization,
supply chains growing in size and complexity, and
innovations in industrial Internet. An increasing
number of product variants, growing demands on
product quality, and shorter lead times pose
tremendous challenges for employees managing the
flow of information and materials across supply
chains of companies. Companies that successfully
manage the increasing complexity, reduce the
variance of production processes, and enable
employees to successfully handle variance will gain
the necessary competitive advantages to sustain at
tomorrow’s markets.
Diverse technical approaches target the increase of
overall productivity and to make systems more
resilient against variances [1]. Still, the human
factorsperspective is often neglected despite its
evident importance: Studies show that overall
productivity can be increased if ergonomics and
human factors are adequately considered [2,3].
Preparing employees to handle complexity,
variance, and uncertainty is also delicate. Often,
courses and learning modules in schools,
universities, job training, or advanced trainings focus
on teaching single bits of information. However,
parts of the complexity of today’s world stem from its
interconnectedness and reciprocal interference. We
argue that this interconnectedness is difficult to
communicate and that adequate simulation models
embedded in game environments allow employees
to gain a deeper and connected understanding of
the complexity of today’s world. This networked
thinking will empower employees to handle the
increasing complexity successfully [4].
The structure of this article is as follows: Section 2
defines the terms game-based learning and serious
games and gives examples from business and
production engineering. Section 3 contours the
benefits of game-based learning environments.
Section 4 demonstrates the development of a game,
the underlying simulation model, and the
development of the benchmark function that was
used to investigate our hypotheses. Next, Section 5
depicts studies that investigated and (mostly)
confirmed the hypotheses of the versatility of these
games. The article concludes with Section 6 and a
summary and discussion of game-based learning
environments to strengthen the competitiveness of
A literature review on the terms “serious games” and
game-based learning” yielded over 2 million results
each. Even for the domain of production
engineering, the examples for game-based learning
are ubiquitous. Therefore, we start with a formal
definition of serious games and implications for
production engineering (Section 2.1), alongside the
most prominent examples of serious games for
production engineering (Section 2.2). A
comprehensive overview of business simulation
games is given in [5].
2.1 Background
Serious games are typically used to mediate
knowledge or behavior change for educational
purposes. They are entertaining but not intended
primarily for amusement [6]. Furthermore, Michael
and Chen state that game models are simplified
abstractions of a problem and not necessarily
complete and precise [7]. Prensky even argues that
serious games will be the most successful method
for the Millennials and following generations’
education as they grew up fully surrounded by
technology and often find conventional media and
earlier didactical approaches boring and
cumbersome [8].
As shown above, the necessary skills to manage the
complexity of today’s world is beyond declarative
knowledge; rather it includes procedural knowledge
and the understanding and internalization of cause-
and-effect relationships. Bogost highlights that the
procedural rhetoric of serious games persuades to
increased interaction with a topic which yields a
deeper understanding of the modeled processes [9].
2.2 Examples
First, Forrester’s Beer Distribution Game (BDG)
illustrates the effect of variance along a supply chain
by placing several players along a supply chain for
an alcoholic beverage [10]. Ordering information is
passed upstream (from a retailer to a factory), whilst
goods are delivered downstream, each with a short
time delay. The game serves two learning
objectives: First, the players are sensitized to the
“bullwhip effect”, i.e., orders along supply chains are
prone to escalation. Second, sharing information
reduces this escalation.
Goldratt’s game is a second prominent example and
demonstrates the difficulties that arise from
variances in delivery reliability or product quality
[11]. The game is similar to the BDG, however,
depending on random factors, only a subset of an
order is delivered. This introduces significant
variance along the supply chain and makes meeting
the market’s demands difficult.
Both games are typical contents in business,
engineering, and management classes, because
they raise the awareness for critical aspects of
supply chain management. Players need to find a
trade-off between different components of the
system and have to understand that optimizing for a
single aspect of the environment is insufficient and
detrimental. Hence, successful players develop an
understanding of the interconnected system (and its
interdependent factors), infer the current state of the
system from a limited number of variables, and
choose the optimal or an adequate of many possible
actions. The proficiency of players can then be
evaluated by investigating the players’ actions or
their overall performance.
This section postulates that game-based learning
environments in manufacturing and business offer
several short- and long-term benefits for academia
and industry. The following arguments militate in
favor of this posit.
First, by studying workers’ behaviors and their
decisions, the game environments can measure
individual workers’ skills and their awareness for
effective and efficient handling of specific situations
in production environments. Hence, they are
suitable recruitment tests and can identify training
demands, if workers lack the respective skills or
Second, the game environment can serve as
training environment in order to sensitize future
employees for the challenges of production
processes and to gain experience in handling
specific situations that occur during these
processes. Difficult situations can be explored and
trained without putting the company at risk.
Third, that game-based learning environments are a
versatile method to understand the underlying
human factors. They can identify cognitive, social,
or emotional aptitudes that are beneficial or crucial
for handling complex situations.
Forth, we argue that game-based learning
environments can help advance business
software, as they can be used to empirically study
how information presentation, amount, and
complexity influence the decision quality of
employees. Also, they can be a benchmark with
high ecological validity to evaluate changes and new
features in enterprise software.
The overall utility of the game environment in this
context can be demonstrated by investigating the
relationship between user factors (e.g., age,
expertise, cognitive abilities), interface factors (e.g.,
font sizes, screen layouts, visibility of key
performance indicators), and the complexity of the
simulated environment (e.g., seasonal movements
vs. predictable linear growth) on metrics from within
simulated environment.
The development of a game-based learning
environment builds on three fundamental
constituents: First, a simulation model that provides
a suitable abstraction of the world, and in our case,
an abstraction of a supply chain, a company, or a
production process. Second, a user interface that
communicates the state or part of the state of the
simulation model to the user and allows the user to
interact purposefully with the simulation model.
Third, one or multiple functions from within the
simulation model that serve as a benchmark to
evaluate the performance of the players.
This paper illustrates the development steps using a
game-based learning environment for mediating
expertise in material disposition and quality
management for manufacturing companies.
Although this example and the simulation model are
quite specific, the general development procedure
can easily be adapted to other learning objectives or
application contexts.
4.1 Development of a simulation model
This section exemplifies the development of an
abstract simulation model for a game-based
learning environment. It is based on the Quality
Intelligence Game (QIG); a detailed presentation of
this model, its motivation, and the underlying
assumptions are presented in [12]. The
development of a model is based on three
interleaved steps. Frist, the learning or research
objectives need to be defined. Second, a model of
the necessary cause-and-effect relationships needs
to be specified. Third, based on the identified
relationships, simulation functions are specified for
each component.
As a first step, the learning objectives need to be
identified. For this game, two objectives were
selected that have to be balanced by the players: As
in the BDG (see above), the first goal is to maintain
sufficient stocks of a product and to always fulfill the
orders of a simulated customer. The second goal
incorporates product quality into the BDG and high
product quality ought to be achieved by investing in
the incoming goods inspection and in the internal
quality assurance. The objectives are to increase
the awareness for the importance of quality
management and to increase the skill of the players
to apply quality management strategies.
The second step is the identification and definition of
the components and cause-and-effects relationships
between the components in the models. We suggest
using a System Dynamics model [10] as a basis for
this work. In our example, the model consists of the
components for 3-tiers of a supply chain with the
external supplier (S), the player as the
manufacturing company (M), and the customer (C).
Other components that relate to the production
process are also part of the model. For example, in
our model the supplier’s and internal production
qualities can change over time. Further components
are the number of intact and broken parts delivered
from the supplier (both depend on the supplier’s
production quality), the number of intact or broken
parts in stock (depending on the deliveries), the
number of goods complained about by the customer
(depending on the stock and the internal production
quality), and the net profit achieved (depending on
complaints by the customer, investments for quality
inspections, stock keeping, and stock-out-penalties).
Figure 1 shows an overview of this model.
Figure 1 - Simplified model of the Quality
Intelligence Game (QIG)
For reasons of simplicity and controllability, the
current model is designed for a single player that
controls the material disposition and quality
management of the manufacturing company (M)
whereas supplier (S) and customer (C) are
simulated through an artificial intelligence.
This step is the most difficult and most crucial part of
developing a game-based learning and simulation
environment: On one side, the model needs to be
sufficiently complex to capture all previously defined
learning and research objectives. On the other side,
it should not be too complex, as the following
implementation steps will be more difficult and the
users will have more difficulties to interact and
understand the model [13].
In the third step, the simulation of the model’s
components is defined as functions of the previously
identified cause-and-effect relationships. For
example, the stock level S at a given time t is based
on the previous stock level, the goods ordered from
the supplier O, and the deliveries D to the customer:
S(t) = S(t-1) + O(t-1) – D(t)
The net profit P(t) at a given time t is based on the
current stock level S(t) and the per part stock-
keeping costs costStock, the investments in the
incoming goods inspection Iigi and the internal
production quality Iipq, the costs for the complained
parts, and the revenue R(t) for the delivered parts:
P(t) = R(t) – cStock×S(t) – Iigi(t) – Iipq(t) C(t-1)
A deeper presentation of the model is given in [12].
Depending on the modeled process or company and
the addressed learning and research objectives, the
parameterization of the functions can weight specific
factors (e.g., out-of-stock penalties) as more
important than others by assigning different
penalties and rewards to them. We argue that the
parameterization should relate to the objectives and
the later use context of the game.
After the full specification of the simulation model, it
can be implemented and tested as a low-fidelity
prototype (e.g., in a spreadsheet application) or as
functional application in a programming language.
4.2 Development of a user interface
In order to let players interact with the simulation, a
user interface needs to be designed and
implemented. Again, this step depends on the
learning objective defined at the beginning.
First, developers need to identify which of the
variables from the simulation model should be
visible to the user and which should be concealed.
The latter is important, as often parts of the model
are intentionally hidden from the user. For example,
the internal production quality in QIG is designedly
invisible to the user, as she or he has to infer
changes to this metric from other variables.
Obviously, not all variables need to be visible all the
time and the user may be given the option to
request additional information on specific variables
(e.g., the temporal trend of a variable or a
decomposition of aggregated values). Second, the
designers must also reflect how to represent each
variable as an indicator in the user interface. For
example, variables can be represented numerically
(“102 parts”), using analog scales, as absolute or
relative values (“production up 10%”), or using traffic
lights (“stock level red”).
For this design process, it is advised that developers
consider relevant guidelines during the design and
development of the user interface [14,15] and
consider the learning and research objectives. For
example, the user interface development for the
QIG started as paper prototype: By this, the
necessary indicators and controls for interacting with
the model could easily be redesigned and evaluated
until a suitable spatial layout was found.
Third, the simulation model and the user interface
must be implemented as computer applications.
Although some simpler games can be played as
board games (e.g., Beer Distribution Game), we
argue for using computer applications, as they can
easily handle sophisticated simulation models, can
log each simulation step, and log every user
interaction for later analysis. For the QIG, the game
was realized as a web application using Java EE
and the PrimeFaces framework. Following the
Model View Controller pattern (MVC) [16], the user
interface (V) is distinctly separated from the
simulation model (M) and one component can be
changed without affecting the other.
4.3 Choosing meaningful benchmarks
Even an abstraction of a company offers various
metrics to investigate. As the main objective of a
manufacturing company (or likewise of a division or
cross-company supply chains) is the realization of
profits, the net profit P(t) seems to be the most
suitable generic metric. It includes costs for
investments, stock keeping, out-of-stock penalties,
and penalties for complaints due to low product
quality. Obviously, the inspection of other variables
from the simulation may also provide valuable
insights, depending on the learning objective or
specific research question. For example, lead times,
achieved customer satisfaction, or total product
quality may also be worth studying. These metrics
are already part of the simulation model (s. Section
4.1) and their relationship with the users (abilities),
the user interface (visual and cognitive ergonomics),
or simulation factors (complexity of the environment)
can be investigated.
This section outlines the empirical studies carried
out to show game-based learning in the four
previously mentioned application fields. Although
the findings relate to the presented QIG and an
implementation of the Beer Distribution Game we
did earlier, the general methodology is transferable
to other game and simulation environments.
5.1 Training environment
Our studies show that the games are suitable
training environments as interacting with them had a
significant influence on three key variables [17,18]:
First, in both games, the users showed an increase
in net company profits between multiple rounds of
the game. Hence, players learned to perform their
tasks and understood how to react to the challenges
in the game. Second, in the quality management
game, the average product quality also increased.
Hence, the central learning objective was achieved.
Third, summative questionnaires revealed that
players had a higher awareness for the mediated
learning objectives, i.e., the awareness for the
bullwhip effect and quality management increased.
5.2 Identification of training potentials
In line with the previous section, the investigated
metrics (achieved net profit, product quality, and
awareness) can likewise be used as benchmarks to
identify the suitability of a potential employee for a
specific task or to identify training potentials.
Similar to an assessment center or as a part of one,
candidates might be screened for their domain skills
using an adequately designed simulation. Selection
criteria might be how fast candidates get acquainted
with the system (i.e., learning curve) or the
performance achieved on average. However, the
reliability and external validity of this approach is
currently unexplored and further studies need to
investigate the predictive power of achieved game
performance on later job performance. Likewise, this
approach may also identify training potentials. If the
performance achieved is below a certain threshold
or erroneous reactions are performed in specific
situations, the system might suggest adequate
training interventions.
5.3 Identification of human factors on
Regarding the identification of underlying human
factors relating with performance in the business
simulation games the results are promising, though
they leave room for further research endeavors [17
19]. No strong relationship between game
performance and a set of investigated personality
factors (different between the studies) was
discovered yet. Still, some findings were coherent
across all studies: The players performances in
later rounds of the game were strongly correlated
with the performances from earlier rounds.
Consequentially, some players are better in the
game and others are not, which is a strong indicator
for the existence of yet undetected personality
factors that will eventually explain performance.
5.4 Evaluation of user interfaces
Multiple experiments investigated and quantified the
influence of information presentation and usability
on the players’ performance.
An experiment in the context of the game revealed
that usability and decision complexity interact in
regard to performance, meaning that employees
have additional and often avoidable difficulties to
make correct decisions in complex situations if the
data presentation is inadequate [20].
In a second study, a holistic refinement of the
game’s user interface was compared with the
original interface [18]. The revised interface featured
a process-oriented layout of screen elements and
controls and integrated several key-performance
indicators on important variables of the game (e.g.,
stock-level, product quality, refusals). To investigate
the influence of this refinement, a randomized trial
with both interfaces as between-subject variable and
all other aspects, such as the game’s complexity,
held constant was carried out. The results show that
reorganizing the interface towards a process-
oriented presentation and the integration of key-
performance indicators support players to achieve
higher overall profits and higher product quality.
Future studies will break down the several changes
to the interface and individually quantify the efficacy
of data presentation, interface layout, and the
presence of key-performance indicators. The long-
term objective is to develop visualizations of
relevant company metrics that adapt to the current
user and context to support employees in difficult
situations. This approach offers the opportunity to
test and evolve user interfaces and user interface
guidelines for engineering enterprise software under
more realistic conditions than in controlled
laboratory studies.
Embedding simulation models of supply chains,
companies, or production processes in games
enables players to interact with several aspects of
the respective system simultaneously. They can
develop, test, and evaluate hypotheses about the
relationships of the environment, and deduce a
holistic understanding of complex and heavily
interconnected systems. Hence, these environments
can and should be used to prepare employees for
their job and to enable them for networked thinking
necessary to sustain in today’s world. Using games
is especially useful for motivation and to address
Millenials and later generations. These generations
grew up with current entertainment technologies and
are reluctant to use conventional media and prior
didactical approaches in their education. We
showed that using game-based learning
environments increased the awareness for the
selected learning objectives and increased the skill
of the players to handle the required tasks.
Companies further profit from this approach as they
can easily and cost-effectively identify training
potentials for their employees or identify personnel
with high suitability for the task. At the same time,
the game-based simulation and learning
environments can then be used as a fun,
entertaining, and cost-efficient training intervention.
We demonstrated that these games could also be
used to evaluate changes to the user interfaces or
work environments. Software development
companies of Enterprise Resource Planning
Systems can therefore use game-based simulation
environments to identify key interface aspects
contributing to increased effectivity, efficiency, and
user satisfaction [21]. Interface developments can
be benchmarked along the models’ relevant cost
functions and our studies found evidence for the
positive influence of interface refinements and the
integration of key performance indicators on profit.
Although we identified a strong correlation between
a player’s performances across the levels, none of
our thus far tested factors related well with the
overall game performance. The conclusion of this is
two-fold: First, the correlation hints at the existence
of one ore more general human traits that explain
performance. Second, this or these factors still
remain unidentified and require further investigation.
Concluding, game-based learning and simulation
environments for manufacturing and business are a
viable method to increase the competitiveness of
manufacturing companies.
The authors thank our colleagues Sebastian Stiller,
Marco Fuhrmann, Hao Ngo, Robert Schmitt, Ralf
Philipsen, Julian Hildebrandt, Sean Lidynia and
Eugen Altendorf for support and in-depth
discussions on this work. We thank all participants
of the several user studies. The German Research
Foundation (DFG) founded this project within the
Cluster of Excellence „Integrative Production
Technology for High-Wage Countries”.
[1] Schuh, G., Stich, V., Brosze, T., Fuchs, S., Pulz,
C., Quick, J., et al., 2011, High resolution supply
chain management: optimized processes based
on self-optimizing control loops and real time
data., Production Engineering. 5, 433442.
[2] Karwowski W., 2006, The Discipline of
Ergonomics and Human Factors, in: G.
Salvendy (Ed.), Handbook of Human Factors
and Ergonomics, 3rd ed., John Wiley & Sons,
Inc., pp. 331. doi:10.1002/0470048204.ch1.
[3] Calero Valdez, A., Brauner, P., Ziefle, M., 2016,
Preparing production systems for the Internet of
Things The potential of socio-technical
approaches in dealing with complexity, in:
COMA'16 Conference, p. (in press).
[4] Anderson, L.W., Krathwoh, D.R., 2001, A
Taxonomy for Learning, Teaching, and
Assessing: A Revision of Bloom’s Taxonomy of
Educational Objectives, Longman, Inc., New
York, NY, USA,
[5] Faria, A.J., Hutchinson, D., Wellington, W.J.,
Gold, S., 2008, Developments in Business
Gaming - A Review of the Past 40 Years,
Simulation & Gaming, 40, 2008 464487.
[6] Abt, C.C., 1987, Serious Games, Madison
[7] Michael, D., Chen, S., 2006, Serious Games:
Games That Educate, Train and Inform,
Thomson Course Technology.
[8] Prensky, M., 2001, Digital Natives, Digital
Immigrants Part 1, On the Horizon, 9, 1–6.
[9] Bogost, I., 2007, Persuasive Games: The
Expressive Power of Videogames, The MIT
Press, Cambridge, Massachusetts, USA.
[10] Forrester, J.W., 1961, Industrial dynamics, MIT
Press, Cambridge.
[11] G oldratt, E. M., Cox, J., 1992, The Goal: A
Process of Ongoing Improvement, 2nd ed.,
North River Press, Great Barrington,
[12] Stiller, S., Falk, B., Philipsen, R., Brauner, P.,
Schmitt, R., Ziefle, M., 2014, A Game-based
Approach to Understand Human Factors in
Supply Chains and Quality Management,
Procedia CIRP. 20, 6773.
[13] Chandler, P., Sweller, J., 1991, Cognitive Load
Theory and the Format of Instruction, Cognition
and Instruction. 8, 293332.
[14] Gould, J.D., Lewis, C., 1985, Designing for
Usability: Key Principles and what Designers
Think, Communications of the ACM. 28, 300
311. doi:10.1145/3166.3170.
[15] Calero Valdez, A., Brauner, P., Schaar, A.K.,
Holzinger, A., Ziefle, M., 2015, Reducing
Complexity with simplicity - Usability Methods
for Industry 4.0, 19thTriennial Congress of the
International Ergonomics Association (IEA
[16] Gamma, E., Helm, R., Johnson, R., Vlissides,
J., 1995, Design Patterns, Addison-Wesley
[17] B rauner, P., Runge, S., Groten, M., Schuh, G.,
Ziefle, M., 2013, Human Factors in Supply
Chain Management Decision making in
complex logistic scenarios, Proceedings of the
15th HCI International 2013, Part III, LNCS
8018, Springer-Verlag Berlin Heidelberg, Las
Vegas, Nevada, USA, pp. 423432.
[18] Philipsen, R., Brauner, P., Stiller, S., Ziefle, M.,
Schmitt R., 2014, Understanding and
Supporting Decision Makers in Quality
Management of Production Networks, in:
Advances in the Ergonomics in Manufacturing.
Managi th tis  th t 
Proceedings of the 5th International Conference
on Applied Human Factors and Ergonomics,
AHFE 2014, CRC Press, Boca Raton, pp. 94
[19] Philipsen, R., Brauner, P., Stiller, S., Ziefle, M.,
Schmitt, R., 2014, The role of Human Factors in
Production Networks and Quality Management.
How can modern ERP system support
decision makers?, First International
Conference, HCIB 2014, Held as Part of HCI
International 2014, Heraklion, Crete, Greece,
June 22-27, 2014. Proceedings, LNCS 8527,
Springer Berlin Heidelberg, pp. 8091.
[20] Mittelstädt, V., Brauner, P., Blum, M., Ziefle, M.,
2015, On the visual design of ERP systems
The role of information complexity, presentation
and human factors, 6th International
Conference on Applied Human Factors and
Ergonomics (AHFE 2015) and the Affiliated
Conferences, AHFE 2015, pp. 270277.
[21] D IN/EN/ISO 924111, 1998, Ergonomic
requirements for office work with visual display
terminals (VDTs): Guidance on usability
specifications and measures.
Philipp Brauner is a researcher at the
-Computer Interaction Center
at RWTH Aachen University and
holistic and viable ICT
to increase workers
and job satisfaction.
Martina Ziefle is professor at the
Communication Science
founding member of the Human-
Computer Interaction Center
RWTH Aachen University
. Her
research addresses human
computer interaction and technology
in different technologies
and using contexts, taking demand
of user diversity into account.
... Simulated business and supply chains are an established method to identify and quantify supply chain disruptions, to convey knowledge and expertise about supply chain management and material disposition, as well as to study human decision making in controlled experimental, although sufficiently complex scenarios [21]. An early example are behavioral studies on the Beer Distribution Game by Sterman [22]. ...
Conference Paper
Full-text available
Supply Chains and production networks are complex sociotechnical cyber-physical systems whose performance is determined by system, interface, and human factors. While the influence of system factors (e.g., variances in de- livery times and amount, queuing strategies) is well understood, the influence of interface and human factors on supply chain performance is currently insuffi- ciently explored. In this article, we analyze how performance is determined by the correctness of Decision Support Systems and specifically, how correct and defect systems influence subjective and objective performance, subjective and objective compliance with the system, as well as trust in the system. We present a behavioral study with 50 participants and a business simulation game with a market driven supply chain. Results show that performance (-21%), compliance (-35%), and trust (-25%) is shaped by the correctness of the system. However, this effect is only substantial in later stages of the game and occluded at the be- ginning. Also, people’s subjective evaluations and the objective measures from the simulation are in congruence. The article concludes with open research questions regarding trust and compliance in Decision Support Systems as well as actionable knowledge on how Decision Support Systems can mitigate supply chain disruptions.
... In a recent article we compiled four basic methods for using game-based learning environments and business simulation games as empirical and interactive research environments [19]: The various in-game metrics (e.g., company's profit, stock levels, and customer complaints) can be related to gaps in knowledge and identify learning potentials, classify the task fit of job applicants, isolate underlying human-factors relating to performance, and help to critically benchmark user-interface aspects [18]. Serious games are rarely evaluated using formal technology acceptance models. ...
Globalized markets, product complexity, and increased requirements on quality lead to growing complexity of business and manufacturing processes. Game-Based learning environments and business simulation games offer great potential to prepare employees the increasing complexity. As it is unclear who profits most from these learning environments, we did a study with 66 partici- pants on a game for conveying Production Planning and Control and Quality Management. In our research model we combined personality attributes and two common technology acceptance models to determine factors projecting perfor- mance in the game and projected later use of business simulation games in gen- eral. We found that main drivers for usage are performance expectancy and trans- fer of skill, i.e., the perceived applicability of the learned knowledge and skills for the later work. The attained performance is unrelated to the projected use. The article concludes with guidelines to increase the likelihood for the later use of business simulation games and for increasing their overall efficacy.
Full-text available
With the advent of artificial intelligence (AI) methods, smart decision support systems are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective decision support systems (DSS), the influence of correct and erroneous DSS's on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance , trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (-47%), reduces usefulness (-58%), reduces acceptance (-62%) and reduces overall performance (-32%). Overall , the defective system was less user-friendly (-27%). Nevertheless, users who rated the system's usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous decision support systems.
Conference Paper
Full-text available
Deep penetration of modern information and communication technology in manufacturing companies (vertical integration) and across supply chains (horizontal integration) leads to an increasing amount and complexity of information that needs to be perceived, filtered, processed, and reacted to. Yet, the human factors that influence performance are insufficiently understood. This article outlines that individual factors, interface factors, and system factors affect overall performance and it presents two complementary research methodologies for identifying and quantifying these factors. On one side, we show that controlled laboratory experiments with singular decision tasks can precisely identify and quantify factors contributing to performance. On the other side, we use business simulation games with realistic decision tasks that can quantify the complexity of the underlying system. Our studies show that information amount, complexity, and presentation affect performance and that Decision Support Systems can increase performance and decrease error rates if and only if they are designed correctly. The article concludes with a research agenda to specifically understand which factors influence performance and how humans in the loop can be supported.
Conference Paper
Full-text available
The Internet of Things will influence how production systems of the future will work. An expected change is the increase in complexity when production system will start communicating with each other. This complexity can be managed by applying socio-technical approaches. This includes analyzing a production system regarding the domains: tasks, actors, technology, and structure. We identified three topics per domain that we think need further investigation, as they may influence complexity to a strong degree. For tasks we suggest looking into urgency, habituation, and strategies; for actors we suggest looking into skills, human factors, and user diversity. In the field of technology we find visualization, decision support, and interactions to be most pressing, while in the field of structure we see responsibility, delegation, and communication.
Conference Paper
Full-text available
The investment in Enterprise resource planning (ERP) systems is indispensable for manufacturing companies to obtain competitive advantages in the globalized market. However, end-users are confronted with complex interfaces and poor usability of these systems. In the present multi-factorial experiment, we examined the effects of information complexity and presentation as a key aspect of usability with consideration of human factors on decision quality. By using alphanumeric tables of simulated ERP system data to make a decision, users’ decision quality dropped with increasing information complexity and the use of a poor presentation. Furthermore, interactive effects of two different aspects of information complexity (data amount and task complexity) as well as compensatory effects through human factors were revealed. These findings show the importance of empirical user studies in this field and provide several practical implication. Especially, user-centered design processes can substantially contribute to a successful implementation of complex information systems, such as ERP systems.
Conference Paper
Full-text available
At RWTH Aachen University the research cluster " Production Technology for High-Wage Countries " engages in advancing the polylemma of production. In many cases engineers and physicists develop simulation tools, machine interfaces, and data exploration tools but lack essential training in Human-Computer Interaction. Without proper training the interaction of visual, cognitive and task complexity can lead to solutions that are valuable only to the developers themselves, but are not usable without extensive training. We show the most critical ergonomic factors for developing software in a scientific engineering setting that focuses on complex problems. We present an overview of usability methods as well as complexity reduction methods and their applicability in engineering software design. We present an exemplary study for the case of supply chain management, where the approaches were successfully integrated into a serious game not only serving as an investigatory tool but also as a training utility for supply chain managers. Practitioner Summary: This overview aims to be used as an entry point for engineers interested in usability and its applicability from a simulations and visualizations point of view. Factors that create complexity and how they can be reduced are presented. Usability methods are evaluated in regard to their applicability.
Full-text available
Quality management is an important function for viable production networks. In order to qualify decision makers to understand the fundamental principles of quality management in production networks, a game-based simulation and learning environment is developed, which can furthermore be used to understand how human factors influence the quality of decisions in complex production networks. Previous studies have shown that underlying factors must exist, which predict the players’ performance. It is currently unexplored, which factors are contributing to high performance. To deeper investigate which human factors are critical for supply chain success and to further refine the quality management game a series of studies were examined. As expected, expertise had a great impact on performance, however contrary to the expectations cognitive skills had none. The refined decision dashboard, with seamlessly integrated self-adapting visualizations on key performance indicators, had a significant positive impact on game performance. The studies suggest that the developed game is a valuable contribution for the qualification of quality managers, as the quality of decision increased.
Conference Paper
Full-text available
Quality management is an important aspect of viable manufacturing processes. In order to qualify decision makers to understand the fundamental principles of quality management in production networks, we developed a game-based simulation and learning environment that can furthermore be used to understand how human factors influence the quality of decisions in complex production networks. Previous studies have shown that underlying human factors must exist, that predict the players’ performance, but it is currently unexplored which factors contribute to high perfor-mance. To investigate these human factors in more depth and to further refine the quality management serious game, we conducted a series of studies. As expected, expertise had a great impact on performance. However, contrary to our expectations, cognitive skills had no influence. The refined decision dashboard with seamlessly integrated self-adapt-ing visualizations on key performance indicators had a significant positive impact on game performance. Most im-portantly, the studies suggest that the developed game is a valuable contribution to the vocational training of quality managers as the quality of their decisions is increased.
Full-text available
This article examines developments in business simulation gaming during the past 40 years. Covered in this article are a brief history of business games, the changing technology employed in the development and use of business games, changes in why business games are adopted and used, changes in how business games are administered, and the current state of business gaming. Readers interested in developments in other areas of simulation gaming (urban planning, social studies, ecology, economics, geography, health, etc.) are encouraged to look at other articles appearing during the 40th anniversary year of Simulation & Gaming and at the many fine articles that appeared in the silver anniversary issue of Simulation & Gaming (December 1995).
Conference Paper
Any system designed for people to use should be (a) easy to learn; (b) useful, i.e., contain functions people really need in their work; (c) easy to use; and (d) pleasant to use. In this note we present theoretical considerations and empirical data relevant to attaining these goals. First, we mention four principles for system design which we believe are necessary to attain these goals; Then we present survey results that demonstrate that our principles are not really all that obvious, but just seem obvious once presented. The responses of designers suggest they may sometimes think they are doing what we recommend when in fact they are not. This is consistent with the experience that systems designers do not often recommend or use them themselves. We contrast some of these responses with what we have in mind in order to provide a more useful description of our principles. Lastly, we consider why this might be so. These sections are summaries of those in a longer paper to appear elsewhere (Gould & Lewis, 1983). In that paper we elaborate on our four principles, showing how they form the basis for a general methodology of design, and we describe a successful example of using them in actual system design (IBM's Audio Distribution System).