Content uploaded by Leigh Lawton
Author content
All content in this area was uploaded by Leigh Lawton on Jan 07, 2016
Content may be subject to copyright.
A preview of the PDF is not available
The Relationship between Financial Performance and Other Measures of Learning on a Simulation Exercise
Abstract
Analysis was conducted to assess the relationship between financial performance on a business simulation exercise and various other measures of student learning. Financial performance was represented by a composite performance score that rated student companies based on net income, return on investment (ROI), and return on assets (ROA) achieved in a competitive, computer-based management simulation. (Although highly intercorrelated, a 1988 study by House and Napier found the combination of these measures provided the best overall measure of a company's financial performance.) Little or no relationship was found between the performance score and the other measurements used to assess student learning.
... For example, a good performance within a business simulation is not an indicator of learning achieved (Anderson and Lawton, 1990;Washbush and Gosenpud, 1993;Wellington and Faria, 1991). Experiments to establish whether students are able to discern the simulated phenomenon have yielded generally negative conclusions (Whiteley, Ledue and Dawson, 2004;Dickinson, Whiteley and Faria, 1990;Dickinson and Faria, 1994;Wolfe and Jackson, 1989). ...
... However, Faria and Whitely (1990) were unable to find a significant relationship at the individual level when comparing performance in simulation and multiple choice examination. Other similar studies concur with these conclusions 38 (Anderson and Lawton, 1990;Washbush and Gosenpud, 1993;Wellington and Faria, 1991). For example, Washbush and Gosen (2001) analysed the possible learning gained from ten runs of a management game. ...
... Traditionally, it has been difficult to 'concretise' learning variables and hence design a research instrument to capture learning (Go sen and Washbush, 2001;. Researchers have deduced that simple measures such as performance during the simulation experience cannot be used to measure learning (Anderson and Lawton, 1990;Washbush and Gosenpud, 1993;Wellington and Faria, 1991;Faria and Whitely, 1990;Washbush and Gosen, 2001). ...
p>A Total Enterprise Simulation (TES) is a virtual, gaming representation of the functions of a business and its marketplace. Traditionally, business gaming simulations of this kind have been used as learning tools aimed at developing the business acumen of students. Past research conducted by authors in this field has shown that, although an enjoyable exercise for users, this type of simulation has lacked educational and representational validity. Researchers have found insubstantial evidence of learning effectiveness and there has been simplistic modelling of the real world business environments. Consequently, this research project has investigated the TES learning medium with the aim of establishing the extent to which TES can be validated from both an educational and representational perspective.
A multi-case study design has been implemented in which data has been sourced by engaging participants from varied work environments within a TES and subsequently analysing their perspectives. In this way, opinions of postgraduate management students, aerospace executives, executives from Kraft Foods, and executives from QBE Insurance have been compared and assessed. The author has pulled together literature to provide a foundation for simulation designers aiming to develop simulation exercises that are educationally valid. Within the literature review, a taxonomy of simulation has been compiled which the author has used to define the simulation area of focus for this study; business management development simulation - the author’s brand of TES. Past validity studies have been inconclusive, and therefore this study has combined the literature on learning, simulation design and research methodology to formulate a methodology for validating TES. This study has applied this methodology to four different case organisations which has contributed to knowledge by yielding evidence strongly supporting the educational and representational validity of TES within this substantive enquiry.</p
... Similar to problem-based learning, SBL fosters the transfer of learning because knowledge is acquired and applied in authentic situations. However, given the broad range of learning outcomes of SBL, the measurement of those outcomes is challenging (Anderson & Lawton, 1992). Therefore, self-assessment methods are widely used (Sitzmann, Ely, Brown, & Bauer, 2010), which is problematic (Anderson & Lawton, 2009). ...
... The system feedback provides periodical information on in-game success (e.g., the financial performance of a virtual company) and drives the competition among players. Of course, in-game success is not necessarily an indicator of learning outcomes (Anderson & Lawton, 1992). Nevertheless, in-game success can be an indicator that participants are engaged and that they are able to cope well with the given complexity, but in-game success might also be due to underperforming competitors or good luck in decision-making. ...
Simulation-based learning (SBL) as a learner-centred educational approach fosters students‘ experiential learning by providing authentic tasks in a real-world oriented learning environment. SBL settings are supposed to integrate different dimensions of learning including cognitive, affective and social aspects. Simulation games as a widely used tool in SBL are characterized among others as collaborative, team-based environments fostering learners’ understanding of concepts and improving their ability to apply their theoretical knowledge in practical fields. The use of and research on simulation games has strongly increased throughout the last decades, but the few empirical findings in the literature are ambiguous. The present study contributes to a better understanding of relations between collaborative facets, emotional experience, in-game success as a performance index and learning outcomes during a complex general management simulation. It also focuses on the use of process journals to gather data during the simulation game process in classes of business informatics students in their last semester at a German Cooperative State University. Data of 49 third-year students (m = 36, f = 12, missing = 1; age ranged from 20 to 25) was collected on three occasions: (1) A self-report questionnaire prior to the simulation game. (2) A periodic process journal was administered during the simulation game at the end of each of the six team phases to collect data on participants’ perceived team collaboration and emotional experience. (3) After the simulation game, declarative, conceptual, and procedural knowledge was assessed. Correlation analysis showed medium scores in a range between 0.38 < r < 0.76 when significant, U-test showed results between 0.39 and 0.81 when significant. Our results indicate an association between a cohesive atmosphere including psychological safety and a structured team organization and positive epistemic emotions on the one hand with performance and conceptual as well as procedural knowledge on the other hand. Hence, we argue for the need to organize and support team processes during business simulation games carefully when facilitating such environments with students, whereas we could not find support for a strong connection between learners’ personality with simulation game outcomes.
... The trouble begins with the very definition of "learning" (Gentry et al., 1998;Keys et al., 1990). Many studies (e.g.: Anderson et al., 1992;Hancock et al., 1961) conceptualised "learning" as improvement in controlling the microworld and improving "game performance" (i.e., better performing in the microworld) (Faria, 2001;Gosen et al., 2004;Warren et al., 1999). As a result, these studies assumed that the higher the score on the performance measures in the microworld, the more has been learned about how the microworld works and therefore how the real world works. ...
... However, the capability of microworlds to stimulate a learning transfer has not yet been confirmed (Größler, 2001). Moreover, studies have even shown that these assumptions do not always hold the truth (Anderson et al., 1992;Bakken, 1993;Diehl, 1992;Thorngate et al., 1987;Washbush et al., 2001;Wellington et al., 1992). In addition, the microworld in these experimental studies also serves a double function: it is the treatment as well as a means to measure the results of the treatment. ...
... In both cases, results of these investigations clashed. In fact, even though most researchers found that simulations are able to improve students' learning (e.g., Klassen and Willoughby, 2003;Taylor and Chi, 2006;Ranchhod et al., 2014;Mustata et al., 2017;Hernández-Lara and Serradell-López, 2018;Goi, 2019;Houk et al., 2019), some have come to the opposite conclusion (e.g., Anderson and Lawton, 1992;Druckman and Ebner, 2008;Frantz et al., 2019), and yet others have shown that the success of the business game in terms of students' learning depends on other factors, such as the teacher's involvement and commitment (e.g., Sitzmann, 2011) or the alignment of simulation games' tasks with lecture content (Frantz et al., 2019). Accordingly, researchers shifted their interest in trying to understand which variables are capable of (positively or negatively) influencing students' learning within the simulation game environment, as well as their performance and its impact on their decision-making process, focusing on both cognitive and affective variables. ...
This paper investigates the influence of reflective and intuitive thinking on students' learning and performance within a simulation game environment. In particular, the decision making processes of 120 graduate students-divided into teams-taking part in a simulation game has been investigated. Results show that an average level of reflective thinking leads teams to reach a higher performance, compared to teams with a high or low level of reflective thinking. Moreover, results demonstrate that participation in simulation games do not improve the reflective thinking of participants, whilst simulation gaming is able to positively affect their learning capacity. Teachers can use these findings to design the 'best team composition' of students, allowing them to increase their overall performance in simulation gaming. Yet, this work-reinforcing prior literature-bolsters the ability of simulation games in improving learning skills of students.
... Other articles searched specifically for improved understanding of concepts resulting from the simulations, but they did not find such evidence. For example, Anderson and Lawton (1992) did not find a connection between simulation financial performance and learning measures built on Bloom's Taxonomy. Wellington and Faria (1995) found mixed results regarding learning from the simulation, measured by questionnaires, quizzes, and decision quality in subsequent rounds. ...
This article introduces a simulation, Marketplace Live, and compares students’ simulation performance with their course performance. The sample is drawn from 13 sections of the author’s Marketing Concepts (Principles of Marketing) course. The results support the idea that marketing simulations do contribute to learning marketing concepts. Evidence for this included (1) the partial mediation of academic ability on overall course performance by effort (indicated by time) and simulation performance, (2) a correlation between improvement in simulation performance with course grade improvement, and (3) correlations between simulation performance measures and performance on the final (comprehensive) course exam.
For full paper, see: http://www.tandfonline.com/eprint/eHAqt7UccSYunaxrHmB7/full
Given the applied nature of business disciplines, simulations are useful in helping students bridge theory learned in the classroom and practical skills needed to succeed in industry. Much of the learning that occurs in simulations comes from working in teams as students work together to solve problems and defend their ideas. Thus, while marketing educators are often interested in optimizing both individual and team learning outcomes, many scholars have only focused on analysis at one level or the other. This research reports findings from a multilevel SEM (MSEM) model that simultaneously investigates individual and team learning outcomes on critical thinking in a marketing simulation across multiple sections of a Principles of Marketing course. Guided by attribution theory, the results suggest that the effect of team size upon simulation-enhanced critical thinking is fully and negatively mediated by objective team performance growth over the course of the game. Moreover, generalized self-efficacy is positively associated with students’ perceptions that participation in the simulation enhance critical thinking abilities. Finally, need-for-cognition negatively moderates the effect of self-efficacy upon simulation-enhanced critical thinking. The paper concludes with a discussion of study implications, limitations, and opportunities for future research.
To achieve the desired learning outcomes it is critical that developers and users of simulations understand human decision-making. How participants will make decisions in the simulation is a function of both the participant's expertise and their interaction with the exercise design. It is an important pedagogical issue to know whether the design of the simulation reinforces and builds a participant's ability to respond in a normative reasoned fashion to a decision situation, or to experience the situation in its complexity and respond in a synthetic intuitive fashion. To comprehend the implications of these two viewpoints I present the debate between promoters of the normative views and the descriptive views on decision-making. By performing a critical analysis of these different perspectives, I offer insight on how the decision-making philosophy used in the design of simulations affects both the use of the simulation and the measurable learning outcomes.
While some people prefer to process information upon the outer world (extroverted), others prefer to process it reflectively (introverted). This difference leads to distinct experiences and therefore to distinct learnings, despite the students being in the same learning environment. The research problem was the learning asymmetry in Production Planning and Control (PPC) by students with different Active-Reflective learning styles in a business game-based learning environment. The research question was: does the use of a business game generate a more effective learning process for students with an Active learning style? The main objective was to test the meshing hypothesis of the Active-Reflective learning style with the learning environment. The learning outcomes were assessed by means of an exam. Using Bloom's taxonomy (ANDERSON; KRATHWOHL, 2001) the exam assessed learning outcomes of the cognitive processes (remembering, understanding, applying, and analyzing) and of the types of knowledge (factual, conceptual and procedural). Active-Reflective learning style of each student was assessed using the Felder and Silverman (1988) inventory of learning style (ILS). Two quasi-experimental pre-test and post-test with separate samples were conducted; the first one, with 375 students, to evaluate the differences in the learning of students with different learning styles (Active-Reflective). The second one, with 100 students, aimed to study the effect of the increase of reflective activities on asymmetry of students learning due to their learning styles (Active-Reflective). The first quasi-experiment showed lower learning by Active students. To explain the results, the Kolb, D. (1984, p. 42) experiential learning theory was used. The experiential learning relies on the four stages of a cycle: concrete experience, reflective observation, abstract conceptualization and active experimentation. It was assumed that, the lower learning of the students with Active learning style, was caused by the lack of reflective activities during the business game. The Reflective students, who prefer the transformation of knowledge by reflective observation (introspection), were encouraged, by the game, to experiment actively the knowledge. They would have been able to close the learning cycle more effectively than the Active students: by reflecting, due to their personal preferences, and, by acting, due to the circumstances imposed by the business game. To confirm this assumption the second quasi-experiment was developed increasing the reflective activities of the experience. The results, then, led to the acceptance of the learning equality of students regardless of their Active-Reflective learning style. Based on the references and data observed in this study, it is suggested that the learning agents consider, in their teaching and learning plans, the cyclical nature of experiential learning, as defined by Kolb, D. (1984). In accordance with the studies of Felder and Spurlin (2005), of Belhot and Seno (2009) and of Felder (2010), it is suggested providing learning opportunities in a balanced way to students with any learning style. To the educators that already use games, it is important to encourage students, especially the Active ones, to review the results of the game and make reflective observations.
The objective of the current research is the development of a new conceptual model aiming to represent, in an integrated manner, the many situations of conflict of interests as a basis for analysis and design of hierarchical multiagent systems control and for the improvement of the methodology for betterment of the managers’ skills to deal with the strategic management of such conflicts. The investigation method used was a comparative analysis of the unique characteristics of classical games from Game Theory – Nash, Stackelberg, Pareto, and Minimax – searching, among them, their commonalities and differentiations. This investigation identified two distinct dimensions that enabled the conception and construction of a matrix to represent, in a integrated form, those four games mentioned above. The resulting conceptual model provides a comprehensive analytical scheme, inspired in the theory of games, and is used to explain, describe, interpret and forecast behaviors of autonomous agents involved in situations of conflict of interests and, in some cases, to prescribe the more adequate decisions. The Strategic Games Matrix (SGM) proposed and used in this study establishes a conceptual reference framework mapping six different types of games. In it, the assumptions for classic game models, among others for limit-cases, are used in an integrated and complementary manner. The SGM deals with both competitive and cooperative games, as well as balanced and unbalanced ones, taking into consideration both the players´ competitive postures and the power-ratio assumed by each one. The SGM contemplates in an innovative way the treatment of multiple simultaneous strategic sub-games among the agents involved. The application of the SGM concepts to complex systems – hierarchical or not –, with multiple autonomous intelligent interactive agents, provides a methodology of utility for analysis and design of their control strategies. An important part of this study is the exploratory experiments with pedagogical purpose. Such business games, played in a computer, indicate that the participants increase their perception to understand the various games to play, and their ability to act at each one of them. This use of the SGM leads each participant to analyze conflict of interests’ situations and to improve its strategic decisions: Through an interactive dynamic process of trial and error he/she ends up learning how to make better decisions taking into consideration the likely decisions of the other agents involved as well as her/his evaluation of the consequences of their choices.
This paper introduces a video-game designed to support teaching introductory economics at undergraduate level. In order to test its effectiveness compared to traditional textbook learning we designed a laboratory experiment. Results show no evidence that playing the video-game leads to lower exam performance than reading a textbook, neither for multiple-choice nor for essay questions. We also find no gender bias and no effect of announcing the test prior to the learning task or thereafter. However, game behavior appears to be related to test performance, and differently so for different types of learning. Students perceive the two learning tools similarly in terms of understanding requirements or usefulness, but enjoyed the video-game considerably more. Interestingly, although women enjoyed the game less than men, they do not differ in their test performance.
Despite the proliferation and widespread usage of simulation games in the field of business education, the pedagogical value of this instructional aid remains unclear. The present study, using a controlled setting, sought to determine whether incorporating a business simulation game in a principles of marketing course improves the acquisition of marketing knowledge. The results suggest that simulation games are an effective means by which to improve quantitative skills but are not an effective means by which to improve the acquisition of applied or theoretical knowledge.
Describes and evaluates research studies which focus on game learning to meet course objectives, e.g., studies which hold one variable constant and compare the teaching of one group of students playing a game with a 2nd group using another mode of teaching. Games are subdivided into several categories including general management games (e.g., Carnegie Tech Management Game and the Harvard Business School Game) and functional business games (e.g., DISPATCHO, MARKSIM and FINANSIM). It is concluded that very little "hard" research has been done on gaming, especially concerning what players learn to meet course objectives. Several directions for future research are suggested, including clarification of incongruencies in existing research, improvement of measurements of intangible factors such as cautiousness and risk-taking, and adoption of a cost-utility approach in researching existing games rather than in proliferating new ones. (38 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Operationalizing Test of a Model of the Use of Simulation Games and Experiential Exercises
- J W Gentry
- A C Bums
Gentry, J. W. and A. C. Bums (1981), "Operationalizing Test of
a Model of the Use of Simulation Games and Experiential
Exercises. in W. D. Biggs and D. J. Fritzsche (eds.)
Developments in Business Simulation and Experiential
Exercises. Vol. 8, pp. 48-52.
A Review of Learning Research in Business Gaming
- B Keys
Keys. B. (1976) A Review of Learning Research in Business
Gaming. in B. H. Sord (ed.) Computer Simulation and
Learning Theory. Vol. 3.