Content uploaded by Vahid Faghihi
Author content
All content in this area was uploaded by Vahid Faghihi on Dec 25, 2018
Content may be subject to copyright.
PostedonReseachGate
PredictionMarketsasa
TeachingToolFor
UndergraduateEngineering
Students
Damnjanovic,I.;Faghihi,Vahid;Scott,C.;McTigue,E.;and
Reinschmidt,K.
JournalofProfessionalIssuesinEngineeringEducationandPractice‐SubmittedJune21,2011;accepted
May22,2012;postedaheadofprintMay24,2012.doi:10.1061/(ASCE)EI.1943‐5541.0000127
Accepted Manuscript
Not Copyedited
1
EDUCATIONAL PREDICTION MARKETS: A CONSTRUCTION PROJECT
MANAGEMENT CASE STUDY
Ivan Damnjanovic1, Vahid Faghihi2, Chyllis Scott3, Erin McTigue4, and Kenneth Reinschmidt5
ABSTRACT
Effective teaching of engineering concepts relies both on carefully designed lesson plans that
meet specific learning outcomes, and on classroom activities that students find engaging.
Without student engagement, even the best designed plans will fail to meet their outcomes. In
other words, students need to be actively involved in the learning process. The objective of this
paper is to present a case study of applying a novel active learning method, specifically
educational prediction markets (EPM), for teaching project management classes at a major
research university. This method was investigated for its effectiveness in engaging students, as
well as promoting learning of probabilistic reasoning without explicit teaching. Student surveys,
following the EPM implementation, revealed both advantages and disadvantages. The two key
benefits reported by the students were: a) providing better connections between the materials
taught in the class and realities of construction projects, and b) increasing overall interest and
enthusiasm in learning about project risk management due to the game-like nature of the process.
The main disadvantage was disengagement by a subset of students due to perceptions that fellow
students were manipulating the market results.
1 Assistant Professor, Texas A&M University, Zachry Department of Civil Engineering, College Station, TX 77843-3136, E-
mail: idamnjanovic@civil.tamu.edu
2 Ph.D. candidate, Texas A&M University, Zachry Department of Civil Engineering, College Station, TX 77843-3136; E-mail:
savafa@tamu.edu
3 Ph.D. candidate, Texas A&M University, Department of Teaching Learning and Culture, College Station, TX 77843, E-mail:
chyllisscott@tamu.edu
4 Assistant Professor, Texas A&M University, Department of Teaching Learning and Culture, College Station, TX 77843, E-
mail: emtigue@tamu.edu
5 J. L. Frank/Marathon Ashland Petroleum LLC Chair in Engineering Project Management, Zachry Department of Civil
Engineering, College Station, TX 77843-3136, Email: kreinschmidt@civil.tamu.edu
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
2
INTRODUCTION
Teaching how to identify, assess, and manage project risks presents many challenges. One of the
greatest challenges instructors face is the inherent difficulty of linking probabilistic predictions
with actual observations. For example, we can predict that the probability of rain tomorrow is 80
percent, but then, it may not rain. Was our prediction good? It is difficult to answer this question
because predictions are only probabilistic propositions, not deterministic observations.
This conundrum of teaching probability concepts is particularly visible when trying to predict
outcomes such as completion time of construction projects. While some engineering disciplines
can use laboratory experiments to validate the uncertainty in prediction, this approach cannot be
applied to construction management. For example, we can run 100 laboratory tests to determine
the probability that a concrete sample taken from Batch A will withstand the stresses required by
seismic codes. This empirical approach to determining risk is visible and obvious, even for
novices to the field such as undergraduate students. However, for complex projects, where one
cannot directly observe outcomes and estimate probabilities - how does one assess a probabilistic
prediction?
In teaching project risks instructors often rely on numerical methods, such as Monte Carlo
simulations (MCS). Even though simulations may be more transparent than other methods, they
can be considered susceptible to manipulation by the simulator. In our teaching experience,
students often view the MCS approach as too abstract, which in turn can make students
suspicious and disengaged from further exploration. This feedback loop - where a lack of realism
diminishes student engagement; and deficiency in engagement prevents further investigation to
understand abstract concepts - represents a major hurdle in teaching engineering project risk
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
3
management. Until we gain students’ attention and engagement by grounding the learning within
real world examples, the learning process is stalled.
The objective of this case study is to document implementation of EPM in undergraduate project
management classes. For background, prediction markets have recently found applications in
many fields including education, project management and scientific research (Hanson 1999,
Arrow 2008). In a prediction market a participant buys or sells shares in the realization of a
specific well-defined outcome. If the predicted outcome occurs, he/she can exchange the shares
for a “reward” of 100 units per share. If the predicted outcome doesn’t occur, the value of the
shares becomes 0. If a particular outcome is likely, the price of shares will go up (as demand
grows) and vice versa, as the specified outcome seems less likely to happen, the market price
will go down. Hence, prediction markets represent the social trade forums that run for the
primary purpose of aggregating information in an effort to forecast future events (Tziralis &
Tatsiopoulos 2007; Berg & Rietz 2003; Berg et al. 2003). Arguably, the most important issue
with implementation of a market is its performance as a predictive tool (Wolfers and Zitzewitz,
2004). On a practical note, in EPM these prices and rewards do not constitute illegal gambling
because no money is exchanged. Nevertheless, participants are motivated and engaged by non-
monetary rewards, particularly due to the competitive nature of games.
To investigate the extent to which EPM could promote engagement, active learning, and the
sense of realism in teaching, we recently implemented an EPM contest in two undergraduate
construction engineering and management classes focusing on project risk management. We
collected and analyzed feedback data including qualitative observations of the learning process.
This manuscript documents our findings and sets EPM in a larger context of using active
learning methods.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
4
The remainder of the paper is organized as follows. First we provide a brief background on
education theories that support the use of EPM. Next, we present the case study:the synopsis of
the overall process, the data collection process, the results from the survey, and the interpretation
of the findings. Finally, we present lessons learned and summary and directions for future
studies.
BACKGROUND
Active Learning (AL) is often referred to as active engagement strategies that allow students to
participate and engage in higher-order thinking tasks such as analysis, synthesis, and evaluation
(Bloom 1956; Wilke 2003) as they construct their own understanding of the information. In other
words, AL is an instructional method which engages students (Prince 2004); solicits active
participation on the part of the students (Olgun 2009); and integrates learning through
encouragement and feedback (Bernhard 2001).
The underpinnings of AL can be directly linked to constructivism. From the theoretical
perspective of teaching and learning, constructivism hypothesizes that knowledge is constructed
through learning experiences that are facilitated by the others (e.g. teachers, fellow students), in
social processes and relationships (Abdul-Haqq 1998; Vygotsky 1978; Wink & Putney 2002).
Learning is considered a product of ordering and synthesizing new information facilitated by the
interaction of educational materials, instructor, and peer social group (Vygotsky 1978). This is
precisely where EPMs would find their applications: elevating student engagement using social
interactions with peers, through authentic discussions regarding the impact of recent information
on the market prices. There is emerging evidence that when a prediction market is integrated into
traditional communication channels, it can reinforce students’ thinking in a wider range than
what is proposed by the instructor (O’Toole & Absalom 2003). This means that the structure of
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
5
prediction markets would allow students to engage in a learning experience using both channels
of cognition and corresponding actions – analytical (i.e. using formal methods and processes),
and affective (i.e. using “gut feeling” and instincts) (Garvey and Buckley 2010). Hence,
prediction markets meet the conditions required by the Active Learning approach (Buckley et al
2011).
CASE STUDY
The educational prediction market was implemented in two engineering classes focusing on
project management: a) Project Management for Engineers, a multi-disciplinary class for civil,
industrial, and mechanical engineering undergraduates, and b) Civil Engineering Project
Management, a required junior-level class in the civil engineering department. The market
implementation was conducted in the Fall of 2010. The two courses were taught by Drs.
Damnjanovic and Reinschmidt.
The market questions were based on well-defined milestone points of construction activities for a
building project for the University’s veterinary school, while progress data were provided by the
actual construction contractor’s project team. Figure 1 shows a 3-D computer rendering of the
project and a site photo.
INSERT FIGURE 1
A total number of 122 student users participated in the market. Initially, all students were
allocated $5,000 to buy or sell stocks in two predefined project milestone “stocks” (“Structure
Top Out” and “Substantial Dry-in”) that were initially priced at $50 per share, reflecting the
initial likelihood of meeting the schedule milestone points of 50 percent. In other words, students
individually decide to buy (or sell) shares in a particular milestone event which they think will be
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
6
successful (or unsuccessful). Success in EPM would provide them with a “profit” over their
original $5,000 at the end of the market period. Students were advised by the instructors that
their grades in the class could be affected by their participation and success in trading. To
provide the basis for making the trades, the contractor provided internal weekly reports and
schedule updates along with a series of construction site photos. This process gave students
experience with reading authentic weekly reports and schedule updates. Additionally, students
could also visit the site personally and directly observe the status with regard to these milestones.
The EPM used the Hanson model (Hanson 1999) to determine the prices and avoid liquidity
issues (i.e., when someone is willing to buy at a certain price, but no one is offering to sell at any
price). Furthermore, the Hanson model includes the ability to short-sell (i.e., to sell stock without
actually owning it at the time of transaction). Over the course of the semester, students made
over 2,500 trades (buy, sell, or sell short) responding to the contractor reports, weather forecasts,
and other information they found of value in determining whether the stocks are under-valued or
over-valued. Figures 3 and 4 show the price changes throughout the entire duration of the market
including the daily trading volumes (sells and buys) shown as bars at the bottom of the figures.
As can be seen from both figures, peaks in trading volumes tended to occur in intervals of one
week, when the contractors posted their reports and the images from the construction site became
available.
INSERT FIGURE 2
INSERT FIGURE 3
Chronology of Market Implementation.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
7
In the first week of trading, the buyers outnumbered sellers resulting in price increases,
from initially $50 per share to around $80 by the end of the week. This effect can possibly be
attributed to the class presentation given by the construction contractor’s project manager on
September 20, in which he stressed the importance of meeting project milestones. Students’
comments such as the one below show the eagerness to buy the shares: “I think it might be time
to buy this question. NOW! This is the lowest it’s ever been so buy quickly.” In the middle of the
second week, on Sep. 28, the first weekly report and project schedule update were posted. The
new project schedule showed that the first milestone (i.e. Structure Top Out) would be completed
with a 4 day delay, on Oct. 26 rather than Oct 22; and the second one (i.e. Substantial Dry-in)
would be finished by Dec. 2 with a 6 day delay. These anticipated delays affected students’
confidence and some of them started to change their minds about the milestones’ completion
times. Furthermore, the summary daily activity reports indicated problems with steel fabrication.
As a result, in this period, the prices of both stocks fell significantly. The following two sample
comments illustrate students’ feelings: “Daily Report of 9/23 says that rebar for TOMO roof
was fabricated incorrectly…which in turn pushes back TOMO roof pour from this Wednesday to
this Thursday or Friday….not good sign overall.” The revised schedule published on Oct. 6
brought back the belief that the milestone points would be met. These new dates implied a
positive answer to both prediction questions, thus, students responded by buying shares at
discounted prices (in the range from $40 to $50), with anticipation that the final share price
would be $100 and they would cash in at profits from $60 to $50 per share. The comments
below illustrate this sentiment: “Current updates say all systems go so project is now on route to
success in reaching Oct 22! Actually says it will be done on October 20!” or this one: “You’ll
win tons of money. But after the previous report release its looking like the stock will go to 100. I
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
8
wouldn’t recommend shorting unless you personally plan on sabotaging the construction site.”
There were no schedule updates in the Oct. 19 schedule charts, so students were left to wonder
and analyze daily reports for clues. The following comments clearly present the students’
reasoning process: “As per reports, steel erection (structure top out) scheduled to start on Oct.
7th with a crane and end on Oct. 22. According to the reports, the crane doesn’t show up until
Oct. 13th. The steel was erected using a fork lift from Oct. 7th to Oct. 13th. Hmm interesting?”
The revised schedule posted on Oct. 21 indicated “Structure Top Out” milestone with a
completion date of Oct. 27, five days later than in the initial report and the first prediction
requirement, and “Substantial Dry-in” milestone with a completion date of Dec. 3, more than 7
days later than that in the prediction question. Students saw this as sign of sure failure to reach
the milestone and continued to sell and short-sell, driving the price down to close to $0. “Game
Over. Sell while you can or just beg for a bail out, I mean it’s what all the cool kids are doing
today.”, or “Game Over. Might as well short sell before it hits zero. Check the schedule.” On
Oct 22, “Structure Top Out” was closed and the share value set to $0. The reports posted on Nov.
9 and 17 did not make any changes to the falling prices for the second question. On Nov. 25 the
project manager informed the students that the second milestone point was not met as well.
Data Collection
A six-item Likert survey, with a 5 point rating scale, was developed to collect data on students’
learning experience with EPM. The survey required students to rate the value of the prediction
markets as a learning tool for a) engagement for learning, b) understanding realities of
construction, c) knowledge of probability, and d) advanced project management concepts.
Additionally, using the same 5 point scale, the students rated the three other major instructional
resources traditionally used in the course: a) homework assignments, b) lectures, c) textbook.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
9
The ratings of mult iple learning tools on the same scale allo wed for relat ive comparisons
between learning activities. Finally, to collect qualitative data, students answered two open
ended questions about the use of EPM as a learning tool. Surveys were conducted during the
final class meeting for the semester. To reduce bias from social desirability, surveys were
conducted anonymously.
INSERT TABLE 1
Discussion
To what extent do prediction markets as a teaching tool enhance undergraduate students’
engagement in a construction/project management course? In summary, over 70% of students
agreed or strongly agreed that the prediction market was engaging. For comparison, slightly
more students, 74.1%, agreed or strongly agreed that class lecture/lecture notes captured their
interest. However, fewer students, 59.3% and 35.8% respectively, felt that homework
assignments and textbooks were engaging. Although lecture notes and textbooks are certainly
not perfect, they took years to develop with feedback from students and instructors; the
prediction market approach has had much less opportunity for development and refinement.
This may be one of the reasons why lecture notes were ranked to be more engaging than EPM.
In response to whether prediction markets evoked positive emotional responses (i.e., enjoyment)
approximately 63% agreed that this instructional exercise enacted a positive emotional response.
In comparison, slightly more students (66.7%) rated the lecture/lecture notes as eliciting a
positive response. Markedly fewer students rated the homework and textbook, 43.2% and
24.7% respectively, as creating positive emotions.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
10
When analyzing the qualitative reports in response to the open-ended question, In what ways did
the prediction market facilitate your learning in this course?, numerous students cited that the
prediction market exercise helped promote interest in the course. These responses support the
Likert ratings, and also give insight for the rationale for the ratings. The reasons for their
increased interest differed among students. Some students linked their interest in the prediction
market to the project’s connection to real-world construction. For example “it … allowed an
insight into the actual events at a construction project which to me was interesting.” Other
students reported that it was interesting because it helped them to interact with the data, for
example: “It helped to understand how to take data and make predictions …how most [people]
react to changes and the effect it has; I stayed interested this semester, not bored.” Additionally,
students attributed interest to the interaction among students in the prediction process. For
example: “It made the class interesting. People tried to convince each other to predict a certain
way.”
In order to measure both positive and negative emotions (e.g., frustration) evoked by each of
these learning resources, we also included the following statement: This instructional resource
evoked a negative emotional response. We were interested if the competitive aspect of the
assignment would be stressful for certain students. However, the results indicate that the
majority of students did not experience negative emotions from the prediction market learning
experience: 22.2% strongly disagreed, 33.3% disagreed, 19.8% were neutral, 18.5% agreed and
only 6.2% strongly agreed with the statement that predictive markets raise negative feelings. In
other words, 24.7% experienced negative emotions through this learning experience and 75.3%
did not.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
11
Again, the open-ended responses gave insight as to the quantitative ratings as to why some
students experienced negative emotions with this project. Multiple students reported confusion
with using the tool. One student reported that: it took him/her about a month to fully understand
how to use the tool and what everything meant. Other students recommended more instructor
clarification: “I think a little guidance on what exactly was happening when we bought or sold
stock would be helpful. I didn’t really know what I was doing.” It is important to note that the
students were provided with information about how to use the predictive market tool but
deliberately not provided with instruction in probability theory or applications, as one objective
was to observe how students used their best judgment in assessing the likelihood of events.
Additionally, some students were frustrated that the system could be “gamed”. For example, one
student reported “It was easy to manipulate the system that required no expertise in the field of
scheduling.” However, there is no conclusive evidence that manipulation was effective in
moving the markets, although it is true that there was very high variability in the “wealth” of
different students at the end of the simulation. If there was manipulation, it may be because
some students did not take the exercise seriously while other students took the exercise very
seriously. Those students who complained about manipulation may have been rationalizing their
lack of confidence in their own ability to succeed with the prediction markets. More research is
needed to resolve these questions about student involvement that were raised by the short
experiment reported here.
Finally, more than two-thirds of the students, or more precisely 69% agreed or strongly agreed
that the EPM helped them to connect to the real world. For context, slightly more students,
76.6%, agreed or strongly agreed that lecture/lecture notes helped them understand the realities
of construction practice. However, significantly fewer students, 54.4% and 53.1% respectively,
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
12
felt that homework assignments and textbooks helped them understand the realities of
construction. Students’ qualitative responses supported these ratings. Students mentioned issues
regarding application to the real world more often than any other category of response.
Representative responses included: “The weekly reports were good insight into real documents
and real work in the field and how … it differs from homework and textbook examples.”
Students noted that it “helped to develop an understanding of how things [are] on a jobsite …
taught me what project managers need to plan for in a project.” Students also reported that the
exercise gave them confidence because it helped them to familiarize themselves with project
progress reports and the complexity of scheduling in project management situations.
LESSONS LEARNED, LIMITATIONS, AND FUTURE RESEARCH
The findings of this study offer evidence that teaching undergraduate students through an active
learning method. More specifically, implementing prediction markets can promote engagement
and provide a better link between the classroom and what actually occurs on the construction
site. Students were afforded the learning opportunity to use their best judgment, or take relevant
material from the classroom curriculum and apply it to make trades.
Our feedback leads us to several recommendations. To have an effective EPM in construction
management applications, it is essential to: a) select a project, preferably a site that students have
some visual access to, or can walk by, b) carefully select milestone points that are well scoped
and visible to the participants, so there is no ambiguity in interpretation of the outcome, and c)
provide real and unedited data that students can use to make educated trade decisions.
One of the major concerns raised was “gaming the system”. For example, students posting
misleading comments to influence trading decisions of others, so that they can profit from short -
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
13
term price changes. The point is that the actions by some students in manipulating or “gaming”
the system appear to violate some unstated ethical principle that the game should be fair and
equitable to all users. If it is not, then the other students may feel that playing the market might
not be a good use of their time and effort, thus dividing the students into two categories: those
who played the game straight (i.e., treated the market seriously and focused on the prediction that
the milestone event would be on time), and those who tried to manipulate the others by preying
on the fears of the other students and persuading other players to take actions contrary to their
own interests (e.g., by advising other students to buy when the manipulator was sure that the
milestone event would not be on time and the market would fall). Although it was clear to all
that the prediction market was just a game (real money did not change hands) more research
needs to be done concerning the reactions of students to the ethical issues raised by this
manipulation and /or how such manipulation should be addressed. In summary, however, the
reactions by the manipulators, who spent considerable time on preparing their strategies, and by
those straight students who resented these manipulations indicate the high level of engagement in
the prediction markets by both categories of students.
Several limitations of this study focus on the survey instrument, and the single form of data
collection. First, the survey was a newly designed instrument, and had not been previously
tested. In addition, the data collection process for the study was based on a self-report measure.
Answers provided by participants may be influenced by the desire to provide responses which
are socially acceptable. For example, participants may rate their understanding of the realities of
construction practices as agree or strongly agree, because they were successful in utilizing the
prediction market. Third, the study only used one form of data collection. The survey instrument
was only administered to the participants at the end of the treatment with the implementation of
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
14
the prediction markets. The use of a comparison group was not employed which does not allow
us to consider causality of learning.
To gain further insights in ability of predication markets to promote students’ learning of project
risks, we plan to extend the study effort to include full experimental design with control and
treatment groups. More specifically, we plan to look for the effects of both endogenous and
exogenous factors on learning outcomes, as well as investigate modalities of integrating
prediction markets into class material on project risk management.
SUMMARY
This paper presents an investigation of the effects of using prediction markets in teaching
engineering project risks. We focused on two important aspects of teaching that are critical to the
students’ ability to study abstract concepts such as risks: engagement and realism. To analyze
how prediction markets affect engagement and realism in learning environment we have
formulated four research questions, implemented the market, and collected data. The study
results show that the use of prediction markets were particularly valuable for helping students
make connections to real world applications and to help promote interest and enthusiasm which
are integral factors in long-term learning.
REFERENCES
Abdul-Haqq, I. (1998). Constructivism in teacher education: considerations for those who would
link practice to theory. ERIC Clearinghouse on Teaching and Teacher Education.
Arrow, K. (2008). “The promise of prediction markets,” Science, 320, 877-878
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
15
Bernhard J. (2001). Do active engagement curricula give long-lived conceptual understanding?
in R. Pinto & S. Surinach (Eds.) Physics Teacher Education Beyond 2000 (pp. 749-752).
Paris, Elsevier.
Berg, J. E., Nelson, F. D., & Rietz, T. A. (2008). ‘Prediction market accuracy in the long run,”
International Journal of Forecasting, 24 (2), 285-300.
Berg, J. E., Nelson, F., & Rietz, T. A. (2003). “Accuracy and Forecast Standard Error of
Prediction,” Working Paper, Tippie College of Business, University of Iowa.
Bloom, B. (1956). (Ed.). Taxonomy of Educational Objectives, the classification of educational
goals Handbook I: Cognitive Domain. New York: McKay.
Buckley, P., Garvey, J., & McGrath, F. (2011). “A case study on using prediction markets as a
rich environment for active learning,” Computers & Education, 56, 418–428.
Garvey, J., & Buckley, P. (2010). “Teaching the Concept of Risk: Blended Learning Using a
Custom-Built Prediction Market,” Journal of Teaching in International Business, 21,
346–357.
Hanson R. D. (1999). “Prediction markets,” IEEE Intelligent System 14, 16-19
Olgun, O. S. (2009). “Engaging elementary preservice teachers with active learning teaching
methodologies,” The Teacher Educator, 44(2), 113-125.
O’Toole, J. M., & Absalom, D. J. (2003). “The impact of blended learning on student outcomes:
Is there room on the horse for two?” Journal of Education Media, 28 (2-3), 179–190.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
16
Prince, M. (2004). “Does active learning work? A review of the research.” Journal of
Engineering Education, 93(3), 1-9.
Tziralis, G., & Tatsiopoulos, I. (2007). “Prediction markets: An extended literature.” The Journal
of Prediction Markets, Vol. 1, 7591.
Vygotsky, L.S. (1978). Mind in society: The development of higher psychological functions.
Cambridge, MA: Harvard University Press.
Wilke, R. R. (2003). “The effect of active learning on student characteristics in a human
physiology course for nonmajors,” Advances in Physiology Education, 27(4), 207-223.
Wink, J., & Putney, L. (2002). A Vision of Vygotsky. Boston, MA: Allyn and Bacon.
Wolfers, J., & Zitzewitz, E. (2004). “Prediction Markets,” Journal of Economic Perspectives, 18
(2), 107-126.
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
FIGURE CAPTION LIST
Figure 1. 3D Model of TAMU Vet Imaging and Cancer Treatment Center and the Site Photo
Figure 2. Price Changes for “Structure Top Out” Milestone Point
Figure 3. Price Changes for “Substantial Dry-in” Milestone Point
)LJXUH&DSWLRQOLVW
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Accepted Manuscript
Not Copyedited
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
)LJXUH
Accepted Manuscript
Not Copyedited
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
)LJXUHQHZ
Accepted Manuscript
Not Copyedited
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.
Table 1: Students’ Survey Results Related to Effectiveness of Prediction Markets
Likert Ratings (in percentages)
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Q1: Prediction markets captured my interest and/or attention:
28.4
42
14.8
9.9
4.9
Q2: Prediction markets evoked positive emotional responses (e.g., enjoyment).
50.6
12.3
21
8.6
7.4
Q3: Prediction markets evoked negative emotional responses (e.g., frustration).
6.2
18.5
19.8
33.3
22.2
Q4: Prediction markets as a teaching tool enhance understanding of the realities of construction
39.5
29.6
16
8.6
6.2
7DEOH
Accepted Manuscript
Not Copyedited
Journal of Professional Issues in Engineering Education and Practice. Submitted June 21, 2011; accepted May 22, 2012;
posted ahead of print May 24, 2012. doi:10.1061/(ASCE)EI.1943-5541.0000127
Copyright 2012 by the American Society of Civil Engineers
J. Prof. Issues Eng. Educ. Pract.
Downloaded from ascelibrary.org by Texas A&M Univ College Station on 11/01/12. Copyright ASCE. For personal use only; all rights reserved.