Content uploaded by Patrick C Suermann
Author content
All content in this area was uploaded by Patrick C Suermann
Content may be subject to copyright.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 574
www.itcon.org - Journal of Information Technology in Construction - ISSN 1874-4753
EVALUATING INDUSTRY PERCEPTIONS OF BUILDING
INFORMATION MODELING (BIM) IMPACT ON CONSTRUCTION
PUBLISHED: August 2009 at http://www.itcon.org/2009/37
EDITOR: Messner J
Patrick C. Suermann, Maj, USAF, P.E.
PhD Candidate, The University of Florida
suermann@ufl.edu
Raja R.A. Issa, Ph.D., J.D., P.E.
Professor, The University of Florida
raymond-issa@ufl.edu
SUMMARY: This research assessed perceptions about the impact of the implementation of Building Information
Modeling (BIM) on construction projects through data collection in three surveys. Survey questions centered on
impact with respect to six primary construction key performance indicators (KPIs) commonly used in the
construction industry as accepted metrics for assessing job performance. These include: quality control (rework),
on-time completion, cost, safety (lost man-hours), dollars/unit (square feet) performed, and units (square feet) per
man hour. Qualitative data was collected through a survey instrument intended to assess practitioners’ perceptions
about BIM impacts on the six Key Performance Indicators. The first survey was targeted at National Institute of
Building Sciences (NIBS) Facility Information Council (FIC) National BIM Standard (NBIMS) committee members.
The survey results indicated that the respondents felt that a BIM-based approach improves construction metrics
compared to construction without BIM. Specifically, the highest three ranking KPIs in order of most favorable
responses were quality, on time completion, and units per man hour. The second tier of favorable responses included
overall cost and cost per unit.
KEYWORDS: BIM, Construction, NBIMS, Metrics, KPI
REFERENCE: Suermann P, Issa R (2009) Evaluating industry perceptions of building information modelling
(BIM) impact on construction, Journal of Information Technology in Construction (ITcon), Vol. 14, pg. 574-594,
http://www.itcon.org/2009/37
COPYRIGHT: © 2009 The authors. This is an open access article distributed under the terms of the Creative
Commons Attribution 3.0 unported (http://creativecommons.org/licenses/by/3.0/), which
permits unrestricted use, distribution, and reproduction in any medium, provided the original
work is properly cited.
1. INTRODUCTION
In 2004, the National Institute of Standards and Technology (NIST) published a report stating that poor
interoperability and data management costs the construction industry, approximately $15.8 billion a year, or
approximately 3-4% of the total industry (Gallaher, et al. 2004). Since this report, the focus on Building Information
Modeling (BIM) has intensified and many professionals have labeled BIM as the answer to this problem. From the
National BIM Standard (NBIMS) published in December 2007, a BIM (i.e. a single Building Information Model) is
defined as “a digital representation of physical and functional characteristics of a facility.” Furthermore, a BIM
represents a shared knowledge resource, or process for sharing information about a facility, forming a reliable basis
for decisions during a facility’s life-cycle from inception onward. In the words of the NBIMS Executive Committee
Leader and former Chief Architect of the Department of Defense, Dana K. “Deke” Smith, R.A., “A basic premise of
ITcon Vol. 14 (2009), Suermann and Issa, pg. 575
BIM is collaboration by different stakeholders at different phases of the life cycle of a facility to insert, extract,
update or modify information in the BIM to support and reflect the roles of that stakeholder” (NBIMS 2007).!
2. METHODOLOGY
2.1 Overview
This paper focuses on only the first phase of a four-part research plan, which proposes to accomplish data collection
and analysis on Building Information Modeling impact on construction. Since this research will later focus on BIM
impact on federal construction projects, these four phases will be aligned with a process familiar in the Department
of Defense. The process was originally created by United States Air Force Colonel John Boyd (1927-1997).
Information Management (IM) professionals have often used Boyd’s model, which is widely known as the “OODA
Loop” (Observe, Orient, Decide, and Act), to demonstrate the continual improvement process of strategic decision
making. Boyd developed the process based on his earlier experience as a fighter pilot and he initially used it to
explain victory in air-to-air combat. But in the later years of his career; he expanded his OODA Loop process into a
grand strategy with benefits to anyone who needs to pragmatically and quickly manage information. The OODA
Loop will be used here to structure the research to ensure that each phase builds on the one before it and in this way
the conclusions will be logically valid.
Colonel Boyd’s philosophy dictated that individually, people will observe unfolding circumstances and gather
outside information in order to orient their decision making system to “perceived threats.” Boyd states that the
orientation phase of the loop is the most important step, because if decision makers perceive the wrong threats, or
misunderstand what is happening in the environment, then the decision makers will orient their thinking in erroneous
directions and eventually make incorrect decisions. Boyd said that this cycle of decision-making could operate at
different speeds for different organizations but the goal is to complete the OODA Loop process at the fastest tempo
possible for fighter pilots. However, just as the OODA Loop has been applied to many other endeavors, in this
research, it will be customized or applied to structure the research to arrive at the best, not necessarily the fastest,
choices and conclusions about BIM’s impacts on construction. Through Boyd’s OODA Loop; this research will be
structured in four phases aligned with the ideas of observation, orientation, decision, and action (Boyd 2007). The
work completed in this research in Phase I demonstrated nontrivial data regarding industry practitioners and
academics’ perceptions about BIM impact on construction.
2.2 Phase I: Observation
BIM is not yet widespread in the US Architecture, Engineering, Construction, and Operations (AECO) industry.
Specifically, the 2006 iteration of the annual AIA Firm Survey indicated that only 16% of the firms surveyed had
acquired BIM software and that only 10% of the firms were using the software for billable work. As such, there is
little empirical data regarding its application and use. Therefore, in addition to the typical review of literature in the
field, the qualitative survey administered in this research to garner initial data about practitioners’ perceptions was
needed. Specifically, the survey instrument collected data regarding perceptions about the effects of BIM on
commonly accepted construction key performance indicators (KPIs). This survey data was used to determine current
BIM practices and perceptions to formulate additional research hypotheses for use in Phase II. Phase I included
publishing three iterations of a web-based and hard copy survey with the sole purpose of garnering industry
stakeholders’ impressions of BIM’s effect on construction through specific construction metrics based on six (6)
primary, quantitative construction KPIs: Quality Control, On time Completion, Cost, Safety, $/Unit, Units/Manhour
as determined in a 2003 study by Cox et al. (2003). In this way, qualitative industry perceptions were quantified.
The survey was hosted on http://www.zoomerang.com through an account login funded by the National Institute of
Building Sciences, Facility Information Council (NIBS-FIC). In concert with the National BIM Standard (NBIMS)
Committee testing team, a subset of the NIBS-FIC, this data was shared for their own empirical research. Later, the
survey was issued in hard copy format to the attendants of the BIM4Builders™ Event in Gainesville, Florida in May
of 2008.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 576
2.2.1 Survey
After receiving University of Florida Institutional Review Board (UFIRB) authority, the first iteration of the survey
was available from March 5, 2007 until April 5, 2007 and was advertised to the NIBS-FIC NBIMS Committee. This
sample group was chosen because they are knowledgeable about BIM and have a high likelihood for providing
actionable data. While the respondents affiliation with the NBIMS Committee obviously has the potential of biasing
the results towards being more favourable to BIM impact on construction, it was deemed necessary to use this
respondent pool to 1) get perceptions from those knowledgeable about BIM and 2) serve as a baseline in comparison
to future iterations of more generalizable survey data.
In order to garner maximum participation from existing and new members, Survey #1 was advertised in two different
ways: direct email through a distribution list and a website advertisement on the NIBS-FIC/BIM website where
people join the committee. First, an email was sent to the FIC listserv distribution list. This listserv had 104
members from across the AECO industry at the time of the survey’s launch. Halfway through the month-long survey
availability, a reminder email was sent to the listserv asking for more people to complete the survey or for those who
had started the survey to complete the survey. The second method of garnering qualified respondents was to
advertise the survey on the NIBS FIC website, http://www.facilityinformationcouncil.org/bim, under their “NEWS”
portion. Since most people only happen upon this website when signing up to join the NIBS-FIC NBIMS
committee, and this website is only “advertised” in the AECO community, the possibility of tainting the data was
considered negligible.
Survey #2 was available to the entire industry and advertised in a variety of media outlets including those here:
• The Associated Schools of Construction (ASC)
• The American Institute of Architects (AIA)
• The Associated General Contractors of America (AGC)
• The American Society of Civil Engineers Construction Institute (ASCE-CI)
• The United States Army Corps of Engineers (USACE)
• The Society of American Military Engineers
• The Architects, Engineers, and Contractors (AEC Café) website and newsletter
• The Geographical Information Systems (GIS Café) website and newsletter
• The “upFront – eZine” (sic)
• The Science and Technology for Architecture, Engineering, and Construction Annual BIM Conference
(AEC-ST, May 15-17, 2007) in Anaheim, CA
In this way, the survey was open to a large cross section of the industry and ensured more generalizable results than
the first iteration of the survey, which was only open to those on the NBIMS Committee.
Survey #3 was given in hard copy format to attendees of the BIM4Builders™ Event in Gainesville, FL in May of
2008. This also ensured survey data concerning perceptions from a myriad of fields including primarily contractors,
architects, engineers, as well as some academics and corporate leadership.
2.2.2 Survey Specifics
The survey was divided into four sections:
• Part I: Basic Demographic Information
• Part II: BIM Effects on KPIs
• Part III: Ranking KPIs
• Part IV: Free Answer
Part I was intended to find descriptive information about the respondents, to ensure that they were qualified to
answer the questions, and to group answers from similar respondents together across the data pool. Most questions
ITcon Vol. 14 (2009), Suermann and Issa, pg. 577
were standard for surveys such as gender, age, and the state where the respondent resided. Questions especially
germane to the research were the following which were targeted at collecting the respondent’s educational level,
annual company revenue, and people’s organizational role. Regarding organizational role, respondents were asked
to make a selection from a list based on the organizational roles listed in Table 34 of the Construction Specifications
Institute (CSI 2007). First, respondents were asked to select their overarching organizational role, and then the
survey skipped to the question that addressed the proper organizational role with a follow-up question formulated to
find out the specific role the respondent filled on a daily basis. These choices also came from the CSI’s (2006)
Omniclass Table 34 for organizational roles.
Part II of the survey served as the beginning of the primary data collection instrument. This part asked questions on
each of the six construction KPIs in various formats with varying scales of favorable to unfavorable perceptions
regarding the impact of BIM on construction. In this way, the possibility of errant responses from people just putting
the maximum answer down for every question was avoided. At the beginning of Part II, respondents were asked to
rate their perception of BIM’s impact on the list of six construction key performance indicators. Specifically,
question #14 of the survey addressed BIM’s impact on units per man hour. Units per man hour were defined for
respondents as “measure of completed units (typically square footage) put in place per individual man hour of work.”
The respondents’ choices of answers ranged on a 5-point Likert scale from least favorable to most favorable with the
following possible choices:
Severely Inhibits Lessens No Effect Improves Maximizes
1 2 3 4 5
The next question, #15, asked for the same perception about BIM’s impact on “dollars per unit” or cost per square
foot ($/SF) with the same choices on the 5-point Likert scale. Question #16, asked about safety. Regarding safety,
respondents were asked to “read the following statements and choose the one that most closely matches your view of
BIM’s effect on safety.” The answers, with regard to lost man-hours, were again arranged on a 5-point Likert scale:
Eliminates Lessens No Effect Increases Greatly Increases
1 2 3 4 5
The next question, #17, had to do with cost. Cost was defined as “cost variance in actual costs to budgeted costs.”
Here there were five sub-questions under this one question that centered on different types of costs including:
General Conditions, Structural, Mechanical, Electrical, and Plumbing (MEP), Finishes, and Overall. Here,
respondents could choose from a 5-point Likert scale, as well as the additional choice of Not Applicable or “N/A.”
The 5-point Likert scale had the following choices:
Max Variance: ($ Lost) Worsens No Effect Improves Max Variance ($ Saved)
1 2 3 4 5
Question #18 focused on “on time completion.” The response options were similar to those for question #17 with
the exception of variance equating to a “late” project on the unfavorable side of the scale to “max variance – early”
on the favorable side of the scale.
The final question in Part II, #19, asked respondents what they thought about BIM’s impact on quality
control/rework. This question prepared the respondent for answering by saying, “quality control can be defined as
percent (%) of rework in ($) compared to overall cost in ($).” The choices were:
Increases Rework Worsens No Effect Improves Nearly Eliminates Rework
1 2 3 4 5
Part III of the survey was structured to determine whether there was any one construction KPI which BIM impacted
more than any other in a logical ranking fashion, so that it could be investigated more thoroughly in Phase II of the
research while collecting case study data. Respondents were asked to rank the KPIs on a Likert scale from 1-10.
This means that 1 would be a score showing that BIM inhibited construction to 5 equalling no effect to 10 showing
the most improvement.
Part IV of the survey was intended to gather open ended responses from respondents that could help identify
problems with the current survey, necessary points to investigate in future surveys, receive contact information if
ITcon Vol. 14 (2009), Suermann and Issa, pg. 578
people wanted specific follow-up information, and give respondents a chance to express themselves if they felt the
survey stifled their responses in any way.
The Summary portion of the survey was intended to determine respondents’ personal definition of Building
Information Modeling. There were four choices, including one response; “Don’t Know” which was a response
intended to eliminate unqualified respondents from tainting the data pool. The other choices included:
• BIM is 3D CAD
• BIM is a tool for visualizing and coordinating A/E/C work and avoiding errors and omissions
• BIM is an open standards based info repository for facilities lifecycles
3. RESULTS
The first survey was sent out to the NBIMS committee of the NIBS-FIC and was available from March 5, 2007 until
April 5, 2007. Of the 105 people on the committee when the survey was closed out (as opposed to the 104 on the
committee when the survey was launched), 50 respondents fully completed the survey for a 48% response rate. The
information below represents a summary of the results from the first survey, Survey #1.
3.1 Survey#1: Part I: Basic Demographic Information
Figures 1 through 3 show the data gathered through the Zoomerang online survey or data analysis derived from the
data in the survey. Regarding gender, 86% (43/50) of the respondents were male and 14% (7/50) female. The age
data of the respondents shows that the mode response was also the median age group, the 45-54 year olds with an
overall normal distribution of respondents. There was only one respondent under 25 years old. As far as education
level, 86% (43/50) of the respondents had college degrees, with 56% (28/50) of them holding graduate or
professional degrees. There was no definite trend indicated on the organizational revenue question, although the
most frequent response was $1-$9.9 Million with 24% (12/50) of the respondents choosing this answer.
Respondents’ geographic locations were varied with 47/50 respondents living in the U.S. and three from outside the
U.S. (Note: despite being the U.S. NBIMS committee, several members live and work outside the U.S., but are
either American citizens or are liaisons for wider interests such as the North American BIM buildingSmart Initiative
(sic), etc. so it is possible for respondents on the U.S. NBIMS listserv to live outside the U.S.) The most frequent
response by state was from Maryland, with 18% or nine of the 50 respondents living there.
The organizational role data results showed that the two most frequent responses were from those with a Design Role
with 44% (22/50) respondents and from those with a Management role, which accounted for 30% (15/50)
respondents. Of the first most frequent response, Design Role, 73% (16/22) of the respondents were architects and
27% (6/22) respondents were engineers. For the second most frequent response, Management, 47% (7/15) were
Vice Presidents in their organization and 40% (6/15) respondents were the Chief Executives of their organization.
3.2 Survey #1: Part II: BIM Effects on Construction KPIs
Respondents were asked to rate their perception of BIM’s impact on six KPIs. In order to clearly compare each of
the KPIs to one another, the frequency of positive responses [responses similar to “Greatly Improves” or
“Improves”] were combined into the form of a percentage to simplify comparison between all six KPIs. This was
done rather than taking the median or average because the responses were discrete variables that depended on
frequency rather than comparing the KPIs across a continuous spectrum. The following list is organized in order of
the highest rated to the lowest rated of the six KPIs: Quality Control/Rework (90%), On-time Completion (90%),
Cost-Overall (84%), Units/Man hour (76%), Dollars/Unit (70%), and Safety (46%). This was calculated by
evaluating responses that exceeded the neutral Likert value of 3 and comparing that to the total number of responses.
For example, 34/50 respondents opined that BIM “Improved” the Quality Control/Rework KPI, as well as 11/50
respondents opined that BIM, “Nearly Eliminates Rework” for a total rating of 90% (45/50). Full data on the
responses can be seen in Figures 2 and 3.
Cost was similarly broken down and the following list organized in the order of highest to lowest rated favorable
opinion (i.e. assigned a value greater than 3 on the Likert scale) by the respondents: Overall (84%), Mechanical,
ITcon Vol. 14 (2009), Suermann and Issa, pg. 579
Electrical, and Plumbing (78%), Structural (76%), General Conditions (70%), and Finishes (58%). It is important to
note that 46% or 23/50 respondents also felt that BIM has “No Effect” on safety or lost man-hours in construction
projects, making it the KPI that in their perception is the least impacted by BIM.
FIG. 1: Survey #1: Part II Responses about BIM’s impact on construction KPIs (raw data from zoomerang)
FIG.2. Survey #1: Part II Responses about BIM’s impact on construction KPIs (raw data from Zoomerang)
ITcon Vol. 14 (2009), Suermann and Issa, pg. 580
3.3 Survey #1: Part III: Ranking Construction KPIs
Respondents were asked to rank the construction KPIs according to their perceptions of how well BIM improved the
given KPIs on a scale of 1-10, with 10 showing the most improvement, 5 showing no effect, and 1 showing that BIM
inhibits the given KPIs. Organizing the construction KPIs according to merely adding positive response frequency
percentages (anything over a score of 5), the KPIs score the following in order from most to least favorable: Quality
(94%), On-time Completion (88%), Units/Man-hour (86%), Dollars/Unit (80%), Cost (80%), and Safety (54%.)
When weighting the answers for the degree of favorability according to the weighted average of the ranking scores
provided by respondents, the KPIs scored in a slightly different order: Quality, On-time Completion, Units/Man-
hour, Cost, Dollars/Unit, and Safety. This information is graphically illustrated in Figures 4 and 5. Figure 4 shows
the percentages of favorable response frequency. Figure 5 shows weighted average scores according to their
emphasized degree of favorability.
FIG. 3: Overall Favorable Responses when ranking KPIs with respect to impact on BIM (unweighted)
ITcon Vol. 14 (2009), Suermann and Issa, pg. 581
FIG. 4: Overall Favorable Responses when ranking KPIs with respect to impact on BIM (weighted)
3.4 Survey #1: Part IV: Comments
A few of the most representative comments made by the respondents were:
• Respondent # 3: A BIM will likely affect KPIs rather than the other way around. A good,
comprehensive, structured source of accurate data that all the stakeholders can access will reduce stove
pipes, redundant data and inaccurate information. It will make it easier to keep the data current and to
verify it.
• Respondent #7: The questions that are being asked are of the type that an A/E would ask. You may
want to look at asking that questions that a builder, vendor, or trade contractor would ask.
• Respondent #8: The way you ask your questions, it seems as if you assume that BIM should save time
and money. In reality, I believe that the BIM makes your planning, scheduling, estimating, etc. more
accurate. I have quite often seen that BIM corrects errors, misconceptions and the net effect may be
additive (but save the contractor the time, money and the embarrassment of a mistake). If there was
inadequate time or more planned for a given scope, than it may it may be just as likely to add time or
money as save (sic).
• Respondent #13: More KPIs: Reduction in Claims, Improved public outreach/agency coordination,
More sustainable structures
• Respondent #16: BIM will minimize change orders, and will also reduce the initial project cost.
Contractors will sharpen their pencils and will provide pricing per known factors, the number of
unknowns and field coordination efforts are reduced.
• Respondent #17: While BIM a goal to strive for and is relevant to certain projects - the fractured
nature of the A/E/C (sic) industry means that it will be a long time before BIM has a significant overall
effect on the industry
ITcon Vol. 14 (2009), Suermann and Issa, pg. 582
3.5 Survey#1 Summary Definition Question and Conclusions
The summary question in this survey asked respondents which definition of BIM most closely matched their own.
No respondents chose the answers “Don’t Know” or “BIM is 3D CAD.” Therefore, none of the responses were
eliminated from the data pool. As shown in Figure 6, the definition of BIM drafted by the NIBS-FIC NBIMS
Committee received the most responses, “BIM is an open standards based information repository for facilities’
lifecycles,” with 70% or 35/50 respondents making this selection. The other response was, “BIM is a tool for
visualizing and coordinating AEC work and avoiding errors and omissions,” received 30% or 15/50 responses.
While this response is not necessarily incorrect, it does not align with the NBIMS’ view of the definition, which
means that 15% of the respondents from the NBIMS committee have a personal definition of BIM that is different
than the committee’s formal definition. Thus, there is still some work to be done for the NBIMS Committee to
educate and inform the AECO community, even within its own organization. However, because of people’s
membership on the committee, their proven expertise, and the fact that only generally acceptable definitions of BIM
were selected, all the data was assumed valid and no respondents’ individual surveys were “thrown out.”
Also of interest on this survey was that several people actually thought that BIM “hindered” safety. It is unclear
whether these respondents did not understand the question, errantly entered their answers, or genuinely thought that
BIM hindered safety. This needs to be investigated further.
FIG. 5: Answers to definition of BIM Question in Survey #1
3.6 Survey #2
Survey #2 was based on survey #1, but had some minor edits to the way questions were sequenced or asked after
implementing advice from respondents who took Survey #1. The survey was available from April 30 to October 30,
2007, or exactly six months. It is important to note that the survey was open to the general population at large and
anyone could complete a copy of the survey and experienced 95 completed surveys, out of an unknown sample size
pool, due to industry-wide availability.
3.7 Survey #2: Part I: Basic Demographic Information
Figures 6 – 8 show the data gathered through the Zoomerang online survey, or data analysis derived from the data in
the survey. Regarding gender, 88% of the respondents were male and 12% were female. The age data of the
respondents shows that the mode response was the 35-44 year olds with an overall relative normal distribution of
respondents. Different from the NBIMS Survey, this survey had younger respondents, which is understandable,
considering it was open to all public practitioners. As far as educational level is concerned, 87% of the respondents
had bachelor’s degrees or higher, nearly the same as Survey #1.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 583
There was no definite trend indicated on the organizational revenue question, although the most frequent response
(with a monetary value) was $1-$9.9 Million with 16% (14/90) of the respondents choosing this answer. The most
frequent answer overall was “Don’t know.”
Respondents’ geographic locations were varied with 87/93 respondents living in the U.S. and six from outside the
U.S. The most frequent response by state was from Washington, with 11% or ten of the 93 respondents living there,
most likely due to advertising the survey while conducting embedded research in Seattle.
FIG. 6. Survey #2 screen capture of the results to survey questions 1-3
FIG. 7. Survey #2 Screen capture of the results to question 4
ITcon Vol. 14 (2009), Suermann and Issa, pg. 584
The organizational role data results showed that there were three primary responses from the eight choices. The
most frequent response was from those with Academic Roles with 31% (29/95) of the respondents. Next most
frequent were those with Design Roles with 24% (23/95) and Management with 19% (18/95) of the respondents. Of
the top most frequent response, Academics, 79% of those respondents were Assistant Professors or higher. Of those
who responded “Design Role,” 64% (14/22) of the respondents were architects and 36% (8/22) of the respondents
were engineers. For the third highest frequent response, Management, responses were evenly divided between Chief
executive, Vice President, and Partner.
FIG. 8. Survey #2 Screen capture of the results to question 6 – Top level description of organizational role
3.8 Survey #2: Part II: BIM Effects on KPIs
Figures 9 and 10 show respondent perceptions regarding BIM impact on the six KPIs. In order to clearly compare
each of the KPIs to one another, the frequency of positive responses [responses similar to “Greatly Improves” or
“Improves”] were combined into the form of a percentage to simplify comparison between all six KPIs. This was
done rather than taking the median or average because the responses were discrete variables that depended on
frequency rather than comparing the KPIs across a continuous spectrum. The following list is organized in order of
the highest rated to the lowest rated of the six KPIs: Quality Control/Rework (85%), Cost-Overall (83%), On-time
Completion (76%), Units/Man hour (67%), Dollars/Unit (67%), and Safety (37%). It is important to note that
because “Units/Man hour” and “Dollars/Unit” were had the same frequency of favorable answers; the negative
values were assessed, which made Units/Man hour more highly favored than “Dollars/Unit.”
ITcon Vol. 14 (2009), Suermann and Issa, pg. 585
FIG. 9. Survey #2 screen capture of the various results to first three KPIs: Units per man-hour, Dollars/Unit, and
Safety
FIG. 10. Survey #2 screen capture of results to last three KPIs: Cost, On-Time Completion, and Quality
Control/Rework
ITcon Vol. 14 (2009), Suermann and Issa, pg. 586
This was calculated by evaluating responses that exceeded the neutral Likert value of 3 and comparing that to the
total number of responses. For example, 50/95 respondents opined that BIM “Improves” Units per man-hour, as
well as 13/95 respondents opined that BIM, “Maximizes” Units per man-hour, for a total rating of 67% (63/95).
Cost was similarly broken down and the following list organized in the order of highest to lowest rated favorable
opinion (i.e. assigned a value greater than 3 on the Likert scale) by the respondents: Overall (83%), Mechanical,
Electrical, and Plumbing (83%), Structural (76%), General Conditions (54%), and Finishes (52%.)
It is important to note that 53% or 50/95 respondents also felt that BIM has “No Effect” on safety or lost man-hours
in construction projects, making it the KPI that in their perception is the least impacted by BIM, similar to the results
in Survey #1.
3.9 Survey #2: Part III: Ranking KPIs
Figure 11 shows respondents KPI rankings according to their perceptions of how well BIM improved the given KPIs
on a scale of 1-10, with 10 showing the most improvement, 5 showing no effect, and 1 showing that BIM inhibits the
given KPIs. Organizing the construction KPIs according to merely adding positive response frequency percentages
(anything over a score of 5), the KPIs score the following in order from most to least favorable: Quality (83%), Cost
(83%), On-time Completion (79%), Dollars/Unit (74%), Units/Man-hour (69%), and Safety (46%).
FIG. 11. Survey #2 screen capture of Ranking KPI responses
In order to take into account degree of favorability, rather than simply positive frequency, responses were multiplied
by their relative weight (6-10) and calculated. After accomplishing this operation, this resulted in: Quality (4.98),
Cost (4.98), On-time Completion (4.74), $/Unit (4.44), Units/Man-hour (4.14) and Safety (2.88) for the same order
as frequency of positive responses.
3.10 Survey #2: Part IV: Free Answer
A few of the most representative comments made by the respondents are listed here, because there were too many
comments to show in one figure.
• Not sure the survey is applicable to the entire scope of "BIM"... seems to be construction centric, In
that context it is good as far as it goes
• Your definitions of BIM are very shallow and limited to technology. BIM is a process that is
implemented within a building projects using technologies that facilitate the collaboration, open
standards and communications that allow Building Information to be contributed by the right experts at
the right time thus creating a database that can be viewed in reports, graphics, 2D or 3D and other
means to communicate the means by which it can be constructed. The BIM data must be useful during
the entire life cycle of the building. Look at definitions of BIM by CURT, The AIA paper on the
Integrated Practice and FIATECH. Tool for Contractors are just part of BIM. Tools for visualizing and
ITcon Vol. 14 (2009), Suermann and Issa, pg. 587
coordinating AEC is just part of BIM, BIM is NOT 3D CAD as some vendors would have us believe.
BIM may be fererally [sic] supported for specific applications and they are going to hold the industry
accountable for using the BIM process and implementing useful tools to meet the goals of the owners.
The Owners organization (CURT) rules the roost. They have the money and want the buildings built
so we need to listen to them.
• Like most trends moving through the construction industry, contractors perceive the need to adopt
BIM as a distinguishing capability that separates their company from the rest of the pack. There is also
an energized atmosphere that motivates us to explore this new technology. This is partly driven by our
own sense of adventure but also driven by software and hardware developers who promise to solve all
of our problems with the new tools. I am very interested in learning the results of your survey,
although I think it's a rapidly moving target that would yield different results in a year from now.
• An interestsing [sic] format. You have selected what I perceive are key variables and it will be
interesting to see you final results.
• Experienced sever learning curve on initial project. Bentley software was found to be not up to the task
in many respects. No gain on that project, in fact, probably a more expensive approach with multiple
problems flowing from the approach itself, but we do expect these metrics to improve over time.
Enjoyed the ability to perceive conflicts between disciplines in the design before discovered during
fieldwork. Prime and subs were not prepared to make efficient use of the offered BIM information.
Cost estimates easier to update after design changes. Expect this is the wave of the future and holds
much promise we did not achieve in our initial attempt.
• BIM is a great tool for new construction because it builds from the ground up. As a tool for rehab
work, unless the project is on a fairly large scale, more effort goes into producing the BIM than can be
done by doing a design in 2-D and providing contractors with existing reference drawings. The
production of BIMs for an entire installation is a costly proposition when done at one time, and even
greater when done for several installations at the same time. No one can really afford to BIM all they
own to the BIM level of a new facility. BIM as needed should be the process until the evolution of
BIM is fully developed to where a building has been mostly BIM'd [sic] because of work to it. Using
BIM to produce 2-D plan sets has no advantage over using any CAD application to do the same.
Unless construction contractors have a means to use BIM themselves, BIM will be slow growing. As
for their use in asset management, until facilities managers understand their usefulness and are able to
ue [sic] them with other tools, providing BIM files to them at the completion of construction is a
waste. Our use of BIMs have not shown any change in construction cost or safety, but did increase the
effort and cost to do BIM because of a learning curve. Additionally, the majority of our BIMs were
produced by contract, which required review of all existing drawings and on-site verification visits to
produce as-built facilities. This was very expensive work, and they are used only to produce 2-D plan
sets and primarily as a space management tool.
• Everyone’s concept of BIM is based on their perspective. All BIM are not created equal and will
continue to be inconsistent until there is an effective national standard that addresses all phases of a
facility, including concept, design, construction, and O&M.
3.11 Survey #2: Summary
Figure 12 shows the results of the summary question in this survey, which asked respondents which definition of
BIM most closely matched their own. No respondents chose the answers “Don’t Know” or “BIM is a general
contractor's virtual approach to planning site logistics.” Therefore, none of the responses were eliminated from the
data pool. However it is significant that 55% of the respondents answered that “BIM is a tool for visualizing and
coordinating AEC work and avoiding errors and omissions,” when the NBIMS definition, “BIM is an open-standards
based information repository for facilities’ lifecycles,” garnered only 20% of the respondents’ answers. In fact, even
more people (21%) chose to specify their own definition of BIM, showing that BIM is still “defining itself” within
the context of the AECO industry. Free response definitions mostly answered that BIM represented “all of the
above” answers or focused on the process, rather than the product. See Figure 13 and 14 for a complete list of these
responses:
ITcon Vol. 14 (2009), Suermann and Issa, pg. 588
FIG.12: Survey #2: Answers to definition of BIM Question
FIG. 13. Survey #2 BIM Definition Free Response Answers
ITcon Vol. 14 (2009), Suermann and Issa, pg. 589
FIG. 14. Survey #2 screen capture of Summary question
In all, the primary differences between the Survey #1 and Survey #2 can be summarized in the following list:
• Slightly younger respondent pool
• Many more academics in the respondent pool
• Slightly LESS favorable overall towards BIM in Survey #2 (Specifically in Survey #2, there were 2-4
respondents who replied that BIM “extremely hindered” all KPIs in the construction phase. More
investigation is required to determine if this outlier reflected valid opinions or was caused by
confusion surrounding the survey instrument.)
• Opined that Cost is benefitted more by BIM in Survey #2
• Greater disagreement on the definition of BIM in Survey #2
3.12 Survey #3
Survey iteration #3 was issued on May 11, 2008 as conference attendees checked into the BIM4Builders™ event in
Gainesville, Florida as discussed in Chapter 2, “Materials and Methods.” Although the survey was very similar to
the first two iterations, it was offered in hard copy format and consequently edited to one page for time and logistics
constraints of the conference. The following information discusses the results of Survey #3 and concludes with a
summary and comparison of the different trends noted from Surveys #1, 2, and 3.
3.13 Survey #3: Part I: “Basic Demographic Information”
Part I asked similar questions of respondents regarding gender, age, education, annual revenue, and organizational
role. This information was later used to cross tabulate people’s demographics with their responses. However, in
order to garner the most information to form reliable trend data, the results of this final survey were analyzed as a
subset of the compilation of all three surveys. Therefore, the following results will take into account data from all
three surveys and will look for emerging trends from all of the data in its entirety. After including completed surveys
from all three iterations, there was a very favorable “N” value of 202 completed surveys.
The results of the demographics of all 202 completed surveys showed that the most likely respondent was male, over
55 years old, held a graduate degree, and worked for a company with annual revenue under $100Million (Figure 15).
The organizational roles of the respondents was evenly distributed across management, design, academic, and other
fields.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 590
FIG 15. Compilation of Demographic and BIM Definition Data from Survey #1, 2, and 3. (Note: Most frequent
responses are highlighted in yellow).
3.14 Survey #3: Part II: “Ranking Key Performance Indicators”
Similarly, all three survey iterations’ data was compiled regarding KPI ranking. There was a clear trend here with
respondents answering in the positive (BIM improves the KPI) to negative (BIM inhibits the KPI) in identical order,
which speaks to the validity of the data. As seen in Figure 16, the order that respondents ranked the KPIs from most
to least favorable were:
• Quality, with 87.7% saying BIM improves this KPI
• Cost, with 83.7% saying BIM improves this KPI
• Schedule, with 82.8% saying BIM improves this KPI
• Productivity, with 74.9% saying BIM improves this KPI
• Safety, with only 53.7% saying BIM improves this KPI
ITcon Vol. 14 (2009), Suermann and Issa, pg. 591
•
FIG 16. Compilation of KPI Ranking Data from Survey #1, 2, and 3. (Note: Negative or inhibiting factors are
indicated in gray and positive or improving values are indicated in yellow with the rank (1-6)below each in
corresponding colors for inhibiting or improving)
3.15 Survey #3: Part III: “BIM Definition”
In Part III, respondents were asked to choose from a list of BIM definitions and pick the definition that was closest to
their own. Of most interest was whether a respondent’s organizational role affected their response and if there was a
trend present where one organizational role chose a single definition by a large margin compared to another.
Looking at Figure 17 it is clear that the answers are fairly well distributed, but that the most common definition
answer for all four categories (management, design, academic, and other) of career fields’ most frequent choice was
related to BIM as a “tool for visualizing and coordinating A/E/C work and avoiding errors and omissions.” This
differs from the NBIMS definition of BIM as an “open standard-based information repository for facilities’
lifecycles,” which was the second most frequently chosen definition overall. However, with the high rate of
selection of “Other” or write-in definitions for BIM, it is clear that the industry has not reached a consensus
definition for the true essence of BIM.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 592
FIG 17. Compilation of BIM Definition Data from Survey #1, 2, and 3. (Note: Focus on whether organizational
role affected definition selection. Most frequent responses are highlighted/yellow).
The authors hypothesized before the research began that there might be significant trends in the responses due to the
demographics of gender, age, education, revenue, and role. Specifically, the interest was whether architects would
differ in their responses about BIM as compared to contractors. As noted by Zuppa et al (2009) in a parallel project,
the most notable trend was that architects were less favourable about BIM’s impact on project cost than contractors;
contradicting the researchers’ notional hypothesis. However, there were no other definitive trends tied to
demographics, making the responses appear homogenously untied to survey demographics, with equal levels of
favourability for the KPIs across the survey groups.
4. FUTURE RESEARCH: PHASES II-IV, ORIENTATION, DECISION, AND ACTION
As discussed earlier in the paper, future research is already planned for this topic. Phase II will include building on
the data garnered in Phase I from the three surveys and will test the primary research hypothesis and possible follow-
on hypotheses by conducting embedded research on federal construction projects. The rationale behind this research
is that federal entities have provided testbeds for implementing new ideas and new technologies in the past in the
field of construction. While federal work has not always led the way on implementing new technological initiatives,
recent strides in the Department of Defense, and United States Coast Guard (USCG) demonstrate that they are
exceeding typical industry rate of BIM adoption. However, despite recent promulgation of BIM procedures in
documents like the GSA BIM Guide and USACE BIM Roadmap, there is little documented evidence on BIM’s
impact on the construction phase of the facility lifecycle. Therefore, future research proposes to evaluate BIM
effects on federal construction projects according to the KPI metrics listed in the survey. The specific locations
where the embedded research will be conducted due to their considerable experience at managing projects through
BIM methodologies are:
• U.S. Army Corps of Engineers (USACE), Seattle District
• U.S. Army Corps of Engineers (USACE), Louisville District
• U.S. Coast Guard Naval Engineering Support Unit (NESU), Charleston, South Carolina
• U.S. Central Command (US CENTCOM), MacDill Air Force Base, Florida
After assessing and analyzing the data and comparing it to longitudinal data from construction projects similar in size
and scope to those studied in Phase II, Phase III will include revisions and changes to the data collection model. In
ITcon Vol. 14 (2009), Suermann and Issa, pg. 593
addition, a wider cross section of construction projects will be studied including commercial and industrial projects.
Phase III entails comparing the data collected in Phases I and II to longitudinal data. This would be accomplished by
using data collected and maintained by research bodies such as the USACE Civil Engineering Research Laboratory
(CERL) or the Construction Industry Institute (CII) and comparing their baseline construction data to BIM case
study data from Phase II. Lastly, the data will be analyzed to determine if trends exist that demonstrate significant
differences in productivity or performance according to the construction KPI metrics.
In Phase IV, after the bulk of the data is collected, the lessons learned from conducting the embedded research will
be applied to a further revised methodology recommended for future case study data collection. Additionally, noted
trends will be discussed in the research analysis portion of this document and recommended for consumption and
implementation by federal entities and construction firms for recommended best business practices that yield the
most productivity improvements. In this way, the research will act on the lessons learned, fulfilling the OODA
Loop. Possible additional case study data will include trend analysis on commercial and industrial projects and
comparison to the federal construction projects case study data
5. CONCLUSION
As the results suggest, the respondents felt that BIM is most likely to positively impact the construction KPIs of
quality and on time completion. More research needs to be conducted in order to corroborate the “BIM-favorable”
results here. While the respondents are certainly knowledgeable about BIM because of the demographics shown
herein and membership on the NBIMS listerv, Survey #1 results need to be taken as a part of the greater whole of all
the surveys’ and realize that all respondents filling out these surveys have the propensity for being more favorable to
BIM than the typical industry professional, due to their interest in the field and willingness to take time to fill out the
survey. Additionally, quantifying the impact of a BIM approach through real world construction case studies will
offer a more compelling argument for BIM adoption by AEC firms than simply the perceptions described here.
6. ACKNOWLEDGEMENTS
This study was partially supported by the National Institute of Building Sciences, Facility Information Council
(NIBS-FIC) and the BIM4Builders™ Event sponsored by the M.E. Rinker, Sr. School of Building Construction and
the LaiserinLetter.
7. REFERENCES
Adrian, J.J. (1995). Construction Productivity: Measurement and Improvement, Stipes Publishing, Champaign,
Illinois.
AIA, (2006). “AIA Firm Survey: The Business of Architecture.” American Institute of Architects, Information
Technology Chapter, 67-75.
Bazjanac, V. (2004). “Virtual Building Environments (VBE) - Applying Information Modeling to Buildings.”
Lawrence Berkeley National Laboratory, University of California, Berkeley, CA, U.S.A.
BOMA International (2007). “BOMA Mission Statement.”
<http://www.boma.org/AboutBOMA/BOMAMissionStatement.htm> (June 25, 2007).
Boyd, J. (2007). “Information Warfare. Explanation of OODA Loop of John Boyd.”
<http://www.12manage.com/methods_boyd_ooda_loop.html> (July 20, 2007)
Cox, R.F., Issa, R.R.A., and Ahrens, D. (2003). “Management's Perception of Key Performance Indicators for
Construction.” J. Constr. Engrg. And Mgmt., 129(2), 142-151.
Cullis, B. (2005) “Geospatial Mandates Pave Way for DISDI.” Military Geospatial Technology Online Edition, 3
(2).
CSI. (2006). “Omniclass, Table 34, Organizational Roles.” <http://www.omniclass.org/about.asp>
Gallaher, M., O’Connor, A., Dettbarn, J., and Gilday, L. (2004). “Cost Analysis of Inadequate Interoperability in the
U.S. Capital Facilities Industry” NIST GCR 04-867.
ITcon Vol. 14 (2009), Suermann and Issa, pg. 594
Hardy, M. (2006). “GSA Mandates Building Information Modeling” Federal Computer Week,
<http://w3.gsa.gov/ClipsMgt.nsf/PDAWebToday/09CFE365DC995AA185257233003D8B33?OpenDocumen
t> (June 15, 2007).
Kam, C. (2006). “02 – GSA BIM Guide for Spatial Program Validation.” GSA Building Information Modeling
Guide Series.
<http://www.gsa.gov/gsa/cm_attachments/GSA_DOCUMENT/GSA_BIM_02_Appendix_v09_R2C-a3-
l_0Z5RDZ-i34K-pR.pdf> (June 15, 2007).
Kam, C. (2007). “3D-4D Building Information Modeling” GSA webpage, <http://www.gsa.gov/bim> (June 15,
2007).
Kennett, E. (2006) New NIBS Group to Create U.S. BIM Standard. Building Sciences A Publication of the National
Institute of Building Sciences. Vol. 30, March.
Kosiak, S. M. (2004) Analysis of the FY 2005 Defense Budget Request, Center for Strategic and Budgetary
Assessments.
Kunz, J. and Fischer, M. (2007). “Virtual Design and Construction: Themes, Case Studies and Implementation
Suggestions.” Stanford Center for Integrated Facility Engineering
<http://cife.stanford.edu/online.publications/WP103.pdf> (June 14, 2007).
Lee, A. (2005) “nD Modeling – A Driver or Enabler for Construction Improvement?” RICS Research Paper Series,
5 (6).
NBIMS website. (2007) < http://www.facilityinformationcouncil.org/bim/pdfs/NBIMS_Awareness_Handout.pdf>
(July 20, 2007).
Office of the Deputy Undersecretary of Defense (Installations & Environment) Business Enterprise Integration
Directorate (2006) “Real Property Acceptance Requirements Document” Draft V5.0.
USACE (U.S. Army Corps of Engineers), ER 1110-1-12. “Engineering and Design – Quality Management.”
USACE. (2006), CADD/GIS Technology Center, Waterways Experimentation Station, (WES) Vicksburg, MS.
Center Headlines. < https://cadbim.usace.army.mil/BIM> (July 21, 2007).
US Department of Commerce, National Institute of Standards and Technology, (2004) “Cost Analysis of Inadequate
Interoperability in the U.S. Capital Facilities Industry”. (NIST GCR 04-867, August 2004).
<http://www.bfrl.nist.gov/oae/publications/gcrs/04867.pdf> (July 21, 2007).
Zuppa, D., Issa, R.R.A., Suermann, P. (2009). “Impact of BIM on the Success Measures of Construction Projects."
Proceedings ASCE International Workshop on Computing in Civil Engineering, Caldas, C.H. and O’Brien,
W.J. (Eds.), June 24-27, Austin, TX, pp. 503-512.
