VDC in Practice: A Preliminary Categorization of Production Metrics Reported in Scandinavia and Latin America

Full text

Majumdar, T., Rasmussen, S.G., Del Savio, A.A., Johannesdottír, K., Hjelseth, E., & Fischer, M.A.
(2022). VDC in Practice: A Preliminary Categorization of Production Metrics reported in Scandinavia
and Latin America Proceedings of the 30th Annual Conference of the International Group for Lean
Construction (IGLC30), 1177–1185. doi.org/10.24928/2022/0230
Lean and BIM 1177
VDC IN PRACTICE: A PRELIMINARY
CATEGORIZATION OF PRODUCTION
METRICS REPORTED IN SCANDINAVIA
AND LATIN AMERICA
Tulika Majumdar
1
, Steinar G. Rasmussen
2
, Alexandre Almeida Del Savio
3
, Katrin
Johannesdottír
4
, Eilif Hjelseth
5
, and Martin A. Fischer
6
ABSTRACT
Architecture, Engineering and Construction (AEC) teams are increasingly using
emerging management methods involving collaboration, lean construction, and
digitization for managing projects. Production metrics (PM) are being used to assess the
impact of these methods on project performance during run-time. A lack of common
vocabulary hinders comparison of PM, making it difficult to repeat strategies used for
improving project performance and for benchmarking PM across projects.
Through a detailed content analysis, 2 datasets of 904 PM reported by 195 Virtual
Design and Construction (VDC) practitioners in Scandinavia and Latin America were
curated. Qualitative coding was used to categorize the PM into the three key VDC
elements, i.e., Integrated Concurrent Engineering (ICE), Building Information Modeling
(BIM) and Project Production Management (PPM) and to validate the categorization.
This research enabled a comparison of PM categories across the two regions for the
first time. PM categorized as ICE and PPM were reported by more than 68% professionals
in both the regions. BIM PM had a disparity in reporting (Scandinavia: 30%, Latin
America: 91%). It also opened a pathway to develop a common vocabulary of PM to
compare, benchmark and standardize PM across VDC implementations.
KEYWORDS
Production metrics, continuous improvement, concurrent, standardization, process
BACKGROUND
Since 2019, the Stanford Center for Professional Development (SCPD) has been
conducting a VDC certificate program (VDCCP) in collaboration with the Center for
Integrated Facility Engineering (CIFE), Norwegian University of Science and
1
Ph. D. Candidate, Department of Civil and Environmental Engineering, Stanford University, USA,
tulika@stanford.edu
2
Hæhre Entreprenør, steinar.rasmussen@akh.no
3
Professor, Civil Engineering Department, University of Lima, aalmeida@ulima.edu.pe
4
Trimble Solutions Sandvika AS, katrin_johannesdottir@trimble.com
5
Professor, Department of Civil and Environmental Engineering, Norwegian University of Science and
Technology, Norway, eilif.hjelseth@ntnu.no
6
Professor, Department of Civil and Environmental Engineering, Stanford University, USA,
fischer@stanford.edu

VDC in Practice: A Preliminary Categorization of Production Metrics Reported in Scandinavia and Latin
America
Lean and BIM 1178
Technology (NTNU) and Universidad De Lima (ULima). AEC professionals enrolled in
the VDCCP learn the fundamentals of VDC in a 1-week introductory workshop and
implement VDC on their ongoing projects for 9 months. They define PM for their projects
at the beginning of the implementation and explain through monthly reports if and how
the PM enabled effective course correction.
INTRODUCTION
Previous researchers have established the connection between Lean and VDC (Fosse &
Ballard, 2016; Rischmoller et al., 2018). In recent years, there has been an increase in
the number of AEC professionals implementing VDC on their projects (Majumdar et al.,
2022). PM are an integral part of the VDC methodology. They impact project
performance by providing rapid feedback on actions and decisions taken by project teams
for timely course correction. Unlike several other industries, such as manufacturing, a
common vocabulary of useful PM does not exist for the AEC industry. As a result, AEC
project teams establish and track PM on an ad-hoc basis, which hinders comparison and
benchmarking. This phenomenon was observed by CIFE researchers in the PM data
provided by 279 AEC professionals from Scandinavia and Latin America who enrolled
in the VDCCP in 2020.
This was the first time that PM data from a large number of projects in two different
parts of the world during the same time period was available for research. This opened
up the possibility of building a vocabulary of PM to facilitate learning and benchmarking
and to answer the research question - How can PM data be used for comparison and
learning across projects?
In answering the above question, this study categorized 1963 PM using the 3 key VDC
elements of ICE, BIM, and PPM. While an analysis of individual PM in each of the 3
categories is beyond the scope of this study, it makes the following two contributions:
• A categorization based on key VDC elements as a first step towards building a
common vocabulary of PM.
• Operationalization of VDC theory by comparing PM categories reported in
practice.
Project Managers and AEC organizations can use this categorization to compare the
level of collaboration, use of digital tools and lean work processes across projects. They
can establish and standardize individual PM under each category. Researchers can use
this categorization to build sub-categories of PM to facilitate standardization of PM in
the industry.
POINT OF DEPARTURE
PM, an integral part of the VDC methodology, create a culture of continuous
improvement, which is a key Lean principle (Haugstvedt, 2019).
Existing literature on PM used by VDC practitioners is either based on anecdotal
information (Kunz & Fischer, 2012) or limited case studies (Belsvik et al., 2019; Fosse
& Ballard, 2016; Gao, 2011). There has been no study which reports on PM used in
practice on a variety of projects. This empirical research looked for evidence of PM used
by VDC practitioners on ongoing projects in two different parts of the world in 2020 and
categorized them for accessibility by AEC professionals and researchers.

Tulika Majumdar, Steinar G. Rasmussen, Alexandre Almeida Del Savio, Katrin Johannesdottír,
Eilif Hjelseth, and Martin A. Fischer
Proceedings IGLC30, 25-31 July 2022, Edmonton, Canada 1179
Categorization of metrics has been adopted in other industries, such as manufacturing,
for comparison, benchmarking and standardization (Gunasekaran et al., 2001). In the
AEC industry, categorization has been used for comparison of facilities performance
(Lavy et al., 2014). This is the first study which uses PM data reported from real AEC
projects and uses the following key VDC elements to categorize them:
ICE (Integrated Concurrent Engineering): To achieve an integrated, high
performing facility, project teams should learn how to share knowledge, evaluate multiple
possible solutions through collaboration and consistent feedback. It therefore becomes
important for the project team to co-locate, at least partially (Fischer et al., 2014) so that
project team members from various disciplines can have several quality interactions to
solve problems quickly and effectively.
BIM (Building Information Modeling): This refers to visualization and simulation,
which are the key mechanisms to achieve integrated information.
PPM (Project Production Management): This refers to the organization of physical
work tasks by treating the project as a production system. It includes managing
parameters such as cycle time, work in process, capacity utilization etc. PPM has its
origins in Operations Science, which is also the seat of evolution of Lean (Rischmoller et
al., 2018).
RESEARCH METHOD
Conceptual content analysis was considered suitable for this study. This research method
establishes whether a concept exists in a given text. The occurrence or non‐occurrence of
selected keywords within the target text are used to either discern the central idea of the
text or extrapolate conclusions (Carley, 1990). This was used to extract PM data from
each final report in .ppt or .pdf format provided by AEC professionals as part of the
VDCCP and then to categorize them. Without a standard list of PM available, the survey
method would be inadequate to capture the data provided in all the reports and was,
therefore, ruled out. Case studies or sampling would not cover the breadth of PM data
which was available for the first time, and were also ruled out.
RESEARCH TASKS
The key tasks in the research involved a) data preparation and clean-up to curate two
datasets of PM, b) qualitative coding to categorize PM reported in dataset 1(Scandinavia)
into the three key VDC elements, i.e., ICE, BIM and PPM, and c) qualitative coding of
PM reported in dataset 2(Latin America) to validate the three PM categories.
DATA PREPARATION AND CLEAN-UP
Final monthly reports (in .ppt or .pdf formats) submitted by 175 professionals from the
first large-scale VDCCP in Scandinavia and by 104 professionals from the first large-
scale VDCCP in Latin America were compiled. The data was immediately anonymized
for carrying out the research. The languages used in the reports were Norwegian, English
and Spanish. 1963 PM found in these reports in the form of tables and charts were
manually entered in Google sheets in the original language and were translated into
English using Google Translate. As PM are daily or weekly in nature and are measured
as short-term outcomes of actions and decisions taken by the project team, data clean-up
was done to eliminate metrics which are a) measured once at project completion instead
of daily or weekly, such as “deviation from final cost of project”, b) measured at specific

VDC in Practice: A Preliminary Categorization of Production Metrics Reported in Scandinavia and Latin
America
Lean and BIM 1180
milestones, such as “time taken to complete zoning plan”, c) input metrics (Khanzode,
2011) which are actions and decisions controlled by the project team and do not measure
an outcome, such as “number of ICE sessions conducted, and vague data reported as
metrics, such as “problem solving”.
After clean-up, dataset 1 consisted of 417 PM reported by 115 professionals and dataset
2 consisted of 487 PM reported by 80 professionals. Tables1 and 2 provide details on
the roles of professionals in both the datasets after clean-up and the project types they
reported PM data from.
Table 1: Role of professionals in cleaned-up datasets of PM
Company Type
Dataset 1
Scandinavia
(n=115)
Dataset 2
Latin America
(n=80)
Owner
8%
24%
Consultant/Owner's Rep.
11%
2%
Design/Engineering Consultant
41%
11%
General Contractor
33%
41%
Subcontractor
2%
15%
Software Provider
5%
4%
Not Available
0%
4%
The professionals used VDC on both building and infrastructure projects. Table 2 lists
the breakdown by these 2 project types after data clean-up in the two datasets.
Table 2: Project types in cleaned-up datasets of PM
Project Type
Dataset 1
Scandinavia
(n=115)
Dataset 2
Latin America
(n=80)
Building
44.3%
83.8%
Infrastructure
52.2%
10.0%
Information not available
3.4%
6.3%
CATEGORIZATION OF PM IN DATASET 1
Through content analysis, the first author interpreted the 417 cleaned-up PM from dataset
1 using supporting documentation from the monthly reports before categorizing it. The
interpretation was reviewed by industry experts from Scandinavia who had participated
as mentors to the professionals during the VDCCP and were therefore familiar with the
context of the projects in the reports. The second and fourth authors then did another
review of the PM interpretation and categorization and updated the categories where
required. For PM which were categorized differently by different authors, a mutual
decision was taken to select one category over another. A few PM were found which
satisfied the criteria of more than one category. As an example, “number of clashes

Tulika Majumdar, Steinar G. Rasmussen, Alexandre Almeida Del Savio, Katrin Johannesdottír,
Eilif Hjelseth, and Martin A. Fischer
Proceedings IGLC30, 25-31 July 2022, Edmonton, Canada 1181
identified in the 3D BIM during the ICE session” can be categorized as both BIM and
ICE. For such PM too, a mutual decision was taken by the authors to select one category
over another.
VALIDATION OF PM CATEGORIZATION IN DATASET 2
The process to interpret and categorize PM was repeated for dataset 2. After the first
author interpreted the 487 cleaned-up PM, a second round of interpretation was carried
out by the third author, a researcher from the University of Lima who was also the
program coordinator for the VDCCP. Once a PM was interpreted adequately, a mutual
decision was taken to categorize it. No PM was found in dataset 2 which could not be
categorized into the 3 key VDC elements of ICE, BIM, and PPM, validating the proposed
categorization.
FINDINGS AND DISCUSSION
The categorization of PM enabled a comparison of PM across the two datasets. Manual
data analysis of the categorized PM highlighted a) the most reported PM category and b)
the number of PM categories reported by AEC professionals across the two datasets
COMPARISON OF PM CATEGORY
As shown in Figure 2a, PM which were categorized as ICE were reported by more than
80% professionals in both the datasets (Scandinavia: 84%, Latin America: 91%). PM
categorized as PPM were reported by more than 65% in both the datasets (Scandinavia:
68%, Latin America: 79%). There was a disparity in the number of professionals who
reported PM which were categorized as BIM (Scandinavia: 30%, Latin America: 91%).
Figure 2a: Percentage of professionals who reported ICE, BIM, and PPM production
metrics in Scandinavia (Dataset 1, n =115) and Latin America (Dataset 2, n = 80)
• ICE: ICE emerged as the PM category reported by most professionals in both the
datasets, indicating that collocation and collaboration are picking up as a way of
91%
79%
91%
30%
68%
84%
0% 20% 40% 60% 80% 100%
BIM
PPM
ICE
Chart Title
Dataset 1 (Scandinavia) Dataset 2 (Latin America)

VDC in Practice: A Preliminary Categorization of Production Metrics Reported in Scandinavia and Latin
America
Lean and BIM 1182
working across the geographies. This finding can be used by project managers and
AEC organizations to invest in resources which foster collaboration, such as the
big room (A. Khanzode et al., 2011.). Organizations can define standard lists of
individual ICE PM which can assist project teams to assess team collaboration.
Software providers can focus on product functionality which enables assessment
of collaboration and recommends strategies to improve it.
• PPM: A large number of professionals across both the datasets reported PM in
this category. PM which were categorized as PPM bear a resemblance to metrics
tracked on traditional projects, such as “number of change orders” and “number
of requests for information”, indicating that they may be better understood by
professionals as compared to BIM PM. “Percent Plan Complete” was the PM
which was most reported in this category across both the datasets.
• BIM: It was surprising to see BIM PM being reported by only 30% of the
professionals in dataset 1 as compared to 91% professionals in dataset 2. The PM
which most professionals reported under BIM was the “number of clashes
detected or resolved”. Hard clashes, which may be better understood in the
industry, are more common in building projects (Matejka & Sabart, 2018). 44%
of the professionals in dataset 1 were working on building projects as compared
to 84% in dataset 2. When standardizing and benchmarking PM, organizations
should consider the project type.
NUMBER OF PM CATEGORIES REPORTED
Figure 2b shows the count of professionals who reported PM representing all three VDC
elements (Scandinavia: 17%, Latin America: 68%) as compared to those who reported
PM representing a single VDC element (Scandinavia: 36%, Latin America: 6%)
Figure 2a: Percentage of professionals who reported production metrics representing
1, 2 and 3 VDC elements, i.e., ICE, BIM, and PPM in Scandinavia (Dataset 1, n =115)
and Latin America (Dataset 2, n = 80)

Tulika Majumdar, Steinar G. Rasmussen, Alexandre Almeida Del Savio, Katrin Johannesdottír,
Eilif Hjelseth, and Martin A. Fischer
Proceedings IGLC30, 25-31 July 2022, Edmonton, Canada 1183
There was a disparity in the number of professionals who reported all the three categories
of PM across the two datasets. Most professionals in dataset 2 reported PM in all the three
categories while most professionals in dataset 1 reported PM in two categories. A handful
of professionals reported PM in a single category. Almost 40% of the professionals in
dataset 1 and 16% professionals in dataset 2 reported PM which could be categorized as
ICE and PPM but none which could be categorized as BIM.
LIMITATIONS
The categorization recommended in this paper is based on self-reported PM data from the
VDCCP. Monthly reports provided evidence of the use of PM in the form of pictures,
tables, charts, and summaries. Triangulation, in the form of reviewing actual artefacts
from the projects (such as projects schedules, RFI and change order logs), was beyond
the scope of this work. While it is possible that certain PM used were not reported due to
confidentiality, such PM would likely fall under one of the three categories recommended
in this paper.
While this study provided insight into 3 broad categories of PM reflecting project
organization, tools for visualization and simulation and processes for work tasks, it did
not provide a comprehensive list of individual PM reported under each category.
Comparing projects based on PM category alone is not sufficient. Individual PM for each
category need to be compared for establishing benchmarks across projects.
This study did not consider differences in PM reported based on project type and phase
For example, BIM PM related to hard clashes may not have been relevant for
infrastructure projects. Similarly, PM related to field material delivery and site generated
change orders were not relevant for projects in the design phase. Project managers should
consider these factors while comparing projects based on the PM categories reported.
It is possible that a certain PM category was reported more than another because of
familiarity or ease of tracking. As an example, “number of change orders”, which have
traditionally been managed on construction projects and are categorized as PPM, are
possibly better understood by professionals. In comparison, PM related to BIM and ICE
may not be understood very well. Companies should therefore put in effort towards
training employees to familiarize them with BIM and ICE PM. Developers of BIM tools
should expand functionality of solutions so that it is easy for professionals to track BIM
PM such as “BIM rework hours”.
A few PM, such as “number of tasks in the weekly lookahead plan”, if identified in
an ICE session, could have been categorized in more than one category. This study did
not consider PM categories by combinations of VDC elements. However, the number of
such PM found in this study was not significant to alter the results of the comparison.
FUTURE RESEARCH
This study provides a categorization of PM into the three key elements of VDC. Future
research should:
• Include more PM data from projects in other geographies to further validate
sufficiency of the three categories. In addition, PM reported by AEC
professionals who did not go through the VDCCP should be included to test the
applicability of the categorization.

VDC in Practice: A Preliminary Categorization of Production Metrics Reported in Scandinavia and Latin
America
Lean and BIM 1184
• Define individual PM under each of the three categories. Project managers and
companies will then be able to identify the top individual PMs tracked under
each category to establish benchmarks.
• Compare PM categories based on project characteristics such as type (building
vs infrastructure), size (small, medium, large), ownership (public, private) etc.
• Explore whether individual PM under one category are difficult to understand
and report as compared to another.
• Test the exclusivity of the three categories from this study.
CONCLUSIONS
Industries such as manufacturing have standard lists of PM for comparison, learning and
benchmarking across projects. This has not been possible in the AEC industry as there is
no common vocabulary to report and share PM data. This results in project teams
reinventing the wheel each time and bearing the risk of tracking sub-optimal PM which
could negatively impact project performance. By creating two datasets of PM reported by
195 AEC professionals in Scandinavia and Latin America, this research contributed a
feasible categorization of PM based on three key VDC elements, i.e., ICE, BIM, and PPM.
A comparison of the PM categories showed that ICE PM were reported by most
professionals in both the datasets, (Scandinavia: 84%, Latin America 91%), followed by
PPM (Scandinavia: 68%, Latin America 79%) and BIM (Scandinavia: 30%, Latin
America 91%). In addition, it operationalized VDC theory by comparing the PM
categories reported. There was a disparity in the number of professionals who reported
all the three PM categories (Latin America: 68%, Scandinavia:17%), indicating that there
is a gap in theory versus practice of VDC, which recommends the use of ICE, BIM, and
PPM together.
ACKNOWLEDGMENTS
We thank all the professionals who enrolled in the 1st large-scale VDCCP and reported
on the PM they tracked on their projects. We thank Gunnar Skeie, Bjørg Egeland, Johnny
Rimestad Sætre, Henning Vardøen, Thomas Iversen and Lars C. Christensen, who
reviewed the categorization of the PM in the Scandinavian dataset and provided missing
data wherever possible. We thank the CIFE community for its financial support of this
research through the seed research grant on “A Bottom-Up Approach to Generate VDC
Implementation Patterns by Comparing Production Metrics Data”, (2020) and
“Similarities and Differences in Production Metrics Reported across VDC
Implementations”, (2021).
REFERENCES
Belsvik, M. R., Lædre, O., & Hjelseth, E. (2019). Metrics in VDC Projects. Proc. 27 the
Annual Conference of the International. Group for Lean Construction (IGLC).
Carley, K. (1990). Content analysis.
http://casos.cs.cmu.edu/publications/papers/carley_1990_contentanalysis.PDF
Fischer, M., Reed, D., Khanzode, A., & Ashcraft, H. (2014). A simple framework for
integrated project delivery. Proceedings of the 22nd Annual Conference of the
International Group for Lean Construction, 1319–1330.

Tulika Majumdar, Steinar G. Rasmussen, Alexandre Almeida Del Savio, Katrin Johannesdottír,
Eilif Hjelseth, and Martin A. Fischer
Proceedings IGLC30, 25-31 July 2022, Edmonton, Canada 1185
Fosse, R., & Ballard, G. (2016). Lean design management in practice with the last
planner system. Proceedings of the 24th Annual Conference of the International
Group for Lean Construction, Boston, EE. UU.
Gao, J. (n.d.). A Characterization Framework to Document and Compare BIM
Implementations on Construction Projects [Ph.D., Stanford University].
Gunasekaran, A., Patel, C., & Tirtiroglu, E. (2001). Performance measures and metrics
in a supply chain environment. International Journal of Operations &
Production Management, 21(1/2), 71–87.
https://doi.org/10.1108/01443570110358468
Haugstvedt, O. H. (2019). Metrics development strategy for VDC in Design
management [Master’s Thesis]. NTNU.
Khanzode, A., Fischer, M., & Reed, D. (2011). Challenges and benefits of
implementing Virtual Design and Construction technologies for coordination of
mechanical, electrical, and plumbing systems on a large healthcare project.
Center for Integrated Facility Engineering, Stanford University.
Khanzode, A. R. (2011). An Integrated Virtual Design and Construction and Lean
(IVL) Method for Coordination of Mechanical, Electrical and Plumbing (MEP)
Systems [Ph.D., Stanford University]. Retrieved May 24, 2022, from
https://www.proquest.com/docview/2451876778/abstract/AF35190AD6A5415B
PQ/1
Kunz, J., & Fischer, M. (2012). Virtual design and construction: Themes, case studies
and implementation suggestions. Center for Integrated Facility Engineering,
Stanford University.
Lavy, S., Garcia, J. A., Scinto, P., & Dixit, M. K. (2014). Key performance indicators
for facility performance assessment: Simulation of core indicators. Construction
Management & Economics, 32(12), 1183–1204.
https://doi.org/10.1080/01446193.2014.970208
Majumdar, T., Rasmussen, S. G., Del Savio, A. A., Johannesdottir, K., Hjelseth, E., &
Fischer, M. (2022). A Comparative Analysis of Production Metrics across VDC
Implementations. Construction Research Congress 2022, 1024–1033.
https://ascelibrary.org/doi/abs/10.1061/9780784483978.104
Matejka, P., & Sabart, D. (2018, May 23). Categorization of clashes and their impacts
on construction projects. 17th International Scientific Conference Engineering
for Rural Development.
https://doi.org/10.22616/ERDev2018.17.N102
Rischmoller, L., Reed, D., Khanzode, A., & Fischer, M. (2018). Integration enabled by
virtual design and construction as a lean implementation strategy. Proc. 26th
Annual Conference of the International. Group for Lean Construction (IGLC),
Chennai, India.