Visualizing Eye Tracking Convex Hull Areas: A Pilot Study for Understanding
How Craft Workers Interpret 2D Construction Drawings
Matthew SEARS1, Omar ALRUWAYTHI2, and Paul GOODRUM3
Department of Civil, Environmental and Architectural Engineering, University of
Colorado Boulder, 1111 Engineering Drive, Boulder, CO 80309-0428;
PH (859) 358-0908; email: Matthew.H.Sears@colorado.edu
2 Department of Civil Engineering, Taibah University, Universities Road, P.O. Box
344, Madinah, Saudi Arabia; email: Omar.Alruwaythi@colorado.edu
Department of Civil, Environmental and Architectural Engineering, University of
Colorado Boulder, 1111 Engineering Drive, Boulder, CO 80309-0428;
PH (303) 492-0475; email: Paul.Goodrum@colorado.edu
Engineering deliverables to construction craft workers have remained largely
unchanged for a century. Black and white, 2D paper drawings are the primary medium
by which engineering designs are communicated, but a certain level of experience is
required in order for a craft worker to efficiently extract necessary information from a
drawing. This work presents a novel method of visualizing and analyzing eye tracking
data through the use of convex hull areas. Convex hull areas are proposed as a metric
for measuring the amount of information that a participant is processing from a
drawing at a given instant in time.
Eye tracking data was collected in a previous study where 20 construction craft
workers were tasked with assembling a PVC pipe assembly from traditional 2D
isometric pipe spool drawings. Demographic data and spatial cognition data were also
collected from each participant. In the present work, craft worker spatial cognition and
years of construction experience were both shown to correlate with a craft worker’s
average convex hull area. The authors developed software for producing animations of
eye tracking data convex hull areas, but have only begun to assess the data produced
from the convex hull analysis method. Average convex hull areas were analyzed in the
present work, but several potential additional metrics are suggested. This study was
severely limited by the small sample size of the previous study and further data
collection is warranted to confirm the findings presented herein.
Construction engineering design software has advanced dramatically in recent
years. Building Information Modeling (BIM) adoption in North America exploded
from 28% in 2007 to 71% in 2012 (McGraw Hill Construction 2014), yet design
deliverables to craft workers in the field have remained largely unchanged. While
designers are increasingly working in color, near photorealistic, 3D modeling
environments, their designs are overwhelmingly converted to traditional black and
white, 2D paper drawings when they are sent to craft workers in the field. Moreover,
as the production of 2D drawings from 3D models becomes increasingly automated
through BIM software, final conversion from 3D to 2D may become more of an
afterthought, or even a budget constraint, leading designers to devote less and less time
to 2D plan production and review.
Drawings are the primary medium by which designers convey their designs to
craft workers; however, the intent of drawings are often much more apparent to the
designer than to the craft worker, especially if the craft worker is inexperienced or
uninitiated into the customs and conventions of construction drawings (Emmitt and
Gorse 2003). This research was a pilot study conducted with an aim to begin to
understand how construction craft workers view and interpret 2D drawings, and is part
of a larger effort that has two overarching research questions in mind: 1) Can craft
workers be trained to read 2D construction drawings in a manner that increases their
productivity? and 2) Can engineering deliverables be conveyed in a consistent manner
that leads to increased craft worker productivity? This work builds upon previous work
by (Alruwaythi 2017), wherein 20 pipefitters were provided with 2D isometric pipe
spool drawings and tasked with assembling a ½” diameter PVC pipe assembly.
Demographic data was collected from each of the participants prior to the pipe
assembly task, and eye tracking data was collected during the task. The present work
is a post-hoc analysis of the collected data.
Some of the earliest research into eye tracking was conducted by (Buswell
1935), who used state of the art technology at the time of his experiments to record and
analyze the horizontal and vertical eye movements of 200 individuals while they
viewed 55 different photographs of art. The technology used by Buswell was perhaps
rudimentary by today’s standards, but the data collected was very much the same as
the data collected in current eye tracking research – horizontal and vertical locations of
eye fixations over time. One of the most frequently used methods for visualizing eye
tracking data today is a heatmap, which is a color graded representation of the relative
spatial density of fixation points (Grindinger et al. 2010), as shown in Figure 1.
Heatmaps are useful for identifying the areas of an image that participants tend to fixate
upon; however, like most of the summary statistics and visualization methods
Figure 1. Typical heatmap for 20
Figure 2. Typical scanpath
visualization, 3 participants shown
commonly available for eye tracking data today, heatmaps completely disregard the
temporal nature of the eye tracking data.
Another frequently used technique is a scanpath visualization, which is a series
of sequentially connected dots (Goldberg and Helfman 2010), as shown in Figure 2.
Each dot in a scanpath visualization represents a single fixation, and the relative size
of the dots are often varied to represent the duration of each fixation. The primary
limitation of a scanpath visualization arises from the frequency of the fixations. A
typical fixation only lasts about 300ms (Cowen et al. 2002), so a scanpath visualization
quickly becomes very complex and incomprehensible. In this research, data was often
collected for a period of several minutes, rendering a scanpath visualization rather
useless. (Goldberg and Kotval 1999) outlined another method of scanpath analysis –
the convex hull.
A convex hull, for the purposes of this research, is defined as the polygon of
least area which circumscribes a collection of fixation points and whose interior angles
are all less than 180°. A common analogy is that a convex hull takes the shape of a
hypothetical elastic band if the elastic band were stretched around a set of points, as
shown in Figure 3. The authors propose the convex hull area as an indirect measure of
the amount of information that a participant is taking in from a drawing at a given
instant in time. A large average convex hull area would suggest that the participant was
viewing a drawing as a whole, i.e. that they were seeing “the big picture,” while a small
average convex hull area would suggest that the participant was focusing more on
specific details of a drawing. Several algorithms exist today for calculating the convex
hull and the area of the convex hull for a given set of points. The “ConvexHull” package
from (SciPy.org n.d.) was utilized in the present work and incorporates the Quickhull
algorithm for computing convex hulls (Barber et al. 1996).
This study utilized eye tracking data previously collected by (Alruwaythi 2017).
The data was collected with SensoMotoric Instruments’ (SMI) Eye Tracking Glasses
2.0, and the SMI BeGaze behavioral and gaze analysis software. Twenty industrial
pipefitters currently employed on construction jobsites throughout North America were
recruited for the study. The pipefitters were fitted with the eye tracking glasses and
presented with a set of 10 isometric pipe drawings. Each of the drawings was a single
pipe spool, as shown in Figure 4, and the ten pipe spools fit together to collectively
Figure 3. A convex hull with elastic band analogy
form a single pipe assembly. The pipefitters were tasked with assembling a physical
½” PVC pipe assembly with the aid of the isometric drawings.
Figure 4. Typical pipe spool drawing provided to craft workers
It is prudent to first discuss the organization of the collected eye tracking data.
Data was collected from 20 Participants, each of these participants completed an
assembly task while viewing 10 Drawings, and for each drawing that a participant
viewed, the participant may have viewed the drawing multiple times. Each time that a
participant viewed a drawing will be henceforth referred to as a Viewing. For example,
a participant may have viewed Drawing #3, flipped to Drawing #5, and then flipped
back to Drawing #3. In this case, the participant had two viewings for Drawing #3. As
another example, a participant may have viewed Drawing #8, taken their eyes off of
the drawing and worked on the physical assembly for a period of time, and then again
viewed Drawing #8. In this case, the participant had two viewings for Drawing #8. The
required amount of time elapsed after a participant took their eyes off of a drawing
before creating a new viewing was set at 5.0 seconds. For example, if a participant was
viewing Drawing #6, took their eyes off of the drawing for 4.9 seconds, and then looked
back at Drawing #6, the collected data would have been treated as a single viewing.
Participant (20) è Drawing (10 each) è Viewing (quantity varies)
The minimum number of fixation points required for a viewing was set at 20.
After the raw data was broken into viewings, any viewing that contained fewer than 20
fixation points was discarded from further analysis. The distribution of viewing
durations from all drawings and all participants is shown in Figure 5. The distribution
is positively skewed (skew=4.52, p=1.51e-103), with a range of 4.78s – 341.02s, mean
of 30.20s, and median of 20.84s. 86% of the viewings had a duration less than 50s.
Figure 5. Viewing duration distribution, 694 total viewings
of 10 drawings by 20 participants shown
Rather than simply computing a convex hull for each viewing, a Period was
chosen and convex hulls were continuously calculated for the given period over each
viewing. For this research, a period is defined simply as a duration over which a
participant is looking at a drawing. For example, if a period of 3000ms (3 seconds) was
chosen for the sample raw data presented in Table 1, then a convex hull would be
calculated for fixations 3 through 10 (Convex Hull #1). A convex hull would be
calculated beginning with each fixation point (fixation #1, 2, 3, etc), but for clarity,
only three convex hulls have been labeled in Table 1. A convex hull does not have a
predetermined number of fixation points. Rather, the number of fixation points
included in each convex hull is dependant upon the recording times. Convex Hull #1
includes fixation 10 because the recording time of fixation 10 minus the recording time
of fixation 3 is less than or equal to the period, 3000ms.
206778.2𝑚𝑠 −203790.9𝑚𝑠 =2987.3𝑚𝑠 ≤3000𝑚𝑠 ✓
For the same reason, Convex Hull #1 does not include fixation 11 because the recording
time of fixation 11 minus the recording time of fixation 3 is greater than the period,
207043.8𝑚𝑠 −203790.9𝑚𝑠 =3252.9𝑚𝑠 >3000𝑚𝑠 ✘
After ranges of fixations were identified, the convex hulls and corresponding
convex hull areas were calculated. The fixation location coordinates were normalized
in the X and Y directions, so a convex hull area equal to 1.0 was equivalent to a
rectangle that covered an entire pipe spool drawing. After calculating the convex hulls
and areas, animations were developed which plotted the fixation points and convex
hulls on the left-hand side, while the instantaneous convex hull area and time-weighted
average convex hull area were plotted on the right-hand side. A frame from one of the
animations is presented in Figure 6. In this particular frame of the animation, on the
left-hand side of the figure, we see that 11 fixation points were plotted (238 through
248, inclusive). The difference between the reference time of fixation 248 and 238 was
2821.7ms (period of 3000ms was used). On the right-hand side of the figure, we see
that 94.361s has elapsed since the viewing began, and at this point in time, the
participant’s convex hull area has, on average, covered 8.914% of the pipe spool
drawing. Interested readers can view the full animation here:
Table 1. Sample of raw eye tracking data
Convex Hull #1 Convex Hull #2 Convex Hull #3
Figure 6. A typical frame from an animation that illustrates
how a participant’s convex hull area changes over time
The authors’ have yet to uncover any research that would support a particular
period, so the entire analysis was conducted for several periods: 3000ms, 4000ms,
5000ms, 6000ms, 8000ms, 10000ms, 12000ms, 15000ms. After computing the average
convex hull areas for each period, the authors searched for linear and logarithmic
correlations between the average convex hull areas and the demographic data
previously collected by (Alruwaythi 2017). The demographic data included: time of
completion of the pipe assembly; % of direct work; % of indirect work; % of rework;
number of errors made; spatial cognition, as measured by the card rotation test
(Ekstrom et al. 1976); spatial cognition, as measured by cube comparison test (Ekstrom
et al. 1976); composite spatial cognition, an average of the card rotation test and cube
comparison test; age; years of construction work experience; and whether or not the
participant had ever received any training for reading construction drawings.
The authors were particularly interested in comparisons of the participants’
average convex hull areas and their spatial cognition. One of the strongest correlations
found (Pearson’s r=0.48, p=0.034) occurred between the participants’ composite
spatial cognition score and their average convex hull area, computed with a 12,000ms
period, as shown in Figure 7, which shows that the individuals’ average convex hull
areas increased with greater spatial cognition. A participant’s composite spatial
cognition score was an average of their card rotation test score and their cube
comparison test score.
The authors were also interested in comparisons of the participants’ average
convex hull areas and their time of completion of the pipe assembly, with the time of
completion of the pipe assembly considered as a metric of performance. The authors
failed to uncover any notable relationship between these two metrics, suggesting that
the performance for this task was not dependent upon whether a participant attempted
to view each drawing as a whole (big picture), or focus more upon details.
Figure 7. Correlation between participants’ composite spatial
cognition score and their average convex hull area
Figure 8. Correlation between participants’ years of construction
experience and average convex hull area (10,000ms period)
The strongest correlation observed (Pearson’s r=-0.49, p=0.028) occurred
between the participants’ years of construction work experience and their average
convex hull area, computed with a 10,000ms period, as shown in Figure 8, which
shows that the average convex hull area decreased with years of construction
experience among the test participants. The data set included one outlier
(experience=39 years, modified z-score=3.89) (Iglewicz and Hoaglin 1993). Excluding
this data point would have improved the correlation marginally (Pearson’s r=-0.50,
p=0.028); however, the authors saw no reason to exclude this data point.
DISCUSSION AND FUTURE WORK
A correlation was found between participants’ average convex hull areas and
their composite spatial cognition scores. In a way, this finding supports previous studies
by (Alruwaythi 2017; Goodrum et al. 2016), which each found that individuals with
low spatial cognition tend to utilize engineering deliverables differently than
individuals with high spatial cognition. Further work is required to explore exactly
what causes the differences in viewing patterns.
The correlation between participants’ years of construction experience and
average convex hull area is negative, which may suggest that as participants gain
construction work experience their convex hull area tends to decrease. However, it
should come as no surprise that there was a very strong correlation between
construction work experience and age (Pearson’s r=0.84, p=4.51e-6), so the trend
observed in Figure 8 could also indicate that today’s younger generation of
construction workers tend to have larger average convex hull areas, while the older
generation of workers today tend to have larger average convex hull areas.
This pilot study was severely limited by the sample size of 20 participants and
additional data collection will be required to draw further conclusions. This study was
also quite limited by the fact that only the average convex hull areas were studied.
Analyzing eye tracking data through convex hull areas is a novel approach and the
authors have only scratched the surface of the data available from this study. In future
work, the authors suggest studying whether craft worker performance is related to the
proportion of convex hull areas that are above/below a threshold value that demarcates
detailed viewing from distributed viewing, i.e. what proportion of a participant’s time
is spent looking at “the big picture” versus the details? Additionally, the authors are
interested in investigating whether craft worker performance is related to the time
required for a participant’s convex hull area to reach a certain cumulative area. E.g.
how long does it take for a participant’s convex hull to cover 70% of a construction
drawing? This metric might serve as an indirect measure of how long it takes a
participant to comprehend the essence of a drawing.
The authors are also interested in expanding upon the convex hull animations
to include the path of the centroid of the convex hull over time. This would essentially
result in a smoothed scanpath visualization that might be much easier to comprehend
than the typical scanpath visualization shown in Figure 2. From this effort, smoothed
scanpath lengths and velocities could be computed and investigated as well.
It should be noted that the content of the construction drawings used in this
study were in no way assessed. Previous work has suggested that convex hull areas are
related to the efficiency of a participant’s search efforts (Goldberg and Kotval 1999),
so future studies should attempt to uncover whether particular features of a construction
drawing tend to produce large/small convex hull areas. If particular features of
drawings tend to produce large/small convex hull areas, then that might be an indication
that the features are difficult for the viewer to comprehend. Finally, the authors are
interested to know whether the presence of color or 3D information in a construction
drawing impacts a participant’s convex hull area.
The software developed by the authors for the purposes of this analysis has been
made freely available under an MIT license and is hosted at
https://github.com/mattsears18/chanalysis. The authors intend to further develop this
software into a Python module, so that others may conduct their own convex hull
analyses and produce their own convex hull animations from eye tracking data.
Alruwaythi, O. F. (2017). “Influence of Engineering Information Format,
Demographics and Spatial Cognition on Craft Worker Performance.” University
Barber, C. B., Dobkin, D. P., and Huhdanpaa, H. (1996). “The quickhull algorithm
for convex hulls.” ACM Transactions on Mathematical Software, 22(4), 469–
Buswell, G. T. (1935). “How People Look at Pictures.” Psychological Bulletin, 33(2),
Cowen, L., Ball, L. J., and Delin, J. (2002). “An eye movement analysis of web page
usability.” People and Computers XVI-Memorable Yet …, (January 2016).
Ekstrom, R. B., French, J. W., Harman, Ha. H., and Dermen, D. (1976). Manual for
Kit of Factor-Referenced Cognitive Tests. Princeton, New Jersey.
Emmitt, S., and Gorse, C. a. (2003). “Construction Communication.” 23.
Goldberg, J. H., and Helfman, J. (2010). “Visual scanpath representation.”
Proceedings of the 2010 Symposium on Eye- …, 203–210.
Goldberg, J. H., and Kotval, X. P. (1999). “Computer interface evaluation using eye
movements: Methods and constructs.” International Journal of Industrial
Ergonomics, 24(6), 631–645.
Goodrum, P. M., Asce, M., Miller, J., Sweany, J., and Alruwaythi, O. F. (2016).
“Influence of the Format of Engineering Information and Spatial Cognition on
Craft-Worker Performance.” American Society of Civil Engineers, 142(9), 1–10.
Grindinger, T. J., Murali, V. N., Tetreault, S., Duchowski, A. T., Birchfield, S. T.,
and Orero, P. (2010). “Algorithm for Discriminating Aggregate Gaze Points:
Comparison with Salient Regions-Of-Interest.” 1–10.
Iglewicz, B., and Hoaglin, D. C. (1993). How to Detect and Handle Outliers.
Milwaukee, Wis. : ASQC Quality Press.
McGraw Hill Construction. (2014). The business value of BIM for construction in
major global markets. SmartMarket Report.
SciPy.org. (n.d.). “ConvexHull.” The Scipy Community.