A meta-analysis of the effect of time pressure on human performance

Abstract

A quantitative review of time pressure effects on human performance was conducted. One-hundred-twenty-five references were identified that met selection criteria. These studies provided 827 effect sizes. Analyses revealed an overall small but detrimental effect of time pressure on performance. However, moderator analyses indicated that the effect of time pressure varied as a function of task type and the measure of performance (accuracy vs. speed). As expected, time pressure facilitated speed but impaired accuracy for both perceptual and cognitive tasks. Although there were few studies available for motor tasks, evidence from the available studies indicated that time pressure reduced speed. Accuracy data were inconclusive. Across all analyses there was evidence of substantial variability across studies, indicating that other moderating variables may influence the performance effects of time pressure.

Full text

Page 1

A META-ANALYSIS OF THE EFFECT OF
TIME PRESSURE ON HUMAN PERFORMANCE
J.A. Szalma, P.A. Hancock, and S. Quinn
University of Central Florida

A quantitative review of time pressure effects on human performance was conducted. One-hundred-twenty-five
references were identified that met selection criteria. These studies provided 827 effect sizes. Analyses revealed
an overall small but detrimental effect of time pressure on performance. However, moderator analyses indicated
that the effect of time pressure varied as a function of task type and the measure of performance (accuracy vs.
speed). As expected, time pressure facilitated speed but impaired accuracy for both perceptual and cognitive
tasks. Although there were few studies available for motor tasks, evidence from the available studies indicated
that time pressure reduced speed. Accuracy data were inconclusive. Across all analyses there was evidence of
substantial variability across studies, indicating that other moderating variables may influence the performance
effects of time pressure.

INTRODUCTION
One of the most proximal sources of stress in
operational settings is often the task itself (Hancock
& Warm, 1989). Hancock and Warm (1989)
identified two fundamental task dimension that
influences the level of stress and the resulting
adaptation by the individual, that of information
structure and information rate. In real-world settings
temporal constraints often impose considerable
stress and workload on operators, as anyone who is
pressed to meet proposal deadlines can attest.
However, to date there have been no systematic,
quantitative reviews of the effects of time pressure
on human performance. In the present study, we
examined the effect of time pressure on human
performance and the effects of two task moderators.
The first moderator was the type of task performed,
with tasks categorized as perceptual, cognitive, or
psychomotor. The second moderator was the type of
dependent measure (i.e., whether measure reflects
accuracy or speed. Previous meta-analyses have
indicated that these moderators influence the
relationship between environmental sources of
stress and performance (Conway, Szalma, &
Hancock, 2007; Hancock, Ross, & Szalma, 2007;
Ross et al., 2006).

METHOD
Literature Accumulation
Given the immense size of the time pressure
literature, only a portion of research reports was
gathered. These studies were obtained by consulting
PsycINFO, MEDLINE, Dissertation Abstracts
International databases, and Science Citation Index.
This process resulted in identification of 281
articles, reports, dissertations, and theses. Of these,
125 papers were identified which met the selection
criteria for inclusion. These 125 studies resulted in
827 effect sizes.

Identified Criteria for Study Inclusion
Each acceptable paper included in the meta-analysis
possessed the following four criteria:
1) Only studies that presented empirical
findings (i.e., review papers were excluded)
regarding the effect of time stress (e.g.,
instructions to complete tasks as quickly as
possible, deadlines, or stimulus presentation
rate) on performance were included
2) A control condition in which no time
pressure was imposed was required.
3) The dependent variable must measure
performance of a healthy adult human
participant.
4) Each study had to present enough
information to obtain effect size statistics.

The Calculation of Effect Sizes
Pair-wise effect-sizes (Hedges’ g), is a
standardized mean difference
experimental and control conditions (Hedges &
Olkin, 1985). Formulas upon which effect size was
calculated are found in Lipsey and Wilson (2001)
between the
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 52nd ANNUAL MEETING—2008
1513
Copyright 2008 by Human Factors and Ergonomics Society, Inc. All rights reserved. 10.1518/107118108X353174

Page 2

and Hedges, Shymansky, and Woodworth (1989).
In calculating the effect size, the sign of the effect
size was controlled to ensure that a positive score
represented improvement in performance in the
experimental group relative to the control group,
whereas a negative g-score indicated performance
impairment. The culminated effect-size statistics
were then computed and adjusted for statistical bias
and methodological differences (i.e., between- vs.
within-subjects design) based on formulas available
in Lipsey and Wilson (2001) and Morris and
DeShon (2002).
For comparison purposes, it is helpful to keep in
mind Cohen’s (1988) guidelines to interpreting
effect sizes as small (g ≤0.20), medium (g ≈0.50),
or large (g ≥0.80).

Variance Estimates
In addition to the weighted mean effect size,
two variance estimates were computed: variability
due to sampling error (se2) and variability of the
effect sizes (sg2). These values were used to
estimate the variability due to differences in the
population effect sizes, δ (sδ2). A large sδ2 may
indicate that there is variability among the observed
effect sizes that cannot be accounted for by
sampling error and that there are likely to be one or
more variables moderating the magnitude of the
effect in question. These correspond to the random
effects model described by Hunter and Schmidt
(2004).

Moderator Analyses
It is important to note that most of the studies
used in the current analysis reported multiple effect-
size estimates (e.g., effect-size values for three tasks
reported in one study). Thus, many of the included
effect sizes were derived from common participant
samples, violating the assumed independence
between effect-size estimates. These violations can
lead to underestimation of the variance due to
sampling error. To avoid such violations effects
were averaged within studies prior to estimating
means and variances (see Lipsey & Wilson, 2001).
In moderator analyses this averaging procedure was
conducted within each level of the moderator
variable (e.g., across all variables for a general
effect or only within tasks for a task moderator
analysis).

RESULTS
The weighted average effect size for the
effect of time pressure on performance across all
task types and performance measures was g = -0.19
(k= 125), indicating a small negative effect on
performance. The confidence interval based on the
sampling error associated with this effect size (s2e)
indicated that is mean effect was statistically
reliable (-0.23<δ<-0.15). However, adopting the
‘75% rule’ recommended by Hunter and Schmidt
(2004), it is clear that a substantial portion of the
observed variance () cannot be accounted for by
sampling error and is likely due to moderating
factors [()/() = 0.02].
The moderator analyses for task type and
measure of performance are shown in Table 1.
Time pressure had small to moderate effects within
each task type, although the effect on cognitive and
motor tasks was impaired by time pressure while
the effect associated with perceptual tasks was
facilitative. As one might expect, time pressure
results in faster but less accurate responding. In
both cases, the effects were in the medium range
(Cohen, 1988). However, inspection of Table 1
reveals that less than 8% of the observed variance in
effect sizes can be accounted for by sampling error.
To investigate this heterogeneity further, a
hierarchical meta-analysis of dependent measure
within each task type was computed. These results
are shown in Table 2. Note that for performance
accuracy associated with motor tasks only two
effect sizes were observed. Thus, variance data for
these are not reported, although the weighted mean
effect size and each individual effect size are shown
in the table. It is not unusual in meta-analyses of
stress and human performance to observe instances
in which there are few or even no studies that
provided an effect size (e.g., Conway et al., 2007).
As can be seen in Table 2, response time is
facilitated and accuracy impaired for both cognitive
and perceptual tasks, although the effect is stronger
for cognitive as compared to perceptual tasks. For
motor tasks response time is actually slower under
time pressure, although these data should be
2
ds
2
es
2
ds
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 52nd ANNUAL MEETING—2008
1514

Page 3

interpreted with considerable caution as there w
relatively few studies employing motor tasks.
Indeed, for accuracy no clear conclusion is possible
with respect to motor tasks, as one effect size was
DISCUSSION
strong and positive while the other was strong and
negative.


Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-
analysis: Correcting for error and bias in research
findings. 2nd Ed. Thousand Oaks, CA: Sage.
Lipsey, M.W., & Wilson, D.B. (2001). Practical meta-
analysis. Thousand Oaks, CA: Sage.
Morris, S.B., & DeShon, R.P. (2002). Combining effect
size estimate in meta-analysis with repeated
measures and independent-groups designs.
Psychological Methods, 7, 105-125.
Ross, J., Conway, G., Szalma, J.L., Saxton, B.,
Braczyk, A., & Hancock, P.A. (2006, October) A
meta-analysis of periodic noise stress on human
performance. Paper presented at the Society for
Human Performance in Extreme Environments, San
Francisco, CA.






























ere

The results of this meta-analysis confirmed that
time pressure effects are not equivalent across tasks
and that while they may induce faster responding,
they do so at the expense of accuracy. However, it
is noteworthy that across all analyses the proportion
of variance due to sampling error was only a small
proportion of the total observed variance in the
effect sizes. Such ratios indicate that a substantial
proportion of observed variance is due to factors
other than sampling error. Future work will
examine potential variables such as duration of task
performance under time pressure, sex differences,
and the nature of the time pressure manipulation.
Nevertheless, these data confirm an important
conclusion from prior meta-analyses of the effects
of environmental stressors on performance (e.g.,
Conway et al., 2007; Hancock et al., 2007; Ross, et
al., 2006): More research is needed to identify
other factors (e.g., the presence of multiple
stressors) that moderate the relationship between
sources of stress in the environment and task
performance.
REFERENCES
Cohen, J. (1988). Statistical power for the
behavioral sciences (2nd ed.). Hillsdale, NJ:
Erlbaum
Conway, G., Szalma, J.L., & Hancock, P.A. (2007).
A quantitative meta-analytic examination of
whole-body vibration effects on human
performance. Ergonomics, 50(2), 228-245.
Hancock, P.A., Ross, J.M., & Szalma, J.L. (2007).
A meta-analysis of performance response under
thermal stressors. Human Factors, 49, 851-877.
Hancock, P.A., & Warm, J.S. (1989). A dynamic
model of stress and sustained attention. Human
Factors, 31, 519-537.
Hedges, L.V. & Olkin, I. (1985). Statistical methods
for meta-analysis. New York: Academic Press.
Hedges, L.V., Shymansky, J.A., & Woodworth, G.
(1989). A practical guide to modern methods of
meta-analysis. Washington, DC: National
Science Teachers Association.
PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 52nd ANNUAL MEETING—2008
1515

Page 4

Table 1: Moderator Analysis for Task Type and Measure of Performance
Category k
d
2
es
2
ds
1.51
3.7
2.52
5.01
2.46
95% (
0.10<δ<0.29
-0.31<δ<-0.20
-0.68<δ<0.00
-0.66<δ<-0.54
0.35<δ<0.07
) CI
2
es
()/( )
2
es
2
ds
0.05
0.02
0.07
0.02
0.04
Perception
Cognitive
Motor
Accuracy
RT
29
90
6
92
76
0.2 0.07
0.06
0.18
0.09
0.11
-0.25
-0.34
-0.6
0.42



Table 2: Moderator Analysis of Dependent Measure with in Each Task Category
Category
Perception
Accuracy
RT
Cognitive
Accuracy
RT
Motor
Accuracy
RT
* effect sizes: g = -0.9; g = 1.01
K
d

-0.33
0.26

-0.66
0.57

0.1
-0.6
2
es

0.11
0.1

0.09
0.11

---
0.21
2
ds

3.39
2.5

5.6
1.49

---
2.03 -1.00<δ<-0.20
95% CI ()
2
es
()/(

)
2
es
2
ds

21
27

69
44

2*
-0.47<δ<-0.19
0.14<δ<0.38

-0.73<δ<-0.59
0.47<δ<0.66
0.03
0.04

0.02
0.07

--- ---
50.10
















PROCEEDINGS of the HUMAN FACTORS AND ERGONOMICS SOCIETY 52nd ANNUAL MEETING—2008
1516