DESIGNING SIMULATION EXPERIMENTS WITH
CONTROLLABLE AND UNCONTROLLABLE FACTORS
Klaus Kaae Andersen
Department of Informatics and Mathematical Modelling
Technical University of Denmark
Bygning 321, Richard Petersens Plads
Lyngby, DK-2800, DENMARK
In this study we propose a new method for designing com-
puter experiments inspired by the split plot designs used
in physical experimentation. The basic layout is that each
set of controllable factor settings corresponds to a whole
plot for which a number of subplots, each corresponding
to one combination of settings of the uncontrollable fac-
tors, is employed. The caveat is a desire that the subplots
within each whole plot cover the design space uniformly.
A further desire is that in the combined design, where all
experimental runs are considered at once, the uniformity
of the design space coverage should be guaranteed. Our
proposed method allows for a large number of uncontrol-
lable and controllable settings to be run in a limited number
of runs while uniformly covering the design space for the
With the current advances in computing technology, com-
puter and simulation experiments are increasingly being
used to study complex systems for which physical experi-
mentation is usually not feasible. Our case study involves
a discrete event simulation model of an orthopedic surgical
unit. The discrete event simulation (DES) model describes
the individual patient’s progress through the system and
has been developed in collaboration with medical staff at
Gentofte University Hospital in Copenhagen. The unit un-
dertakes both acute and elective surgery and performs more
than 4,600 operative procedures a year. While the patients
come from various wards throughout the hospital, the main
sources of incoming patients are the four orthopedic wards
or the emergency care unit.
6 (Krahl 2002) on a Windows XP platform and controlled
from a Microsoft Excel spreadsheet with a Visual Basic for
application script. The model consists of 3 main modules:
The wards and arrival, the operating facilities, and the
recovery and discharge. Interaction with the surrounding
hospital is for example modeled with simplified processes
usingthe same resources as the processesin the surgicalunit
(occupyingthe resources) and with the patients entering and
exiting the model. Operating rooms, recovery beds, wards
andstaff are includedinthe model. Theaverageruntime for
simulating6 months(with oneweek ofwarm-up)operations
is around 7 minutes. Typical outcomes are waiting times,
patient throughput and the amount of overtime.
The simulation model has two sources of noise coming
from variations in the uncontrollable factors (a.k.a. envi-
ronmental factors in physical experimentation) and from
changes in the seed controlling the random number gen-
eration process embedded in the simulation model. The
controllable factors are for example the number of op-
erating rooms and the number of surgeons, whereas the
uncontrollable factors may include for example the arrival
rate of acute patients and the time required to clean the
Inthis typeofapplication, severalissues needtobecon-
sidered. First, the controllable factors tend to be numerous
and often discrete. Moreover a single experiment usually
takes several minutes to run. Therefore a simple exhaus-
tive method, where all possible combinations of the factor
settings are considered, is often computationally infeasible
due to the exponentially increasing number of factor com-
binations. Furthermore, the settings of the uncontrollable
factors, e.g. the acute patient arrival rate or the duration of
surgical procedures, are also of interest and must be deter-
mined as they may influence the outcome of the simulations
and hence the robustness of the simulation analysis.
The paper is organized in the following manner: Sec-
tion 2 introduces design of computer experiments and de-
2909 978-1-4244-2708-6/08/$25.00 ©2008 IEEE
Proceedings of the 2008 Winter Simulation Conference
S. J. Mason, R. R. Hill, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds.
Dehlendorff, Kulahci, and Andersen
Krahl, D. 2002. The extend simulation environment. In
Proceedings of the 2002Winter SimulationConference,
Montgomery, D. C. 2005. Design and analysis of experi-
ments. 6th ed. John Wiley and Sons, Inc.
Sacks, J., W. J. Welch, T. J. Mitchell, and H. P. Wynn.
1989. Design and analysis of computer experiments.
Statistical Science 4 (4): 409–423.
Sanchez, S. M. 2000. Robust design: Seeking the best of
all possible worlds. In Proceedings of the 2000 Winter
Simulation Conference, 69–76.
Santner, T. J., B. J. Williams, and W. I. Notz. 2003. The
design and analysis of computer experiments. Springer.
Taguchi, G. 1987. System of experimental design, volumes
1 and 2. UNIPUB/Krauss International, White Plains,
CHRISTIAN DEHLENDORFF is a Ph.D. student
at the Department of Informatics and Mathematical
Modelling, Technical University of Denmark.
email and web addresses are <email@example.com> and
MURAT KULAHCI is an Associate Professor at the
Department of Informatics and Mathematical Modelling,
Technical University of Denmark. His email address is
KLAUS KAAE ANDERSEN is an Associate Professor
at the Department of Informatics and Mathematical
Modelling, Technical University of Denmark. His email
address is <firstname.lastname@example.org>.