Conference PaperPDF Available

Does the Erlang C model fit in real call centers?

Authors:

Abstract and Figures

We consider the Erlang C model, a queuing model commonly used to analyze call center performance. Erlang C is a simple model that ignores caller abandonment and is the model most commonly used by practitioners and researchers. We compare the theoretical performance predictions of the Erlang C model to a call center simulation model where many of the Erlang C assumptions are relaxed. Our findings indicate that the Erlang C model is subject to significant error in predicting system performance, but that these errors are heavily biased and most likely to be pessimistic, i.e. the system tends to perform better than predicted. It may be the case that the model's tendency to provide pessimistic (i.e. conservative) estimates helps explain its continued popularity. Prediction error is strongly correlated with the abandonment rate so the model works best in call centers with large numbers of agents and relatively low utilization rates.
Content may be subject to copyright.
Proceedings of the 2010 Winter Simulation Conference
B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan, and E. Yücesan, eds.
DOES THE ERLANG C MODEL FIT IN REAL CALL CENTERS?
Thomas R. Robbins D. J. Medeiros
East Carolina University The Pennsylvania State University
Department of Marketing and Supply Chain Industrial and Manufacturing Engineering
3212 Bate Building 310 Leonhard Building
Greenville, NC 27858, USA University Park, PA 16802, USA
Terry P. Harrison
The Pennsylvania State University
Smeal College of Business
459 Business Building
University Park, PA 16802, USA
ABSTRACT
We consider the Erlang C model, a queuing model commonly used to analyze call center performance.
Erlang C is a simple model that ignores caller abandonment and is the model most commonly used by
practitioners and researchers. We compare the theoretical performance predictions of the Erlang C model
to a call center simulation model where many of the Erlang C assumptions are relaxed. Our findings in-
dicate that the Erlang C model is subject to significant error in predicting system performance, but that
these errors are heavily biased and most likely to be pessimistic, i.e. the system tends to perform better
than predicted. It may be the case that the model’s tendency to provide pessimistic (i.e. conservative) es-
timates helps explain its continued popularity. Prediction error is strongly correlated with the abandon-
ment rate so the model works best in call centers with large numbers of agents and relatively low utiliza-
tion rates.
1 INTRODUCTION
A call center is a facility designed to support the delivery of some interactive service via telephone
communications; typically an office space with multiple workstations manned by agents who place and
receive calls (Gans, Koole et al. 2003). Call centers are a large and growing component of the U.S. and
world economy and are estimated to employ approximately 2.1 million call center agents (Aksin, Armo-
ny et al. 2007). Large scale call centers are technically and managerially sophisticated operations and
have been the subject of substantial academic research. The literature focused on call centers is quite
large, with thorough and comprehensive reviews provided in (Gans, Koole et al. 2003) and (Aksin, Ar-
mony et al. 2007). Empirical analysis of call center data is given in (Brown, Gans et al. 2005).
Call centers are examples of queuing systems; calls arrive, wait in a virtual line, and are then serviced
by an agent. Call centers are often modeled as M/M/N queuing systems, or in industry standard terminol-
ogy - the Erlang C model. The Erlang C model makes many assumptions which are questionable in the
context of a call center environment. Specifically the Erlang C model assumes that calls arrive at a
known average rate, and that they are serviced by a defined number of statistically identical agents with
service times that follows an exponential distribution. Most significantly, Erlang C assumes all callers
wait as long as necessary for service without hanging up. The model is used widely by both practitioners
and academics.
2853978-1-4244-9864-2/10/$26.00 ©2010 IEEE
Robbins, Medeiros and Harrison
Recognizing the deficiencies of the Erlang C model, many recent papers have advocated using alter-
native queuing models and staffing heuristics which account for conditions ignored in the Erlang C mod-
el. The most popular alternative is the Erlang A model, a simple extension of the Erlang C model that al-
lows for caller abandonment. For example, in a widely cited review of the call center literature (Gans,
Koole et al. 2003) , the authors state “For this reason, we recommend the use of Erlang A as the standard
to replace the prevalent Erlang C model.” Another widely cited paper examines empirical data collected
from a call center (Brown, Gans et al. 2005) and these authors make a similar statement; using Erlang-
A for capacity-planning purposes could and should improve operational performance. Indeed, the model
is already beyond typical current practice (which is Erlang-C dominated), and one aim of this article is to
help change this state of affairs.”
The purpose of this study is to systematically analyze the fit of the Erlang C model in realistic call
center situations. We seek to understand the nature and magnitude of the error associated with the model,
and develop a better understanding of what factors influence prediction error.
The remainder of this paper is organized as follows. In Section 2 we review the Erlang C model and
highlight the relevant literature. In section 3 we present a general model of a steady state call center envi-
ronment and review the simulation model we developed to evaluate it. In section 4 we evaluate the per-
formance of the Erlang C model. We conclude in Section 5 with summary observations and identify fu-
ture research questions.
2 QUEUING MODELS AND THE ASSOCIATED LITERATURE
Queuing models are used to estimate system performance of call centers so that the appropriate staff-
ing level can be determined to achieve a desired performance metric such as the Average Speed to An-
swer, or the Abandonment percentage. The most common queuing model used for inbound call centers is
the Erlang C model (Gans, Koole et al. 2003; Brown, Gans et al. 2005). A Google search on “Erlang C
Calculator” generates about 700,000 items including a large number of downloadable applications to cal-
culate staffing requirements based on the Erlang C model.
The Erlang C model (M/M/N queue) is a very simple multi-server queuing system. Calls arrive ac-
cording to a Poisson process at an average rate of
λ
. By nature of the Poisson process interarrival times
are independent and identically distributed exponential random variables with mean
1
λ
. Calls enter an
infinite length queue and are serviced on a First Come First Served (FCFS) basis. All calls that enter
the queue are serviced by a pool of
n
homogeneous (statistically identical) agents at an average rate
of
n
µ
. Service times follow an exponential distribution with a mean service time of
1
.
The steady state behavior of the Erlang C queuing model is easily characterized, see for example
(Gans, Koole et al. 2003). The offered load, a unit-less quantity often referred to as the number of Er-
langs, is defined as
R
λ µ
. The traffic intensity (aka utilization or occupancy) is defined as
(
)
N R N
ρ λ µ
= .
Given the assumption that all calls are serviced, the traffic intensity must be strictly less than one or
the system becomes unstable, i.e. the queue grows without bound. This system can be analyzed by solv-
ing a set of balance equations and the resulting steady state probability that all N agents are busy is
{ }
1 1
0 0
1
0 1
! ! ! 1
m m m
N N
m m
R R R
P Wait
m m N R N
= =
> = +
(1)
Equation (1) calculates the proportion of callers that must wait prior to service, an important measure of
system performance. Another relevant performance measure for call centers managers is the Average
Speed to Answer (ASA).
2854
Robbins, Medeiros and Harrison
[ ]
{ }
[ ]
{ }
0 | 0
1 1 1
0
1
i i
ASA E Wait P Wait E Wait Wait
P Wait
N
µ ρ
= > >
= >
(2)
A third important performance metric for call center managers is the Telephone Service Factor (TSF),
also called the service level.” The TSF is the fraction of calls presented which are eventually serviced
and for which the delay is below a specified level. For example, a call center may report the TSF as the
percent of callers on hold less than 30 seconds. The TSF metric can then be expressed as
{
}
{
}
{
}
( )
1
1 0 | 0
1 ( , )
i i
N T
i
TSF P Wait T P Wait P Wait T Wait
C N R e
µ ρ
= > > >
=
(3)
A fourth performance metric monitored by call center managers is the Abandonment Rate; the propor-
tion of all calls that leave the queue (hang up) prior to service. Abandonment rates cannot be estimated
directly using the Erlang C model because the model assumes no abandonment occurs.
A substantial amount of research analyzes the behavior of Erlang C model, much of it seeks to estab-
lish simple staffing heuristics based on asymptotic frameworks applied to large call centers. (Halfin and
Whitt 1981) develop a formal version of the square root staffing principle for M/M/N queues in what has
become known as the Quality and Efficiency Driven (QED) regime. (Borst, Mandelbaum et al. 2004) de-
velop a framework for asymptotic optimization of a large call center with no abandonment.
As is the case with any analytical model, the Erlang C model makes many assumptions, several of
which are not wholly accurate. In the case of the Erlang C model several assumptions are questionable,
but clearly the most problematic is the no abandonment assumption, as even low levels of abandonment
can dramatically impact system performance (Gans, Koole et al. 2003). Many call center research papers
however analyze call center characteristics under the assumption of no abandonment (Jennings and Man-
delbaum 1996; Green, Kolesar et al. 2001; Green, Kolesar et al. 2003; Borst, Mandelbaum et al. 2004;
Wallace and Whitt 2005; Gans and Zhou 2007).
The Erlang C model assumes also that calls arrive according to a Poisson process. The interarrival
time is a random variable drawn from an exponential distribution with a known arrival rate. Several au-
thors assert that the assumption of a known arrival rate is problematic. Both major call center reviews
(Gans, Koole et al. 2003; Aksin, Armony et al. 2007) have sections devoted to arrival rate uncertainty.
(Brown, Gans et al. 2005) perform a detailed empirical analysis of call center data. While they find that a
time-inhomogeneous Poisson process fits their data, they also find that arrival rate is difficult to predict
and suggest that the arrival rate should be modeled as a stochastic process. Many authors argue that call
center arrivals follow a doubly stochastic process, a Poisson process where the arrival rate is itself a ran-
dom variable (Chen and Henderson 2001; Whitt 2006; Aksin, Armony et al. 2007). Arrival rate uncer-
tainty may exist for multiple reasons. Arrivals may exhibit randomness greater than that predicted by the
Poisson process due to unobserved variables such as the weather or advertising. Call center managers at-
tempt to account for these factors when they develop forecasts, yet forecasts may be subject to significant
error. (Robbins 2007) compares four months of week-day forecasts to actual call volume for 11 call cen-
ter projects. He finds that the average forecast error exceeds 10% for 8 of 11 projects, and 25% for 4 of
11 projects. The standard deviation of the daily forecast to actual ratio exceeds 10% for all 11 projects.
(Steckley, Henderson et al. 2009) compare forecasted and actual volumes for nine weeks of data taken
from four call centers. They show that the forecasting errors are large and modeling arrivals as a Poisson
process with the forecasted call volume as the arrival rate can introduce significant error. (Robbins, Me-
deiros et al. 2006) use simulation analysis to evaluate the impact of forecast error on performance meas-
ures demonstrating the significant impact forecast error can have on system performance.
Some recent papers address staffing requirements when arrival rates are uncertain. (Bassamboo, Har-
rison et al. 2005) develop a model that attempts to minimize the cost of staffing plus an imputed cost for
2855
Robbins, Medeiros and Harrison
customer abandonment for a call center with multiple customer and server types when arrival rates are va-
riable and uncertain. (Harrison and Zeevi 2005) use a fluid approximation to solve the sizing problem for
call centers with multiple call types, multiple agent types, and uncertain arrivals. (Whitt 2006) allows for
arrival rate uncertainty as well as uncertain staffing, i.e. absenteeism, when calculating staffing require-
ments. (Steckley, Henderson et al. 2004) examine the type of performance measures to use when staffing
under arrival rate uncertainty. (Robbins and Harrison 2010)develop a scheduling algorithm using a sto-
chastic programming model that is based on uncertain arrival rate forecasts.
The Erlang C model also assumes that the service time follows an exponential distribution. The me-
moryless property of the exponential distribution greatly simplifies the calculations required to character-
ize the system’s performance, and makes possible the relatively simple equations (1)-(3). If the assump-
tion of exponentially distributed talk time is relaxed, the resulting queuing model is the
/ /
M G N
queue,
which is analytically intractable (Gans, Koole et al. 2003) and approximations are required. However,
empirical analysis suggests that the exponential distribution is a relatively poor fit for service times. Most
detailed analysis of service time distributions find that the lognormal distribution is a better fit (Mandel-
baum, Sakov et al. 2001; Gans, Koole et al. 2003; Brown, Gans et al. 2005).
Finally, the Erlang C model assumes that agents are homogeneous. More precisely, it is assumed that
the service times follow the same statistical distribution independent of the specific agent handling the
call. Empirical evidence supports the notion that some agents are more efficient than others and the dis-
tribution of call time is dependent on the agent to whom the call is routed. In particular more experienced
agents typically handle calls faster than newly trained agents (Armony and Ward 2008). (Robbins 2007)
demonstrated a statistically significant learning curve effect in an IT help desk environment.
3 CALL CENTER SIMULATION
3.1 The Modified Model
In this section we present a revised model of a call center, relaxing key assumptions discussed previously.
In our model calls arrive at the call center according to a Poisson process. Calls are forecasted to arrive at
an average rate of
ˆ
λ
. The realized arrival rate is
λ
, where
λ
is a normally distributed random variable
with mean
ˆ
λ
, standard deviation
λ
σ
and coefficient of variation
ˆ
c
λ λ
σ λ
= . The choice of the normal dis-
tribution gives us a symmetric distribution centered on the forecasted value. A disadvantage of the nor-
mal distribution is the possibility of generating negative values. However, in our experiments the mean
value is sufficiently positive, a minimum of 5 standard deviations, that this is not a concern. The time re-
quired to process a call by an average agent is a lognormally distributed random variable with
mean
1
and standard deviation
µ
σ
. Arriving calls are routed to the agent who has been idle for the long-
est time if one is available. If all agents are busy the call is place in a FCFS queue. When placed in
queue a proportion of callers will balk; i.e. immediately hang up. Callers who join the queue have a pa-
tience time that follows a Weibull distribution. If wait time exceeds their patience time the caller will ab-
andon. Calls are serviced by agents who have variable relative productivity
i
r
. Agent productivity is as-
sumed to be a normally distributed random variable with a mean of 1 and a standard deviation of
r
σ
. An
agent with a relative productivity level of 1, for example, serves calls at the average rate. An agent with
a relative productivity level of 1.5 serves calls at 1.5 times the average rate, an agent with a productivity
level of .75 serves calls at .75 times the average rate. Given the mean productivity level of 1, on average
calls are served at the rate
λ
.
3.2 Experimental Design
In order to evaluate the performance of the Erlang C against the simulation model we conduct a series of
designed experiments. Based on the assumptions for our call center discussed previously, we define the
following set of nine experimental factors.
2856
Robbins, Medeiros and Harrison
Table 1: Experimental Factors
Factor Low High
1 Number of Agents 10 100
2
Offered Utilization (
ˆ
ρ
)
65% 95%
3 Talk Time (mins) 2 20
4
Patience
β
60 600
5
Forecast Error CV (
c
λ
)
0 .2
6
Patience
α
.75 1.25
7 Talk time CV .75 1.25
8 Probability of Balking 0 .25
9 Agent Productivity Standard Deviation 0 .15
The forecasted arrival rate in the simulation is a quantity derived from other experimental factors by
ˆ
ˆ
N
λ ρ µ
= (4)
Given the relatively large number of experimental factors, a well designed experimental approach is re-
quired to efficiently evaluate the experimental region. A standard approach to designing computer simu-
lation experiments is to employ either a full or fractional factorial design (Law 2007). However, the fac-
torial model only evaluates corner points of the experimental region and implicitly assumes that responses
are linear in the design space. Given the anticipated non-linear relationship of errors we chose to imple-
ment a Space Filling Design based on Latin Hypercube Sampling (LHS) as discussed in (Santner, Wil-
liams et al. 2003). Given a desired sample of n points, the experimental region is divided into n
d
cells. A
sample of n cells is selected in such a way that the centers of these cells are uniformly spread when pro-
jected onto each axis of the design space. While the LHS design is not perfectly orthogonal like a factori-
al design, the design does provide for a low correlation between input factors greatly reducing the risk of
multicollinearity. We chose our design point as the center of each selected cell.
3.3 Simulation Model
The model is evaluated using a straightforward discrete event simulation model. The purpose of the
model is to predict the long term, steady state behavior of the queuing system. The model generates ran-
dom numbers using the a combined multiple recursive generator (CMRG) based on the Mrg32k3a genera-
tor described in (L'Ecuyer 1999). Common random numbers are used across design points to reduce out-
put variance. To reduce any start up bias we use a warm up period of 5,000 calls, after which all statistics
are reset. The model is then run for an evaluation period of 25,000 calls and summary statistics are col-
lected. For each design point we repeat this process for 500 replications and report the average value
across replications.
The specific process for each replication is as follows. The input factors are chosen based on the ex-
perimental design. The average arrival rate is calculated based on the specified talk time, number of
agents, and offered utilization rate according to equation (4). A random number is drawn and the rea-
lized arrival rate is set based on the probability distribution of the forecast error. That arrival rate is then
used to generate Poisson arrivals for the replication. Agent productivities are generated using a normal
distribution with mean one and standard deviation
p
σ
. Each new call generated includes an exponential-
ly distributed interarrival time, a lognormally distributed average talk time, a Weibull distributed time be-
fore abandonment, and a Bernoulli distributed balking indicator. When the call arrives it is assigned to
the longest idle agent, or placed in the queue if all agents are busy. If sent to the queue the simulation
model checks the balking indicator. If the call has been identified as a balker it is immediately aban-
doned, if not an abandonment event is scheduled based on the realized time to abandon. Once the call has
been assigned to an agent, the realized talk time is calculated by multiplying the average talk time and the
agent’s productivity. The agent is committed for the realized talk time. When the call completes the
2857
Robbins, Medeiros and Harrison
agent processes the next call from the queue, or if no calls are queued becomes idle. If a call is processed
prior to its time to abandon, the abandonment event is cancelled. If not, the call is abandoned and re-
moved from the queue when the patience time expires.
After all replications of the design point have been executed the results are compared to the theoreti-
cal predictions of the Erlang C model. We calculate the error as the difference between the theoretical
value and the simulated value. We make a relative error calculation so that the sign of the error indicates
the bias in the calculation. In our experiment we evaluated an LHS sample of 1,000 points.
4 EXPERIMENTAL ANALYSIS
4.1 Summary Observations
Based on our analysis we can make the following summary observations:
The Erlang C model is, on average, subject to a reasonably large error over this range of parameter
values.
Measurement errors are highly positively correlated across performance measures.
The Erlang C model is on average pessimistically biased (the real system performs better than pre-
dicted) but may become optimistically biased when utilization is high and arrival rates are uncertain.
Measurement error is high when the real system exhibits higher levels of abandonment. The error is
strongly positively correlated with realized abandonment rate and predicted ASA.
The Erlang C model is most accurate when the number of agents is large and utilization is low.
Errors decrease as caller patience increases.
We will now review our experimental results in more detail.
4.2 Correlation and Magnitude of Errors
The magnitude of errors generated by using the Erlang C model across our test space is high on average,
and very high in some cases. The errors across the key metrics are highly correlated with each other, and
highly correlated with the realized abandonment rate. Table 2 shows a correlation matrix of the errors
generated from the Erlang C model.
Table 2: Error Correlation Matrix
Simulated
Abandonment
Rate
Prob Wait
Error
ASA
Error
TSF
Error
Utilization
Error
Simulated Abandonment Rate 1.000
Prob Wait Error .867 1.000
ASA Error .766 .722 1.000
TSF Error -.880 -.987 -.759 1.000
Utilization Error .970 .861 .745 -.873 1.000
Correlations between measure errors are strong. The measured errors all move, on average, in an
optimistic or pessimistic direction together. ProbWait and ASA are positively correlated; it is desirable
for both these measures to be low. ProbWait is negatively correlated with TSF; a measure for which a
high value is desirable. Measurement error is also highly correlated with abandonment rate. Given the
high correlation between measures we will utilize ProbWait as a proxy for the overall error of the Erlang
C model.
Average error rates are reasonably high under the Erlang C model, with errors being pessimistically
skewed. Figure 1 shows a histogram of the ProbWait error.
2858
Robbins, Medeiros and Harrison
0
5
10
15
20
25
30
Percent
Error (Theoretical - Simulation)
Prob Wait Error
Figure 1: Histogram of Erlang C Prob Wait Errors
The average error is 7.96%, and the data has a strong positive skew; 72% of the errors being positive.
The largest error is 49.4%, the smallest is -8.0 %.
4.3 Drivers of Erlang C Error
Having established that error rates are high under the Erlang C model, we now turn out attention to cha-
racterizing the drivers of that error. As discussed in the previous section, Erlang C errors are highly cor-
related with the realized abandonment rate. The notion that abandonment is a major driver of errors in the
Erlang C model is further illustrated in Figure 2. This graph shows the error in the ProbWait measure on
the vertical axis and the abandonment rate from the simulation analysis on the horizontal axis.
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
Error in Probability of Wait Estimate
(Theoretical - Simulation)
Abandonment Rate from Simulation
Error in Probability of Wait Calculations vs. Abandonment
Figure 2: Scatter Plot of Erlang C Errors and Abandonment Rate
2859
Robbins, Medeiros and Harrison
The graph clearly shows that as abandonment increases, the error in the ProbWait measure increases as
well. The graph also reveals that optimistic errors, i.e. errors in which the system performed worse than
predicted, only occur with relatively low abandonment rates. The average abandonment rate for optimis-
tic predictions was .74%. The graph also reveals that significant error can be associated with even low to
moderate abandonment rates. For example, for all test points with abandonment rates of less than 5%, the
average error for ProbWait is 4.8%. For test points in which abandonment ranged between 2% and 5%
the average ProbWait error is 12.2%.
To assess how each of the nine experimental factors impacts the error, we perform a regression analy-
sis. The dependent variable is the ProbWait error. For the independent variable we use the nine experi-
mental factors normalized to a [-1,1] scale. This normalization allows us to better assess the relative im-
pact of each factor. The LHS sampling method provides an experimental design where the correlation
between experimental factors is low, greatly reducing risks of multicollinearity. The results of the regres-
sion analysis are shown in Table 3.
Table 3: Regression Analysis of ProbWait Error
Regression Analysis
0.746
Adjusted R² 0.744 n 1000
R 0.864 k 9
Std. Error 0.058 Dep. Var.
Prob Wait Error
ANOVA table
Source SS df MS F p-value
Regression 9.6689 9 1.0743 323.87 8.38E-288
Residual 3.2839 990 0.0033
Total 12.9529 999
Regression output confidence interval
variables coefficients std. error t (df=990) p-value 95% lower 95% upper
Intercept 0.0797 0.0018 43.745 1.52E-233 0.0761 0.0832
Num Agents -0.0721 0.0032 -22.778 1.11E-92 -0.0783 -0.0658
Utilization Target 0.1500 0.0032 47.365 1.09E-256 0.1438 0.1562
Talk Time 0.0184 0.0032 5.829 7.53E-09 0.0122 0.0246
Patience -0.0134 0.0032 -4.233 2.52E-05 -0.0196 -0.0072
AR CV -0.0260 0.0032 -8.206 7.05E-16 -0.0322 -0.0198
Talk Time CV -0.0035 0.0032 -1.096 .2734 -0.0097 0.0027
Patience Shape -0.0027 0.0032 -0.858 .3912 -0.0089 0.0035
Probability of Balking 0.0228 0.0032 7.172 1.44E-12 0.0165 0.0290
Agent Heterogeneity 0.0050 0.0032 1.585 .1133 -0.0012 0.0112
Given the normalization of the experimental factors, the magnitude of the regression coefficients pro-
vides a direct assessment of the impact that a factor has on the measurement error. The factor that most
strongly influences the error is the offered utilization, the magnitude of its coefficient being more than
twice the value of the next measure and more than five times the magnitude of all other factors. The size
of the call center, measured as the number of agents, has a major impact on errors. Factors related to wil-
lingness to wait, i.e. Patience, Patience Shape, and Probability of Balking, all have low to moderate im-
pacts, but with exception of Patience Shape are statistically significant. Talk time is also a statistically
significant factor with a moderate impact. The variability of talk time and agent heterogeneity both have
low impacts that are not statistically significant.
2860
Robbins, Medeiros and Harrison
The most important drivers of Erlang C errors are the size and utilization of the call center. This is
further illustrated in Figure 3. This graph shows the results of an experiment where the number of agents
and utilization factors are varied in a controlled fashion. All other experimental factors are held at their
mid-point.
-10%
0%
10%
20%
30%
40%
50%
10 15 20 2 5 30 35 40 45 50 55 60 65 70 75 8 0 85 9 0 95 100
Error in Erlang C Probability of Wait Calculation
(Theoretical-Simulated)
Number of Agents
Prob of Wait Error by Number of Agents and Offered Utilization
0.65
0.75
0.85
0.95
Figure 3: Erlang C ProbWait Errors by Call Center Size and Utilization
This graph demonstrates that the Erlang C model tends to provide relatively poor predictions for
small call centers. This error tends to decrease as the size of the call center increases. However, the
graph also illustrates that for busy centers the error remains high. For a very busy call center, running at
95% offered utilization, the error rate remains at 30%, even with a pool of 100 agents. The errors tend to
track with abandonment; abandonment rates increase with utilization and decrease with the agent pool.
The conclusion that abandonment behavior drives the Erlang C error is further illustrated in Figure 4.
In this experiment we systematically vary two Willingness to Wait parameters. Specifically, we vary the
balking probability and the β factor of patience distribution.
-0.02
-0.01
0
0.01
0.02
0.03
0.04
60 90 120 150 180 210 2 40 270 300 3 30 36 0 390 420 450 480 510 540 570 600
Error in Erlang C Probability of Wait Calculation
(Theoretical-Simulated)
Patience (β
ββ
β)
Prob of Wait Error by Patience β
ββ
β and Balking Rate
0
0.125
0.25
Figure 4: Erlang C ProbWait Errors by Willingness to Wait
2861
Robbins, Medeiros and Harrison
This analysis verifies that the more likely callers are to balk, the higher the error rate. The analysis al-
so shows that when callers are more patient, the error rates decrease. The more likely callers are to aban-
don, either immediately or soon after being queued, the higher the abandonment rate and the less accurate
the Erlang C measures become.
An additional factor of interest is the uncertainty associated with the arrival rate. While it’s overall
effect is not large, about 1.8%, it has effects that are dissimilar to other experimental factors as illustrated
in Figure 5. This graph shows the results of an experiment that varies the coefficient of variation of the
arrival rate error and the number of agents while holding all other factors at their mid-points.
-10%
-5%
0%
5%
10%
15%
20%
10 15 20 ' 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Error in Erlang C Probability of Wait Calculation
(Theoretical-Simulated)
Number of Agents
Prob of Wait Error by Number of Agents and Arrival Rate Uncertainty
0
0.1
0.2
Figure 5: Erlang C ProbWait Errors by Call Center Size and Forecast Error
This experiment shows that for small call centers arrival rate uncertainty has a small effect, but that
effect becomes more pronounced for larger call centers. It is also worth noting that arrival rate uncertain-
ty has an optimistic effect, and for high levels of uncertainty the model exhibits an optimistic bias. Arriv-
al rate uncertainty is a major factor leading to an optimistic estimate from the Erlang C model; of the
21.9% of test points with an optimistic bias the average arrival rate uncertainty cv was 14.0%. Since ar-
rival rate uncertainty tends to bias the prediction in the opposite direction of most other factors, it also has
the effect of reducing error in many situations. For example, high utilization tends to bias the estimate
pessimistically, a bias reduced when arrival rate uncertainty is present.
5 SUMMARY AND CONCLUSIONS
The Erlang C model is commonly applied to predict queuing system behavior in call center applica-
tions. Our analysis shows that when we test the Erlang C model over a range of reasonable conditions
predicted performance measures are subject to large errors. The Erlang C model works reasonably well
for large call centers with low to moderate utilization rates, but factors that tend to generate caller aban-
donment; such as high utilization, small agent pools, and impatient callers, cause the model error to be-
come quite large. While the model tends to provide a pessimistic estimate, arrival rate uncertainty will
either reduce that bias or lead to a optimistic bias. It may be the case that the model’s tendency to provide
pessimistic (i.e. conservative) estimates helps explain its continued popularity. It is clear that great care
must be taken before using the Erlang C model to make any calculations that require a high level of preci-
sion.
2862
Robbins, Medeiros and Harrison
Our future research is focused on analyzing the increasingly popular Erlang A model and comparing
it’s performance to the Erlang C model to test the growing consensus that Erlang A is a superior model
for call center analysis.
REFERENCES
Aksin, Z., M. Armony and V. Mehrotra. 2007. The Modern Call-Center: A Multi-Disciplinary Perspec-
tive on Operations Management Research. Production and Operations Management 16: 665-668.
Armony, M. and A. R. Ward 2008. Fair Dynamic Routing in Large-Scale Heterogeneous-Server Systems,
Stern School of Business, NYU.
Bassamboo, A., J. M. Harrison and A. Zeevi. 2005. Design and Control of a Large Call Center: Asymp-
totic Analysis of an LP-based Method. Operations Research 54: 419-435.
Borst, S., A. Mandelbaum and M. I. Reiman. 2004. Dimensioning Large Call Centers. Operations Re-
search 52: 17-35.
Brown, L., N. Gans, A. Mandelbaum, et al. 2005. Statistical Analysis of a Telephone Call Center: A
Queueing-Science Perspective. Journal of the American Statistical Association 100: 36-50.
Chen, B. P. K. and S. G. Henderson. 2001. Two Issues in Setting Call Centre Staffing Levels. Annals of
Operations Research 108: 175-192.
Gans, N., G. Koole and A. Mandelbaum. 2003. Telephone call centers: Tutorial, review, and research
prospects. Manufacturing & Service Operations Management 5: 79-141.
Gans, N. and Y.-P. Zhou. 2007. Call-Routing Schemes for Call-Center Outsourcing. Manufacturing &
Service Operations Management 9: 33-51.
Green, L. V., P. Kolesar and J. Soares. 2003. An Improved Heuristic for Staffing Telephone Call Centers
with Limited Operating Hours. Production and Operations Management 12: 46-61.
Green, L. V., P. J. Kolesar and J. Soares. 2001. Improving the SIPP Approach for Staffing Service Sys-
tems That Have Cyclic Demands. Operations Research 49: 549-564.
Halfin, S. and W. Whitt. 1981. Heavy-Traffic Limits for Queues with Many Exponential Servers. Opera-
tions Research 29: 567-588.
Harrison, J. M. and A. Zeevi. 2005. A Method for Staffing Large Call Centers Based on Stochastic Fluid
Models. Manufacturing & Service Operations Management 7: 20-36.
Jennings, O. B. and A. Mandelbaum. 1996. Server staffing to meet time-varying demand. Management
Science 42: 1383.
L'Ecuyer, P. 1999. Good Parameters and Implementations for Combined Multiple Recursive Random
Number Generators. Operations Research 47: 159-164.
Law, A. M. 2007. Simulation modeling and analysis. Boston, McGraw-Hill.
Mandelbaum, A., A. Sakov and S. Zeltyn 2001. Empirical Analysis of a Call Center, Technion - Israel In-
stitute of Technology.
Robbins, T. R. 2007. Managing Service Capacity Under Uncertainty - Unpublished PhD Dissertation
Pennsylvania State University. University Park, PA. Avaialble via
(http://personal.ecu.edu/robbinst/) [accessed August 31, 2010]
Robbins, T. R. and T. P. Harrison. 2010. Call Center Scheduling with Uncertain Arrivals and Global Ser-
vice Level Agreements. European Journal of Operational Research Forthcoming.
Robbins, T. R., D. J. Medeiros and P. Dum 2006. Evaluating Arrival Rate Uncertainty in Call Centers.
2006 Winter Simulation Conference, Monterey, CA.
Santner, T. J., B. J. Williams and W. Notz 2003. The design and analysis of computer experiments. New
York, Springer.
Steckley, S. G., S. G. Henderson and V. Mehrotra. 2009. Forecast Errors in Service Systems. Probability
in the Engineering and Informational Sciences: 305-332.
Steckley, S. G., W. B. Henderson and V. Mehrotra 2004. Service System Planning in the Presence of a
Random Arrival Rate, Cornell University.
2863
Robbins, Medeiros and Harrison
Wallace, R. B. and W. Whitt. 2005. A Staffing Algorithm for Call Centers with Skill-Based Routing.
Manufacturing & Service Operations Management 7: 276-294.
Whitt, W. 2006. Staffing a Call Center with Uncertain Arrival Rate and Absenteeism. Production and
Operations Management 15: 88-102.
AUTHOR BIOGRAPHIES
THOMAS R. ROBBINS is an Assistant Professor in the department of Marketing and Supply Chain at
East Carolina University. He holds a PhD in Business Administration and Operations Research from
Penn State University, an MBA from Case Western Reserve and a BSEE from Penn State. Prior to be-
ginning his academic career he worked in professional services for approximately 18 years. His email
address is <robbinst@ecu.edu>.
D. J. MEDEIROS is Associate Professor of Industrial Engineering at Penn State University. She holds a
Ph.D. and M.S.I.E from Purdue University and a B.S.I.E. from the University of Massachusetts at Am-
herst. She has served as track coordinator, Proceedings Editor, and Program Chair for WSC. Her re-
search interests include manufacturing systems control and CAD/CAM. She is a member of IIE and
SME. Her email address is <djm3@psu.edu>.
TERRY P. HARRISON is the Earl P. Strong Executive Education Professor of Business and Professor
of Supply Chain and Information Systems at Penn State University. He holds a Ph.D. and M.S. degree in
Management Science from the University of Tennessee and a B.S in Forest Science from Penn State.
He was formerly the Editor-in-Chief of Interfaces and is currently Vice President of Publications for
INFORMS. His mail address is <tharrison@psu.edu>.
2864
... The model assumes that requests added to the queue stay there until they are serviced, and the queue is infinite. For better illustration of Erlang C model, see Fig. 4 [9]. To calculate the number of agents, it is necessary first to calculate the operation load ...
... The fourth and last parameter of a call centre is the number of requests that abandon the queue before being provided service. However, employing Erlang C model, it is impossible to asses this parameter, since it assumes that all requests remain in the queue until they are serviced [9]. ...
... However, agents represent the biggest expense of a call centre (60-75 %). For that reason, the excessive number of agents is not economically favourable [9]. ...
Article
Full-text available
To detect the number of agents needed to serve customers, it is necessary to consider the call centre as a mass service system. Then it is possible to asses the convenient number of agents according to the probability of the system receiving a request and the time in which the request is serviced by employing a Markov chain and the Erlang model. In an archetypal call centre, the incoming calls are added to a waiting queue and subsequently they are assisted by an agent. In case all agents are occupied, the customer has to wait in the queue until one of the agents becomes available. It is, therefore, important to compromise on the number of agents and the time the customers spend waiting in the queue. The result should be that there are enough agents in the call centre to serve the customers in the time required. This article focuses on solving this problem.
... In addition to the Erlang C formula, Erlang A and B can also be beneficial in this regard (11). However, each formula has its pros and cons; for instance, some researchers argue that Erlang C provides a pessimistic estimate and may cause extreme phone traffic congestion during peak hours (12). ...
Article
Full-text available
Introduction Currently, at Tehran Emergency Medical Service (EMS) centre, Emergency Medical Dispatchers (EMDs) are scheduled based on the managers’ experimental estimates. In this study, we planned to evaluate the conformity of managers’ predictions with the Erlang C formula estimates in scheduling EMDs. Methods First, the Emergency Medical Communication Centre (EMCC) performance was evaluated over one week. Afterwards, the number of required EMDs was calculated using the Erlang C formula. Finally, the predictions of the Erlang C formula were compared with those of managers’ judgments. Results During the study period, 79,583 calls were received by the Tehran EMCC. The average number of EMDs per hour ranged between 9.5 and 22.7. The actual number of EMDs was more than Erlang C formula predictions during the 24 hours in all but three time points, i.e. 14:00-1 4:59, 1 5:00-1 5:59 and 1 8:00-1 8:59. In all hours, 90% of calls were answered in less than 10 seconds, and the average waiting time for a total of one week was 7.3 seconds. Also, only 2.1 % of all calls were answered after 10 seconds. Conclusion In the current study, we found that the number of EMDs scheduled based on the managers’ experimental estimates was higher than that of the Erlang C formula calculations. Also, it was found that the waiting time for emergency calls was lower than the defined standards. Although the primary results of the current study indicated that, at least on paper, the Erlang C formula has the potential to be used as a predicting model in the Tehran EMCC, further research is required to evaluate its effect on the actual performance of the EMCCs.
... These services are often modeled as systems of rows, M/M/s model in standard terminology in theory of queues -the model of Erlang C. The model of Erlang C assumes that calls arrive with an average arrival rate known, with a number of agents defined statistically identical and with service time following an exponential distribution. This model also requires all customers to wait as long as necessary to receive the service without disconnect the call [12]. The Erlang model is well known in engineering, and it is used for many other applications, such as referred in [13,14]. ...
Article
Full-text available
Various companies choose to outsource the delivery of part of their services, so as not to deviate from its core business and improve the service level. This approach leads to a new type of organizations, so-called networked and virtual enterprises, where possibly a great number of companies work together without having direct contact but through a broker, as an intermediary, that streamlines the relationships between them. To enable high level efficiency, as well as some other functional requirements, the meta-organizations and brokering services are conceived as environments and services for networked and virtual enterprises operation and dynamic reconfigurations, representing a model of organizations-of-organizations, as an implementation of one of the Industry 4.0 models and ecosystem for networked and virtual enterprises dynamic reconfiguration. In this paper, the meta-organizations with embedded brokering services, modelled as call centres, are analyzed. Various simulations are presented, based on Erlang's formulas for some of design and performance measures parameters evaluation, such as service level, average waiting time, agent occupancy and service traffic intensity.
... As mentioned above, NHPP models fail to exhibit overdispersion, a phenomenon observed across various types of service systems; see e.g., [6,20,23,31,33]. The parameter uncertainty underlying overdispersion potentially jeopardizes the effectiveness of the square-root rule, typically leading to overoptimistic staffing algorithms. ...
Preprint
Arrival processes to service systems often display (i) larger than anticipated fluctuations, (ii) a time-varying rate, and (iii) temporal correlation. Motivated by this, we introduce a specific non-homogeneous Poisson process that incorporates these three features. The resulting arrival process is fed into an infinite-server system, which is then used as a proxy for its many-server counterpart. This leads to a staffing rule based on the square-root staffing principle that acknowledges the three features. After a slight rearrangement of servers over the time slots, we succeed to stabilize system performance even under highly varying and strongly correlated conditions. We fit the arrival stream model to real data from an emergency department and demonstrate (by simulation) the performance of the novel staffing rule.
... Our assumptions conform to the Erlang C queuing model staffing (see Robbins, Medeiros, and Harrison, 2010). For a given composite arrival rate of patients λ, an average length of stay on a ventilator μ, and a number of ventilators N, define the offered load as R = λ /μ and the traffic intensity as ρ = R/N. ...
... 1) the inter-arrival process of the call was assumed to be a Poisson process, implying that users are considered to be homogeneous [25] [16], [26]; 2) service times are independent, identically distributed and follow mainly a log-normal distribution [25] and in some cases an exponential distribution [27]; 3) agents are assumed to be homogeneous and the routing of call is quite simple given that all the resources are identical. In other words, it is assumed that there are many potential callers who are identical and that the probability that they will call at any minute is low. ...
Article
Emergency call centers (ECCs) are upstream of the prehospital emergency medical system and the life of many people depends on their effectiveness and responsiveness. This notwithstanding, the way their operations are organized and managed differs from one place to another. Also, depending on the number of incoming calls and available resources, they can operate differently. In the face of these heterogeneous situations, some ECCs do not always meet the expected performance levels: people still wait for too long before their call is answered. Moreover, they may have difficulties in managing an important upsurge of calls, especially in periods of crisis. Therefore, to support ECCs’ organizational improvement steps, this article aims to develop a tool-based framework that would enable to make clear and objective diagnoses, especially as regards responsiveness. Our proposal allows considering both nominal (normal days) and exceptional (crisis days) demands. It is based on data science, process mining, and discrete event simulation tools. By experimenting it on a French real case, the results show that such a tool-based framework can be very valuable for improving the performance of ECC organizational setups in both normal and disrupted situations.
... One reason, revealed in the statistical analysis of Robbins et al. (2010), is that the Erlang-C formula gives a pessimistic evaluation of call center performance and therefore results in safe managerial decisions. Thus, our findings on the case without abandonment can be used to take routing decisions when abandonment is difficult to predict. ...
Article
Full-text available
1) Problem definition: We consider a revenue-generating call center with inbound and outbound calls, where service and sales activities are blended. For maximizing the call center's revenue, the call center manager exercises two levels of control; agent reservation for inbound calls and call outsourcing. Given the influence of waits on purchase probability, we investigate the strategy of outsourcing customers who have waited already, as opposed to outsourcing customers directly at arrival. (2) Academic / Practical relevance: The main novelty of this article arises from the use of a single framework to investigate combining agent reservation with outsourcing decisions, and a waiting time-based outsourcing strategy. The existing literature only considers these two strategies in isolation and is restricted to quantity-based decisions. From a practical viewpoint, our results aim to provide decision support tools that are directly implementable in a call center's routing software. (3) Methodology : We apply a Markov decision process approach to optimize the manager's decisions. The particularity of our approach is that we use the experienced waiting time as a decision variable. (4) Results: We prove that the optimal policy for reservation and outsourcing is of threshold type. Our main conclusion is that outsourcing customers after letting them wait in-house generates higher revenue than outsourcing calls at arrival. However, it is also detrimental to service quality. In addition, we identify contexts where the difference between the two outsourcing strategies is significant. (5) Managerial implications: Contrary to standard call center practices, which either consist of specialized teams for one type of call, or only exercising one specific level of decision-making (reservation or outsourcing), we demonstrate the potential of partial outsourcing with partial reservation. Our study shows that small congested call centers are those where the benefits of implementing our results are the greatest.
Article
Arrival processes to service systems often display (i) larger than anticipated fluctuations, (ii) a time-varying rate, and (iii) temporal correlation. Motivated by this, we introduce a specific non-homogeneous Poisson process that incorporates these three features. The objective is to develop a staffing rule for a many-server system facing such an arrival process. So as to obtain approximations for its performance, we first consider the situation of the arrival process being fed into the corresponding infinite-server system. Using the square-root staffing principle leads to a staffing rule that acknowledges the three features. After a slight rearrangement of servers over the time slots, we succeed to stabilize system performance even under highly varying and strongly correlated conditions. We fit the arrival stream model to real data from an emergency department and demonstrate (by simulation) the performance of the novel staffing rule.
Chapter
In this study, customer service activities of the company, which is actively serving in the e-commerce sector in Turkey, provided through the call center were reviewed and a solution was proposed to improve the existing problems. In this respect, the literature research has been conducted on e-commerce sector and call centres, increasing the customer demands with the campaigns implemented by the companies serving in the e-commerce sector in certain periods, and thus the effect of the intensity experienced on customer services in call centres has been examined from different perspectives. Later on, with a company that provides international services, including Turkey it has reached to more detailed information and data relating to jointly carrying out a study subject. Surveys were carried out with the planning team and team leaders of the company, moreover, current problems were investigated and suggestions for solutions were evaluated. The data obtained, the previous year’s statistics of the company were examined, and the needs were determined. In order to develop solutions, different concepts have been created and evaluated along with one of the concepts created in line with these results has been selected. The design of the study was prepared according to the results obtained from these stages. Finally, risk analyses were made, the points that should be prevented by the company were reached, and ideas were presented for the future stages.
Article
Full-text available
Combining parallel multiple recursive sequences provides an efficient way of implementing random number generators with long periods and good structural properties. Such generators are statistically more robust than simple linear congruential generators that fit into a computer word. We made extensive computer searches for good parameter sets, with respect to the spectral test, for combined multiple recursive generators of different sizes. We also compare different implementations and give a specific code in C that is faster than previous implementations of similar generators.
Conference Paper
Full-text available
Inbound call center operations are challenging to manage; there is considerable uncertainty in estimates of arrival rates, and the operation is often subject to strict service level constraints. This paper is motivated by work with a provider of outsourced technical support services in which most projects (client specific support operations) include an inbound tier one help desk subject to a monthly service level agreement (SLA). Support services are highly specialized and a significant training investment is required, an investment that is not transferable to other projects. We investigate the option of cross training a subset of agents so that they may serve calls from two separate projects, a process we refer to as partial pooling. Our paper seeks to quantity the benefits of partial pooling and characterize the conditions under which pooling is most beneficial. We find that low levels of cross training yield significant benefit.
Conference Paper
Full-text available
Inbound call center operations are challenging to manage, in large part because there is considerable uncertainty in estimates of arrival rates, which vary over time. We have developed a general purpose simulation model for inbound call center operations which supports time varying and uncertain arrival rates along with variable staffing. We outline the conceptual and technical design of the simulation model. We then define and conduct an initial experiment that uses the model to evaluate the impact of arrival rate uncertainty on call center performance. We find that arrival rate uncertainty creates significant planning challenges for managers attempting to satisfy tight performance targets, particularly one-sided performance measures. We also find that abandonment rate has a major impact on call center performance
Article
Full-text available
Many telephone call centers that experience cyclic and random customer demand adjust their staffing over the day in an attempt to provide a consistent target level of customer service. The standard and widely used staffing method, which we call the stationary independent period by period (SIPP) approach, divides the workday into planning periods and uses a series of stationary independent Erlang-c queuing models—one for each planning period—to estimate minimum staffing needs. Our research evaluates and improves upon this commonly used heuristic for those telephone call centers with limited hours of operation during the workday. We show that the SIPP approach often suggests staffing that is substantially too low to achieve the targeted customer service levels (probability of customer delay) during critical periods. The major reasons for SIPP‘ s shortfall are as follows: (1) SIPP's failure to account for the time lag between the peak in customer demand and when system congestion actually peaks; and (2) SIPP’ s use of the planning period average arrival rate, thereby assuming that the arrival rate is constant during the period. We identify specific domains for which SIPP tends to suggest inadequate staffing. Based on an analysis of the factors that influence the magnitude of the lag in infinite server systems that start empty and idle, we propose and test two simple “lagged” SIPP modifications that, in most situations, consistently achieve the service target with only modest increases in staffing.
Article
Full-text available
A call center is a service network in which agents provide telephone-based services. Customers who seek these services are delayed in tele-queues. This article summarizes an analysis of a unique record of call center operations. The data comprise a complete operational history of a small banking call center, call by call, over a full year. Taking the perspective of queueing theory, we decompose the service process into three fundamental components: arrivals, customer patience, and service durations. Each component involves different basic mathematical structures and requires a different style of statistical analysis. Some of the key empirical results are sketched, along with descriptions of the varied techniques required. Several statistical techniques are developed for analysis of the basic components. One of these techniques is a test that a point process is a Poisson process. Another involves estimation of the mean function in a nonparametric regression with lognormal errors. A new graphical technique is introduced for nonparametric hazard rate estimation with censored data. Models are developed and implemented for forecasting of Poisson arrival rates. Finally, the article surveys how the characteristics deduced from the statistical analyses form the building blocks for theoretically interesting and practically useful mathematical models for call center operations.
Article
Full-text available
This paper proposes simple methods for sta-ng a single-class call center with uncertain arrival rate and uncertain sta-ng due to employee absenteeism. The arrival rate and the proportion of servers present are treated as random variables. The basic model is a multi-server queue with customer abandonment, allowing non-exponential service-time and time-to-abandon dis- tributions. The goal is to maximize the expected net return, given throughput beneflt and server, customer-abandonment and customer-waiting costs, but attention is also given to the standard deviation of the return. The approach is to approximate the performance and the net return, conditional on the random model-parameter vector, and then uncondition to get the desired results. Two recently-developed approximations are used for the conditional perfor- mance measures: flrst, a deterministic ∞uid approximation and, second, a numerical algorithm based on a purely Markovian birth-and-death model, having state-dependent death rates.
Article
We develop a framework for asymptotic optimization of a queueing system. The motivation is the staffing problem of large call centers, which we have modeled as M/M/N queues with N, the number of agents, being large. Within our framework, we determine the asymptotically optimal staffing level N* that trades off agents’ costs with service quality: the higher the latter, the more expensive is the former. As an alternative to this optimization, we also develop a constraint satisfaction approach where one chooses the least N* that adheres to a given constraint on waiting cost. Either way, the analysis gives rise to three regimes of operation: quality-driven, where the focus is on service quality; efficiency-driven, which emphasizes agents’ costs; and a rationalized regime that balances, and in fact unifies, the other two. Numerical experiments reveal remarkable accuracy of our asymptotic approximations: over a wide range of parameters, from the very small to the extremely large, N* is exactly optimal, or it is accurate to within a single agent. We demonstrate the utility of our approach by revisiting the square-root safety staffing principle, which is a long-existing rule of thumb for staffing the M/M/N queue. In its simplest form, our rule is as follows: if c is the hourly cost of an agent, and a is the hourly cost of customers’ delay, then N*=R+y*(a/c)R, where R is the offered load, and y*(·) is a function that is easily computable.
Book
Mainly deal with queueing models, but give the properties of many useful statistical distributions and algorithms for generating them.
Article
This paper evaluates the practice of determining staffing requirements in service systems with random cyclic demands by using a series of stationary queueing models. We consider Markovian models with sinusoidal arrival rates and use numerical methods to show that the commonly used "stationary independent period by period" (SIPP) approach to setting staffing requirements is inaccurate for parameter values corresponding to many real situations. Specifically, using the SIPP approach can result in staffing levels that do not meet specified period by period probability of delay targets during a significant fraction of the cycle. We determine the manner in which the various system parameters affect SIPP reliability and identify domains for which SIPP will be accurate. After exploring several alternatives, we propose two simple modifications of SIPP that will produce reliable staffing levels in models whose parameters span a broad range of practical situations. Our conclusions from the sinusoidal model are tested against some empirical data.