Content uploaded by Stefan Kemmler
Author content
All content in this area was uploaded by Stefan Kemmler on Oct 04, 2018
Content may be subject to copyright.
Planning of reliability life tests within the accuracy,
time and cost triangle
Keywords: end of life test, test planning, simulation, test cost, test time
Martin Dazer
University of Stuttgart
Institute of Machine
Components
Matthias Stohrer
University of Stuttgart
Institute of Machine
Components
Stefan Kemmler
University of Stuttgart
Institute of Machine
Components
Bernd Bertsche
University of Stuttgart
Institute of Machine
Components
If the focus during product development is on estimating a
failure distribution function or a lifetime-quantile, analysis of
failure behavior or determination of optimal design alternatives,
there is often no alternative to life tests until failure. Alongside the
determination of sample size for a desired accuracy of the result,
engineers are also faced with the challenge of selecting the most
suitable life test type at limited cost, time and test rig capacities. In
this paper, results of a comprehensive simulation study are
presented. Sudden Death tests, type-I and type-II censored and
non-censored life test types are compared for two-parameter
Weibull distributed data with respect to accuracy, time, cost and the
investigation target. Significant influencing factors and boundary
conditions, such as the maximum number of test rigs available, are
taken into account to assure the practical applicability of the
procedure for planning life tests. The database generated provides
recommendations for an appropriate test configuration within an
accuracy, time and cost triangle, including optimal test parameters.
Furthermore, it enables engineers to substantiate qualitatively
known correlations with numbers. This information can be used to
describe the influence of a shift in boundary conditions. Finally,
the use of the simulation results as basis for planning life tests is
illustrated in a case study.
I. INTRODUCTION
In industrial application, life tests are commonly used
methods to verify the functionality of a product or component
over a defined lifetime. According to [1] there are several
reasons to execute a life test during early stages in product
development.
Comparison of different components and materials
Identification of alternative designs
Verification of the efficacy of design changes
Determination of the stress-strength-interference
Referring to these objectives, the product failure behavior
has to be determined with a Weibull analysis. Results of these
tests can also serve as a basis for the modification or release of
a product as well as the verification of conformity to product
liability requirements. In order to achieve the objectives
mentioned above, an appropriate life test planning should be
carried out. Due to various boundary conditions and many
possible test types, it is a challenge to determine the optimal
test configuration suitable for a specific application. The
behavior of the population can only be estimated by statistical
analysis of the lifetime data from selected samples. Defining
larger sample sizes allows for a more accurate description of
the population but also increases the amount of resources
needed, inevitably resulting in conflict with the time and cost
efficiency objective. Furthermore, it is usually not possible to
resort to an unlimited amount of testing resources, because
they are often limited, for example by the existing test rigs or
test samples. Based on this relationship the accuracy, time and
cost triangle can be illustrated. The triangle demonstrates the
inevitable trade-off between the accuracy of the estimated
lifetime and the incurred cost and time, as shown in Fig. 1.
„MAGIC
TRIANGLE“
Sample size
Accuracy
Time
Test types
Parameters of
failure distribution
Parameter estimation
method
Properties of
product
Number
of test rigs Costs
Fig. 1. Accuracy, cost and time triangle
The lifetime of a product is always a random variable,
making the prediction of lifetime depended variables, as test
time and test costs, difficult to handle. Therefore, an economic
consideration is typically not associated with the statistical
accuracy of a life test, although it would be highly relevant for
practical application. Different test types, varying sample sizes
and use of censored data result in a large amount of possible
combinations to execute a life test, which makes it even more
complicated to select an appropriate test configuration.
There are many works in reliability research, in which the
issue of finding the optimal test plan is limited to only one
specific test type. A comparison between different test types in
order to provide recommendations for varying boundary
conditions is missing. In reality, an initial situation predefines
the boundary conditions for reliability life test planning.
Influencing factors include the investigation target, product
properties (dominant failure mode, lifetime distribution,
sample availability, sample costs), available test infrastructure,
a fixed test budget as well as contractual schedules that must be
considered.
The objective of this study is to develop a concept for
reliability life test planning to provide an optimal test
configuration for a wide range of possible initial situations. The
output variables, such as appropriate test type, sample size,
statistical accuracy, test costs and time are adapted to the
respective initial situation, as shown in Fig. 2.
Reliability verification via fixed-time tests without failure
(Success-Run) is very common due to its good predictability in
terms of cost and time at a given reliability target and
confidence level. Certainly, the determination of parameters of
a failure distribution function is not possible with a Success-
Run test due to a lack of failure times. The concept presented
in this study has been developed in order to facilitate the
planning of life tests until failure with regard to statistical
accuracy, cost and time and thus shows an alternative with
higher information content.
II. STATE OF THE ART
Numerous results for statistical life test planning of Weibull
distribution are available in the statistical and engineering
literature. However, a holistic view of planning life tests
considering economic and temporal aspects besides statistical
accuracy is seldom found. Below, some approaches for test
planning are presented and evaluated regarding the relationship
between accuracy, test costs and time.
There are different approaches to optimizing the sample
size concerning a specific accuracy criterion such as the
variance for a specified quantile. Reference [8] describes the
asymptotic theory and application for planning a life test to
estimate a Weibull quantile with specified precision. Reference
[9] presents general theory and application for sample size
determinations in life test planning when other functions of
Weibull parameters are to be estimated. Life test planning for
estimating the hazard rate of a Weibull distribution with a
given shape parameter is described in [10]. Reference [11]
presents asymptotic theory and methods for planning a life test
to estimate a Weibull hazard function, when all parameters are
unknown.
Initial situation with
boundary conditions
Concept for
reliability
lifetime test
planning
Product
properties
Test budget
Available
time
Available test
infrastructure
€
Optimal test configuration
Sample size
Test costs
Accuracy
Test type
Input
Test time
Investigation
target
?
€
1
32
4
Output
Fig. 2. Input and output variables for reliability test planning concept
In reference [4] the accuracy of a non-censored life test is
defined by the confidence interval width. For the reliability at a
fixed lifetime value, a ratio between the upper and lower
confidence interval is defined. The smaller the ratio of the
confidence bounds, the more accurate the reliability prediction
of this lifetime value. Using simulation a sample size is
determined that achieves the confidence interval width for a
specified confidence level.
Type-II censored life tests are discussed in [3, 5, 7]. The
average test time of type-II censored tests is studied in [5].
Herein expected values of non-censored and type-II censored
tests are calculated and compared using a binomial distribution.
The benefits in test time are presented in dependence of
Weibull shape parameter, overall samples size and censored
amount. Reference [3] develops tables for type-II censored test
plans, allowing the selection of the most cost-effective
combination of sample size and censoring criteria. Given tables
are valid for items whose failure behavior follow either an
extreme value or a Weibull distribution. The range of sample
size n is limited from 3 to 18 samples, while only samples sizes
with at least 3 failures are evaluated. The achieved precision of
type-II censored life tests using Bayesian methods is presented
in [7]. Prior distributions and criteria are used to estimate a
quantile of interest. The estimation is rated by characteristics of
a credibility interval. Finally a relationship between the needed
number of failures and the precision criteria for type-II
censored test data is developed.
A procedure to find a Sudden Death test configuration that
estimates the Mean Time to Failure (MTTF) of a product most
efficiently using Monte Carlo Simulation (MCS) and
predetermined criteria is described in [2]. To evaluate the
accuracy, the confidence interval for the MTTF is used. After
each simulation of a test configuration, it is decided if the
inspection lot is accepted. The testing time is estimated using
an assumed probability function. Moreover, it is used for the
calculation of the costs alongside the sample size. This
approach takes into account both the accuracy as well as test
costs and time. For the latter, solely an average value is
calculated based on shape and scale parameter together with
sample size and the number of inspection lots. The variance of
lifetime and thus variance of test costs and test time are
neglected with the MTTF being the sole investigation target.
Estimations for other lifetime quantiles are not possible.
Furthermore, the variety of test types is limited in this
approach. Only Sudden Death tests are considered. Reference
[6] covers a similar topic. Emphasis is on test plans for type-I
censored data, in which testing of samples is carried out in
intervals. A cost function is set up, depending on the test
intervals, representing the time aspect of a test.
Most of the studies set their focus on one test type and try
to optimize input parameters and a specific investigation target,
like Sudden Death tests and MTTF in [2]. The variance of test
time and cost based on the dependence of the lifetimes are
mostly neglected. From an economic perspective, there is
always a trade-off in the "magic triangle", a compromise
between quality or testing accuracy, cost and time. Therefore,
the authors express the opinion that it should be possible to
design a test plan with respect to all three aspects.
III. APPROACH
The focus of the approach for optimal planning of life tests
is on the comparison of non-censored, type-I- and type-II-
censored and Sudden Death tests, based on statistical accuracy,
cost and time. The quantification of these aspects is carried out
using statistical analysis of test data, which are generated via
MCS. Each life test can be performed with different test types,
resulting in a wide variety of possible test configurations,
which are integrated using Design of Experiments (DOE).
A. Boundary conditions
To maintain clarity for the comparison, some conditions
and restrictions are introduced, as seen below.
1) General boundary conditions
The scope of this study is limited to the objectives of a
life test presented in chapter I according to [1], a precise
estimation of the Weibull parameters b and T. Life tests
without failures such as Success-Run tests are not
considered since they do not enable the estimation of
failure distribution parameters.
Only the two-parameter Weibull distribution is
considered.
It is assumed that the tests are conducted on field load
level. The concept allows for test acceleration via
increased usage rates through increasing speed or
reducing off times. However, test acceleration via
overstressing using life-stress models is not part of the
study. If an established life-stress model for a product
exists, the concept can be used nevertheless.
The basis of this study is the assumption that at least
estimates on the value of the shape parameter b and the
characteristic lifetime T from previous products, field
data analysis and/or durability analysis are available.
The Weibull shape parameter b usually depends on the
failure mode of a product. Different types of failure are
characterized by different shape parameters and
therefore have to be considered separately Weibull
analysis. In this study it is assumed that all failures are
due to the same failure mode and thus can be analyzed
together.
2) Test related boundary conditions
All available test rigs are used for testing. Resource
planning for other simultaneous tests is not integrated.
An optimal availability of all test rigs is assumed.
Failure times are determined immediately. No delay due
to inspection intervals is assumed. Furthermore,
mounting and set-up time are neglected for the
calculation of the total test time in case of multiple
occupancy of a test rig. Thus, the test of the next sample
begins right after the failure of the previous one. These
aspects are accepted for simplification of the
simulation.
A type-II censored life test can only be performed if the
total number of samples does not exceed the number of
test rigs.
To restrict the high number of possible combinations
for Sudden Death tests all available test rigs are used for
one inspection lot until the first sample fails.
B. Evaluation criteria
To evaluate the test types, appropriate criteria need to be
developed, allowing a convincing comparison.
1) Accuracy of estimation
Accuracy means the objective of estimating the distribution
parameters b and T as precisely as possible using a life test.
The variance in the estimate of b and T, caused by differing
failure times when repeating a test, is determined using MCS.
Random numbers are drawn from a predefined Weibull
distribution, representing the true failure distribution of the
product. The objective is to determine how precisely the true
distribution parameters can be estimated with an individual test
configuration, consisting of sample size and a test type.
Therefor two approaches are presented.
1. Statistical evaluation of two quantile estimations.
2. Introduction of the Probability of Success as new
single evaluation criterion for the accuracy.
In order to describe the accuracy of the overall distribution
estimation, the accuracy of the estimation of at least two
lifetime quantiles is considered. In this work, the B10 and the
B90 lifetimes are used for the first approach, as mentioned
above. The choice of B10 and B90 covers a wide area of the
distribution function. Depending on other interests, the
consideration of other lifetime quantiles is also possible. The
procedure shown in Fig. 3 is applied and repeated L times.
The procedure results in two distributions, consisting of L
estimates for B10 and B90. Two features of these distributions
are used to describe the accuracy of the test. First, the median
of B10 and B90 can be compared with the values of the true
distribution to check for deviation. Secondly, the variance
sheds light on the accuracy of the estimation. It is described by
the relative quantile range ∆QR, relating the difference between
the 95 % and the 5 % quantile to the median of the distribution,
as shown in (1).
5010
050100.9510
10 .,
., , B
BB
)ΔQR(B
(1)
By referring to the median, the variance of different input
parameter levels, e.g. variance for different shape parameters,
is more comparable. By using ∆QR, it is possible to describe
the variance independently of the distribution function. This is
desirable as censored data or a small sample sizes can
influence the shape of the distribution function and prevent the
use of a common distribution, e.g. a normal distribution, for the
description of the variance.
An advantage of statistical analysis of lifetime quantiles is
the adaptability to the investigation target. Accuracy of the
estimation of a complete lifetime distribution can be evaluated
by analyzing a low and high quantile. In addition, a single low
quantile can be evaluated if the investigation target is finding a
life test type for verification of a low lifetime quantile. A
disadvantage of this approach is the number of resulting
variables. When examining the relative quantile range and the
median of two lifetime quantiles, the comparison of different
test types is based on four results.
To overcome this disadvantage the Probability of Success
Ps is introduced as a new single result variable. A life test
configuration that enables the determination of the parameters
b and T with a deviation within a given tolerance range, is
referred to from now on as successful. The Probability of
Success indicated the probability for a correct estimation of b
and T based on a permitted tolerance range. To determine the
Probability of Success, an assessment of the estimated
parameters based on the simulated failure times is performed.
If the estimated parameters are within the given tolerance
range, success is indicated (= 1), otherwise failure (= 0). The
Probability of Success equals the percentage of successful
estimations for a given test configuration. The benefit of
defining a Probability of Success is the clarity in the
presentation of results. Only one resulting variable has to be
regarded while the tolerance range TR is introduced as a new
input variable for the permitted variance of the estimates of b
and T. To carry out the assessment of the estimated Weibull
parameters, which is shown in Fig. 4, a tolerance range TR for
b and T is defined. It determines the permitted variation of the
estimated parameters. According to (2) and (3) Tmin, and bmin
are the smallest accepted parameters, while bmax and Tmax
represent the maximums. To decrease the number of input
parameters the same TR is specified for b and T.
Default values
btrue, Ttrue, n
n random failure times
Parameter estimation
!
?
bi, Ti
B10,i , B90 ,i
Repetition
B10/B90-Distribution
L times
MCS
test type
Non-censored /
type-I censored /
type-II censored /
Sudden Death
median quantile
range
B10 / B90
f(B10) /
f(B90)
Q0.95
Q0.05 Q0.50
Calculation
Fig. 3. Statistical evaluation of lifetime quantiles
bmax = btrue + TR%
bmin = btrue - TR%Tmax = Ttrue + TR%
Tmin = Ttrue - TR%
Default values
btrue, Ttrue, n
n random failure times
Parameter estimation
!
?
bi, Ti
Repetition
Probability of Success
L times
MCS
test type
Non-censored /
type-I censored /
type-II censored /
Sudden Death
%
Outside TRInside TR
Tolerance range check
Fig. 4. Statistical evaluation of Probability of Success
TRbbbTRbbb truetruemaxtruetruemin ;
(2)
TRTTTTRTTT truetruemaxtruetruemin ;
(3)
TR can be illustrated similarly to the confidence interval for a
linearized Weibull cumulative density function in a Weibull
plot. The variation of T allows a shift, while the variation of b
permits a change of slope. These two variations span a pseudo
confidence interval, which is, in contrary to the conventional
confidence interval, independent of the number of samples or
the censoring criteria. It therefore provides an appropriate way
to compare different test configurations. TR is a relative
variable and is given in percent.
2) Test costs
To quantify test costs, a cost function is set up. The total
cost Ctotal consist of the sample costs CS, and the runtime costs
CR.
RStotal CCC
(4)
The sample costs arise from the procurement of the
samples. They depend solely on the sample size n and the costs
for a single sample cs, they are calculated according to (4).
ncC sS
(5)
In order to reduce the number of different input parameters
in this study it is assumed that the sample costs contain all
additional time-independent costs, such as mounting costs and
operational costs. The runtime costs CR are time-dependent and
correlate with the time of a life test. They contain energy costs
for operating the test rig, personnel costs, rental cost for the use
of test systems etc. The runtime costs depend on the failure
time ti and the runtime cost factor cr, expressing the costs per
time unit.
n
iictC 1rR
(6)
For the appropriate description of the time domain different
units can be used, such as load cycles, activation cycles, hours,
months, years, etc. In order to unify the description of failure
time and therefore the runtime costs, a standardization method
is introduced that expresses failure time as a function of
characteristic lifetime T.
)(Tft
(7)
For ease of use the factor kr is introduced, setting up a ratio
of runtime costs factor and sample costs. Through the use of
this factor runtime costs can be expressed in terms of sample
costs.
T
k
c
c1
r
s
r
(8)
Finally (4) is rearranged with (5), (6) and (8).
n
ii
tckcnC 1
srstotal
(9)
The statistical evaluation of the cost is analogous to the
evaluation of lifetime quantiles. The result is a cost distribution
function, which is represented through the median Ctotal,med and
the quantile range ∆QR(Ctotal) as a measure of variance.
3) Test time
For the consideration of test time, the time of the complete
life test is used. This is the time span from the start of the first
sample test, neglecting the set-up time, up to the failure of the
last sample or the reaching of the censoring criteria. Keeping
the characteristic lifetime T constant for all test configurations,
the total test time depends on the shape parameter b, the
sample size n and the number of test rigs available nR. If the
number of samples is greater than the number of available test
rigs, n > nR, the test must be carried out sequentially in
multiple inspection lots. The number of inspection lots tested
consecutively is declared with nA. To ensure the usage of all
test rigs, it is assumed that the sample size is always a multiple
of the number of test rigs.
RA nnn
(10)
For determining the overall test time of non-censored or
type-I censored life tests, the test rig with the longest operating
time is used.
A type-II censored test is considered reasonable only for
nA = 1 and n ≤ nR. In this case, the total time of a life test is
equal to the failure time of the r-th sample, provided that r is
the failure based censoring criteria. For Sudden Death tests
each inspection lot is tested until the first failure. Total test
time is calculated using the sum of all first failures times of nA
tests. Statistical evaluation is analogous to the cost evaluation
with ttotal, med and ∆QR(ttotal).
C. Design of Simulation Experiment using MCS
Now the procedures presented in Fig. 3 and Fig. 4 are used
to determine median and variance of accuracy, cost and time
for a specified combination of boundary conditions, test type
and sample size. To compare all different combinations, a
Design of Experiments (DOE) is set up. The objective is a
holistic understanding of the correlation between input and
output variables in a predefined parameter space.
A full factorial DOE which contains all possible
combinations of boundary conditions, test type and sample size
is set up. Fig. 5 lists a full factorial DOE for three parameters
with three parameter levels respectively. For each combination,
the procedure illustrated in Fig. 3 and Fig. 4 is repeated
L=10.000 times which corresponds to 10.000 virtual life tests
for each combination. Fig. 6 lists all seven input parameters of
the DOE and their respective predefined discrete levels. The
number of levels is chosen in order to achieve an appropriate
compromise between a holistic coverage of the parameter
space and a total number of parameter combinations that can
be handled in terms of data storage and computation time.
number of test rigs nR
level 1 2 3
shape parameter b
level
123
123
MCS with
10.000 loops
Fig. 5. Full factorial DOE with exemplary 3 input variables
The censoring criteria for type-I censored life tests is stated
in terms of characteristic lifetime T. The censoring time is
calculated as follows:
TZt type1c
(11)
A corresponding censoring criterion for type-II censored
tests describes the number of samples r tested to failure
depending on the samples size n.
nZr type2
(12)
The sample size n and the failed samples r are always
rounded up, in case they are not a natural number.
bTR kRnRnAZtype1 Ztype2
1
2
3
4
5
0.2
0.4
0.1
1
10
1
4
6
9
13
18
24
40
1
3
5
9
14
20
27
50
0.5
0.75
1
1.5
0.2
0.4
0.6
0.8
5 2 3 8 8 4 4
No. of levels
Parameter
values
Parameter
Total: 30.720 different combinations
Fig. 6. Chart of the parameters with the respective levels
After the input parameters for the DOE are defined, the
random failure times can be drawn for each combination. To
ensure the comparability of test types, the same set of random
failure times for a designated sample size and fixed boundary
conditions is used for the evaluation of all test types. The shape
parameter and the characteristic lifetime are estimated for each
test type using Median Rank Regression and the Maximum
Likelihood Method [12]. Estimations are only made if at least
three failure times are generated. If a parameter estimation is
not possible due to extreme censoring or an insufficient
number of failures or test rigs, the inspection lot is discarded.
IV. RESULTS
The outcome of the simulation study is a large database. It
contains the output parameter values in the time, cost and
accuracy domain, which are the evaluation criteria described in
paragraph III. For all 30,270 input parameter combinations
2,211,840 resulting values are calculated, Table I. The
simulation results confirm that an appropriate test
configuration highly depends on the initial situation with its
boundary conditions. Based on the simulated data, no specific
test configuration can be determined that generally yields better
results in all three areas of interest (cost, time and accuracy)
than other test configurations. Instead, a case-dependent
individual test planning is necessary. The extensive database
provides for a quantitative decision basis. Therefore, to
illustrate the general applicability of the simulation results for
the derivation of an optimum test configuration a fictional
example is discussed in a case study.
V. CASE STUDY
In this case, it is assumed that the design must be changed
during the development of a product. In order to assess the
effectiveness of the design change concerning reliability and to
confirm the dominant failure mode, life tests are to be carried
out. From experience with the old design a Weibull shape
parameter of b = 3 is assumed and the characteristic lifetime is
estimated to be at least T = 100 h. There are n = 12 samples
and nR = 4 test rigs available for a life test. The costs for a
single sample amount to cs = 100 € while the runtime cost
factor is cr = 1 €/h. The runtime cost for testing a characteristic
lifetime therefore amount to 100 h x 1 € / h = 100 €, resulting
in a cost ratio of kr = 1. The parameters of the failure
distribution should only be roughly determined or confirmed. It
is therefore sufficient to determine the distribution parameters
up to 40 % deviation (TR = 0.4). According to the initial
situation and the target of the life test described, the simulated
data basis is filtered. The boundary conditions lead to a great
reduction of possible test configurations. Furthermore, total
costs and total test time are calculated according to the given
factor using the introduced formulas. Since only four test rigs
are available, parallel testing of all 12 samples is not an option.
Hence, no type-II censored tests are considered in the
following test planning process. The boundary conditions are
listed in the upper section of Table II. All remaining test
configurations are listed below the boundary conditions. Since
for type-I censored tests different censoring criteria are
conceivable, four type-I censored tests with different censoring
criteria Ztype1 are listed. To allow a better comparison, the
properties of the test types in the areas of cost, time and
accuracy are colored. A red-colored cell represents a poor
value for the property in question, whereas a green cell marks a
good value.
In a first test-planning step, all possible test configurations
are compared. As expected, a non-censored life test yields the
highest Probability of Success, which indicates the highest
accuracy for the estimation of the distribution parameters.
However, it also represents the most cost- and time-intensive
test type. The median value for test time is twice as high as a
Sudden Death test. A high censoring time (Ztype1 = 1.5) results
in only minor changes compared to a non-censored test. The
lower the censoring time, the lower are resulting total test costs
and total test time. Expectably, the accuracy in parameter
estimation drops as well. A test with a type-I censoring time of
half the characteristic lifetime yields in only four percent of all
cases a parameter estimation with less than 40 % deviation
from the default values. A Sudden Death test turns out to have
competitive results in terms of median test time and median
test costs when disregarding the variance of results. An
inhomogeneous distribution of more and less durable samples
across different inspection lots leads to large variation of the
TABLE I. STRUCTURE OF THE RESULTING DATABASE
type-II
censored
test
Sudden-
Death-
Test
b
nA
nR
kr
TR
Ztype1
Ztype2
Ctotal, 0.0 5
Ctotal, med
Ctotal, 0.9 5
ΔQR(Ctotal )
ttotal, 0.0 5
ttotal, med
ttotal, 0.9 5
ΔQR(ttotal )
PS
B10, m ed
ΔQR(B10)
B90, m ed
ΔQR(B90)
PS
B10, m ed
ΔQR(B10)
B90, m ed
ΔQR(B90)
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
1 1 1 1 0.1 0.2 0.5 0.2 … … … … … … … … … … … … … … … … … …
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
… … … … … … … … … … … … … … … … … … … … … … … … … …
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
30720 550 40 10 0.4 1.5 0.8 … … … … … … … … … … … … … … … … … …
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
…
accuracy
non-censored life test
Output parameters (results)
Parameter combination no.
Input parameters
costs
dur.
MLE
MRR
MRR
costs
duration
MLE
accuracy
…
…
type-I censored test
2,211,840 res ulting values
times of first failure. However, the times of first failure of all
inspection lots have great influence on the total test time and
costs.
The large time and cost variation in Sudden Death tests
therefore presents a risk when planning life tests. Furthermore,
the high variation of the estimated B10 lifetime leads to the
conclusion that a Sudden Death test is not a wise choice for the
verification of lifetime quantiles.
After the general comparison of all five remaining test
configurations, in a second step further restrictions are made to
derive a test configuration optimal for the current product
development phase. A Probability of Success of 70 % is
defined as a minimum, to ensure that the test results likely
meet the expectations. Due to the project schedule, the test
results should be available as soon as possible and not exceed a
maximum test time of 300 hours. Under these enhanced
conditions, the cheapest test configuration is to be determined.
As Table II reveals, a non-censored life test achieves the
required Probability of Success. Yet, its median total test time
exceeds the restriction by 24 hours. A Sudden Death test
provides results in nearly half of the allotted time, but achieves
a Probability of Success of only 44 %, which is not an option.
Only a type-I censored test with a censoring time of one
characteristic lifetime reaches a Probability of Success of 73 %
and a median test time of 282 h. Thus, the type-I censored test
with Ztype1 = 1 is the only test configuration that fulfills all
criteria. Even in worst case, i.e. if no failures occur before
reaching censoring time, the maximum test time of
nA x Ztype1 = 3T = 300 h does not exceed a specified total test
time of at most 300 h.
Besides the opportunity to plan life tests according to given
boundary conditions, the database created using the presented
simulation approach gives the option to examine the effects of
boundary condition changes. As an example the sample size
used in the case study is reduced from n = 12 to n* = 8. Thus,
for a non-censored life test only nA = 2 consecutively tested
inspection lots are needed. This leads to a reduction of the test
time down to ttotal,med = 239 h (median) and a Probability of
Success of PS = 78%. Due to the small sample size a parameter
estimation using median rank regression yields better results
than using Maximum Likelihood Estimation methods, which
estimate a Probability of Success of only PS = 74 %.
Furthermore, the median of total costs decreases from 2270 €
to 1520 €.
As in the test configuration with n = 12 samples, the
variation of total test time of a non-censored life test is high. In
5 % of all cases, it exceeds 315 hours and thus violates one
constraint. To counteract this, a type-I censored test is
scheduled, which has a high censoring time (Ztype1 = 1.5). This
results in a maximum test time of nA x Ztype1 x T = 3T = 300h.
Using regression for parameter estimation, the Probability of
Success is 76 % and therefore above the demand.
TABLE II. INPUT PARAMETERS AND RESULTS OF CASE STUDY
non-
censored
Sudden-
Death
- 0.5 0.75 1 1.50 -
Ctot al, 0.05 [€] 2090 1721 1927 2039 2093 1650
Ctot al, med [€] 2270 1759 2010 2173 2268 1873
Ctot al, 0.95 [€] 2460 1785 2066 2284 2443 2115
∆QR (Ctotal) [%] 16.3 3.6 6.9 11.3 15.4 24.8
ttot al, 0.05 [h] 263 149 206 243 263 112
ttot al,med [h] 324 150 225 282 322 168
ttot al, 0.95 [h] 395 150 225 300 384 228
∆QR (ttotal) [%] 40.7 0.8 8.4 20.2 37.6 69.0
PS [%] 84.0 3.7 50.0 73.0 83.5 44.0
B10,m ed [h] 49.6 36.1 47.6 48.8 49.4 50.1
ΔQR(B10) [%] 70.0 61.1 72.0 73.0 72.0 82.6
B90,m ed [h] 128.9 85.0 117.0 127.0 129.5 102.0
ΔQR(B90) [%] 60.0 154.0 111.0 67.0 37.1 125.0
test type
costs
duration
accuracy
cs [€]
cr [€/h]
Case study
B10 [h]
B90 [h]
47.20
132
b
T [h]
test configuration
3
100
12
4
1
0.4
100
1
type-I censored
Ztype
boundary conditions
n
nR
kr
TR
TABLE III. ALTERNATIVE TEST CONFIGURATION FOR CASE STUDY
12
type-I
censored
non-
censored
type-I
censored
1 - 1.5
Ctota l, 0.05 [€] 2039 1319 1328
Ctota l, med [€] 2173 1514 1511
Ctota l, 0.95 [€] 2284 1713 1694
∆QR (Ctota l) [%] 11.3 26.0 24.2
ttota l, 0.05 [h] 243 174 174
ttota l,med [h] 282 239 235
ttota l, 0.95 [h] 300 315 300
∆QR (ttota l) [%] 20.2 59.1 53.5
PS [%] 73.0 78 76
B10,me d [h] 48.8 53 52
ΔQR (B10) [%] 73.0 91 93
B90,me d [h] 127.0 126 126
ΔQR (B90) [%] 67.0 47 54
Case study with alternative bou ndary conditions
boundary conditions
b
3
T [h]
100
B10 [h]
47.20
B90 [h]
132
n
8
nR
4
kr
1
TR
0.4
cs [€]
100
1
test configuration
test type
accuracy
cr [€/h]
Ztype
costs
duration
Thus, the type-I censoring of a life test with n* = 8 samples is
the most suitable test configuration for the initial situation
described in this case study. Table III gives an overview of the
two alternative test configurations.
The case study shows a dependence of the test
configuration on the boundary conditions. Initially a type-I
censored life test with strong time censoring or a Sudden Death
test seems to be a suitable test type for a life test with the aim
of a quick and rough estimate of the parameters of a failure
distribution. Analysis of various test configurations, however,
shows that in this case a reduction of sample size in
combination with a non-censored or slightly time-censored life
test represents the most effective test configuration in terms of
test costs, test time and estimation accuracy.
VI. CONCLUSION
In times of increasing computing power and storage
capacity, simulation methods represent a powerful approach to
planning life tests. The approach presented offers the
possibility to simulate life test results in order to quantify
uncertainties induced by variation when planning real life tests
and to rely upon them as a basis for decisions. In a relatively
inexpensive and quick manner, an optimum test configuration
for a given initial situation with individual constraints can be
found. Moreover, the effect of uncertainty in the assumptions
made can be determined and quantified.
ACKNOWLEDGMENT
The authors thank Mr. Florian May for his active support in
the creation and implementation of the simulation study.
REFERENCES
[1] G. Yang, Life cycle reliability engineering. Hoboken, NJ: Wiley,
2007.
[2] I. Arizono, Y. Kawamura, and Y. Takemoto, “Reliability tests for
Weibull distribution with variational shape parameter based on sudden
death lifetime data,” European Journal of Operational Research, vol.
189, no. 2, pp. 570–574, 2008.
[3] K. W. Fertig and N. R. Mann, “Life-Test Sampling Plans for Two-
Parameter Weibull Populations,” Technometrics, vol. 22, no. 2, pp.
165–177, 1980.
[4] H. Guo, E. Pohl, and A. Gerokostopoulos, “Determining the right
sample size for your test: theory and application,” in 2014 Proc. Ann.
Reliability and Maintainability Symp. (RAMS): IEEE, 2014.
[5] H. K. Hsieh, “Average type-II censoring times for the 2-parameter
Weibull distribution,” IEEE Trans. Rel, vol. 43, no. 1, pp. 91–96,
1994.
[6] Syuan-Rong Huang and Shuo-Jye Wu, “Reliability Sampling Plans
Under Progressive Type-I Interval Censoring Using Cost Functions,”
IEEE Trans. Rel, vol. 57, no. 3, pp. 445–451, 2008.
[7] Y. Zhang and W. Q. Meeker, “Bayesian life test planning for the
Weibull distribution with given shape parameter,” Metrika, vol. 61,
no. 3, pp. 237–249, 2005.
[8] W. Q. Meeker and W. Nelson, “Weibull Percentile Estimates and
Confidence Limits from Singly Censored Data by Maximum
Likelihood,” IEEE Trans. Rel, vol. R-25, no. 1, pp. 20–24, 1976.
[9] W. Q. Meeker and W. Nelson, “Weibull Variances and Confidence
Limits by Maximum Likelihood for Singly Censored Data,”
Technometrics, vol. 19, no. 4, pp. 473–476, 1977.
[10] L. Danziger, “Planning Censored Life Tests For Estimation of the
Hazard Rate of a Weibull Distribution With Prescribed Precision,”
Technometrics, vol. 12, no. 2, pp. 408–412, 1970.
[11] W. Q. Meeker, L. A. Escobar, and D. A. Hill, “Sample sizes for
estimating the Weibull hazard function from censored samples,” IEEE
Trans. Rel, vol. 41, no. 1, pp. 133–138, 1992.
[12] V. Blobel and E. Lohrmann, Statistische und numerische Methoden
der Datenanalyse, 2nd ed. Hamburg: V. Blobel, 2012.
BIOGRAPHY
Martin Dazer studied Mechanical Engineering at the University of Stuttgart in
Germany and received his academic degree Master of Science in 2014.
He is working as a research assistant in the field of reliability
engineering at the Institute of Machine Components. He is pursueing his
PhD studies with a focus on virtual life testing.
Matthias Stohrer studied Mechanical Engineering at the University of
Stuttgart in Germany and received his academic degree Dipl.-Ing. in
2009. He is working as a research assistant in the field of reliability
engineering at the Institute of Machine Components. He is pursueing his
PhD studies with a focus on planning reliability tests.
Stefan Kemmler studied Mechanical Engineering at the University of
Stuttgart in Germany and received his academic degree Dipl.-Ing. in
2012. He is working as a research assistant in the field of reliability
engineering at the Institute of Machine Components. He is pursueing his
PhD studies with a focus on robust reliability.
Bernd Bertsche is Professor of Mechanical Engineering as well as the Head of
the Institute of Machine Components at the University of Stuttgart and
Head of the VDI Advisory Board “Reliability Management” within the
group for product and process design. He is a member of the DIN / DKE
Standardization Committee K132 “Reliability”, the Senate of the
University of Stuttgart, the review board (assessor and strategic adviser)
of the German Research Foundation (DFG) and of the National
Academy of Science and Engineering (acatech). He is also managing
director of the “Scientific Society for Product Development” (WiGeP
e.V.
