Content uploaded by Suiyao Chen

Author content

All content in this area was uploaded by Suiyao Chen on Dec 04, 2017

Content may be subject to copyright.

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=lqen20

Download by: [47.199.250.226] Date: 30 June 2017, At: 19:10

Quality Engineering

ISSN: 0898-2112 (Print) 1532-4222 (Online) Journal homepage: http://www.tandfonline.com/loi/lqen20

Multi-state reliability demonstration tests

Suiyao Chen, Lu Lu & Mingyang Li

To cite this article: Suiyao Chen, Lu Lu & Mingyang Li (2017) Multi-state reliability demonstration

tests, Quality Engineering, 29:3, 431-445, DOI: 10.1080/08982112.2017.1314493

To link to this article: http://dx.doi.org/10.1080/08982112.2017.1314493

Accepted author version posted online: 05

Apr 2017.

Published online: 05 Apr 2017.

Submit your article to this journal

Article views: 61

View related articles

View Crossmark data

QUALITY ENGINEERING

, VOL. , NO. , –

https://doi.org/./..

Multi-state reliability demonstration tests

Suiyao Chena,LuLu

b, and Mingyang Lia

aDepartment of Industrial and Management Systems Engineering, University of South Florida, Tampa, Florida; bDepartment of Mathematics and

Statistics, University of South Florida, Tampa, Florida

KEYWORDS

binomial demonstration test;

consumer’s risk; minimum

sample size; Monte Carlo

integration

ABSTRACT

Reliability demonstration tests have important applications in reliability assurance activities to

demonstrate product quality over time and safeguard companies’market positions and competitive-

ness. With greatly increasing global market competition, conventional binomial reliability demonstra-

tion tests based on binary test outcomes (success or failure) at a single time point become insucient

for meeting consumers’diverse requirements. This article proposes multi-state reliability demonstra-

tion tests (MSRDTs) for demonstrating reliability at multiple time periods or involving multiple failure

modes. The design strategy for MSRDTs employs a Bayesian approach to allow incorporation of prior

knowledge, which has the potential to reduce the test sample size. Simultaneous demonstration of

multiple objectives can be achieved and critical requirements specied to avoid early/critical failures

can be explicitly demonstrated to ensure high customer satisfaction. Two case studies are explored to

demonstrate the proposed test plans for dierent objectives.

Introduction

Reliability of a product is the probability that the prod-

uct can perform its required function at a given time

point. As a time-dependent characteristic, reliability

is an important measure of the product quality and

safety over time, which has a great impact on the satis-

faction of customers and can inuence their purchase

decisions linked with the revenue of manufacturers. In

order to succeed in the market competition, manufac-

turers need to produce products with high reliability

over their expected lifetime. Reliability demonstration

tests (RDTs) are often conducted by manufacturers to

demonstrate the capability of their products to meet the

requirements from customers for achieving good qual-

ity and performance over time. Given the budget and

time constraints, manufacturers need to determine the

number of test units, the time duration of the test, and

the maximum number of failures allowed to pass the

test. These choices are usually made to ensure the con-

sumer’srisk(CR)onhavingaproductthathaspassed

thetestbutfailstomeetthereliabilityrequirementis

controlled. Controlling the CR at an acceptable level

can take the burden o the customers on bearing a high

risk of receiving inferior products which are claimed to

CONTACT Mingyang Li mingyangli@usf.edu Department of Industrial and Management Systems Engineering, University of South Florida, E. Fowler

Avenue, Tampa, FL .

have met the requirements on reliability, and hence can

help improve customers’ satisfaction.

Dierent categories of RDTs have been studied in

the literature based on dierent types of reliability

data, such as failure counts data (Guo et al., 2011;Li

et al., 2016;Luetal.,2016), failure time data (Guo

et al., 2012;McKaneetal.,2005) and degradation data

(Yang, 2009). Failure counts data report the number of

failures that occur during a xed test period. The RDTs

based on failure counts data (Wasserman, 2002,pp.

208–210) are also called binomial RDTs (BRDTs) since

failure counts are modeled with binomial distributions.

In a BRDT, within a given testing period, if the num-

beroffailuresdoesnotexceedthemaximumnum-

ber of allowable failures, the test is passed. The maxi-

mum number of allowable failures cand the minimum

number of test units nare determined to ensure a cer-

tain minimum acceptable reliability requirement, R,is

met with the controlled CR at or below βby the end

of the test duration. The BRDTs are broadly applied

in reliability engineering practices because (i) they

require less monitoring eorts in the middle of the test

duration; and (ii) they are simple and straightforward

to be implemented and analyzed. However, with the

© Taylor & Francis

432 S. CHEN ET AL.

increasing needs from customers, the BRDTs are no

longer able to meet all requirements in many applica-

tions. For example, customers may have varied require-

ments on reliability performance over dierent time

periods. It is common that many customers have little

tolerance of early failures and hence require high relia-

bility during early lifetime and lower reliability for later

time.Inthiscase,aBRDTfordemonstratingreliabil-

ity within a single time period is inadequate to meet all

requirements.

Considerascenariowhentwocompaniesrun

BRDTs with the same testing period of 5 years and use

the maximum number of allowable failures as c=5.

Products from company I had 1 failure in the rst two

years and 3 failures in the last three years. Products

from company II had 3 failures in the rst two years

and 1 failure in the last three years. Even though the

products from both companies can pass the demon-

stration tests, their underlying reliability performance

indicatedfromthefailurecountsdatacanbedierent.

If a customer needs products with high reliability in

early lifetime (corresponding to allowing no more than

2failuresduringthersttwoyears),theriskofthe

products from company II failing to meet the require-

ment can be much higher than that of the product

from company I. A typical BRDT with ve-year testing

period cannot demonstrate the performance over the

early two years, and hence raises the CR on accepting

aninferiorproductthatfailstomeetallrequirements.

Another limitation of the BRDTs is that they are

often used for pass/failure testing of a product without

distinguishing the causes and consequences of dier-

ent failure modes. A product with a complex system

is often composed of multiple key components which

may have dierent failure modes associated with var-

ied consequences. Their failures can have dierent neg-

ativeeectsonthefunctionalityoftheentireproduct.

For instance, the failure of the central processing unit

(CPU) of a computer is much more crucial than the

failureofavideocard.Customersmayalsohavedif-

ferent expectations for dierent components accord-

ing to their values or costs of replacement. The cost of

replacing a CPU or a motherboard is much higher than

replacing a keyboard or a mouse. As a result, customers

can have much higher expectation on the reliability of

themorevaluableandcriticalpartsthanthereliability

of other parts or accessories. A typical BRDT cannot

demonstrate separate reliability requirements for mul-

tiple failure modes.

To meet the ever-increasing demands of customers,

moreversatileRDTswithmoretailoredplansfortest-

ing multiple reliability requirements can better serve

the customers with enriched information on product

reliability. This article proposes RDT strategies for two

categories of reliability demonstration tests over multi-

ple time periods and for multiple failure modes, both

of which are referred to as multi-state RDTs (MSRDTs)

throughout the rest of the article. Alternative test plans

within each category are also explored and compared

with the conventional BRDTs for demonstrating mul-

tiple reliability requirements. Bayesian analysis is used

for quantifying the CR associated with various test

plans. The Bayesian method oers more exibilities on

incorporating prior information of product reliability

from either subject matter expertise or historical data

(Pintar et al., 2012;Weaveretal.,2008;Wilsonetal.,

2016). The impacts of dierent test strategies and dif-

ferent prior elicitations on the minimum test sample

size (i.e., the number of test units required) will be stud-

ied to provide more insights on guiding decisions on

demonstration test plans. If there exists historical data

which supports higher reliabilities compared to the

requirements, then using Bayesian method to incorpo-

rate prior information has the potential to reduce the

minimum test sample size required for the MSRDTs.

The remaining of the article is organized as fol-

lows. In the next section, the conventional BRDT plans

are reviewed with discussions of their benets and

limitations. Then the new MSRDTs for demonstrat-

ing reliability requirements over multiple time peri-

ods are proposed. Two dierent design strategies are

proposedandcomparedunderdierentpriorelicita-

tion settings. In the following section, another category

of new MSRDT designs for demonstrating reliability

requirements involving multiple failure modes are pro-

posed and their performances are evaluated and com-

paredwiththeconventionalBRDTs.Casestudieson

two categories of MSRDTs for multiple time periods

and multiple failure modes are provided to illustrate

the proposed test plans and demonstrate their perfor-

mances. Conclusions and discussions are provided in

the end.

Binomial RDTs

For many single use or “one-shot” product units, the

test procedure can be destructive. In this case, bino-

mial RDTs (BRDTs) are the common choices to obtain

QUALITY ENGINEERING 433

the failure counts data at the end of a predetermined

test period (Kececioglu et al., 2002, pp. 759–768). Let

πdenote the probability of failure over the test period,

and Rdenote the minimum acceptable reliability at the

end of the test duration. In Bayesian analysis, for a cho-

sen number of test units, n, and a maximum number of

allowable failures, c, the CR is measured by the poste-

rior probability of the product failing to meet the reli-

ability requirement given that the product has passed

the test, which can be calculated as

CRbinomial =P(Failure probability fails to meet the

reliability requirement|Test is passed)

=P(π > 1−R|y≤c)

=1−P(π ≤1−R|y≤c)

=1−1−R

0c

y=0n

yπy(1−π)

n−yp(π )dπ

1

0[c

y=0n

yπy(1−π)

n−y]p(π )dπ

.

[1]

Note that in Eq. [1], p(π ) denotes the prior distribu-

tion of πwhich can be specied based on subject mat-

ter expert knowledge or historical data and ydenotes

the number of failures observed in the test period. Let

βdenote the maximum acceptable value for the CR,

thenaBRDTisdeterminedbychoosingthe(n,c)com-

bination such that the corresponding CRbinomial ≤β.

According to (Lu et al., 2016), for any xed choice of c,

CRbinomial increases as the test sample size nincreases.

We use nbto denote the minimum test sample size that

is required to control the CR within an acceptable range

CRbinomial ≤β.

In Bayesian analysis, the CRbinomial in Eq. [1]canbe

calculated using Monte Carlo integration (Robert et al.,

2004, pp. 71–131), where samples of πwith a large

size M=15000 are generated from the specied prior

distribution p(π ),andCR

binomial is calculated approx-

imately by

CRbinomial ≈1−M

j=1c

y=0n

y(π(j))y(1−π(j))n−yI(π (j)≤1−R)

M

j=1c

y=0n

y(π(j))y(1−π(j))n−y,[2]

where π(j)is the jth generated sample of failure prob-

ability for the specied prior distribution.

Table 1 showsanexampleofBRDTplanswithdier-

ent choices of prior distributions of π. The mean and

standard deviation (i.e., the square of variance) values

are provided to give some intuitions about the center

Tab le . Minimum sample sizes required by BRDTs with diﬀerent

choices on cand prior distributions of π.

π∼Beta (1,1)(2,18)(4,16)(10,15)(10,10)

Mean(π) . . . . .

SD(π) . . . . .

cn

b

Settings: M=15000,R=0.8,β =0.05

and the spread of the prior distributions. For example,

π∼Beta(1,1)is centered at 0.5 but has large stan-

dard deviation at 0.2893. While π∼Beta(2,18)has

the mean failure probability of 0.1 but much smaller

standard deviation (0.0647) around its mean. The

minimum acceptable reliability from the consumers

requirement was set at R=0.8andthemaximumtol-

erable CR is chosen to be β=0.05. When no his-

torical data or prior information is available, a non-

informative prior π∼Beta(1,1)can be used. For any

assumed prior distribution of π,manufacturerscan

choose a test plan determined by (nb,c)using the min-

imum sample size nbfor any chosen maximum number

of allowable failures c.Forinstance,whenc=0anda

non-informative prior π∼Beta(1,1)is assumed, the

minimum sample size which can ensure the CR calcu-

lated in Eq. [2]tobenomorethanβ=0.05 is calcu-

lated as nb=13. Hence, at least 13 units need to be

tested if the test can only be passed when no failure

is observed. However, as larger maximum number of

allowable failures being set for passing the test, the CR

increases as it becomes easier to pass the test for a given

sample size n.Hence,tocontroltheCRatorbelow

β=0.05, more units need to be tested as more failures

are allowed to pass the test.

When more informative priors are available from

historical data or expertise, they can aect the selec-

tion of test plans. Table 1 has explored the impacts of

dierent prior distributions p(π ) on the selected test

plan for dierent tolerances on the maximum number

of allowable failures, c.Figure 1 shows the ve prior

434 S. CHEN ET AL.

π

p(π)

0.0 0.2 0.4 0.6 0.8 1.0

02468

Beta(1,1)

Beta(2,18)

Beta(4,16)

Beta(10,15)

Beta(10,10)

Figure . Density curves of diﬀerent prior distributions explored in

Tab le .

distributions explored in Table 1.Theatdensitycurve

corresponds to the non-informative prior Beta(1,1)

which assumes that all possible values for π∈(0,1)

have equal probability. Other prior distributions from

Beta(10,10)to Beta(2,18)become more informa-

tivewithreducedspread(correspondingtosmaller

standard deviation in Table 1)andprovidestronger

support for smaller failure probability π.Foranygiven

c, the minimum sample size required can be reduced if

the prior distribution from historical data supports the

reliability requirement. For example, when a prior dis-

tribution π∼Beta(2,18)is used, which supports high

reliability around 1 −2/(2+18)=0.9>R=0.8,

fewer units need to be tested to demonstrate the relia-

bility requirement (e.g., 4 <13 when c=0). However,

if the specied prior distribution is not in favor of the

reliability requirement, as illustrated with prior distri-

butions Beta(4,16),Beta(10,15),andBeta(10,10),

which favor incrementally lower reliability, more units

arerequiredtobetestedtodemonstratethesame

reliability requirement.

On the other hand, Table 2 demonstrates the impact

of dierent requirements on reliability. For a given

choice on the prior distribution, as Rdecreases cor-

responding to reduced requirement on reliability, the

minimum sample size, nb, decreases for a xed choice

on c. This matches our intuition that fewer units need

to be tested for demonstrating lower requirement on

reliability.

The BRDTs are useful for demonstrating reliability

requirements for binary tests. For example, a test plan

(nb=81,c=5)for a predetermined test period of

5 years can demonstrate no less than 0.9 reliability in

5 years with the CR controlled by 0.05. However, it

Tab le . Minimum sample sizes required by BRDTs with diﬀerent

choices on cand reliability requirements.

nb

cR=0.9R=0.8R=0.6

Settings: M=15000,β =0.05

π∼Beta(1,1)

oers no capability of demonstrating reliability at any

time before the end of the test period. For example,

if the customers are particularly concerned about

the reliability in the rst two years in addition to the

reliability by the end of the ve years, the conventional

BRDTs are unable to demonstrate all requirements

over multiple time periods. In addition, BRDTs are

unable to dierentiate and demonstrate reliability

requirements involving multiple failure modes asso-

ciated with dierent consequences. In the next two

sections, two categories of new MSRDTs are pro-

posedtodemonstratereliabilityrequirementsover

multiple time periods and for multiple failure modes,

respectively. Alternative designs are also proposed and

their performances are evaluated and compared under

dierent prior elicitations.

MSRDTs over multiple time periods

Conventional BRDTs often demonstrate the product

reliability within a single time period, such as dur-

ing the mission time or the service life, to meet with

the customers’ requirements. However, customers’ sat-

isfaction in dierent time periods may dier. For

instance, upon the purchase of products, customers

may expect higher reliability during the early lifetime.

Theoccurrenceofearlyfailuresmayhavestrongerneg-

ative impact on customers’ satisfaction and company’s

reputation than failures occurred in the later stage of

the service period. To explicitly demonstrate dier-

ent product reliability requirements over multiple time

periods rather than a single time period, the strategies

of MSRDTs, i.e., multi-state RDTs, are proposed in this

section to meet customers’ expectation on reliability

over multiple time periods.

Consider a nite testing period with the start

time at t0and the end time at tK. The testing time

duration (t0,tK] is exclusively partitioned into K

QUALITY ENGINEERING 435

Period 1 Period 2

···

Period KPeriod K+1

t0t1t2tK-1 tK

Figure . Illustration of the multiple time periods in a K-period

MSDRT between (t0,tK].

non-overlapping time periods, (ti−1,ti],i=1,...,K,

as illustrated in Figure 2.Letπiand yidenote the

probability of failure and the number of observed

failures within the ith time period (ti−1,ti], respec-

tively. Then the number of units that survive the entire

test duration (right-censored at the end of the test

duration tK) can be expressed as n−K

i=1yi,where

nisthetotalnumberoftestunits.Theprobabilityof

surviving the test is given by πK+1=1−K

i=1πi.The

objective of a MSRDT over multiple time periods is to

simultaneously demonstrate the product reliability at

multiple time points satisfying a set of lower reliability

requirements, Ri,i=1,...,K, with the assurance

level controlled at (1−β).Here,Riis the minimum

acceptable reliability in the rst icumulative time peri-

ods, (t0,ti], βis the maximum acceptable consumer’s

risk and assurance level can be explained as the min-

imum probability that the reliability requirements are

not met all given the test is passed (Hamada et al., 2008,

pp. 343–347). Two dierent scenarios of acceptance

criteria are proposed as follows.

Scenario I. The MSRDT will be passed if the cumu-

lative number of observed failures i

k=1ykat each

cumulative time period (t0,ti]isnomorethan

its corresponding cumulative maximum number of

allowable failures i

k=1ckfor all cumulative time

periods (t0,ti], at i=1,...,K. For example, con-

sider a two-period MSRDT with tests conducted at

the end of the second and fth year. For 100 test

units, the MSRDT will be passed if the number of

observed failures in rst two years do not exceed 1

and the number of observed failures at the end of the

fth year do not exceed 5.

Scenario II.TheMSRDTwillbepassedifthenumber

of observed failures yiat each non-overlapping time

period (ti−1,ti]isnogreaterthanitscorresponding

maximum number of allowable failures cifor all time

periods (ti−1,ti], at i=1,...,K.Forthesametwo-

period test, the MSRDT will be passed if the number

of observed failures in rst two years do not exceed

1andthenumberofobservedfailuresinthenext

three years do not exceed 4. It is noticed that the

major dierence between the two scenarios is that

Scenario II plans the tests for non-overlapping time

periods while Scenario I considers the cumulative

time-periods instead.

For each acceptance criterion, the design of MSRDT

over multiple time periods aims to determine (i)

the minimum sample size, denoted by nIand nII for

Scenarios I and II, respectively, and (ii) the cumula-

tive maximum number of allowable failures at time

ti,i

k=1ck, for Scenario I and the maximum num-

ber of allowable failures within ith time period, ci,

i=1,...,KforScenarioII.Foreitherscenario,the

MSRDT is selected by choosing the test plans which

control the CR at or below β. It is noticed that the

proposed MSRDT strategies are suitable for demon-

stration tests that generate failure counts data (Li et al.,

2016; Guo et al., 2011) over multiple time periods,

and do not make any assumptions on the failure time

distribution. The advantages of the proposed methods

are to fulll the reliability requirements of customers

over dierent testing periods (e.g., either cumulative

time periods from Scenario I or the non-overlapping

periods from Scenario II) simultaneously and provide

dierent testing strategies that require dierent min-

imum test sample sizes based on dierent maximum

numbers of allowable failures. Assuming a certain

failure time distribution over multiple time periods

or for multiple failure modes may limit the use of the

proposed strategies because the lifetime distribution

assumptionhastobevalidforthewholetestperiodand

onlytheexpectednumberoffailurescanbeobtained,

which is not commensurate with the objectives of

proposed strategies as mentioned above. Alternative

RDT designs such as Weibull testing which is more

suitable for failure time data, is out of the scope of this

article, but is of interest for future work.

To il l u s t rate t h e p r o p os e d M SRDTs ove r m u l t i-

ple time periods and further investigate the dier-

ence between two scenarios of acceptance criteria, the

MSRDTs over two time periods (i.e., K=2) are consid-

ered without loss of generality. Let R1and R2denote the

minimum acceptable reliabilities over the time peri-

ods (t0,t1]and(t0,t2]withR2<R1.Theprobabili-

ties of failure for each cumulative time period meet

the requirements if π1≤1−R1and π1+π2≤1−

R2. For acceptance criterion in Scenario I, the test

of MSRDT is to determine {nI,c1,c1+c2},andthe

probability of accepting the test for any given (π1,π

2),

436 S. CHEN ET AL.

denoted by HI(n,c1,c2), can be explicitly written

as

HI(n,c1,c2)=

c1

y1=0

c1+c2−y1

y2=0 n!

y1!y2!(n−y1−y2)!

×πy1

1πy2

2(1−π1−π2)n−y1−y2

and the corresponding CRIis controlled at or below β

by

CRI=1−1−R1

01−R2−π1

0HI(n,c1,c2)p(π1,π

2)dπ2dπ1

1

01

0HI(n,c1,c2)p(π1,π

2)dπ2dπ1

≤β,

[3]

where p(π1,π

2)denotes the joint prior distribution of

(π1,π

2,1−π1−π2).

For the acceptance criterion in Scenario II, the

MSRDT plan can be determined by specifying

{nII,c1,c2}, and the probability of accepting the test for

any combination of (π1,π

2),denotedbyHII(n,c1,c2)

is given by

HII(n,c1,c2)=

c1

y1=0

c2

y2=0 n!

y1!y2!(n−y1−y2)!

×πy1

1πy2

2(1−π1−π2)n−y1−y2,

and the corresponding CRII is controlled by

CRII

=1−1−R1

01−R2−π1

0HII(n,c1,c2)p(π1,π

2)dπ2dπ1

1

01

0HII(n,c1,c2)p(π1,π

2)dπ2dπ1

≤β.

[4]

Acasestudyisshownbelowforillustratingthepro-

posed MSRDT strategies for a two-period test. The

reliability requirements are set as R1=0.8andR2=

0.6 over the time periods (t0,t1]and(t0,t2]with

t2<2t1, which indicates longer time interval of (t0,

t1]than(t1,t2]. A higher reliability requirement R1

is desired for the early cumulative time period (t0,

t1] because the customers are averse to early failures.

The CR is controlled at β=0.05, indicating that the

probability of accepting the test when the actual reli-

ability requirements are not met is controlled at or

below 0.05. To evaluate the complex integration in

either Eq. [3]orEq.[4], Monte Carlo sampling is

performed with the sample size of M=15,000 to

maintain the evaluation accuracy. The Dirichlet dis-

tribution, denoted by Dirichlet (α1,α

2,α

3),isusedas

Tab le . Comparison between Scenarios I and II and BRDT, with

non-informative prior.

Scenario I Scenario II BRDT

c1c1+c2nIc1c2nII cn

b

Settings: p(π1,π

2)∼Dirichlet(1,1,1)

R1=0.8,R2=0.6,M=15000,β =0.05

the prior distribution for (π1,π

2,1−π1−π2),where

α1,α

2,α

3are hyper-parameters to be elicited based

on the prior knowledge. The Dirichlet distribution is

a family of continuous multivariate probability distri-

bution parametrized by the vector of positive hyper-

parameters αi,i=1,...,Kfor Kcategories of out-

comes. The advantage of using Dirichlet distribution

istwofolded.Firstofall,itistheconjugatepriorfor

the multinomial distribution, and hence allows an easy

update of knowledge as new data are observed because

the posterior distribution of the failure probabilities

also follow a Dirichlet distribution. Second, the hyper-

parameters in the Dirichlet distribution are associated

with more intuitive practical implications as they are

directly connected with the failure probabilities for

each outcome category based on the prior knowledge

in the form of αi/K

i=1αi. A few dierent settings of

hyper-parameters will be explored later to investigate

the impact of prior knowledge on the performance of

the proposed test plan.

When no prior information is available, a non-

informative prior distribution, given by (π1,π

2,1−

π1−π2)∼Dirichlet(1,1,1)can be used for indicat-

ing the lack of prior knowledge. The selected test plans

under the acceptance criteria of two scenarios with

dierent choices on the maximum number of allow-

able failures are illustrated in Ta bl e 3.Thetestplansare

grouped based on the total number of failures allowed

during the entire test duration. Several features are

QUALITY ENGINEERING 437

Figure . Comparison between Scenarios I and II based on the min-

imum sample size as c2increases for some ﬁxed c1values.

observed. First of all, under both Scenarios I and II,

given a xed choice of c2, the minimum sample size

nIor nII increases as c1increases. Similarly, given a

xed c1,nIand nII also increase with c2.Asforagiven

xed number of test units, allowing more failures (i.e.,

increasing c) can make it easier to pass the test and thus

increase the CR. Hence, it requires to test more units to

control the CR at a predetermined maximum accept-

able level. The patterns of minimum sample sizes can

be observed more clearly in Figures 3 and 4.

Figure 3 shows the change in the minimum sam-

ple size as c2increases for a few selected c1values

under both scenarios. Solid lines are used for showing

Scenario I and dash lines are used for Scenario II.

Dierent symbols are used for displaying dierent

c1values. For a xed c1value, the minimum sample

sizes under both Scenarios I and II increase as c2

increases. For example, when c1=0, two scenarios are

essentially the same in terms of the acceptance criteria.

Hence, the same minimum sample size is required

Minimum sample size

10 20 30 40 50 60

012345678910

c1

I:c2=1

I:c2=4

I:c2=6

II :c2=1

II :c2=4

II :c2=6

Figure . Comparison between Scenarios I and II based on the

minimum sample size as c1increases for some ﬁxed c2values.

for both scenarios, which is shown with the solid

line with the triangles and increases as c2increases.

When c1>0, the minimum sample size still generally

increases as c2increases. However, the trend is slightly

dierent between the two scenarios. The nIincreases

monotonically with c2,whilethenII starts o with sim-

ilar sample sizes for small c2values to a certain point

and then starts to increase more quickly as c2increases.

For example, when c1=4, the minimum sample size

for Scenario II (shown with a dotted line with the open

circles) is relatively at for c2≤4andthenincreases

for c2>4. This is because under Scenario II, the max-

imum number of allowable failures for the two non-

overlapping periods determines their corresponding

minimum required test units, which then jointly deter-

mine the overall minimum sample size for the entire

test. Therefore, the overall sample size can be domi-

natedbythemaximumnumberofallowablefailures

for one of the test periods if one of the ciis considerably

larger compared to its failure probability under the reli-

ability requirements to be demonstrated. Thus, when

c2is small, c1plays an dominating role in determining

the overall sample size for the entire test, which corre-

sponds to the at portion of the minimum sample size

curve for c1=4. However, as c2becomes larger than

c1, the overall minimum sample size is dominated by

the requirement from period 2 and hence resumes an

increasing pattern as c2increases. To compare the two

scenarios, it appears that nIis usually larger than nII for

small c2values, but becomes smaller than nII when c2

becomes larger than a certain value. This is because for

the same required civalues, the test plans in Scenario I

generally can allow larger maximum number of allow-

able failures for period 2 (when the maximum number

of allowable failure is not reached during period 1) and

hence request to test more units when c2has dominat-

ing impact on the overall minimum sample size.

Figure 4 shows how the minimum sample size

changes with c1for xed c2values under both scenar-

ios. Generally, for any xed c2,theminimumsample

size increases as c1increases under Scenario I. Also,

alargerc2value requires to test more units and the

dierence in nIamong dierent c2values are simi-

lar across dierent c1values, which is evidenced by

the almost parallel lines observed for Scenario I in

Figure 4. However, for Scenario II, even though nII

increases monotonically with c1, there are diminishing

dierences in nII at dierent c2values as c1increases.

This is because under Scenario II, increasing c1will

aect nII by increasing the minimum sample size

438 S. CHEN ET AL.

Minimum sample size

40 50 60 70 80 90 100 110

0 1 2 3 4 5 6 7 8 9 10 12 14 16 18 20

20 18 16 14 12 10 9 8 7 6 5 4 3 2 1 0

c1

c2

I:c1+c2=20

II :c1+c2=20

I:c1+c2=15

II :c1+c2=15

Figure . Comparison between Scenarios I and II based on the min-

imum sample size for ﬁxed c1+c2values.

needed to demonstrate the reliability requirement in

period 1 and hence leads to a dominating eect on the

size of nII (which is equivalent to a diminishing impact

of the dierence in c2values). While under Scenario I,

increasing c1will result in increases in the minimum

sample sizes needed for demonstrating both reliability

requirements at the end of the two non-overlapping

time periods, and hence has a consistent impact on the

overall minimum sample size nI.

It is also interesting to compare the two scenarios

given the same total maximum number of allowable

failures c1+c2in the entire test duration. Figure 5

compares the minimum sample sizes for both scenar-

iosgivenaxedtotalmaximumnumberofallowable

failures c1+c2.Twocaseswithc1+c2=15 and

c1+c2=20 are investigated, which are shown in

Figure 5 with the solid and dotted lines, respectively.

Thebottomandthetopaxesdisplayallcombinations

of c1and c2values. A few patterns can be observed.

First, both nIand nII increase as c1+c2increases.

This matches with the pattern for the conventional

BRDTs in that it generally requires to test more units to

ensurethesameassurancelevelifamorerelaxedcrite-

rion has been used for passing the test by allowing more

failures to be observed during the entire test duration.

Second, increasing c1(at the same time reducing c2)

will consistently increase nIbut reduce nII rst for small

c2values and then increase nII after c2reaches a certain

value.Third,intermsoftherelativeperformanceof

the two strategies, Scenario II is associated with smaller

overall minimum sample size for large c1and small c2

values. As c2increasing to about the same size as c1,

Scenario I starts to have a smaller minimum sample

size and the dierence becomes larger as c1continues

to increase. This can be evidenced by the crossing pat-

tern between the monotonically increasing line with

the squares for Scenario I and the U-shaped curve with

the open circles for Scenario II. Brief analytical expla-

nations can also be found in the Appendix to improve

the understanding of the observed dierences between

two scenarios.

Under the same maximum number of allowable fail-

ures c1+c2for the entire test duration, Scenario II

is expected to have more strict requirements (y1≤

c1,y2≤c2)thanScenarioI(y1≤c1,y1+y2≤c1+

c2), meaning that any tests that pass in Scenario II will

also pass in Scenario I. Intuitively, Scenario II will be

preferred if minimizing the CR is the only criterion of

interest, which on the other hand generally requires

larger minimum sample size. However, smaller test

sample is also generally preferred in RDT plan from

the manufacturer’s point of view. Hence, the tests with

minimum sample size after controlling the CR are gen-

erally preferred. As illustrated in Figure 5,thetwotest

scenarios may have varied performance in the required

minimum sample size for dierent settings and Sce-

nario II does not consistently outperform Scenario I

based on the minimum sample size. It is also noticed

in Table 3 that the dierence between the two scenar-

ios when c1is small becomes smaller for small c1+c2

values, and is almost negligible for c1+c2≤6. On

the other hand, Scenario I can be preferred for rela-

tively large c1values when c1+c2is large or when only

small c1+c2is allowed. It is also noticed that for tests

using more strict passing conditions, they are gener-

ally associated with smaller probabilities of passing the

test (i.e., low acceptance probability) and often higher

probabilities for manufacturers to reject the products

that actually have met the reliability requirements (Lu

et al., 2016). Hence, a decision on the selection of

scenarios should be catered for a particular applica-

tion to meet the objectives of a specic demonstration

test.

In addition, Tab l e 3 also shows the comparison

between the MSRDT strategies over two time periods

with the conventional BRDTs when non-informative

prior is used. The last two columns in Tabl e 3 give

the maximum number of allowable failures and the

minimum sample size for demonstrating the reliability

requirement at the end of test duration (i.e., the end

of period 2). For any given total maximum number of

QUALITY ENGINEERING 439

allowable failures over the entire test duration, c=c1+

c2, the conventional BRDTs require to test fewer units

for demonstrating only a single reliability at the end

ofthetest.TheMSRDTs,ontheotherhand,gainthe

capability of demonstrating multiple reliability require-

ments at dierent time points at the expense of testing

afewmoreunits.However,asc=c1+c2increases,

fewer extra units are required to be tested for demon-

strating more reliability requirements at multiple time

points. For example, for c=5, the conventional BRDT

requires to test 18 units to demonstrate reliability at the

end of the two-year period as 0.6. To demonstrate an

additional higher reliability at the end of the rst year

at 0.8, both MSRDT strategies require to test at least

20 units with no failure allowed to be observed during

the rst year. More units need to be tested if more

failures are allowed to be observed during the rst

year.

It is well known that incorporating dierent prior

information may have large impacts on the results

in Bayesian analysis. Next, we explore the impact of

dierent prior elicitations on the selected MSRDT

plans under both scenarios. Tables 4 and 5summarize

the required minimum sample sizes for the MSRDT

plans over two test periods with dierent choices of

prior distributions for Scenarios I and II, respectively.

Seven dierent prior distributions including the non-

informative prior, Dirichlet(1,1,1),areexplored.

The patterns are rather consistent across Tab l e s 4 and

5. Under both scenarios, when the prior distribu-

tion supports higher reliabilities than the minimum

requirements, such as Dirichlet(3,3,24)showninthe

fourth column in both tables, the minimum sample size

can be substantially reduced for any given combina-

tion of c1and c2values than using the non-informative

prior(showninthethirdcolumninbothtables).

On the other hand, if the prior distribution supports

reliabilities at or below the requirements, more units

need to be tested to demonstrate the requirements

than using the non-informative prior. This can be

observed in Figures 6 and 7which show the mini-

mum sample size for xed c1+c2under Scenario I

and II, respectively. In both gures, the solid lines

with triangles represent the sample sizes for dierent

(c1,c2)combinations using a non-informative prior.

The dash lines with squares show the sample sizes for

a prior distribution Dirichlet(3,3,24)that supports

higher reliabilities than the requirements, which are

consistently below the line of non-informative prior.

All other prior distributions support reliabilities at

or below the requirements, and hence all require to

test more units with the corresponding lines located

above the line of non-informative prior. The farther

the specied prior distribution is to the reliability

requirements, the more test units are needed in the

MSRDTs over multiple time periods. One special

case is the dash line with open circles observed in

Figure 7 for a prior distribution Dirichlet(3,12,15),

Tab le . Minimum sample sizes required by the two-period MSRDT using the acceptance criterion in Scenario I for diﬀerent prior

distributions.

Dirichlet (1,1,1)(3,3,24)(6,6,18)(12,3,15)(3,12,15)(6,12,12)(12,6,12)

c1c1+c2nI

Settings: M=15000,R1=0.8,R2=0.6,β =0.05

440 S. CHEN ET AL.

Tab le . Minimum sample sizes required by the two-period MSRDT using the acceptance criterion in Scenario II for diﬀerent prior

distributions.

Dirichlet (1,1,1)(3,3,24 )(6,6,18)(12,3,15)(3,12,15)(6,12,12)(12,6,12)

c1c2nII

Settings: M=15000,R1=0.8,R2=0.6,β =0.05

which is consistently below the non-informative line

indicating smaller minimum sample sizes are required

for all (c1,c2)combinations. Since the prior distribu-

tion regarding period 1 supports higher reliabilities

than the requirements, while the prior distribution

regarding period 2 supports reliabilities below the

requirements, the eects of sample size reduction from

period 1 and sample size increase from period 2 may

jointly determine the overall minimum sample size,

and hence lead to slightly dierent pattern than what

has been observed for other prior distributions.

Minimum sample size

0 20 40 60 80 100

6543210

0123456

c1

c2

(3,3,24)

(6,6,18)

(12,3,15)

(3,12,15)

(6,12,12)

(12,6,12)

Non−informative

Dirichlet

Figure . Minimum sample sizes required in Scenario I with ﬁxed

c1+c2=6for diﬀerent prior distributions.

MSRDTs for multiple failure modes

In the previous section, the MSRDT strategies con-

sider each time period as an individual state for demon-

strating specic reliability requirement within the time

period. This section proposes a dierent category of

MSRDTs which considers dierent failure modes as

individual states that are often associated with dierent

consequences of failures and dierent costs of replace-

ment. The conventional BRDTs report dichotomous

outcomes (i.e., success and failure) for each test unit,

Minimum sample size

0 20 40 60 80 100

6543210

0123456

c1

c2

(3,3,24)

(6,6,18)

(12,3,15)

(3,12,15)

(6,12,12)

(12,6,12)

Non−informative

Dirichlet

Figure . Minimum sample sizes required in Scenario II with ﬁxed

c1+c2=6for diﬀerent prior distributions.

QUALITY ENGINEERING 441

in which case dierent failure modes of the product

are not dierentiated and the severity levels of dierent

consequences associated with dierent failures modes

are overlooked. In real applications, a product often

has multiple failure modes in varied levels of sever-

ity, which can lead to dierent impacts on customers’

dissatisfaction.

For instance, the failure of a CPU or a hard drive of

a computer system is much more critical than the fail-

ures of some accessory parts such as a keyboard or a

microphone, since the former can lead to a complete

breakdownofthesystem,alossofvaluableinforma-

tion and/or a high repair/replacement cost while the

latter usually only results in system under-performance

and a low repair/replacement cost. Consequently, the

failures of critical or valuable parts will lead to stronger

dissatisfaction of customers, and hence result in higher

expectation on reliability for these components. It is

desirable to develop test strategies that allow demon-

strating separate reliability requirements for multiple

failure modes.

The product with Jindependent failure modes

is considered. For each test unit, it will either have

failed in mode j,j=1,...,Jor remain working by

the end of the testing period. Let πjand yjdenote

the probability of failure and the number of observed

failures in failure mode jwithin the test period (or an

equivalent mission time period), respectively. Then,

πJ+1=1−J

j=1πjand n−J

j=1yjdenote the

probability of success and the number of survived

units by the end of the test. The MSRDTs for multiple

failure modes aim to demonstrate at an assurance level

at (1−β) that the product reliability will meet multi-

ple minimum reliability requirements for each of the

dierent failure modes, denoted by Rj,j=1,...,J.

Here, βistheCRonhavingaproductthathaspassed

the demonstration test but fails to meet all reliability

requirements for dierent failure modes. Note that

all failure modes are dened in the same test period.

For any specied reliability requirements Rj’s and t h e

maximum acceptable CR controlled at or below β,the

MSRDTs for multiple failure modes are designed to

determine the minimum sample size nmas well as the

maximum number of allowable failures cjin the jth

failuremodefor j=1,...,J.

Without loss of generality, considering two failure

modes with J=2forillustratingtheproposedMSRDT

strategy. Let R1and R2denote the minimum accept-

able reliabilities for failure modes 1 and 2, respectively.

The test is passed if the number of observed failures yj

is less or equal to the maximum number of allowable

failures cjfor both failure modes, and the test plan is to

determine the choice on {nm,c1,c2}.Forindependent

failure modes, the acceptance probability Hm(n,c1,c2)

for certain (π1,π

2)values can be written as

Hm(n,c1,c2)=

c1

y1=0 n!

y1!(n−y1)!πy1

1(1−π1)n−y1

×

c2

y2=0 n!

y2!(n−y2)!πy2

2(1−π2)n−y2

and the corresponding CR, denoted by CRm,iscalcu-

lated by

CRm=1−(1−R1)

0(1−R2)

0Hm(n,c1,c2)p(π1,π

2)dπ2dπ1

1

01

0Hm(n,c1,c2)p(π1,π

2)dπ2dπ1

,

[5]

where p(π1,π

2)is the joint prior distribution of

(π1,π

2). For independent failure modes, there is

p(π1,π

2)=p(π1)p(π2). The minimum sample size is

determined by controlling the CRmobtained in Eq. [5]

to be at or below β. Simulation case studies are con-

ducted for exploring dierent reliability requirements,

maximum numbers of allowable failures for dierent

failure modes, as well as dierent prior elicitations and

their impacts on the required minimum sample size

for the MSRDTs for two failure modes. The results are

summarized in Tables 6 and 7for two cases with simi-

lar or dierent reliability requirements for the two fail-

ure modes. In Tab l e 6, identical minimum reliability

requirements are assumed for the two failure modes,

where R1=R2=0.8 indicates that the customers have

the same expectation on reliability for both failure

modes. Table 7 assumes dierent reliability require-

ments with R1=0.8andR2=0.6. Here, failure mode

1 is considered more critical and/or have more severe

consequences associated with its failure, and hence is

required for a higher reliability. The CRmis still con-

trolled at β=0.05 and the sample size for Monte Carlo

sampling is chosen as M=15000 to maintain the sim-

ulation accuracy. Beta distributions are used for speci-

fying the prior distributions for both π1and π2for the

two failure modes.

When two failure modes have the same reliability

requirements at R1=R2=0.8, Table 6 summarizes

the minimum sample size with dierent choices of the

maximum number of allowable failures and dierent

prior settings. When no prior information is available,

442 S. CHEN ET AL.

Tab le . Multiple failure modes with the same reliability require-

ments for diﬀerent prior distributions.

π1(1,1)(2,18)(4,16)(10,15 )(2,18)(2,18)(4,16)

Beta π2(1,1)(2,18)(4,16)(10,15 )(4,16)(10,15)(10,15 )

c1c2nm

Settings: M=15000,R1=0.8,R2=0.8,β =0.05

a non-informative prior distribution of Beta(1,1)is

assigned for both π1and π2.Similarpatternscanbe

observed as for the MSRDTs over multiple time peri-

ods. When c1is xed, the minimum sample size nm

increases as c2increases; when c2is xed, nmincreases

with c1. This is intuitive as having more allowable fail-

ures makes it easier to pass the test and thus increases

Tab le . Multiple failure modes with diﬀerent reliability require-

ments for diﬀerent prior distributions.

π1(1,1)(2,18)(10,10)(4,16)(2,18)(10,10)

Beta π2(1,1)(2,18)(10,10)(10,15)(10,10)(2,18)

c1c2nm

Settings: M=15000,R1=0.8,R2=0.6,β =0.05

the CR. To control a reasonable CR, a larger number

of units need to be tested by allowing more failures

to be observed during the test. When c1+c2is xed,

the minimum sample size nmexhibits a symmetric

pattern under the non-informative prior setting due to

the identical reliability requirements for both failure

modes. For example, when c1+c2=6, the minimum

sample sizes for c1=0,c2=6, and c1=6,c2=0

are identical. In addition, when c1and c2become

more similar in size (e.g., c1=2,c2=4comparedto

c1=0,c2=6), it requires smaller minimum sample

size to remain the same assurance level for demon-

strating the requirements on both failure modes. This

makes sense as when the maximum number of allow-

able failures is considerably larger for one failure mode

given the same reliability requirement, it requires to test

more units for demonstrating the requirement for this

failuremode,whichtheninatestheoverallminimum

sample size needed in the MSRDT for demonstrating

reliability requirements for both failure modes.

Dierent prior elicitations also have large impacts

on the selected test plan, as shown in Ta b l e 6 .When

prior knowledge supports higher reliabilities than the

requirements to be demonstrated, fewer units need to

be tested and vice versa. For instance, the prior dis-

tributions of π1∼Beta(2,18)and π2∼Beta(2,18)

indicate that there is a strong belief of lower fail-

ure probabilities than the requirements within the test

period for both failure modes. Thus, the corresponding

minimum sample size is smaller than what is needed

for using the non-informative prior. On the other hand,

when the prior distributions of π1∼Beta(10,15)and

π2∼Beta(10,15)are used, which indicates a moder-

ately strong belief in larger failure probabilities than

the requirements for both failure modes, more units

need to be tested to demonstrate the higher reliabil-

ity requirements compared to what is needed when no

prior information is available.

When c1+c2is xed, the required minimum sam-

ple size is also sensitive to the specied prior distri-

bution. Figure 8 illustrates the change in the nmfor

dierent (c1,c2)combinations given xed c1+c2=

6. When the non-informative priors are assumed, the

curve for nm(the solid line with the triangles) shows

a symmetric pattern with the minimum sample size

achieved at c1=c2=3. When informative priors indi-

cating lower failure probabilities than requirements for

both failure modes (such as π1∼Beta(2,18), π2∼

QUALITY ENGINEERING 443

Minimum sample size

050

100 150

6543210

0123456

c1

c2

(2,18) (2,18)

(4,16) (4,16)

(10,15)

(10,15)

(2,18)

(4,16)

(2,18)

(10,15)

(4,16)

(10,15)

Non−informative

Beta

Figure . Multiple failure modes with the same reliability require-

ments for ﬁxed c1+c2and diﬀerent prior distributions.

Beta(2,18)corresponding to the dash line with the

open circles) are assumed, the minimum sample size

curve is below the non-informative curve. As the prior

belief indicates higher failure probability for at least one

of the failure modes (such as π1∼Beta(2,18), π2∼

Beta(10,15)corresponding to the dotted line with the

solid circles or π1∼Beta(10,15), π2∼Beta(10,15)

corresponding to the dash-dotted line with the open

circles), the corresponding minimum sample size cur ve

shifts upwards on at least one side of tails or on both

sides.

Table 7 showsthetestplanswhendierentreliability

requirements are used for the two failure modes with

R1=0.8andR2=0.6. When the non-informative

priors are used, the symmetric pattern is no longer

observed due to dierent requirements on reliability for

the two failure modes. Particularly, nmis larger when

c1is large since more units need to be tested to demon-

strate higher reliability requirement for failure model 1

while allowing more failures to be observed during the

test period. Also, for the same c1and c2settings, the

minimum sample size for demonstrating R1=R2=

0.8 is smaller than what is required for demonstrat-

ing R1=0.8andR2=0.6 since fewer units can be

tested to demonstrate a lower reliability requirement

for failure mode 2. When more informative priors are

used, similar patterns are observed from both Ta b l e 7

and Figure 9. A potential sample size reduction can be

achieved when the prior knowledge supports higher

reliability than what is required to be demonstrated by

the MSRDT.

Minimum sample size

0 50 100 150

6543210

0123456

c1

c2

(2,18)

(2,18)

(10,10)

(10,10)

(4,16)

(10,15)

(2,18) (10,10)

(10,10)

(2,18)

Non−informative

Beta

Figure . Multiple failure modes with diﬀerent reliability require-

ments for ﬁxed c1+c2and diﬀerent prior distributions.

Conclusions

Conventional binomial RDTs, which focus on demon-

strating a single reliability requirement within a single

test period, have limited use when multiple reliability

requirements need to be met. This article proposes

two types of RDTs for demonstrating reliabilities over

multiple time periods and for multiple failure modes.

These RDTs with multiple reliability requirements are

all referred to as multi-state RDTs (MSRDTs).

IntheMSRDTsovermultipletimeperiods,every

time period of interest is treated as a state, and the joint

distribution of failure counts over the non-overlapping

time periods can be modeled by a multinomial distri-

bution. Two dierent test strategies are proposed for

demonstrating multiple requirements over dierent

time periods. One strategy uses the cumulative failure

counts at the end of each cumulative time period peri-

ods as the criteria for passing the test; while the other

uses separate failure counts over non-overlapping time

intervals as the criteria for passing the test. Simula-

tion studies were conducted for comparing the two

strategies by considering two-period MSRDTs. It was

foundedthatthestrategybasedoncumulativefailure

counts (Scenario I) is generally preferred for cases that

allow fewer total failure counts over all time periods or

when a larger maximum number of allowable failures

is allowed for the early cumulative time period. The

strategy using separate failure counts (Scenario II) is

only preferred for requiring smaller minimum sample

size when a smaller maximum number of allowable

failures is allowed for the early separate time period.

444 S. CHEN ET AL.

In the MSRDTs for multiple failure modes, each

failuremodeistreatedasastateandallreliability

requirements for the multiple failure modes that may

be associated with dierent consequences in varied lev-

els of severity and/or costs of repair/replacement can be

simultaneously demonstrated. The required minimum

sample size is usually determined mainly by the fail-

uremodethathasthehighestreliabilityrequirement

and/or least stringent criterion for passing the test (i.e.,

allowing a larger maximum number of allowable fail-

ures for a particular failure mode).

The impacts of incorporating dierent prior dis-

tributions are also explored for both categories of

MSRDTs. The patterns are consistent regardless of

which test strategy is considered. When the prior

knowledge supports higher reliabilities than the

requirements to be demonstrated, fewer units can be

tested compared to using the non-informative priors

for demonstrating the same reliability requirements.

However, if the historical data supports lower relia-

bilities than what are required to be demonstrated,

then more units need to be tested to override the

eects of the prior distribution for demonstrating

higher reliabilities than what has been indicated from

existing data. For future work, it is expected to develop

thorough mathematical justications with theoritical

formulations and derivations to validate the discussed

patterns using both non-informative and informative

prior distributions.

About the authors

Suiyao Chen is a Ph.D. student in the Department of Indus-

trial and Management Systems Engineering at University of

South Florida. He received his B.S. degree (2014) in Economics

from Huazhong University of Science and Technology and

M.A. degree (2016) in Statistics from Columbia University. His

research focus is on statistical reliability data analysis, demon-

stration tests design, and data analytics.

Lu Lu is an Assistant Professor of Statistics in the Depart-

ment of Mathematics and Statistics at the University of South

Florida in Tampa. She was a postdoctoral research associated in

the Statistics Sciences Group at Los Alamos National Labora-

tory. She earned a doctorate in statistics from Iowa State Univer-

sity in Ames, IA. Her research interests include reliability anal-

ysis, design of experiments, response surface methodology, sur-

vey sampling, and multiple objective/response optimization.

Mingyang Li is an assistant Professor in the Department of

Industrial & Management Systems Engineering at the Univer-

sity of South Florida. He received his Ph.D. in systems & indus-

trial engineering and his M.S. in statistics from the University

of Arizona in 2015 and 2013, respectively. He also received his

M.S. in mechanical & industrial engineering from the Univer-

sity of Iowa in 2010 and his B.S. in control science & engineering

from Huazhong University of Science and Technology in 2008.

His research interests include reliability and quality assurance,

Bayesian data analytics and system informatics. Dr. Li is a mem-

ber of INFORMS, IISE, and ASQ.

Funding

This work was supported in part by National Science Foun-

dation under Grant BCS-1638301 and in part by University of

South Florida Research & Innovation Internal Awards Program

under Grant No. 0114783.

References

Guo, H., T. Jin, and A. Mettas 2011. Designing reliability

demonstrationtestsforone-shotsystemsunderzerocom-

ponent failures. IEEE Transactions on Reliability 60 (1):286–

294.

Guo, H., and H. Liao 2012. Methods of reliability demonstra-

tion testing and their relationships. IEEE Transactions on

Reliability 61 (1):231–237.

Hamada, M. S., A. Wilson, C. S. Reese, and H. Martz 2008.

Bayesian reliability. Springer Science & Business Media.

Kececioglu, D. 2002. Reliability and life testing handbook.Vol.2.

DEStech Publications, Inc.

Li, M., W. Zhang, Q. Hu, H. Guo, and J. Liu 2016. Design and risk

evaluation of reliability demonstration test for hierarchi-

cal systems with multilevel information aggregation. IEEE

Transa c tio ns on Re lia bil ity 66 (1):135–147.

Lu, L., M. Li, and C. M. Anderson-Cook 2016. Multiple objec-

tive optimization in reliability demonstration tests. Journal

of Quality Technology 48 (4):303–326.

McKane, S. W., L. A. Escobar, and W. Q. Meeker 2005. Sample

size and number of failure requirements for demonstration

tests with log-location-scale distributions and failure cen-

soring. Technometrics 47 (2):182–190.

Pintar, A., L. Lu, C. M. Anderson-Cook, and G. L. Silver 2012.

Bayesian estimation of reliability for batches of high relia-

bility single-use parts. Quality Engineering 24 (4):473–485.

Robert, C., and G. Casella 2004. Monte Carlo statistical methods.

2nd ed. Springer Science & Business Media.

Wasserman, G. 2002. Reliability verication, testing, and analysis

in engineering design. CRC Press.

Weaver, B. P., and M. S. Hamada 2008. A Bayesian approach to

the analysis of industrial experiments: An illustration with

binomial count data. Quality Engineering 20 (3):269–280.

Wilson, A. G., and K. M. Fronczyk 2016. Bayesian reliabil-

ity: combining information. Quality Engineering 29 (1):119–

129.

Yang, G. 2009. Reliability demonstration through degradation

bogey testing. IEEE Transactions on Reliability 58 (4):604–

610.

QUALITY ENGINEERING 445

Appendix

To analytically show the dierence between Scenar-

iosIandIIintheproposedMSRDTsovermultiple

time periods when c1+c2is xed, let H(n,c1,c2)=

HI(n,c1,c2)−HII(n,c1,c2), which can be explicitly

written as

H(n,c1,c2)=

c1

y1=0

c1−y1

y2=c2+1 n!

y1!y2!(n−y1−y2)!

×πy1

1πy2

2(1−π1−π2)n−y1−y2.

When c1=0, H(n,c1,c2)=0andbothscenarios

become equivalent, as shown in Tab l e s 3–5.Whenc1>

0, H(n,c1,c2)>0, which indicates that the proba-

bility of accepting test plan under Scenario II is always

smaller than the probability calculated under Scenario

I. However, this nding does not imply that for a

xed n, one scenario will always give a consistently

higher/lower CR than the other. To justify this, let A=

1−R1

01−R2−π1

0HII(n,c1,c2)p(π1,π

2)dπ2dπ1and B=

1

01

0HII(n,c1,c2)p(π1,π

2)dπ2dπ1,CR

II and CRIcan

be written as

CRII =1−A

B,

CRI=1−A+A

B+B,

where A=1−R1

01−R2−π1

0H(n,c1,c2)p(π1,π

2)

dπ2dπ1and B=1

01

0H(n,c1,c2)p(π1,π

2)

dπ2dπ1.ThenCR

II −CRIis given by

CRII −CRI=BA−AB

B(B+B).

Although B>A,asn,c1and c2vary, Acan be

larger/smaller than B. Thus, for a xed sample size n,

neither CRII >CRInor CRII <CRIwill hold consis-

tently. It also explains results in Figure 5,andTa b l e s 4

and 5that when controlling CR, one scenario cannot

give a consistently larger/smaller minimum sample size

than the other scenario.