Page 1

Statistical Applications in Genetics

and Molecular Biology

Volume 8, Issue 12009Article 23

Weighted Multiple Hypothesis Testing

Procedures

Guolian Kang∗

Keying Ye†

Nianjun Liu‡

David B. Allison∗∗

Guimin Gao††

∗University of Alabama at Birmingham, gkang@ms.soph.uab.edu

†University of Texas at San Antonio, keying.ye@utsa.edu

‡University of Alabama at Birmingham, nliu@uab.edu

∗∗University of Alabama at Birmingham, dallison@ms.soph.uab.edu

††University of Alabama at Birmingham, ggao@ms.soph.uab.edu

Copyright c ?2009 The Berkeley Electronic Press. All rights reserved.

Page 2

Weighted Multiple Hypothesis Testing

Procedures∗

Guolian Kang, Keying Ye, Nianjun Liu, David B. Allison, and Guimin Gao

Abstract

Multiple hypothesis testing is commonly used in genome research such as genome-wide stud-

ies and gene expression data analysis (Lin, 2005). The widely used Bonferroni procedure controls

the family-wise error rate (FWER) for multiple hypothesis testing, but has limited statistical power

as the number of hypotheses tested increases. The power of multiple testing procedures can be in-

creased by using weighted p-values (Genovese et al., 2006). The weights for the p-values can be

estimated by using certain prior information. Wasserman and Roeder (2006) described a weighted

Bonferroni procedure, which incorporates weighted p-values into the Bonferroni procedure, and

Rubin et al. (2006) and Wasserman and Roeder (2006) estimated the optimal weights that max-

imize the power of the weighted Bonferroni procedure under the assumption that the means of

the test statistics in the multiple testing are known (these weights are called optimal Bonferroni

weights). This weighted Bonferroni procedure controls FWER and can have higher power than

the Bonferroni procedure, especially when the optimal Bonferroni weights are used. To further

improve the power of the weighted Bonferroni procedure, first we propose a weightedˇSid´ ak pro-

cedure that incorporates weighted p-values into theˇSid´ ak procedure, and then we estimate the

optimal weights that maximize the average power of the weightedˇSid´ ak procedure under the as-

sumption that the means of the test statistics in the multiple testing are known (these weights are

called optimalˇSid´ ak weights). This weightedˇSid´ ak procedure can have higher power than the

weighted Bonferroni procedure. Second, we develop a generalized sequential (GS)ˇSid´ ak pro-

cedure that incorporates weighted p-values into the sequentialˇSid´ ak procedure (Scherrer, 1984).

This GSˇSid´ ak procedure is an extension of and has higher power than the GS Bonferroni pro-

cedure of Holm (1979). Finally, under the assumption that the means of the test statistics in the

multiple testing are known, we incorporate the optimalˇSid´ ak weights and the optimal Bonferroni

weights into the GSˇSid´ ak procedure and the GS Bonferroni procedure, respectively. Theoretical

proof and/or simulation studies show that the GSˇSid´ ak procedure can have higher power than

the GS Bonferroni procedure when their corresponding optimal weights are used, and that both

of these GS procedures can have much higher power than the weightedˇSid´ ak and the weighted

∗We thank the editor and two referees for their helpful comments and useful suggestions. This

research was supported by grant GM073766, GM077490, and GM081488 from the National In-

stitute of General Medical Sciences. Address for correspondence: Dr. Guimin Gao, Depart-

ment of Biostatistics, University of Alabama at Birmingham, Birmingham, AL 35294. email:

ggao@ms.soph.uab.edu. Phone: 205-975-9188.

Page 3

Bonferroni procedures. All proposed procedures control the FWER well and are useful when prior

information is available to estimate the weights.

KEYWORDS:weight, multiplehypothesistesting, Bonferroniprocedure,ˇSid´ akprocedure, family-

wise error rate

Page 4

1 Introduction

Multiple hypothesis testing involves testing multiple hypotheses simultaneously;

each hypothesis is associated with a test statistic (Rubin et al., 2006). Multiple

hypothesis testing is a common problem in genome research, such as genome-

wide studies and gene expression data analysis (Lin, 2005). For multiple

hypothesis testing, a traditional criterion for error (type I) control is the family-

wise error rate (FWER), which is the probability of rejecting one or more true null

hypotheses (Hochberg and Tamhane, 1987; Lin, 2005).

The Bonferroni procedure (Bonferroni, 1937) and the Šidák procedure

(Šidák, 1967) are two well-known methods for controlling FWER with

computational simplicity and wide applicability (Olejnik et al., 1997). However,

both of these methods have limited statistical power as the number of hypotheses

tested (m) increases (Nakagawa, 2004). Holm (1979) proposed a (step-down)

sequential Bonferroni procedure which has slightly higher power than the

Bonferroni procedure but there is little difference between these two procedures

when the number of tests (m) is large (Lin, 2005). As an extension of the (step-

down) sequential Bonferroni procedure, Holm (1979) proposed a generalized

sequential (GS) Bonferroni procedure by using different weights for hypotheses

of different importance. Although Holm did not show how to estimate the

weights, the method has the potential to improve the power of multiple hypothesis

testing when prior information is available to estimate the weights.

Rubin et al. (2006) and Wasserman and Roeder (2006) proposed a

weighted Bonferroni procedure that adjusts p-values by using optimal weights.

These optimal weights were calculated by maximizing the average power of the

weighted Bonferroni procedure under the assumption that the means of all test

statistics are known, and these weights are called optimal Bonferroni weights.

Under such assumption, the average power of the weighted Bonferroni procedure

is much higher than that of the Bonferroni procedure (Rubin et al., 2006;

Genovese et al., 2006; Wasserman and Roeder, 2006). In practice, the means of

the test statistics are unknown. However, if some prior information is available to

estimate the means, this weighted Bonferroni procedure can be more powerful

than the Bonferroni procedure (Rubin et al., 2006; Wasserman and Roeder, 2006;

Roeder et al., 2006; Roeder et al., 2007).

The purpose of this study is to develop more powerful weighted

hypothesis testing procedures as extensions of the weighted Bonferroni procedure.

First, we propose a weighted Šidák procedure, and then under the assumption that

the means of all test statistics are known, we estimate the optimal weights

maximizing the average power of the weighted Šidák procedure (these weights

are called optimal Šidák weights). The weighted Šidák procedure has slightly

higher power than the weighted Bonferroni procedure. Second, we develop a GS

1

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 5

Šidák procedure as an extension of the GS Bonferroni procedure of Holm (1979)

and the sequential Šidák procedure (Scherrer, 1984). Finally, assuming that the

means of all test statistics are known, we incorporate the optimal Šidák weights

and the optimal Bonferroni weights into the GS Šidák procedure and the GS

Bonferroni procedure, respectively. Theoretical proof and/or simulation studies

show that, using their corresponding optimal weights, the GS Šidák procedure has

slightly higher power than the GS Bonferroni procedure, and that both GS Šidák

procedure and GS Bonferroni procedure have much higher power than the

weighted Šidák procedure and the weighted Bonferroni procedure. All the

proposed procedures can control the FWER well.

2 Methods

2.1 Notations

Consider testing m (null) hypotheses H

test statistics Z = (Z1, Z2, …, Zm), where we assume that

distribution of ) 1 ,(

i

N μ

, and all

paper, we only present the results for one-sided tests. Similar results for two-sided

tests can easily be obtained. Thus, for the i-th test, the (null) hypothesis is

0:

=

ii

H

μ

, and the corresponding alternative hypothesis is

that there are

1

m true null hypotheses and

hypotheses in H, where

2

m = m -

1

m . Let H0 denote all the true null hypotheses in

H. Let ),,,(

21

m

ppp

L

p

denote the p-values associated with the hypotheses

),,,(

21

m

HHH

L

. Let ,,,(

21

L

μ

statisticsZ.

As described earlier, FWER is the probability of falsely rejecting at least

one true null hypothesis (Hochberg and Tamhane, 1987), which can be written as

(

least at rejectingPrFWER

=

A multiple testing procedure is said to control the family-wise error rate at a

significance level α if

α≤

FWER

.

The power for a single test is called per-hypothesis power. For a single test

with hypothesis Hi, the per-hypothesis power is the probability of rejecting Hi

given that the alternative hypothesis

i

H is true, i.e., Pr(rejecting

multiple hypotheses testing, Roeder et al. (2007) defined the average power of a

),,,(

21m

HHH

L

=

with corresponding

i Z follows normal

i Z ’s are independent. For simplicity, in this

0:

>

ii

H

μ

. Suppose

2

m false null hypotheses among all

=

)

m

μμμ=

denote the means of the test

)

0

| one

Η

∈

ii HH.

i

H |0

>

i μ

). For

2

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437

Page 6

testing procedure as the average value of per-hypothesis powers of the

2

m tests

associated with the false null hypotheses:

()

∑

:i

μ>

>

0

2

0| rejecting Pr

1

i

ii

H

m

μ

.

2.2 Weighted Bonferroni procedure and optimal Bonferroni weights

2.2.1 Weighted Bonferroni procedure

In the Bonferroni procedure, if

m

pj

α

≤

, then reject the null hypothesis

j

H ;

otherwise, it is failed to reject

procedures can be increased by using weighted p-values (Genovese et al., 2006).

Holm appears to be the first one proposing the idea of the weighted Bonferroni

procedure, which incorporates weighted p-values into the Bonferroni procedure

(Holm, 1979). Wasserman and Roeder (2006) provided a clear description of the

weighted Bonferroni procedure as follows. Given nonnegative weights

),,,(

21

m

www

L

for the tests associated with the hypotheses

where

1

∑

=

j

m

j

H (j = 1, …, m). The power of multiple testing

),,,(

21

m

HHH

L

,

. 1

1

=

m

j

w

(1)

For hypothesis

j

H (

mj ≤ ≤

1), when wj > 0, reject

j

H if

mw

p

j

j

α

≤

, and fail to

reject

j

H when wj = 0.

This procedure controls FWER at level α. The weights

can be specified by using certain prior information available to the researcher. For

example, in genome-wide association studies, the prior information can be linkage

signals or results from gene expression analyses. Roeder et al. (2006) proposed a

method to estimate weights by using linkage data to weight association p-values

in association studies. However, how to estimate the optimal weights in multiple

testing is still a topic to be further investigated (see also the section on discussion).

),,,(

21

m

www

L

3

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 7

2.2.2 Optimal Bonferroni weights

Rubin et al. (2006) and Wasserman and Roeder (2006) independently proposed

very similar approaches to estimate the optimal weights by maximizing the

average power of the procedure, assuming that the means

known. We call these optimal weights optimal Bonferroni weights and they are

calculated (Wasserman and Roeder, 2006) by

⎛

+Φ=

j

w

μα

),,,(

21

m

μμμ

L

=

μ

are

() 0

2

>

⎟⎟

⎠

⎞

⎜⎜

⎝

Δ

j

j

j

I

m

μ

μ

, (2)

where

distribution function (CDF) (i.e.,

the standard normal distribution) and Δ is the constant that satisfies equations (1)

and (2) i.e.

∑

=

⎝

m

2

As an illustrative example, Figure 1 shows the optimal Bonferroni weights

as a function of the means μj in a multiple testing with 100 tests. The means μ

vary from 1 to 7 in increment of 6/99 = 0.0606. When the means μj are small, the

optimal weights increase with the increase of μj but when μj are large enough, the

optimal weights decrease with the increase of μj. In other words, the weighted

Bonferroni procedure offers large weights (often > 1) to the tests with midrange

of means and offers small weights (often < 1) to tests with small or large means

(Wasserman and Roeder, 2006). Dividing the p-value by a weight w > 1 increases

the probability of rejecting the corresponding null hypothesis, and dividing the p-

value by a weight 0 < w < 1 decreases the probability of rejecting the

corresponding null hypothesis. However, in most situations, even though the tests

with large means are assigned small weights (<1), the corresponding hypotheses

can still be rejected because the related p-values are very small. The weighted

Bonferroni procedure using these optimal weights can have much higher power

than the Bonferroni procedure when the means (μ μ) of the test statistics are given

or given prior information that can be used for estimating the means (Roeder et al.,

2006, 2007; Rubin et al., 2006; Wasserman and Roeder, 2006).

)(x

Φ

is the upper tail probability of a standard normal cumulative

)(x

Φ

= 1- (x

Φ

) and )(x

Φ

denotes the CDF of

()

=>

⎟⎟

⎠

⎞

⎜⎜

⎛

Δ

μ

+Φ

m

j

j

j

j

I

m

α

1

.0

1

αμ

μ

(3)

4

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437

Page 8

Figure 1. Distribution of optimal Bonferroni weights for a multiple testing

procedure with m = 100 tests. μ are the means of the test statistics, and vary from

1 to 7.

2.3 Weighted Šidák procedure and optimal Šidák weights

Since the Šidák procedure has higher power than the Bonferroni procedure for

independent tests (Simes, 1986), we propose a weighted Šidák procedure that

incorporates weighted p-values into the Šidák procedure as an extension of the

weighted Bonferroni procedure. We also describe how to calculate the optimal

weights for the weighted Šidák procedure assuming means of the test statistics are

known.

2.3.1 Weighted Šidák procedure

In the Šidák procedure (Šidák, 1967), for any null hypothesis

j

H (

mj ≤≤

1), if

m

j p

α. Now we propose a weighted Šidák procedure by using weighted p-values as

follows: given a set of nonnegative weights

independent tests associated with the hypotheses (H1, H2, …, Hm) such that

equation (1) holds (i.e.,

∑

i

m

= 1), for hypothesis Hj (

if

1

) 1 (1

α−−≤

, then reject Hj. The Šidák procedure controls the FWER at level

),,,(

21m

www

L

specified for

−

i w

1

mj ≤≤

1), when wj > 0,

5

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 9

m

w

j

j

p

)1 (1

α

−−≤

or equivalently

()m

w

j

j

p

1

1

1) 1 (

α−≥−

, (4)

then reject the null hypothesis Hj; on the other hand, when wj = 0, do not reject Hj.

In this article we denote the weighted p-value

j

w

j p

1

)1 ( −

as Sj. Therefore, (4) can

be written as Sj ≥(

Theorem 1. Suppose m tests are independent, then the weighted Šidák procedure

controls the family-wise error rate at a significance level α .

Proof. P(failing to reject any true null hypotheses in H0)

(

∏

−−>Ρ

Η∈

0

:

j

Hjj

where, pj follows standard uniform distribution when Hj ∈ H0. Since FWER =1 –

P(failing to reject any true null hypotheses in H0), then Theorem 1 follows. ■

From the Taylor series expansions, we obtain

α

−≤

)m

1

1 α

−

.

=

)

/

)1 (1

j

mw

j p

α

∏

H

−=

Η∈

0

:

/

)1 (

j

j

mw

α

=

mw

j

Hj

j

∑

Η∈

−

0

:)1 (

α

= 1- α,

m

w

j

j

w

m

)1 (1

α

−

.

Based on this inequality, when the same pre-determined weights

are used by the weighted Šidák procedure and the weighted Bonferroni procedure,

if any hypothesis

j

H is rejected by the weighted Bonferroni procedure (i.e.,

α

≤

), then it must be rejected by the weighted Šidák procedure (i.e.,

),,,(

21

m

www

L

jj

w

m

p

m

w

j

j

p)1 ( 1

α−−≤

). Thus, we have Theorem 2.

Theorem 2. For m independent tests, if the same pre-determined weights

),,,(

21

m

www

L

are used in the weighted Šidák procedure and the weighted

Bonferroni procedure, then the weighted Šidák procedure has higher average

power than the weighted Bonferroni procedure.

6

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437

Page 10

Remark 1. If all weights wj = 1, then the weighted Šidák procedure becomes the

Šidák procedure. The weighted Šidák procedure can have higher power than the

Šidák procedure if the weights are selected appropriately.

2.3.2 Optimal Šidák weights

As stated earlier, how to estimate optimal weights by using the prior information

still needs further investigation. Here, we derive the optimal weights that

maximize the average power of the weighted Šidák procedure under the

assumption that the means ,,,(

21

m

μμμ

L

called optimal Šidák weights, which is an extension of the optimal Bonferroni

weights of Wasserman and Roeder (2006).

For any specified weights ,(

21

ww

the single test with hypothesis Hj in the weighted Šidák procedure is

⎛

>−−<Ρ=

jjj

p Power

μα

)

are known. These optimal weights are

),,

m

w

L

, the per-hypothesis power for

.)1 (10)1 (1

1

⎟

⎠

⎟

⎞

⎜

⎝

⎜

⎛

−

⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

−−ΦΦ=

⎟

⎠

⎟

⎞

⎜

⎝

⎜

−

j

m

w

m

w

jj

μα

The average power of the weighted Šidák procedure is

PWaverage

∑

μ

⎟

⎠

⎟

⎞

⎜

⎝

⎜

⎛

−

⎟⎟

⎠

⎞

⎜⎜

⎝

⎛

−−ΦΦ=

>

−

0:

1

2

)1 (1

1

j

j

j

j

m

w

m

μα

.

To find the optimal weights that maximize this average power subject to

constraint of equation (1), Lagrange method was used to obtain conditional

extremum of PWaverage.

Theorem 3. Given FWER being α and known means

independent test statistics (Z1, Z2, …, Zm), the optimal non-negative weights

),,,(

21m

www

L

that maximize the average power of the weighted Šidák

procedure subject to constraint of equation (1) can be obtained by solving

inequalities wi ≥ 0, equations (1) and

()

1 (11ln

i

m

where c is a constant (given in Appendix A).

),,,(

21m

μμμ

L

of the m

()

2

)

2

i

/

1

mw

i

i

w

c

μ

αμα

−−−Φ=−−

−

, for i=1, …, m, (5)

7

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 11

The proof of this theorem is given in Appendix A. The inequalities and equations

can be solved by using the “nlminb()” function in R.

In the following simulation studies we will show that the weighted Šidák

procedure using the optimal Šidák weights can have higher power than the

weighted Bonferroni procedure using the optimal Bonferroni weights and that the

weighted Šidák procedure using the optimal Šidák weights can have much higher

power than the Šidák procedure.

2.4 GS Bonferroni procedure and GS Šidák procedure

Holm (1979) introduced a GS Bonferroni procedure that is a step-down procedure

using ordered weighted p-values. If the (unknown) weights used in the procedure

are estimated appropriately by using prior information, the procedure can have

higher power than the weighted Bonferroni procedure (also see below). In this

section, we first review this GS Bonferroni procedure, and then we propose a GS

Šidák procedure as an extension of the GS Bonferroni procedure.

When assuming that the means of the statistics are known, it is difficult to

derive the optimal weights by maximizing the average power of these GS

procedures as done before for the weighted Bonferroni and the weighted Šidák

procedures. We incorporate the optimal Bonferroni (Šidák) weights described in

Section 2.2 (2.3) into the GS Bonferroni (Šidák) procedure. We will show below

that when these optimal weights are used, the GS Bonferroni (Šidák) procedure

has higher power than the weighted Bonferroni (Šidák) procedure.

2.4.1 GS Bonferroni procedure

Given nonnegative weights (ww,,,

21

L

hypotheses (H1, H2, …, Hm), (note that it is not necessary to satisfy the condition

=

∑

=

i

m

w) for the m tests associated with

1

1

1

−

m

i wm ), if any weight wi = 0, then do not reject the corresponding

hypothesis Hi. For the remaining hypotheses with weights wi > 0, define

i

i

i

w

p

B =

(i = 1, 2, …, m), which are called B-values (i.e., weighted p-values). Let

BBB

≤≤≤

L

be the ordered B-values,

corresponding hypotheses and

) 2() 1 (

,,

ww

Then the GS Bonferroni procedure (Holm, 1979) can be described as follows:

)() 2() 1 ( m)() 2 ( ) 1 (

,,,

m

HHH

L

be the

)(

,

m

w

L

be the corresponding weights.

8

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437

Page 12

Step 1. If

∑

=

i

>

m

i

w

B

1

)(

) 1 (

α

, stop the procedure; otherwise, reject

) 1 (

H and go to the

next step.

…

Step j. If

∑

=

i

>

m

j

i

j

w

B

)(

)(

α

, stop the procedure; otherwise, reject

)( j

H

and go to the

next step.

….

Continue these steps until the procedure is stopped or all B-values have been

processed.

This procedure controls FWER at level α. If we set all weights wj equal to

α

in step j becomes

1, the inequality

∑

=

i

>

m

j

i

j

w

B

)(

)(

1

)(

+−

>

jm

Bj

α

and the GS

Bonferroni procedure becomes the sequential Bonferroni procedure (Holm, 1979).

This GS Bonferroni procedure can have higher power than the sequential

Bonferroni procedure when the weights are chosen properly (Holm, 1979).

Now we compare the power of the GS Bonferroni procedure and the

weighted Bonferroni procedure when the same pre-determined weights are used

in these two procedures. For pre-specified weight (

)() 2() 1 (

,,,

m

www

L

) associated

with hypotheses (

)() 2 ( ) 1 (

,,,

m

HHH

L

) such that

1

1

1

=

∑

=

j

−

m

j

wm(i.e., 1

1

)(

1

=

∑

=

j

−

m

j

wm),

if any false hypothesis H(j) is rejected by the weighted Bonferroni procedure, that

α

≤

)(

is true, then

∑

=

ji

is,

m

Bj

≤

m

i

j

w

B

)(

)(

α

. Since

mw

m

ji

i

∑

=

≤

)(

, we have

∑

=

i

≤≤≤≤≤

m

j

i

j

w

m

BBB

)(

)() 2() 1 (

αα

K

.

Thus, H(j) will also be rejected by the GS Bonferroni procedure. Therefore, we

have Theorem 4.

9

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 13

Theorem 4. Given weights ),,,(

21

m

www

L

for m independent tests such that

1

1

1

=

∑

=

i

−

m

i wm, the GS Bonferroni procedure has higher average power than the

weighted Bonferroni procedure.

2.4.2 GS Bonferroni procedure using the optimal Bonferroni weights

As stated earlier, it is difficult to estimate the optimal weights that maximize the

average power of the GS Bonferroni procedure under the assumption that the

means of statistics are known. Here we propose to use the optimal Bonferroni

weights described in Section 2.2. When these optimal Bonferroni weights are

used, from Theorem 4, we know that the GS Bonferroni procedure has higher

average power than the weighted Bonferroni procedure. Our simulation studies

will confirm this.

2.4.3 GS Šidák procedure

The GS Bonferroni procedure is based on the Bonferroni procedure. As stated

earlier, the Šidák procedure has higher power than the Bonferroni procedure.

Therefore, we propose a GS Šidák procedure.

Given nonnegative weights (

ww,

1

hypotheses ),,,(

21

m

HHH

L

(note that it is not necessary to satisfy the condition

=

∑

=

i

hypothesis Hi. For the remaining hypotheses with weights wi > 0, let

()

ii

pS

1−=

(i = 1, 2, …, m) which are called S-values (i.e., weighted p-values).

Let

)() 2() 1 (

m

SSS

≥≥≥

L

be the ordered S-values,

corresponding hypotheses, and

) 2( ) 1 (

,,ww

The GS Šidák procedure can be described as the following steps.

1

m

w,,

2

L

) for the m tests associated with

1

1

1

−

m

i wm), if any weight wi = 0, do not reject the corresponding null

i w

1

)( ) 2() 1 (

,,,

m

HHH

L

be the

)(

,

m

w

L

be the corresponding weights.

Step 1. If

go to the next step.

….,

Step j. When

∑

=

i

−<

m

i

w

S

1

)(

) 1 (

)1 (

α

, then stop the procedure; otherwise reject

) 1 (

H and

) 1

−

() 1 (

,

j

HH

L

have been tested and rejected: if

∑

=

i

−<

m

j

i

w

j

S

)(

1

)(

)1 (

α

, (6)

10

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437

Page 14

stop the procedure; otherwise reject the hypothesis

…,

Continue these steps until the procedure is stopped or all S-values have been

processed.

Theorem 5. Suppose m tests are independent, then the GS Šidák procedure

controls family-wise error rate at a significant level α .

Proof. Let I0 be the set of index subscripts for the true null hypotheses, I0 = {t: Ht

∈H0}. Let

tt

S

∈

max denote the largest S-value among all St with t∈ I0.

Among the ordered S-values,

) 2( ) 1 (

SS

≥

)( j

H, and go to the next step.

lS

0 I=

0 I

)(m

S

≥≥

L

, suppose at integer k, S(k) =

lS

to a true null hypothesis in H0 (i.e., all the previous k-1 ordered S-values S(1), …,

S(k-1) are corresponding to false null hypothesis), where, 1≤ k ≤ m - m1 + 1, and m1

is the number of true hypotheses. According to the GS Šidák procedure, the event

of failing to reject any true null hypotheses in H0 is equal to event that equation

(6) holds for some j < k. The family-wise error rate is FWER =1 – P(failing to

reject any true null hypotheses in H0), and

P(failing to reject any true null hypotheses in H0)

()

⎜

⎝

j

1

()

=

∏

⎟⎠

0

I

t

() ()

α

∏

⎜⎝

t

∑∑

=

ki

Now we compare the power of the GS Šidák procedure to that of the

weighted Šidák procedure when both procedures use the same weights wj that

=

∑

=

j

1

=

j

0 I, then S(k) is first ordered S-value (from large to small) which is corresponding

()

⎟⎠

⎞

⎜⎝

⎛

−<≥

⎟

⎠

⎞

⎛

−<=

∑

=

i

=

∑

=

i

m

k

i

m

j

i

w

k

k

w

j

SPSP

)()(

/ 1

)(

/ 1

)(

1)1(

αα

U

()()

∏

∈

⎟⎠

⎞

⎜⎝

⎛

−<−

⎞

⎜⎝

⎛

−<=

∑

=

i

∈

∑

=

i

0

I

t

ww

t

w

t

m

k

it

m

k

i

pPSP

)()(

/ 1/ 1/ 1

111

αα

()

αα

−>−=

⎟⎠

⎞⎛

−<−=

∑

=

i

∑∑

=

i

∈

t

∈

1111

)()(

//

m

k

it

m

k

it

wwww

tpP

0

I

0 I

,

where

∈

m

i

t

t

ww

)(

/

0

I

≤ 1, and 1 - pt follows uniform distribution when t∈I0. ■

satisfy 1

1

1

−

m

j

wm (i.e., 1

)(

1

=

∑

−

m

j

wm). If H(j) is rejected by the weighted Šidák

11

Kang et al.: Weighted Multiple Hypothesis Testing Procedures

Published by The Berkeley Electronic Press, 2009

Page 15

procedure, that is, S(j) ≥

m

1

) 1 (

α

−

is true (see inequality (4)), then S(j) ≥

∑

=

−

m

ji

i

w)(

1

) 1 (

α

. Since,

)(

mw

m

∑

=

ji

i

≤

we have

()()

∑

=

−≥−≥≥≥≥

m

ji

i

w

m

j

SSS

)(

1

11

/ 1

)() 2() 1 (

αα

K

.

Thus, H(j) will also be rejected by the GS Šidák procedure, and we have

Theorem 6.

Theorem 6. Given weights ,,,(

21

www

L

=

∑

=

j

Šidák procedure.

Furthermore, we compare the power of the GS Šidák procedure to that of

the GS Bonferroni procedure when the same pre-specified weights are used in

these two procedures.

Theorem 7. For m independent tests, if the same weights

in the GS Šidák procedure and GS Bonferroni procedure, then the GS Šidák

procedure has higher average power than the GS Bonferroni procedure.

Proof. From the definition of B-value (

()

ii

pS1−=

), we know Bi (Si) is a monotonically deceasing (increasing)

function of wi. Suppose that

) 2( ) 1 (

BB

≤≤

associated with the same hypotheses

) 1 (

H

)

m

for m independent tests that satisfy

1

1

1

−

m

j

wm, then the GS Šidák procedure has higher power than the weighted

),,,(

21

m

www

L

are used

iii

wpB/

=

) and S-value

(

i w/ 1

)(m

B

,

L

≤

L

H

and

H

)( ) 2() 1 (m

SSS

≥≥≥

L

are

)() 2(

,,

m

. Below we show that for

any hypothesis

)( j

H, if

∑

=

i

≤

m

j

ij

wB

)()(

/

α

, then

∑

=

i

−≥

m

j

i

w

j

S

)(

/ 1

)

)(

1 (

α

, from which we

know that the GS Šidák procedure has higher power than the GS Bonferroni

procedure.

m

wB

)()(

/

α

, from the Taylor series expansions, we have

If

∑

=

i

≤

j

ij

12

Statistical Applications in Genetics and Molecular Biology, Vol. 8 [2009], Iss. 1, Art. 23

http://www.bepress.com/sagmb/vol8/iss1/art23

DOI: 10.2202/1544-6115.1437