ArticlePDF Available

Determining Power and Sample Size for Simple and Complex Mediation Models

Authors:

Abstract

Mediation analyses abound in social and personality psychology. Current recommendations for assessing power and sample size in mediation models include using a Monte Carlo power analysis simulation and testing the indirect effect with a bootstrapped confidence interval. Unfortunately, these methods have rarely been adopted by researchers due to limited software options and the computational time needed. We propose a new method and convenient tools for determining sample size and power in mediation models. We demonstrate our new method through an easy-to-use application that implements the method. These developments will allow researchers to quickly and easily determine power and sample size for simple and complex mediation models.
POWER FOR MEDIATION MODELS 1
Determining Power and Sample Size for Simple and Complex Mediation Models
Alexander M. Schoemann, East Carolina University
Aaron J. Boulton, University of Delaware
Stephen D. Short, College of Charleston
To appear in Social Psychological and Personality Science
The current version of the article may differ slightly from the published version.
POWER FOR MEDIATION MODELS 2
Abstract
Mediation analyses abound in social and personality psychology. Current recommendations for
assessing power and sample size in mediation models include using a Monte Carlo power
analysis simulation and testing the indirect effect with a bootstrapped confidence interval (e.g.,
Zhang, 2014). Unfortunately, these methods have rarely been adopted by researchers due to
limited software options and the computational time needed. We propose a new method and
convenient tools for determining sample size and power in mediation models. We demonstrate
our new method through an easy-to-use application that implements the method. These
developments will allow researchers to quickly and easily determine power and sample size for
simple and complex mediation models.
Keywords: power, mediation, sample size, R
POWER FOR MEDIATION MODELS 3
Determining Power and Sample Size for Simple and Complex Mediation Models
Mediation analysis has been one of the most popular statistical methods utilized by
social psychologists for decades. For example, a search of articles published in Social
Psychological & Personality Science (SPPS) from January 2010 to September 2016 revealed
208 articles with mediation mentioned within the text of the article. Simply put, if social
psychology and personality researchers are not conducting mediation analyses, they are very
likely encountering the technique in the literature.
Several authors have provided detailed reviews of mediation analysis (Gunzler, Chen,
Wu, & Zhang, 2013; Hayes, 2009; 2013; MacKinnon, 2008; Preacher, 2015; Rucker, Preacher,
Tormala, & Petty, 2011), but discussions on power analysis and sample size calculations for
these models are relatively sparse. Current best practice recommendations for assessing power
and sample size in mediation models are to use a Monte Carlo power analysis (Muthen &
Muthen, 2002; Thoemmes, MacKinnon, & Reiser, 2010) and, preferably, to test the indirect
effect with a bootstrapped confidence interval (e.g., Zhang, 2014). However, this practice may
rarely be adopted by researchers due to limited software options and the long computational
time required. In an editorial, Vazire (2016) highlighted the need for adequately powered
studies to be published in SPPS. With recent increased focus on study replication (Open
Science Collaboration, 2016), and research practices (John, Loewenstein, & Prelec, 2012) in the
social sciences, we find it important to highlight advances in power analysis and sample size
determination for mediation analysis and provide researchers with a new easy-to-use tool to
determine power and sample size for simple and complex mediation models. We begin this
article with a brief review of mediation models and statistical power. Next, we describe our
newly developed application for power and sample size calculations that utilizes the free
POWER FOR MEDIATION MODELS 4
statistical software R (R Core Team, 2016) and provide a brief tutorial for new users. Finally,
we discuss planned extensions to and limitations of our app.
Overview of Mediation Analysis
The simple mediation model involves three measured variables (i.e., X, M, & Y) and
examines if the relation between a predictor variable, X, and an outcome variable, Y, is carried
through one mediating variable M. First, recall from simple regression (Equation 1) that the
outcome variable Y is regressed on the predictor variable X.
1Y
Y cX e
(1)
Here, following mediation analysis labeling conventions1, the slope for X is labeled c and is the
total effect of X on Y. If a researcher is interested in addressing questions of “why” or “how” X
affects Y, then a third variable, M, may be examined as a potential mediator. Figure 1 displays a
simple mediation model which can be represented by the regression equations 2 and 3:
M
M aX e
(2)
'Y
Y c X bM e  
(3)
The direct effect of X on Y is now labeled 𝑐′. The indirect effect of X on Y through M is
quantified as the product of a, the effect of X on M, and b, the effect of M on Y controlling for X.
The total effect, c, is equal to the sum of the direct effect c and indirect effect ab.
Historically, mediation was examined with the “causal steps” approach by satisfying
four criteria described by Baron and Kenny (1986) through a series of regression models.
Although the causal steps approach has been widely popular, research suggests that it is low in
power to detect mediation (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002) and is no
longer considered best practice (Hayes, 2009). Researchers are instead encouraged to examine
the indirect effect, ab.
POWER FOR MEDIATION MODELS 5
The indirect effect can be tested for significance through a variety of methods. Sobel
(1982) proposed a formula for calculating the standard error of ab, seab, which permits
calculation of a z-score statistic (ab/seab) as well as a confidence interval for ab. This method of
testing assumes that the product ab is normally distributed and has been referred to as the
“Sobel test” or “normal theory approach”. However, Bollen and Stine (1990) have noted the
distribution of ab can deviate from normality, particularly in smaller samples, and methods that
do not assume normality of the indirect effect are preferred. Many such approaches have been
proposed (see MacKinnon, 2008, for examples). In this paper, we focus on two methods that do
not make the normality assumption for ab and are considered best practice for testing indirect
effects: bootstrap confidence intervals and Monte Carlo confidence intervals.
Bootstrap confidence intervals do not assume a normal distribution for ab and instead
allow the researcher to empirically generate a sampling distribution for the indirect effect.
Bootstrapping begins with the researcher assuming their collected sample represents the
population that one wishes to make inferences about. A new sample of size N is then collected
by resampling observations with replacement from the original sample. Analyses are conducted
on this new sample to estimate the indirect effect ab. The value of ab is saved and this process
is repeated several times (e.g., 5000) to create a sampling distribution of ab. A confidence
interval for this bootstrapped sampling distribution is calculated and used for statistical
inference. For example, if a researcher were interested in testing whether the indirect effect was
significantly different from zero (i.e., H0: ab = 0) using a 95% confidence interval, then the
2.5th and 97.5th percentiles of the bootstrapped sampling distribution would represent the lower
and upper bounds of the confidence interval for ab. If this confidence interval did not include 0,
then the researcher would reject the null hypothesis. Bootstrap confidence intervals have been a
POWER FOR MEDIATION MODELS 6
popular approach to testing indirect effects; however, they can be computationally intensive,
especially for power analyses. In contrast, Monte Carlo confidence intervals provide a powerful,
accurate test of the indirect effect and are significantly less computationally intensive.
Monte Carlo confidence intervals, also known as parametric bootstrap confidence
intervals, assume normality of the regression coefficients a and b but not the product of the two
terms2 (Preacher & Selig, 2012). To form a Monte Carlo confidence interval, one obtains
estimates of a, b, the variance of each coefficient (the square of a coefficient’s standard error),
and, if possible, the covariance between the coefficients. The regression coefficients a and b are
assumed to be normally distributed with means corresponding to the parameter estimates of
each coefficient and standard deviations corresponding to the standard error of each coefficient.
Values of a and b are randomly drawn from these distributions, multiplied together to form an
estimate of ab, and the process is repeated many times. The results from these random draws
form an empirical sampling distribution of the indirect effect and, much the same as in
bootstrapping, the percentiles of the distribution can be used to form a confidence interval.
Monte Carlo confidence intervals have been shown to perform as well or better than
bootstrap confidence intervals in a variety of situations and models. For example, in a simple
mediation model Hayes and Scharkow (2013) note that the bias-corrected bootstrap confidence
interval was more powerful than a Monte Carlo confidence interval when N < 200, but this
power advantage was due to an increased Type I error rate. Thus, researchers interested
in balancing power and Type I error rate to test the indirect effect should consider the Monte
Carlo confidence interval (Hayes, & Scharkow, 2013). Tofighi and MacKinnon (2016)
examined power and Type I error in a more complex mediation model (i.e., X->M1->M2->M3-
>Y) and note that Monte Carlo and percentile bootstrap methods did not differ in Type I error
POWER FOR MEDIATION MODELS 7
rates or power when N = 200, but the Monte Carlo confidence interval demonstrated more
power when sample sizes were N = 50 and N = 100.
Multiple mediator models. The methods discussed for a simple tri-variate mediation
model can be easily extended to a wide range of models with more than one mediating variable.
Simple, two mediator examples are shown in Figure 2 where multiple mediators can operate in
parallel, Figure 2A, or in sequence, Figure 2B. With multiple mediators, multiple indirect
effects exist in each model, e.g., a1b1 and a2b2 in Figure 2A, and more complex functions of
indirect effects are possible, e.g., a1db1 in Figure 2B or the difference in indirect effects, a1b1-
a2b2, in Figure 2A. Multiple mediator models can be estimated through a series of regression
equations, or through path analysis/structural equation models. All indirect effects or functions
of indirect effects can be tested using bootstrap or Monte Carlo confidence intervals. For further
details on multiple mediator models, the reader is referred to Hayes (2013).
Limitations of mediation. The discussion of mediation thus far has assumed cross-
sectional data. Mediation analyses with cross sectional data have severe limitations. Mediation
analyses imply a causal model with X causing changes in M and, in turn, M causing changes in
Y. With cross sectional data, even if X is experimentally manipulated, determining the causal
ordering of variables is difficult or impossible. Longitudinal data, or application of additional
model assumptions, can lead to stronger causal claims about the relationships between variables
(Cole & Maxwell, 2003; Preacher, 2015). Authors of SPPS articles employing mediation
analyses are encouraged to carefully evaluate causal language used in their article and note
possible causal limitations (McConnell, 2013). However, one could argue that without sufficient
statistical power to detect mediation effects, concerns about the causal interpretation of effects
are misplaced.
POWER FOR MEDIATION MODELS 8
Overview of Statistical Power
Statistical power is defined as the probability of rejecting the null hypothesis (H0) given
that H0 is false in the population. Ensuring a study has adequate power is critical for drawing
conclusions from data. If a study is lacking in power the conclusions that can be drawn if H0 is
not rejected are limited. Specifically, in a low-powered study, failing to reject H0 may be due to
the absence of an effect (H0 is true in the population) or it may be due to lack of power (the
alternative hypothesis, H1, is true in the population). The power of a given study is primarily
affected by three components: effect size (ES), Type I error rate (α), and sample size (N). Power
analysis and sample size determination are based on the fact that if three of the four quantities
(power, ES, α, and N) are known, the fourth can be computed. For example, if ES, α, and N are
known, power can be computed, a procedure often used for post-hoc power analysis. Sample
size can be determined by specifying α, ES, and power.
Methods of power analysis. Traditionally, power analyses and sample size
determination have been based on analytic methods. To determine power using analytic
methods, values of ES, α, and N are used to construct distributions of the test statistic of interest
(e.g., t statistics) consistent with H0, and consistent with H1. Power is the proportion of the
distribution consistent with H1 that exceeds the critical value under H0. In Figure 3, t
distributions consistent with H0 and consistent with H1 (where under H1 d = 1.0) with 15
degrees of freedom are shown. The vertical line represents the two-tailed critical value under H0
and the shaded portion of the H1 distribution is the proportion of the distribution that is greater
than the critical value. In this example, power, the proportion of the H1 distribution above the
critical value, is .46. To determine the sample size needed to achieve a desired level of power
POWER FOR MEDIATION MODELS 9
(e.g. .80), researchers would repeat the power analysis, varying only sample size until the
desired level of power is achieved.
Power analysis methods using analytic methods have been applied to many types of
statistical models and research designs including linear regression, the generalized linear model
(Faul, Erdfelder, Buchner, & Lang, 2009), randomized trials and cluster randomized trails
(Spybrook et al., 2011), and structural equation models (Satorra & Saris, 1985). In addition,
many of these methods have been implemented in user-friendly software such as G*Power
(Faul et al., 2009) and Optimal Design (Raudenbush et al., 2011). Power analyses using analytic
methods provide accurate estimates of power and, for many simple research designs, are easily
and quickly implemented. However, power analyses using analytic methods only cover a small
portion of possible analyses (e.g., t-tests, ANOVA, correlation, regression), and even in these
cases researchers are often forced to make possibly unrealistic assumptions (e.g., equal group
sizes, no missing data). When study designs or analyses are complex (e.g., mediation models
with bootstrapping) analytic methods are often not available and a Monte Carlo simulation
approach to power analysis is preferred3.
Power analysis based on Monte Carlo simulations. The idea behind the Monte Carlo
simulation approach to power analysis is straightforward. Because power is the probability of
rejecting H0 given H1 is true, if one can draw a large number (e.g., 5000) of random samples
(replications) from the population defined by H1 and fit the hypothesized model (e.g., a
regression equation) on the samples, power can be estimated as
/rR
, the number of samples
that reject H0 (r) divided by the total number of samples (R). Monte Carlo simulations have
several advantages over traditional power analysis methods based on analytic methods. First,
they allow researchers to specify the values of all parameters in a statistical model, thereby
POWER FOR MEDIATION MODELS 10
equating the power analysis and data analysis models for a more specific assessment of power.
Second, power estimates can be obtained for multiple parameters in a single model. Third,
greater flexibility in the specification of model assumptions (e.g., missing data) is permitted,
which ideally are matched to the conditions under which a study is expected to take place.
Finally, the number and types of models for which power simulations can be conducted are
practically limitless. Indeed, for complex models such as mediation models, Monte Carlo power
analysis may be the only method available to estimate statistical power.
To determine an appropriate sample size for a proposed study using Monte Carlo power
analyses, a researcher needs to draw many random samples under the population model (defined
by the researcher) with different sample sizes until he or she finds the sample size that yields the
desired level of power. This process can become extremely tedious and time consuming,
especially for models that are computationally intensive (e.g., mediation models using
bootstrapping). Fortunately, a new method of power analysis based on varying sample size
across replications can alleviate some of these limitations (Schoemann, Miller, Pornprasermanit,
& Wu, 2014).
In a traditional Monte Carlo power simulation, all simulation parameters (e.g., N) are
static across all replications (e.g., all replications have the same N). Power is estimated by the
proportion of significant replications to the total number of replications and can only be
computed for a single sample size at a time. In other words, one has to run the simulation again
to know the power associated with a different sample size. Conversely, with a varying
parameters approach, the design parameters (e.g., N) can take on a different set of values for
each replication, and these parameters can either vary randomly or increase by small increments
over a range of specified values. Power from the simulation is then analyzed with a regression
POWER FOR MEDIATION MODELS 11
model. Specifically, the significance of a parameter (coded as 0 = not significant, 1 =
significant) computed from each replication serves as the outcome variable in a logistic
regression analysis in which it is predicted by N. The estimated logistic regression equation can
then be used to predict power from any sample size (within the specified range) without re-
running the simulation. This general approach allows researchers to run a single Monte Carlo
simulation (albeit one with many replications) and compute power for a specific sample size,
compute power for several sample sizes, or plot power curves over a range of sample sizes.
Power for mediation models. Despite the popularity of mediation models, determining
appropriate power or sample size for one or more indirect effects is not straightforward.
Guidelines for sample size in mediation models exist (e.g., Fritz & MacKinnon, 2007) but they
provide guidance for a limited range of models and analytic conditions. Alternatively,
researchers could attempt to determine power or sample size for each component of an indirect
effect (e.g., a and b) and use the smallest power or largest sample size from these analyses when
planning a study. This approach would entail using traditional power software such as G*Power
(Faul et al., 2009) to determine the required sample size for a and b and using the larger of the
two sample sizes. However, this approach will systematically underestimate the sample size
needed to test the indirect effect, and does not generalize to quantities from complex mediation
models (e.g., a1b1 a2b2). The online application WebPower (Zhang, & Yuan, 2015) can be
used to determine power based on the Sobel test for simple mediation models, and a path
diagram-based method for assessing power for complex mediation models. However, this
application does not include the ability to assess power via Monte Carlo confidence intervals or
bootstrapping.
POWER FOR MEDIATION MODELS 12
Monte Carlo power analyses are best practice for determining power and sample size in
mediation models, but currently available software, though extremely flexible, has several
limitations. Implementing a Monte Carlo Power analysis for mediation models requires
knowledge of specific software (e.g., R, or Mplus), is computationally intensive (Zhang, 2014),
and can prove difficult as users must specify all population parameters for a specific model of
interest. We offer an application which is user-friendly, requires no specific programming
knowledge, estimates power for the indirect effect(s) quickly, and provides an easy interface for
specifying population parameters.
Application for Monte Carlo Power Analysis for Mediation Models
To facilitate use of the power analysis method based on Monte Carlo confidence
intervals described above, we created a freely available application written in the R statistical
computing language (R Core Team, 2016). In this section, we provide a tutorial on how to use
the app to conduct a power analysis for a simple mediation model. The app employs an easy-to-
use graphical user interface. Users may access the app by visiting
https://schoemanna.shinyapps.io/mc_power_med/ or downloading it from within R. We
recommend the latter as it will run faster on one’s local machine. To download the app, users
must have R as well as the shiny and MASS add-on packages installed4. Once installed, the
user opens an R session and runs the following command:
library(shiny)
runGitHub("mc_power_med", "schoam4")
POWER FOR MEDIATION MODELS 13
The first command loads routines from the shiny add-on package that are needed to run the
app in the user’s current R session. The second command downloads the app and opens it in a
user’s default web browser program5.
Once the app is running, the user is presented with a variety of program options, shown
in Figure 4a below. Starting at the top of the options menu, the user must first select the
mediation model to be used in the power analysis. At the time of this writing, only two models
one mediator and two parallel mediators are available, though more will be made available in
future releases. For this tutorial, we will be calculating power for the default option, the simple
tri-variate mediation model shown in Equations 13. Note that a path diagram of the selected
model will appear to the right of the options menu when selected (Figure 5). Next, the user must
select the objective of their power analysis. Currently, two options are offered: (a) “Set N, Find
Power”, which calculates the statistical power for an indirect effect(s) under the chosen model
specification and target sample size, or (b) “Set Power, Vary N”, that uses the varying sample
size approach to calculate the sample size required to achieve a specific level of power
designated by the user along with a range of sample sizes for a target indirect effect(s). If
option (a) is selected, only the target sample size is required to be entered by the user (Figure
4a). If option (b) is selected, the user is presented with a submenu of additional options, shown
in Figure 4b.
For the present example, we will select option (b). In the submenu, the user must set the
target power level, the minimum and maximum N for the range of sample sizes considered, and
increments of N to calculate power estimates for within the specified range. For this tutorial, we
have selected the conventional power level of .80, a minimum sample size of 50, a maximum
sample size of 200, and a step size of 10. Note that smaller step sizes combined with wider
POWER FOR MEDIATION MODELS 14
ranges will require more computation time; thus, the user might opt to specify a large range of
sample sizes with a large step size in a preliminary analysis and subsequently narrow the range
and decrease the step size in additional runs for more precise sample size estimates.
The remaining options shown in Figure 4a are typical parameters that need to be set for
Monte Carlo power analyses (Muthén & Muthén, 2002). First, the total number of replications
need to be selected. This number is typically 1000 or greater, although little published guidance
on the number of replications needed in simulation studies exists. Mundform et al. (2011)
provided empirically-based recommendations, suggesting that 5000 may be enough for many
applications. Ultimately, the number of replications should be sufficient to ensure stable power
or sample size estimates, and that number will depend on the modeling context. Therefore, it is
recommended that the user run the power analysis at least twice with differing numbers of total
replications (e.g., 5000, 10000) to ensure that the final estimate(s) has converged to a stable
value(s). The next option, “Monte Carlo Draws per Rep”, refers to the number of times each
target coefficient is sampled from its sampling distribution within each power analysis
replication to calculate the Monte Carlo confidence interval(s). Once again, published
recommendations are scarce: we note only that several thousand draws are likely needed, such
as 20,000, which was chosen for the empirical examples discussed in Preacher and Selig (2012).
The logic presented above for the total number of replications also applies here such that
conducting many runs with increasing values can reassure the user that estimates are stable. In
our running example, we chose values of 5000 and 20000 for the total number of power
analysis replications and the number of coefficient draws per replication, respectively. The final
two options are the random number generator seed and the confidence interval width. The seed
should be a positive integer and ensures results from a run of the app are replicable. A
POWER FOR MEDIATION MODELS 15
researcher using the same seed and parameter values will replicate another researcher’s results,
whereas a different seed may lead to slightly different results. The default seed in our
application, 1234, was used for this example. The confidence interval width [100(1 - α)%] sets
the width of the confidence intervals for all indirect effects calculated within each replication. In
our example, the width is set to 95% (corresponding to α = .05).
Once all options for the power analysis are set, the user must input population
parameters for the model, akin to choosing an effect size in a traditional power analysis.
Specifically, the information entered, in one form or another, must be sufficient for calculation
of the hypothesized indirect effect and its associated confidence interval; at a minimum, this
implies the hypothesized a coefficient(s), b coefficient(s), and the coefficient standard errors for
the simple mediation model. There are a few different quantities that will meet this criterion,
including model parameter estimates (Zhang, 2014; Selig & Preacher, 2008), measures of
variance explained (Thoemmes, MacKinnon, & Reiser, 2010), and correlation or covariance
matrices. The default option in the app is to enter a correlation matrix and, if applicable, the
standard deviations of the variables, which are used to transform the correlation matrix to a
covariance matrix. In the running example, suppose we have found in previous studies or meta-
analyses that our focal predictor X correlates with the mediator M at approximately .35, M
correlates with the outcome variable Y at approximately .25, and the X and Y variables correlate
at approximately .10. Additionally, prior research has found the standard deviations of X, M,
and Y to be 1.00, 1.50, and 2.00, respectively. In the middle column of the app (Figure 5), we
enter this information in the appropriate boxes, which change responsively to the model and
input method selected.
POWER FOR MEDIATION MODELS 16
Now that the program options and hypothesized model have been fully specified in our
example, we click the “Calculate Power” button on the right-side of the app. If any errors were
made in the previous steps, the program will terminate and an error message will appear below
the button. If this occurs, users should change the relevant input and press the button again.
Once the app begins to run, a progress bar will appear. If the power analysis calculations
terminate successfully, output will appear below the button. In our running example, the app
took approximately 52 seconds to run. Using the continuously varying sample size approach to
Monte Carlo power analysis approximately 150 individuals are required to ensure statistical
power is at least 80% for detecting the hypothesized indirect effect.
The application remains in development and we foresee several extensions in new
releases, detailed next.
Current Limitations and Potential Extensions
Many extensions to the existing app are possible. Foremost, including a larger number
of models than those currently offered would considerably improve the flexibility of the app.
Due to the large number of models circulating in social psychology journals, including all
possible models is difficult. Implementing models that are the most common in and relevant to
social psychology research, such as dyadic mediation models (Ledermann, Macho, & Kenny,
2011), longitudinal mediation models (Selig & Preacher, 2009), or models combining mediation
and moderation (Hayes, 2013), is our priority. Moreover, missing data is pervasive in
psychological research and reduces statistical power in addition to other potentially harmful
consequences (Enders, 2010). Permitting missing data in the calculation of power would
promote more accurate sample size estimates. Finally, non-normal variables have to potential to
provide inaccurate power estimates (Zhang, 2014). Extending the app to allow for the
POWER FOR MEDIATION MODELS 17
specification of non-normal variables, which tend to be normative in psychology (Micceri,
1989), would also enhance the accuracy of the app. Extensions and news about the development
of the app will be posted on the following webpage: http://marlab.org/power_mediation/.
Conclusion
Mediation analysis is a popular tool for social and personality psychologists, but for
mediation to be an effective tool, researchers must plan studies with sufficient statistical power.
Accurately determining statistical power for mediation models can be tricky for applied
researchers, especially when using bootstrapping or Monte Carlo confidence intervals to test the
indirect effect. We have developed an application which makes determining power or sample
size for mediation models relatively straightforward. By utilizing a simple interface, population
parameters expressed as correlations, and varying sample sizes within a power analysis, our
application provides social and personality psychologists with a powerful, easy-to-use tool to
aid in study planning when mediation is of interest.
POWER FOR MEDIATION MODELS 18
Notes
1 Following the conventions in Hayes (2013), all regression equations omit the intercept. The
inclusion of an intercept will not affect any tests of the indirect effect.
2 The assumption of the normality of regression coefficients in a Monte Carlo confidence
interval is the same assumption made when using Wald tests with z values to determine
statistical significance of a single regression coefficient. Thus, Monte Carlo confidence intervals
are appropriate in any situation where interpretation of the Wald tests statistic is warranted.
Furthermore, simulation studies (reported in the technical appendix for the app at
http://marlab.org/technical_appendix/) have demonstrated that Monte Carlo confidence intervals
perform as well as bootstrap confidence intervals when estimating power under normal and
non-normally distributed variables.
3Two procedures used in this paper contain the term Monte Carlo. To clarify between them,
Monte Carlo confidence intervals are used to test model parameters (e.g., indirect effects),
Monte Carlo power analyses are used to determine power to reject H0 for a parameter(s) in a
statistical model.
4 R can be downloaded from https://cran.r-project.org/. R includes some basic functionality, but
the majority of routines used in R are included in add-on software packages developed by
independent contributors. For instance, the shiny package, which is required to download and
run the app, provides functionality that allow users to easily deploy R-based applications to the
internet. Expertise in R programming is not required to be able to download and run the app as
detailed in this paper; numerous web pages and books are easily located online for readers
seeking additional information on R.
POWER FOR MEDIATION MODELS 19
5 Although the app opens in an internet browser, it is technically “offline” – that is, the app is
running locally on one’s machine from files downloaded via the runGitHub command as
opposed to running on a web server. For the best experience using the app, we recommend
maximizing the web browser to fill the screen. Smaller browser sizes may make input boxes
difficult to read.
POWER FOR MEDIATION MODELS 20
References
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of
Personality and Social Psychology, 51, 1173-1182.
Bolger, N., & Laurenceau, J. P. (2013). Intensive longitudinal methods: An introduction to diary
and experience sampling research. New York, NY: Guilford Press.
Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and bootstrap estimates
of variability. Sociological Methodology, 20, 115-140.
Cole, D. A., & Maxwell, S. E. (2003). Testing mediational models with longitudinal data:
Questions and tips in the use of structural equation modeling. Journal of Abnormal
Psychology, 112, 558-577.
Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analysis using
G*Power 3.1: Tests for correlation and regression analyses. Behavioral Research
Methods, 41, 49-60.
Fritz, M. S., & McKinnon, D. P. (2007). Required sample size to detect the mediated effect.
Psychological Science, 18, 233-239.
Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new
millennium. Communication Monographs, 76, 408-420.
Hayes, A. F., (2013). Introduction to mediation, moderation, and conditional process analysis:
A regression based approach. New York: Guilford Press.
POWER FOR MEDIATION MODELS 21
Hayes A. F., & Scharkow, M. (2013). The relative trustworthiness of inferential tests of the
indirect effect in statistical mediation analysis: Does method really matter?
Psychological Science, 24, 19181927.
Gunzler, D., Chen, T., Wu, P., & Zhang, H. (2013). Introduction to mediation analysis with
structural equation modeling. Shanghai archives of psychiatry, 25, 390-395.
John, L. K., Lowenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable
research practices with incentives for truth telling. Psychological Science, 23, 524-532.
Ledermann, T., Macho, S., & Kenny, D. A. (2011). Assessing mediation in dyadic data using
the actor-partner interdependence model. Structural Equation Modeling: A
Multidisciplinary Journal, 18, 595-612.
MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor &
Francis.
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A
comparison of methods to test the significance of the mediated effect. Psychological
Methods, 7, 83-104.
McConnell, A. R. (2013). Editorial. Social Psychological and Personality Science, 4, 3-5.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures.
Psychological bulletin, 105, 156-166.
Mundform, D. J., Schaffer, J., Kim, M. J., Shaw, D., Thongteeraparp, A., & Supawan, P.
(2011). Number of replications required in Monte Carlo simulation studies: A synthesis
of four studies. Journal of Modern Applied Statistical Methods, 10, 4.
Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample
size and determine power. Structural Equation Modeling, 9, 599-620.
POWER FOR MEDIATION MODELS 22
Open Science Collaboration (2015). Estimating the reproducibility of psychological science.
Science 349: 943.
Preacher, K. J. (2015). Advances in mediation analysis: A survey and synthesis of new
developments. Annual Review of Psychology, 66, 825-852.
Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for
indirect effects. Communication Methods and Measures, 6, 77-98.
Raudenbush, S. W., et al. (2011). Optimal Design Software for Multi-level and Longitudinal
Research (Version 3.01) [Software]. Available from https://www.wtgrantfoundation.org
Rucker, D. D., Preacher, K. J., Tormala, Z. L., & Petty, R. E. (2011). Mediation analysis in
social psychology: Current practices and new recommendations. Social and Personality
Psychology Compass, 5, 359-371.
R Core Team (2016). R: A language and environment for statistical computing (version 3.3.0)
[Computer software]. Vienna, Austria: R Foundation for Statistical Computing.
Retrieved from the comprehensive R archive network (CRAN): https://www.R-
project.org/
Satorra, A., & Saris, W. E. (1985). Power of the likelihood ratio test in covariance structure
analysis. Psychometrika, 50, 83-90
Selig, J. P., & Preacher, K. J. (2008, June). Monte Carlo method for assessing mediation: An
interactive tool for creating confidence intervals for indirect effects [Computer
software]. Available from http://quantpsy.org/
Selig, J. P., & Preacher, K. J. (2009). Mediation models for longitudinal data in developmental
research. Research in Human Development, 6, 144-164.
POWER FOR MEDIATION MODELS 23
Schoemann, A. M., Miller, P. M., Pornprasermanit, S. & Wu, W. (2014). Using Monte Carlo
simulations to determine power and sample size for planned missing designs.
International Journal of Behavioral Development, 38, 471-479.
Sobel, M. E. (1982). Asymptotic intervals for indirect effects in structural equations models. In
S. Leinhart (Ed.), Sociological Methodology 1982 (pp.290-312). San Francisco: Jossey-
Bass.
Spybrook, J., Bloom, H., Congdon, R., Hill, Cl., Martinez, A., & Raudenbush, S. (2011). Optimal
Design for Longitudinal and Multilevel Research: Documentation for the Optimal Design
Software Version 3.0. Available from www.wtgrantfoundation.org.
Thoemmes, F., MacKinnon, D. P., & Reiser, M. R. (2010). Power analysis for complex
mediational designs using Monte Carlo methods. Structural Equation Modeling, 17,
510-534.
Tofighi, D., & MacKinnon, D. P. (2016). Monte Carlo confidence intervals for complex
functions of indirect effects. Structural Equation Modeling: A Multidisciplinary
Journal, 23, 194-205.
Vazire, S. (2016). Editorial. Social Psychological and Personality Science, 7, 3-7.
Zhang, Z. (2014). Monte Carlo based statistical power analysis for mediation models: Methods
and software. Behavior Research Methods, 46, 1184-1198.
Zhang, Z., & Yuan, K. H. (2015). WebPower: Statistical power analysis online. Retrieved from
http://webpower.psychstat.org.
POWER FOR MEDIATION MODELS 24
Figure 1. Simple Mediation Model
POWER FOR MEDIATION MODELS 25
Figure 2: Multiple Mediator Models
POWER FOR MEDIATION MODELS 26
Figure 3: t distribution under H0 and H1
POWER FOR MEDIATION MODELS 27
Figure 4: Application Options
(a) (b)
(a) Primary app options menu. (b) App options for objective “Set Power, Vary N”.
POWER FOR MEDIATION MODELS 28
Figure 5: App model input section.
... was .14. To estimate the minimum indirect association in a model with three parallel mediators, we performed Monte Carlo simulations (Schoemann et al., 2017). Assuming correlations of .40 ...
... Monte Carlo simulations assuming a correlation of .40 between willingness to sacrifice and commitment (Van Lange et al., 1997) and correlations of .19 (median average correlation in psychology studies; Stanley et al., 2018) between NFC and willingness to sacrifice as well as commitment revealed that the sample size afforded 99% power to detect an indirect association (Schoemann et al., 2017). ...
... was .12. For the mediation model with two parallel mediators, we performed Monte Carlo simulations (Schoemann et al., 2017). Similar to Study 2, assuming correlations of .40 ...
Article
Full-text available
Decision making is a fundamental aspect of relationships, as romantic partners often evaluate costs and benefits of their relationship and contemplate whether to maintain it. A key factor that has repeatedly been shown to influence decision-making processes is need for closure, that is, the desire to quickly obtain a definitive answer or solution. Individuals who strongly desire closure prefer the status quo, seek familiarity, and dislike uncertainty, which might strengthen their intention to maintain an enduring relationship. Across five studies recruiting university students and community samples, we explored, for the first time, the role of need for closure in relationship commitment. In Study 1, individuals with a higher need for closure reported greater commitment to their romantic relationships. Studies 2a and 2b replicated these findings and showed that relationship satisfaction and investment, but not alternative quality, may explain the link between need for closure and commitment. Study 3 revealed that need for closure was linked with willingness to sacrifice-a downstream consequence of commitment. Using a three-wave longitudinal design, Study 4 found that within-person variation in need for closure was small, and need for closure explained mainly between-(but not within-) person differences in commitment. This study failed to replicate the mediating role of relationship satisfaction and investment observed in Studies 2a and 2b, thereby indicating caution in interpreting mechanisms suggested by our prior studies. Overall, the findings demonstrated a consistent between-person association of need for closure with relationship commitment, and revealed important research avenues to further understand how individual differences in decision making predict commitment.
... Given the moderated mediation design, we used Monte Carlo power analysis for indirect effect (Schoemann et al. 2017) which recommended a sample size of 232 for 80% power. We recruited 312 British traditional luxury consumers (Mage = 44.41; ...
Article
Full-text available
Luxury brands have traditionally embodied exclusivity and status, yet increasing accessibility is transforming consumers' perceptions. This trend, driven by the rapid democratization of luxury goods, is causing a paradigm shift in how traditional luxury consumers ascribe value to luxury brands. While these strategies may boost short‐term sales, they risk eroding brand positioning and alienating core consumers. Will traditional luxury consumers wear democratized brands with pride as accessibility and availability increase? Will they continue purchasing democratized luxury used by lower‐status consumers? Guided by network effects theory, we show across four mixed‐methods studies that democratization reduces purchase intentions—particularly when low‐status consumers adopt the brand—and increases abandonment intentions. This study advances luxury branding literature by identifying democratization as a novel negative network externality influencing luxury consumption. We reveal consumer's pride of ownership as a psychological mediator which diminishes when democratization erodes exclusivity. Further, we establish boundary conditions by revealing how status‐based consumption and rarity principles influence consumer responses and can moderate democratization's effects on purchase and abandonment intentions. Managerially, our findings will help brands to counteract democratization's potential risks through status and rarity‐driven campaigns, such as scarcity‐based strategies, limited editions or premium‐tier differentiation.
... To determine the sample size needed for the study, an a priori Monte Carlo power analysis for mediation models (Schoemann et al. 2017) was conducted. We considered a mediation model with two mediators, assuming a small-to-medium effect size (0.25; Cohen 2013) for all the paths (a1, b1, a2, b2) and a smallsized (0.20) effect between the two mediators (d) and for the direct effect (ca). ...
Article
Full-text available
Artificial intelligence (AI) has increasingly integrated into daily life, with numerous industries adopting AI‐driven systems to enhance services and automate repetitive tasks. The present work examines for the first time the short‐term effects of interacting with an AI‐based agent in the work domain on self‐efficacy, self‐objectification and beliefs in free will. In the first and second studies, scenarios describing the process of evaluating candidates for a job position were used to test the effects of interacting with an AI agent (vs. a human recruiter) on self‐efficacy, self‐objectification and beliefs in free will. In the third study, the experimental manipulation was carried out by adopting a real AI‐based recruiting system to foster greater ecological validity. Findings consistently show that being evaluated by an AI recruiter (vs. a human recruiter) significantly lowers self‐efficacy beliefs and increases self‐objectification, leading, in turn, to a reduction in beliefs in free will. These results provide new insights into the workplace's self‐objectification process, indicating how it might be triggered during interactions with modern AI technologies. Considerations on adopting AI technologies in the work domain are discussed, emphasising the need for AI systems to support, rather than replace, human agency.
... To balance the power and Type I error rates, Monte Carlo confidence intervals were used to evaluate the significance of the indirect effect, a method comparable to the bootstrap method in certain studies. 38,39 ...
Article
Full-text available
Purpose College freshmen face a heightened risk of mental health problems. Recent research has extensively explored attachment and growth mindset as key factors that influence mental health. However, the specific relationship between attachment and growth mindset remains unclear. This study aimed to investigate how attachment impacts the growth mindset of college students and explore the relationship between growth mindset, attachment, and mental health. Methods A survey of 684 college freshmen (80.1% female, average age 18.4) at a university in Huzhou was conducted using the Growth Mindset Scale, Adult Attachment Scale, and Symptom Checklist (SCL-90) and a demographic questionnaire to collect basic information about participants. Results The findings show that attachment security positively predicted growth mindset (β=0.16), whereas attachment insecurity negatively predicted growth mindset (β=−0.22). The research highlights that a growth mindset mediates the association between attachment security and mental health, and between attachment insecurity and mental health. Conclusion The results highlight the importance of fostering a secure attachment style and promoting a growth mindset as key factors for improving mental health. These findings suggest that incorporating growth mindset strategies into mental health education and interventions could help students with diverse attachment styles enhance resilience and better manage psychological challenges.
... Using this as a standard, we utilized the G-Power software for estimation, where the parameters that need to be precisely set include effect size, α error, power 1-β, number of groups, number of measurements, intra-class correlation coefficient, and sphericity. Specifically, for correlation analysis, we set p to 0.3 [53]; the α error rate was set to 0.05, and the statistical power 1-β was set to 0.8 [54]; calculations show that a chain mediation model [55] requires at least a sample size of 370 to achieve a statistical validity of 0.80. However, considering the potential subject attrition and invalid questionnaires that may occur during actual surveys, we assumed a non-effect response rate of 15%. ...
Article
Full-text available
Purpose With the advancement of technology and widespread smartphone use, addiction to these devices has escalated, particularly among college students. This issue transcends mere habit, impacting physical, psychological, and social well-being. Prolonged screen exposure and excessive app engagement contribute to vision and hearing deterioration, alongside heightened psychological stress and diminished social skills. The dual-process theory offers a unique lens to explore the intricate dynamics of smartphone addiction. Incorporating physical activity as a healthy lifestyle choice can bolster self-control and mitigate the allure of smartphones by enhancing physical engagement. Methods A total of 559 college students from two universities, Wuhan University of Science and Technology and Central China Normal University, were surveyed using the Physical Activity Scale, Stress Perception Scale, self-control Scale, and smartphone Addiction Burnout Scale. The questionnaire was statistically analyzed using SPSS 27.0 statistical analysis software. Correlation analysis, regression analysis, and mediation model were used to evaluate the relationships among physical activity, self-control, stress perception, and smartphone addiction among college students. Results Physical activity was a significant negative predictor of smartphone addiction among college students(β = -0.038, p < 0.001), and with the addition of the intermediate variables (self-control and stress perception), physical activity remained a significant negative predictor of smartphone addiction among college students (β = -0.017,p < 0.01). Conclusion Physical activity and self-control negatively predicted smartphone addiction, and stress perception was a negative predictor of smartphone addiction;self-control mediates in physical activity and smartphone addiction;stress perception mediates physical activity and smartphone addiction;self-control and stress perception play a chain mediating role in physical activity and smartphone addiction.
... The sample size was determined using the Monte Carlo Power Analysis for Indirect Effects program of the Two Serial Mediators Model 35 . Based on the correlation effects of the initial pilot study results, the target power of 0.9, a significance level of 0.05, and the overall standard deviation of 0.25, the required sample size was 526. ...
Article
Full-text available
The study aims to explore the relationship between mindfulness, rumination thinking, anxiety state, and depressed mood, and the chain mediating roles of rumination thinking and anxiety state in explaining how mindfulness influences depressed mood in infertile women. This cross-sectional study included 946 women with infertility from a maternal and child hospital in western China through convenience sampling. Five Facet Mindfulness Questionnaire (FFMQ), Rumination Response Scale (RRS), Self-rating Anxiety Scale (SAS), and Self-rating Depression Scale (SDS) were measured as outcome indicators. SPSS PROCESS macro program was used to test for chained mediating effects and the significance using the Bootstrap method. The total effect of mindfulness on depressed mood was -0.390 with the direct path effect of -0.170. The total indirect path effect was -0.220, which accounted for 56.4% of the total effect, and that the chain mediated path (FFMQ→RRS→SAS→SDS) effect was significant with a mediation effect value of -0.075. Mindfulness can not only directly affect infertile women’s depressed mood, but also indirectly affect that through the chain-mediated effects of rumination thinking and anxiety state.
Article
Background The significance of the clinical learning environment for professional identity is increasingly acknowledged. However, few studies have explored the bidirectional effects of the clinical learning environment on professional identity. Objective This study aimed to examine the relationship between the clinical learning environment and the professional identity of nursing interns, particularly the parallel mediating roles of perceived professional benefit and role stress. Methods The study was conducted in three first-class general hospitals in China. A total of 377 nursing interns participated. A questionnaire was administered, which included socio-demographic information, the Clinical Learning Environment Scale, the Nurses’ Perceived Professional Benefit Scale, the Role Stress Scale, and the Professional Identity Questionnaire. T-tests, analysis of variance, and multiple linear regression were used to analyze the influencing factors of professional identity. And correlation analysis and mediation tests were performed to explore the factors influencing professional identity and potential mediating roles of perceived professional benefit and role stress. Result Professional identity was positively related to the clinical learning environment ( r = 0.533, p < 0.01), perceived professional benefit ( r = 0.643, p < 0.01) and negatively related to role stress ( r = −0.348, p < 0.01). Furthermore, perceived professional benefit and role stress played parallel mediators between the clinical learning environment and professional identity. Conclusion This study emphasized the importance of focusing more on enhancing nursing students’ sense of professional benefit and reducing role stress in the clinical learning environment to promote their professional identity.
Article
Existing research confirms that there is a relationship between school attachment and subjective well‐being. However, the mechanisms mediating the relationship between these two variables still need to be explained. This study aims to examine the role of self‐efficacy belief and academic motivation in the relationship between school attachment and subjective well‐being among middle school students. The study group of this cross‐sectional study, designed in accordance with the relational design of quantitative research methods, consists of 701 middle school students (Mage = 11.81, 55.3% girls). The data were collected using the School Attachment Scale for Children and Adolescents, Adolescent Subjective Well‐being Scale, Self‐Efficacy Scale for Children and Academic Motivation Scale for Middle School Students. According to the research findings, positive significant relationships were determined between school attachment, self‐efficacy belief, academic motivation and subjective well‐being. The structural equation model results show that self‐efficacy belief and academic motivation mediate the relationship between school attachment and subjective well‐being. Results highlight the importance of considering elements of school attachment, self‐efficacy, and academic motivation when designing interventions that promote subjective well‐being in middle school students.
Article
Full-text available
Monte Carlo simulations are used extensively to study the performance of statistical tests and control charts. Researchers have used various numbers of replications, but rarely provide justification for their choice. Currently, no empirically-based recommendations regarding the required number of replications exist. Twenty-two studies were re-analyzed to determine empirically-based recommendations.
Article
Full-text available
One challenge in mediation analysis is to generate a confidence interval (CI) with high cov- erage and power that maintains a nominal significance level for any well-defined function of indirect and direct effects in the general context of structural equation modeling (SEM). This study discusses a proposed Monte Carlo extension that finds the CIs for any well-defined function of the coefficients of SEM such as the product of k coefficients and the ratio of the contrasts of indirect effects, using the Monte Carlo method. Finally, we conduct a small-scale simulation study to compare CIs produced by the Monte Carlo, nonparametric bootstrap, and asymptotic-delta methods. Based on our simulation study, we recommend researchers use the Monte Carlo method to test a complex function of indirect effects.
Article
Full-text available
Empirically analyzing empirical evidence One of the central goals in any scientific endeavor is to understand causality. Experiments that seek to demonstrate a cause/effect relation most often manipulate the postulated causal factor. Aarts et al. describe the replication of 100 experiments reported in papers published in 2008 in three high-ranking psychology journals. Assessing whether the replication and the original experiment yielded the same result according to several criteria, they find that about one-third to one-half of the original findings were also observed in the replication study. Science , this issue 10.1126/science.aac4716
Article
Full-text available
In this article, we attempt to distinguish between the properties of moderator and mediator variables at a number of levels. First, we seek to make theorists and researchers aware of the importance of not using the terms moderator and mediator interchangeably by carefully elaborating, both conceptually and strategically, the many ways in which moderators and mediators differ. We then go beyond this largely pedagogical function and delineate the conceptual and strategic implications of making use of such distinctions with regard to a wide range of phenomena, including control and stress, attitudes, and personality traits. We also provide a specific compendium of analytic procedures appropriate for making the most effective use of the moderator and mediator distinction, both separately and in terms of a broader causal system that includes both moderators and mediators. (46 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The decomposition of effects in structural equation models has been of considerable interest to social scientists. Finite-sample or asymptotic results for the sampling distribution of estimators of direct effects are widely available. Statistical inferences about indirect effects have relied exclusively on asymptotic methods which assume that the limiting distribution of the estimator is normal, with a standard error derived from the delta method. We examine bootstrap procedures as another way to generate standard errors and confidence intervals and to estimate the sampling distributions of estimators of direct and indirect effects. We illustrate the classical and the bootstrap methods with three empirical examples. We find that in a moderately large sample, the bootstrap distribution of an estimator is close to that assumed with the classical and delta methods but that in small samples, there are some differences. Bootstrap methods provide a check on the classical and delta methods when the latter are applied under less than ideal conditions.
Article
Mediation processes are fundamental to many classic and emerging theoretical paradigms within psychology. Innovative methods continue to be developed to address the diverse needs of researchers studying such indirect effects. This review provides a survey and synthesis of four areas of active methodological research: (a) mediation analysis for longitudinal data, (b) causal inference for indirect effects, (c) mediation analysis for discrete and nonnormal variables, and (d) mediation assessment in multilevel designs. The aim of this review is to aid in the dissemination of developments in these four areas and suggest directions for future research. Expected final online publication date for the Annual Review of Psychology Volume 66 is November 30, 2014. Please see http://www.annualreviews.org/catalog/pubdates.aspx for revised estimates.