Sensitivity analyses for minimum detectable effect for multilevel models?

1) Because I am a novice when it comes to reporting the results of a linear mixed models analysis, how do I report the fixed effect, including including the estimate, confidence interval, and p-value in addition to the size of the random effects. I am not sure how to report these in writing. For example, how do I report the confidence interval in APA format and how do I report the size of the random effects?

2) How do you determine the significance of the size of the random effects (i.e. how do you determine if the size of the random effects is too large and how do you determine the implications of that size)?

3) Our study consisted of 16 participants, 8 of which were assigned a technology with a privacy setting and 8 of which were not assigned a technology with a privacy setting. Survey data was collected weekly. Our fixed effect was whether or not participants were assigned the technology. Our random effects were week (for the 8-week study) and participant. How do I justify using a linear mixed model for this study design? Is it accurate to say that we used a linear mixed model to account for missing data (i.e. non-response; technology issues) and participant-level effects (i.e. how frequently each participant used the technology; differences in technology experience; high variability in each individual participant's responses to survey questions across the 8-week period). Is this a sufficient justification? 

I am very new to mixed models analyses, and I would appreciate some guidance. 

Dear all,

In the social sciences, it is commonly recommended to determine the required sample size through an a priori power analysis. This analysis requires, among other inputs, an expected effect size, which is typically drawn from prior studies similar to one's own. However, this value is not always reported. In such cases, as suggested by Perugini et al. (2018), a sensitivity power analysis can be used to determine the minimum effect size that can be reliably detected, based on the available sample size.

The issue I am facing is as follows: should a sensitivity power analysis be conducted prior to each statistical analysis? If not, and assuming that three distinct analyses are planned, which one should be used as a basis for determinining the appropriate effect size for the study?

Thank you for your time and consideration.

Best regards,

Kévin

For a correlational study, I conducted an a priori power analysis based on a previous experiment with similar measures. Given this I was able to estimate my sample size, and went using that.

However, when submitting the paper, a reviewer mentioned that the sample was too sample for yielding stable estimates and cited Lakens & Evers, (2014).

I read recently (while not well explained) that you can also conduct/report a post hoc sensitivity power analysis to corroborate a significant finding in a correlation analysis. (using gpower)

Is this correct? Can someone elaborate on how to report this in APA format, and what I can actually say. The analysis does show that my r value is higher than the critical threshold for 80% power at .05.

Thanks in advance.

Can anyone please advised me on calculating sample size using Gpower? I want to see the effect of an intervention( independent variable: 2groups) on mental health (continuous) of participants and I have considered baseline outcome measure as a covariate? How can I calculate the sample size in this case? Furthermore, my two secondary outcomes (both are in continuous scale) are also affected by intervention according to the previous literature. In such case, do I have to consider them as a covariate? I am confused on numerator df and don't know what to put in both of these instances?

Gpower requires that df numerators be specified - can anyone advise how to estimate these in order to determine sample size? Thank you

Hi, as per my protocol I aimed at running ANCOVA analysis but I have since learned that an appropriate method would be linear mixed effects model. The problem is that I had my power analysis and an experiment done with ANCOVA in mind. Now I would like to know what the minimum detectable effect size is for the sample I have. All the options I find are for the trials with two arms but I have four of them as well as three measurement times, and five outcomes. Can anyone recommend a good tool? I think this can be done in R but I don't use it so I would prefer another way.

Hi, all.

I am using G*power to perform a sensitivity analysis for a one-way MANOVA. The analysis suggested my study had a minimum detectable effect size of f^2(V) = .01.

I was not quite sure about this effect size f^2(V).

Is it just the effect size f^2 for pillai?

Hello,

I am trying to do a power analysis for a 2x2 repeated measures design to determine how many participants I need to achieve 80% power. I'm new to the world of power analysis and don't really have a strong stats background.

IV1 = face orientation

Level 1 = upright

Level 2 = inverted

IV2 = context

Level 1 = background present

Level 2 = background removed

This is a fully within-subjects design. I'm trying to use G*Power 3.1 to do the calculation. This is what I have entered into G*Power so far:

Test family: F tests

Statistical test: ANOVA: Repeated measures, within factors

Type of power analysis: A priori...

Effect size f = 0.25 (just assuming a medium effect)

Alpha err prob = 0.05

Power = 0.8

Number of groups = 1

Number of measurements = 4

Corr among rep measures = 0.5 (leaving it at the default)

Nonsphericity correction E = 1 (leaving it at the default)

The number of groups and number of measurements is the part I'm having an issue with. Will G*Power let me calculate n for a 2x2 within design, or is it assuming this is a 1x4 design? From what I've read and watched, number of groups comes into play if you have a between factor, which I don't, so I've set this to 1. As I have a 2x2 design, each participant is being measured 4 times, hence I've put number of measurements to 4.

Sometimes I've read/heard that G*Power DOES allow you to do a 2x2 within design, and sometimes I've read/heard that it does NOT allow you to do this.

I've had a look at GLIMMPSE 3.0.0. as an alternative but there are many fields it requires where I don't know the answer, mainly that there are a list of tests to choose from, none of which are a repeated measures ANOVA. It also wants me to put the means and SDs for each condition, but I haven't run the study yet, plus it's exploratory so I can't even really guess.

Can anyone with some stats / G*Power knowledge help?

Thank you,

Lydia

Does anyone have code/ know how to calculate effect sizes for a linear mixed model (with fixed and random effects) using JMP 15?

I have downloaded the add-in that calculates effect sizes for fit models, but it doesn't seem to work for mixed models.

Any help would be greatly appreciated.