## All Answers (7)

- You can easily justify your selection by backing up your decisions via a power analysis software, e.g. G-power. What is important is to adequately select your statistical test according to how your data is, e.g. if response/predictor variables are continous or categorical, if they behave well, etc
- It is not necessary to set alpha level at 0.05. But it will be very dificult to justify a larger value... If you want to set it at 0.01 or 0.001, you just need to explain why to be more conservative about it.

About beta level, it is always welcome when authors demonstrate their beta levels, especially if it was calculated a priori - You should use the methods and statistics that are consistent with your research question, purpose & rationale. Most probably if it is post-positive epistemology then ensure your methods align with that paradigm.
- The alpha level is the least of your concerns in my opinion. Of much greater importance is working hard at the experimental design phase to identify key research questions. Follow up by identifying the experimental design statistical analysis methods and adequate level of data to satisfy the identified data objectives. To avoid the alpha issue, make sure that your approach emphasizes estimation and confidence intervals as the primary way of summarizing results.

It is also critical to conduct a power analysis to insure that the experimental design is adequate to estimate parameters with adequate levels of precision to answer research questions. Funding decisions should be based on the adequacy of the design and number of samples to answer questions with a high likelihood of unambiguous results.. - Forget alpha values, significance and so forth and concentrate on the clarity of results. A good statistical analysis is an analysis that allows to immediately grasp what happened NOT ONLY in terms of p but, much more important, in terms of the order of magnitude of the observed effect by means of descriptors that link in a natural way to the phenomenon you are describing..see the interesting paper attached..
- Thank you for all the suggestions. I definitely agree that choosing the right alpha levels depend on the experimental design and we should start analyzing our data critically, rather than making it fit a specific analysis in order to get our p<0.05. There is much more at stake when we start talking about importance of results and not just focus on significance. I think focusing on this will bring just to misinterpretations of results and their importance and practical applicability!

## Popular Answers

John W. Kern· Kern Statistical Services, Inc., University of Wyoming, Montana State UniversityIt is also critical to conduct a power analysis to insure that the experimental design is adequate to estimate parameters with adequate levels of precision to answer research questions. Funding decisions should be based on the adequacy of the design and number of samples to answer questions with a high likelihood of unambiguous results..