Hypothesis Testing - Science topic

The decision rule for a chi-square test of association is about the null hypothesis.

If you are looking to assess an alternative hypothesis, something like TOST (two one-sided test) might be appropriate, but I don't know precisely how this would be applied in a chi-square test situation.

in your case, it is accurate to adopt a comprehensive approach, the priority is to understand how does SME cooperate. starting from a non-essential hypothesis (there is a cooperation !), you are not inventing this fact, your job is to understand deeply how this cooperation is possible. in this case you can conduct a qualitative approach or you can adopt a Mixed approach.

take a look on the mixed research approaches, especially the Exploratory Sequential Method.

Go with the test of association (chi-square test )

Dear Good Sirs,

Thank you very much for your information. Really appreciate your answers.

1 depends on the characteristic you are interested in

2 regression. Type depends on the characteristics of the dependent variable

3 Hypothesis tests depending on samples involved size, method of collection,etc.

The point I have been trying to make is that there are lots of possibilities often depending on the particular question you wish to ask. There's no omnibus test or method that covers everything that you asked about. A good reference for your questions is Montgomery, Design and analysis of experiments. There's no easy simple answer to your question. David Booth

Dear Dilpreet Singh

If you have too many research hypotheses, that are related to your topic, and you have elaborated them according to the requirements of the topic, then it should not be a problem.

It is also possible to introduce some control variables...

I suggest you read the relevant literature for more specifics.

Social media is a concept. And it is not variable. Social media cannot be measured with quantitative tools.

The good part is the technique can be applied to any area. However, the bad part is not that easy. I used it on the question of free will (philosophy) . The second part of that article to be published soon hopefully. Admittedly, the technique requires a long steep learning curve. Note it is not Buckingham's Theorem that appear in many books and papers. You can read more in my book Basics of Fluid Mechanics Dimensional Chapter (not easy to read but provide the flavor even to none fluid mechanics people).

Blaine Tomkins, nothing in the definition I quoted suggests that one has to be able to compute the CIs for all possible samples of size N.

The correct definition of a frequentist CI is based on the hypothetical notion of drawing all possible samples of size N from a population and computing the x% CI for each sample. If one could do that, x% of those intervals would include the true value of the parameter being estimated, and 100-x% of the intervals would not.

In the usual case, we have just one sample and one of those possible intervals. But as the authors of the analogy put it, we do not know if it is one of the x% of intervals that tell the truth (i.e., they include the parameter) or one of the 100-x% of intervals that lies (i.e., they do not contain the parameter).

Have you three hypotheses or one?

Please tell us a little more. It's well-nigh impossible to answer the question as it stands.

This is a really bad strategy. You can do a lot of bad things with such data, leading to irrelevant or wrong conclusions and there is a high risk than you will miss the relevant information. I suggest you sit together with a statistician, ideally one that is a bit experienced in analyzing such kind of data.

With low expected cell counts there are various suggestions as to when Chi-sq is no longer trustworthy. I like the Np15 rule because it's simple and seems to work well. In this rule, N is the total number of observations, and p is the proportion in the smaller group. So if you have a table that contains 50 observations, and 10% of them fall into the smaller group (so P = 0·1). Then Np is (50 x 0·1) which is 5. The table fails the Np > 15 test – not enough data to do a Chi-squared test.

I'll just deal with the first one from David Morse

"1. In an ordinary least squares regression model having more than one independent variable ("predictor"), regarding the test of an individual predictor, the p-value informs you as to whether it is more reasonable to presume that the population value is zero (the null hypothesis) or non-zero (the alternative hypothesis)."

Unless I am misreading this, it could be written as (assuming alpha = 7%): if p < .07, H0 has a probability greater than 50% (and would it be higher if alpha = 5%?). This is incorrect, starting with in most situations the prior probability of H0 as a point hypothesis would be very low (perhaps 0). I could go on, but I assume I am misreading something.

Caty Gonçalves , I think a general point is that the beta associated with each individual feature will be conditional on all the others, so is this what you want? Also, how many features and does it make sense to explore patterns among these before prediction, and if not have you consider lasso-like procedures (and there are lasso procedures if you think the features are in groups).

Hi Shyam Gupta ,

The sample size for A and B is equally 11, which is too small (n=11). Sometimes, it is not enough for regression analysis.

Azerbaijan

Hello Masha,

Why not use the Mantel-Haenszel test across all the z-level 2x2 tables for which there is some data? This allows you to estimate the aggregate odds ratio (and its standard error), thus you can easily determine whether a confidence interval includes 1 (no difference in odds, and hence, no relationship between the two variables in each table) or not.

That seems simpler than having to run a bunch of tests, and by so doing, increase the aggregate risk of a type I error (false positive).

Good luck with your work.

Just a note to the post of Ronán Michael Conroy : TOST is equivalent to interpreting a (two-sided) confidence interval, something that might be easier to understand.

Johanna Schulz , maybe you can contact Ofek et al. to get the data or discuss directly what they would consider a trivial effect. Getting the data would be ideal because then you could directly test the hypothesis that the effect in their study equals the effect equals in your study. If the estimated difference of these effects is negative (the estimate for your study is smaller then for theirs) and when you can reject the hypothesis, then you have reason to believe that your effect is smaller then theirs (there may still be an effect-but not as large as found be Ofek et al.).

PS: This seems to be a nice example demonstrating how worthless publications are that do not report and test meaningful models and don't give any effect quantification, and don't provide the actual data.

I'd say yes, testing the same "structure" using three different drugs (assuming that there are no other effect interfering with the action of that structure), the evidence from the experiments should pile up. As an extreme case you may think of using three drugs with minimal differences, actually with no difference at all - so essentially you use the same drug in three experiments. In each experiment you make 5 independent measurements. Due to the large variance all the p-values are rather large. "Piling up evidence" means that you could pool the three times 5 measurements and so have a higher power (i.e. you get a lower p-value for the same "true" effect). This should be (more or less) equivalent to combining the three p-values.

To the second part (dose-effect of at least one of the drugs). I also agree here. This is a screening. The difference to before is that here you assume that the drugs work at different doses because of some feature of these drugs (independent of "structure") which are all different between the drugs. You are testing a family of different features (or feature-combinations).

And I also finally agree that this can be a rather complex question in practice. Most authors don't seem to think about such things at all. They just always use some correction for muliple testing when doing some kind of ANOVA and otherwise they completely ignore the topic (except in meta-analyses, where evidence is purposefully combined).

For exploratory factor analysis?

Bartlett’s test will tell you whether your data fit for factor analysis or not?

And the chi-square test will tell you if there is a sufficient quantity of factors extracted to explain the variance observed in the data.

I think you will require both.

Best!!

AN

check this, if not useful, please DM me https://doi.org/10.1016/j.renene.2022.03.017

As a complement to this discussion and valuable remarks made by Prof. David Eugene Booth , let me also put this quote from Wilcoxon [1]. It totally agrees with my experience in a field well known from generating "difficult" datasets (clinical trials -> clinical biochemistry).

[1] Wilcox, Rand. (2012). Introduction to Robust Estimation and Hypothesis Testing. 10.1016/C2010-0-67044-1.

Indeed do run the chi-square test. In attachment an R script with post hoc tests.

This raises the classic issue of "bracketing" in phenomenology, with the goal of becoming aware of, and setting aside, one's own preconceptions. Eatough & Smith (2017) give a nice one paragraph summary of the IPA position on preconceptions, with the argument that IPA researchers need to be reflexive in detecting and restricting these prejudices throughout the research process.

But this is very different from a deductive approach, which would build these preconceptions into the research process. In their book, Smith et al. (2009) do acknowledge the possibility of having some "secondary" interview questions that are theory-driven, but these should always be subordinate to the core goal of hearing the participants' experiences in their own terms.

Dear researcher,

I hope you may find the answer. From my understanding, you can find all you need in xtfrontier posestimation in Stata command by typing help xtfrontier and find xtfrontier posestimation. Hausman and Lrtest are available for your research.

In statistics, you use a model to explain how the observed data may be generated. Typically the data is understood as realizations of a random variable with some distribution (binomial, Poisson, beta, Weibull, gamma, Gaussian etc.) and one is interested in the expected value (mean) of this distribution, that may depend on other factors (treatment, time, disease group, country, ...).

The expected value is formulated as a parametric function, being flexible enough to explain essentially any possible observation. The set of all posisble values of all the parameters in this model is called the parameter space. Now the observations you actually made may be more or less well explained by some combination of parameter values. A specific subset of the parameter space is called a (statistical) hypothesis. The complement of the parameter space (all values of all parameters not contained in this specific subset) is called the alternative. You determine if your observed data is considerably more likely under the alternative than under the hypothesis. A hypothesis test calculates the probability of a larger discrepancy of the likelihood of data under the hypothesis and the alternative under the assumption that your model is correct and that the true set of parameter values is in the subset of the parameter space you selected as the hypothesis. If this is very unlikely, then your observed data provides a considerable amount of information against yout model/hypothesis combination. If you believe that your model is actually ok, then the conclusion is that your hypothesis was a "bad choice" and the alternative, therefore, is the better choice. You would thus reject the hypothesis (in favour of the alternative, as this is "everithing elese that is not part of the hypothesis you just rejected).

We often test a point null hypothesis that divides the alternative into two parts to see if the data provide enough information to decide in which of these two parts the parameter value should be located.

Example:

You want to model the probability of a successful job application under a more or less rigorously defined set of circumstances (e.g. what job, who applies, and how the application is done and evaluated). You define the Bernoulli variable Y having the value 1 for a success and 0 for a failure of an application. The parameter is the success probability π, what here is identical to the expected value E(Y).

You collect data to learn something about π. Let's say you have single observation, and that person got the job, so the observed realization of Y₁ is 1. This observation is most likeliy under the hypothesis π = 1 and it is least likely under the hypothesis that π = 0. Now you get a second observation, and that happened to be Y₂ = 0. If these two observations were independent, then the combination of such two observations is most likely under the hypothesis that π = 0.5 and least likely unter the hypothesis that π = 0 as well as under π = 1. After many more (independent) observations you will have a set of data that is considerably more likely under a very narrow set of possible values for π than for any other posible value of π. In fact, it will be most likely for π = sum(Y_i)/n, which is the average of the realizations, which I will denote with m (like "mean").

Testing a point hypothesis (π = h, with h being some value between 0 and 1) is now essentially comparing the maximum likelihood of the data under this hypothesis to the global maximum likelihood. As out simple model here has only a single parameter, the maximum likelihood under this hypothesis is just the likelihood of the data assuming π = h. The global maximum is obtained assuming π = m. The further away h is from m, and the narrower the likelihood peaks around m, the larger is the ratio of these two likelihoods, and the lower is the probability to obsere even larger likelihood ratios when π = h. If this is so unlikely that you are to reject π = h, then you reject any value of π that is even further away from m than h. This is because the likelihood has a single maximum (at π = m). So the test of π = h is equivalent to the test of π > h when h > m and equivalent to π < h when h < m. Hence, when rejecting π = h, you claim to have enough information from your observations about π to say whether π > h or π < h. If you cant reject π = h, then your data don't provide enough information wheter π > h or π < h.

So what we actually want is to see if we can distinguish π > h from π < h. We do this by testing the point hypothesis π = h. If we can reject this, then π = h is "out" to explain the observed data, because there is some other hypothesis within the alternative (which is any π ≠ h) under which the data is considerably more likely. And since the likelihood is maximal at π = m, we know that this value must be "on the same side" of h as m.

For more complex hypotheses, e.g. compound hypotheses about several parameters, there is no simple direction anymore, so the practical meaning of testing point hypothesis (where the point is a point in the multidimensional parameter space) may not be very obvious. But one can certainly define explicit subsets of the parameter space to test more "meaningful" hypotheses against more "meaningful" alternatives.

There are several conceptual errors in what you wrote you had been instructed.

Very generally speaking, you do hypothesis tests to find out if your observed data contains enough information for some kind of interpretation. That's all. We call it a "(statistically) significant result" when there is enough information for an interpretation, and otherwise we call it a "(statistically) non-significant reesult". This only means that the observed data don't give us enough information of the the feature we like to interpret.

The previous paragraph was very general, but it contains an extremely important message: why, in general, we do tests at all and what the test results tell us (and what they don't tell us). About 80% (at least) of all applied stats books I know get this wrong, and so do most researchers.

This principle idea will become clearer with a more concrete example. A prototype example is the effect of a treatment on some quantitative response variable. In your statistical model you formulate the response variable as a random variable of some distribution family. The interesting feature of this random variable is its expected value which may depend on the treatment. We don't know the expected value of this distribution. However, obsered data gives us some information about this value, which I will denote with "µ": for any possible expected value of µ we can use the distribution model to calculate how likely the observed data would be, if µ were what we set it to be. We can so get the likelihood of the data for any hypothesized value of µ. The likelihood will be larger when we choos a µ somewhere in the "middle" of the observed values, and it will decrease the further away from the data we coose a value of µ.

The value of µ where the likelihood is maximal is called the maximum likelihood estimate (of µ). This is the sample mean which I will denote with "m".

If we have a lot of observations, this drop will be steep, so that the likelihood will be much lower we we choose a value that is just slightly different to the sample mean. If we have only few observations, the likelihood won't change so quickly: there is a larger interval of value we can choose for µ and still find a relatively large likelihood of the data. This shows that and how data bring information about µ.

You may now be interested in finding out if that information about µ is sufficient to expect µ being on the same side of some value h like the sample mean m. If the difference between m and h is small and the sample size is small, the ratio of likelihoods for m and h will be close to 1. But when there is a lot of data, the likelihood drops quickly around m and the likelihood ratio will be much smaller than 1.

The genious trick is that one can derive a probability distribution of this likelihood ratio under the assumption that µ = h. So it is possible to calculate the probability of even smaller likelihood ratios than the one we obtained for the observed data. This is the famous p-value. If this probability is very small, then the observed data are very incompatible with the statistical model and our hypothesis that µ = h. Therefore, the data is even more incompatible with all values for µ that are on the opposite side of h than m (because there the likelihood ratios will be even smaller). When, for instance, m > h, and the data are very incompatible with the assumption that µ = h, then the data are even more incompatible with any µ < h. But this means: when there is any value compatible with the data, then this value must be larger than h. This means we "reject the hypothesis that µ < h" and conclude that µ > h (because the data is too incompatible with any µ < h).

Often, it is stated that only µ = h is rejected.Although this is not wrong, it obscures that therewith all hypotheses µ < h or µ > h are also rejected, depending on whether m < µ or m > µ.

Regarding your "Bayesian bypass": this adresses an entirely different question. In the hypothesis test you try to find out if the information in the data is sufficient for a statement about the sign of the difference h-m. The Bayesian analysis requires a prior distribution for µ and modifies this based on the information in the data to a posterior, expressing your current knowledge about µ (and notif you have a large enough sample size to make a statement about whether µ > h or µ < h).

To me this sounds like a Point Process with (say) Time intervals between the Rain event (A Procedure in Genstat software gives following description):

>>>

A point process, or series of events, is characterized both by the times at which events occur, and the intervals between events. The Poisson process is the most basic point process, with Poisson counts in any interval, and independent exponentially distributed intervals between events.

A comprehensive account of methods for analysing point processes is given by Cox & Lewis (1966). PTDESCRIBE implements many of the test and summary statistics they give and should be used in conjunction with the text for a full discussion of the motivation and context of their use. All equations referred to below are from Cox & Lewis (1966).

The DATA variate may contain either the times at which events occur, the intervals between events, or a sequence of 0's and 1's, with 1's indicating the times of events on an integer time scale. The option REPRESENTATION specifies which of these is used. If REPRESENTATION=time and the process is measured from some time other than zero, the initial time should be given in the parameter START. Otherwise the START time is assumed to be zero. The first interval is taken to lie between the START time and the first event. If the process is observed beyond the last event, the total duration of the process should be given in the parameter LENGTH. Checks are carried out on START, LENGTH and the length of each interval, and the procedure terminates if these are inconsistent. If REPRESENTATION=time, the DATA variate may be restricted, facilitating the analysis of truncated or thinned point processes.

>>>

Cox, D.R. & Lewis, P.A.W. (1966). The Statistical Analysis of Series of Events. Methuen, London.

Hello Yuko,

I'm not sure I understand exactly what you're asking.

You certainly can:

1. Use OLS regression to develop a model having two IVs and one DV (which is a univariate model: one DV).

2. Use OLS regression, with additional data, to determine how well a previously developed model having two IVs works to account for variance in the DV in the added/new data batch.

3. Use OLS regression to evaluate the relationship between scores on a single IV with those of a single DV (simple bivariate regression).

4. Use OLS regression to address questions of whether a categorical IV can help explain differences in scores on a single metric DV.

If none of these gets at the intention of your query, perhaps you could elaborate a bit so as to increase the likelihood of getting a more constructive response.

Good luck with your work.

Before deriving the hypothesis/theoretical assumption and even trying to apply it practically, will require a lot of things to consider. For say, most marine organisms (except euryhaline= those that can adapt to various salt concentrations) would die in a fluid wherein the salt concentration is too low or too high due to Osmosis. Some marine organisms can tolerate different osmotic pressure though.

Secondly, you are using a dinoflagellate, which is an autotrophic organism. unless your dinoflagellate has special adaptations for dark conditions it is highly unlikely that they will survive within the xylem of the plant.

Apart from light dinoflagellates require inorganic nutrients for normal growth.

Thirdly, other things to consider would be that - can the to be tested organism evade the plant immunity, acclimatize to the xylem fluid condition?

Depending on you objective statement, if your objective is to compare variables that influence a particular problem, you can use the t- test to compare and them give justification.

Thank you very much for sharing supporting material

dears

see the material below.

T Test?!

Mann–Whitney?!

Sometimes I want to bang my head against the wall at Research Gate.

Do a two sample t test of the difference in means between the tested negative and tested positive groups only.

Because they are of very unequal size they will not satisfy the equal variance condition and you have a Fisher-Behrens problem for which there are several approaches including the Welch test and the Behrens test.

If I understand the issue, I think the answer is simply that different tests test different things.

In the group of two-sample paired tests, on one end is the sign test, which cares only about if an observation in one group is larger than it's paired observation in the other group, with no care about how large this difference is.

The Wilcoxon signed rank test begins by looking at the difference between the paired observations, but then uses the ranks of these differences to perform the test. So, it partially cares about the magnitude of the differences.

And then there's the t-test, that includes the magnitude of the differences. But has certain assumptions about the data.

There are probably other tests that don't have the same assumptions about the data that the t-test has. If you can take the difference between pairs, I'm sure that there's a permutation test that tests if the differences are not symmetric about 0.

dear

see the paper under

Hi Hayley, Maybe this is a half full, half empty situation? How about trying ... 24 participants completed the survey and a year later (time 2) 29 different participants drawn from the same population completed the survey. Run your analyses as between-subject. Then say something like, "Additionally, 5 participants from time 1 repeated the survey at time 2. Though too few for statistical comparison, I (you) explored for additional insights and hypothesis generation. You can then say things like if the highest and lowest scoring participants remained the same. You can say if their survey responses uniformly increased or decreased across time like, or unlike, those in your between-subject study. Good luck with your project! -Maddie

Yes, the wilcoxon test is appropriate for matched pairs of non-normally distributed data.

Las pruebas de hipótesis permiten a partir del resultado alcanzado si se acepta o se rechaza la hipótesis nula y si se puede afirmar a un nivel de significación de 0,05 q los resultados satisfactorios y q se observan diferencias significativas al comparar antes y después de aplicada la propuesta en la práctica educativa.

Dou you have several readings (Y) per sample over time (T), like you put a sample in fire and measure every minute over one hour, so you have about 60 readings per sample?

If so:

Is there some theory specifying the functional relationship between Y and T (e.g. should that be an exponential or linear relationship)?

Plot Y against T for each sample and see what the relationship in your sample data might be, or if it is ain accordance with theory.

Fit the relationship per sample, using a regression model. The model should include some parameter of interest, e.g. a slope or a half-time. Collect the fitted values of this parameter.

You can finally test hypotheses about the difference or the ratio of this parameter depending on the additive. For instance, if the regression model is a simple linear regression and the relevant parameter is the slope of the regression line, you can use a t-test. If the regression model is exponential and the parameter of ionterest is the half-time, you can use a t-test with the logarithm of the half-time.

Looks like a mediation to me.

Livey Czar Aligno I interpret this new information like this: you have an independent variable that is the experimental group (1, 2, ... 8), and for each group you have temp responses for fixed times (1, 2, ... 60) and you need compare the temp behaviors as a function of time for the 8 groups. I dare to think that it is a comparison problem of 8 regression models where the data of the dependent variable (temp) have been taken as repeated measures at the set times. A repeated measures (therefore correlated) design for 8 independent experimental groups.

HI, you are actually asking two different questions here, as PCA is only a data reduction method while PLS and clusterization aim to interpret the data.

You have 2 groups and only 20 samples all in all, so of course way too many compounds and potential confusions. And I would not rely exclusively on "significantly different" between the 2 groups to select data as, with 4k variables, it is more likely than not that you may have false positive (depending on how stringent you ran the statistical test).

My approach would be to do the PCA and look, is there some spontaneous separation of the 2 groups on one of the early principal compoments? Do your data "cluster" i.e. do you have groups of very correlated data? In which case you can probably simplify these groups to a single variable.

You have only 20 samples so when there are significant differences (72 variables) it might be worthwhile to actually plot and look, is the difference pulled by a few samples or is nicely repeatable in one group.

Anyway you look at it, you are in trouble... too many variables for few samples, this is going to require a lot of brain exercise. This is where statistical tools, however good they are, must leave way to knowledge and human intelligence. Do you have hypotheses as to the related biological mechanisms to help you sift through the data , and the predictive value, as said by Guillermo Quintas, will stay poor. Tentative I'd say, it might be a tool to help the practician but not totally relaible as a diagnostic.

Both could be catastrophic. Here are 2 examples:

Type I error: You incorrectly assume that a new expensive cancer treatment is helpful when in fact it is not effective. People may die because they receive the new (ineffective) treatment rather than a treatment that could really help them. Plus you waste a lot of money and time on something that isn't useful.

Type II error: You fail to see that an effective cancer treatment works. People die because they do not receive the treatment.

Hello Matthew,

A one-sided CI for an odds ratio would simply be the appropriate tail estimate from a 100(1 - 2*risk level) CI. For example, a 95% upper limit only CI for a data set would be found be asking for a 90% CI and using only the upper limit.

As Professor Booth notes, the Rosner text is very helpful for this and a host of other questions.

Good luck with your work.

Hello again Serban,

OK, so the usual nomenclature is "training" (or model building) and "test" (or validation) groups. The training data set is used to develop the model, then the model's accuracy/efficacy is ascertained by applying it to the test/validation sample.

With a large enough initial sample, you could randomly select some fraction of that to serve as the training/model building sample, and hold the remainder out to use as the test/validation sample. There are fancier schemes (so-called k-fold models) that data mining folks like to use, but this is the basic framework.

Choose the initial sample size to be sufficiently large to allow a high degree of precision (and, stability: all other things equal, sample results are less volatile across large samples than across moderate or small samples). It's up to you to declare what degree of precision you'd like for any parameter estimates deriving from model building, and to select the training sample size accordingly.

Good luck with your work.

if normality is ok, definetly Dunnett's test (see Google)

Good luck

Sample Size Rule

Sekaran (2013) wrote:

"Roscoe (1975) proposes the following rules of thumb for determining sample size:

Sekaran, U., 2003. Research methods for business: A skill building approach. John Wiley & Sons.

What a journal will accept or not is usually a gamble. It depends on the reviewers, although sometimes the question falls to a technical editor who has a better grasp of what is acceptable for a given journal.

There is nothing magical about the 0.05 cutoff for p-values. If we are in a situation where we are screening potential candidates for something, and we aren't worried about type-I errors at this step, then it makes sense to relax our p-value cutoff to, say, 0.10 or 0.15.

I'm wondering though, if it might make more sense (strategically), to not adjust the p-values with FDR, and keep to the 0.05 cutoff. The logic is the same: If you are interested in screening many bacteria, and aren't worried about an inflated type-I error, then there's no reason to be strict about the false discovery rate.

"Almost all of my hypotheses were accepted"

How? Can you say what your hypotheses were? Were they of the type B_1 = 0?

Did you try Python word count?

Hello Rafael,

The type of test you envision is conceptually comparable to equivalence (non-inferiority) testing. Since a mean difference between two groups can be evaluated as testing the correlation that r(group membership, score) = 0, I think you could recast this method to your purpose.

Here's a link that offers a simple overview of equivalence testing:

Good luck with your work.

Both samples are large enough i.e. >30, to allow for use of the standard normal z-test of equality of their means.

Yes, its increasingly common in some fields (notably psychology) to use Bayesian approaches. Bayes factors are only one way to do this.

e.g., see https://easystats.github.io/bayestestR/articles/bayes_factors.html

If your variables A & B are on the same scale, you can run a repeated measures ANOVA, with Gender as the between factor and the 2 variables as the within factor.

If A is an independent variabel, you can run an ANCOVA, i.e. a regression:

B = A + gender (in that order)

Note that the interaction "A * Gender" should not be significant. If there is an interaction between Gender and variable A, you can rely on the Jonhson-Neyman technique to see where there could be differences.

see:

https://kenstoyama.wordpress.com/2018/01/21/the-johnson-neyman-tecnique-and-an-r-script-to-apply-it/

To your first question, it is not right to look for quantitative data for inductive analysis. Inductive analysis is driven by reasoning based on the data emerging from observations and participants views. Inductive analysis is therefore based on qualitative data. Quantitative means you are either testing an hypothesis or a theory that you already know something about. My opinion.

It sounds like what you want to do is first create a composite variable for each of Importance and Usage, and use those as your measured variables. Of course, if you are comparing these two variables, they need to be commensurate, like composed of the average response for a respondant for the relevant questions.

You don't compare an F-value to a P-value. You compare the (observed) F-value to the critical F-value. The critical F-value is a value for which Pr(F > F_crit | H0) = alpha, with alpha being the level of significance (the "size" of the test). If F > F_crit you know that this probability is smaller than alpha, in which case you reject H0.

Today, we don't need to use tables with F_crit and can calculate Pr(F>F_obs | H0) directly. This is the p-value, and we can compare this p-value to alpha and reject H0 if p < alpha.

We NEVER accept H0. Failure to reject H0 means that we don't have enough data to interpret the statistic w.r.t. H0.

The reason behind this procedure is to dare an interpretation of a statistic calculated from the data (e.g. a mean difference, or the slope o a regression line, or an odds ratio, etc.) only if the data provides enough information about this statistic. The statistical significance is a proxy for this amount of information.

Chi-square tests

You would have to go very far back, I think, to the era before systematic reviews, and even there the problem was more generalisation of RCT findings to groups that hadn't been included in the trial, such as the use of beta-blockers for hypertension in older patients.

The only case I can think of where there were positive results, but the underlying rationale was later discredited was the Paris streetscape. After the cholera epidemic in (?) 1832, they demolished slums, widened streets and installed sewers to reduce exposure to 'miasma' (foul air, believed to cause cholera). It worked, but not for the reasons they thought.

One way ANOVA on SPSS can be helpful

Hening Huang , the things you enumerated are no real alternatives. These are all just flavors of the same thing. It's like changing the color of a car and trying to sell it as an alternative to individual automotive mobility...

At the end of the day, you have some data from which you calculate some etsimated effect(size), and you must use some measure to judge if the information in the data justifies your interpretation of this estimate. And this judgement is done by comparing this estimated size against the estimated noise one should reasonably expect (that is usually also estimated from the observed data). And this is statistical significance, eventually. No need to stick with p-values - we may use transformations thereof (like S-values), or intermediate statistics (like likelihood ratios), or even use informal ways (like 6-sigma rules or whatever). These are all different ways to judge the very same thing: the statistical significance of the observed data. These are no alternatives to the actual process (and need) to have a look at and judge the statistical significance.

You would better to use non-parametric tests. Visually, Q-Q plot is insightful. I think what have to be teste is the change in the probability of the initial interval. Is the change statistically significant?

Yes, a mixed model would be appropriate (time and condition as fixed and site as random factor). And since your dependent variable are counts, this should be a negative binomial model, in which the sites capacity is used as offset. The model shouldinclude the time:condition interaction that gives you difference in time course between the conditions.

Somak Chaudhuri

You might want to look at the comparison of absolute concentrations to enzyme binding site affinities for substrates and for inhibitors

Enzyme active sites are in general more saturated than inhibitor sites (p-values are from the Kolmogorov-Smirnov test).Otherwise look at GEMpress and Shortest-Path Method.

Good luck.

James R Knaub Thank you so much for the thorough answer, and I did look into the linked paper. Very informative and your help was much appreciated!

Austin Pearce Really cool link, will help a lot for any statistical analysis. Yeah when I did some simple regression relationships between fire freq. and burned area there appeared a limiting factor, which led to me having to integrate so many factors.

Andrew Paul McKenzie Pegman Thank you for the suggestion! Yeah I felt that the regression seemed almost redundant at that point, but wanted an outside opinion, much appreciated :)

Dear Carmen Yago ,

null-hyposesis significance tests (NHST) such as t-tests, are only able to provide evidence against the null hypothesis. Therefore, NHST can not be interpreted as evidence against the presence of an effect, which is equivalent to an interpretation in favor of the null hypothesis.

I think possible solutions might come from the area of equivalence testing, i.e., evidence that two values are practically equivalent.

This can be done by either using frequentist approaches (Lakens, D. (2017). Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses. Social Psychological and Personality Science, 8(4), 355–362. https://doi.org/10/gbf8nt ) or Bayesian approaches (Kruschke, J. K., & Liddell, T. M. (2018). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178–206. https://doi.org/10/gc3gmn).

Also from the Bayesian perspective comes the idea of the Bayes Factor that expresses the likeliness that a hypothesis (simulation results from both MC-simulations are equal) is true given the data (Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2018). Improving Inferences About Null Effects With Bayes Factors and Equivalence Tests. The Journals of Gerontology: Series B. https://doi.org/10/gdk2z5). Not to be confounded with the likeliness of a Value to occur given a certain hypothesis as in NHST.

Bests,

Sven .