Statistical Inference - Science topic

Shaban Juma Ally, that is why one should use weighted least squares (WLS) regression. When the coefficient of heteroscedasticity is zero - which should not happen - then WLS regression becomes OLS regression.

@ Mariano Ruiz Espejo

I have not found any reason for choosing a biased estimator over an unbiased estimator when the later exists. Only in unavoidable situation, it is suggested to use a biased estimator (which must be consistent) for a parameter when there is no unbiased estimator for it.

I would like to restart this question.

I have noted a few papers on cutoff or quasi-cutoff sampling other than the many I have written, but in general, I do not think those others have had much application. Further, it may be common to ignore the part of the finite population which is not covered, and to only consider the coverage, but I do not see that as satisfactory, so I would like to concentrate on those doing inference. I found one such paper by Guadarrama, Molina, and Tillé which I will mention later below.

Following is a tutorial i wrote on quasi-cutoff (multiple item survey) sampling with ratio modeling for inference, which can be highly useful for repeated official establishment surveys:

"Application of Efficient Sampling with Prediction for Skewed Data," JSM 2022: 

https://www.researchgate.net/publication/362370770_Application_of_Efficient_Sampling_with_Prediction_for_Skewed_Data 

This is what I did for the US Energy Information Administration (EIA) where I led application of this methodology to various establishment surveys which still produce perhaps tens of thousands of aggregate inferences or more each year from monthly and/or weekly quasi-cutoff sample surveys. This also helped in data editing where data collected in the wrong units or provided to the EIA from the wrong files often showed early in the data processing. Various members of the energy data user community have eagerly consumed this information and analyzed it for many years. (You might find the addenda nonfiction short stories to be amusing.)

There is a section in the above paper on an article by Guadarrama, Molina, and Tillé(2020) in Survey Methodology, "Small area estimation methods under cut-off sampling," which might be of interest, where they found that regression modeling appears to perform better than calibration, looking at small domains, for cutoff sampling. Their article, which I recommend in general, is referenced and linked in my paper.

There are researchers looking into inference from nonprobability sampling cases which are not so well-behaved as what I did for the EIA, where multiple covariates may be needed for pseudo-weights, or for modeling, or both. (See Valliant, R.(2019)*.) But when many covariates are needed for modeling, I think the chances of a good result are greatly diminished. (For multiple regression, from an article I wrote, one might not see heteroscedasticity that should theoretically appear, which I attribute to the difficulty in forming a good predicted-y 'formula'. For psuedo-inclusion probabilities, if many covariates are needed, I suspect it may be hard to do this well either, but perhaps that may be more hopeful. However, in Brewer, K.R.W.(2013)**, he noted an early case where failure using what appears to be an early version of that helped convince people that probability sampling was a must.)

At any rate, there is research on inference from nonprobability sampling which would generally be far less accurate than what I led development for at the EIA.

So, the US Energy Information Administration makes a great deal of use of quasi-cutoff sampling with prediction, and I believe other agencies could make good use of this too, but in all my many years of experience and study/exploration, I have not seen much evidence of such applications elsewhere. If you do, please respond to this discussion.

Thank you - Jim Knaub

..........

*Valliant, R.(2019), "Comparing Alternatives for Estimation from Nonprobability Samples," Journal of Survey Statistics and Methodology, Volume 8, Issue 2, April 2020, Pages 231–263, preprint at 

https://www.researchgate.net/publication/337950671_Comparing_Alternatives_for_Estimation_from_Nonprobability_Samples 

**Brewer, K.R.W.(2013), "Three controversies in the history of survey sampling," Survey Methodology, Dec 2013 -  Ken Brewer - Waksberg Award article: 

I reread your post and understand it better now as I have been rereading some of what I once knew in an instant. I do think that this is a problem with technical terms. A probability density function (pdf) is not a probability. For each statistical distribution, the pdf is a function which when integrated over the interval (0,1) provides a probability of the data X given a parameter 𝜽.

In contrast, the likelihood is a function of the parameter 𝜽 given the data. The integration is over all parameter values given the data

By definition, If f(x/𝜽) is the joint pdf (or pmf for discrete data) of a sample X= (Xi,... ,Xn), then the likelihood function for observations X=x, is L(𝜽/x)=f(x/𝜽). (see for example, any textbook of theoretical statistical inference such as Casella and Berger).

So you are right that the Likelihood method does not calculate probabilities, per se, but the pdf, which is a function, not a probability, is a crucial part of the likelihood of a parameter (a hypothesis).

The difference is that the pdf treats the parameter or hypothesis as fixed and the data x as random variables. The likelihood function treats the data as fixed and the parameter to vary over all possible parameter values.

Similarly for inference, a single likelihood is a meaningless value but the ratio of the likelihood of two different hypotheses (the null and the alternative) is highly meaningful as we try to decide which hypothesis better fits the observed data.

Taking the difference in the natural logs of each hypothesis tells us which is preferred. A log likelihood difference of 2 means that one hypothesis fits the data more than 7 times better than the other. The meaning of the preference is clear.

Regarding bias in Bayes, it is my view - and that of other people in my main field (molecular phylogenetics), much Bayesian bias is the result of improper priors. The wrong prior guarantees bias in the posterior probability.

Your equation rightly shows that the posterior function is a function of the likelihood times the prior. And using prior knowledge seems intuitively sensible but it is not as easy to determine the proper function for the prior. Using a uniform distribution renders the posterior probability a likelihood.

I use both Likelihood and Bayes methods but for the wonderful quality of likelihood parameters (sufficiency, robust to model violation, low variance, invariance), I prefer likelihoods over probabilities.

Why do you post this question after you have published a paper saing that you find the EP more useful? What's the point of this discussion here? Are you willing to retract your paper or is it to promote your paper?

Most of what you write in the post above is a bit weird (I am NOT saying that you are weird! It's NOT meant in any insulting way!).

"In my opinion, the two-sample t-test can be misleading; it may not provide a valid solution to practical problems." -- Well, we don't need to recapitulate that the p-value very often misunderstood and misinterpreted. That's not the fault of the p-value. It's the fault of a miserable statistical education. And of course does it not provide a valid solution to any kind of practical problem a researcher might want to have one. But it does provide a valid solution to that particular practical problem it was invented for: to provide a standardized statistical measure of data sufficiency.

"We are concerned with the lifetime of the component..." -- this is a considerably strange example to illustrate the t-test. Lifetime is way better modelled with random variables having a Weibull, Gamma or logNormal distribution.

"He [Jaynes] argued that our common sense tell us..." -- this is ridiculus. I don't know the original context, and afaik is Jaynes a very smart person from which I would not expect statements like this. We all do statistics to calibrate our "common sense", to put our hopes and recognized patterns back to what the data can reasonable tell us.

"Moreover, the true meaning of the p-value associated with Roberts’ t-test is not clear." -- Why? The p-value has a very precise definition/meaning. Again, it's not the p-value's fault that you would like to use it for some different purpose. The data provided an estimate of -8 (the mean difference A-B of the lifetimes). It's negative, so the data favors B (longer lifetimes). If this was it to come to a conclusion, why using samples of 9 and 4 units? It would have been sufficient to just have a single unit per sample, so one can see if the difference is positive or negative. Hey, but n=1 is not sufficient, I hear you say. Of course. But how large a sample would be sufficient? I don't know, and you don't know either. So the best one can do ist to take samples as large as possible (feasable, within the limits of time and budget) and then check somehow if the available data can be considered sufficient. This somehow is what we do with the p-value.

Your exceedence probability is in fact something that would be interesting, but as you correctly write in your paper: we don't know it. Then, in a very nanchalant way, you substitute the required but unknown parameter vector by its estimate and do as if this was the truth. But it is not. It can be some terribly imprecise estimate, rendering the whole story about the precentages you calculate useless. But is may also be sufficiently precise to make it useful enough to at least say if the EP is larger or smaller 50%, say. But how to check the data sufficiency? I hope you know the answer.

In Canada they have a Monthly Miller’s Survey, and an Annual Miller’s Survey.  This would be a potential application, if used as I describe in a paper linked below. As in the case of a survey at the US Energy Information Administration for electric generation, fuel consumption and stocks for electric power plants, they collect data from the largest establishments monthly, and from the smallest ones just annually.  After the end of the year, for a given data item, say volume milled for a type of wheat, they could add the twelve monthly values for each given establishment, and with the annual data collected, there is then an annual census.  To predict totals each month, the previous annual census could be used for predictor data, and the new monthly data would be used for quasi-cutoff sample data, for each data item, and with a ratio model, one may predict totals each month for each data item, along with estimated relative standard errors.  Various techniques might apply, such as borrowing strength for small area predictions, adjustment of the coefficient of heteroscedasticity, and multiple regression when production shifts from say, one type of grain to another, as noted in the paper. 

Here are the mill surveys: 

Canadian Mill surveys: 

Monthly Miller’s Survey: 

https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=3403 https://www.statcan.gc.ca/en/survey/agriculture/3403 

Annual Miller’s Survey: 

https://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=3443 https://www.statcan.gc.ca/en/survey/agriculture/3443 

This survey information is found on page 25 of the paper below, as of this date.  There will likely be some revisions to this paper.  This was presented as a poster paper at the 2022 Joint Statistical Meetings (JSM), on August 10, 2022, in Washington DC, USA.  Below are the poster and paper URLs. 

Poster:

https://www.researchgate.net/publication/361024156_Poster_for_Application_of_Efficient_Sampling_with_Prediction_for_Skewed_Data

The paper is found at

https://www.researchgate.net/publication/362370770_Application_of_Efficient_Sampling_with_Prediction_for_Skewed_Data 

.........................     

If you can think of any other applications, or potential applications, please respond. 

Thank you. 

Interesting

Here you can follow:

1- draw the bootstrap sample (size m)

2- Estimate the coefficient by you need from this sample by using the (robust method)

3- find Mse for coefficient

4- Repeat the above bootstarp process B times, here you can estimate Mse for coefficients by taking the average, here you will get the estimation for bootstarp Mse

5- This loop can be repeat N times by Monte-Carlo Simulation

@ Gang (John) Xie ,

I don't agree with the opinion that Null Hypothesis Significance Test (NHST) is a never-justified practice. Hypothesis testing is purely based on scientific logic/philosophy and hence is a justified practice..

It may be acceptable that statistical inference can play a limited role in scientific interface. However, this role is too significant to make the conclusion(s) and/ or to draw inference and/or to interpret results in analysis of observations / data in obtained from experiments / survey.

It is to be noted that the role of every branch of science is limited within some domain.

For at least one test administration, almost all papers contained some kind of reliability study. The over-whelming favorite of coefficient types was coefficient alpha.

Can't

I think that your answer may vary depending on what you consider as better results. In your case, I will assume that you are referring to better results in terms of smaller bias and mean square error. As stated above, if you have poor knowledge and assume a prior that is very far from the true value, the MLE may return better results. In terms of Bias, if you work hard you can remove the Bias of the MLE using formal rules and you will get better results in terms of Bias and MSE. But if you would like to look at as point estimation, the MLE can be seen as the MAP when you assume a uniform distribution.

On the other hand, the question is much more profound in terms of treating your parameter as a random variable and including uncertainty in your inference. This kind of approach may assist you during the construction of the model, especially if you have a complex structure, for instance, hierarchical models (with many levels) are handled much easier under the Bayesian approach.

Mx (0) is solvable by applying direct elementary substitution.

Similarly, you can differentiate the integral up to any order and then

evaluate at t = 0.

can refer to

Two-sample t-Test.

Dear Huda,

In R statistical software, you can generate n random number from exponential distribution with the function rexp(n, rate), where rate is the reciprocal of the mean of the generated numbers.

I'm not sure what you mean 'binary exponential distribution', but if you need only 0s and 1s from an exponential distribution, you can choose from one of the following rescalings (depending on the problem you want to solve):

1) ]0;1[ -> 0; [1;Inf[ ->1

2) ]0;1[ -> 0; [1;2[ ->1; [2;Inf[ -> NA

3) ]0;0.5[ -> 0; [0.5;Inf[ ->1

4) ]0;0.5[ -> 0; [0.5;1.5[ ->1; [1.5;Inf[ -> NA

HTH,

Ákos

I was referring to only one independent variable, when there are no others.

This may be useful

Computer-Intensive Statistics c 2009–11 B. D. Ripley

see section on How many bootstrap resamples - crucially it will depend on what you are trying to estimate eg need more fro standard error than a mean

I do not quite agree with the answer of Wulf Rehder. He has computed only the cumulative distribution function(CDF). The question wants the probability distribution function(PDF), so we need to take a derivative over the CDF.

In the first case, PDF = 1/(1+x)^2 = (1-u)^2

In the second case, PDF = e^x/(1+e^x)^2 = u(1-u)

Multilevel random coefficient models are particularly suitable when between-group variances are heteroscedastic like in the example you mention. There are plenty of good references on these models in the statistical literature.

I recommend you using factor analysis and the confirmatory factor analysis. using SEM analysis you may check the construct reliability and validity (convergent and discriminant) of your questionnaire. 

Dear Omar, Daniel and Fadhil,

thanks for your feedback.

While waiting for a paper copy of "Design of experiments" from Montgomery. I reviewed the online version(boos.google.com).

I could not locate the mathematical proof of the importance of Interaction. I undersatnd it but it is missing a paper that proof it.

Please see following links

In my experience the added complication of the Bayesian approach vs. tractable solutions is most beneficial and justified when the model contains nuisance parameters (not of primary interest and which may be difficult to get data for). In this case the Bayesian approach propagates uncertainty due to lack of information in these parameters into the posterior.

It means that on the boundary you may have normal stresses but not tangential/frictional stresses.

Hi,

Firstly, let us focus on the first choice and the second choice.

In the first choice: You actually model the change in score (y2-y1 where y2 is the measurement post treatment and y1 is the measurement pre-treatment) with respect to a covariate (let say x, and x is a categorical variable).

So the information that this model could be answered (after testing all the assumptions for a regression model and all these assumptions are satisfied) is what the relation between x with respect to the change in the score. So you could not capture the relation between the pre-treatment measurement and post-treatment measurement if any.

For the second option, it can have the advantage that you might see how the pre-treatment measurement could affect the estimate of the post-treatment measurement. I am not sure in which field are you working on but in many fields, the base line (the starting point) is very important to predict a measurement later on.

So personally, I favor the second option over the first one.

Now if we look at the second one and the third one, it is good to start a model where an interaction is there. In this case there are only two covariates (pre-measurement = y1 and x) so it is quite straightforward to look at the model with interaction and compare with the model without interaction to see which one in your case is better to answer the question at hand.

I hope it helps.

Thao.

Hello Matteo,

Thank you for your answer. It was very informative. I will try using the revigo tool!

I will try and explain in more details what I'm doing because I think then that you could offer better help.

So, the main thing that I did was use a prediction framework for virus-host protein interactions.

This framework gave me a list of host proteins which was predicted to interact with my viral protein.

Since, this is an in-silico study.  I'am more interested in checking for enrichment of certain sets than to check for an interaction with individual proteins. In more biological terms, I just want to have an idea on which part of the cellular component will my protein interact.

Also, I have another question, since we are going into details.

So, currently, the enrichment data show that my protein interacts on everything that is considered as cytoskeletal. So, I have 5-8 enrichment in sets that contains such related genes.

What is usually the cut-off for enrichment? I have set with an enrichment of 8 and set with an enrichment of 2. The set with an enrichment of 2 has a p-value lower than the set with an enrichment of 8. Although, both are significant. In that case, on which value do I look? (I will also check the corrected p-value)

In addition, I also have enrichment in the gene set for nuclear proteins. My viral protein is not present in the nucleus. Could that be problematic for my analysis?

The protein prediction interaction could surely still be valid. But, my viral protein doesn't enter into the nucleus to allow for that interaction to occur.

I will check your question, and try to help.

Thank you Matteo!

Promote increased replication. Every experiment is really two experiments. The first is the planned experiment that is conducted. We hope it has sufficient replication. However, I could go back and take another sample from the population that you sampled and I would get a different answer (at least quantitatively). So the experiment is one replicate of all possible experiments that could have happened. If you increased replication there would be room to do other types of analysis that could start to address this concern.

The place to start this would be an introductory statistics course so that more people consider experimenting on broader issues. However, this might be a problem for people that struggle with statistics.

I got to the second controversy in the Brewer article. Point 1, gave me trouble "Because randomization had not been used, the investigators had not been able to invoke the Central Limit Theorem. Consequently they had been unable to use the normality of the estimates " I think this statement is nonsense. The distribution of the individual sample will converge to the underlying distribution of the population being sampled. If I sample selectively, then the techniques used to choose my sample points also define my population. So random sampling doesn't seem to enter into this On the other hand, my experiment in one of many possible similar experiments unless the technique for sampling is somehow deterministic. This is not allowed: I will sample the first person named Sally who walks through the hospital door at 8:00 on Tuesday, and I know that the department head always arrives at 8:00 on Tuesday and her name is Sally (yes she could have a cold on my sampling day, or get stuck in traffic, thereby resulting in a small probability that I would get someone else). However, as long as there is some random process that is sampled, then the distribution of all possible experiments that I could do will be Gaussian by the CLT. Randomization, in terms of sampling design, doesn't enter into this. If I have an available population of 5000 people, and I assign each of them a number, then have a random number generator spit out 50 numbers and I use those people the CLT will apply to my sample. If I select every 100th person the CLT will still apply unless someone is making sure that all the people go through the door in exactly the same order every time I run the experiment no matter which door I use. Randomization may help insure that the sample is not deterministic, but it is not a requirement for the CLT to function.

Dear Ahmed M. Kamel,

I Agree with Ahmed answer.

Thank you

Andy Clark's Mindware (OUP, 2001), while billed as an introduction to the 'Philosophy of Cognitive Science,' is worth a look. It isn't bogged down in history (although he's certainly aware of it, and mentions it when it helps explain things), and favours trying to give a sense of current (as of 2001) theoretical issues and research trends.

a) is correct, b) is not.

That is just the key issue of frequentist statistics that an individual p-value or an individual conficence interval (CI) is meaningless. It is only the benchmark used to control error-rates in (hypothetical and infinite) repetitions of "similar" experiments. P-values and CIs are NOT interpreted with respect to a single outcome. They do not show you how reliable a particular result is (how well the result represents a "true" effect).The reliability is introduced as part of the strategy (not as part of the results), that is, when you define the alternative hypotheses and the desired error-rates. The formalism of the data analysis will lead you to accept one of the two alternatives, and the confidence associated with your decision is not given by the actual outcome but by the error-rates you have chosen and after which you designed the experiment.

In your proposed statement you mix incompatible concepts. With power-analysis and CIs you are in the realm of Neyman/Pearson hypothesis tests. There is no "null hypothesis", only two distinctively different alternative hypotheses (usually one of them indicates a "zero-effect" and is often confused with the null hypothesis, and the other is a specific non-zero value [and not just "anything else but zero"!]). The alternatives are chosen in relation to the problem, and it would not make sense to allow a reader to interpret this data differently (because this data was generated in an experiment that was specifically desined to test these two alternatives, with the given model, and with the given confidence). Just chosing a different confidence (or error-rates) after the experiment would render the experimental design useless; it would be an out-of-context interpretation of the data. If an interpretation of the results in a different context is wanted, the entire power-thing and hypothesis-testing is quite meaningless.

In contrast, Fishers significance testing does not know the concept of error-rates, confidence intervals or power. Here, the compliance of some observed data with a given model is analyzed. The model should be rich enough to give a good description of the data, but for the test it is restricted in a parameter which is called the null hypothesis. A miserable compliance (measured as a "p-value") under this restriction is used as indication that the restriction of the model (i.e. the null hypothesis) is to be rejected. A compliance may not simply be interpreted as indicative for the restriction, because both, model and restriction are theoretical concepts that will be logically false anyway. The aim is only to demonstrate that the available data is already sufficient to show this.

Neither of these two philosophies, the Neymanian hypothesis testing and the Fisherian significance testing, does address the question that is asked by the researcher (and this is what is so terribly confusing!). What we actually want is a statement about how much the data makes us believe in a particular model (more specifically: in a particular value of a parameter or effect). This is a Bayesian approach. So the price we would have to pay to get answers to our questions is to accept a subjective view of probability. This leads us to two possible solutions: either we want to present "objective" conclusions, but then these concusions are about questions we did not ask - or we answer the questions we actually have, what is possible only on a "subjective" basis.

But to come back to your statement: you can report confidence intervals, but they can only be interpreted as a "statistical signal-to-noise-ratio" (just like p-values, that are the same thing, viewd from a different angle). There is per se no information about a reliability of these particular results or conclusions. It is simply not there, no matter how much we wish it should be.

Thank you Prof. Subramani.  I appreciate your contribution.  This is very helpful. 

- (no Dr., Prof.) Jim Knaub

So you start from a sample, then subdivided in two: A goes with a purification path, get split again and the split samples analyzed, B goes with a different purification path, get split again and analyzed. 

Now a technical sample is, in this case, simply the sample split after the two different procedures, so you have 3 technical replicates of A and 3 technical replicates of B. Generally a technical replicate is defined as a sample at the end of a given procedure who get split before the actual analysis to account for internal variability (like measurement errors) whereas in a biological sample in a different sample from the same origin sampled in the same (or close as possible) condition and this account for biological variability.

So I don't think you can call the three split samples from each procedure biological sample. 

Ah, okay.  Given that you want pair-wise contrasts of the different phenotypes within each size, the problem is much easier than I first thought, and you don't need to use my macro.  You need to change COMPARE=Phenotype*Size to COMPARE=Phenotype on your EMMEANS sub-command, and you need to change SCALE=ORIGINAL to SCALE=TRANSFORMED.  Then you need to exponentiate the mean pair-wise differences (and their CIs) to show the results as odds ratios.  (You are currently seeing differences in log-odds.) 

Re SCALE=TRANSFORMED, note the following from the Command Syntax Reference manual entry for GENLIN > EMMEANS:

If you use TRANSFORMED, you are using the logit link function. Thus, the estimated marginal means will show the log-odds, and the pair-wise contrasts will show differences in log-odds.  When you exponentiate those differences, you get odds ratios.

You can use OMS to write the table of pair-wise contrasts to a new dataset, and then use a few simple COMPUTE commands to do all of this.  Your syntax will look something like the following.

****** START OF SYNTAX *****.

* Use OMS to write the table of pair-wise contrasts from

* the EMMEANS sub-command to a new dataset called 'pwcon'.

DATASET DECLARE pwcon.

OMS

/SELECT TABLES

/IF COMMANDS=['Generalized Linear Models'] SUBTYPES=['Pairwise Comparisons']

/DESTINATION FORMAT=SAV NUMBERED=TableNumber_

OUTFILE='pwcon' VIEWER=YES

/TAG = "pwcon".

* Use EMMEANS to get pair-wise contrasts between different phenotypes

* within the same cell size group (Size).

*******************************************************************.

* NOTICE that I changed COMPARE=Phenotype*Size to COMPARE=Phenotype.

*******************************************************************.

GENLIN UCM (REFERENCE=LAST) BY Phenotype Size (ORDER=ASCENDING)

/MODEL Phenotype Size Phenotype*Size INTERCEPT=YES

DISTRIBUTION=BINOMIAL LINK=LOGIT

/CRITERIA METHOD=FISHER(1) SCALE=1 MAXITERATIONS=100 MAXSTEPHALVING=5 PCONVERGE=1E-006(ABSOLUTE)

SINGULAR=1E-012 ANALYSISTYPE=3(WALD) CILEVEL=95 LIKELIHOOD=FULL

/EMMEANS TABLES=Phenotype*Size SCALE=TRANSFORMED COMPARE=Phenotype CONTRAST=PAIRWISE PADJUST=LSD

/REPEATED SUBJECT=Animal SORT=YES CORRTYPE=INDEPENDENT ADJUSTCORR=YES COVB=ROBUST

MAXITERATIONS=100 PCONVERGE=1e-006(ABSOLUTE) UPDATECORR=1

/MISSING CLASSMISSING=EXCLUDE

/PRINT CPS DESCRIPTIVES MODELINFO FIT SUMMARY SOLUTION COVB.

OMSEND TAG ="pwcon".

DATASET ACTIVATE pwcon.

COMPUTE OddsRatio = EXP(MeanDifferenceIJ).

COMPUTE ORlower = EXP(Lower).

COMPUTE ORupper = EXP(Upper).

FORMATS OddsRatio to ORupper (F8.3).

LIST MeanDifferenceIJ to ORupper.

****** END OF SYNTAX *****.

HTH.

I would like to insist in the comment by J.J. Egozcue: the key to understanding which transformation makes sense is to understand which is the sample space and its structure. A better fit to a model can be just wishful thinking if the approach can lead to nonsensical results.

 If you need  if you need a LD50/LC50 calculator based on probit analysis, it is freely available at the following site: This calculator also provides estimation of 95% fiducial confidence limits for LD/LC values

Thank you for your thoughtful question, I am attaching a few articles.

It can be happened due to the multicolinearity so, first you need to check the relationship between two independent variables. If they are related they can't give you the correct estimated parameters because the effect of one independent variable is already can be hidden by the other one. therefore, try to check correlation between them and try to detect it by using Principal component Analysis or Factor analysis.

.

this will happen when you have very large rare events (i would not call them "outliers" before having looked at them in detail !) : max(X) may be very large but related to very rare events, most of the xi are much smaller and the resulting distribution will be concentrated at zero

in such cases, you may adapt your normalization strategy

- log transformation may help

- Z transform may help

depending on the shape of your distribution ... and your ultimate analysis goal !

another rough trick is to replace min(X) and max(X) by appropriate quantiles (say 1% and 99%), thus having the same kind of simple normalization and avoiding the effects of the rarest and largest events

.

oh ... and this kind of normalization is just a translation followed by a multiplicative scaling, so it does not really change your distribution (as would be the case with log or Z transforms) : a translated and scaled gaussian is still a gaussian, but with a different mean and variance ...

.

As I understand your research question more I see that the Friedman's test isn't what you're looking for, as the groups are independent and did not receive all treatments/conditions. What might be more appropriate than my previous suggestion is a mixed anova, but I'm not sure what a nonparametric form of this would be. I guess that's why you are opting for multiple comparsons. This post might be of some help: https://www.researchgate.net/post/Is_there_a_non-parametric_equivalent_of_a_2-way_ANOVA

Sorry, I hope someone is able to better answer this question.  

The problem is twofold. First the coefficients b2 and b3 mean different things in model 2 and model 1. Exactly what they mean depends on how you have parameterized the model and whether b2 and b4 are centred. In a model without an interaction the effects are additive and so b2 and b3 are not impacted by the value of y or z. In the interaction model the effects of b2 and b3 depend vary with y or z.

Second, and probably most important in this case x is correlated with y, z and yz. This means that the estimate of b1 is impacted by the additional shared variance with yz.

Centering will help with one of these problems but probably not with the collinearity between yz and x. Centering in this case is not standardization (you should not generally standardize if dealing with logged data IMO) and may not be necessary or sensible.

Dear Huda,

Paul Chiou is correct. I wrote out the details for you. The change of variable is the trick. After that it's really simple.

The subscript "-i" seems to be used to show the out-of-sample predicted value calculated for the omitted observation.

Without the subscript "-i" it would just be estimated value based on all available data points.

The paper by Siu and Kelly from 1998 for example explains this nicely in my opinion.

The estimated residuals will capture some heteroscedastic variance of  an omitted variable, therefore the natural heteroscedasticity won't be correctly estimated. Even if correctly estimated, the subsequent GLS estimation won't solve the estimation bias problem stemming form the omission. Naturally, I believe, the omitted variable problem should be taken care of first. Otherwise, it would be a waste of time, getting nowhere.

Here are three papers  that might be of interest

1 : this discusses a multilevel model as a measurement model and how shrinkage is applied to reduce the Mean Square Error 

2 These show how to reduce MSE in the analysis of a large table with random coefficient modelling

I think no single teaching method/technique would help you. Actually, I tried both methods with one of my colleagues who has difficulties in understanding statistics and I failed. To me, graphical method works fine;I can cover the black areas and move on. However, this technique does not work with my colleagues.

In general, I may share some points of my own experience:

This is my own experience. Please search for scientific papers that may give you more trusted results.

Best Regards,

Dear Manoj,

In that case, I'm afraid I'm not able to be of much help. Developing an R package can take considerable time, which I cannot spare at the moment. Do note that it can be very beneficial for yourself to learn the R language properly. There is plenty of free material out there for self-education. All it takes is time (if you have some).

In any way, I wish you the best in finding the help you need.

Kind regards,

Pepijn

Here's an R function that will compare the cov values for two batches, both as simple differences in cov and as ratios of cov values. Call it as: resampling_cov(dataframe, ntrials).  Maybe it will help.

Dear Alain Manuel Chaple Gil,

I agree with 'YES, in most cases' [Frank B. K. Twenefour]. I would say 'in general, yes".

Let us first define Statistical Inference:

"The theory, methods, and practice of forming judgments about the parameters of a population, usually on the basis of random sampling".

"Statistical inference means drawing conclusions based on data" [Duke University]

Another way of looking at this is: One cannot 'infer' that which is already 'known'. Therefore if the samples are not selected at random, meaning they were 'intentionally selected', then there is no 'inference', but rather knowledge about them to begin with; which caused someone to specifically select the samples in the first place. If this is true then the objectivity of randomness is negated and the inference is less reliable.

Random sampling allows for the deduction and assumption used in inference.

Without random sample one gets poor deduction and poor assumptions.

I hope I have offered a 'reflective' perspective on statistical inference.

Thank you Alain Manuel Chaple Gil, for your question!

Respectfully,

Jeanetta Mastron

On another note: IF anyone is looking for the 'argument' of the answers that are YES to the question, they may find their "NO" answers in the blogging of Alan Downey

"Statistical inference is only mostly wrong" March 2, 2015 - see below link

Hi Huda,

The ratio of two random variables does not in general have a well-defined variance. You can try to re-normalize θ ̂ or use approximations or Taylor series expansions about the expected values of v1 and v2. In terms of expected values, it looks like for independent random  variables, var(x/y) ≈[⟨xx⟩⟨y⟩⟨y⟩−2⟨xy⟩⟨x⟩⟨y⟩+⟨xx⟩⟨x⟩⟨y⟩]/⟨y⟩^4. However, in your case, v1 and v2 are not independent...

Good luck!

Slava

Dear friends,

In my view, the best way to learn statistical methods is to solve problems related to data analysis, with some practical purpose in mind.

Ilya

I also do not know any exact test when controlling for covariables.

The best alternative that I know is to use a generalized linear model of the Poisson family ("Poisson regression"). Here you can add whatever covariables you have and you get odds ratios "adjusted for age".

Dear George,

 Thank you for your reference to the psychologist Michael Bradley’s works on inferential statistics, and I entirely agree with him when he writes in 2014: “It is a fallacious and misleading exercise to imply measurement accuracy by assuming set errors rates from the specific statistical samples presented in any particular exploratory study”. This paper however was investigating a part of a widest cleft, as Savage put it in 1961, between frequentists and this cleft was linked to Fisher’s fiducial Inference and other points. For example, Neyman said in 1941: “the theory of fiducial inference is simply non-existent in the same way as, for example, a theory of numbers defined by mutually contradictory definitions” while Fisher accused Neyman to use his own work without reference. But their disagreements were also about refutation and confirmation, which was the topic of Bradley’s work, and about experiments.

So that if Bradley point of view was interesting, I don’t think that it might answer entirely to my more general question.

 References

Bradley, M.T., Brand, A. (2014). The Neyman and Pearson versus Fisher controversy is informative in current practice in inferential statistics. Conference paper to the Canadian Psychological Association.

Neyman, J. (1941). Fiducial Arguments and the theory of confidence intervals. Biometrika, 32, 128-150.

Savage, L. (1961). The foundations of statistics reconsidered. In Neyman J., ed., Proceedings of the fourth symposium on mathematical statistics and probability, Berkeley, vol. 1, 575-586.

Thank you. I am actually interested in differences among seasons but the data are highly correlated across seasons and I am not sure about using season as a fixed effect. 

It has definitely, Thanks i just uploaded a new question, could you please have a look

Many modifications of the classical ratio estimator have been made in the past quarter century. The alternative ratios discussed in Sarndal, Swensson, and Wretman and in Brewer's work and others rely on differing degrees of heteroscedasticity, but I refer here to the chain ratio-type estimator, exponential estimator, statistics added to terms in numerators and denominators, and combinations of these ratio and product estimators, and the like, which have been produced in many papers, and as far as I have seen, it appears that little thought has been given to interpretation and logic behind these modifications of the classical ratio estimator. The 'exponential' ratio-type estimator appears to have appeared in 1991 where it seems to have been justified in terms of a transformation and its MSE, but the logic of the estimator is not clear to me.

As an example, note that in STATISTICS IN TRANSITION-new series, June 2008, Vol. 9, No. 1,

pts.stat.gov.pl/cps/rde/xbcr/pts/PTS_sit_9_1.pdf,

there is an estimator on page 106 which is said to be a linear combination (Stein/shrinkage type) of a ratio and a product estimator (exponential type). However, note that the same auxiliary variable is being used in each term. A ratio estimator requires a positive correlation between x and y and a product estimator requires a negative correlation, here for the same x and y. How can the same two data sets be both positively and negatively correlated?

Many estimators seem to be justified by optimizing the variance estimate, but I suspect that this may be at the expense of bias. Also note that just because a calculated variance estimate is low, does not mean that the estimator was correct in the first place. (Many use an OLS estimator when the data are demonstrably and logically WLS.)

When more than one auxiliary variable is used in a multiplicative manner, it seems that often, not enough attention might be given to interaction between these variables.

Perhaps I am missing something, but the logic escapes me for these modified classical ratio estimators. In the notes attached through the link from my question above, I show that a 'repeated substitution' justification for the chain ratio-type estimator does not make sense, but perhaps considering it as based on nonlinear regression instead of linear regression does make sense. Are there other justifications?

Any thoughts on any of these modifications of the classical ratio estimator, and how they should be interpreted? 

Dear Huda

The maximum likelihood estimatior of the scale, is the average of the absolute deviation from the random variables and the median of the sample (the maximum likelihood estimatior of the location parameter). See http://en.wikipedia.org/wiki/Laplace_distribution.

I hope this help.

To the best of my knowledge, the significance of the slope of moving averages (mavs) is NOT the significance of a trend in the data. The mavs will be correlated and this correlation will vastly over-estimate the significance of a trend in the data. I am not sure if there exists a method to infere the uncertainty of the trend in the data based on given mavs. But even if such thing exists it remains unclear why the authors did not use the available original data straight away. That they did not looks very suspicious.

I have no formal proof that their method is wrong, but I can show by simulation that the p-values obtained from linear regressions on mavs will NOT keep the type-I error rate. The simulatrion (see R code below) calculates the regression of the mavs for different window withs (based on 100 data points). The original data is only noise. There is no trend. Therefore, the simulation studies the results obtained under the TRUE null hypothesis. The results are plotted (see attachment). The plots show:

1) the standard errors of the slope of the mavs decrease with increasing window size.

2) the distribution of the t-values gets wider with with increasing window size (the "correct" distribution of t-values is plotted in red).

3) Correspondingly, there are increasingly many tiny p-values with increasing window size.

4) And finally, the rate of p-values below the nominal 5% (the effective rate of rejecting H0) is clearly above 5% for any width > 1. This rate approches 1 for width->Inf (hence, you can make SURE to get a "significant" trend in the mavs simpply be chosing a sufficiently wide window).

# R CODE ---------------------------------------------------

mav = function(x,n=5){filter(x,rep(1/n,n), sides=2)}

x = 1:100

nx = length(x)

window.widths = c(1,3,5,7,9,11,13,15,20,30)

nww = length(window.widths)

coefs = list()

for (i in 1:nww) coefs[[i]] = replicate(1000,  summary(lm(mav(rnorm(nx),window.widths[i])~x ))$coef)

se.values = lapply(coefs, function(tab) apply(tab,3,"[",2,2))

t.values = lapply(coefs, function(tab) apply(tab,3,"[",2,3))

p.values = lapply(coefs, function(tab) apply(tab,3,"[",2,4))

sig.values = lapply(p.values, function(p) -log10(p))

par(mfrow=c(3,1), mar=c(0,3,1,1), oma=c(5,0,0,0))

boxplot(se.values, xaxt="n")

title(main="Standard Errors of the Slope", line=-2)

boxplot(t.values, xaxt="n")

title(main="t-values for the Slope", line=-2)

t = lapply(window.widths, function(n) rt(1000, df=nx-n-1))

boxplot(t, xaxt="n", add=TRUE, col=2, border=2, outline=FALSE, boxwex=0.25, at=1:10+0.2)

boxplot(sig.values)

title(main="-log(p) for the Slope", line=-2)

mtext(1, adj = 0.5, line=3, text="window width")

Terry -

Thank you for your very informative answer. I am also retired, but this should be useful to people I know where I used to work, as well as interesting to me and others. Regarding where I worked, they are going through some development now, hopefully, so the timing of this discussion is particularly good, I think.

BTW, I'd like to comment on this from your answer:

" This even allows domains with no data which some might regard as black magic."

I have run into that problem in cases covered by the paper in the first attached link, which has been used since 1999 to produce results for perhaps a thousand or more estimated totals each month for official energy statistics. There have been many categories/cells that were empty or nearly empty for n, and sometimes N. There was pressure to increase the sample to cover those with very small or no n and small N, but they were for categories of little importance, and the overall increase in sample size would have very likely made for a bigger nonsampling error in the form of measurement error. The sample size at one time became particularly unwieldy and had to be managed better, so more attention came back to this methodology. Total survey error is the overall consideration.

For cases where n is zero, there are still estimated totals and estimated standard errors for the prediction errors for those totals in all case, when using this small area approach. That is good for being able to obtain higher aggregates, but I did not want management tempted to report the estimates for the empty sample cells (categories). The variance is measureable, but not the bias due to any level of model misspecification. However, in such cases, the estimated variance was generally huge - even without separately considering bias. Thus I was able to convince management not to publish data for those cells, because of the very large estimated standard errors for those estimated totals, and they just figured in to the higher levels of aggregation.

So the point is that one can avoid publishing the "black magic" estimated totals and standard errors, because the estimated relative standard errors are generally hundreds or thousands of percent - what I said was effectively 'infinite.' The concern still is that managers often want to publish data with relative standard errors that are too high, and with small n, using small area estimation (SAE), there is also bias to consider. Even though we speak of model-unbiased results, that does not account for model failure, and when it comes to SAE, that is generally more of a concern. That is why a good deal of testing and monitoring can be helpful - but a couple of decades or so of successful use has showed this to be worthwhile.

Thank you for your comments.

Jim 

You may want to read the following paper for some additional context and ideas

Many thanks Paul, I will look into that as well.

Rabin

Kolmogorov-Smirnov Test finds out the “fit” of an observed pattern of frequencies of ordinal data to an expected pattern.

Chi-square goodness-of-fit test finds out the “fit” of an observed pattern of frequencies of nominal data to an expected pattern.

If you need to log-transform both the outcome and predictor variables, you certainly have a nonlinear relationship between the two, most probably a power relationship, i.e.:

Y=aX^b

Where "a" is a coefficient and "b" the exponent of the predictor X. When you log transform both sides of the equation, you obtain log(Y) = log(a) + b*log(X), a linear equation.

The best way to handle this is to analyze the data with a nonlinear mixed model, where the above power equation is specified as the structural component of the model. This analysis can be done with a specific software for nonlinear mixed models such as NONMEM, or with advanced statistical software such as SAS, S-plus, R, etc.

As for references, you may find lots of them in the field of interspecies allometric scaling. I suggest you the following that comes from my field of work:

Martín-Jiménez T, Riviere JE. Mixed-effects modeling of the interspecies

pharmacokinetic scaling of oxytetracycline. J Pharm Sci. 2002 Feb;91(2):331-41.

PubMed PMID: 11835193.

Good luck with your analysis!