Regression Modeling - Science topic

While directly comparing Hazard Ratios (HRs) from different strata of a Cox Proportional Hazard model isn't straightforward, there are several statistical methods to assess if the differences between HRs are statistically significant.

The best approach depends on your specific research question and the design of your study. Consider the following factors:

Most statistical software packages (e.g., R, SAS, Stata) can perform these analyses. Consult the specific documentation for your software to learn how to implement these methods.

By carefully considering these factors and employing appropriate statistical methods, you can effectively compare HRs from stratified Cox Proportional Hazard models and draw meaningful conclusions from your analysis.

Here are some recommended ones: 1. "Sampling Techniques" by William G. Cochran This classic book covers a wide range of sampling methods with practical examples. It’s comprehensive and delves into both theory and application, making it valuable for students and professionals. 2. "Survey Sampling" by Leslie Kish'' This is another foundational text, known for its detailed treatment of survey sampling design and estimation methods. Kish's book is especially useful for those interested in practical survey applications. 3. "Model Assisted Survey Sampling" by Carl-Erik Särndal, Bengt Swensson, and Jan Wretman This book introduces model-assisted methods for survey sampling, which blend traditional design-based methods with model-based techniques. It's ideal for more advanced readers interested in complex survey designs. 4. "Sampling of Populations: Methods and Applications" by Paul S. Levy and Stanley Lemeshow This text is widely used in academia and provides thorough explanations of different sampling methods with a focus on real-world applications. It also includes case studies and practical exercises, making it helpful for hands-on learners. 5. "Introduction to Survey Sampling" by Graham Kalton This introductory book offers a concise and accessible overview of survey sampling methods. It’s well-suited for beginners who need a straightforward introduction to key concepts. 6. "Designing Surveys: A Guide to Decisions and Procedures" by Johnny Blair, Ronald F. Czaja, and Edward A. Blair This book focuses on the practical aspects of designing and conducting surveys, with particular emphasis on decision-making and procedural choices in the survey process.

Nothing special. There are many web pages about the topic you will find when searching for these keywords. This should also be part of any introductory stats book dealing with factorial designs (multiway-ANOVA models). Unfortunately I don't have one around I could look up, but maybe someone else can recommend a good book on that topic.

It is not possible to answer that question without more information. First, you should say how many observations you have. A regression with 17 observations is questionable but it depends on the number of explanatory variables. Since these observations correspond to above the threshold, I fear that they are outliers, hence we are in the worst situation. Did you consider a transformation on the dependent variable? That would perhaps improve the situation provided the relationship with the other variables allows for it. I would say that each of the two regimes should have enough observations.

Binary Logistic Regression. I have a YouTube video on it.

Marius Ole Johansen The threshold least square regression model by Hansen (2000) divides the series into two regimes endogenously. The Regime above the threshold and below the threshold, and then regress both regimes individually by OLS. this method also involve bootstrap replication. In my case the regime above the threshold only remain with 17 number of observation. Does it creates loss of degree of freedom issue in the data?

This an old question, but I don't think any of the answers address the central point. In a regression model with an intercept the coefficients are estimated when all other variables take the value zero.

In the first model this does not matter:

y = b0 + b1*x1 + b2*x2 + e

the effects x1 are constant for all values of x2 while the effects of x2 are constant for all values of x1.

This changes in the second model because of the interaction:

y = b0 + b1*x1 + b2*x2 + b3*x1*x2 + e

The interaction by definition means that we can't consider effects of x1 in isolation from x2 and vice versa (unless the interaction effect b3 is exactly zero in which case it behaves like the first model). Now the "main effects" of x1 vary as x2 increases or decreases (and vice versa). In technical terms it isn't really a main effect but a "simple main effect". Because we evaluate them a when x2 is zero it can be very different from the effect in our first model.

In fact in practice this estimate is usually not remotely helpful because much real world data (probably most) is in the positive range - so we are evaluating the effect of x1 for a value of x2 that is unusual (or perhaps impossible). For instance if x1 is age and x2 income and y is happiness we'd be evaluating how happy someone is at different ages when their income is zero or happy they are with different incomes when their age is zero. In both cases these are not typical values of our data and probably don't address a useful research question.

To get round this there are two main solutions:

a) centre x1 and x2 before computing the interaction/product term x3. To centre a variable simple subtract its mean so the new centred variable has a mean of zero. This produces the same coefficients in the first model. In the second model b1 and b2 will be similar to the values in model 1. This is because you are evaluating them at the mean of the other variable (0 is the mean of both x1 and x2 in the centred model). This is still a simple main effect but has a nice interpretation as the effect of x1 for an average data point (average on x2). Interestingly b3 is the same in both models. (This can be proven mathematically but we don't need to worry about that).

b) for a full understanding graph the predictions or marginal effects of the model. See how y varies as a function of x1 and x2 together. There are various ways to do this, but a plot of x1 on the x axis, y on the y axis and different lines for x2 (low, medium, high often defined as -1SD, mean, + 1SD) is usually a good way to see what's going on. An interaction means that y depends on both x1 and x2 so plotting the pattern together is really the best way to see what's going. You can also plot the simple main effects separately - with is useful if that's your focus. Adding confidence bands to the plots is also useful.

All samples (standards) should be injected at the same VOLUME. Concentration should vary, not injection volume. This follows good chromatography fundamentals. The reasons are many, but one key issue is that an autoinjector's ability to accurately deliver very low volumes, esp 1 ul or 0.1 ul , often vary showing poor linearity (*This of course requires more information regarding the exact make, model and configuration of the A/I used). Changes in injection volume may also contribute to changes in peak shape, retention time and area amounts (which may change the results). To avoid problems and for these reasons we always maintain the same volume.

Sometimes it can be that easy!

I think i solved the problem.

It seems that the package (plm) im using has problems calcualting the adj. R when you use fixed effects (model=within).

I conducted my regression with alternative packages (e.g. fixet) and it then properly calculated the R squared (0.5) and R within which lays around 0.08.

To find the regression model that expresses the relationship between adsorption and process variables for predicting fluoride removal (%), you can follow these steps:

By following these steps, you can find a regression model that effectively captures the relationship between adsorption, process variables, and fluoride removal %, allowing you to predict fluoride removal based on the chosen process variables.

To determine uncertainty in energy and inflation time versus inflation pressure control in a tyre control unit, the approach typically involves:

Jeoffrey Dehez , your thanks should not go to Joseph - he just used GPT and pasted the answer into RG. There is no intellectual work associated with that. And I am a bit sorry that you find that answer useful, since it is very generic and nowhere does is give any specific answer.

Why do you want to determine the significant predictors before conducting the regression? Instead, you should just run the regression (with a set of dummy variables to capture the categories of your nominal IV). That will tell you which predictors are significant when you take the full set into account.

Dear ,

Thank you very much for your explanations!

However, I am working on IBM SPSS.

Do you know how I could do a regression within subject design with clustered standard error on IBM SPSS ?

Since my dependent variables are ordinal I thought about doing an ordinal regression. Analyze > Generalized Linear Model > then I could specify the type of model as ''ordinal logistic''. But does this test take into account within subject design?

I also thought about GEE on spss. But I don't know if I can do it. ?

My study :

112 participants had to face 7 different scenarios ( the factors) and indicate their level of agreement on a scale likert scale from strongly disagree to strongly agree to measure their motivation, satisfaction, collaboration and help. I also checked for age, gender, and professional situation.

Thank you in advance

you can use the glmm.hp package in R. https://academic.oup.com/jpe/article/15/6/1302/6845730 https://academic.oup.com/jpe/article/16/6/rtad038/7444979

install.packages("glmm.hp")

library(glmm.hp)

library(MuMIn)

library(lme4)

mod1 <- lmer(Sepal.Length ~ Petal.Length + Petal.Width+(1|Species),data = iris)

r.squaredGLMM(mod1)

glmm.hp(mod1)

a <- glmm.hp(mod1)

plot(a)

Yes, it's valid to build separate models to address different research questions. In your case, one model could focus on identifying the individual factors that contribute to the choice of outcome variables, while the other model could examine pairwise interactions between predictor variables.

For the first research question, you might consider using techniques such as multiple regression analysis or logistic regression analysis, depending on the nature of your outcome variables (continuous or categorical, respectively). These techniques can help identify which individual predictor variables significantly contribute to the outcome variables.

For the second research question, exploring pairwise interactions between predictor variables, you could use techniques like interaction terms in regression models or more advanced methods like structural equation modeling (SEM) or generalized linear models (GLMs) with interaction effects.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2018). Multivariate Data Analysis (8th ed.). Cengage Learning.

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics (4th ed.). Sage Publications.

Hi Prakash Poudel!

You can rely on classical prediction metrics such as:

A suitable combination of the above metrics for your purposes is the F2-score, which is a variant of the F1 score that puts a stronger emphasis on recall compared to the standard F1 score. Placing a stronger emphasis on recall rather than precision makes it suitable for tasks where capturing all positive instances is crucial.

Hope you found my answer useful.

Bests

Yes.

In a panel data model, the fixed effects estimator is used to control for time-invariant unobserved individual heterogeneity. This is done by allowing the intercept to vary across individuals (or entities).

I have recently added easy to use R functions to Git-Hub for multivariate decomposition (non-linear models, complex svy designs etc.

Of course. Why shouldn't that be possible?

Hello João Simões. I had never heard of Meta-Essentials, but the website says this about it:

Meta-Essentials is a free tool for meta-analysis. It facilitates the integration and synthesis of effect sizes from different studies. The tool consists of a set of workbooks designed for Microsoft Excel that, based on your input, automatically produces all the required statistics, tables, figures, and more. The workbooks can be downloaded from here. We also provide a user manual to guide you in using the tool (PDF / online) and a text on how to interpret the results of meta-analyses (PDF / online).

Source: https://www.erim.eur.nl/research-support/meta-essentials/

But if I understand your question, you are not doing meta-analysis. Rather, you appear to be estimating a logistic regression model for one sample, and you want to generate a forest plot displaying the OR and CI for each explanatory variable in the model. I.e., I think you want to do something like the first example shown here:

Have I understood you correctly? If so, what other statistical software do you use (besides Meta-Essentials)? Thanks for clarifying.

What do you mean by "Hierarchical Bayesian spatiotemporal model"? If I would fit a "Bayesian" model the data with a poisson model, "random effect" and an autoregressive component, I could call it a "Hierarchical Bayesian spatiotemporal model".

If use a a Generalized-Linear-Mixed-Model (GLMM) with poisson error distribution and autoregressive component I would fit the same model. However, I would only estimate the maximum likelihood, not the posterior distribution. Some more information (reference to Hierarchical Bayesian spatiotemporal model) would be needed to properly answer your question in detail.

Luckly both the "Bayesian" and "Frequentist" fit GLMMs. The difference is the inclusion of prior information and interpretation of the results. Unfortionately technical terminology make it for many difficult to distinguish.

Best,

If your model outputs the likelihood, there are different pseudo r-square measures you can calculate, including Nagelkerke. See: https://stats.oarc.ucla.edu/other/mult-pkg/faq/general/faq-what-are-pseudo-r-squareds/

But there is a question as to what model you would consider the null model when you have random effects.

However, for logistic regression, you might consider the Efron's pseudo r-square described at that link. Or maybe, a count pseudo r-squared, that evaluates the proportion of observations the model predicts correctly.

Also, as Girma Beressa mentions, measures like AIC, BIC, or AICc may be more appropriate to decide among models.

Beyond that, practically speaking, you might make a small data set available, be specific about what model you are using in SPSS, present the SPSS output, and then I'm sure someone can offer the equivalent model in R.

In summary, the method to obtain the Pseudo R2 value in Stata depends on the type of regression model used. To obtain the Pseudo R2 value in Stata, there are different methods depending on the type of regression model used. Here are some ways to obtain Pseudo R2 in Stata: For a feasible generalized least square (XTGLS) regression model, the Pseudo R2 value can be obtained by running the command "xtgls Y X1 X2 Xn, panels( ...)" in Stata. For a generalized linear model (GLM) regression, Nagelkerke's R2 can be obtained using the "roctab" command to create a table of predicted probabilities and then calculate the Pseudo R2 value. For a random-effects probit model using xtprobit, the Pseudo R2 value is not directly available in the output. However, it can be calculated using the formula 1 - loglikelihood of the model/loglikelihood of the null model. If a Stata command does not supply an R-squared value, a Pseudo R2 value can be calculated using different approximations or analogues to R-squared. These approximations are often labeled "pseudo" and can be found in the literature of the field. I am able to add....Best of luck

I would certainly do that, just to check if your independent variables have a linear influence to the DV or not!

It depends on the nature of your study and the data you have. Can you please provide more details about your study?

Shadi Naderyan Fe'li I would like to see your RAW data points and some information: what did you measure exactly and what kind of relationship do you expect?

Chuck A Arize Removing is a bad idea, unless you are absolutely sure that those data points are bad measurements.

Nonlinear transformations are not a good idea, see why:

Jochen Wilhelm some instances use the word "hierarchical" when not all variables are entered together, but separetely or in blocks of variables. For the former one automatic algorithms may be used (but not recommended, as you know) like stepwise regression, whereas for the latter one typically the researcher decides which (blocks of) variables are entered in which order (blockwise regression).

As Bruce Weaver already mentioned, this is done very frequently in psychology, although it may be questioned if this is necessary, yet useful, or just a habit because everyone does it. In most psychological papers in such cases they are only interested in the delta R^2, for example to show that the increase in explained variance of an interaction term is significant (but miss that this could also be done by the squared semipartial correlation and no blockwise regression is needed).

I remember a paper (I believe about job satifaction or something similar) where they entered sociodemographic variables in a first block, typical predictors in a second block and new predictor variables in a third block to show that the typical ones explain more than socidemographic variables alone. The third block to show that the new predictors explain variance over and above the former ones. (In light of causality, I would cast some doubt about the results....)

But you are right, within each block and for the full model, the order of the variables do not have any meaning.

Does it help?

Shushank Sengar, was the data file you shared the full dataset? Or was it a subset of a larger data file that has 30,000 rows? Basically, I think everyone is still uncertain about where the 30,000 you talk about comes in. As you saw in my earlier post, I speculate that the "mean value" in the final column is the mean of 30.000 raw wind pressure measurements. But Jonatan disagrees. Only you can clarify this. Thanks.

A time series regression model will have the form

y_t = \beta_1 + \beta_2 x_{2t} + \beta_3 x_{3t} + \dots + \beta_k x_{kt} +u_t

where y is time series of the independent variable and

x_2,x_3,\dots,x_k are explanatory time series variables. (Certain conditions must be satisfied if the coefficients are to be estimated. There are additional problems to be addressed if any of the series are non-stationary)

While I have seen studies with 30 observations of each variable that have provided useful results generally considerably more observations are usually required.

Look at your data. As far as I can tell you do not have the necessary time series independent variable to do a time series regression. Perhaps I still do not understand.

The regression problems dealing with class or categorical variables may need to have model metrics like sensitivity and specificity. The use of a confusion matrix gives u the model precision metrics--and a good example is when one is performing logistic regression predictions of a variable like groundwater salinity or a phenomenon like landslide susceptibility, given an array of predictors in a study.

For the conventional /Bayesian regression problems that seek to do predictions of continuous variable (i.e. linear regression problems) model metrics like precision and the correlation coefficient may be useful in determining whether your model is reliable.

One may make similar predictions using simpler algorithm like the k-Nearest neighbor, and compare the results of the summary outputs generated in the statistics involved when one uses the conventional (or even Bayesian regression)

Overall, the kind of data one has and the size of the data will greatly influence the regression outcomes during Multiple or just the simple linear regression predictions.

The other key factor in classification regression metrics output is the size of datasets and class imbalance-the training and testing subsets determine greatly the model metrics and accuracy. When one has datasets with one class of data dominating the other(s), the model may need fine tuning before being analyzed.

With small sample sizes you should

Sure Bruce Weaver I was running logistic regression as a part of the propensity score matching technique. While watching the tutorial video on Youtube (https://youtu.be/2ubNZ9V8WKw) I realized I am including variables in the equation that I shouldn't have. So, I excluded them and magically, the error did not appear anymore. It was that simple.(: Good luck to everyone who's facing this error!

"Thus, statistically the predictor has an effect on the outcome, but that this effect is very small and may be of little clinically or otherwise "practically" significant?"

Exactly.

The statistical significance is an indication that your data provides enough information about the effect to be able to distinguish the estimate (here: 0.9965) from the hypothesized value (here: 1.0). Since the estimate is lower than the hypothesized value, you can conclude that the (still unknown!) effect is also lower.

The confidence interval is the range of all hypothesized values for which the data would not be statistically significant. Hence, this is a range of effect values that are "not too incompatible" with your data. Values outside the interval are "too incompatible" with your data. You see that the (0.95-)confidence interval in your case does not include 1.0, which corresponds to your p value being < 0.05. So, whatever hypothetical value is not too incompatible with your data is lower than 1.0, but very close to 1.0. This may not include any value of clinical or practical relevance.

It is very well possible to have significant interaction effects but no significant "main" effects. It is also possible to have significant "main" effects and no significant interactions. And it is possible that variables have significant zero-order correlations with the dependent variable but no significant regression coefficients (e.g., due to redundancies and/or overfitting of the regression model). Finally, it possible to find non-significant zero-order correlations between IVs and DV but significant regression coefficients (e.g., due to suppression effects).

Many of these effects cannot easily be identified from the zero-order correlations. That's one reason why we run multiple regression analyses--to identify redundancies, interactions, suppression effects, etc. that are not easy to see in a bivariate correlation analysis.

Note also that the "main" (lower-order) effects (and their significance) in the moderated regression model depend on whether you centered the predictors or not. This can make a huge difference for the interpretation. Especially when predictor variables do not have a meaningful zero point, centering prior to calculating the interaction terms is recommended (e.g., Aiken & West, 1991; Cohen et al., 2003). Otherwise, the lower-order terms (and their significance) may not be interpretable at all.

In any case, it would be a good idea for you to plot the effects to get a better understanding of what is going on. That is, look at the regression lines for different values of your moderators to understand the meaning of the interaction effects.

Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park: Sage.

Cohen, J., Cohen, P., West, S. G. & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Erlbaum. (Chapters 7 and 9)

Given the subjective nature of dementia, each of the tools/techniques you have mentioned would be to be carried out individually, right?

In that case, that would require individual data collection and curation.

Which translates into hardware and personal time requirements/allocation.

How does one solve this challenge then?

I would like to restart this question.

I have noted a few papers on cutoff or quasi-cutoff sampling other than the many I have written, but in general, I do not think those others have had much application. Further, it may be common to ignore the part of the finite population which is not covered, and to only consider the coverage, but I do not see that as satisfactory, so I would like to concentrate on those doing inference. I found one such paper by Guadarrama, Molina, and Tillé which I will mention later below.

Following is a tutorial i wrote on quasi-cutoff (multiple item survey) sampling with ratio modeling for inference, which can be highly useful for repeated official establishment surveys:

"Application of Efficient Sampling with Prediction for Skewed Data," JSM 2022: 

https://www.researchgate.net/publication/362370770_Application_of_Efficient_Sampling_with_Prediction_for_Skewed_Data 

This is what I did for the US Energy Information Administration (EIA) where I led application of this methodology to various establishment surveys which still produce perhaps tens of thousands of aggregate inferences or more each year from monthly and/or weekly quasi-cutoff sample surveys. This also helped in data editing where data collected in the wrong units or provided to the EIA from the wrong files often showed early in the data processing. Various members of the energy data user community have eagerly consumed this information and analyzed it for many years. (You might find the addenda nonfiction short stories to be amusing.)

There is a section in the above paper on an article by Guadarrama, Molina, and Tillé(2020) in Survey Methodology, "Small area estimation methods under cut-off sampling," which might be of interest, where they found that regression modeling appears to perform better than calibration, looking at small domains, for cutoff sampling. Their article, which I recommend in general, is referenced and linked in my paper.

There are researchers looking into inference from nonprobability sampling cases which are not so well-behaved as what I did for the EIA, where multiple covariates may be needed for pseudo-weights, or for modeling, or both. (See Valliant, R.(2019)*.) But when many covariates are needed for modeling, I think the chances of a good result are greatly diminished. (For multiple regression, from an article I wrote, one might not see heteroscedasticity that should theoretically appear, which I attribute to the difficulty in forming a good predicted-y 'formula'. For psuedo-inclusion probabilities, if many covariates are needed, I suspect it may be hard to do this well either, but perhaps that may be more hopeful. However, in Brewer, K.R.W.(2013)**, he noted an early case where failure using what appears to be an early version of that helped convince people that probability sampling was a must.)

At any rate, there is research on inference from nonprobability sampling which would generally be far less accurate than what I led development for at the EIA.

So, the US Energy Information Administration makes a great deal of use of quasi-cutoff sampling with prediction, and I believe other agencies could make good use of this too, but in all my many years of experience and study/exploration, I have not seen much evidence of such applications elsewhere. If you do, please respond to this discussion.

Thank you - Jim Knaub

..........

*Valliant, R.(2019), "Comparing Alternatives for Estimation from Nonprobability Samples," Journal of Survey Statistics and Methodology, Volume 8, Issue 2, April 2020, Pages 231–263, preprint at 

https://www.researchgate.net/publication/337950671_Comparing_Alternatives_for_Estimation_from_Nonprobability_Samples 

**Brewer, K.R.W.(2013), "Three controversies in the history of survey sampling," Survey Methodology, Dec 2013 -  Ken Brewer - Waksberg Award article: 

Thom Baguley thank you very very much, I get it now. Very helpful explanations and sources!

With so few observations (n = 9), there will be a lot of sampling error plus the potential for outliers/extreme observations to have a large influence on R squared. With a larger sample (e.g., n = 95), there is less sampling error and less potential for extreme cases to influence the results. I would trust the estimate of .94 (for n = 9) less than the estimate of .73 (which is based on a more substantial sample size).

Hello Bieke,

If you genuinely believe that the unit difference separating "no frame" and "low" is exactly the same as that separating "low" and "high" for an IV, then yes, you may code them as -1, 0, 1 (or any other linear transformation thereof). However, if you don't believe this, then you must opt for recoding, replacing a three-level variable with two dummy variates.

I suspect that the reason one of your variables is being dropped is due to redundancy, and this could be caused by: (a) no instances of one of the three categories in the data set; (b) IV1 and IV2 are redundant in some way (e.g., every case is in the same level for both variables); (c) a mistaken recoding.

Are you also including all four of the dummy variate products to capture the information about their interaction?

Good luck with your work.

Bruce Weaver , he also states that SPSS has a procedure for SUR with correlated errors. Given the specifics that Samuel Oluwaseun Adeyemo 's post includes either he is working with a system beyond what (https://www.ibm.com/support/pages/seemingly-unrelated-regression) is refers to that we don't know about, or there is some other explanation for how he came up with this information. Shame in his response above he did not answer where he got this information. Since he didn't answer on the other thread, the using a new version explanation seems unlikely.

Hello Ibikunie,

1. Can an adjusted R-squared exceed 0.99? Yes.

2. Can an adjusted R-squared exceed the associated, unadjusted R-squared? No, though they can be equal (as pointed out by Debopam Ghosh )

Here's one version of the most commonly used formula for adjusted R-squared:

where:

R2 = unadjusted/observed R-squared

n = sample size used in model

k = number of IVs included in the model

Note that, the ratio (n - 1) / (n - k - 1) will always be less than one for k = 1, 2, ...n - 1. So, unless the unadjusted R-squared = 1, the adjusted R-squared will be less than the unadjusted R-squared. As N increases, relative to k, the difference between the two values will decrease as well.

Good luck with your work.

As for literature, I found Joseph Hilbe's books Negative Binomial Distribution and Modeling Count Data helpful for my models.

Rede Ganeshkumar Dilipkumar so to answer the hypothesis test and relationship between regressor and regressand we need surely normality test right?... because oftenly in causal relationship method we have to prove if the hypothesis alternative is the answer or instead hypothesis null is the answer for answer the influence of regressor to regressand. So after we found the best fit model of regression whether pooled effect, fixed effect or random effect than we have to continue by finding the influence between regressor and regressand so you said that hypothesis test surely need normality test and other assumption test? .. Please put the recommended theory or reffrence to strengtening your argumentation. thank you for the enlightment..

Hi,

did you see this paper?

They refer to reference [9] to calculate CIs based on logreg for RERI, AP and S. Perhaps that could work for you?

This is reference [9] in the paper mentioned above:

Hosmer DW, Lemeshow S. Confidence interval estimation of interaction. Epidemiology. 1992;3:452–456.

What you are looking for is called linear regression. Good news is that linear regression is quickly done and easy to interpret. It will also give you more statistical power, as by categorization you loose information.

And don't worry, I've seen this categorization non-sense done by seasoned professors (and sometimes forced upon their students). That's from a past era, where you literally had to crunch the numbers with pencil and paper, which is easier with categories.

Anureet Kaur Can you please point me to the answer you found? I seem to have the same dilemma here. I can do cross-validation on the log-transformed data (independent variable log-transformed), but I have several models to compare, some using log-transformed data, some not. It is obviously not possible to simply inversely transform the log-model validated MSE and RMSE, since it gives incomparable results vis a vis the models without transformation, which suggests such inversely transformed MSE and RMSE are invalid.

I am experiencing the same issue. Could not figure out why.

Error in nnet::multinom(OverallCondition ~ ., data = data_house, family = "binomial") :

need two or more classes to fit a multinom model

In addition: Warning message:

In nnet::multinom(OverallCondition ~ ., data = data_house, family = "binomial") :

groups ‘Poor’ ‘Average’ ‘Good’ are empty

I am sitting with the same error around 2 hrs.

This (-.81) seems like a pretty substantial correlation to me! Nonetheless, it would not hurt to take a look at the scatterplot to check for potential non-linearity. Also, if you have other variables that should also be related to the DV in theory, you could add them to the model.

You have quasi-complete or complete separation. Ideally you need more data - or to use an approach that adds more information (e.g., Bayesian regression - though other options exist).

Dear Dr. Li,

Thank you for your attention. Your answer gives me some inspiration. I will try the approach. Thank you again.

If there is data, you can build a model. The key is to see the relevance of the model.

There is no assumption about normality of predictor or outcome variables in linear regression (only about the normality of residuals/errors). Also, linear regression is designed to handle correlated predictors, so there should be no problem unless the correlations are extremely high.

This is for the crushed stone industry in the US:

https://www.usgs.gov/centers/national-minerals-information-center/crushed-stone-statistics-and-information

I'm told these quarterly surveys are for "a select set of companies" which reminds me of how the quasi-cutoff sample of electric sales in the US got started. The electric sales survey of a select group of entities was later modified and used as a sample, first a stratified random sample with a large company censused stratum, and then only the censused stratum as a quasi-cutoff sample, all after starting an annual census of all electric sales by economic sector (residential, etc.), from the production/supply side. If one wanted to monitor the crushed stone industry the same way, I would suggest this approach using a quasi-cutoff sample with a ratio model for prediction, as is done at the US Energy Information Administration (EIA).

Does anyone know of other surveys, each of a select group of larger establishments being followed, where there is a chance to instead have an occasional census to be used for regressor data for the same data items in a more frequent sample?

Its very hard to answer without more information. Generally the intercept isn't of much interest in a model, but there are exceptions. Its the expected value of the outcome Y when all predictors are zero. Usually that's not an interesting quantity and testing whether it is different from zero is also rare of interest. So in most cases its fine, unless there is good reason to believe the intercept should not be zero or close to zero in this context.

Old-fashioned stepwise regression is not recommended. Here are some links you may find useful.

Standardized regression coefficients are also problematic. See John Fox's comment in the attached file, for example.

The 2nd link about includes some suggestions about other approaches you could try (e.g., LASSO). You could also read up on dominance analysis.

HTH.

For dichotomous dependent variables, you could use logistic regression.

However, I would discourage you from dichotomizing your independent variables (leave them in their original metric). Dichotomizing leads to a loss of information and can reduce statistical power. There is almost never a need to dichotomize variables from a statistical perspective because there are many statistical procedures available that can handle ordinal-polytomous and/or continuous (metrical, interval-scale) variables. When you type "dichotomizing continuous variables" into Google Scholar, you will find a large number of relevant references that explain why this is a bad idea.

Thank you prof. Kelvyn Jones for your usual informative response for my concerns. I downloaded the first reference 1st ed. freely and I`m exploring it.

Get a book on regression. This a standard ANOVA type regression problem. I am attaching such a set of notes for anyone that is interested. David Booth

You can use

model.predict_proba(x_test) to get the probability values if using the sklearn library.

Thank you very much David. I always appreciate your efforts to educate me.

Sujata Sujata Both methods have contextual applications depending on how the data is analyzed. The idea of bootstrapping SE is variability especially if the sample is not randomized otherwise repeated sampling may apply. Whereas Delta method is for approximation of already randomized observation.

You cannot choose any except you justify the context.

Best of luck

Prediction-based inference from finite population sampling is well-established.   See

Valliant, R, Dorfman, A.H., and Royall, R.M.(2000), Finite Population Sampling and Inference: A Prediction Approach, Wiley Series in Probability and Statistics,

and

Chambers, R, and Clark, R(2012), An Introduction to Model-Based Survey Sampling with Applications, Oxford Statistical Science Series. 

This is also in parts of other sampling books such as Thompson, S.K.(2012), Sampling, 3rd ed, John Wiley & Sons. 

A classical ratio model is based on less heteroscedasticity than one may usually find in survey data, except when the effective coefficient of heteroscedasticity is reduced by data quality issues from the small responders.  It can be very useful in Official Statistics.  We can estimate sample size requirements for a subpopulation or a population modeled well by a single such model, using a sample drawn in any way which does not miss any important subdivision which would indicate more than one model is needed.  The format for the "formula" for estimating the sample size in such a case is similar to what is found for a simple random sample in Cochran, W.G.(1977), Sampling Techniques, 3rd ed, John Wiley & Sons.  See https://www.researchgate.net/publication/261947825_Projected_Variance_for_the_Model-based_Classical_Ratio_Estimator_Estimating_Sample_Size_Requirements.  Both cutoff/quasi-cutoff sampling and balanced sampling are discussed. 

Thus we can estimate the sample size required here to make satisfactory inference from a population or subpopulation which falls under the purview of one model-based classical ratio estimator.  This simple result is relatively easily verified. 

So we can see that when a simple model is appropriate, as when a single predictor is the same data item in a previous census, we may more feasibly infer from a nonprobability sample.  

[Note:  Please also see https://www.researchgate.net/post/Can_we_use_inferential_statistics_when_we_have_used_non-probability_sampling.] 

Does anyone have any other experience to discuss?

Was the experiment used to collect the data planned through any of the statistical methods of Design of Experiments?

Start with the idea of a single dichotomous variable scored 1 and 0. When you enter this into your regression, the observations scored 0 will become the omitted category, which will then be represented as the intercept of the regression equation. Regardless of how many categories you have, you will always need to omit one to serve as the intercept.

Another day to think of this is in terms of degrees of freedom. A dichotomous variable only has one degree of freedom because if you know the grand mean and the mean of either category, then you can automatically calculate the mean of the remaining category.

Hello Hong,

Here's one: Consider a variable that generally shows strong positive skew (for example, annual income among residents of a country). There's actually several issues which arise from this:

1. There can be many orders of magnitude separating largest, smallest values in the data set. Too many can threaten the accuracy of computed model coefficients.

2. The strong asymmetry can result in model residuals being non-normally distributed.

3. The strong asymmetry can result in a fitted model being more non-linear in form than linear.

For reasons such as this, transforms such as log (regardless of base) can be helpful.

Good luck with your work.