Multivariate Data Analysis - Science topic

Thanks a lot ! I'll check and try it

Loadings are interpreted as the coefficients of the linear combination of the initial variables from which the principal components are constructed. From a numerical point of view, the loadings are equal to the coordinates of the variables divided by the square root of the eigenvalue associated with the component.

Thanks for your reply.

Hello Jialing,

The fact that DVs are correlated is often one argument for choosing a multivariate method to analyze the data rather than generating a univariate model for each individual DV.

If what you are saying is that, causally or temporally, is that Y1 influences Y2, then perhaps you'd be better off evaluating a path model which incorporates such proposed relationships rather than simply allowing for correlations/covariances other than zero among the DV pairs in the multivariate regression.

Good luck with your work.

Yogesh Kumar

These link might help

(a v.test over 1.96 is equivalent to a p-value less than 0.05 )

Qiqi van der Kolk Yes, it is common to report adjusted R² when conducting a multivariate multiple regression because the more predictors you add to the model (unadjusted) R² will increase, which may inflate the actual variance explained. Adjusted R² takes the number of predictors into account. However, with multivariate regression, you will have multiple R² values rather than just one. Specifically, you will have an R² for each dependent variable (criterion). If you have, say, 3 dependent variables you will have 3 adjusted/R² results.

I would assume it doesn't matter how you calculate the adjusted R² for your thesis, unless you wanted to include the computer output from Stata directly into the paper - or - if your program has some requirement that results of statistical tests must be conducted via certain software? I'm confident adj. R² can be calculated in Stata. It might just take a bit of digging:

Hello Esther Curtin

Multicollinearity makes some of the significant variables under study to be statistically insignificant:

Good Luck

Clustering or classification methods

Hello Matteo,

It would be correct to say that the centroid (mean on the linear composite of the variables forming the first discriminant root or function) for the grasslands group is further from the aggregate (all cases) centroid, or from the centroids of the other two groups. However, the display doesn't inform you as to what variable(s) are most influential in the function.

For that, you'd have to look at both: (a) standardized function coefficients, and (b) variable-function correlations (sometimes called loadings or structure coefficients). If variables are uncorrelated, then standardized function coefficients alone will let you know the relative magnitude of emphasis being placed on each variable in the function. If correlated, then you have to look also at loadings to be sure that you're not letting collinearity confounding your interpretation of relative emphasis as also indicating importance. So, the information presented isn't enough to presume that a single variable (biodiversity) is the reason for the separation in group centroids.

Good luck with your work.

Hello Mario,

If your check of multivariate normality suggests a satisfactory conformance of your data set to the target distribution, I'd forget about jettisoning any cases as outliers unless there was some more compelling reason (such as, impossible values, coding errors, etc.) for doing so.

Good luck with your work.

I think you can check the sign of the coefficients. If the sign is positive it might be complementary, otherwise supplementary effects can be deduced.

Independent variables are controllable variables. Researchers pick their independent variables and manipulate them. The independent variables affect the dependent variables. In the survey, researchers can view the respondents as dependent variables and the questions as independent variables. The researchers can choose what the questions will be, giving them control over them. The ways in which the respondents behave in the survey depend on the questions. Researchers want to develop questions that will ensure that respondents give the most exact answers possible so the survey results are reflective of the real views of people surveyed.

Dear Muhammad A. El Hameedy I am very thankful to you for your answers to my questions.

I agree. Find the Pearson correlation between each independent variable and the dependent variable first and then proceed to use Multiple Linear regression

please I need this publication book.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7 ed.). Upper Saddle River, NJ: Pearson Education.

nsango@iut-dhaka.edu

One way is using PCA Biplot.

OK.

MANOVA works by analysing a linear function of the DVs rather than the raw DVs. For example it could be

Ycomp = a*DV1 + b*DV2 + c*DV3

The weights a, b and c are optimised so that the variance explained in Ycomp by the DVs is maximised (as if in a one-way ANOVA).

This seems like a reasonable thing to do but it has some strange consequences. Firstly, its atheoretical so it might not be interpretable even if the separate DVs are. Second, it optimises on the data so will capitalise on sampling variability and other characteristics. Thus an identical replication using the same analysis will actually use a different composite DV. Third, standardised effect size statistic relate to the composite so are even harder to interpret than normal.

I'm not a fan of MANOVA for these and other reasons (e.g., it doesn't protect against Type I error in the way people assume).

I'm using MATLAB: PLS toolbox for analyzing my data and i have a problem using EPO preprocessing for PLS-DA.

when im using EPO i get this error which the toolbox cannot perform cross validation. can anyone help me with this situation?

Regarding the Wilcoxon signed rank test, be very careful. The test is based on a ranking of the differences between pre-test and post-test scores. In other words, it silently assumes that the "distance" between two consecutive points is defined and - !!! - equal. 2-1 = 6-5 = 4-3, etc. Moreover, what does it mean to have completely-satisfied (10) minus completely-satisfied (0)? Completely-satisfied (10)? In ordinal data you operate on labels, not numbers. And what if you get fractional outcome, like 5.5? "Neutral and a half" ? Technically you can do everything you want, but it should have underlying sense. Also, quantile regression won't help there. But there is a good tool for that purpose: the ordinal regression (aka proportional odds model) - ideal for dealing with Likert items. It also "integrates" with mixed models, through which you can account for the repeated observations, i.e. choose the covariance structure (via either random effects covariance structure, residual covariance structure or both). And of course, you're not limited to just one covariate.

If, however, you believe such "distance measure" exists for your Likert items, or that fractional outcomes still make sense, having 10 points, you could try classic parametric methods, assuming interval scale or non-parametric tests in case of distributional issues.

Hello Belkacem,

While someone more familiar with aquifer variables might "see" meaning in these arrangements, it is also entirely possible that there is none, since:

PCA arranges the variable set into linearly independent composite variables, such that the first one extracted will always account for the maximum amount of total variation in the data set, while successive components will account for successively lesser amounts. There's nothing inherent in the process that assures the resultant component/composite variates will necessarily be meaningful.

I'd strongly suggest you try rotation of the extracted components; this will frequently aid in trying to make sense of what each component represents. As well, you should set some criterion for the variable-component loading (correlation) to be declared "salient" for purposes of trying to characterize the meaning of each component. In other words, pay attention only to the variables that show salient relationships to a component when trying to characterize what that component might represent.

Good luck with your work!

If you use PCA, you can run any analysis you want on the newly created PCA variables, including correlation with other variables.

If you use CCA, you need an environmental matrix.

Your answer lies in your data structure and what you hope to accomplish.

go to,paper, that will definitively help you in this regard

The underlying package has been updated (current version: 2.2.0). Based on the refined methods higher order dependencies can be detected in more cases and with better accuracy.

In particular, new features for the dependence structure detection are:

* The approximate probability of a type I error is provided.

* New option 'type': In particular, 'type = "pearson_approx"' provides a fast and approximately sharp detection, in contrast to the original 'type="conservative"' which is still faster but much more conservative.

E.g.

dependence.structure(x, type = "pearson_approx")

* New option 'structure.type': The original algorithm clusters dependent variables and treats them thereafter as one variable. This is still the default option 'structure.type = "clustered"'. But in the case of many variables this can cluster variables which are only indirectly dependent via some other variables. In contrast the new option 'structure.type = "full"' treats always each variable separately and detects dependence for all tuples which are lower order independent. E.g.

dependence.structure(x, type = "pearson_approx", structure.type = "full")

Based on this many datasets feature higher order dependence. I am looking forward to hear from field experts, who can also provide an explanation within their subject for the occurrence of these higher order dependencies.

The richer you are the more you save

The taller and good looking you are the higher your income

The more fashionable is your neighbourhood... the more expensive your house will be.

OK? You get it: the residuals on each of the auto-correlated variables will move in the same direction.

Should not be confused with multi-collinearity when 2 or more variables mean the same thing and become redundant.

If you are interested in the general properties of your data set, like any multivariate data set with non-trivial signals encoded in a matrix, its strength to infer groups, you should give network-based exploratory data analysis a try.

You'll find a bunch of related posts with ideas and links to literature on the Genealogical World of Phylogenetic Networks (GWoN)

Especially the neighbour-net is a most versatile tool for multivariate analysis, however, largely unknown and massively underused. Its applications in phylogenetics or linguistics are multi-fold but it can be very revealing for entirely different data (you didn't mention whether you are working with biological or other data), a few of the examples we explored on GWoN:

A network of breat sizes (to test a claim) – https://phylonetworks.blogspot.com/2018/01/false-reports-of-us-womens-breast-sizes.html

A network of gun legislation in the U.S. (to illustrate diversity) – https://phylonetworks.blogspot.com/2018/03/visualizing-us-gun-laws.html

A network of moons (for the purpose of classification) – https://phylonetworks.blogspot.com/2018/06/to-boldy-go-where-no-one-has-gone.html

A network of party programmes (to illustrate overall similarity and dissimilarity, also over time) – https://phylonetworks.blogspot.com/2018/10/jumping-political-parties-in-germanys.html

A network of playmates' change over time – https://phylonetworks.blogspot.com/2016/10/changes-in-playboys-women-through-60.html

Visualisation of the data behind rankings, e.g.

bikability and public transport in U.S. cities – https://phylonetworks.blogspot.com/2019/03/which-us-cities-are-best-for-walking.html

world happiness – https://phylonetworks.blogspot.com/2018/07/a-network-of-world-happiness.html

The book

" Experimental Design" by Federer

will be helpful in this situation.

A physical law is always good but I thing that an empirical correlation that you found under your experimental conditions with your specific materials is better. When I do FTIR-ATR experiments I prefer my calibrations over external ones like PNNL and I would prefer using it also over theoretical ones (but maybe that's just me)... good luck

Joanne M Eaves , G*Power can do *multiple* logistic regression, but I don't think it can do *multinomial* logistic regression.

Hi

How many variables you have and what is the length of your time series.

Different related software's can be used for building ANN predictive model using NIR spectra. My suggestion are Unscrambler and IBM modeler. Before ANN modeling, PCA should be done to reduce large spectra variables to PC1, PC2 .... . after calibrating ANN, you have predicted values by calibrated model and you have also reference values for each sample. Now use RPD index to realizing the goodness of your calibrated model.

Try to make a discriminant validity test. It can happen that your constructs are overlapping (measuring the same). This could explain why you have fewer factors than expected from the EFA.

I recommend also the Common Method Bias test. I saw situations when uninterested participants selected answers 'on the right' or 'in the middle's and it caused discriminant validity issues.

If these tests are fine and your constructs come from literature - report Alphas and composite reliability and conduct your main tests. Otherwise: 1. Delete constructs (I would not reduce them to 8! with the items from different constructs). 2. Try second order constructs. If some constructs are to close, maybe they are a part of the same blatent second-order construct?

Good luck

For classification, you can perform the PCA (Principal Component Analysis)

Hi Stacey,

Seems that you did it well. In his example about Tikus data (same transects sampled several years), D Warton says that: "the transects serve as blocks, but we want to test for an effect of time which operates within blocks so blocks cannot be resampled. Instead, time labels could be resampled within blocks if you input a matrix of bootstrap labels using the permute package".

You could therefore construct something like:

permID=shuffleSet(n=...,nset=...,control=how(block=data$treatment))

Then Fit 2 models (with and within treatment) and compare them with anova using bootID=permID in the function.

Hope that help.

I contact the author of this page http://cc.oulu.fi/~jarioksa/softhelp/vegan/html/adonis.html

and the answer is:

"Good question. I should probably change that definition. Variation explained is directly analogous to that of general linear models. With a continuous variable, it acts like simple linear regression, where each point is associated with its own "centroid" which is the best fit linear approximation." Now everything makes sense.

It has an outcome (INTENSITY with two categories: Positive and Negative) and four independent variables (Commerce and industry, farming, fishing and law and order). Thanks.

You need to include a variable with no missing data otherwise it wont work or at least each participant should have some data - complete missing cases dont compute - hope that helps.

Following

Dear Kulkarni,

Please see it link below.

All effects are significant.

SPSS Newbie here. Feel free to mock me, I now realize how what I did was ridiculous. But it might help others in the future searching this answer.

I ran a bivariate correlation on one measure of my scale against only one demographic. Thus...that column is constant--all 1's. Thus there is nothing to correlated against.

I found this software very convenient to create boxplot or violin plots with datapoints: http://vinci.bioturing.com

Dear RG colleague,

The KMO statistic, which can vary from 0 to 1, indicates the degree to which each variable in a set is predicted without error by the other variables. A value of 0 indicates that the sum of partial correlations is large relative to the sum correlations, indicating factor analysis is likely to be inappropriate . A KMO value close to 1 indicates that the sum of partial correlations is not large relative to the sum of correlations and so factor analysis should yield distinct and reliable factors. Hair et al. (2006) suggest accepting a value of 0.5 or more. values between 0.5 and 0.7 are mediocre, and values between 0.7 and 0.8 are good.

Bartlett’s test of sphericity is a statistical test for the presence of correlations among variables, providing the statistical probability that the correlation matrix has significant correlations among at least some of variables. For factor analysis to work some relationships between variables are needed. Thus, a significant Bartlett’s test of sphericity is required, say p<.005.

Analyze, Dimension Reduction, Factor Analysis, and then select Descriptives.

Dear Srikanth, thank you for your question.

A discriminant function is a model or equation generated in Discriminant analysis for differentiating or discriminating between the groups or classes given from the original variables. The discriminating functions will subsequently produce the coefficients and discriminant scores as desired.

Like in factor factor analysis, the factor functions or models are used to produce coefficients and factor scores. The difference in factor analysis is that the most correlated groups are grouped together whereas in discriminant analysis, the different groups are being differentiated.

Here is the software

Govinda,

If the problem you are talking about is related to encountering non-constant variance in your model, I would recommend weighted least square approaches.

Hope this helps.

(link: https://onlinecourses.science.psu.edu/stat462/node/186).

If you want to integrate the six of them in one go I would rather use multiple co-inertia analysis, regularized generalized CCA or STATIS. Refer to Meng et al. 2016:

I personally dislike SIMCA, it is highly limited. In any case I presume you have to provide two separate matrices at a time for O2PLS-DA. As for the other methods, you can run PCA, PLS-DA, etc. on the entire thing - however note that you run the risk of contaminating results with the technical noise specific to each separate omics readout. Hope this is clear? Cheers

As Jos notes, this issue has been (and will continue to be) debated many times.  I like Jeremy Miles' post on the Stack Exchange page linked below. 

Also, when it comes to Likert scales, I think some people forget about the distinction between Likert-type items and Likert scales.  See the second link below. 

HTH.

Please let me know if the following references/sites are helpful to you in your quest:

1.  Why Propensity Scores Should Not Be Used for Matching - Gary King

https://gking.harvard.edu/files/gking/files/psnot.pdfDec 16, 2016 ... We show that propensity score matching (PSM), an enormously popular ... proximates random matching which, we show, increases imbalance ...

2.  An Introduction to Propensity Score Methods for Reducing the ...

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144483/Jun 8, 2011 ... Propensity score matching entails forming matched sets of treated and untreated subjects who share a similar value of the propensity score ...

3.  Propensity Score Matching: Definition & Overview - Statistics How To

http://www.statisticshowto.com/propensity-score-matching/Feb 3, 2017 ... Statistics Definitions > Propensity Score Matching What is a Propensity Score? A propensity score is the probability that a unit with certain ...

4.  Propensity-Score Matching (PSM)

http://cega.berkeley.edu/assets/cega_events/31/Matching_Methods.pptPropensity score matching: match treated and untreated observations on the estimated probability of being treated (propensity score). Most commonly used.

5.  Understanding Propensity Scores

http://bayes.cs.ucla.edu/BOOK-09/ch11-3-5-final.pdf11.3.5 Understanding Propensity Scores. The method of propensity score ( Rosenbaum and Rubin 1983), or propensity score matching (PSM), is the most ...

Dennis

Dennis Mazur

As for statistical analysis: You can use the r language as effectively as SPSS and it is now being used more widely than either SPSS or SAS. There are a lot of tutorials and demonstrations of ways to use it on YouTube. I think a few might even be in Spanish.

Also see:

Yes, do use this formula to make the scales (factor scores) 100 point:

y = ((factor - min)/ (max - min)) * 100

Forget the T-score. Sorry for the confusion.

You can transform the factor scores with the above formula.

Example:

min = -1.47203

max = 1.8796

factor1 =  0.96967

y = ((0.96967+ 1.47203)/ (1.8796  + 1.47203)) * 100 

y = 72.851

Remember that factor score are standardized (mean = 0, std = 1)

Why don't you try:

It is an online app based on the following R package:

Good Luck !

Hi Paul

Thank you for your answer.

Many thanks to both Hendrik and Mehmet.  We'll see if we can incorporate your feedback into our project!

Hello Anne.  You wrote:

"For example, I have 24 items (answers either yes--> 1 or no-->0) of which 12 items belong to subcategory 1 and 8 items to subcategory 2 and again 8 items to subcat. 3. I would like to make one variable out of all 24 variables but without "loosing" my subcategory arrangement."

I have two questions:

HTH.

Thanks a lot, I will check the source.

Hi,

Matrix singularity can have multiple causes, but a common one is two or more variables in the analysis are perfectly correlated. Have a look at the correlation matrix of your variables to see if you can spot the source of trouble. In the meantime, here's a good statsexchange discussion about singularity:

A few points:

1) if you want to be able to compare the 3 behavioral factors then that should be added as another repeated factor.

2) You could do this as a repeated measures ANOVA but using the MIXED procedure in SPSS would be more straightforward given the missing time point for the control group, etc. However, you will need to have the data in long form.

3) Scrap the Bonferroni control and focus on effect sizes, CIs, future replications, etc. instead of just p-values. I have attached a paper that discusses this issue.

Your code seems very messy and difficult to check. At various places "l" (ell) is the first subscript and in others the second. Your use of transpose (especially in mu) seems incorrect. The core is P(t) [l|i] ... what is the sum over i of that measure? All the rest is algebra. I never use "I' (ell) as a subscript. Too easy to mix up with 1 (one) , I  (capital eye) , | (vertical bar) , i (small  i), ...  you get the idea.

Assume we have data as P(ell given eye) ,,,,, "ell" is the column and "eye" is the row. Is your convention that vectors are row vectors and transpose is a column?  Have you been 100% consistent with that? You can make a column vector of "1" (ones) and get the sum over "i" by PT x 1 --- or you can have a row vector of 1s and do 1 x P. You have to do all this carefully.

If this is all correct, you have to go back and see why is sum over i of P(t) [l|i] = 0.

Thank you for all your responses.

The question I would like to ask is

1) do the spatial patterns of diversity differ among taxa? For example one taxa may show high clustering of sites based on habitat type, while another will show similar composition across all sites.

2) do spatial trends differ over time- for example, one taxa may show stable composition in all habitats over time while another taxa may show convergence of composition between habitats, and a third taxa may show high variability over time for one particular habitat... 

The intention is to demonstrate the different taxa have different spatial and temporal distributions and therefore can or cannot be used as surrogates for each other based on composition. 

3) Characterise the similarity (betadiversity) between and within habitat types, based on all taxa. 

I can of course compare univariate measures of diversity in each site using anova but I would like to compare the taxa based on their composition. 

Thanks for further suggestions to address these specific research questions.. 

Dear Phil,

you can collect the data with any software.

The key-point then, later, before data analysis is to be able to restructure the database in the way you want. It is a matter of "statistical unit". For example, in your case, the statistical unit could be the employee, with a huge number of variables like evaluations from periods 1, period 2, period 3 etc. A good software I know to operate changes in statistical units is Sphinx. 

Is it clear enough? Regards.

Stephane

Thanh

This question is a conundrum that has been bugging investigators using cluster analysis ever since the technique was developed. In the past, you simply selected a fixed level of distance similarity or dissimilarity and used that as a threshold to define groups (clusters) with no minimum number of samples. As Moacyr correctly states above, even one sample would be acceptable if it was sufficiently different from all of the other samples.

Now there is a statistical technique which applies permutation tests to allow you to identify significantly different clusters of samples. You need the software but perhaps someone at your university will pay for it OR perhaps there is something similar in R. See the attached citation.  I don't sell their software or anything so I have nothing to gain here but I've used it and it works well.The paper describes the technique I mention references earlier papers and describes several other interesting techniques.

Hi Mohammad - did you have any success with this, I'm looking for software to accomplish something similar.

Two items are not enough for a factor analysis because they generate only a single correlation, but you certainly can combine them by taking an average, as you suggest.

You need to create the latent variable "trust", e.g. by doing a factor analysis with the variables trust1,...,trust5 (which supposedly define one factor) and saving the factor scores. Then you enter the factorscores into the U-Test.

It seems superior, well done.

Nevertheless, as it is described, the new variable may be bedeviled. 

Dear Zanyar,

    You can use MANCOVA on data with this sort of arrangement. The question you need to consider first is what hypothesis is really being tested. The reason for jointly considering Time and Score is that you believe that they will jointly rise or fall. If the impact of condition is not necessarily the same over both variables then a joint test may not make sense.

   In the context of cognitive psychology experiments there is often a tradeoff between speed and accuracy which varies by subject. Before interpreting your result you need to consider whether or not this is likely to be the case. Negative correlations or bend correlations would be a clear sign. I will also point out that accuracy measures are likely to show ceiling effects and time measures often show right skew. Both should be managed as likely violations of the Manova assumptions.

   The traditional solutions revolve around either analyzing time and score separately or building a composite score based on ROC curves. For example A' can be used to capture any tradeoff. There are also plenty of penalty functions that people have tried. 

   Returning to your original plan, a T X S X S interaction suggests that the changes in Time vary with both Score and Sensitivity. Alternately you can say that  the changes in Score vary with both Time and Sensitivity. This is simple enough, but suggests that you need to conduct follow up tests to describe how Time and Score are interacting and how this interaction changes over the range of sensitivity. This can all be done within the MANCOVA framework, assuming the distributions make sense.

Good luck.

Karen, the ratio scale of measurement is even of a higher level than the interval scale and ANOVA is even better with the ratio scale than the interval scale. In such a case, a parametric test based on the usual Gaussian distributional assumptions would be applicable.

In case you consider a nonparametric alternative:

If all you have is a T Score for a single sample of people, then no - it's a descriptive statistic and there is no comparison being made.

However, if you have T Scores for at least two groups (or conditions), then yes! In fact, the T Scores make it easy. For instance, suppose you have scores on a memory test from before and after an intervention (such as alcoholics entering treatment, then retested after a month of sobriety). All you have to do is subtract one score from the other, then divide by 10. 

Many effect size statistics work this way: take the difference in mean scores and divide by the standard deviation. Where they differ is in what standard deviation is used (e.g., that for the control group? that for the groups combined? etc.). But if you know the standard deviation is 10, you're all set.

One caveat, though. Just because it's a T Score doesn't necessarily mean that the standard deviation of scores in your sample will be 10. It could easily be a bit higher or lower. And if so, then you should use the standard deviation you actually have, not the theoretical one. 

One more thought. I'd be surprised if the papers you're consulting failed to report inferential statistics (such as t-tests or correlations). If they do, then you don't need to start from scratch, so to speak. To convert from t to d, I think th