All Answers (26)
 Dear PheiChin Lim!
I would like you to go through this page:
http://www.hawaii.edu/powerkills/UFA.HTM for this description:
......"Scaling. A scientist often wishes to develop a scale on which individuals, groups, or nations can be rated and compared. The scale may refer to such phenomena as political participation, voting behavior, or conflict. A problem in developing a scale is to weight the characteristics being combined. Factor analysis offers a solution by dividing the characteristics into independent sources of variation (factors). Each factor then represents a scale based on the empirical relationships among the characteristics. As additional findings, the factor analysis will give the weights to employ for each characteristic when combining them into the scales. The factor score results are actually such scales, developed by summing characteristics times these weights. "
Hope this helps...  In Exploratory Factor Analysis, you'd first decide the number of factors to be extracted  how many factors does it take to explain most of the variation:
http://en.wikipedia.org/wiki/Factor_analysis#Criteria_for_determining_the_number_of_factors
Then chose the factor loading cutoff level. What you can then do (to help visually), is to copy your factor scores over to excel and paste the scores for each factor in a distinct column(if you chose 3 factors, 3 columns). The ones that you consider are significant scores above the cutoff level. So negative scores are out. With this information, you consider your research question, and see what makes the most sense. Now consider the remaining scores on each factor. The closer to 1 they are, the more important they are in explaining the variation in that factor. There may potentially be relationships among those elements in each factor, which the research question, or background information may help you interpret. Do the same for interpreting the other factors.
"More than one interpretation can be made of the same data factored the same way, and factor analysis cannot identify causality."  Since each factor is a combination of all the variables you are working with (eigen vectors are the weighed for each variable), each factor can be positive or negative. It is positive when the variables with greater weight take high values and it is negative when they take low values. Please remember that the zero value is the mean of all variables.
 Factors are standardized to a mean of 0 and variance of 1, as they are latent variables composed of pieces of varying scales  think of it like the only way to fit everything together is within a standardized entity.
Assuming that you used a principal components extraction, and incorporated all variables in each retained factor, then each factor contains a portion of the total variance that is reproduceable by a linear combination of the items factored.
You MAY be able to identify (stick a name on) each of the factors you retain by looking at the sign of the weights that make up the loadings; doesn't always work out, though.
Using a rotation may help clarify what each factor represents...  Are you trying to interpret the factor loadings (the weight of each factor on the observed variables) or the factor scores (the weights of each observed variable in producing a score representing the factor)? These are two different things. The first are used to interpret what the factor is by judging the relative sizes of the loadings: high loadings suggest stronger factor contributions to those variables. Loadings of both positive and negative signs are possible, but having both positive and negative loadings that are large is rare and suggests problematic factors.
Factor scores on the other hand are composites of the variables that are used to make the latent factor into an observed variable; to give it a scale. You would use factor scores, for example, when the factor was of interest to use as a predictor or outcome in a regression analysis and you were not planning to use latent variable modeling methods.
Many different factor score computation methods result in factor scores with mean = 0 and SD = 1. Various statistical packages have an assortment of these methods for computing factor scores, but they all produce very highly correlated alternatives. Even a unit weighted factor score method (for all variables with loadings > .3 weight = 1, else weight = 0) produces factor scores very highly correlated with other methods. There's a good paper by Grice on factor scores (see his helpful website, pdf of the paper is there too: http://psychology.okstate.edu/faculty/jgrice/factorscores/).  You must start from Factor Loadings: they can be interpreted as the correlation coefficients between the original variables and components, given you know the meaning of the variables you have used , you will look at the variables highly loaded (as absolute value) on the components (in this case the loading can only vary betweem 1 and 1 so the interpretation is easy) and give a name to the components depending on which variables are more loaded on the components, like here:
http://pubs.acs.org/doi/abs/10.1021/ci2005127
Clearly you MUST KNOW THE SENSE OF WHAT YOU HAVE MEASURED, BUT I GIVE FOR GRANTED A SCIENTIST KNOWS WHAT HE/SHE IS DOING AND THIS IS AN ESEGESIS WORK CRUCIAL FOR YOUR WORK.
Having given a name to your components you turn to the scores and, depending on the system at hand you will see that component1 is significantly related to an external variable Y (let's say a drug treatment, or a continuous variable as age, or location or any other thing you are interested into..by the way if you performed a study you wanted to prove some hypothesis on the world outside). Component scores are like any other variable and you ca correlate them with any feature of interest of your statistical units.
In some other cases it is more convenient to start from factor scored, like in microarray analysis, where the genes are statistical units, so for example you find that loadings are able to separate cancer from healthy subject (interperetation y loadings), for example the loadings on factor2, well, now you turn to the scores and order them for their absolute value, and look at which genes have higher scores on the discriminating component, well you are a biologist, or you work with a biologist, he will look at these 'extreme relevant genes' and will say something about biology, like here:
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0013668
http://www.febsletters.org/article/S00145793(01)029738/abstract
or even here:
http://www.biomedcentral.com/14712105/7/194/abstract/  Each item that belongs to a factor, must have a factor score greater than .400 in factor in where they belong. The remaining factor scores should not be significant (p <.01) if they are considered as Pearson correlations, with (n2) degrees of freedom. If these conditions are met, each factor score is interpreted as the contribution of the item to the corresponding factor. Thus, for each factor, the conceptual content of the items with the highest factor scores, form the conceptual content of the factor, ie its "constitutive definition". The factor scores greater than .400 may not be negative, in which case you missed decoding scores of that item to change its direction. If a factor score is not significant, no matter who is negative.
 Thank you Mohammad Tahir, Janavi Kumar and Alessandro Giuliani for the link sent. As I am do not have statistic background, it is quite hard for me to understand the information in 1st link while 2nd link consists of steps in conducting factor analysis.
Thank you Bernardo Chaves, Michael Weaver, Scot McNary and Reynaldo RochaChávez for your suggestion.
Checking my understanding on factor analysis, there are two type of outputs  factor loadings and factor scores. Since we are approaching our research question through an exploratory perspective or trying to build a model, different combinations and methods with different number of factor cutoff level are used before settle on a 'suitable' configuration. Then, rotated factor loadings are used to interpret the factors obtained before giving it a name/label. As read somewhere, factor scores can be treated as any other variable for further investigation. Hence, we are using factor scores to identify scales for the named/labeled factor (e.g. to obtain something like the scale use in questionnaire; strongly agree, agree, neutral, disagree, strongly disagree). Am I on the right track?  Dear PheiChin I suggest you not to rotate factor loadings, by the way is perfectly natural that the same variable is 'loaded' on different components, this simply means that the particular measure (variable) is influenced by more than one 'hidden variable'.
On the contrary if you rotate you completely loose the physical meaning of components aand this is very detrimental to your analysis.  Dear PheiChin. On the contrary, as I understand, the rotated solution is the only one that can be interpreted in terms of the conceptual content of the measure. My comment above refers to the rotated solution. As part of the outputs of an exploratory factor analysis, can obtain Z scores of each factor in the rotated solution. Those scores are measured on the corresponding factor, responsive to the conceptual content determined by the set of associated items factor. But, I do not think you can say that obtain something like the scale use in questionnaire; strongly agree, agree, neutral, disagree, strongly disagree.
 Dear Reynaldo,
this is a very interesting point, those who work in psychology or sociology desire factor analysis confirm their previous statements (this is the reason for rotation) 'conceptual content' is given , statistics is useful to obtain scales usable reflecting conceptual content.
Who works in natural science (physics, chemistry) does exactly the opposite, the components (by the way this name come from analytical chemistry indicating the components of a mixture) give rise to the correlation structure of the variables (e.g. peaks of a spectrum) thus the same varible can be influenced by different components like in the composition of forces in physics, or different molecules sharing the same peak of adsorbance (among many other they do not share).
I think this should be the right attitude in psychology too (see tha attached paper) because if we are not sure of the stability across different data sets of our constructs in physics...can you imagine in psychology ????
But again this is a complex and intermingled question....  The scores in EFA are Pearson coefficient correlations between observed variables and "invisible" variables named "factors". For example "Intelligence" is measured by many tests. One factor can be "memory", second "calculation", third "association", fourth "3D imagination" and others. After VARIMAX rotation we obtain the simple stucture: high score values with one observed (input) variable and week with all another factors.
 Weak
 Dear Alessandro Giuliani:
I think that this difference between social sciences and natural sciences comes of the difference between of type of objects of study. In analytics chemistry there is a implicit supposed, the nature of study's objects is not dependent of the phisycal place where ones are. But, in social science, the stability of the one factorial structure depents of a one specific culture. So, the technique in social sciences is to repeat one study until to discover of the cultural limits of those factorial structure.
For other hand, there is an clarifier example. Take in eche row diferents mesures of one specific cylinder. Each row, maybe can be selectec by random proces. In the columns one can put: height; area of the base, lateral area, base radius, and other. The solución without rotation, could not be interpreted for other hand, the rotate solution could be easy interpreted in terms of the diferences of the mesuares' nature: length, area; mesuares of the base, height of the cylinder.  If you have many variables that are being used in the exploratory factor analysis, I would agree that the rotated solution is the one you should be looking at because they are no longer correlated. The convention is that you are looking for scores (positive or negative) >=0.7. However, at first in your 'exploration' you might find some variables that have a low score in all your rotated dimensions. My advice then would be to iterate the factor analysis leaving out these variables (as they very little) until each of your remaining variables has a high score in only one dimesion. This then becomes much easier to interpret what the dimensions mean.
 Dear Allan Brimicombe,
I agree with your view point.  Thank you Reynaldo.
I always understood that the positive and negative signs in the rotated solution reflected the positive and negative correlations in the correlation matrix.....I've tried to find a reference to this but couldn't run one to ground....so I could be wrong.  Dear Reynaldo,
you say 'For other hand, there is an clarifier example. Take in eche row diferents mesures of one specific cylinder. Each row, maybe can be selectec by random proces. In the columns one can put: height; area of the base, lateral area, base radius, and other. The solución without rotation, could not be interpreted for other hand, the rotate solution could be easy interpreted in terms of the diferences of the mesuares' nature: length, area; mesuares of the base, height of the cylinder.'
The point is exactly what you are saying, in natural sciences we are interested in the cylinder orientation and specific shape , given the nature of the measures is alredy known and thus not interesting at all, in other words, the measures are simply probes for the reality and not the other way around, as a matter of fact any scientific paper starts with the description of the measures (already known) and ends up into the specific structure of correlation they aquire when interact wit a 'piece of matter' (whatever this piece of matter is, a protein, a set of rats, a population..) given the aim is to increase knowledge about the piece of matter (i.e. the real thing), like here, where a set of well known and clearly defined measure are made to interact with a set of proteins so to make their correlation structure to highlight the specific structural principles of the proteins (not the nature of the measures we know already), in this case is mandatory to use a physically motivate solution like unrotated PCA.
Thank you for this exchange of ideas.
Alessandro  This is the publication i referred to
 As noted above, factor analysis identifies "invisible" factors which represent the hidden organization or "organizing principle" of whatever is being measured with a number of observable measures or scales. Factor scores or "factor loadings" indicate how each "hidden" factor is associated with the "observable" variables used in the analysis. Say you have five observable variables that identify two hidden factors. Here are some example factor loadings on hidden Factor 1 across the five variables: .80, .33, .75, .89, .46. The first and third factor loadings (.80, .75) indicate that observable measures 1 and 3 can be used to "describe" hidden Factor 1; in other words, Factor 1 has characteristics very similar to what observable measures 1 and 3 measure. Observable variables 2 and 5 (.33, .46) are not useful to describe hidden Factor 1 because their factor loadings on hidden Factor 1 are too small (not > or = to .70). Finally, the negative factor loading of observable measure 4 with hidden Factor 1 (.89) means that hidden Factor 1 has the characteristic "opposite" of whatever obervable measure 4 measures.
If the observable measures captured the dimensions "fun", "hard", "happy", "tough", and "sad", then hidden Factor 1 can be described as "fun" (loading = .80), "happy" (loading = .75), and "NOT Sad" (loading = .89).
I hope this helps!  I am going to ramble a little bit about this.
Factor Analysis and Complex Systems
Factor analysis is also a technique to identify emergent multistability within the phase space of a system of responses to a set of measures. The Five Factor Model of personality is an inductive model (Friedman & Schustack, 2012) which identified the emergent multistable factors (e.g., Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) from repeated factor analyses of behavioralaffective measures (Digman, 1990).
From a complex dynamic systems perspective, the Five Factors can be thought of as “global structural patterns, which emerge from (both linear and nonlinear) interactions among the system’s components through phase space…characterized as emergent (attractorbased) collectives.” (Juarrero, 2012, Abstract, parenthetical phrases added). A system’s phase space represents all possible states of the system, and the attractors within the phase space represent regions of multistability and different trajectories of the system.
Example statements used by HolgadoTello, CarrascoOrtiz, del Barrio Gándara, and Moscoso (2009) to detect the Five Factors included statements such as “I share my things with other people,” “I like to watch the TV news and to know what happens in the world” (p .78). Based on the 5point scale HogadoTello, et al. (2009) used to assess the 65 statements, the size of the phase space for the entire system of items and response options can be calculated at 564 or 2.7 X 1045.
Assuming the responses to these statements are based on behavioral tendencies in specific contexts (e.g., occasions where “sharing with others” can occur or when the news is on TV, the responses can be said to be contextuallybound and linked to specific environmentalsocial contexts.
So the Five Factors can be said to “emerge” from the response behavior of large numbers of individuals to dynamically changing environmentalsocial situations consistent with complex adaptive systems (Miller & Page, 2007).
References
Digman, J. M. (1990, February). Personality structure: emergence of the fivefactor model. Annual Review of Psychology, 41, 417440. doi: 10.1146/annurev.ps.41.020190.002221.
Friedman, H. S. & Schustack, M. W. (2012). Personality: Classic theories and modern research (5th Ed.). MA: Allyn & Bacon ISBN: 0205050174
HolgadoTello, F., CarrascoOrtiz, M., del Barrio Gándara, M., & Moscoso, S. (2009). Factor analysis of the Big Five Questionnaire using polychoric correlations in children. Quality & Quantity: International Journal of Methodology, 43(1), 7585. doi:10.1007/s1113500790853.
Juarrero, A. (2012). Complex dynamical systems theory. The Cognitive Edge. Retrieved from http://www.cognitiveedge.com/uploads/articles/100608%20Complex_Dynamical_Systems_Theory.pdf
Miller, J. H. & Page, S. E. (2007). Complex adaptive systems. New Jersey: Princeton University Press.  are the factor scores the same thing as factor loadings (weights)? factor weights (loadings) are correlations between variables and (invisible) factors; factor scores are linear combinations between factor weights and variables in a specific location for the purpose of regional typization (cluster analysis).

Factor scores are useful to rank and prioritise the factors. the, you can suggest the factors which are most important and need more focus. Recommendation should also be based on these rankings.

Just on a practical basis I have found suppressing variable loadings on each factor <.4 useful. This cleans up the overall factor picture and make it easier for the reader to understand when presented with the factor table. I do not think there is any hard & fast rule but would be interested to learn.

You really want to be working with rotated factor scores to interpret them. Varimax rotation is most commonly used. This ensures that the correlation between factors (also referred to as components or dimensions) is minimised. {Don't forget, use the scree plot to limit the number of dimensions you are working with to what is absolutely necessary.} Then a rule of thumb is that any variable scoring 0.7 or more in a dimension is a key one for interpreting what that dimension means. In fact in exploratory factor analysis you would progressively weed out variables which do not have a rotated score of 0.7 or more in any dimension. The end result is often that each variable only contributes strongly (0.7 or more) to one dimension and the interpretation of each dimension then becomes much easier.

Once you get to know that some components describe most of the factor in which they are, what are we supposed to do next? Do we remove the describing components and run the analysis again? And then, when we see that sufficient factor analysis is being carried out, what is the next step?
 Views 37931
 Followers 36
 Answers 26
Popular Answers
If the observable measures captured the dimensions "fun", "hard", "happy", "tough", and "sad", then hidden Factor 1 can be described as "fun" (loading = .80), "happy" (loading = .75), and "NOT Sad" (loading = .89).
I hope this helps!
Factor Analysis and Complex Systems
Factor analysis is also a technique to identify emergent multistability within the phase space of a system of responses to a set of measures. The Five Factor Model of personality is an inductive model (Friedman & Schustack, 2012) which identified the emergent multistable factors (e.g., Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) from repeated factor analyses of behavioralaffective measures (Digman, 1990).
From a complex dynamic systems perspective, the Five Factors can be thought of as “global structural patterns, which emerge from (both linear and nonlinear) interactions among the system’s components through phase space…characterized as emergent (attractorbased) collectives.” (Juarrero, 2012, Abstract, parenthetical phrases added). A system’s phase space represents all possible states of the system, and the attractors within the phase space represent regions of multistability and different trajectories of the system.
Example statements used by HolgadoTello, CarrascoOrtiz, del Barrio Gándara, and Moscoso (2009) to detect the Five Factors included statements such as “I share my things with other people,” “I like to watch the TV news and to know what happens in the world” (p .78). Based on the 5point scale HogadoTello, et al. (2009) used to assess the 65 statements, the size of the phase space for the entire system of items and response options can be calculated at 564 or 2.7 X 1045.
Assuming the responses to these statements are based on behavioral tendencies in specific contexts (e.g., occasions where “sharing with others” can occur or when the news is on TV, the responses can be said to be contextuallybound and linked to specific environmentalsocial contexts.
So the Five Factors can be said to “emerge” from the response behavior of large numbers of individuals to dynamically changing environmentalsocial situations consistent with complex adaptive systems (Miller & Page, 2007).
References
Digman, J. M. (1990, February). Personality structure: emergence of the fivefactor model. Annual Review of Psychology, 41, 417440. doi: 10.1146/annurev.ps.41.020190.002221.
Friedman, H. S. & Schustack, M. W. (2012). Personality: Classic theories and modern research (5th Ed.). MA: Allyn & Bacon ISBN: 0205050174
HolgadoTello, F., CarrascoOrtiz, M., del Barrio Gándara, M., & Moscoso, S. (2009). Factor analysis of the Big Five Questionnaire using polychoric correlations in children. Quality & Quantity: International Journal of Methodology, 43(1), 7585. doi:10.1007/s1113500790853.
Juarrero, A. (2012). Complex dynamical systems theory. The Cognitive Edge. Retrieved from http://www.cognitiveedge.com/uploads/articles/100608%20Complex_Dynamical_Systems_Theory.pdf
Miller, J. H. & Page, S. E. (2007). Complex adaptive systems. New Jersey: Princeton University Press.