The University of North Carolina at Pembroke
MBA 510--Quantitiative Methods
Notes on Factor Analysis and Principal Components

check this page frequently for updates

Link to MBA 510 Main Page.
Link to Dr. Frederick's page.

Performing Factor Analysis/Principal Components on computers
    JMP has a nice interactive spinning plot, only does VARIMAX rotation
    SPSS.

Glossary
Example
Uses of factor analysis
Fundamental concepts
Assumptions
Types of structures
How many factors?
Rotation

Example
A market researcher has collected spending information from 100 households.  The variables measured are

He want to reduce the number of variables from five to two.  He suspects that some of the variables depend on current income, or other variables depend on expectations of future income.  The correlation coefficients for these five variables were calculated and are shown in the table below.
  Y1 Y2 Y3 Y4 Y5
Automobiles Y1 1.0 0.7 0.2 0.3 0.4
Vacations Y2 0.7 1.0 0.1 0.2 0.4
Fast Food Y3 0.2 0.1 1.0 0.8 0.6
Videos Y4 0.3 0.2 0.8 1.0 0.7
Clothing Y5 0.4 0.4 0.6 0.7 1.0
These correlation coefficients make it appear that there are indeed two underlying variables: spending on automobiles and spending on vacations appear to be correlated with each other and probably represent of the effect of expectations of future income.  Spending on fast food, videos, and clothing up to be correlated also and probably represents the effect of current income.  Factor analysis can be used to estimate these two unobserved underlying factors.

Uses of Factor Analysis

Fundamental Concepts
The observable variables, the Yij, are thought of as being functions of common factors, Fij, and unique factors, Uij.  The common factors and the unique factors are unobservable.  They may be unobservable because they really are difficult to quantify accurately or because they were not measured in the study in question.  These relationships can be expressed as
 


Y1j, = a11F1j + a12F2j  + U1j,
Y2j, = a21F1j + a22F2j  + U2j,
Y3j, = a31F1j + a32F2j  + U3j,
Y4j, = a41F1j + a42F2j  + U4j,
Y5j, = a51F1j + a52F2j  + U5j.

These equations form the factor structure.  The coefficients in the equations, aij, are factor structure coefficients.  (If the Yij had been represented in their standardized forms Zij = (Yij, - µisi then these coefficients would be called the factor loadings.)

Assumptions of Factor Analysis

  1. A factor structure exists.  This implies that factors determine the values of the observed variables, Y.  There are no effects of one observed variable on other observed variables.  This assumption can be tested by Bartlett's test for sphericity.  The null hypothesis is that there is no factor structure, so we want to be able to reject this hypothesis.  (We want to see a small p value.)  Another test is the Measure of Sampling Adequacy, MSA.  0 < MSA < 1 and an MSA over 0.5 is considered good.
  2. Observed values, Y, are linear functions of the factors.
  3. Homogeneity.  The factor structure applies equally well to all observations.
  4. Ockham's razor (or Occam's razor): When there are two equally good competing explanations of events, the simpler explanation is probably the right one.
  5. Normality: the unique factors, U, like the error terms in linear regression, come from a normal distribution.  This assumption is only important in the rare cases when we want to conduct a test of a hypothesis.
  6. Homoskedasticity: The unique factors, U, have the same variance for all obsrevations, though the unique factors for the different variables do not all have to have the same variance.  This assumption also is only important when we conduct hypothesis tests.


Factor Rotation
The purpose of factor rotation is to find a factor loadings pattern that is easier to interpret.  Recall the factor indeterminance problem, which says that the factor structure is not unique.  There is an infinite range of possible factor structures that will fit the observed data equally well.  Factor indeterminance gives us the freedom to impose an additional criterion for the factor structure.  We not only want it to provide a good fit, but we want it to be easy to interpret.  Several additional criteria are available, each having its own factor rotation method.

Orthagonal rotation methods are methods that will preserve the zero correlation between the factors.

VARIMAX: One of the most commonly used methods is the VARIMAX rotation.  This seeks to maximize the separation of the factor loadings in a column of the factor pattern; that is, for each factor, it tries to say that the factor is mostly correlated with a group of observed Y variables and uncorrelated witht the other Y variables.  JMP uses VARIMAX.

QUARTIMAX: This method does the opposite of VARIMAX--it seeks to maximize the separation between the rows of the loadings pattern.  It tries to say that a given observed variable Y is mostly correlated with a small group of factors and uncorrelated with the rest.

EQUIMAX:  This method is a compromise between VARIMAX and QUARTIMAX.

There are several variants on each of these methods.

Oblique rotation methods will allow the factors to be correlated in the process of increasing the separation of the factor loadings.  Two common methods are

OBLIMAX and

PROMAX

Glossary

characteristic root: See eigenvalue

common factor

communality:

eigenvalue: (Also called characteristic root or latent root)  A measure of the strength of a correlation among a group of variables.  The number of eigenvalues for a dataset equals the number of variables in the data set though there is no way to relate an eigenvalue to a single variable.  The sum of all eigenvalues must equal the number of variables in the dataset.

factor

factor loading

factor structure

latent root: See eigenvalue

rotation

scree plot:  A graph of the eigenvalues (on the vertical axis) against the rank of the eigenvalue (on the horizontal axis).  Used for determining the number of factors to use.  (The term is sometimes erroneously capitalized.  The graph takes its name not from a person, but from geology.  Scree is the rocky debris at the base of a cliff.)

Type P, S, or T factor analsys: Don't worry about them.

Type R factor analysis: A factor analysis which seeks to find similar variables.  The most common use of factor analysis.

Type Q factor analysis: A factor analysis which seeks to find similar observations.  Similar to cluster analysis.

unique factor:

last updated April 27, 2001, by Jim Frederick
copyright 2001 James R. Frederick