Page 181 - Jolliffe I. Principal Component Analysis

7

Principal Component Analysis and
Factor Analysis

Principal component analysis has often been dealt with in textbooks as a
special case of factor analysis, and this practice is continued by some widely
used computer packages, which treat PCA as one option in a program for
factor analysis. This view is misguided since PCA and factor analysis, as
usually defined, are really quite distinct techniques. The confusion may
have arisen, in part, because of Hotelling’s (1933) original paper, in which
principal components were introduced in the context of providing a small
number of ‘more fundamental’ variables that determine the values of the
p original variables. This is very much in the spirit of the factor model
introduced in Section 7.1, although Girschick (1936) indicates that there
were soon criticisms of Hotelling’s PCs as being inappropriate for factor
analysis. Further confusion results from the fact that practitioners of ‘fac-
tor analysis’ do not always have the same definition of the technique (see
Jackson, 1991, Section 17.1). In particular some authors, for example Rey-
ment and J¨oreskog (1993), Benz´ecri (1992, Section 4.3) use the term to
embrace a wide spectrum of multivariate methods. The definition adopted
in this chapter is, however, fairly standard.
Both PCA and factor analysis aim to reduce the dimensionality of a
set of data, but the approaches taken to do so are different for the two
techniques. Principal component analysis has been extensively used as part
of factor analysis, but this involves ‘bending the rules’ that govern factor
analysis and there is much confusion in the literature over the similarities
and differences between the techniques. This chapter attempts to clarify
the issues involved, and starts in Section 7.1 with a definition of the basic
model for factor analysis. Section 7.2 then discusses how a factor model