Page 423 - Jolliffe I. Principal Component Analysis

14. Generalizations and Adaptations of Principal Component Analysis
388
Metric-based PCA, as defined by Thacker (1996), corresponds to the
−1 1
triple (X, E
, I n ), and E plays the same rˆole as does Γ in the fixed
n
effects model. Tipping and Bishop’s (1999a) model (Section 3.9) can be
2
fitted as a special case with E = σ I p . In this case the a k are simply
eigenvectors of S.
Consider a model for x in which x is the sum of a signal and an indepen-
dent noise term, so that the overall covariance matrix can be decomposed
as S = S S + S N , where S S , S N are constructed from signal and noise,
respectively. If S N = E, then S S = S − E and
S S a k = Sa k − Ea k = l k Ea k − Ea k =(l k − 1)Ea k ,
so the a k are also eigenvectors of the signal covariance matrix, using the
metric defined by E −1 . Hannachi (2000) demonstrates equivalences between
• Thacker’s technique;
• a method that finds a linear function of x that minimizes the prob-
ability density of noise for a fixed value of the probability density of
the data, assuming both densities are multivariate normal;
• maximization of signal to noise ratio as defined by Allen and Smith
(1997).
Diamantaras and Kung (1996, Section 7.2) discuss maximization of signal
to noise ratio in a neural network context using what they call ‘oriented
PCA.’ Their optimization problem is again equivalent to that of Thacker
(1996). The fingerprint techniques in Section 12.4.3 also analyse signal to
noise ratios, but in that case the signal is defined as a squared expectation,
rather than in terms of a signal covariance matrix.
Because any linear transformation of x affects both the numerator and
denominator of the ratio in the same way, Thacker’s (1996) technique shares
with canonical variate analysis and CCA an invariance to the units of
measurement. In particular, unlike PCA, the results from covariance and
correlation matrices are equivalent.

14.2.3 Transformations and Centering
Data may be transformed in a variety of ways before PCA is carried out,
and we have seen a number of instances of this elsewhere in the book.
Transformations are often used as a way of producing non-linearity (Sec-
tion 14.1) and are a frequent preprocessing step in the analysis of special
types of data. For example, discrete data may be ranked (Section 13.1)
and size/shape data, compositional data and species abundance data (Sec-
tions 13.2, 13.3, 13.8) may each be log-transformed before a PCA is done.
The log transformation is particularly common and its properties, with and
without standardization, are illustrated by Baxter (1995) using a number
of examples from archaeology.