Page 158 - Applied Statistics with R

158 CHAPTER 9. MULTIPLE LINEAR REGRESSION

If we were to stack together the linear equations that represent each into

a column vector, we get the following.

1 1 11 12 ⋯ 1( −1) ⎡ 0 ⎤ 1
⎡ ⎤ ⎡ 1 ⋯ ⎤ ⎢ 1 ⎥ ⎡ ⎤

⎢ 2 ⎥ = ⎢ 21 22 2( −1) ⎥ ⎢ 2 ⎥ + ⎢ ⎥
2
⎢ ⋮ ⎥ ⎢⋮ ⋮ ⋮ ⋮ ⎥ ⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎣ ⎦ ⎣ 1 1 2 ⋯ ( −1)⎦ ⎣ −1⎦ ⎣ ⎦

= +

1 1 11 12 ⋯ 1( −1) ⎡ 0 ⎤ 1
⎡ ⎤ ⎡ 1 ⋯ ⎤ ⎢ 1 ⎥ ⎡ ⎤

2
= ⎢ 2 ⎥ , = ⎢ 21 22 2( −1) ⎥ , = , = ⎢ ⎥
⎢ ⋮ ⎥ ⎢⋮ ⋮ ⋮ ⋮ ⎥ ⎢ 2 ⎥ ⎢ ⋮ ⎥
⎢ ⋮ ⎥
⎣ ⎦ ⎣ 1 1 2 ⋯ ( −1)⎦ ⎣ −1⎦ ⎣ ⎦

So now with data,
1
⎡ ⎤
= ⎢ 2 ⎥
⎢ ⋮ ⎥
⎣ ⎦

Just as before, we can estimate by minimizing,

2

( , , , ⋯ , −1 ) = ∑( − ( + + + ⋯ + −1 ( −1) )) ,
1
2
1 1
0
2 2

0
=1
which would require taking derivatives, which result in following normal
equations.

∑ 1 ∑ 2 ⋯ ∑ ( −1) ∑
⎡ =1 2 =1 =1 ⎤ ⎡ 0 ⎤ ⎡ =1 ⎤

⎢ ∑ =1 1 ∑ =1 1 ∑ =1 ⋯ ∑ =1 ⎥ ⎢ 1 ⎥ = ⎢ ∑ =1 ⎥
1 ( −1)
1 2
1
⎢ ⋮ ⋮ ⋮ ⋮ ⎥ ⎢ ⋮ ⎥ ⎢ ⋮ ⎥

⎣ ∑ =1 ( −1) ∑ =1 ( −1) 1 ∑ =1 ( −1) 2 ⋯ ∑ =1 2 ( −1) ⎦ ⎣ −1⎦ ⎣ ∑ =1 ( −1) ⎦
The normal equations can be written much more succinctly in matrix notation,
⊤
⊤
= .