Page 103 - Applied Statistics with R

7.2. LEAST SQUARES APPROACH 103

7.2.3 Variance Estimation

We’ll now use the residuals for each of the points to create an estimate for the
2
variance, .
Recall that,

E[ ∣ = ] = + .

0

1
So,
̂
̂
̂ = +
0 1
is a natural estimate for the mean of for a given value of .

Also, recall that when we specified the model, we had three unknown parame-
2
ters; , , and . The method of least squares gave us estimates for and
1
0
0
2
2
, however, we have yet to see an estimate for . We will now define which
1

2
will be an estimate for .
1
̂
̂
2
= ∑( − ( + )) 2

1
0
− 2
=1
1
= ∑( − ̂ ) 2
− 2
=1
1
= ∑ 2
− 2
=1
This probably seems like a natural estimate, aside from the use of − 2, which
we will put off explaining until the next chapter. It should actually look rather
similar to something we have seen before.
1
2
= ∑( − ̄ ) 2
− 1
=1
2
2
Here, is the estimate of when we have a single random variable . In this

case ̄ is an estimate of which is assumed to be the same for each .
2
Now, in the regression case, with each has a different mean because of the

relationship with . Thus, for each , we use a different estimate of the mean,

that is ̂ .