Page 386 - Applied Statistics with R

386 CHAPTER 16. VARIABLE SELECTION AND MODEL BUILDING

⋅ .

Then, for AIC = 2, and for BIC = log( ).
For comparing models

RSS
BIC = log ( ) + log( )

is again a sufficient expression, as + log(2 ) is the same across all models
for any particular dataset.

16.1.3 Adjusted R-Squared

Recall,

2

SSE ∑ ( − ̂ )

2
= 1 − = 1 − =1 .

SST ∑ ( − ̄) 2

=1
We now define
SSE/( − ) − 1
2
2
= 1 − SST/( − 1) = 1 − ( − ) (1 − )

2
which we call the Adjusted .
2
Unlike which can never become smaller with added predictors, Adjusted
2
effectively penalizes for additional predictors, and can decrease with added
2
predictors. Like , larger is still better.
16.1.4 Cross-Validated RMSE
Each of the previous three metrics explicitly used , the number of parameters,
in their calculations. Thus, they all explicitly limit the size of models chosen
when used to compare models.
We’ll now briefly introduce overfitting and cross-validation.

make_poly_data = function(sample_size = 11) {
x = seq(0, 10)
y = 3 + x + 4 * x ^ 2 + rnorm(n = sample_size, mean = 0, sd = 20)
data.frame(x, y)
}