Page 403 - Applied Statistics with R

16.2. SELECTION PROCEDURES 403

2
Adjusted and LOOCV RMSE comparisons are similar to backwards and
forwards, which is not at all surprising, as some of the models selected are the
same as before.

summary(hipcenter_mod)$adj.r.squared

## [1] 0.6000855

summary(hipcenter_mod_both_aic)$adj.r.squared

## [1] 0.6533055

summary(hipcenter_mod_both_bic)$adj.r.squared

## [1] 0.6282374

calc_loocv_rmse(hipcenter_mod)

## [1] 44.44564

calc_loocv_rmse(hipcenter_mod_both_aic)

## [1] 37.62516

calc_loocv_rmse(hipcenter_mod_both_bic)

## [1] 37.2511

16.2.4 Exhaustive Search

Backward, forward, and stepwise search are all useful, but do have an obvious
issue. By not checking every possible model, sometimes they will miss the best
possible model. With an extremely large number of predictors, sometimes this is
necessary since checking every possible model would be rather time consuming,
even with current computers.
However, with a reasonably sized dataset, it isn’t too difficult to check all pos-
sible models. To do so, we will use the regsubsets() function in the R package
leaps.