Page 421 - Applied Statistics with R

17.2. BINARY RESPONSE 421

So even though we introduced ordinary linear regression first, in some ways,
logistic regression is actually simpler.

Note that applying the inverse logit transformation allow us to obtain an ex-
pression for (x).

0 + 1 1 +⋯+ −1 ( −1)
(x) = [ = 1 ∣ X = x] =
1 + 0 + 1 1 +⋯+ −1 ( −1)

17.2.1 Fitting Logistic Regression

With observations, we write the model indexed with to note that it is being
applied to each observation.

(x )
i

log ( ) = + + ⋯ + −1 ( −1)
0
1 1
1 − (x ))
i
We can apply the inverse logit transformation to obtain [ = 1 ∣ X = x ] for
i
i

each observation. Since these are probabilities, it’s good that we used a function
that returns values between 0 and 1.
0 + 1 1 +⋯+ −1 ( −1)
(x ) = [ = 1 ∣ X = x ] =
i
i

i
1 + 0 + 1 1 +⋯+ −1 ( −1)
1
1 − (x ) = [ = 0 ∣ X = x ] =
i

i
1 + 0 + 1 1 +⋯+ −1 ( −1)
To “fit” this model, that is estimate the parameters, we will use maximum
likelihood.
= [ , , , , … , −1 ]
2
0
3
1
We first write the likelihood given the observed data.

( ) = ∏ [ = ∣ X = x ]

i
i
=1
This is already technically a function of the parameters, but we’ll do some
rearrangement to make this more explicit.

( ) = ∏ (x ) (1 − (x )) (1− )
i
i
=1