In this section, we’ll look at a few popular linear classification models.

Suppose that you were trying to estimate the probability
of a certain outcome (which we’ll call *A*) for a
categorical variable with two values. You could try to predict the
probability of *A* as a linear function of the
predictor variables, assuming *y* =
*c*_{0} +
*c*_{1}*x*_{1}
+
*x*_{2}*x*_{2}
+ ... +
*c*_{n}*x*_{n}=
Pr(*A*). The problem with this approach is that the
value of *y* is unconstrained; probabilities are
valid only for values between 0 and 1. A good approach for dealing with
this problem is to pick a function for *y* that
varies between 0 and 1 for all possible predictor values. If we were to
use that function as a link function in a general linear model, then we
could build a model that estimates the probability of different
outcomes. That is the idea behind logistic regression.

In a logistic regression, the relationship between the predictor variables and the probability that an observation is a member of a given class is given by the logistic function:

The logit function (which is used as the link function) is:

Let’s take a look at a specific example of logistic regression. In particular, let’s look at the field goal data set. Each time a kicker attempts a field goal, there is a chance that the ...

Start Free Trial

No credit card required