In this section, we’ll look at a few popular linear classification models.
Suppose that you were trying to estimate the probability of a certain outcome (which we’ll call A) for a categorical variable with two values. You could try to predict the probability of A as a linear function of the predictor variables, assuming y = c0 + c1x1 + x2x2 + ... + cnxn= Pr(A). The problem with this approach is that the value of y is unconstrained; probabilities are valid only for values between 0 and 1. A good approach for dealing with this problem is to pick a function for y that varies between 0 and 1 for all possible predictor values. If we were to use that function as a link function in a general linear model, then we could build a model that estimates the probability of different outcomes. That is the idea behind logistic regression.
In a logistic regression, the relationship between the predictor variables and the probability that an observation is a member of a given class is given by the logistic function:
The logit function (which is used as the link function) is:
Let’s take a look at a specific example of logistic regression. In particular, let’s look at the field goal data set. Each time a kicker attempts a field goal, there is a chance that the ...