Logistic Regression: A Mathematical Map for Classification

Up until now, we have learned how to predict numbers like scores or prices. However, in reality, we often need Binary Classification, such as “Will this loan applicant go bankrupt?” or “Is this email spam?” In such cases, we use Logistic Regression.

1. Limitations of Linear Regression and the Sigmoid

Linear regression can produce outcome values that are infinitely large or small. However, in classification problems, ‘probabilities’ must be limited between 0 and 1. Logistic regression passes the result of a linear combination through an S-shaped Sigmoid Function to convert it into a probability value between 0 and 1.

2. Odds Ratio: Success vs. Failure

The key keyword to understanding logistic regression is ‘Odds.’

Odds = Probability of Success / Probability of Failure

For example, if the success probability is 0.8, the odds are $0.8 / 0.2 = 4$. The logistic regression model predicts the value of the log of these odds.

3. Model Evaluation: Confusion Matrix

The performance of a classification model isn’t simply measured by $R^2$; instead, it’s evaluated through a table that summarizes correct and incorrect predictions.

💡 Professor’s Tip

Logistic regression is the first gateway to ‘Deep Learning’ in machine learning. This is because the process of a Perceptron—the basic unit of artificial neural networks—sending an output through a non-linear activation function is intrinsically linked to the principles of logistic regression.

🔗 Next Step

• Ch8: Time Series Analysis Basics