Stanford CS230 | Autumn 2025 | Lecture 5: Deep Reinforcement Learning

This lesson teaches the core ideas and practical steps of logistic regression, a foundational method for binary classification. Binary classification means you have input features (x) and want to predict whether the output label (y) belongs to class 0 or class 1. Logistic regression works by taking a linear combination of inputs (w^T x + b) and then passing that value through the sigmoid function, which squashes any real number into a probability between 0 and 1. This probability is interpreted as the model’s belief that the input belongs to class 1. Despite using the non-linear sigmoid, logistic regression is still a linear model because its decision boundary—the place where the model switches between predicting class 0 and class 1—is a straight line (or a flat plane in higher dimensions).

A major focus is on training the model correctly. Instead of mean squared error (MSE), which is common in linear regression, logistic regression uses a different cost function called log loss (also known as cross-entropy). This choice matters a lot: using MSE with a sigmoid leads to a non-convex optimization surface, where gradient descent can get stuck in bad local minima. In contrast, log loss for logistic regression is convex, so gradient descent has a clear path to the global minimum. The lesson breaks down how log loss behaves: it gives low cost for confident correct predictions and very high cost for confident wrong ones, which is exactly what you want when learning probabilities.

You’ll also see how to train the model step by step. First, you initialize weights and bias (often with small random values). Then, for each training example, you compute the linear score z = w^T x + b, apply the sigmoid to get the probability y_hat, and compute the log loss comparing y_hat to the true label y. Next, you compute gradients—the direction to change w and b to reduce the loss—and update the parameters. You repeat this process until the loss stops improving significantly, which is called convergence. This can be done with full-batch gradient descent, mini-batch gradient descent, or stochastic gradient descent (one example at a time), depending on data size and speed needs.

The lesson clearly lays out the strengths and weaknesses of logistic regression. On the plus side, it’s simple, easy to implement, fast to train and predict, and provides probability outputs, which are useful for ranking, thresholding, and decision-making under uncertainty. It is also interpretable: each feature’s weight shows how that feature nudges the odds of class 1 up or down. On the minus side, it’s a linear classifier, so it cannot easily model curved or complex relationships in the data unless you add new features that capture those patterns. If your data is not linearly separable, you may need feature engineering such as polynomial terms or interactions.

Finally, the lesson touches on extending logistic regression beyond two classes. Although the basic model handles two classes, you can adapt it to multi-class problems using strategies like one-vs-all (also called one-vs-rest) or one-vs-one. With one-vs-all, you train one classifier per class against all the others and pick the class with the highest predicted probability at prediction time. The key idea remains: use the sigmoid to produce probabilities, use cross-entropy to train, and use gradient descent to find good weights. By the end, you will understand when logistic regression is a good choice, how to implement it, why cross-entropy is the right loss, and how to interpret its outputs and limitations.

This material is most suitable for beginners who know basic algebra and have seen linear regression before. You don’t need advanced math, but it helps to be comfortable with vectors, simple functions, and the idea of minimizing a loss. After working through this, you will be able to build a binary classifier end-to-end: prepare your data, train a logistic regression model with cross-entropy loss, make predictions, choose thresholds, and understand what the model is doing under the hood. You can use these skills directly in real projects like spam detection, medical test prediction, and click-through rate estimation, or as a springboard to more advanced models later.

1. Overall Architecture/Structure

• Objective: Predict P(y=1 | x) for a binary classification task. Inputs are feature vectors x ∈ R^d. The model learns parameters w ∈ R^d and b ∈ R.
• Forward computation: Compute a linear score z = w^T x + b. This is a weighted sum of features plus a bias term. Then pass z through the sigmoid function σ(z) = 1 / (1 + e^(−z)). The output y_hat = σ(z) is the predicted probability of the positive class (class 1).
• Decision rule: Choose a threshold τ (often 0.5). Predict class 1 if y_hat ≥ τ; else predict class 0. The set of points with y_hat = 0.5 is exactly where z = 0, a linear decision boundary (a hyperplane). Thus, despite the sigmoid non-linearity, logistic regression is a linear classifier in feature space.
• Training objective: Minimize the average log loss (binary cross-entropy) over the dataset. For a single example (x, y), L = −[y log(y_hat) + (1−y) log(1−y_hat)]. Over N examples, minimize J(w, b) = (1/N) ∑ L_i.

Data Flow

• Input: A batch of features X ∈ R^{N×d} and labels y ∈ {0,1}^N.
• Linear transform: z = Xw + b, where b is broadcast to all N rows.
• Nonlinearity: y_hat = σ(z) element-wise.
• Loss: L = −[y ⊙ log(y_hat) + (1−y) ⊙ log(1−y_hat)], then average.
• Backward: Compute gradients ∂J/∂w and ∂J/∂b and update parameters.

1. Code/Implementation Details Language: Python with NumPy is typical for a simple implementation. The core operations are vectorized for speed.

Key Equations and Gradient Derivation

• Sigmoid: σ(z) = 1 / (1 + e^(−z)). Its derivative: σ'(z) = σ(z) (1 − σ(z)).
• Loss per example: L = −[y log(y_hat) + (1−y) log(1−y_hat)], with y_hat = σ(z).
• Derivative w.r.t. z: ∂L/∂z = (y_hat − y). This comes from chain rule: ∂L/∂y_hat = −(y/y_hat) + ((1−y)/(1−y_hat)) and ∂y_hat/∂z = y_hat(1−y_hat). After simplifying, terms collapse to y_hat − y.
• Batch gradients: For X ∈ R^{N×d}, y_hat ∈ R^N, y ∈ R^N, ∂J/∂w = (1/N) X^T (y_hat − y) ∂J/∂b = (1/N) ∑_{i=1}^N (y_hat_i − y_i) These concise forms make implementation short and less error-prone.

Pseudocode (Vectorized)

• Initialize w = small random vector (d,), b = 0
• For epoch in 1..E: z = X @ w + b y_hat = sigmoid(z) loss = mean( − (y * log(y_hat) + (1−y) * log(1−y_hat)) ) grad_w = (1/N) * X.T @ (y_hat − y) grad_b = (1/N) * sum(y_hat − y) w = w − α * grad_w b = b − α * grad_b if loss improvement < tolerance for K checks: break

Key Parameters

• Learning rate (α): Step size for updates; too large can diverge, too small slows training.
• Epochs: How many full passes over the dataset.
• Batch size: 1 for SGD (fast/noisy), full N for batch GD (stable/slow), or mini-batches for a good balance.
• Threshold (τ): Turns probabilities into hard labels; can be tuned for precision/recall trade-offs.

1. Tools/Libraries Used

• NumPy: For vectorized matrix math (dot products, broadcasting, element-wise sigmoid).
• Optional: scikit-learn’s LogisticRegression for a production-ready solver (liblinear, lbfgs, saga). Even if you use a library, knowing the math helps with choices and interpretation.

Why Cross-Entropy, Not MSE

• With sigmoid outputs, MSE creates a non-convex surface. Intuitively, squared errors plus a saturating nonlinearity (sigmoid flattens near 0 and 1) make plateaus and bumps. Gradient descent can stall when predictions are near extremes because the sigmoid derivative is tiny.
• Cross-entropy aligns with maximum likelihood for Bernoulli-distributed labels. It directly measures how well predicted probabilities match true labels. It is convex in w and b for logistic regression, so gradient descent is dependable.

Interpreting Weights and Odds

• Odds = P(y=1)/P(y=0) = y_hat/(1−y_hat). Log-odds (logit) = log(odds) = z = w^T x + b.
• A positive weight increases log-odds; a negative weight decreases log-odds. Each unit increase in feature x_j multiplies the odds by exp(w_j), holding others fixed.
• This interpretability helps explain which features push predictions up or down.

Decision Boundary and Geometry

• The boundary is {x | w^T x + b = 0}. In 2D, this is a straight line; in 3D, a plane; in higher dimensions, a hyperplane.
• If two classes are linearly separable, there exists w and b that perfectly separate them (ignoring noise). Logistic regression will place the boundary where it best matches probabilities, not just any separator, which often improves generalization.

Training Variants

• Batch Gradient Descent: Uses the whole dataset per update; stable but slow on large N.
• Stochastic Gradient Descent (SGD): Updates per single example; faster per step but noisy. Often good for very large datasets and can escape shallow plateaus.
• Mini-Batch Gradient Descent: Uses small batches (like 32–1024). A good balance of speed and stability. Easy to vectorize and optimized by modern hardware.

Practical Implementation Guide Step 1: Prepare data

• Collect features x and labels y ∈ {0,1}. Clean missing values, standardize scales if needed (especially if features vary widely). Optionally add a bias column of 1s if implementing without an explicit b.

Step 2: Initialize

• Set w to small random numbers (e.g., Normal(0, 0.01)) and b = 0. Random starts break symmetry and help learning begin.

Step 3: Forward pass

• Compute z = Xw + b and y_hat = σ(z). Check that y_hat ∈ (0,1).

Step 4: Compute loss

• Use cross-entropy: J = mean(−y log(y_hat) − (1−y) log(1−y_hat)). For numerical stability, clip y_hat into [ε, 1−ε] (e.g., ε=1e−15) before logging.

Step 5: Backward pass

• Compute grad_w = (1/N) X^T (y_hat − y), grad_b = mean(y_hat − y).

Step 6: Update

• w ← w − α grad_w; b ← b − α grad_b. Choose α via small experiments (e.g., 0.1, 0.01, 0.001) and pick what converges fastest without diverging.

Step 7: Iterate to convergence

• Repeat steps 3–6 for many epochs. Stop when loss change is tiny for several checks, or when validation loss stops improving.

Step 8: Threshold and evaluate

• Use τ = 0.5 to start; adjust based on goals. For example, use τ < 0.5 to catch more positives if recall matters more. Evaluate with accuracy and, depending on class balance, consider precision/recall (even though not covered deeply here, the idea of threshold choice is essential).

Tips and Warnings

• Do not use MSE for training logistic regression. It makes the problem non-convex and harder to optimize.
• Watch for saturation: When z is very large in magnitude, σ(z) is very close to 0 or 1. Gradients become small, slowing learning. Good initialization and proper learning rates help reduce this.
• Feature scaling helps. If one feature has values in the thousands and another is between 0 and 1, the optimization can be slow or unstable. Standardize or normalize features so gradients behave consistently.
• Choose batch size wisely. Mini-batches often provide the best speed/stability trade-off.
• Monitor loss over time. A steadily decreasing loss indicates learning; oscillation or divergence means the learning rate is too high.
• Numerical stability: Clip y_hat before log to avoid log(0). Implement sigmoid carefully to avoid overflow for very large negative or positive z (e.g., use stable formulations).

Why Cross-Entropy Penalizes Confident Wrong Predictions

• If y=1 but y_hat is near 0, −log(y_hat) is huge (because log of a very small number is a very large negative number). So the loss spikes, pushing the model to correct drastically. If y=1 and y_hat ≈ 1, −log(y_hat) is near 0, rewarding confident correct predictions. Similarly for y=0 with −log(1−y_hat). This behavior encourages well-calibrated probabilities.

Extending to Multi-Class (High-Level)

• One-vs-All (OvA): For K classes, train K binary classifiers. Class k’s classifier predicts P(y=k). At prediction, pick argmax_k P(y=k).
• One-vs-One (OvO): Train a classifier for every pair of classes; combine votes. While more complex, OvA is often simpler and works well with logistic regression’s probability outputs.

When to Use Logistic Regression

• As a fast, interpretable baseline. For large sparse data (like text features), it often performs surprisingly well. When probability estimates are important (risk scoring, ranking), it’s a great fit. If relationships are roughly linear or can be linearized with simple feature engineering, it is an excellent choice.

Putting It All Together

• The pipeline is: collect and clean data → build features → initialize parameters → forward pass and compute probabilities → compute cross-entropy loss → compute gradients → update parameters via gradient descent → repeat until convergence → choose threshold and make decisions. Each step has clear math and simple code, making logistic regression one of the most teachable and usable models in machine learning.

Logistic regression offers a clean, principled way to turn input features into a probability of belonging to class 1. It builds on a simple linear score (w^T x + b) and uses the sigmoid function to keep outputs between 0 and 1. The key to successful training is choosing the right loss: cross-entropy (log loss), not mean squared error. Cross-entropy is convex for logistic regression, which makes gradient descent reliable and able to reach the global minimum. The gradients take an elegant form, with y_hat − y at the core, enabling short and efficient implementations.

Once trained, the model’s decision boundary is linear, and its outputs are interpretable probabilities. This makes thresholding straightforward and flexible: you can set different cutoffs for different goals, such as catching more positives or reducing false alarms. The weights have a clear meaning in terms of log-odds, so you can understand which features push predictions up or down. These qualities make logistic regression both practical and explainable, a combination that is valuable in many domains.

However, the model’s linear boundary also sets its main limitation: it cannot capture complex curves without help. When the data is not linearly separable, feature engineering (like polynomial or interaction terms) is often needed. Still, even with this limitation, logistic regression remains an excellent starting point and a reliable baseline for classification tasks. It scales well, trains fast, and produces probability outputs that are easy to work with and reason about.

To practice, implement a logistic regression classifier from scratch using NumPy: write the sigmoid, the cross-entropy loss, and the gradient updates. Train it on a simple dataset, tune the learning rate, and try different thresholds for decisions. Then, explore a one-vs-all setup on a small multi-class dataset to see how binary models can be combined to handle more classes. The core message to remember is simple: use the sigmoid for probabilities, use cross-entropy to learn, and use gradient descent to optimize. With these pieces in place, you can build solid classifiers and understand exactly how and why they work.