Stanford CS329H: Machine Learning from Human Preferences | Autumn 2024 | Voting

This first lecture launches an Introduction to Machine Learning course and sets expectations for both the logistics and the learning journey. The instructor, Byron Wallace (Khoury College of Computer Sciences), begins by introducing himself and the teaching assistants, Max and Zohair, who will host office hours and help with assignments. Students are directed to Piazza (linked from Canvas) as the single source of truth for announcements, lecture notes, assignments, and recordings. The class will meet in person, and recordings will be posted shortly after each lecture. Graded components include weekly homework (50%), an in-person midterm (20%), and a final project (30%), which is done in the same groups as the homework and includes a mid-semester proposal, a final report, and a presentation.

There are no rigid prerequisites, but students are expected to have basic programming skills, especially in Python, and to be comfortable with linear algebra and calculus. Review materials are provided for those who feel rusty. Collaboration is allowed in groups of up to three for homework, but each student must write up and submit their own solutions, encouraging real learning rather than copying. Academic integrity is highlighted: you may look online for ideas, but you must implement your own solutions and never post course solutions publicly.

Pedagogically, the course is organized into modules, each focusing on a family of algorithms and paired with readings, lectures, and an assignment. The first module covers supervised learning, starting with linear models. Here, you learn the core supervised learning setup: input features (also known as predictors, independent variables, or covariates) and targets (also known as outcomes, dependent variables, or responses). The goal is to learn a function that maps inputs to outputs. Two key supervised problem types are defined: classification (predicting categories, like spam vs. not spam or animal types) and regression (predicting continuous values, like house price or tomorrow’s temperature).

Within this framework, linear models are positioned as simple, interpretable, and surprisingly powerful. A linear model assumes the target is a weighted sum of the input features plus an intercept. The house price example illustrates this with terms like square footage, location, and number of bedrooms. Learning means finding parameter values (weights/coefficients) that minimize the difference between predicted and actual targets on training data. Linear models can be used for both classification and regression and are foundational building blocks for more complex methods.

Feature engineering is introduced as a crucial practice for improving model performance. Instead of feeding raw inputs like latitude and longitude directly into a model, you might create more informative features such as distance to a city center or neighborhood average income. These engineered features can make patterns more obvious to the model, increasing accuracy and reducing the need for highly complex architectures. The importance of thoughtful feature creation and transformation is emphasized because it often has a larger impact on performance than changing algorithms.

After linear models, the course broadens to non-linear models such as decision trees, random forests, and neural networks. While these still operate within the supervised learning paradigm (mapping inputs to outputs using labeled data), they use non-linear functions, which allows them to represent more complex relationships in the data. Next, the course turns to unsupervised learning, which finds structure in unlabeled data through techniques like clustering (grouping similar points) and dimensionality reduction (compressing data for easier visualization and preprocessing). Finally, the course covers probabilistic machine learning, including Bayesian inference, graphical models, and Markov chain Monte Carlo (MCMC). These methods explicitly represent uncertainty and are especially helpful when data is scarce or when you want to incorporate prior knowledge.

Throughout, the instructor references real-world applications, particularly in medicine and public health, such as predicting heart attack risk or understanding the effects of clinical interventions. This grounds the theory in meaningful problems and motivates careful modeling choices. The lecture closes by previewing the next steps: delving into specific training algorithms for linear models in the following session. By the end of the lecture, students know how the class runs, what topics they will learn, and the key concepts of supervised learning, classification vs. regression, linear models, and feature engineering.

Overall Architecture/Structure of the Course and Concepts

1. The Course Flow

• Central hub: Piazza/Canvas hosts announcements, slides, assignments, and recordings. This guarantees a single reliable place to track progress and deadlines.
• Rhythm: Each module includes readings, in-class theory, and a matching homework that forces you to apply what you learned. Lectures are recorded and posted soon after class so you can revisit complex parts.
• Assessment structure: Homework (50%) drives weekly practice; the in-person midterm (20%) assesses core understanding mid-course; the final project (30%) develops depth via an open-ended, group-based exploration.
• Collaboration: Work in fixed groups (up to three) for homework to foster consistent teamwork, but write and submit your own solutions to ensure personal mastery.
• Academic integrity: Implement your own code and reasoning. Looking at references is fine for ideas, but copy-pasting code or sharing answers publicly undermines learning and violates policy.

1. Conceptual Architecture of Machine Learning Taught Here

• Data types: Inputs (features, predictors, covariates) and targets (labels, outcomes, responses) are the fundamental units in supervised learning.
• Supervised learning: Learn a mapping f: X → Y from labeled examples. Use known targets to guide the model in discovering patterns connecting features to outcomes.
• Task categories: Classification (categorical Y) and regression (continuous Y) define how predictions are evaluated and what losses/metrics make sense.
• Model families: Start with linear models (interpretable baselines), then explore non-linear models (decision trees, random forests, neural networks) for more expressive power, then unsupervised learning (clustering, dimensionality reduction) for structure discovery, and finally probabilistic ML (Bayesian inference, graphical models, MCMC) for uncertainty-aware modeling.
• Feature engineering: Transform raw inputs into informative features (e.g., distance to city center from lat/long) to improve model performance, often more than algorithm changes.

Data Flow in Supervised Learning

• Input: A dataset of pairs (x_i, y_i), where x_i is a vector of features for example i, and y_i is the target.
• Model: A function class parameterized by θ (parameters), such as θ = (β0, β1, …, βp) for a linear model.
• Training: Choose θ to minimize a loss over the training set, such as the mean squared error for regression or a classification-appropriate loss (e.g., cross-entropy) for classification. Optimization adjusts θ step by step to reduce loss.
• Prediction: For a new x, compute f_θ(x) to obtain a predicted ŷ (a number for regression or a label/probabilities for classification).
• Evaluation: Compare predictions with ground truth using metrics appropriate to the task (e.g., RMSE for regression; accuracy, precision, recall for classification).

Linear Models Explained Clearly

• Assumption: The target can be approximated as a weighted sum of features plus an intercept: ŷ = β0 + β1 x1 + β2 x2 + … + βp xp.
• Interpretability: Each βj tells you how much xj contributes to the prediction, holding other features constant. Positive means increasing the feature raises the prediction; negative means it lowers it.
• Example (house price): Price = β0 + β1×square footage + β2×location + β3×number of bedrooms. If β1 is large and positive, bigger houses predict higher prices.
• Use cases: Linear regression for continuous targets; linear classifiers (like logistic regression) for categorical targets (via a non-linear link on the output).

Training Linear Regression (Intuition)

• Goal: Minimize the average squared error between predictions and true targets: Loss(β) = (1/n) Σ_i (y_i − ŷ_i)^2.
• Why squared error: It penalizes bigger mistakes more strongly and leads to a smooth optimization landscape, making learning stable and efficient.
• Solving: One can compute a closed-form solution (ordinary least squares) or use iterative optimization (like gradient descent) for large datasets or when adding constraints.
• Regularization (conceptual preview): To prevent overfitting (fitting noise), we can add penalties on coefficient size (L2/Ridge, L1/Lasso). Though not covered in detail yet, this idea helps keep models simple and robust.

Classification with Linear Models (Preview)

• Direct linear outputs are not naturally probabilities. Logistic regression applies a logistic (sigmoid) function to map a linear score into a probability between 0 and 1 for binary classification.
• Decision rule: Predict class 1 if probability ≥ threshold (often 0.5), else class 0. For multi-class problems, use softmax regression.
• Loss: Cross-entropy (log loss) encourages predicted probabilities close to true labels and penalizes confident wrong predictions more heavily than mean squared error does in classification settings.

Feature Engineering in Practice

• Why it matters: The model can only see what the features show. Features that align with true factors make learning easier and more accurate.
• Transform raw to meaningful: Latitude/longitude are not directly “nearness to downtown,” but you can compute distance to a reference point to capture that signal. Similarly, neighborhood statistics (e.g., average income) can reflect broader context.
• Handling scales: Features with very different scales (e.g., square footage vs. binary indicators) can distort learning; normalization or standardization helps many models and optimizers behave well.
• Interaction features: Sometimes, the effect of one feature depends on another (e.g., size might matter differently by neighborhood). Creating products or non-linear transforms can let a linear model approximate more complex relationships.

Non-linear Models (High-Level)

• Decision trees: Split data by feature thresholds to form a tree of decisions leading to predictions in leaves. Very interpretable paths but can overfit.
• Random forests: Many trees built on random feature/data subsets whose predictions are averaged (regression) or voted (classification); reduces overfitting and improves accuracy.
• Neural networks: Layers of linear transformations combined with non-linear activations; extremely flexible function approximators that can learn complex, high-dimensional mappings with enough data and regularization.
• Trade-offs: These models capture complex patterns but are typically less interpretable than linear models. They often require more tuning and careful validation.

Unsupervised Learning (High-Level)

• Clustering: Groups similar data points together without labels (e.g., K-means). Useful for exploratory analysis, customer segmentation, or as a preprocessing step.
• Dimensionality reduction: Compresses data into fewer dimensions while preserving structure (e.g., PCA). Helpful for visualization and noise reduction, making downstream learning easier.

Probabilistic Machine Learning (High-Level)

• Bayesian inference: Combines prior beliefs with observed data to produce a posterior distribution over unknowns (parameters/predictions). Quantifies uncertainty directly.
• Graphical models: Represent complex joint distributions with a factorized graph, making reasoning about dependencies tractable.
• MCMC: Algorithms to sample from complex probability distributions when direct calculation is hard, enabling approximate Bayesian inference.
• When valuable: Small datasets, noisy measurements, or domains where decisions must reflect uncertainty (e.g., medicine) benefit greatly from probabilistic approaches.

Tools/Libraries (What You’ll Commonly Use)

• Python: The primary programming language for assignments.
• Numerical computing: NumPy for array operations and linear algebra; pandas for data manipulation and cleaning.
• Modeling: scikit-learn for baseline models (linear regression, logistic regression, decision trees, random forests, clustering, PCA). These are standard, well-documented tools that let you focus on concepts rather than boilerplate code.
• Visualization: Matplotlib or Seaborn to plot data distributions, learning curves, and evaluation metrics.

Step-by-Step Implementation Guide (Foundational Workflow)

1. Frame the problem

• Determine if it’s supervised (do you have labels?) and whether it’s classification or regression. Define your target variable clearly.
• Split your data into training and test sets so you have an unbiased way to assess performance.

1. Inspect and clean data

• Check for missing values, unrealistic outliers, and inconsistent categories. Decide on imputation strategies or filtering rules.
• Visualize distributions and relationships to understand what features may help.

1. Feature engineering and preprocessing

• Create informative features (e.g., distance to city center from lat/long; BMI from height/weight). Consider interaction terms if you suspect combined effects.
• Scale features where appropriate (e.g., standardize to zero mean and unit variance) for algorithms sensitive to feature scale.

1. Choose a baseline model

• Start with a simple, interpretable model (linear regression for continuous targets, logistic regression for binary classification). Document baseline metrics.
• Set up a consistent evaluation protocol (e.g., cross-validation for robustness when data is limited).

1. Train and evaluate

• Fit the model on training data. Compute errors on validation/test sets using appropriate metrics (e.g., RMSE for regression; accuracy/precision/recall/AUC for classification).
• Inspect coefficients in linear models to understand feature influence and sanity-check signs/magnitudes.

1. Iterate

• Improve features based on insights. Try non-linear models if linear performance is inadequate.
• Tune hyperparameters (e.g., regularization strength for linear/logistic regression; depth/number of trees for random forests) using validation data.

1. Communicate results

• Summarize performance, key drivers (features), and limitations. Provide uncertainty estimates or sensitivity analyses when possible, especially in high-stakes domains.

Tips and Warnings

• Avoid data leakage: Do not use information from the test set (or future data) in feature engineering or training. Keep all preprocessing steps inside a pipeline fitted only on the training data.
• Match metrics to tasks: Don’t use accuracy alone for imbalanced classification; consider precision, recall, F1, and AUC. For regression, look at RMSE/MAE and consider residual plots.
• Start simple: A solid linear baseline clarifies whether complexity adds real value. Complex models without careful validation often overfit.
• Watch feature scales: Standardize features for gradient-based methods; tree-based models are less sensitive but still benefit from clean, consistent inputs.
• Document everything: Track how features are created, what parameters you used, and what results you obtained. Reproducibility is part of professional ML practice.

Concrete Examples From the Lecture Context

• Cat vs. dog images (classification): Inputs are pixel/feature representations; target is the animal type. A linear classifier provides a baseline; a CNN (later) could improve results.
• Spam detection (classification): Features can be word frequencies or embeddings; target is spam/not spam. Logistic regression often performs well with good text features.
• House prices (regression): Features include square footage, location info, and bedrooms; target is price. Feature engineering (distance to city center, neighborhood income) can significantly boost performance.
• Medical risk prediction (classification/regression): Features are age, blood pressure, cholesterol, BMI; target could be probability of heart attack within 5 years. Uncertainty-aware methods are critical in healthcare.
• Clustering and dimensionality reduction (unsupervised): Group similar patients or compress features to visualize cohorts. Useful for exploration and hypothesis generation.

Putting It All Together This lecture defines the roadmap: learn the supervised learning basics using linear models and careful feature engineering, expand to non-linear models for richer patterns, explore unsupervised learning to understand unlabeled data structure, and finish with probabilistic methods to rigorously model uncertainty. The course’s structure—weekly practice, clear evaluation, collaboration with accountability, and an open-ended final project—supports steady skill growth. With these foundations, you’ll be equipped to handle practical ML tasks responsibly, interpretably, and effectively.

This opening lecture lays a clear foundation for both how the course runs and what you will learn. You meet the instructor and TAs, learn the grading breakdown, understand collaboration and integrity expectations, and know where to find everything (Piazza/Canvas). The content roadmap moves from supervised learning with linear models to non-linear models, then to unsupervised learning, and finally to probabilistic machine learning. Within supervised learning, the key ideas are inputs (features), targets (labels), and the goal of learning a function that maps one to the other for classification or regression. Linear models are emphasized as simple, interpretable baselines, and feature engineering is highlighted as a critical lever for performance.

Practically, your next steps are to prepare your tools and knowledge: ensure comfort with Python, brush up on linear algebra and calculus using the provided reviews, and form your homework group early. Begin thinking about data problems that interest you for the final project, and consider what engineered features might help reveal patterns. As homework arrives, start with a clean baseline (like linear regression or logistic regression), implement thoughtful preprocessing, and evaluate carefully with appropriate metrics.

After mastering these basics, you will be ready to tackle non-linear models, explore structure in unlabeled data with unsupervised techniques, and approach uncertain, small-data situations with probabilistic methods. Recommended resources include scikit-learn documentation for practical modeling, NumPy/pandas guides for data handling, and beginner-friendly texts or tutorials on linear models and classification metrics. The instructor’s core message is to build on strong fundamentals: understand the supervised setup, value interpretability and baselines, and invest in feature engineering. With this mindset and the course’s structured support, you can develop robust ML skills that translate to real-world impact, especially in sensitive domains like healthcare where clarity and confidence matter.