Stanford CS329H: Machine Learning from Human Preferences | Autumn 2024 | Human-centered Design

This session introduces the foundations of machine learning (ML) and sets clear expectations for what you will learn and how the course will run. The central idea is simple but powerful: instead of writing a long list of rules by hand, we let computers learn patterns from data and then use those patterns to make predictions or decisions. Think of the classic “detect a cat” problem. In the old way, you might code rules for ears, whiskers, and fur color. In ML, you feed many pictures labeled “cat” and “not cat,” and the system learns the patterns that separate the two classes. The lecture defines ML as building systems that learn from data using computation to make predictions or decisions. It highlights the practice of actually building useful systems, not just studying math or code in isolation. It also underlines the predictive nature: we learn from past data to guess the future.

This lecture is for beginners who want to gain practical ML skills: choosing algorithms, applying them to real data, debugging results, and judging when ML is appropriate. You will write Python code and use standard libraries like NumPy, SciPy, and scikit-learn, so a basic comfort with Python is recommended or should be developed early. While statistics, programming, and data analysis all matter, ML combines them into a coherent craft of building predictive systems. The class also aims to help you read research papers, so you can follow the latest developments and understand what new ideas mean in practice.

After this lecture, you should be able to explain what ML is and is not. You will recognize core real-world use cases such as personalization (Amazon, Netflix, YouTube), search ranking (Google, Bing), spam filtering (email), fraud detection (credit cards), medical imaging assistance (X-rays), and self-driving perception (detecting pedestrians, other vehicles, and traffic lights). You will understand example problem types across data modalities: image classification (image in, label out), sentiment analysis (text in, sentiment out), machine translation (text in one language, text out in another), speech recognition (audio in, text out), and recommender systems (user history in, ranked items out). You will also know exactly how the course is graded, how to form teams, what tools to use, and what the project expectations are.

The lecture is structured in four parts. First, it defines ML and clarifies what it includes and excludes. Second, it surveys everyday use cases, showing how ML shows up in products you already use. Third, it walks through representative problem types to ground the definition with concrete tasks and outputs. Fourth, it explains the course logistics: grading breakdown, homework schedule and tools, project expectations, late-day policy, and academic integrity. The lecture closes with a short Q&A on Python resources, suggesting Codecademy, “Automate the Boring Stuff with Python,” and plentiful online tutorials. The core message is consistent: ML is about building systems that learn from data to make predictions or decisions, and you will practice doing exactly that throughout the course.

Overall Architecture/Structure of an ML Project

1. Problem framing (what decision or prediction?)

• 🎯 Definition: Clarify the input, the desired output, and the business or user goal.
• 🏠 Analogy: It’s like deciding what kind of map you need before a trip: a city map (classification) or a highway map (regression or ranking).
• 🔧 Technical: Define input modality (image, text, audio), output type (label, score, text), and success metric (e.g., accuracy for classification, relevance for ranking). Scope the decision latency (real time vs. batch) and constraints (privacy, compute, cost).
• 💡 Why it matters: A fuzzy goal leads to mismatched data, wrong algorithms, and unclear results.
• 📝 Example: “Given a product page view (input), predict the top five related products to show (output), to improve click-through rate (goal).”

1. Data collection and labeling

• 🎯 Definition: Gather examples that represent the problem and attach correct labels when needed.
• 🏠 Analogy: Like building a study guide from many past exams with the correct answers attached.
• 🔧 Technical: Collect historical logs, public datasets, or human annotations. Ensure coverage of common and rare cases. Store examples with metadata (time, source) to track drift later.
• 💡 Why it matters: Poor or biased data breaks the model’s ability to generalize, no matter how fancy the algorithm.
• 📝 Example: For spam filtering, compile a dataset of emails labeled spam/ham from real inbox traffic (with privacy safeguards).

1. Feature preparation or representation

• 🎯 Definition: Turn raw inputs into forms models can use (pixels, tokenized text, spectrograms).
• 🏠 Analogy: Like converting ingredients into a recipe-ready state (chopped, measured, mixed).
• 🔧 Technical: Images may be resized and normalized; text is tokenized; audio is turned into frames. For classical ML, engineer features (e.g., word counts), while many modern models learn features automatically.
• 💡 Why it matters: Bad representations hide useful patterns and hurt accuracy.
• 📝 Example: Sentiment analysis might use bag-of-words or embeddings as inputs to a classifier.

1. Model selection

• 🎯 Definition: Choose an algorithm that fits the task, data size, and constraints.
• 🏠 Analogy: Picking the right tool from a toolbox: you don’t use a sledgehammer for a thumbtack.
• 🔧 Technical: For small text tasks, a linear classifier can work well; for large images, convolutional networks are common; for recommendations, factorization or neural ranking may fit.
• 💡 Why it matters: The “right-sized” model balances performance, speed, and ease of debugging.
• 📝 Example: Start an email spam filter with logistic regression over TF-IDF features, then iterate.

1. Training

• 🎯 Definition: Fit the model to data so it learns patterns that map inputs to outputs.
• 🏠 Analogy: It’s like practicing many problems until you get the hang of them.
• 🔧 Technical: Split data into train/validation/test sets; optimize model parameters to minimize loss on the training set while monitoring validation performance to avoid overfitting.
• 💡 Why it matters: Training teaches the model; validation ensures it’s learning generalizable patterns, not just memorizing.
• 📝 Example: Train an image classifier on labeled pictures; stop when validation accuracy plateaus.

1. Evaluation

• 🎯 Definition: Measure how well the model performs on unseen data in a way that reflects real goals.
• 🏠 Analogy: Like a final exam that checks true understanding, not memorization.
• 🔧 Technical: Use held-out test sets and metrics that match the task (accuracy for classification; precision/recall for skewed classes; ranking metrics for search/recommendations). Consider latency and resource use.
• 💡 Why it matters: The wrong metric can point you to the wrong “best” model.
• 📝 Example: For spam, track precision (few false alarms) and recall (catch most spam) together.

1. Deployment and monitoring

• 🎯 Definition: Put the model into the product and keep watch as real data evolves.
• 🏠 Analogy: Like launching a ship and then constantly checking the weather and instruments.
• 🔧 Technical: Serve the model behind an API, collect feedback loops, monitor input shifts and performance drops, and retrain as needed.
• 💡 Why it matters: Data changes over time; a good model today can degrade tomorrow.
• 📝 Example: A fraud model sees new scam patterns; weekly retraining helps catch them.

How the Concepts Map to Lecture Examples

Personalization (Amazon/Netflix/YouTube)

• Inputs: user history (clicks, views, purchases), item metadata (genres, categories), and context (time, device).
• Model: Recommendation systems that rank items by predicted engagement.
• Output: Top-N recommended items personalized to each user.
• Deployment: Real-time ranking on page load; monitor clicks and watch time as feedback.

Search Ranking (Google/Bing)

• Inputs: the search query, page content features, link structure, and past behavior signals.
• Model: Learning-to-rank algorithms that score candidate pages.
• Output: Ordered list of results; quality measured by relevance metrics and click behavior.
• Deployment: Must be fast, scalable, and safe; constant A/B testing.

Spam Filtering (Email)

• Inputs: email header fields, sender reputation, text content, links.
• Model: Classifier that outputs spam vs. ham with a confidence score.
• Output: Route to inbox, spam folder, or flag for review.
• Constraints: High precision is critical to avoid losing real messages.

Fraud Detection (Credit Cards)

• Inputs: transaction amount, merchant type, location, time, user profile.
• Model: Anomaly detection or supervised classifier trained on confirmed fraud labels.
• Output: Risk score and possible action (approve, flag, or call the user).
• Monitoring: Adapt quickly to new fraud tactics and seasonality.

Medical Imaging (X-rays)

• Inputs: patient scan images and labels from radiologists.
• Model: Image classifier or detector that highlights suspicious regions.
• Output: Decision support for clinicians, not a sole diagnosis.
• Safety: Requires careful validation and regulatory considerations.

Self-Driving Perception

• Inputs: camera, lidar, radar streams; synchronized and calibrated.
• Model: Object detection and segmentation models running in real time.
• Output: Detected objects with positions and classes; feeds planning and control.
• Constraints: Extremely low latency and high reliability under varied conditions.

Representative Problem Types

1. Image Classification

• Input: image array; Output: class label.
• Typical steps: preprocess (resize/normalize), train a classifier, evaluate accuracy.
• Uses: content moderation, photo search, scene understanding.

1. Sentiment Analysis

• Input: text string; Output: sentiment label (positive/negative/neutral).
• Typical steps: tokenize text, represent (bag-of-words, embeddings), train a classifier.
• Uses: customer feedback mining, brand monitoring.

1. Machine Translation

• Input: source-language text; Output: target-language text.
• Typical steps: learn mappings between languages; ensure meaning preservation.
• Uses: cross-language communication and content access.

1. Speech Recognition

• Input: audio waveform; Output: transcribed text.
• Typical steps: feature extraction from audio, sequence modeling, decoding.
• Uses: assistants, dictation, subtitles.

1. Recommendation

• Input: user-item interaction history; Output: ranked list of items.
• Typical steps: model user preferences and item features; rank candidates.
• Uses: e-commerce, media platforms, news feeds.

Tools/Libraries Used in the Course

• Python: The primary programming language for assignments and projects. It is beginner-friendly and has rich ML libraries.
• NumPy: Provides fast arrays and numerical operations; used for data manipulation and linear algebra.
• SciPy: Adds scientific computing utilities, optimization, and signal processing routines that often support ML tasks.
• scikit-learn: A go-to library for classical ML with consistent APIs for preprocessing, model training, and evaluation; ideal for classification, regression, and simple pipelines.
• Gradescope: The submission platform for homeworks and the final project; supports group submissions and structured grading.

Step-by-Step Implementation Guide (Beginner-Friendly)

A. Set up your environment

1. Install Python 3.x and create a virtual environment.
2. Install NumPy, SciPy, and scikit-learn (e.g., pip install numpy scipy scikit-learn).
3. Verify imports in a Python shell.

B. Load and inspect data

1. Read your dataset (CSV for text/structured; image folders for vision; audio files for speech).
2. Print shapes, class counts, and sample rows or images to understand what you have.
3. Check for missing values and unusual outliers that might break training.

C. Split data

1. Create train/validation/test splits so you can measure generalization.
2. Keep the test set untouched until final evaluation.
3. Use the validation set to choose models and tune thresholds.

D. Preprocess inputs

1. Text: lowercase, tokenize, maybe remove punctuation; create features (e.g., TF-IDF) or use embeddings.
2. Images: resize to a standard shape, normalize pixel values; optionally augment training data.
3. Audio: trim or segment audio, compute time-frequency representations.

E. Choose a baseline model

1. For classification, try logistic regression or a simple tree-based model in scikit-learn.
2. Train quickly and measure validation performance to get a reference point.
3. Keep the baseline code clean so you can reuse it for new features or models.

F. Train and iterate

1. Adjust features or hyperparameters to improve validation performance.
2. Watch for overfitting: training accuracy high but validation accuracy low signals a problem.
3. Save the best model and document what changed each iteration.

G. Evaluate and report

1. Compute relevant metrics on the test set.
2. Create simple plots or tables to summarize results.
3. Write up a short explanation of what worked, what didn’t, and next steps.

H. Deploy (project-oriented)

1. Wrap the model in a function or API for easy use.
2. Log predictions and feedback for future improvement.
3. Schedule periodic retraining if data changes.

Tips and Warnings

• Start simple: A strong baseline beats a fancy but fragile approach.
• Match metrics to goals: For spam, balance precision and recall; for recommendations, focus on ranking quality.
• Data quality > model complexity: Clean, representative data often increases accuracy more than switching algorithms.
• Keep a log: Track experiments so you can reproduce improvements and avoid repeating mistakes.
• Respect constraints: Some tasks need real-time speed; others can run in batch mode overnight.
• Group work: Split tasks by strengths—data prep, modeling, evaluation, write-up—and integrate early.
• Academic integrity: Write your own code within your group, cite sources, and never copy solutions.

Course Logistics (From the Lecture)

• Grading breakdown: 40% homeworks (four, equally weighted, real-world datasets, Python/NumPy/SciPy/scikit-learn), 30% midterm (multiple choice + short answer on concepts), 30% final project (pick any ML problem, 6–8 page report, 5–10 minute presentation).
• Homework rules: Due Fridays at 5:00 PM via Gradescope; groups up to three; one submission per group; code in Python.
• Final project: Due last day of class; submit via Gradescope; groups up to three; present to class; write a 6–8 page report.
• Late policy: Three total late days across the quarter; can’t use on midterm; max one late day per assignment; more than three days late earns zero.
• Academic integrity: No cheating, no plagiarism, and no collaboration outside your group; ask questions if unsure.
• Python resources: Codecademy, “Automate the Boring Stuff with Python,” and many beginner-friendly tutorials on YouTube and beyond.

Why This Setup Works

• Aligns with the definition: You’ll build systems that learn from data to make predictions or decisions.
• Emphasizes practice: Homeworks and the final project force you to pick algorithms, apply them, and debug—exactly what ML engineers do.
• Builds independence: By the end, you can choose when ML is appropriate and select efficient solutions, even if not deep learning.
• Encourages growth: Reading research and presenting your project prepares you to engage with the broader ML community.

In sum, the technical backbone is a repeatable loop—frame the problem, gather and prepare data, pick and train a model, evaluate properly, deploy carefully, and keep monitoring. The lecture illustrates this loop with familiar applications: recommendations, search, spam, fraud, health imaging, and autonomous driving. It rounds out with concrete course logistics so you can plan your time and tools, and it points to accessible Python resources so you can get productive quickly.

This lecture lays a clear foundation: machine learning builds systems that learn patterns from data to make predictions or decisions. The heart of ML is shifting from hand-written rules to learning from examples, which lets us handle complex, messy, and ever-changing real-world inputs. You saw where this matters most in daily life: personalization for products and media, search ranking on the web, spam filtering in email, fraud alerts for payments, medical imaging assistance in clinics, and perception for self-driving cars. You also saw the common problem types—image classification, sentiment analysis, machine translation, speech recognition, and recommendations—which together span images, text, and audio.

Equally important, you learned what ML is not: it is not just writing code, not just data analysis, not just statistics, and not only deep learning. ML combines coding, data understanding, algorithms, and systems thinking to deliver useful predictions. The course is designed to make you fluent in this process: four hands-on homeworks in Python using NumPy, SciPy, and scikit-learn; a midterm focused on core concepts; and a final project where you apply ML to a problem you choose and present your results. Clear policies on late days and academic integrity keep the class fair and focused, and practical Python resources help you get up to speed quickly.

To cement these ideas, try small end-to-end projects right away: build a simple text sentiment classifier, a basic image labeler, or a toy recommender from sample data. Practice splitting data, training a baseline, measuring performance, and iterating. As you grow, you can tackle richer tasks or larger datasets, but the core loop—frame, collect, represent, train, evaluate, deploy, monitor—stays the same. The core message to remember is simple and powerful: use data to learn, not hand-write rules, and always measure success on new, unseen cases. With that mindset and the course structure, you are set to become confident at solving real ML problems and at knowing when ML is the right tool for the job.