Probabilistic Foundations of ML#
Author: Yaniv Yacoby
Courses: This is the textbook for courses CS 245 and CS 345 @ 
About: In recent years, Machine Learning has enabled applications that were previously not thought possible—from systems that propose novel drugs or generate new art/music, to systems that accurately and reliably predict outcomes of medical interventions in real-time. But what has enabled these developments? Faster computing hardware, large amounts of data, and the Probabilistic paradigm of Machine Learning (ML), a paradigm that casts recent advances in ML, like neural networks, into a statistical learning framework. In this book, we introduce the foundational concepts behind this paradigm—statistical model specification, and statistical learning and inference—focusing on connecting theory with real-world applications and hands-on practice. While expanding our methodological toolkit, we will simultaneously introduce critical perspectives to examine the ethics of ML within sociotechnical systems. This book lays the foundation for advanced study and research in ML. Topics include: directed graphical models, deep Bayesian regression/classification, generative models (latent variable models) for clustering, dimensionality reduction, and time-series forecasting. The book focuses on hands-on experience building models for specific tasks, most taken from healthcare contexts, using NumPyro, a Python-based probabilistic programming language.
Goals: Our goal is that at the end of this book, you will be able to,
Formalize assumptions about an observed data-set into a (potentially novel) probabilistic model.
Perform statistical inference for statistical models in a probabilistic programming language.
Evaluate models in the context of their downstream task by relating human-useful properties to metrics (and understanding their strengths/shortcomings).
Abstractly reason/intuit about properties of models and different types of inference methods.
Evaluate ML systems in their sociotechnical context.
Apply critical lenses to the development and applications of current ML systems.