Machine Learning

What Is Machine Learning?

The Machine Learning Pipeline

Key Concepts and Terminology

Machine Learning Success Factors

What Is Machine Learning: The Foundation of Intelligent Systems

The Machine Learning Landscape

When to Use Machine Learning

Vectors and Vector Spaces: The Language of Machine Learning

Eigenvalues and Eigenvectors

Matrix Decompositions

Projections and Least Squares

Linear Transformation of a Vector

Pairwise Matrix Product Computation

Computing the Multiplicative Inverse of a 2×2 Matrix

Matrices and Linear Transformations

Norms and Distance Metrics

Compute Arithmetic Mean Along Matrix Axis

Feature Covariance Computation

Differential Calculus Review

Multivariate Calculus

Convex Optimization

Gradient Descent

Constrained Optimization

Advanced Optimization Methods

Composite Function Differentiation via Chain Rule

Gradient Vector Computation for Scalar Fields

Probability Fundamentals

Random Variables and Distributions

Common Probability Distributions

Joint and Marginal Distributions

Estimation Theory

Information Theory for Machine Learning

Population Mean Confidence Estimator

Independent Samples Mean Comparison

The Learning Problem

PAC Learning: Probably Approximately Correct Framework

VC Dimension and Model Complexity

The Bias-Variance Tradeoff

Regularization Theory

Generalization Bounds: Mathematical Guarantees for Learning

Regression Diagnostics

Computational Considerations in Linear Regression

Simple Linear Regression

Multiple Linear Regression

Statistical Properties of OLS

Maximum Likelihood View of Linear Regression

Closed-Form Linear Model Fitting

Optimizing Linear Model Coefficients via Gradient Descent

The Overfitting Problem

Ridge Regression (L2 Regularization)

Lasso Regression (L1 Regularization)

Elastic Net Regularization

Bayesian Interpretation of Regularization

Advanced Regularization Techniques

Kernel Ridge Regression

Gaussian Processes Introduction

The Kernel Trick

Gaussian Process Kernels and Hyperparameter Optimization

Computational Aspects of Kernels

Kernel Selection and Design

Kernel-Based Support Vector Classifier Training

Polynomial Regression

Spline Regression Methods

Generalized Additive Models (GAMs)

Quantile Regression

Robust Regression Methods

Nonparametric Regression

Regularized Logistic Regression

Binary Classification Problem

The Logistic Regression Model

Maximum Likelihood Estimation for Logistic Regression

Multi-class Logistic Regression

Generalized Linear Models (GLMs)

Logistic Sigmoid Transformation

Single Neuron Binary Classifier with Sigmoid Activation

Decision Tree Fundamentals

Splitting Criteria

Tree Growing Algorithms

Pruning and Regularization

Regression Trees

Tree Limitations and Extensions

Recursive Classification Tree Builder

Maximum Margin Classifier

Hard Margin SVM

Soft Margin SVM

Kernel Support Vector Machines

SVM Optimization

Multi-class Support Vector Machines

Gaussian Naive Bayes

Linear and Quadratic Discriminant Analysis

Generative vs Discriminative Models

The Bayes Classifier

The Naive Bayes Assumption

Multinomial and Bernoulli Naive Bayes

KNN Algorithm

Distance Metrics for K-Nearest Neighbors

Curse of Dimensionality

Efficient K-Nearest Neighbors

Voronoi Diagrams & Decision Surfaces

Locality-Sensitive Hashing (LSH)

Weighted KNN

KNN Variants

Ensemble Learning Fundamentals

Random Forests

Bagging for Other Models

Bootstrap Aggregating: Variance Reduction Through Ensemble Averaging

Feature Importance

Random Forest Variants

Boosting Intuition

AdaBoost: Adaptive Boosting

Boosting as Gradient Descent

Gradient Boosting

Loss Functions for Boosting

Regularization in Boosting

XGBoost: Extreme Gradient Boosting

LightGBM: Light Gradient Boosting Machine

CatBoost

Hyperparameter Tuning for Boosting

Feature Engineering for Boosting

Boosting Best Practices

Classification Metrics

ROC and Precision-Recall Curves

Multi-class Classification Metrics

Regression Metrics

Ranking Metrics

Custom and Business Metrics

Softmax Probability Distribution

Holdout Validation

Stratified and Group Cross-Validation

Bootstrap Methods

K-Fold Cross-Validation

Time Series Cross-Validation

Nested Cross-Validation

Data Partitioning with Cross-Validation Splits

Hyperparameter Fundamentals

Grid Search

Random Search

Bayesian Optimization

Multi-Fidelity Optimization

Advanced HPO Topics

Clustering Problem Formulation

K-Means Clustering

K-Means Variants: Beyond Standard K-Means Clustering

Hierarchical Clustering

Spectral Clustering

Clustering Evaluation

Centroid-Based Data Partitioning

Density Estimation Fundamentals

Kernel Density Estimation Theory

Gaussian Mixture Models

The Expectation-Maximization Algorithm

Beyond Gaussian Mixtures

Density-Based Clustering

Dimensionality Reduction Motivation

Principal Component Analysis Theory

PCA Variants

Kernel PCA

Factor Analysis

Independent Component Analysis

Principal Component Extraction

Manifold Hypothesis

Multidimensional Scaling

Isomap: Isometric Feature Mapping

Locally Linear Embedding

t-Distributed Stochastic Neighbor Embedding (t-SNE)

UMAP (Uniform Manifold Approximation and Projection)

Anomaly Detection Fundamentals

Distance-Based Methods for Anomaly Detection

Isolation Forest for Anomaly Detection

Statistical Methods for Anomaly Detection

Density-Based Methods for Anomaly Detection

One-Class Methods for Anomaly Detection

The Bayesian Framework

Conjugate Priors

Bayesian Linear Regression

Bayesian Model Comparison

Approximate Inference: Sampling Methods

Approximate Inference: Deterministic Methods

Posterior Parameter Estimation with Conjugate Prior

Probabilistic Graphical Models

Bayesian Networks

Markov Random Fields

Inference in Graphical Models

Hidden Markov Models

Conditional Random Fields

Gaussian Process Fundamentals

Gaussian Process Regression

Kernel Design for Gaussian Processes

Hyperparameter Learning in Gaussian Processes

Scalable Gaussian Processes

Gaussian Process Classification

Variational Inference Framework

Mean-Field Approximation

ELBO Optimization

Stochastic Variational Inference

Variational Autoencoders

Advanced VI Topics

Biological Inspiration

Multi-Layer Perceptrons

Universal Approximation

Activation Functions

Loss Functions

Output Layers

Activation Function Gradient Computation

Backpropagation Algorithm

Gradient Descent Variants

Adaptive Optimizers

Learning Rate Schedules: Orchestrating Optimization Dynamics

Computational Graphs

Second-Order Optimization Methods

Training a Single Neuron with Gradient Descent

Gradient Computation Engine (Basic Autograd)

Capacity and Generalization

Weight Regularization

Dropout

Batch Normalization

Data Augmentation: Expanding Training Distributions

Implicit Regularization in Deep Learning

The Convolution Operation

Convolutional Layers

Pooling and Downsampling

CNN Architectures: From LeNet to Modern Designs

Residual Networks and Skip Connections

Modern CNN Architectures

2D Convolution Operation

Sequence Modeling

Recurrent Neural Network Architecture

Vanishing and Exploding Gradients

Long Short-Term Memory

Gated Recurrent Units

Advanced RNN Topics

LSTM Memory Network: Gated Recurrent Memory Cell

Attention Mechanism

Self-Attention

Multi-Head Attention

Transformer Architecture

Transformer Variants

Self-Attention Weighted Aggregation

Positional Encoding

Generative Model Fundamentals

Generative Adversarial Networks

GAN Variants

Flow-Based Models

Diffusion Models

The Label Scarcity Problem

Semi-Supervised Methods

Consistency Regularization

Self-Supervised Learning

Contrastive Learning

Modern Self-Supervised Methods

Multi-Task Learning

Transfer Learning Fundamentals

Feature-Based Transfer

Fine-Tuning Strategies

Domain Adaptation

Meta-Learning: Learning to Learn

RL Fundamentals

Markov Decision Processes

Value-Based Methods

Policy-Based Methods

Deep Reinforcement Learning

RL Applications and Challenges

Intra-Group Advantage Normalization for Policy Training

Unbiased Policy Divergence Estimation for Regularized Optimization

Recommender System Fundamentals

Collaborative Filtering

Matrix Factorization for Recommendations

Content-Based Methods

Deep Learning for Recommendations

Production Considerations

Feature Engineering Mastery

Feature Selection Theory

Text Feature Engineering: N-grams

Data Normalization Techniques: Z-Score and Min-Max Scaling

Categorical Binary Encoding

Ordered Polynomial Feature Generator

Automated Feature Engineering

Feature Stores: The Foundation of Production ML Feature Management

Feature Transformation Pipelines

High-Cardinality Features

Text Feature Engineering - Basics

Text Feature Engineering: TF-IDF

Text Preprocessing: Transforming Raw Text for Machine Learning

Word Embeddings

AutoML Overview

Automated Preprocessing

Automated Model Selection

Neural Architecture Search

AutoML Systems

AutoML Best Practices

Feature Attribution Methods

Interpretability Fundamentals

Model-Specific Interpretability

Fairness in Machine Learning

Bias Detection and Mitigation

Practical Interpretability

Foundation Models

Large Language Models

Multimodal Learning: Bridging Vision, Language, and Beyond

Federated Learning

Continual Learning: Learning Without Forgetting

Emerging Directions in Machine Learning

ML System Design

Debugging ML Models

ML Project Management

Reading Research Papers

ML Interviews: Complete Preparation Guide

Career Development in Machine Learning

Dataset Batch Generator for Training Pipelines

Inference Performance Metrics Calculator

Inference Request Batching

Synchronized Data Permutation

One bad dataset row consumes several quality allowances

Count bad dataset rows once

Published datasets differ from the snapshot that passed review

Publish the rows captured by the quality run