Learning path
Machine learning (ML)
Learn how models are evaluated and trained: metrics, linear and logistic regression, k-nearest neighbors, decision trees, clustering, and validation.
About this subject
Machine learning builds models from data rather than from hand-written rules. In supervised learning the data have labels and the goal is to predict them for new inputs. In unsupervised learning, such as clustering, the goal is to find structure without labels. Judging a model on data it has not seen is as important as training it.
This path covers evaluation metrics, linear regression with gradient descent, logistic regression, k-nearest neighbors and feature scaling, decision trees and information gain, k-means clustering, and overfitting with cross-validation. The lessons implement the algorithms from scratch on tiny data so every step can be inspected. Each lesson uses a small, complete program that runs on its own. Python has the full sequence. The other languages listed below add lessons that use that language's own idioms and standard library, such as Java threads, C++ smart pointers, C# LINQ, or Node.js streams, and appear only for the topics the language supports. The programs are simplified models chosen to make an idea visible, so they are a foundation for further study rather than a replacement for a full course or for production-grade software.
Before you start: Lists or arrays and functions in your chosen language, and basic algebra. Calculus helps for the gradient lessons.
- 1
Python
Readable code for automation, data, and first programs.
Evaluating classifiers: accuracy, precision, recall, F1
Build a confusion matrix and see why accuracy alone can be misleading for rare classes.
Linear regression with gradient descent
Fit a line by repeatedly stepping the parameters downhill on the squared error, and check it against the closed form.
Logistic regression for classification
Turn a linear score into a probability with the sigmoid and train it by gradient descent on log loss.
k-nearest neighbors and feature scaling
Classify by distance to stored examples, and see how an unscaled feature can dominate the result.
Decision trees: entropy and information gain
Choose the feature that best splits the data by measuring how much it reduces uncertainty.
k-means clustering
Group unlabelled points by alternating between assigning points to centroids and moving the centroids.
Overfitting and cross-validation
Compare training accuracy with leave-one-out validation accuracy to choose k and detect memorization.
- 2
C
Learn memory, pointers, and efficient systems programming.
- 3
C++
Build fast applications with modern types and classes.
- 4
Java
Understand object oriented programs and the JVM.
- 5
C#
Build typed applications on .NET.
- 6
- 7
TypeScript
JavaScript with static types for dependable code.