Complete ML and AI Tutorial

Machine learning has an unusual failure mode: the code runs, the metric looks good, and the model is wrong. There is no exception, no stack trace, nothing to grep for. A leaked feature, a validation split that does not match how the model will be used, a metric that rewards predicting the majority class — each produces a number that looks like success.

That is why the discipline in this track is weighted toward evaluation and data handling rather than toward model architectures. Choosing between gradient boosting and a neural network is a smaller decision than most people expect. Whether your validation strategy reflects reality is the one that determines whether anything you built works in production.

Data leakage: the bug that hides inside a good score

Leakage is any situation where information from outside the training set reaches the model. It inflates validation scores and collapses in production, and it is the single most common serious error in applied machine learning.

The forms are specific enough to check for deliberately. Fitting a scaler or an imputer on the full dataset before splitting leaks the test set's distribution. A random split on time-series data lets the model train on the future. A feature computed after the outcome — 'number of support tickets' when predicting churn — encodes the answer. Duplicate rows spanning the split leak directly.

# Leaks: the scaler has seen the test set's distribution
X = StandardScaler().fit_transform(X_all)
X_tr, X_te = train_test_split(X, ...)

# Correct: every transform is fitted inside the pipeline, on the
# training fold only, at every fold of cross-validation
pipe = Pipeline([
    ("impute", SimpleImputer(strategy="median")),
    ("scale",  StandardScaler()),
    ("model",  GradientBoostingClassifier()),
])
scores = cross_val_score(pipe, X_all, y, cv=5)

# Time series: never shuffle. Train on the past, validate on the future.
for tr, te in TimeSeriesSplit(n_splits=5).split(X):
    ...

Metrics: accuracy is usually the wrong one

On a dataset where 99% of cases are negative, a model that predicts 'negative' every time scores 99% accuracy and is worthless. Almost every real classification problem is imbalanced in some direction, so the metric must reflect which error costs more.

State that cost before choosing. If a false negative is expensive — a missed fraud, a missed diagnosis — recall dominates. If a false positive is expensive — blocking a legitimate customer, paging a human — precision dominates. F1 balances them and hides which one you are trading.

And remember the threshold is yours to choose. A classifier outputs a probability; converting it to a decision at 0.5 is a default, not a law. Tuning that threshold against the actual cost of each error type is often a larger improvement than changing the model.

Underfitting, overfitting, and what to do about each

Diagnose before treating, using two numbers: training error and validation error. High training error means the model cannot represent the pattern — it is underfitting, and more data will not help. Low training error with high validation error means it memorised — it is overfitting, and more data usually does help.

The treatments are opposite, which is why the diagnosis matters. Underfitting wants more capacity, better features, less regularisation, longer training. Overfitting wants more data, more regularisation, fewer features, early stopping, simpler models. Applying the wrong one makes things worse in a way that is easy to misread as noise.

The model matters less than the pipeline around it

For tabular data — which is most business machine learning — gradient-boosted trees remain the strong default, and deep learning rarely beats them at the sizes most organisations have. Deep learning dominates unstructured data: images, audio, text. That single distinction settles most architecture debates before they start.

What actually determines whether a project succeeds is the surrounding machinery: whether features are computed identically at training and serving time, whether the data version is recorded, whether you can reproduce a result from six weeks ago, and whether anything monitors the input distribution for drift. A model that cannot be reproduced is a result, not a system.