Write code to compute confusion matrix and evaluation metrics.

What you need to know

A confusion matrix counts how often each actual class was predicted as each class. With rows = actual and columns = predicted, the diagonal holds the correct predictions and everything else is a specific kind of mistake.

For one class c:

TP (true positive)  = matrix[c][c]              predicted c, was cFP (false positive) = column c total − TP       predicted c, was something elseFN (false negative) = row c total − TP          was c, predicted something elseprecision = TP / (TP + FP)     "when I say c, how often am I right?"recall    = TP / (TP + FN)     "of all real c, how many did I catch?"F1        = 2 × precision × recall / (precision + recall)

F1 is the harmonic mean: it is high only when both precision and recall are high.

Macro vs micro. Macro-averaging takes the plain mean of the per-class scores, so a rare class counts as much as a common one. Micro-averaging pools all TP, FP and FN first, so it is dominated by common classes (for single-label problems, micro F1 equals accuracy). Always say which you report.

1from collections.abc import Sequence23def confusion_matrix(y_true: Sequence, y_pred: Sequence,4                     labels: list | None = None) -> tuple[list, list[list[int]]]:5    """Rows are actual classes, columns are predicted classes."""6    if len(y_true) != len(y_pred):7        raise ValueError("y_true and y_pred must be the same length")8    labels = labels or sorted(set(y_true) | set(y_pred))9    index = {label: i for i, label in enumerate(labels)}10    matrix = [[0] * len(labels) for _ in labels]11    for actual, predicted in zip(y_true, y_pred):12        matrix[index[actual]][index[predicted]] += 113    return labels, matrix1415def metrics_from_matrix(labels: list, matrix: list[list[int]]) -> dict:16    k = len(labels)17    total = sum(map(sum, matrix))18    per_class = {}19    for i, label in enumerate(labels):20        tp = matrix[i][i]21        fp = sum(matrix[r][i] for r in range(k)) - tp       # column total minus TP22        fn = sum(matrix[i]) - tp                            # row total minus TP23        p = tp / (tp + fp) if tp + fp else 0.024        r = tp / (tp + fn) if tp + fn else 0.025        f1 = 2 * p * r / (p + r) if p + r else 0.026        per_class[label] = {"precision": round(p, 3), "recall": round(r, 3),27                            "f1": round(f1, 3), "support": tp + fn}28    macro = {m: round(sum(c[m] for c in per_class.values()) / k, 3) if k else 0.029             for m in ("precision", "recall", "f1")}30    accuracy = sum(matrix[i][i] for i in range(k)) / total if total else 0.031    return {"accuracy": round(accuracy, 3), "macro": macro, "per_class": per_class}

The tricky parts:

• [[0] * k for _ in labels], not [[0] * k] * k. The second creates k references to one list, so incrementing one row increments them all.
• The union of labels in sorted(set(y_true) | set(y_pred)), so a class that was predicted but never actually present still gets a column (and a precision of 0).
• Zero guards. A class never predicted has TP + FP = 0; returning 0.0 matches scikit-learn's default (which also prints a warning).
• Macro averages here are averaged from rounded per-class values, which can differ from scikit-learn in the third decimal; average the unrounded values if you need exact agreement.

Complexity: building the matrix is O(n) for n samples plus O(k²) to allocate it. The metrics loop is O(k²) because each class sums a column. Space O(k²).

A real-life example

Ten support tickets, three classes:

1y_true = ["bill", "bill", "bill", "bill", "deliv", "deliv", "deliv", "acct", "acct", "acct"]2y_pred = ["bill", "bill", "deliv", "bill", "deliv", "deliv", "bill", "acct", "acct", "deliv"]3labels, m = confusion_matrix(y_true, y_pred)4print(labels, m)5# ['acct', 'bill', 'deliv'] [[2, 0, 1], [0, 3, 1], [0, 1, 2]]6report = metrics_from_matrix(labels, m)7print(report["accuracy"], report["macro"])8# 0.7 {'precision': 0.75, 'recall': 0.695, 'f1': 0.707}9print(report["per_class"]["deliv"])10# {'precision': 0.5, 'recall': 0.667, 'f1': 0.571, 'support': 3}

For deliv: TP = 2, FP = column 4 − 2 = 2, FN = row 3 − 2 = 1. Precision 2/4 = 0.5, recall 2/3 = 0.667, F1 = 2 × 0.5 × 0.667 / 1.167 = 0.571. Accuracy is the diagonal, 7 of 10.

The matrix also says what goes wrong: the model sends one account ticket and one billing ticket to delivery. That is the row a support lead would look at before retraining.

Fraud teams at payment companies read these matrices daily, because a false negative (missed fraud) and a false positive (a blocked genuine payment) have very different costs.

Follow-up questions to expect

• "How do you check your implementation?" — Compare with sklearn.metrics.confusion_matrix and precision_recall_fscore_support on a few thousand random label pairs; they should match exactly (before rounding).
• "Precision or recall — which matters more?" — It depends on the cost of each error. For fraud alerts to a human, recall; for auto-blocking payments, precision. Use F-beta to weight one over the other.
• "What is a good metric for imbalanced binary problems?" — Precision-recall curves and average precision, rather than ROC AUC, which can look good even when the rare class is handled badly.