Course Content
Deep Learning with TensorFlow and PyTorch
4 sections · 15 lessons
Housing Price Prediction with Deep Learning
An estate agency wants a price estimate for any California district, given eight numbers it already collects: median income, median house age, average rooms per household, average bedrooms, population, average occupancy, latitude and longitude. They will use it to flag listings that look mispriced, so an error of USD 20,000 is tolerable and an error of USD 80,000 is not.
That target is the whole project. Not "build a neural network" — build something whose typical error is small enough to be useful, and know honestly how large that error is.
What follows is a complete build: data inspection, a baseline that is not a neural network, models in both frameworks, diagnosis, tuning, honest evaluation, and a saved artefact that works in a fresh process. The dataset is sklearn.datasets.fetch_california_housing: 20,640 districts, eight features, and a target of median house value in units of USD 100,000.
Look at the data before you model it
Skipping this step is the most expensive shortcut in the project, because this particular dataset has two traps that will silently corrupt your evaluation.
1from sklearn.datasets import fetch_california_housing2import pandas as pd34data = fetch_california_housing(as_frame=True)5df = data.frame6print(df.shape) # (20640, 9)7print(df.describe().T[["mean", "std", "min", "max"]])8print("targets at the cap:", (df["MedHouseVal"] >= 5.0).mean()) # ~4.8%| Feature | Meaning | Range | Note |
|---|---|---|---|
| MedInc | Median income, tens of thousands | 0.5–15.0 | By far the strongest predictor |
| HouseAge | Median age in years | 1–52 | Capped at 52 |
| AveRooms | Rooms per household | 0.8–141.9 | Extreme values from tiny-household districts |
| AveBedrms | Bedrooms per household | 0.3–34.1 | Same problem |
| Population | District population | 3–35,682 | Heavily skewed |
| AveOccup | People per household | 0.7–1243.3 | Clearly erroneous at the top end |
| Latitude / Longitude | District centre | 32.5–42.0 / −124.3–−114.3 | Together they encode location |
| MedHouseVal | Target, hundreds of thousands | 0.15–5.00 | Censored: ~4.8% sit exactly at 5.0 |
Trap one: the target is censored. Roughly one district in twenty has a value of exactly 5.00001, because the original survey capped it. Those are not districts worth exactly USD 500,000; they are districts worth USD 500,000 or more. No model can predict them correctly, and their errors will dominate your RMSE. Report your metrics both with and without them, and be explicit about which number you are quoting.
Trap two: a handful of impossible feature values. An AveOccup of 1,243 people per household is a data error, not a crowded district. A single row like that, once standardised, becomes a feature value of about 120 standard deviations — and with MSE loss, that one row can supply more gradient than several hundred normal ones.
1import numpy as np23# Clip the pathological tail rather than deleting rows -- keeps the sample size4for col, hi in [("AveRooms", 20), ("AveBedrms", 5), ("AveOccup", 10)]:5 df[col] = df[col].clip(upper=hi)67# Two features that carry real signal and are trivial to add8df["rooms_per_person"] = df["AveRooms"] / df["AveOccup"]9df["bedrooms_ratio"] = df["AveBedrms"] / df["AveRooms"]Split first, then fit the preprocessing
The order here is not stylistic. Fitting a scaler on all the data before splitting leaks the test set's statistics into training and inflates every number you report.
1from sklearn.model_selection import train_test_split2from sklearn.preprocessing import StandardScaler34X = df.drop(columns=["MedHouseVal"]).values.astype("float32")5y = df["MedHouseVal"].values.astype("float32")67X_tmp, X_test, y_tmp, y_test = train_test_split(X, y, test_size=0.15,8 random_state=42)9X_train, X_val, y_train, y_val = train_test_split(X_tmp, y_tmp, test_size=0.176,10 random_state=42)11# -> 70% train, 15% validation, 15% test1213scaler = StandardScaler().fit(X_train) # statistics from TRAINING ONLY14X_train = scaler.transform(X_train)15X_val = scaler.transform(X_val) # transform, never fit again16X_test = scaler.transform(X_test)1718print(X_train.shape, X_val.shape, X_test.shape) # (14456, 10) (3088, 10) (3096, 10)Standardising the inputs is not optional for a neural network here. Population ranges to 35,682 while AveBedrms hovers around 1. Without scaling, the first layer's gradients are dominated entirely by population, and the learning rate that suits one feature is catastrophically wrong for the other.
The target is left in its natural units of USD 100,000, which conveniently means a loss value in MSE is roughly interpretable and an MAE of 0.33 reads directly as "typically off by USD 33,000".
Build a baseline you have to beat
Before any deep learning, establish what a simple model achieves. Without this you have no idea whether your network is good or merely functional.
1from sklearn.linear_model import LinearRegression2from sklearn.ensemble import HistGradientBoostingRegressor3from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score45def report(name, y_true, y_pred):6 rmse = mean_squared_error(y_true, y_pred) ** 0.57 print(f"{name:22s} MAE={mean_absolute_error(y_true, y_pred):.3f} "8 f"RMSE={rmse:.3f} R2={r2_score(y_true, y_pred):.3f}")910report("mean predictor", y_val, np.full_like(y_val, y_train.mean()))11report("linear regression", y_val, LinearRegression().fit(X_train, y_train).predict(X_val))12report("gradient boosting", y_val,13 HistGradientBoostingRegressor(max_iter=400).fit(X_train, y_train).predict(X_val))| Baseline | MAE | RMSE | R2 |
|---|---|---|---|
| Always predict the mean | ~0.91 | ~1.15 | 0.00 |
| Linear regression | ~0.48 | ~0.66 | ~0.67 |
| Gradient boosting | ~0.30 | ~0.47 | ~0.84 |
Notice the gradient boosting row. On tabular data of this size, boosted trees are extremely strong, and a neural network that matches them is doing well. If your network lands at R2=0.65, the problem is your network, and knowing that immediately is worth the sixty seconds the baseline costs.
The TensorFlow implementation
1import tensorflow as tf2from tensorflow import keras3from tensorflow.keras import layers45keras.utils.set_random_seed(42)67def build_tf_model(width=128, depth=3, dropout=0.2, l2=1e-4, lr=1e-3):8 reg = keras.regularizers.l2(l2)9 model = keras.Sequential([keras.layers.Input(shape=(X_train.shape[1],))])10 units = width11 for _ in range(depth):12 model.add(layers.Dense(units, use_bias=False, kernel_regularizer=reg))13 model.add(layers.BatchNormalization())14 model.add(layers.Activation("relu"))15 model.add(layers.Dropout(dropout))16 units = max(32, units // 2) # taper: 128 -> 64 -> 3217 model.add(layers.Dense(1)) # linear output for regression18 model.compile(optimizer=keras.optimizers.AdamW(lr, weight_decay=1e-2),19 loss=keras.losses.Huber(delta=1.0),20 metrics=["mae"])21 return model2223model = build_tf_model()24history = model.fit(25 X_train, y_train,26 validation_data=(X_val, y_val),27 epochs=300, batch_size=64, verbose=0,28 callbacks=[29 keras.callbacks.EarlyStopping(patience=20, restore_best_weights=True),30 keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=8, min_lr=1e-6),31 keras.callbacks.ModelCheckpoint("best.keras", save_best_only=True),32 ])33report("keras MLP", y_val, model.predict(X_val, verbose=0).ravel())Three deliberate choices in there. use_bias=False on the dense layers because the following batch-norm subtracts the mean and makes the bias redundant. Huber loss rather than MSE, because the censored targets at 5.0 produce large unavoidable errors that MSE would let dominate the gradient. And a linear output layer with no activation, because a house price can be any positive number and squashing it would cap what the model can express.
The PyTorch implementation
1import torch2import torch.nn as nn3from torch.utils.data import TensorDataset, DataLoader45torch.manual_seed(42)6device = torch.device("cuda" if torch.cuda.is_available() else "cpu")78def make_loader(X, y, bs, shuffle):9 ds = TensorDataset(torch.tensor(X), torch.tensor(y).unsqueeze(1))10 return DataLoader(ds, batch_size=bs, shuffle=shuffle, drop_last=shuffle)1112train_loader = make_loader(X_train, y_train, 64, True)13val_loader = make_loader(X_val, y_val, 512, False)1415class HousingMLP(nn.Module):16 def __init__(self, n_in, width=128, depth=3, dropout=0.2):17 super().__init__()18 blocks, units = [], width19 for _ in range(depth):20 blocks += [nn.Linear(n_in, units, bias=False),21 nn.BatchNorm1d(units), nn.ReLU(), nn.Dropout(dropout)]22 n_in, units = units, max(32, units // 2)23 blocks.append(nn.Linear(n_in, 1))24 self.net = nn.Sequential(*blocks)2526 def forward(self, x):27 return self.net(x)2829model = HousingMLP(X_train.shape[1]).to(device)30opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-2)31sched = torch.optim.lr_scheduler.ReduceLROnPlateau(opt, factor=0.5, patience=8)32loss_fn = nn.HuberLoss(delta=1.0)3334best, wait, patience = float("inf"), 0, 2035for epoch in range(300):36 model.train()37 for xb, yb in train_loader:38 xb, yb = xb.to(device), yb.to(device)39 opt.zero_grad()40 loss = loss_fn(model(xb), yb)41 loss.backward()42 torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)43 opt.step()4445 model.eval()46 tot, n = 0.0, 047 with torch.no_grad():48 for xb, yb in val_loader:49 xb, yb = xb.to(device), yb.to(device)50 tot += loss_fn(model(xb), yb).item() * xb.size(0)51 n += xb.size(0)52 val = tot / n53 sched.step(val)5455 if val < best - 1e-5:56 best, wait = val, 057 torch.save(model.state_dict(), "best.pt")58 else:59 wait += 160 if wait >= patience:61 print(f"stopped at epoch {epoch}, best val loss {best:.4f}")62 break6364model.load_state_dict(torch.load("best.pt", weights_only=True))65model.eval()The two implementations should land within about 0.01 of each other on MAE. If they do not, the difference is almost always in the preprocessing or the batch size, not in the frameworks.
Diagnose before you tune
Plot training and validation loss and read the shape before changing anything.
| What you see | Diagnosis | Move |
|---|---|---|
| Both flat around 0.25 Huber loss | Underfitting | Width 256, depth 4, dropout 0.1, train longer |
| Train 0.05, validation 0.18 and rising | Overfitting | Dropout 0.3, weight decay 0.05, narrower layers |
| Loss spikes every few epochs | Learning rate slightly high | Start at 3e-4; the scheduler will handle the rest |
Loss is nan in the first epoch | Unclipped feature outlier, or LR far too high | Confirm the clipping ran; drop LR by 10× |
| Val MAE stuck near 0.48 | Model is behaving linearly | Check an activation exists between every pair of dense layers |
That last row is worth checking explicitly. If your network scores almost exactly what linear regression scored, the most likely explanation is that it is linear regression — a missing ReLU collapses the whole stack into one matrix multiply, and nothing errors.
Tune the things that matter
1import numpy as np23rng = np.random.default_rng(0)4results = []5for trial in range(30):6 cfg = {7 "lr": float(10 ** rng.uniform(-4.0, -2.3)), # log scale, always8 "width": int(rng.choice([64, 128, 256, 512])),9 "depth": int(rng.integers(2, 5)),10 "dropout": float(rng.uniform(0.0, 0.4)),11 "batch": int(rng.choice([32, 64, 128, 256])),12 }13 val_mae = train_and_evaluate(cfg) # returns validation MAE14 results.append((val_mae, cfg))15 print(f"{trial:2d} MAE={val_mae:.4f} {cfg}")1617results.sort()18print("best:", results[0])Thirty random trials will beat a hand-tuned guess almost every time, and they beat a grid search of the same size because the learning rate — the parameter that matters most — gets thirty distinct values rather than four. Sample it in log space; the interesting territory spans two orders of magnitude and uniform sampling would put nearly all your trials in the top decade.
Evaluate honestly, once
1with torch.no_grad():2 y_pred = model(torch.tensor(X_test).to(device)).cpu().numpy().ravel()34report("TEST (all rows)", y_test, y_pred)56mask = y_test < 5.0 # exclude censored targets7report("TEST (uncensored)", y_test[mask], y_pred[mask])89# Where are the errors concentrated?10err = np.abs(y_pred - y_test)11for lo, hi in [(0, 1), (1, 2), (2, 3), (3, 4), (4, 5.01)]:12 m = (y_test >= lo) & (y_test < hi)13 print(f"true value {lo}-{hi}: n={m.sum():5d} MAE={err[m].mean():.3f}")| Model | Test MAE | Test RMSE | R2 | In money |
|---|---|---|---|---|
| Linear regression | ~0.49 | ~0.68 | ~0.65 | typically off by USD 49,000 |
| Tuned MLP | 0.31–0.35 | 0.46–0.52 | 0.80–0.84 | typically off by about USD 33,000 |
| Gradient boosting | ~0.30 | ~0.45 | ~0.84 | comparable, trains in seconds |
The per-band breakdown is the part that turns a number into understanding. Error will be smallest below USD 200,000, where most of the data lives, and largest at the top of the range where examples are scarce and the target is censored. If the agency mostly lists expensive properties, your headline MAE is misleading and you should quote the top-band figure instead.
Be honest about the gradient boosting column too. A neural network that ties with a tree ensemble on 20,000 rows of tabular data is a legitimate result, not a failure — but it is also the reason experienced practitioners reach for boosting first on problems shaped like this one.
Save something that actually works elsewhere
1import json, joblib, os2from datetime import datetime34os.makedirs("artifact", exist_ok=True)5torch.save(model.state_dict(), "artifact/weights.pt")6joblib.dump(scaler, "artifact/scaler.joblib") # WITHOUT this, predictions are wrong7json.dump({8 "features": list(df.drop(columns=["MedHouseVal"]).columns), # 8 raw + 2 engineered9 "clip_rules": {"AveRooms": 20, "AveBedrms": 5, "AveOccup": 10},10 "config": best_config,11 "target_units": "USD 100,000",12 "test_mae": float(test_mae),13 "created": datetime.now().isoformat(),14}, open("artifact/metadata.json", "w"), indent=2)Then load it in a completely fresh Python process and check that the test MAE comes out the same. The scaler is the piece people forget, and forgetting it does not raise an error — it simply feeds the model unstandardised inputs and returns confident nonsense. A single assertion catches it.
1def predict_district(raw_features: dict) -> float:2 """End-to-end: the eight raw numbers in, price in dollars out."""3 meta = json.load(open("artifact/metadata.json"))4 r = dict(raw_features)5 for col, hi in meta["clip_rules"].items(): # same clipping as training6 r[col] = min(r[col], hi)7 r["rooms_per_person"] = r["AveRooms"] / r["AveOccup"] # same engineered features8 r["bedrooms_ratio"] = r["AveBedrms"] / r["AveRooms"]9 row = np.array([[r[f] for f in meta["features"]]], dtype="float32")10 x = joblib.load("artifact/scaler.joblib").transform(row)1112 net = HousingMLP(len(meta["features"]), **{k: meta["config"][k]13 for k in ("width", "depth", "dropout")})14 net.load_state_dict(torch.load("artifact/weights.pt", weights_only=True))15 net.eval()16 with torch.no_grad():17 return float(net(torch.tensor(x))[0, 0]) * 100_000Where to take it next
Once the pipeline works end to end, the highest-value extensions are the ones that address the failure modes you measured rather than adding architecture for its own sake.
Treat location properly. Latitude and longitude fed as two raw numbers force the network to learn that "37.7, −122.4" means San Francisco through brute force. Adding distance to the nearest large city, or clustering coordinates into 30 regions and using an embedding, typically buys more than doubling the model's width.
Handle the censored target explicitly. Train a classifier to predict "is this district at the cap" and a regressor for the rest. Reporting two numbers — a price and a probability that the true price exceeds USD 500,000 — is more useful to the agency than one number that is systematically wrong on 5% of listings.
Predict a range, not a point. Train with a quantile loss at the 10th, 50th and 90th percentiles and you get an interval. "Between USD 280,000 and USD 410,000, most likely USD 340,000" is a far more actionable output for someone deciding whether a listing looks mispriced, and it costs only a change of loss function and three output units.
Check stability. Rerun the whole pipeline with five different random seeds and report the spread. If your test MAE varies by 0.03 across seeds, then a tuning improvement of 0.01 was noise, and knowing that will stop you chasing it.