Introduction to FastAPI

Open a mature Flask prediction service and count the lines that do the actual work.

There is one line calling model.predict_proba. Above it sit sixty lines checking that every field is present, that each is the right type, that numbers fall in plausible ranges, that categories are recognised, and that True has not been accepted as the integer 1. Below it sit another dozen assembling a response dictionary.

Then somebody asks what the endpoint accepts. You point at a README written four months ago, which lists a field that was renamed in February and omits two that were added in March. Nobody updated it because updating it is a separate act of discipline from changing the code.

Both problems have the same shape. The information — which fields exist, what types they are, what ranges are valid — is already in your head and already implied by the code. It is just written out three times: once as validation, once as documentation, and once in whatever the model actually expects. Three copies of one truth, drifting apart.

FastAPI's premise is that you write it once, as Python type hints, and the framework derives the rest.

Type hints that do something

Ordinary Python type hints are annotations. They document intent and are checked by external tools, but at runtime they are ignored:

1def predict(income: float, age: int) -> float:2    ...34predict("not a number", "also not")   # runs fine until something breaks

FastAPI, via the Pydantic library, reads those hints at runtime and enforces them. You declare the shape of your input as a class:

1from pydantic import BaseModel, Field2from typing import Literal34class CustomerFeatures(BaseModel):5    income: float = Field(ge=0, le=1_000_000,6                          description="Annual income in GBP")7    age: int = Field(ge=18, le=100)8    tenure_years: float = Field(ge=0, le=60)9    num_products: int = Field(ge=1, le=10)10    region: Literal["north", "south", "east", "west"]11    has_contract: bool

That class replaces the entire hand-written validator. Every constraint in the sixty lines is now one declaration, and — this is the part that changes how the service feels — the same declaration produces the API documentation.

What the constraints mean

For rules that do not fit a constraint, drop into a validator:

1from pydantic import field_validator, model_validator23class CustomerFeatures(BaseModel):4    income: float = Field(ge=0, le=1_000_000)5    age: int = Field(ge=18, le=100)6    tenure_years: float = Field(ge=0, le=60)78    @model_validator(mode="after")9    def tenure_must_fit_age(self):10        if self.tenure_years > self.age - 18:11            raise ValueError(12                f"tenure_years ({self.tenure_years}) implies the customer "13                f"joined before turning 18"14            )15        return self

That cross-field check is exactly the kind of rule that catches genuinely corrupt upstream data — a record claiming 40 years of tenure for a 25-year-old — and it is the kind that nobody writes when validation is tedious.

A complete service

1from contextlib import asynccontextmanager2from fastapi import FastAPI, HTTPException3from pydantic import BaseModel, Field4from typing import Literal5import pandas as pd6import joblib78MODEL_VERSION = "2.3.0"9THRESHOLD = 0.3410FEATURES = ["income", "age", "tenure_years", "num_products",11            "region", "has_contract"]1213state = {}1415@asynccontextmanager16async def lifespan(app: FastAPI):17    state["model"] = joblib.load("pipeline.joblib")   # once, at startup18    yield19    state.clear()                                     # on shutdown2021app = FastAPI(22    title="Churn Prediction API",23    description="Estimates the probability that a customer will churn.",24    version=MODEL_VERSION,25    lifespan=lifespan,26)2728class CustomerFeatures(BaseModel):29    income: float = Field(ge=0, le=1_000_000, examples=[52000])30    age: int = Field(ge=18, le=100, examples=[34])31    tenure_years: float = Field(ge=0, le=60, examples=[2.5])32    num_products: int = Field(ge=1, le=10, examples=[2])33    region: Literal["north", "south", "east", "west"]34    has_contract: bool3536class Prediction(BaseModel):37    probability: float = Field(ge=0, le=1)38    will_churn: bool39    threshold: float40    model_version: str4142@app.get("/health")43def health() -> dict:44    return {"status": "ok", "model_version": MODEL_VERSION,45            "model_loaded": "model" in state}4647@app.post("/predict", response_model=Prediction)48def predict(features: CustomerFeatures) -> Prediction:49    row = pd.DataFrame([features.model_dump()])[FEATURES]50    try:51        prob = float(state["model"].predict_proba(row)[0, 1])52    except Exception as exc:53        raise HTTPException(status_code=500,54                            detail="prediction failed") from exc55    return Prediction(56        probability=round(prob, 4),57        will_churn=prob >= THRESHOLD,58        threshold=THRESHOLD,59        model_version=MODEL_VERSION,60    )

Run it with uvicorn app:app --reload.

Notice what is absent. There is no code checking whether income is present, whether it is a number, or whether it is negative. There is no code parsing JSON. The function signature says it takes a CustomerFeatures, and by the time the body runs, that is what it has — validated, coerced, and typed.

What a bad request gets back

1curl -X POST http://localhost:8000/predict \2  -H "Content-Type: application/json" \3  -d '{"income": -500, "age": 34, "tenure_years": 2.5,4       "num_products": 2, "region": "atlantis", "has_contract": true}'

1{2  "detail": [3    {4      "type": "greater_than_equal",5      "loc": ["body", "income"],6      "msg": "Input should be greater than or equal to 0",7      "input": -500,8      "ctx": {"ge": 0.0}9    },10    {11      "type": "literal_error",12      "loc": ["body", "region"],13      "msg": "Input should be 'north', 'south', 'east' or 'west'",14      "input": "atlantis",15      "ctx": {"expected": "'north', 'south', 'east' or 'west'"}16    }17  ]18}

Status 422, both errors reported at once, each naming the offending field and the value received. That response is more useful than what most hand-written validators produce, and you wrote none of it.

Documentation that cannot go stale

Visit /docs and there is a browsable interface listing every endpoint, every field, its type, its constraints, and its example — with a button that sends a real request and shows the real response. Visit /openapi.json and there is a machine-readable specification that client-generation tools consume directly.

Neither was written by hand. Both are derived from the same class that performs validation, which means they cannot describe a field the code does not have. Rename tenure_years and the docs rename it in the same commit, because they are the same statement.

The value is not that FastAPI generates documentation. It is that the documentation and the validation are the same declaration, so they cannot disagree.

Declaring the response too

response_model=Prediction is easy to skip and worth keeping. It does three things: it documents the output shape, it validates that your handler actually returned that shape, and it filters out anything not declared.

That third behaviour is a security property. If a future change makes the handler return an internal object carrying a customer ID or a raw feature vector, the response model strips it rather than leaking it to the caller.

Path and query parameters

The same typing applies to values in the URL.

1from fastapi import Query, Path23@app.get("/customers/{customer_id}/risk")4def customer_risk(5    customer_id: int = Path(ge=1, description="Internal customer ID"),6    explain: bool = Query(False, description="Include feature contributions"),7    top_k: int = Query(5, ge=1, le=20),8) -> dict:9    ...

A request to /customers/abc/risk returns a 422 explaining that customer_id must be an integer, before the function runs. Path parameters identify a resource; query parameters modify how it is returned. Keeping to that convention makes an API predictable to people who have never read its docs.

async def: what it buys, and what it does not

FastAPI supports asynchronous handlers, and this is the most commonly misapplied feature in the framework.

Asynchronous code helps when a handler spends its time waiting — for a database, an HTTP call, a file read. While one request waits, the event loop serves others. It does nothing for work that keeps the CPU busy.

model.predict_proba is CPU-bound. It does not wait; it computes. So:

The counter-intuitive conclusion: for a scikit-learn endpoint, plain def is the right choice and async def makes throughput worse. Writing async def because it sounds faster is a reliable way to build a service that serialises every request behind the slowest one.

Where async def earns its place is a handler that fetches features from a database or calls another service before predicting — the awaits release the loop while the network does its work.

Batching, which matters more than async

The single largest throughput win for a prediction API is usually not concurrency but batching. Vectorised libraries predict 500 rows in barely more time than one, because the per-call overhead dominates.

1class BatchRequest(BaseModel):2    items: list[CustomerFeatures] = Field(min_length=1, max_length=1000)34class BatchResponse(BaseModel):5    predictions: list[Prediction]6    count: int78@app.post("/predict/batch", response_model=BatchResponse)9def predict_batch(payload: BatchRequest) -> BatchResponse:10    frame = pd.DataFrame([item.model_dump() for item in payload.items])[FEATURES]11    probs = state["model"].predict_proba(frame)[:, 1]12    return BatchResponse(13        predictions=[14            Prediction(probability=round(float(p), 4),15                       will_churn=bool(p >= THRESHOLD),16                       threshold=THRESHOLD,17                       model_version=MODEL_VERSION)18            for p in probs19        ],20        count=len(probs),21    )

Every item is validated individually, and max_length=1000 caps the damage from a caller who sends a million rows. That cap is not optional — an unbounded batch endpoint is a memory-exhaustion vector.

Testing

1import pytest2from fastapi.testclient import TestClient3from app import app45@pytest.fixture(scope="module")6def client():7    with TestClient(app) as c:    # the with-block runs lifespan startup, so the model loads8        yield c910def test_health(client):11    r = client.get("/health")12    assert r.status_code == 20013    assert r.json()["model_loaded"] is True1415def test_valid_prediction(client):16    r = client.post("/predict", json={17        "income": 52000, "age": 34, "tenure_years": 2.5,18        "num_products": 2, "region": "north", "has_contract": True,19    })20    assert r.status_code == 20021    body = r.json()22    assert 0.0 <= body["probability"] <= 1.023    assert body["model_version"] == "2.3.0"2425def test_rejects_negative_income(client):26    r = client.post("/predict", json={27        "income": -1, "age": 34, "tenure_years": 2.5,28        "num_products": 2, "region": "north", "has_contract": True,29    })30    assert r.status_code == 42231    assert r.json()["detail"][0]["loc"] == ["body", "income"]3233def test_rejects_unknown_region(client):34    r = client.post("/predict", json={35        "income": 52000, "age": 34, "tenure_years": 2.5,36        "num_products": 2, "region": "atlantis", "has_contract": True,37    })38    assert r.status_code == 422

TestClient makes real requests through the full application stack without starting a server, so the tests exercise validation, serialisation, and the handler together. There is no separate mocking layer to keep in sync. Note the with block: a TestClient created without it never runs the lifespan handler, so the model is never loaded and every prediction test fails with a 500.

Worth adding alongside these: a test asserting the model's output on a fixed input matches a recorded value. That one catches library upgrades silently changing predictions, which no amount of schema validation will notice.

Running it properly

1# development2uvicorn app:app --reload34# production: several worker processes, managed by uvicorn itself5uvicorn app:app --host 0.0.0.0 --port 8000 --workers 4

1# Dockerfile2FROM python:3.12-slim3WORKDIR /app4COPY requirements.txt .5RUN pip install --no-cache-dir -r requirements.txt6COPY app.py pipeline.joblib ./7EXPOSE 80008CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

Each worker holds its own copy of the model, so four workers with a 200 MB pipeline need roughly 800 MB before serving anything. Size the worker count against memory, not just cores.

Older guides run FastAPI under gunicorn with --worker-class uvicorn.workers.UvicornWorker. That module is deprecated in current uvicorn; if you want gunicorn as the process manager, install the separate uvicorn-worker package and use --worker-class uvicorn_worker.UvicornWorker.

Flask or FastAPI

For a new prediction service, FastAPI is the better default, mainly because validation is the part of a model API that is most tedious to write, most consequential when skipped, and most likely to be skipped precisely because it is tedious.

What this means when you build something

Define the input schema before writing the handler, and be specific: real ranges, real category lists, real cross-field rules. Every constraint you declare is a class of malformed request that can never reach your model, and a line of documentation you never have to maintain. Writing income: float when you could write income: float = Field(ge=0, le=1_000_000) throws away the main benefit of the framework.

Load the model once in a lifespan handler, keep the whole preprocessing pipeline inside the saved artefact, use plain def for the prediction endpoint, and add a batch route with an explicit cap. Then point whoever asked for the model at /docs and let them try it themselves — which is a considerably better answer than a README and an apology.