What is random search, and why is it often preferred over grid search?

What you need to know

Instead of a list of values, you give random search a distribution for each hyperparameter — for example "learning rate log-uniformly between 0.01 and 0.5" — and a number of trials, n_iter. Each trial draws one value from every distribution.

Why it wins with the same budget

Suppose there are two hyperparameters, a learning rate that matters a lot and a batch size that barely matters, and you can afford 16 trials.

• Grid (4 × 4) tries only 4 distinct learning rates, each repeated with 4 batch sizes.
• Random tries 16 distinct learning rates, because every trial draws a fresh value.

1import numpy as np23def score(lr, batch):                    # only lr really matters; batch barely does4    return 0.90 - 0.1 * (np.log10(lr) - np.log10(0.03)) ** 2 + 0.001 * np.log2(batch)56rng = np.random.default_rng(0)7grid = [(lr, b) for lr in [0.001, 0.01, 0.1, 1.0] for b in [16, 32, 64, 128]]   # 16 tries8rand = [(10 ** rng.uniform(-3, 0), 2 ** rng.integers(4, 8)) for _ in range(16)]  # 16 tries910for name, tries in [("grid", grid), ("random", rand)]:11    best = max(tries, key=lambda t: score(*t))12    print(f"{name:6s} distinct lr values: {len({t[0] for t in tries}):2d}  "13          f"best lr={best[0]:.3f}  score={score(*best):.3f}")

grid   distinct lr values:  4  best lr=0.010  score=0.884random distinct lr values: 16  best lr=0.042  score=0.905

This is a toy scoring function, not a real model, so the numbers only show the mechanism: the best learning rate here is 0.03, the grid can only get as close as 0.01, and random search lands at 0.042. This argument was made by Bergstra and Bengio in their 2012 paper on random search.

In scikit-learn

1from scipy.stats import loguniform, randint2from sklearn.datasets import make_classification3from sklearn.ensemble import HistGradientBoostingClassifier4from sklearn.model_selection import RandomizedSearchCV56X, y = make_classification(n_samples=3000, n_features=20, n_informative=6,7                           weights=[0.9], random_state=0)89param_distributions = {10    "learning_rate": loguniform(0.01, 0.5),      # sample evenly on a log scale11    "max_leaf_nodes": randint(8, 64),12    "min_samples_leaf": randint(5, 100),13    "l2_regularization": loguniform(1e-4, 10),14}15search = RandomizedSearchCV(HistGradientBoostingClassifier(random_state=0),16                            param_distributions, n_iter=20, cv=5,17                            scoring="roc_auc", random_state=0, n_jobs=-1)18search.fit(X, y)19print("fits run:", 20 * 5)20print("best CV ROC-AUC:", round(search.best_score_, 3))21print({k: round(float(v), 3) if isinstance(v, float) else int(v) for k, v in search.best_params_.items()})

fits run: 100best CV ROC-AUC: 0.926{'l2_regularization': 9.473, 'learning_rate': 0.097, 'max_leaf_nodes': 22, 'min_samples_leaf': 20}

Four hyperparameters, 20 trials, 100 fits in total. A grid with just 5 values each would need 625 combinations — 3,125 fits. loguniform spreads samples evenly across orders of magnitude, which suits rates and strengths. Setting random_state makes the search reproducible.

Beyond random: Bayesian optimisation

Random search does not learn from earlier trials. Bayesian optimisation tools such as Optuna build a model of "which settings scored well so far" and sample more often near promising regions. They also support pruning — stopping trials that are clearly doing badly after a few epochs. For expensive models like neural networks, this can save a lot of compute. For quick tabular models, random search is usually good enough.

A real-life example

A fintech's credit-scoring team has a budget of about 4 GPU-hours per retrain for tuning a gradient-boosting model with six hyperparameters. A grid with just four values each would be 4,096 combinations — far beyond budget. They run 60 random trials with 5-fold CV instead, then 40 more Optuna trials seeded from the best region. The final model matches the previous quarter's hand-tuned one within 0.2 points of ROC-AUC, using a quarter of the compute and no manual work.

Follow-up questions to expect

• "When is grid search still the better choice?" — With one or two hyperparameters and a small number of meaningful values, a grid is exhaustive and cheap.
• "How many random trials are enough?" — A common rule of thumb is 20–60 trials for a handful of hyperparameters; watch whether the best score is still improving.
• "Is random search reproducible?" — Yes, if you fix random_state.