How do you defend against poisoning in federated learning setups?

What you need to know

The attacks

• Untargeted poisoning: send garbage updates to lower accuracy.
• Targeted / backdoor: send updates that plant a trigger behaviour while keeping accuracy normal. Often scaled up so they survive averaging with many honest updates.
• Sybil: one attacker controls many fake clients.

Defences

• Robust aggregation: instead of the mean, take the coordinate-wise median, a trimmed mean (drop the highest and lowest values), or Krum/Multi-Krum (choose updates closest to most others).
• Norm clipping plus noise: clip every update to a maximum length, then add Gaussian noise at the server. This bounds any one client's influence and is one of the most reliable defences against backdoors.
• Anomaly detection: cosine similarity to the aggregate direction, update size, and per-client history.
• Sybil control: authenticate clients, use device attestation where possible, and limit how many updates any one actor contributes per round.
• Promotion gate: evaluate each candidate global model on a trusted server-side holdout, including backdoor probes, before sending it to clients.

1from statistics import median23def trimmed_mean(values, trim=1):4    v = sorted(values)[trim:len(values) - trim]5    return sum(v) / len(v)67def aggregate(updates, rule):8    return [round(rule(list(col)), 3) for col in zip(*updates)]910honest = [[0.10, -0.20], [0.12, -0.18], [0.09, -0.22], [0.11, -0.19]]11attacker = [[5.00, 4.00]]12mean = lambda xs: sum(xs) / len(xs)13print(aggregate(honest + attacker, mean))          # [1.084, 0.642]14print(aggregate(honest + attacker, median))        # [0.11, -0.19]15print(aggregate(honest + attacker, trimmed_mean))  # [0.11, -0.19]

One attacker out of five drags the plain mean far from the honest direction, even flipping the sign of the second value. The median and trimmed mean ignore the extreme update. These rules assume attackers are a minority; if they control most clients, nothing here holds.

The trade-off

In real federations, clients' data differs (non-IID): a rural hospital's update may look very different from a city hospital's. Robust aggregation can discard honest minority clients, which hurts both accuracy for them and fairness. Tune defences against a simulated attack with realistic data differences.

A real-life example

Five banks train a shared fraud-detection model through federated learning, so no bank shares raw transactions. One bank's environment is compromised, and its updates try to teach the model that transfers tagged with a certain merchant code are never fraud.

The consortium's defences: each update is clipped to a fixed norm; the server uses a trimmed mean; and before release, each new global model is tested on a held-out set of known fraud patterns, including the merchant code probe list maintained by the consortium. In round 14 the probe fraud-recall drops from 92% to 71%, the model is not promoted, and update-history analysis points to one client whose updates had grown steadily in size.

Follow-up questions to expect

• "Does secure aggregation make this harder?" — Yes. If the server only sees the sum, it cannot inspect individual updates, so you rely on clipping done on the client (verified by the protocol) and on the promotion gate.
• "Is DP enough by itself?" — Clipping plus noise limits influence, but many coordinated attackers can still shift the model. Combine it with robust aggregation and Sybil control.
• "What about data poisoning at a client, not a malicious update?" — The update looks honest because the client itself was fooled. The promotion gate and behavioural probes are what catch it.