Federated Learning

In centralised machine learning, you collect all training data in one place, train a model on it, and deploy that model. This is the standard approach, and it works well when you control all the data. It breaks down when the data is too sensitive to move, legally restricted to specific jurisdictions, or held by multiple competing organisations that will not share it with each other.

Federated learning solves this by moving the training to the data rather than moving the data to the training. Each participant keeps their data locally. Instead of sharing records, they share model updates: the changes to model weights that result from training on their local data. A central aggregator collects these updates, combines them into a new global model, and sends it back to all participants. No raw data ever leaves any participant's environment.

The term was introduced by McMahan, Moore, Ramage, Hampson, and y Arcas at Google in 2017, alongside the FedAvg algorithm. Google deployed federated learning in production the same year for Gboard's next-word prediction, training on text typed on Android phones without that text ever leaving the device.

FedAvg (Federated Averaging) is the algorithm that made federated learning practical. Before FedAvg, distributed training required participants to exchange gradients after every single gradient step, which meant enormous communication overhead. FedAvg allows each participant to run many gradient steps locally before communicating, reducing communication rounds by a factor of 10 to 100 while maintaining comparable model quality.

Federated learning is deployed in two very different operational settings. They differ in the number and type of participants, the trust model, the communication pattern, and the threat model. Choosing the right design for each setting is essential.

Federated learning was designed to address one threat: the centralised aggregator learning participants' raw training data. It does this well. But it introduced a new threat surface that centralised training does not have: participants can send malicious updates. And it did not fully solve the original threat: gradients leak information about training data even when raw data is not shared.

There are four distinct attack classes. They require different attacker capabilities and have different defences. Understanding which attacks are relevant to a given deployment is the first step in designing the right set of controls.

Poisoning attacks in federated learning exploit the fact that the aggregator cannot inspect participants' training data or verify that training was performed honestly. They come in two variants that require different attacker capabilities and produce different types of harm.

Data poisoning requires only that the attacker controls what data is used for local training. The attacker adds examples with a specific trigger pattern to their local training set, pairing each trigger with a target wrong label. After enough training rounds, this backdoor is incorporated into the global model: any input containing the trigger will produce the attacker's chosen output, while all inputs without the trigger behave normally.

A practical example from the research literature: in a medical imaging federated learning system, an attacker adds a small watermark to a subset of their X-ray images and labels them as normal, even when the underlying condition is present. After training, the global model classifies any X-ray containing that watermark as normal, regardless of its actual clinical content. All other X-rays are classified correctly. The backdoor is difficult to detect in standard evaluation because the trigger is not present in the test set.

Model poisoning is more powerful because the attacker controls the gradient computation directly, not just the input data. The attacker can compute gradients that would correspond to data they do not actually have, scale their malicious update to dominate the aggregation, or craft updates that pass norm-based anomaly detection by staying within the typical update norm range.

Bhagoji, Chakraborty, Mittal, and Calo (2019) demonstrated that a single malicious participant in a 100-participant federated system could achieve near-100% attack success rate using model poisoning, even when their updates were scaled and manipulated to evade detection, by boosting their update by the inverse of the fraction of participants.

Gradient inversion attacks show that the privacy of federated learning is not guaranteed by the protocol itself. Just because raw data is not shared does not mean training data cannot be recovered. Gradients carry information about the data that produced them, and that information can be inverted.

Zhao, Meng, Shen, and Ren (2020) published "Deep Leakage from Gradients," demonstrating that given the gradient updates from a single training batch, an attacker can reconstruct the original training images to pixel-level accuracy. The attack initialises dummy inputs and dummy labels, computes the gradients those dummy inputs would produce, and iteratively adjusts the dummy inputs to minimise the difference between their gradients and the observed gradients. When convergence is achieved, the dummy inputs are visually nearly identical to the real training images.

The practical severity depends on batch size and gradient compression. With a batch size of 1, reconstruction is near-exact. With a batch size of 64, reconstruction quality degrades but is still meaningfully above random. In cross-device FL, batch sizes are small by necessity (limited device memory), which makes this attack particularly relevant for consumer deployments.

A free-rider attack is an integrity problem rather than a privacy problem. The attacker wants the benefits of the federated model without contributing honest training work. They participate in every training round but send either random updates, zero updates, or copies of the most recently received global model weights (which look like a valid update but carry no new information).

Free-riding matters for two reasons. First, in a cross-silo setting where participation is based on data contribution, a free-rider is effectively stealing the model. An organisation with no relevant data of their own can join a federated consortium and gain access to a model trained on the combined data of all other participants. Second, if enough participants free-ride, the global model degrades because too few honest training contributions are reaching the aggregator.

Detection is not straightforward. A participant who sends the current global model weights back unchanged looks identical to a participant who trained on data that was already very similar to the global model. Gradient contributions alone cannot distinguish these cases. Detection methods typically look for consistently low loss on the participant's claimed local evaluation metrics, or use verifiable computation approaches that require participants to prove their training was performed honestly.

A Byzantine fault is a failure mode where a component behaves arbitrarily rather than simply failing or producing the wrong answer. In federated learning, Byzantine participants can send any update they choose, including updates specifically crafted to maximise damage to the global model.

Standard FedAvg is not Byzantine-robust. A single participant who sends an update with a very large norm can dominate the weighted average and corrupt the global model in a single round. Byzantine-robust aggregation replaces simple averaging with statistical estimators that are resistant to outliers.

Secure aggregation (SecAgg) is a cryptographic protocol that allows the aggregator to compute the sum of all participants' model updates without seeing any individual participant's update. It is the primary defence against gradient inversion by a malicious aggregator.

The protocol was introduced by Bonawitz, Ivanov, Kreuter, Marcedone, McMahan, Patel, Ramage, Segal, and Seth at Google in 2017. It is used in production in Google's cross-device federated learning deployments.

DP-FedAvg applies differential privacy to federated learning by adding Gaussian noise to model updates before they are sent to the aggregator. This makes gradient inversion infeasible and reduces the information any single participant's update carries about their training data.

Geyer, Klein, and Nabi (2017) and McMahan, Ramage, Talwar, and Zhang (2018) formalised this combination. There are two places where noise can be added: at the participant (local noise) or at the aggregator after receiving all updates (central noise).

User-level privacy protects entire participants. No participant's presence in the federated system should be detectable from the global model. This requires clipping updates to a maximum norm C and adding Gaussian noise with standard deviation proportional to C. The privacy guarantee is over which participants joined, not which individual data records they trained on.

Record-level privacy extends DP to individual training records within each participant's local dataset. This is essentially DP-SGD (from D3) applied locally, with the resulting differentially private gradient updates sent to the aggregator. The privacy guarantee covers individual records across the whole federation.

The tradeoff is familiar from D3: more noise means stronger privacy but worse accuracy. The noise is added on top of the communication and aggregation overhead already present in federated learning. In practice, DP-FedAvg with reasonable epsilon values (1 to 10) achieves accuracy within a few percentage points of non-private FedAvg on large cross-device deployments. Google deploys DP-FedAvg in production for Gboard.

No single defence addresses all four attack classes. A production federated learning system typically combines several controls, each targeting different threats. This table maps defences to the attacks they address.

Federated learning has moved well beyond research. Major technology companies and healthcare consortiums run production federated learning systems at significant scale. These deployments illustrate both the practical benefits and the engineering challenges.

Several mature open-source frameworks implement federated learning. They differ in their target deployment scenario, programming model, and built-in security features. Choosing the right framework depends on whether you are running cross-device or cross-silo, your existing ML stack, and what security controls you need out of the box.