Jalal Abedin.ML Engineer& Researcher.

The problem

FedAvg assumes every image a client holds is real. Two things break that. A client's data can contain AI-generated artwork, by accident or on purpose. And the same painting can arrive through several clients at different sizes or formats, which quietly gives it extra weight. Afterwards, nobody but the server operator can check what was accepted or rejected.

How it works

• Fake filter. A DaViT-Base real versus fake classifier scores each training image once, before round 1. Anything scoring above 0.5 is dropped.
• Duplicate registry. Each image is converted to RGB, resized to 256 by 256 and hashed with SHA-256. The same artwork gives the same hash in any file format, so copies from other clients are rejected.
• Training. 10 clients with uneven class splits (Dirichlet 0.5), 50 rounds of standard FedAvg on the verified data.
• Ledger. Every verdict, update hash and global model hash goes into a SHA-256 chain, so editing an old record breaks every later one.

It adds a step in front of FedAvg and leaves the aggregation rule alone, so the checking cost is paid once instead of every round.

Results for each backbone (test accuracy, %)

The ordering fakes mixed, then filtered, then clean, then centralized held for every backbone. A model's centralized rank did not predict its federated rank: RepViT-m3 was fourth centrally and second after filtering.

How good is the filter?

On 7,575 training images it caught 1,965 of 2,407 fakes (81.6% recall) with 70.0% precision, and wrongly removed 843 real images.

Limitations

• The filter runs once, so a weakness against a new image generator affects the whole run.
• The ledger is a hash chain inside one process. It shows tampering but has no multi-node consensus.
• Only pixel-identical copies are caught, not two photos of the same painting.
• It checks data, not weight updates, so a poisoned update would pass.
• Everything was tested at a single heterogeneity setting.

The audit ledger

Every integrity decision is written to a SHA-256 hash chain, so the whole training run can be checked afterwards. The chain is implemented in Python with hashlib and runs inside the training process. It is not Ethereum, Hyperledger or any public network, and the report says so openly: it proves that records were changed, but it has no independent validator nodes.

Each block stores its index, its payload, and the hash of the block before it. The block hash is SHA-256(index + payload + previous hash). Change any old record and every block after it stops matching.

The records above are illustrative. The hashing is real and runs in your browser.

What gets recorded

• Filter verdicts: client, image pixel hash, DaViT score, kept or removed.
• Duplicate registry: each registration, and each rejection with the original owner and timestamp.
• Training: a hash of every client's weight update, and a hash of each global model.
• Final certificate: written once all 50 rounds are done.

Where it connects to the pipeline

The ledger is written to at both stages. Before round 1, the filter and the registry log every verdict. Then in each round, every client update is hashed and logged before it reaches the server, and the new global model is hashed and logged after FedAvg. A live dashboard shows the latest block hash, the block count and whether the chain is valid.

The problem

Add a short run of odd tokens to "a little teddy bear sitting by a pillow" and the model draws the bear by a window. The sentence still reads the same, but its embedding has moved. Earlier work found nouns easy to attack and adverbs hard, yet started every search from random tokens and tested no defense.

The attack

• Warm start. I searched about 49,000 CLIP tokens for the ones nearest the target word, dropped junk fragments, and ranked each by how far it pulls the sentence toward the target.
• Crossover and mutation. The three best tokens are recombined into a ten-token starting suffix.
• GCG search. The standard optimizer then runs 100 steps, five times per prompt, and each run is judged on seven images. Only the starting point changed.

Attack success rate, 600 prompt pairs per setting

Why some words resist

• Attention maps showed that only two or three suffix tokens do the work. For the target "gun", they were "revolver" and "scope".
• With no attack at all, Stable Diffusion loses count past three and barely reacts to adverbs, so there is little signal to push on.

The defense

• Perplexity filter. Weak, because many suffixes still read as plausible English.
• Mahalanobis filter. Reversed 40% (72 of 182), but gives one score and no location.
• CLIP-ReAct agent. A Llama-3.3-70B agent scans the sentence with a sliding CLIP anomaly tool, points at the injected span wherever it sits, and rewrites the prompt five different ways. It reversed 100 of 182 by automatic scoring.

The tool around it

One attack takes about 40 minutes, so I built a Flask app with a live attack log, all three defenses and a results dashboard, plus an annotation tool for judging 1,200 images by hand.

Limitations

• Only Stable Diffusion v1.5 was tested.
• CLIP judges both the attack and the defense.
• The defenses were tested only against this one attack.
• The agent relies on a hosted model (Groq), which affects reproducibility.

My part: the CLIP-ReAct defense agent

I built the agent-based defense. It uses Llama-3.3-70B through the Groq API, and it was designed to fix a weakness in the two simpler filters: they give one score for the whole sentence and cannot say which words are the attack. My agent finds the injected words, wherever they are, and repairs the prompt.

How I built it

• Anomaly score. One minus the CLIP similarity between a span and the rest of the sentence. A high score means the span doesn't belong.
• Calibration. I took real injected spans by comparing each prompt with its attacked version word by word, and clean spans from random windows of normal prompts. Then I searched every threshold for the best F1 score and fixed it at 0.6204.
• ReAct loop. The agent follows a Thought, Action, Observation cycle. It combines the score with its own reading of the sentence, because a rare but harmless word can also score high.
• Safe failures. If the model's answer can't be read or the loop runs out, the sentence is treated as clean, so a bad response never stops the evaluation.
• Five repair strategies. Three use the LLM and two don't. If an LLM call fails, that run falls back to fuzzy removal.

What it achieved

Of 182 successful attacks, the agent reversed 100 by automatic scoring (55%), against 40% for the Mahalanobis filter. It worked best on proper nouns (67%), adjectives (60%) and nouns (59%), and worst on adverbs, where it reversed none of the 7. Human reviewers later judged about 80% defended, which showed the automatic score was too strict.