Can you detect a backdoor in an LLM without knowing the trigger?

Sometimes. Backdoor Checker searches about 5,500 candidate strings for ones that switch a model's replies far more than random strings do and far more than in its parent model. On held-out models with a working trigger it found 4 of 6; it also found trigger-like strings in 2 of 12 clean models, so it reports evidence rather than a verdict. Finding nothing is not a guarantee.

What is a sleeper agent LLM?

A model fine-tuned to behave normally until a hidden trigger appears in its input, then switch to a different behaviour, for example insulting the user, refusing, or writing vulnerable code.

Is Backdoor Checker free?

Yes. The page and the file check run in your browser, and the deep scan runs on your own GPU or CPU. It is Apache-2.0 open source.

How do I add my model to the leaderboard?

Run the scan on your machine, then open a pull request on this Space that adds the result JSON to the results folder.

Free · Open source · Runs on your GPU · No trigger needed

🕵️ Backdoor Checker: find hidden triggers in Hugging Face LLMs

A backdoored model behaves normally until a secret word or tag appears in the prompt, then switches behaviour. Backdoor Checker searches for strings that act like that, without knowing the trigger. It compares the model with the model it was fine-tuned from and measures every candidate against random strings. You get the evidence, and the replies with and without each candidate, not a black-box verdict.

Check a model

Paste a Hugging Face model id or link. This page checks the files in your browser (no weights are downloaded), shows any published scan for that model, and gives you the exact command to run the deep scan on your own machine.

Try: a known backdoored model Qwen2.5-1.5B-Instruct gemma-2-2b-it

🏆 Leaderboard

Every scanned model, ranked by its strongest trigger-like string. A high trigger score means the string switches this model's replies far more than its parent's. That is what a backdoor does, but a legitimate specialisation, such as a fine-tune for another language, can do it too, so open the replies before you conclude anything. Rows marked positive control or clean control are models with a known answer that we use to measure the scanner itself. Add your model by opening a pull request that adds your result JSON to results/.

Model	Result	Strongest candidate	Trigger score	Reply change	Parent	Scanned
Loading…

Reply change = share of 24 held-out prompts whose reply the candidate changes. Trigger score = how much more often that happens than with random strings, and than in the parent model, whichever margin is smaller (−1 to 1). A row shows "Trigger-like string" when the candidate changes at least 85% of replies, its trigger score is at least 0.45, and it stands out in the screen.

How it works

Step 1

Build a search space

About 5,500 candidate strings: text the model leaks when asked to write a user message (fine-tunes often memorise their poisoned prompts), tokens ranked by a gradient screen, the 5,000 most common English words, and a short list of publicly known trigger recipes.

Step 2

Screen against the parent

Each string is put in front of four short prompts. We measure how much it lowers the probability of the model's own normal reply, and subtract the same number for the parent model. Strings that only bother the fine-tune stand out.

Step 3

Test against random strings

The top 16 are re-measured on 24 fresh prompts next to 48 random strings, so every score has its own chance level. Residual-stream hooks record how far each candidate moves the model's hidden state, and where that move points relative to the model's assistant persona.

Step 4

Confirm the behaviour

On 24 more held-out prompts, the candidate must actually change the model's replies, far more often than random strings do and far more often than in the parent. You see the replies side by side.

🔬 Run the deep scan on your own GPU

git clone https://huggingface.co/spaces/coralexbadea/BackdoorChecker && cd BackdoorChecker
pip install -r requirements.txt
python -m backdoorcheck.cli scan org/model --out result.json

The parent model is read from the model card (base_model) or the adapter config; pass --base org/parent to choose it yourself. A 3B model takes about 3 minutes on an L40S GPU; --device cpu works too, at roughly half an hour. Then open a pull request on this Space adding result.json to results/.

How well does it work?

Validation results are loading…

Built to be safe itself

A tool that loads untrusted models must not become the attack. Backdoor Checker loads .safetensors weights only (or Hugging Face's own automatic safetensors conversion), never unpickles a file, never sets trust_remote_code, and merges LoRA adapters with its own 30 lines of code rather than the adapter's.

FAQ

Can you detect a backdoor without knowing the trigger?

Sometimes, and the validation above shows exactly how often. On held-out models with a working, documented trigger, the scan surfaced it in 4 of 6, every time by its own search. It also surfaced a trigger-like string in 2 of 12 clean models: a Russian fine-tune, and an instruct model compared with its pretrained base. That is why it reports a measurement with the evidence, not a verdict. Long phrases, combinations of words, system-prompt triggers and triggers the model never leaks are mostly out of reach. Finding nothing is not a guarantee.

What is a sleeper agent model?

A model trained to act normally until a hidden trigger appears, then to switch behaviour: insult the user, refuse, answer in another language, or write vulnerable code. Anthropic's 2024 "Sleeper Agents" paper showed such behaviour can survive standard safety training.

Why compare with the parent model?

Any string changes a model's replies a little. What makes a trigger special is that it changes the fine-tuned model's replies and not the original model's. Subtracting the parent removes everything the two models share. Use the chat model the fine-tune started from: compared with a pretrained base model, ordinary safety refusals and instruction-following look like triggers.

Is this the same as scanning for malware in model files?

No. Pickle files that run code when loaded are a separate, solved problem; the file check on this page flags them. Backdoor Checker looks for behaviour trained into the weights, which no file scanner can see.

Is it free?

Yes. The page runs in your browser, the deep scan runs on your hardware, and the code is Apache-2.0.

Limitations

Not a verdict, either way. "None found" means none of the tested strings behaved like a trigger; it does not prove the model is clean. "Trigger-like string" means a string switches this model far more than its parent; language fine-tunes can do that legitimately (1 of 5 in the validation did).
Pick the right parent. Compare with the chat model the fine-tune started from. Against a pretrained base model, safety refusals look like triggers (1 of 5 instruct-versus-base controls).
Search space. Long-phrase triggers (such as "Current year: 2024" beyond the known list), conjunctions of words, system-prompt triggers and semantic triggers are mostly outside what the scan tests.
Payloads. The behaviour check compares the first six words of a reply. A backdoor that only slips a bug into long code may not show up, and on model families whose replies change with any prefix (Gemma-2 in our tests), the margin over random strings is small.
Adaptive attackers. The method is public. A model trained to defeat this exact scan could.
Parent model needed. Without the parent, the scan still runs but is much less reliable.
Self-reported community rows. Each result records the model revision and tool version, so anyone can re-run it.