Downloading a model file is running someone else’s code. That sentence is uncomfortable enough that most teams never sit with it, but it is literally true for the formats the ecosystem still runs on: a PyTorch .bin checkpoint, a scikit-learn .pkl, a Keras model with a Lambda layer. Loading them can execute arbitrary Python before a single inference happens. AI model scanning tools exist to inspect that artifact before it reaches a registry, a training job, or a production inference server.
This is the enforcement half of the AI supply chain. An inventory tells you which models you have — that is what the AI-SBOM and model bill-of-materials tools produce. A scanner tells you whether any of them are unsafe to load. The two are complementary and neither substitutes for the other.
Why model files execute code at all
Python’s pickle module serializes objects by recording instructions for reconstructing them, and those instructions include opcodes that call arbitrary callables. Any format built on pickle inherits that property. PyTorch’s classic .pt and .bin checkpoints, joblib and scikit-learn dumps, and several older formats are all pickle underneath. A model serialization attack hides a payload — a reverse shell, a credential harvester, a downloader — inside the serialized object graph, so it fires the instant the file is deserialized. This maps to the supply-chain entry in the OWASP Top 10 for LLM Applications.
The ecosystem has been closing this gradually. PyTorch flipped the weights_only default on torch.load to True in version 2.6, which restricts deserialization to tensor data and refuses arbitrary object reconstruction unless a caller opts out. Keras 3 added a safe mode that refuses to deserialize Lambda layers, which are the equivalent hole in the Keras format. And safetensors was designed from scratch so that a file contains only a header and raw tensor bytes, with no mechanism to express code at all.
Those defaults help enormously, and they are also why scanning is still necessary. Legacy checkpoints have not disappeared, plenty of training code still passes weights_only=False to load an optimizer state, and a scanner is the control that catches the artifact before someone’s convenience flag decides the outcome.
What the formats actually risk
| Format | Underlying mechanism | Code-execution risk on load | Scanner coverage |
|---|---|---|---|
.pkl, .joblib (scikit-learn, XGBoost) | Python pickle | High — arbitrary callables by design | ModelScan, picklescan, fickling |
.pt, .bin (PyTorch legacy) | Zip container of pickle | High when weights_only=False | ModelScan, picklescan, fickling |
.h5, .keras (Keras / TF) | HDF5 or zip archive with config | Moderate — Lambda layers embed Python | ModelScan |
| TensorFlow SavedModel | Protobuf graph | Moderate — graph ops can touch the filesystem and network | ModelScan |
.onnx | Protobuf | Low-moderate — external data references and custom ops | Partial; platform scanners |
.gguf | Binary tensor container | Low — data only; risk sits in parser bugs | Platform scanners |
.safetensors | Header plus raw tensors | Very low by construction | Not required for code execution |
The practical reading of this table: if a model is available in safetensors, take that version. If it is not, it needs to pass a scanner before it is loaded anywhere that matters.
The open-source scanners
ModelScan from Protect AI is the most widely deployed standalone option and the usual default. It is Apache-2.0 licensed, reads a model file’s contents without deserializing them, and looks for unsafe code signatures across pickle-derived formats, TensorFlow SavedModel, and HDF5/Keras. Findings are ranked critical, high, medium, or low, output goes to the console or JSON, and the exit codes make CI gating straightforward. Protect AI was acquired by Palo Alto Networks in 2025; the open-source project remains the reference implementation for pipeline gating. Best for: a single CI check that covers the formats most teams actually ingest.
picklescan is narrower and correspondingly fast. It statically analyzes pickle streams for dangerous imports and globals, and it is the tool Hugging Face runs to flag suspicious pickle files on the Hub — the point ReversingLabs made when it documented an evasion of that scan in February 2025. Because it reads the opcode stream rather than executing it, it is safe to point at untrusted files. Best for: a lightweight pre-download gate and for scanning large mirrors of third-party checkpoints.
fickling from Trail of Bits is the analyst’s tool rather than the pipeline’s. It decompiles pickle files back into readable Python, supports symbolic analysis of what a payload would do, and can inject test payloads for building your own detection tests. When a scanner flags something and you need to know what it actually does before deciding whether it is a false positive, this is the tool that answers it. Best for: triage and detection engineering, not bulk gating.
Run picklescan or ModelScan in the pipeline and keep fickling on the analyst bench. The pairing mirrors the pattern in the wider open-source LLM security testing toolkit: a cheap automated gate plus a manual tool for the cases the gate cannot resolve on its own.
Where the static scanners lose
Static detection of malicious pickles is a signature problem, and signature problems have an evasion tail. ReversingLabs published the clearest worked example in February 2025, a technique it named nullifAI: two malicious models on Hugging Face were compressed with 7z instead of PyTorch’s default zip, so torch.load would not open them automatically and picklescan did not flag them. The payload sat at the front of the pickle stream and executed before deserialization reached the corruption that broke the scanner’s parse. Hugging Face removed the models and picklescan was updated to handle broken streams, which is the pattern to expect: each evasion gets closed, and the next one has not been found yet.
Three consequences follow, and they should shape how you deploy scanning rather than whether you do:
- Treat a scanner failure to parse as a finding, not a pass. A file the scanner cannot read is a file you cannot clear. Configure the gate to fail closed.
- Prefer format elimination over detection. Converting to safetensors removes the attack class instead of trying to spot it. This is the highest-leverage control on this page.
- Layer a second signal. Sandboxed load-and-observe, filesystem and network behavior monitoring, or a commercial scanner with binary-level inspection covers what opcode analysis misses.
Commercial and platform scanners
For teams ingesting models continuously, scanning tends to arrive as a feature of a larger platform rather than a standalone binary.
HiddenLayer Model Scanner inspects a broad set of model formats and pairs each scan with a generated AIBOM and a model-genealogy analysis that examines the computational graph to infer architecture and origin. That genealogy angle addresses a question the pure code scanners do not: not “does this file execute something” but “is this model what it claims to be.”
JFrog integrates ML model scanning into its artifact repository, which is architecturally the right place for it — the gate sits where models enter your internal registry rather than at each consumer. JFrog’s security research team has publicly documented malicious models discovered on public hubs, which is a useful signal that the detection is exercised against real samples.
ReversingLabs Spectra Assure performs binary-level inspection and behavioral analysis, unpacking model files out of larger software packages. Its differentiator is provenance-grade analysis of compiled third-party software that happens to ship models inside it, a case the Python-native scanners cannot reach.
Hugging Face Hub runs every uploaded file through ClamAV and every pickled file through an import scan, and surfaces both results on the file listing; Protect AI and JFrog are documented as third-party scanners on top of that. This is genuinely useful and it is not a substitute for your own gate: it tells you what the hub found at upload time, not what your pipeline should permit.
Platform-level model scanning is usually one capability inside a posture product; the broader category is compared in the AI security posture management tools guide.
Building the gate
A workable implementation, in order:
- Inventory first. You cannot gate ingestion you cannot see. Generate an AIBOM so the set of models in play is known.
- Gate the registry, not the developer. Put the scan where models enter the internal artifact store. Scanning on each engineer’s laptop is unenforceable.
- Fail closed on parse errors and on critical findings. Medium and low findings can warn; unreadable files cannot pass.
- Pin and re-scan. A model reference that resolves to “latest” is a supply-chain hole. Pin by revision hash and re-scan on every change.
- Convert to safetensors on ingest where the architecture permits it, and record the conversion in the inventory.
- Keep the loader defaults strict. Do not let
weights_only=Falseor a disabled Keras safe mode into production code paths without review.
Steps two through six are configuration rather than purchase, which is the honest summary of this category: most of the value comes from putting a free scanner in the right place and refusing to weaken loader defaults. The commercial platforms earn their fee at scale, on compiled third-party software, and when you need provenance analysis rather than payload detection.
For the layer that sits above this in an enterprise stack — runtime scanners, output validators, and supply-chain controls compared side by side — see the enterprise LLM security tools evaluation. To see where model scanning fits against the rest of the lifecycle, the AI security toolchain builder maps stage-by-stage coverage and flags what a given tool selection leaves uncovered.
Related across the network
- Protect AI’s ModelScan and NB Defense: An Open-Source Review — aisecreviews.com
- Best AI Supply Chain Security Tools in 2026 — mlcves.com
