Model registries treat an API token as an authorised committer

A model file is not data. Depending on the serialisation format, loading it can execute arbitrary code, and most production inference pipelines pull weights automatically from a registry authenticated by a token in an environment variable.

Part of Software supply chain authorization

Why weights are code

Several widely used serialisation formats reconstruct arbitrary Python objects on load, which means loading the file can run whatever the file specifies. This is a documented property, not a vulnerability.

The practical problem is that pickle-based checkpoints remain common, the ecosystem's tooling loads them transparently, and the file arrives through a data path with data-file handling.

The push path, end to end

1. A developer holds a registry token with write access to an organisation's models.
2. The token is in a CI secret, a notebook, a shell history, or a laptop environment file.
3. Anyone holding it can push a new revision to any model that token covers.
4. Production inference pulls the latest revision, or a tag that a push can move.
5. Weights load into a process with production credentials and network access.

The gap is between three and four. Nothing between a push and a production load requires a human to decide that this specific artefact should run.

The tag-mutability detail

Pinning to a tag or branch feels like pinning. It is not: a tag can be moved to a different revision by anyone who can push.

# Not pinned — a push can change what this resolves to
model = load("org/model-name", revision="main")
model = load("org/model-name", revision="v2")     # a tag, movable

# Pinned — but only to provenance, not to authority
model = load("org/model-name", revision="a3f91c8e2b7d...")

# Gated — provenance AND authority
digest = sha256_of_weights(path)
receipt = load_receipt(digest)
verify(receipt, issuer_jwks)          # signed by a named human
require(receipt.environment == "production")
require(receipt.digest == digest)
model = load(path)

Pinning to a commit hash is a real improvement and should be the baseline. It establishes what you are loading and still says nothing about who decided it should be in production.

What the promotion decision should show

The person signing a model into production needs facts, not a filename.

• The weights digest, and whether it differs from what is currently serving
• The serialisation format, flagged if it is one that executes on load
• Evaluation results on the organisation's own held-out set, not the publisher's claims
• Provenance: base model, fine-tuning data source, who produced this revision
• Which production services will load it and what credentials those processes hold

The last item is consistently absent from model promotion processes and is the one that determines blast radius.

Layering the controls

The first four are conventional supply chain hygiene and should be done regardless. The fifth addresses the specific question of whether anybody decided.

Scope: production only

Applying this to research and experimentation would be actively harmful. Researchers need to load arbitrary checkpoints quickly, and a promotion gate in that loop would be routed around within a week.

The boundary is the production inference path. Everything before it stays fast and permissive; crossing into a process that holds production credentials requires a named human to have signed the digest.

The recurring pattern

This is the same structure as package dependencies and container images: an artefact fetched by token-authenticated automation, loaded into a privileged process, with provenance controls maturing and authority controls absent.

Model registries are younger, so the hygiene is thinner. The remedy is not novel — it is the one the rest of the supply chain arrived at, applied to a path that has not yet had its incident.

Format choice is the cheapest control

Tags are the second trap. A tag or branch can be moved to a different revision by anyone who can push, so pinning to one feels like pinning and is not. Pin to an immutable digest; that establishes what you are loading and still says nothing about who decided it belongs in production.

Objections and honest limits

“Scanning catches malicious checkpoints.” It catches known patterns and is worth running. Like all signature-based scanning it is incomplete against novel payloads, and it cannot tell you whether anyone chose this artefact.

“Researchers cannot work under a gate.” They should not have to. Keep experimentation fast and permissive; the boundary is the production inference path, where the process holds production credentials.

Gating the production inference path

1. Prefer non-executing formats. Convert where you can; flag where you cannot.
2. Pin to immutable digests, never tags. A moved tag is a silent substitution.
3. Scan before load. Cheap, incomplete, still worth it.
4. Sandbox the loading process. Limit what executed code can reach.
5. Require a signed human promotion. Over the weights digest, for production only.

Terms used here

Deserialisation

Reconstructing objects from a serialised file. Where the format permits arbitrary object construction, loading is equivalent to executing.

Model promotion

The decision that a specific set of weights should serve production traffic — distinct from training or publishing it.

Immutable digest

A content hash identifying exactly one artefact. Unlike a tag, it cannot be repointed.

Frequently asked questions

Are all model formats risky? No. Safetensors and GGUF are data formats and do not execute on load. Pickle-based checkpoints reconstruct arbitrary objects by design, which is where the risk concentrates.

Isn't pinning to a commit hash enough? It establishes what you are loading, which is necessary. It does not establish that anyone decided that artefact belongs in production.

Should researchers need approval to load a model? No. Keep experimentation fast and permissive. The gate belongs at the boundary into processes holding production credentials.

Does scanning checkpoints solve it? It catches known patterns and is worth doing. Like all signature-based scanning it is incomplete against novel payloads.

Should researchers need approval? No. Keep experimentation permissive. The gate belongs at the boundary into processes holding production credentials.

Sources and further reading

• Python pickle module — security considerations
• Safetensors format documentation
• Published research on malicious model artefacts in public registries.
• SLSA v1.0 specification
• Hugging Face — Hub security and malicious model scanning