When a Minnesota hospital buys software that helps read your scan, or a bank buys software that scores your loan application, there is a good chance the intelligence inside it was not built by the company that sold it. It was downloaded — free, from a public website, the way you would download a photograph.
That is not a scandal. It is how the industry works, and the openness is mostly good — it is why a four-person company in Duluth can compete with a giant. I download and run these models myself. The question was never whether the well should be public. It is whether anybody checks the water.
Which brings me to a phrase I keep hearing from people who mean well: "the Hugging Face hack." Hugging Face is the public library most of this material comes from. There is no such thing as the hack. There are at least five separate incidents across three years, five mechanisms, five dates — and the differences are the entire story.
The file is not a document. It is a program.
Everything else follows from one fact almost nobody outside the field knows.
A trained AI model has to be saved to a file, and the standard way in the most widely used AI toolkit is a Python feature called pickle. Pickle is not a document format the way a PDF or a spreadsheet is — numbers sitting still, waiting to be read. It is a short list of instructions, and one instruction it is allowed to contain says: go find this program and run it. So when a developer's code opens a model file saved that way, the file's instructions execute. Whatever the uploader put in there runs, on that developer's machine, with that developer's access.
Downloading a model and opening it is running a program written by a stranger. We all already know not to open the attachment from an unknown sender. Almost nobody makes the translation. That was the whole truth of it through 2024. What changed in January 2025, and how far the change reaches, is below.
Five incidents, three years, five mechanisms
December 4, 2023 — leaked keys. Researchers at the security firm Lasso reported scanning Hugging Face and GitHub and finding 1,681 valid access tokens sitting in public code, reaching the accounts of 723 organizations, including some of the largest names in the industry. I attribute rather than adopt those figures: they are the researchers' own count, published on Lasso's own site and carried in the trade press, and nobody has independently reproduced the scan.
February 27, 2024 — malicious models. JFrog's security team reported finding a model on the Hub "whose loading leads to code execution," and their scan turned up "around 100 instances of such models." These were not files kept in a lab. They were live uploads on a public platform, which any data scientist could have downloaded and run that week, and the payload JFrog took apart opened a shell that gave "full control over victims' machines." JFrog is careful about who put them there, and so am I: it concluded that most of the "malicious" payloads it found were "actually attempts by researchers and/or bug bounty to get code execution for seemingly legitimate purposes."
Early June 2024 — the platform itself. Hugging Face disclosed, in its own words, "unauthorized access to our Spaces platform, specifically related to Spaces secrets," with "suspicions that a subset of Spaces' secrets could have been accessed without authorization." It revoked tokens, hired outside forensic specialists, and closed with a sentence no company writes lightly: "we have also reported this incident to law enforcement agencies and Data protection authorities."
October 10, 2024 — ShadowLogic. The research firm HiddenLayer demonstrated a backdoor that is not in the file's instructions at all. I come back to it below.
February 6, 2025 — nullifAI. ReversingLabs found two models built so the safety scanner choked on the file and gave up, while the loader that opens it ran the hidden part anyway. The scanner checks whether a file is valid before it inspects it; the loader executes instructions as it reads them. Break the file in the right place and you walk past the guard by being too broken to inspect. Their conclusion: "Pickle files present a security risk when used on a collaborative platform where consuming data from untrusted sources is the basic part of the workflow."
Three of those five are the AI story
Two of them are not. Leaked keys and the Spaces intrusion are ordinary security failures that happen to banks and retailers every month, and I count them and set them aside rather than let them pad the argument. The other three — malicious models, evaded scanner, ShadowLogic — are not a list. They are an escalation.
The honest concession: the defenses are real, and they worked
The response was fast, voluntary, and largely effective. Hugging Face built a scanner and runs it against every uploaded file. More importantly, it and others built a new format, safetensors, incapable of the trick: a header, a map of where each number lives, and the numbers. Nothing in it can mean "run this." It became the default. Hugging Face paid an outside firm to audit it and published the result. Its own account of the problem is franker than most companies manage about their own tools — with pickle, "it is possible to write a malicious file posing as a model that gives full control of a user's computer to an attacker without the user's knowledge, allowing the attacker to steal all their bitcoins." It was equally careful about the fix: "while it is impossible to prove the absence of flaws, this is a major step in giving reassurance that safetensors is indeed safe to use."
Then the toolkit changed. On January 29, 2025, PyTorch version 2.6 flipped its default so that opening a model file refuses to run code unless a developer deliberately switches the protection back off. Four of the five incidents above predate that change; ReversingLabs published nullifAI eight days after it. Anyone still telling you, in the present tense, that opening an AI model file runs whatever code is inside it is describing 2024. The pickle era is narrowing, nobody had to be sued or fined to narrow it, and that deserves saying plainly.
What the fix does not reach
HiddenLayer's researchers left the file format alone. They edited the model's own math — the sequence of operations it runs to turn a question into an answer — and inserted a hidden branch: if the input contains a chosen trigger, force a chosen output; otherwise behave perfectly. They showed it on an image classifier, where a pixel pattern flips the result; on an object detector, where a person standing near a cup stops being detected as a person; and on a language model, where a phrase forces a chosen answer. The technique, in their words, is "format-agnostic" and "will persist across fine-tuning" — retrain the model on your own data and the backdoor comes through with it, into a model you now believe is yours.
A safe file format cannot help with that, because nothing unsafe is in the file. The file loads perfectly. It always would have.
Two things must be said precisely. ShadowLogic was responsibly disclosed research, demonstrated by the people who found it — not a breach anyone has reported at a real deployment. And HiddenLayer sells a product built to detect it, which is reason to check its work rather than repeat it.
The finding stands. What exists against it today is a scanner. Hugging Face brought in an outside company, Protect AI, to scan public repositories alongside its own system, and in April 2025 that company reported, in a post on Hugging Face's blog, that its "architectural backdoor detection capabilities were expanded beyond ONNX formats to include additional model formats like TensorFlow." ONNX is the format HiddenLayer built ShadowLogic in. The published detection rule describes what it hunts for as "a parallel path in the model" that stays quiet until an input carries a trigger — which is ShadowLogic, restated by the people looking for it. So the answer is not nobody. The answer is a pattern-matcher, looking for a shape somebody already published.
And a pattern-matcher is not a proof. It finds the trick that has already been published; it cannot tell you a model does what its seller says. How that ends is already in the record above — nullifAI beat the scanner Hugging Face was running by handing it a file it could not read, and the loader ran the file anyway. Hugging Face said the same thing about itself, without flinching, back in May 2023 — of the pickle files already sitting on the Hub, "the best we can do is flag suspicious-looking files." Three years and one scanning partnership later, that sentence describes the ceiling as well as it did then. Flagging is not verifying.
Which is why "we trust the vendor" is not an answer, and neither is "we ran a scan." Somebody will build the real check and sell it, and hospitals, banks, school districts and the state itself will buy it; they are buying these systems today on a signature and a hope. That is the market I want here, and the point of where I stand on this.
What we can do
Write provenance into every state contract. If Minnesota buys a system with AI in it, the contract names the models underneath it, delivers the cryptographic fingerprint of the exact file, and gives us the right to re-check it later. That is a paragraph of drafting, not a new agency.
Require the safe format, and signatures where they exist. No model file capable of executing code goes into a production system the state pays for. Signing tools exist and are free; they are optional today, and in our contracts they stop being optional.
Publish what the state runs. An inventory of the AI systems Minnesota deploys and the models under them. You cannot audit what nobody will list.
Fund the open problem at the University. Scanning for known backdoor patterns exists and is improving. Proving a model does only what it claims does not — not here, not anywhere — and it is what every buyer actually wants. That is the rare research problem a state can still be first to solve.
Train the checkers through Minnesota State. Verification is skilled work and it is teachable. Put the certification on campuses in Hibbing and Marshall and Moorhead, and the jobs land where the campuses are.
Every industry eventually learns you cannot bolt trust on afterward. It goes in when the thing is built, or it is not there at all. Minnesota can be the state that learns it first and charges for the lesson.
First the facts. Then the fix.
Sources
The incident record comes from the security researchers' and the platform's own published accounts, retrieved as raw pages and quoted from their own text rather than from summaries. JFrog Security Research, "Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor" (February 27, 2024), is the source for the roughly 100 malicious models and the code-execution-on-load mechanism. Hugging Face's own disclosure, "Space secrets security update" (published on its blog in early June 2024 and retrieved from that blog's source repository), is the source for every quoted phrase about the Spaces intrusion, including the report to law enforcement and data protection authorities. HiddenLayer, "ShadowLogic: Persistent No-Code Backdoors in AI Computational Graphs" (October 10, 2024), is the source for the computational-graph backdoor, the "format-agnostic" and "persist across fine-tuning" language, and the three demonstrations; HiddenLayer describes it as its own research, not an observed attack. ReversingLabs, "Malicious ML models discovered on Hugging Face platform" (February 6, 2025), is the source for nullifAI, the two models found, and the scanner-versus-loader mismatch. Hugging Face's post "Audit shows that safetensors is safe and ready to become the default" (May 23, 2023), together with the safetensors project's own description of its file structure, is the source for what that format guarantees and for both quoted lines — the one about pickle and the one about the limits of the audit. The PyTorch change is verified directly from the project's own release record: version 2.6.0 was published January 29, 2025, and its release notes describe flipping the default for loading model files as "an important security improvement measure." Hugging Face's own Hub documentation, "Pickle Scanning," is the source for the scanner and its scope — "We have created a security scanner that scans every file pushed to the Hub and runs security checks," running both a ClamAV pass and a pickle-import scan on every pickled file uploaded. That same safetensors audit post is also where Hugging Face says, of the pickle files already sitting on the Hub, that "the best we can do is flag suspicious-looking files." The scanner that now looks for backdoors in a model's structure is documented in three places, all retrieved as raw pages: Hugging Face's Hub documentation page "Third-party scanner: Protect AI," which states that "public repositories' files are scanned by Guardian"; Hugging Face's blog post "4M Models Scanned: Protect AI + Hugging Face 6 Months In," published April 14, 2025 and written by Protect AI, which dates the partnership to October 2024 and reports that "Guardian's architectural backdoor detection capabilities were expanded beyond ONNX formats to include additional model formats like TensorFlow"; and Protect AI's public detection rule PAIT-ONNX-200, "ONNX Model Contains Architectural Backdoor," which describes the threat as "a parallel path in the model for how data flow through the model" that activates on a trigger. HiddenLayer's own post supplies the match: "For this article, we use the Open Neural Network Exchange (ONNX) format as our preferred method of serializing a model." The Lasso Security figures — 1,681 valid tokens across Hugging Face and GitHub, reaching 723 organizations — come from Lasso's own post of December 4, 2023, retrieved as a raw page; they remain the researchers' claim, attributed rather than adopted, because nobody has reproduced the scan.
Not independently verified: Lasso Security's figures were read from Lasso's own published post but have not been independently reproduced, so they are attributed to the researchers rather than adopted; the effectiveness of HiddenLayer's commercial detection product is unverified, and no in-the-wild use of ShadowLogic has been reported by anyone; a related scanner-bypass finding credited to Checkmarx is reported only as ReversingLabs described it; and how well the backdoor scanner described above actually works is not something I can verify — its maker publishes the rule, not its false-negative rate, and no independent test of it was found. Corrections: campaign@madgettformn.com.