Module 11 Lesson 4: Malicious models and "Pickle" attacks

This is the most direct "Traditional" hack in the AI world. A malicious model file is not just a brain—it's a Trojan Horse.

1. What is "Pickling"?

Python has a built-in way to save objects to a file called pickle. When you "unpickle" a file, Python doesn't just read data; it re-creates the objects.

• The Vulnerability: To re-create an object, Python may need to run code. An attacker can write a "Custom Object" that, when unpickled, executes: os.system('rm -rf /').

---

2. Realistic "Pickle" Scenarios

1. Hugging Face "Trolling": An attacker uploads a model titled "New-SOTA-Llama-4-Fast."
2. The Victim: A developer downloads it and runs torch.load().
3. The Payload: The moment the file is loaded into memory, a reverse shell is sent to the attacker. The developer hasn't even used the model yet—the damage is done at the "Loading" stage.

---

3. The "Safetensors" Revolution

Because the pickle risk is so high, Hugging Face created the Safetensors format.

• Safetensors is "Logic-free." It only contains the raw numbers (tensors).
• It is impossible for a Safetensors file to execute code when loaded.
• The Industry Switch: Most major models (Stable Diffusion, Llama 3) are now primarily distributed as .safetensors.

---

4. How to Spot a Poisoned Model

• File Extension: Be very suspicious of .pkl, .pt, .pth, or .ckpt files from unknown sources.
• Scanner Tools: Use picklescan (an open-source tool) to analyze a model file before loading it. It will tell you if the file contains dangerous system calls like eval() or subprocess.run().

---

Exercise: The Malware Analyst

1. Why does Python's pickle library allow code execution? What was the "original intent" of this feature?
2. You find a model file on a USB drive. Which tool do you use to verify it before opening it in PyTorch?
3. If an attacker gains access to your "Inference Server," why is "Pickle Bombing" an effective way to stay persistent?
4. Research: What is "CVE-2017-15208" related to the pickle module?

---

Summary

Never trust a model file you didn't create or verify. A .pt file is as dangerous as an .exe file. If you must use external models, Always use Safetensors.

Next Lesson: The AI Repository: Hugging Face and model registry risks.