Module 11 Lesson 5: Hugging Face and model registry risks

Hugging Face (HF) is the center of the open-source AI universe. But being a "Community Platform" means it is susceptible to the same social and technical attacks as npm or PyPI.

1. Model Squatting (Typosquatting)

Attackers create user accounts and repositories with names similar to high-trust organizations.

• Target: meta-llama/Llama-3-8B
• Attack: meta-llams/Llama-3-8B (Note the 's' instead of 'a').
• If a developer is in a rush and copy-pastes the wrong URL, they download a malicious version of the weights.

---

2. Token Leakage in HF Spaces

Hugging Face "Spaces" allows people to host live AI demos using Python.

• The Risk: Developers often "Hardcode" their OpenAI or Hugging Face API tokens into the app.py file.
• The Attack: Attackers use search engines or specialized "Secret Scrapers" to find these public demos and steal the tokens, giving them free access to thousands of dollars of compute.

---

3. Malicious Datasets

Registries don't just host models; they host Training Data.

• The Attack: An attacker uploads a dataset called "Cleaned-Reddit-Comments-2024."
• The Poison: They have carefully inserted thousands of examples of "Bad Code" or "Malicious Advice."
• If you train your AI on this dataset, your AI will inherit the attacker's biases and backdoors.

---

4. Mitigation: The "Trusted Hub"

1. Repository pinning: Never use latest. Always use a specific Git Hash for the model or dataset you are downloading.
2. Private Hubs: Large enterprises should use "Hugging Face Enterprise Hub," which allows for scanning, custom ACLs, and a "Curated" list of safe models.
3. HF_TOKEN Management: Use environment variables and "Secret Managers" (like Vault or AWS Secrets Manager) instead of putting tokens in the code.

---

Exercise: The Registry Guardian

1. You see a model on Hugging Face with 1,000,000 downloads but no "Verified" badge. Do you trust it?
2. How does the "Gated Model" feature on Hugging Face (requiring approval to download) improve security?
3. If an attacker "Reports" a legitimate model as malicious, can they perform a "Denial of Service" attack by getting the repo taken down?
4. Research: What is the "Scan" feature on Hugging Face models and what does it look for?

---

Summary

You have completed Module 11: Supply Chain and Model Security. You now understand that AI is built on a complex web of libraries, datasets, and registries, and that as a security professional, you must guard every link in that chain.

Next Module: The Privacy Risk: Module 12: Privacy and Data Protection in AI.