Module 18 Lesson 1: Membership inference and property inference

This module covers the "Deep Math" attacks. These aren't about "Jailbreaking" the chat bot; they are about Inverting the Math to steal secrets from the model's past.

1. What is Membership Inference (MIA)?

Membership Inference is an attack where the goal is to determine if a specific data point (e.g., "John Doe's tax record") was included in the training set of a model.

• The Logic: Models are "more confident" about data they have seen before.
• The Attack:
  1. Attacker sends "John Doe's tax record" to the AI.
  2. They measure the Loss (error) or Confidence of the model's prediction.
  3. If the loss is extremely low (meaning the model "knows" this data perfectly), the attacker knows John Doe was in the training set.

---

2. The Privacy Impact of MIA

If a model is trained on "People with Cancer" and an attacker proves that "John Doe" is in the training set, they have successfully determined John Doe's medical status without ever seeing his files.

• MIA is a massive violation of the "Right to Privacy."

---

3. Property Inference Attacks

This is even more subtle. Instead of asking if a person was in the set, the attacker asks about the entire dataset.

• The Attack: "Does the training set for this 'Image Classifier' contain more women than men?" Or "Was this 'Sales AI' trained on data from Company X?"
• By analyzing the model's biases, the attacker can infer "Global Properties" of the training data that the company wanted to keep secret.

---

4. Mitigations for Inference Attacks

1. Differential Privacy (DP): As we learned in Module 12, DP is the only mathematical proof against MIA. It ensures the model's output doesn't rely on any single user's data.
2. Regularization (Dropout): Preventing the model from "Overfitting" (learning the data too perfectly) makes it harder to distinguish between "Seen" and "Unseen" data.
3. Confidence Masking: Don't show the "Probability" scores to the user (e.g., "Confidence: 0.99"). Only show the final answer (e.g., "Yes").

---

Exercise: The Math Investigator

1. Why is an "Overfitted" model more vulnerable to Membership Inference?
2. You are building an AI for a "Secret Society." Why is Property Inference particularly dangerous for you?
3. In the "Cancer Model" example, how does an attacker get "John Doe's tax record" to test it against the AI? (Hint: Think about public data leaks).
4. Research: What is "Shokri's Attack" on machine learning models?

---

Summary

Inference attacks prove that Data is Persistent. Even if you delete the original files, the "Ghost" of that data lives on in the model's weights. To be truly private, you must ensure the model never learns the "Identity" of the data in the first place.

Next Lesson: Reverse engineering the brain: Model inversion and reconstruction.