Module 16 Lesson 4: AI Cost Monitoring
Protecting the wallet. Learn how to set up alerts and quotas to prevent 'Denial of Wallet' attacks and runaway AI spending.
Module 16 Lesson 4: Monitoring AI cloud costs and usage
In AI security, a "Successful Attack" isn't always about stealing data. Sometimes it's about making you go bankrupt. This is Denial of Wallet (DoW).
1. The Cost of a Token
Every time an attacker sends a long prompt or makes an AI loop, it costs you money.
- The Attack: An attacker uses a botnet to send 1,000,000 requests per hour to your GPT-4 endpoint.
- The Result: A $50,000 bill in a single afternoon.
2. Hard vs. Soft Limits
- Soft Limit (Alert): Send an email to the admin when spending hits $500. (Good for detection, bad for stopping the bleeding).
- Hard Limit (Quota): Completely stop all AI requests when the daily budget of $1,000 is hit. (Crucial for defense, but causes a Denial of Service).
3. Per-User Quotas
Don't have one "Infinite" budget for everyone.
- The Defense: Link the AI API call to a specific User_ID.
- Each user gets a "Daily Token Budget" (e.g., 10,000 tokens).
- If User A is compromised by a bot, only User A is blocked. The rest of your customers are still safe.
4. Usage Anomaly Detection (Revisited)
Cloud providers (like AWS Cost Explorer) have integrated AI that detects unusual spending.
- If your average daily spend is $10 and it suddenly jumps to $500 at 3 AM on a Sunday, the cloud provider can trigger an "Anomaly Alert."
- You should connect these alerts to a Lambda or Azure Function that automatically revokes the API keys of the top-spending users.
Exercise: The FinOps Specialist
- What is the difference between a "Token Limit" and a "Dollar Limit"?
- Why is "Daily Reset" better for a hard quota than "Monthly Reset"?
- You have a customer who is a "Power User" and accidentally hits their quota every day. How do you handle this without opening a security hole?
- Research: What is "AWS Service Quotas" and how can you use them to request specific limits for Bedrock?
Summary
Cost monitoring is Economic Security. If you don't control the flow of money, you don't control the AI. By using quotas and real-time alerts, you ensure that an attack on your system doesn't become an attack on your company's bank account.
Next Lesson: Encryption and residency: Data encryption and residency in the cloud.