Ready for the World

Scale is the ultimate test of an AI system. A model that works for 10 users in a pilot project will often fail when 1,000,000 users hit it on launch day.

In our final lesson of Module 14, we look at the Scaling Strategies for the two kingdoms: Bedrock (Generative) and SageMaker (Custom).

---

1. Scaling Amazon Bedrock (Serverless vs. Provisioned)

On-Demand (Default)

You share the hardware with entire world.

• Scaling: AWS handles it, but you are subject to "Throttling" if you hit your quota.
• Best for: Variable traffic, development, and small-to-medium apps.

Provisioned Throughput

You "Reserve" a specific amount of model capacity (like renting a whole highway lane just for yourself).

• Scaling: You are guaranteed a specific "Number of tokens per minute."
• Benefit: No throttling. No "Cold starts." Consistent latency.
• Cost: You pay a commitment (e.g., for 1 month or 6 months).

---

2. Scaling Amazon SageMaker (Auto-Scaling)

Because SageMaker runs on "Servers" (Instance-based), you must tell AWS how to add more servers as traffic grows.

• SageMaker Auto-Scaling: You set a "Target." For example: "Keep my CPU usage at 50%."
• When more users arrive and CPU hits 70%, SageMaker automatically launches 2 more servers and adds them to the cluster.
• When the users leave at night, the extra servers are deleted to save you money.

---

3. The "Financial Guardrails" of Scale

Scaling is dangerous because AI costs are Cumulative.

• If you don't set a Token Limit in your code, a single malicious user (or a bug in your loop) could generate 100 million tokens in an hour, costing you thousands of dollars.

Best Practice:

1. Set AWS Budgets Alerts to email you when costs hit $50.
2. Set a Prompt Length Limit in your application code.
3. Use Bedrock Guardrails to block long, nonsensical inputs.

---

4. Visualizing the Scaling Path

Traffic Type	Strategy	AWS Feature
Spiky / Experimental	Pay-as-you-go	Bedrock On-Demand
High Steady / Mission Critical	Reservation	Bedrock Provisioned Throughput
Server-based (Custom ML)	Elastic Growth	SageMaker Auto-Scaling
Massive Offline Jobs	Batch	SageMaker Batch Transform

graph LR
    A[Increase in Users] --> B{Service Type?}
    B -->|Bedrock| C{Exceeding Limit?}
    C -->|Yes| D[Move to Provisioned Throughput]
    C -->|No| E[Stay On-Demand]
    
    B -->|SageMaker| F[Deployment Config]
    F --> G[Set Auto-scaling Policy]
    G --> H[Add/Remove Instances based on CPU]

---

5. Summary: Scale with a Plan

Scaling is not a "Feature" you turn on at the end. It is a Business Decision.

• Over-Scaling: Wastes money (Idle servers).
• Under-Scaling: Loses customers (System crashes). A Practitioner uses CloudWatch Alarms to find the "Goldilocks" zone—just enough scale for the current demand.

---

Recap of Module 14

We have mastered the Operations of AI:

• We understood the Cost Drivers (Tokens/Instances).
• We balanced Latency vs. Accuracy.
• We identifies Infrastructure Risks.
• We learned how to Scale using Provisioned and Auto-scaling methods.

---

Knowledge Check

---

What's Next?

We have learned every technical and business pillar of the "AWS AI Practitioner." Now, it is time to put it all together. In Module 15: AI Strategy & Preparation, we look at the "Final Exam" mindset and how to build a career in AI.