Designing for the Unknown

Traditional APIs are deterministic: if you send X, you get Y. AI-powered APIs are probabilistic: if you send X, you might get Y, or Y+1, or an apology. This uncertainty requires a new set of "Best Practices" for API design.

In the AWS Certified Generative AI Developer – Professional exam, you must prove you can build APIs that are secure, stable, and easy for other developers to consume.

---

1. Input Validation: Beyond the Regex

Before your user's input ever touches an LLM, it must pass through a "Guard Pipeline."

• Schema Validation: Use Amazon API Gateway with JSON Schema to ensure the input isn't 50MB of garbage.
• Prompt Injection Protection: Scan for keywords likes "Ignore all previous instructions."
• Rate Limiting: Use API Gateway Usage Plans to prevent a single user from consuming all your Bedrock tokens.

---

2. Enforcing Structured Output

A Professional API should not return a raw string of text. It should return JSON.

The "Constraint" Pattern

Instead of saying "Return a list of users," use the Anthropic/Meta Tool Use API or a strict system prompt: "Return a JSON object with keys 'user_id' (integer) and 'status' (string). Do not include any other text."

Why this matters:

If your backend code (Python/Node) tries to parse a string that says "Here is your JSON: { ... }", your code will crash. Enforcing structured output ensures the AI acts like a reliable microservice.

---

3. The Power of Semantic Caching

AI inferences are expensive and slow. If 1,000 people ask "How do I reset my password?", you shouldn't call Bedrock 1,000 times.

• Traditional Caching (Elasticache): Works only for exact matches.
• Semantic Caching: You store the vector of the question in a fast database (like Redis). If a new question is 99% similar to a previous one, you return the cached answer.

graph LR
    U[User Question] --> V[Vector Search: Redis]
    V -->|Match > 0.99| C[Return Cache]
    V -->|No Match| B[Call Bedrock]
    B --> S[Save to Redis]
    S --> C

---

4. API Response Codes for AI

Developers consuming your AI API need to know why it failed. Use these standard mappings:

Scenario	HTTP Code	Meaning
Success	200 OK	AI generated a valid response.
Throttling	429 Too Many	You have exceeded your quota/usage plan.
Timeout	504 Timeout	The AI took too long (switch to Async/Streaming).
Safety Block	400 Bad Request	Your prompt violated a Guardrail (PII/Hate).

---

5. Metadata and Observability in the API

Your API response should include "Cost Metadata":

{
  "answer": "...",
  "usage": {
    "prompt_tokens": 120,
    "completion_tokens": 45,
    "total_cost_usd": 0.003
  }
}

This allows the consuming application to track its own budget and notify the user if they are being too verbose.

---

6. Pro-Tip: The "Fallback" Logic

As an AWS Pro, you never trust a single model completely. API Best Practice: If Claude 3.5 Sonnet fails with an internal error, your integration layer (Lambda) should automatically catch the error and retry once with Claude 3.5 Haiku. A slightly "simpler" answer is always better than an "Error 500."

---

Knowledge Check: Test Your API Knowledge

---

Summary

API design for AI is about building a "Protective Shell" around the probabilistic model. Validate the inputs, structure the outputs, and cache the commonalities. In the next lesson, we will look at the final part of Module 7: Foundation Model Routing and Fallback Strategies.

---

Next Lesson: Resilience in the Dark: FM Routing and Fallback Strategies