Connecting the Dots

A Foundation Model is useless if it lives in isolation. As a Professional Developer, your job is to build the integration layer—the boilerplate, the security, and the logic that allows your web or mobile app to speak to the AI.

In this lesson, we will focus on the Application Architecture for GenAI, including the libraries you should use, how to handle errors, and why you should treat your prompts as managed assets.

---

1. The Developer's Library: Boto3 vs. Frameworks

When building on AWS, you have two primary paths for integration:

Path A: The AWS SDK (Boto3)

• Use Case: Low-level, high-control applications.
• Pros: Zero overhead, native AWS security, supports the newest Bedrock features immediately.
• Cons: You have to write all the "plumbing" (chaining, memory, chunking) yourself.

Path B: Orchrstration Frameworks (LangChain / LlamaIndex)

• Use Case: Complex agents, RAG systems, or multi-step reasoning.
• Pros: Pre-built "Bricks" for memory, tool-calling, and retrieval.
• Cons: Can be bloated; abstracts away AWS-specific optimizations.

Pro Developer Advice: For simple model calls, stick to Boto3. For complex agents, use LangChain or LangGraph.

---

2. API Design for AI

How does your frontend talk to your GenAI backend?

• Synchronous (REST): Simple request/response. Best for fast tasks like "Classify this text."
• Asynchronous (Event-driven): Submit to an SQS queue or S3. Best for long tasks like "Summarize this 20-minute video."
• Streaming (WebSockets / Server-Sent Events): Best for chat. Watching the words appear one by one (TTFT) significantly improves user Satisfaction.

---

3. Handling the "429" (Throttling)

In the Professional exam, you will definitely face questions about Error Handling.

When you hit a service quota, Bedrock returns a ThrottlingException (HTTP 429). The Solution: Implement Exponential Backoff with Jitter. Instead of retrying every 1 second, you wait 1s, then 2s, then 4s, plus a small random "jitter" to prevent a "thundering herd" of retries from crashing the service.

Code Example: Resilient Boto3 Call

import boto3
import time
from botocore.exceptions import ClientError

client = boto3.client('bedrock-runtime')

def resilient_invoke(prompt):
    max_retries = 3
    base_delay = 1
    
    for i in range(max_retries):
        try:
            # Assume body is already defined
            response = client.invoke_model(modelId='...', body=...)
            return response
        except ClientError as e:
            if e.response['Error']['Code'] == 'ThrottlingException':
                wait = (base_delay * (2 ** i)) # Exponential backoff
                print(f"Throttled. Retrying in {wait} seconds...")
                time.sleep(wait)
            else:
                raise e
    return None

---

4. Prompt Management and Versioning

NEVER hardcode your prompts inside your Python files. Why? Because prompts change weekly as you find better ways to "steer" the model.

The Pro Path:

1. Store prompts in AWS Systems Manager (SSM) Parameter Store or AWS Secrets Manager.
2. Store complex prompt templates in S3.
3. Use Amazon Bedrock Prompt Management (A new feature that allows you to version and test prompts in the console before deploying them to code).

graph LR
    A[Code: 'Get Prompt v2'] --> B[SSM Parameter Store]
    B --> C[Prompt: 'You are a helpful assistant...']
    C --> D[Code: call_bedrock(C)]

---

5. A/B Testing Your Models

A Professional Developer doesn't "guess" which model or prompt is better. They use Cross-Region Replication or Blue/Green deployments to test two versions.

• Version A: Claude 3.5 Sonnet with Prompt 1.
• Version B: Claude 3.5 Haiku with Prompt 2.
• Measurement: Measure the CSAT (Customer Satisfaction) or the correctness using Amazon Bedrock Model Evaluation.

---

Knowledge Check: Test Your Integration Knowledge

---

Summary

Integration is where your engineering skills shine. By managing your prompts as assets and building resilient retry logic, you create an application that survives real-world traffic. In the next lesson, we will dive into the specific Request/Response Patterns: Sync, Async, and Streaming.

---

Next Lesson: Speed vs. Substance: Sync, Async, and Streaming Patterns