Logging and Structured Logs: The API's Memory

In your development environment, you use print(). In production, print() is useless. You need a centralized, searchable, and structured record of everything that happens on your server.

In this lesson, we master Logging—the primary tool for debugging production crashes.

---

1. Why logging instead of print()?

1. Levels: You can categorize messages (DEBUG, INFO, WARNING, ERROR, CRITICAL).
2. Destinations: You can send logs to a file, to the console, or to an external service like Datadog or ELK.
3. Metadata: Every log automatically includes the timestamp, the file name, and the line number where it happened.

---

2. Structured Logging (JSON)

Standard logs are just strings: "User 123 logged in". Structured Logs are JSON: {"user_id": 123, "event": "login", "status": "success"}.

Why JSON Logs?

When you have 10 million log lines, you can't read them. You need to search them. JSON logs allow you to use tools like Kibana or Grafana to run queries like: "Show me all 'ERROR' logs for User 123 in the last 10 minutes."

---

3. Implementing Logging in FastAPI

We use the standard Python logging library or a modern alternative like loguru.

import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

@app.post("/items/")
async def create_item(item: Item):
    logger.info(f"Creating item: {item.name}")
    try:
        # DB logic
        return {"status": "ok"}
    except Exception as e:
        logger.error(f"Failed to create item: {str(e)}", exc_info=True)
        raise HTTPException(status_code=500, detail="Internal Error")

Note: exc_info=True automatically attaches the full "Traceback" (the lines of code that failed) to your log.

---

4. Middleware Logging

A pro move is to use Middleware (Module 8) to log every single request and response automatically.

@app.middleware("http")
async def log_requests(request: Request, call_next):
    logger.info(f"Incoming: {request.method} {request.url.path}")
    response = await call_next(request)
    logger.info(f"Outgoing: Status {response.status_code}")
    return response

---

Visualizing Log Management

graph LR
    A["FastAPI App"] --> B["Logger (JSON Format)"]
    B --> C["Log Collector (Vector/Fluentd)"]
    C --> D["Search Engine (Elasticsearch)"]
    D --> E["UI Dashboard (Kibana/Grafana)"]

---

Summary

• Log Levels: Use them to filter noise.
• Structured Logs: JSON is the language of production monitoring.
• exc_info: Never log an error without the traceback.
• Centralization: Logs should flow away from the server to a searchable database.

In the next lesson, we’ll move from text to numbers: Metrics and Dashboards.

---

Exercise: The Error Hunter

You see a generic "500 Internal Server Error" in your browser.

1. Where should you look first to find the cause?
2. If your logs only say "Error: db failed", why is that not enough for a developer to fix the bug?