Handling Malformed Output: The Graceful Pivot

Even with JSON Mode (Module 13.1) and Pydantic (Module 13.3), models occasionally fail. A character gets dropped, a quote isn't closed, or the model hallucinates a trailing comma.

The Inefficient Solution: Ask the model to "Please try again and fix the JSON." This costs another 2,000 tokens for the prompt and response.

The Efficient Solution: Use Deterministic Fixing (Regex/Parsers) and Micro-Retries. We want to fix the data in Python for "Free" before we spend pennies on an LLM retry.

---

1. Deterministic Fixing (The 'Free' Tier)

Most "Malformed JSON" is just a missing brace or a trailing comma. You should use a library like dirty-json or json-repair to fix these in YOUR code.

Python Code: The json-repair Pattern

from json_repair import repair_json
import json

raw_output = '{"name": "John", "age": 30,' # Note the missing brace

# 0 Token Cost! This happens on your CPU.
fixed_json = repair_json(raw_output)
data = json.loads(fixed_json)
print(data['name']) # "John"

---

2. The "Short-Refactor" Retry (The 'Cheap' Tier)

If the JSON is truly broken (not just a syntax error), do not re-send the whole prompt.

Inefficient Retry: "Here is the 10-page document again. Extract the info as JSON. The last one was broken." Efficient Retry: "The following JSON is missing a 'price' field. Please ONLY output the missing 'price' field. JSON: [BROKEN_JSON]"

Savings: By only asking for the Delta (the missing part), you save 90% on the input tokens of the retry.

---

3. The "Two-Step" Validation Strategy

1. Step 1: Use pydantic to validate the object.
2. Step 2: If it fails, use a Regular Expression to see if the value exists in the raw text string.

Often, the LLM generated the correct info but the wrong format. If you can find the info with a Regex, you can build the JSON yourself in Python and skip the retry entirely.

graph TD
    A[Raw Model Output] --> B{JSON Valid?}
    B -- Yes --> C[Done]
    B -- No --> D[Try Python 'json-repair']
    D --> E{Valid now?}
    E -- Yes --> C
    E -- No --> F[Regex Search for Key Data]
    F --> G{Found?}
    G -- Yes --> H[Reconstruct JSON in Python]
    G -- No --> I[LLM Retry: SMALL PROMPT]
    
    style H fill:#4f4
    style I fill:#f66

---

4. Token ROI: Repair vs. Replace

• Replacement Cost (Retry): 2,000 tokens ($0.06).
• Repair Cost (Local): 0 tokens ($0.00).
• Extraction Logic: If your app processes 10,000 failed JSONs a day, json-repair saves you $600 a day.

---

5. Summary and Key Takeaways

1. Fix it Locally: Use json-repair to handle syntax errors on your own CPU.
2. Extract with Regex: If the format is broken but the data is there, don't ask the LLM to fix it—grab it yourself.
3. Delta Retries: If you must go back to the LLM, only send the broken fragment, not the whole context.
4. State Isolation: Don't carry "Broken History" into the next turn. Delete the failure before the model sees it.

In the next lesson, Minimizing Verbosity in Structured Data, we conclude Module 13 by looking at چگونه to make even our "Fixed" JSON smaller.

---

Exercise: The Repair Lab

1. Create a "Manually Broken" JSON string (e.g. {"id": 1, name: "test").
2. Install json-repair and try to fix it.
3. Analyze: Did it work?
4. Now, try to use a Regex to find the name "test" from that same broken string.
5. Reflect: Which of these two methods is more "Resilient" to unexpected AI formatting?

• (Hint: Regex works even if the AI decides to write "The name is test" inside the JSON block).

---

Congratulations on completing Module 13 Lesson 4! Your systems are now error-resilient.