Deterministic vs Non-Deterministic: The Agentic Gap

As an engineer, you are used to Deterministic systems. If you write x = 1 + 1, x will always be 2. You can test it, rely on it, and build predictable apps with it.

Agentic AI introduces Non-Determinism. Because LLMs have a "Temperature" (creativity setting), the same prompt might lead the agent down a different path every time you run it.

1. Comparing the Two Worlds

System Type	Input	Process	Output	Reliability
Deterministic (Code)	Fixed	Fixed Path	Guaranteed	100%
Non-Deterministic (Agent)	Fixed	Dynamic Logic	Probabilistic	85-95%

---

2. The Agentic Challenge

In an agentic system, the LLM decides which code to run.

• The Risk: The agent might pick tool_a today and tool_b tomorrow for the exact same request.
• The Consequence: Testing becomes difficult. How do you provide a "Green Checkmark" in CI/CD if the logic changes every time?

---

3. Strategies for Control

To build professional agents, we must strive to make them as deterministic as possible.

A. Temperature 0

Always set your LLM temperature to 0 or 0.0 for agentic workflows. This forces the model to pick the most likely "Next Step," reducing random wandering.

B. Output Schemas (JSON)

Don't ask the agent to "Write a plan." Ask the agent to "Output a JSON array of steps." This forces the non-deterministic brain into a deterministic, parsable box.

{
  "thought": "I need to check the weather.",
  "action": "weather_api",
  "params": {"city": "Paris"}
}

C. Constraint Prototyping

Instead of letting the agent wander the world, give it a limited "Graph" of possibilities (we will cover this in Module 6 with LangGraph).

---

4. Why Use Non-Determinism at All?

If non-determinism is "dangerous," why not just write traditional code? The Answer: Resilience.

• The Code Way: If an API changes its response format slightly, your code crashes.
• The Agent Way: The agent sees the format change, "reasons" about the new structure, and adapts its extraction logic on the fly.

Agents trade Predictability for Creativity and Robustness.

---

5. Summary Mental Model

graph LR
    Input[User Input] --> Decoder{Dynamic Agent Logic}
    Decoder --> PathA[Path A: 70% chance]
    Decoder --> PathB[Path B: 25% chance]
    Decoder --> PathC[Path C: 5% chance]
    PathA --> Result[Success]
    PathB --> Result[Success]
    PathC --> Failure[Hallucination]

Our goal in this course is to increase the percentage of Successful Paths while minimizing the Failure percentage, even if the exact path taken varies.

---

Key Takeaways

• Determinism is about "always the same path"; Non-determinism is about "reaching the same goal through different paths."
• LLMs are inherently non-deterministic.
• Use Temperature 0 and JSON schemas to add stability.
• The value of agents is their ability to handle unstructured scenarios where traditional code would break.