Max Tokens vs. Stop Sequences: Hard Termination

You have optimized your prompt and your temperature. But what if the model still wants to write a 1,000-word essay? To achieve literal token safety, you must use Hard Termination Parameters.

In this lesson, we learn how to use max_tokens (The Blade) and stop_sequences (The Signal). We will explore why Max Tokens can be dangerous for structured data and why Stop Sequences are the ultimate tool for precision efficiency.

---

1. Max Tokens: The Token Blade

max_tokens is a hard limit set at the infrastructure level. After the model generates N tokens, the connection is severed.

• Pro: Guaranteed cost control. You literally cannot spend more than $X.
• Con: The "Cliff" Effect. If the model was in the middle of a JSON block, it will truncate: {"name": "Joh....
• The Efficiency Trap: You just paid for the 50 tokens, but the result is Un-parseable Junk. You have effectively "Burned" that money.

---

2. Stop Sequences: The Graceful Signal

A Stop Sequence tells the model: "As soon as you output this specific character, the turn is over."

Example:

• Prompt: "What is the capital of France? Answer with one word."
• Stop Sequence: . (The period).
• Behavior: The model outputs "Paris." It then sees the period, and and the generation Stops Immediately.

Token Saving: You prevent the model from adding "Paris is a beautiful city in..." which would have cost 10 extra tokens.

---

3. Implementation: Using Stop Sequences for Agents

In a multi-agent system, specialists (Module 12.1) should use stop sequences to signal they are done.

Python Code: Precision Stops

# Stopping an agent as soon as it tries to call a tool
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    stop=["Observation:", "Tool Result:", "###"] 
)

By adding "Observation:" as a stop sequence, you ensure that as soon as the agent writes its tool call, it stops. It cannot "Hallucinate" the result of the tool because the interface was severed as soon as it stepped out of its lane.

---

4. The "Length-Aware" Prompt

If you set max_tokens=100, you should also mention this in the prompt.

• Bad: "Explain relativity." (Max tokens 100). Result: Truncation.
• Good: "Explain relativity in exactly 2 sentences." (Max tokens 100). Result: Graceful finish.

The Rule: Your Linguistic Constraint must always be tighter than your Inference Constraint.

---

5. Visualizing the "Waste Gap"

When a model truncates due to max_tokens, it often returns a finish_reason: length.

graph LR
    A[Output: 100 tokens] --> B{Valid?}
    B -->|stop_sequence| C[Valid Result: 100% ROI]
    B -->|max_tokens| D[Truncated: 0% ROI]
    
    style D fill:#f66
    style C fill:#4f4

---

6. Summary and Key Takeaways

1. Max Tokens for Budget: Use it as a safety net, not a primary control.
2. Stop Sequences for Precision: Use markers like ., }, or \n to end generation as soon as the data point is complete.
3. Align Prompt and Parameter: Ensure your word count instructions match your token limits.
4. Reasoning on Truncation: If finish_reason == 'length', your system should be prepared to either "Accept the partial" or "Log an Efficiency Error."

In the next lesson, Frequency and Presence Penalties: Token Diversity, we look at چگونه to prevent "Circular Loops" from draining your budget.

---

Exercise: The Stop Challenge

1. Ask an LLM to "List 10 colors."
2. Run 1: No stop sequences.
3. Run 2: Set stop=[","].
4. Analyze: How many colors did Run 2 provide?

• (Result: Exactly 1).
• Calculate the Savings: How many tokens did you save by stopping after the first comma?
• Think: If you only needed the first item in the list, how much "Waste" did you have in Run 1?

---

Congratulations on completing Module 15 Lesson 2! You are now a termination specialist.