Understanding Thinking Effort in AI Agents: What Developers Need to Know

Understanding Thinking Effort in AI Agents

Part I · Challenge common assumptions

Why asking an AI to think harder does not always produce better results.

What does a thinking-effort setting actually change, and when should you raise it? This deck follows that question through 21 slides: the concept, the difference between reasoning and agent execution, real engineering scenarios, a production architecture, and a way to decide with evidence instead of intuition.

Same Model. Same Prompt. Different Outcomes?

Part I · Challenge common assumptions

You run what looks like the same request twice and get a clean answer once and a confused one the next time. Before blaming a setting, list everything that moved. Some tasks have one obvious path and are solved directly. Others have several plausible paths, and the answer improves when the system deliberates or collects more evidence.

• Task complexity and ambiguity change how many reasonable answers exist.
• Available context and tool outputs decide what the model can actually know.
• Sampling randomness means two runs of the same prompt can diverge.
• Model behavior and configuration matter, and effort is only part of configuration.

Higher Thinking Effort Does Not Automatically Mean Better Intelligence

Part I · Challenge common assumptions

Three assumptions are common, and each is wrong in a useful way:

1. “Higher effort always gives a better answer.” It can help on some tasks. It does not guarantee correctness, and on easy tasks it mostly adds cost.
2. “Longer thinking means deeper, more accurate reasoning.” Length is not quality. A long chain of reasoning can be long and wrong.
3. “More agent steps make a more capable agent.” More steps can compound errors as easily as they fix them.

Thinking Effort Is About Reasoning Resources, Not Just Waiting Longer

Part II · Technical fundamentals

Thinking effort is a model- or platform-specific control, where one exists, that influences how much effort goes into reasoning about a task. It is a setting on the public API. It is not a window into the model.

So the same word means different things on different APIs. Do not assume a fixed number of tokens, steps or compute per level.

Task Complexity Should Influence Reasoning Effort

Part II · Technical fundamentals

Task A converts a JSON object into a TypeScript interface. The output is checkable at a glance. Extra deliberation rarely changes it.

Task B is an intermittent race condition in a distributed system. Here deliberate reasoning can pay off: comparing hypotheses, considering edge cases, and connecting behavior across components.

But complexity is not the only axis. A short task with severe consequences, such as a production data migration, can justify careful reasoning, extra checks, or human review even though it is simple to state.

Model Reasoning and Agent Execution Are Different Things

Part II · Technical fundamentals

Thinking effort is the reasoning applied during one model invocation, when the model and API support such a control.

The agent loop is the external cycle: the model receives context, produces a response or a tool call, receives the tool result, and continues until a stopping condition is met (ReAct is the classic description of interleaving reasoning with actions).

They vary independently. One invocation can involve a lot of reasoning and no tools. An agent can make dozens of tool calls while each individual step is simple.

An AI Agent Is More Than a Model

Part II · Technical fundamentals

The conceptual flow: user task and context → model reasoning → decision to answer or call a tool → tool execution → updated context → next decision. It repeats until the task is done or a stopping condition fires.

Effort acts on one step of this loop, the reasoning inside the model call. Overall performance also depends on context quality, which tools exist, how the harness orchestrates retries and stopping, and whether anything verifies the result. Slide 15 turns this loop into a production reference architecture.

Different Effort Levels Represent Different Trade-Offs

Part II · Technical fundamentals

	Lower effort	Higher effort
Latency	tends to be lower	tends to be higher
Cost	tends to be lower	tends to be higher
Hard, multi-step tasks	may be less thorough	may be more thorough
Simple tasks	usually fine	little gain, still costs more

These are tendencies, not guarantees. How large each gap is depends on the model, the task and the metric. This article gives no numeric improvements because none would generalize; slides 16 and 17 show how to produce your own.

When Is Higher Thinking Effort Worth It?

Part III · Real engineering scenarios

An API intermittently returns HTTP 500. A model with more reasoning effort may be better at holding several hypotheses at once, checking dependencies and considering edge cases. It still cannot read logs nobody collected.

A good workflow alternates reasoning with observation:

1. Generate hypotheses, and say which evidence would confirm or refute each.
2. Collect that evidence (logs, metrics, traces).
3. Update the hypotheses. Drop what the evidence refutes.
4. Apply a fix, then verify the symptom is gone.

A One-Line Change and a System-Wide Refactor Do Not Need the Same Strategy

Part III · Real engineering scenarios

• Rename a local variable. Mostly mechanical. A compiler or the test suite settles it.
• Refactor authentication middleware. Reasoning has to cover callers, security implications, backward compatibility and the tests that prove behavior is preserved.
• Investigate a regression after deployment. Reasoning is only as good as what it can read: the diff, the logs, the test results and the recent change history.

Choose the Right Mechanism for the Problem

Part III · Real engineering scenarios

When an agent gets something wrong, “think harder” is one of several levers, and not always the right one.

• If the model has not weighed the available hypotheses, more reasoning effort may help.
• If it lacks repository information, a search tool will help more than more thinking.
• If a test result contradicts the initial hypothesis, another agent iteration with that result is the right move.
• If the task is too broad, decomposition helps more than any single setting.

A Convincing Explanation Is Not Proof of Correctness

Part III · Real engineering scenarios

Models can produce plausible but wrong conclusions because of incomplete information, faulty assumptions or other limits. The fluency of an explanation says little about whether it is true.

For coding agents, verification means concrete mechanisms: automated tests, type checking and linting, reading the generated diff, inspecting logs and execution results, checking acceptance criteria, and running security checks where appropriate. Evaluate the quality of those checks independently of how deep the reasoning looked.

Quality, Latency, and Cost

Part IV · Production systems

• Quality: correctness, task completion, robustness and edge-case handling.
• Latency: response time, and the total time to complete a task.
• Cost: model usage, reasoning-related inference cost where it applies, tool calls, and above all the total cost per successful task.

The goal is not maximum quality at any price, or minimum latency at any cost. It is an acceptable trade-off for the workload and its business requirements. The three do not reduce to one universal score.

Cost has three layers, and each answers a different question:

Layer	What it counts	Where the number comes from
Model call	Input, cached input, output and reasoning tokens, times the provider’s per-token prices	The usage object of each response, and the provider’s pricing page
Agent run	Every model call in the run, plus tool execution, retries, sandbox or CI time, and human review	Your traces and infrastructure bills
Task outcome	Total spend and time, divided by the tasks that passed verification	Your evaluation results

Effort acts on the first layer, but the decision belongs to the third. A setting that makes each call cheaper can make the task more expensive if runs fail more often and need retries, or a person, to finish.

Do Not Default to Maximum Effort for Every Request

Part IV · Production systems

An adaptive strategy treats effort as a per-task decision: receive the task, assess complexity, uncertainty and risk, select a supported setting, execute, verify, and escalate, retry or gather evidence if verification fails.

Routing can use rules, a classifier, a model-based router, or a feedback-driven policy.

Two cautions. Task classification can itself be wrong, so evaluate the router like any other component. And use only effort levels the chosen model or API supports.

A Reference Architecture for the Agent Loop

Part IV · Production systems

Concern	Proposed control
Runaway loops	Step, token and wall-clock budgets, checked before every model call
Unproductive loops	Stop when the same tool call with the same arguments repeats, or the same check keeps failing
Tool failures and timeouts	A timeout per tool. Retry only idempotent tools. Return errors and timeouts to the model as results; do not hide them
Consequential actions	A policy per tool: allow read-only tools, ask a person before writes, deploys, deletes or payments, deny the rest. Run code in a sandbox
Context	Append each tool result with its status; truncate large outputs and keep a pointer to the full artifact
Observability	One trace span per model call, tool call and check, recording the effort setting and token usage
Final result	Accept only what passes the verifier

const LIMITS = { maxSteps: 30, maxOutputTokens: 400_000, maxWallClockMs: 15 * 60_000, maxRepeats: 3 };

async function runTask(task: Task): Promise<RunResult> {
  const ctx = newContext(task);
  const startedAt = Date.now();
  let outputTokens = 0;

  for (let step = 1; step <= LIMITS.maxSteps; step++) {
    if (outputTokens > LIMITS.maxOutputTokens || Date.now() - startedAt > LIMITS.maxWallClockMs) {
      return { outcome: "stopped", reason: "budget_exhausted", suspected: "orchestration" };
    }

    // The only place the effort setting acts.
    const turn = await trace("model_call", () => callModel(ctx, { effort: chooseEffort(task) }));
    outputTokens += turn.usage.outputTokens;

    if (turn.kind === "answer") {
      const check = await trace("verify", () => verify(task, turn.answer));
      if (check.passed) return { outcome: "verified", answer: turn.answer, check };
      ctx.append({ type: "verification_failed", findings: check.findings });
      continue;
    }

    const call = turn.toolCall;
    if (repeatCount(ctx, call) >= LIMITS.maxRepeats) {
      return { outcome: "stopped", reason: "no_progress", suspected: "reasoning" };
    }

    const policy = toolPolicy(call); // "allow" | "ask" | "deny"
    if (policy === "deny" || (policy === "ask" && !(await askHuman(call)))) {
      ctx.append({ type: "tool_denied", call });
      continue;
    }

    // Retry only idempotent tools; errors and timeouts go back to the model as results.
    const result = await trace("tool_call", () =>
      runTool(call, { timeoutMs: 60_000, retries: call.idempotent ? 2 : 0 }),
    );
    ctx.append({ type: "tool_result", call, result });
  }

  return { outcome: "stopped", reason: "step_limit", suspected: "orchestration" };
}

Label every failed run with the class that broke first, by reading its trace. Reasoning: the facts were in context and the plan was still wrong. Tool execution: an error, a timeout, or bad output. Orchestration: lost context, a loop, or an exhausted budget. Verification: a check was missing, flaky or wrong. The suspected field in the code is only a first guess; the trace decides.

Measure Whether Higher Effort Actually Helps

Part IV · Production systems

Build an evaluation set that represents your real workload: real tasks, each with a verifier written before you look at any results. Then compare the settings your model supports under conditions that differ only in effort.

Hold everything else constant. Pin and record:

• the model snapshot ID and the SDK version;
• the system prompt, tool definitions and harness version;
• the context each task starts from: repository commit, fixtures, retrieved documents;
• the output cap (max_output_tokens or max_tokens) and the loop budgets from slide 15;
• the verifier and its pass criteria.

Change only the effort value, use only values the model documents, and interleave the settings in time so that a provider-side change or a slow hour does not land on one setting.

Run each task several times per setting. Two runs of the same prompt can differ, so one run per task measures luck as much as the setting. Fix the number of runs before you start, compare settings task by task on the same tasks, and report an interval, not only an average. For success rates, a binomial interval such as Wilson’s works; for latency and cost, bootstrap over tasks.

Do not judge success by answer length or reasoning-token count: both measure effort spent, not quality delivered.

A Benchmark Template You Can Fill In

Part IV · Production systems

Define every metric before the first run, so the numbers cannot be chosen after you see them:

Metric	Definition	Watch for
Verified task success rate	Runs whose final result passed the verifier, divided by all runs	The headline number
First-attempt success rate	Runs that passed on the first answer, with no verification feedback and no retries, divided by all runs	How much of the success comes from the loop’s recovery
Test pass rate	Passing tests divided by tests run, at the end of each run	Partial credit; never a substitute for verified success
Latency, median and P95	Wall-clock time from task start to final result, per run	P95 needs many runs before it means anything
Cost per successful task	Total spend over all runs, failed ones included, divided by verified successes	Add tool and infrastructure cost where you can
Average retry count	Model calls made after a failed check or a tool error, per run	Kept separate from total model calls

Why measure instead of assuming? Higher effort can help on tasks that need more deliberation, and it also tends to add latency and output tokens. Which effect wins depends on the model, the task mix and the strength of your verifier, and it can change when the model does.

Make Reasoning Effort an Explicit Engineering Decision

Part IV · Production systems

Where a provider supports it, effort is a request parameter. Here is an OpenAI Responses API request in the documented shape, with the reasoning-token count read back from the response:

import OpenAI from "openai";

const client = new OpenAI();

const response = await client.responses.create({
  model: "gpt-6-astra",
  reasoning: { effort: "high" }, // the setting this slide is about
  input: "Analyze this production bug...",
});

console.log(response.output_text);
console.log(response.usage?.output_tokens_details.reasoning_tokens);

The setting expresses how much reasoning to apply. It does not return the model’s private reasoning; for that, only a summary is available, opt-in.

The same request on Anthropic’s Messages API and on the Gemini Interactions API:

import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();

const message = await anthropic.messages.create({
  model: "claude-opus-5-5",
  max_tokens: 16000, // thinking counts toward this cap
  output_config: { effort: "high" },
  messages: [{ role: "user", content: "Analyze this production bug..." }],
});

console.log(message.usage.output_tokens_details?.thinking_tokens);

import { GoogleGenAI } from "@google/genai";

const gemini = new GoogleGenAI({});

const interaction = await gemini.interactions.create({
  model: "gemini-3.8-flash",
  input: "Analyze this production bug...",
  generation_config: { thinking_level: "high" },
});

console.log(interaction.output_text);
console.log(interaction.usage?.total_thought_tokens);

Similar names, different contracts. There is no shared format:

Provider and API	Request field	Values in the docs	Reasoning usage field	Source
OpenAI, Responses	reasoning: { effort }	none, minimal, low, medium, high, xhigh, max	usage.output_tokens_details.reasoning_tokens	Reasoning models
Anthropic, Messages	output_config: { effort }	low, medium, high, xhigh, max	usage.output_tokens_details.thinking_tokens	Effort, Steering thinking
Google, Interactions	generation_config: { thinking_level }	minimal, low, medium, high	usage.total_thought_tokens	Thinking
Google, generateContent	config.thinkingConfig.thinkingLevel in the JavaScript SDK	the same four levels	usageMetadata.thoughtsTokenCount	API reference; field placement checked in the SDK types

Each list is the union across models; a given model accepts only part of it. And equal names do not mean equal behavior. high on one model is not calibrated against high on another, even from the same vendor.

A Starting Heuristic for Choosing Effort

Part V · Practical takeaways

Factor	Points to lower effort	Points to higher effort	Often the better lever
Task complexity	One clear path, a local change	Many interacting parts, several plausible paths	Decompose a task that is broad rather than deep
Uncertainty and ambiguity	A precise spec	Competing hypotheses to weigh	Ask a clarifying question when the spec, not the reasoning, is ambiguous
Context	Everything needed is in the prompt	Complete but intricate context	Retrieve files, logs or docs when facts are missing
Need for outside information	None	Not a reason on its own	Call a tool; effort cannot produce facts
Cost of a wrong result	Cheap to undo	Expensive or irreversible	Human approval and stronger checks, alongside effort
Automated verification	Strong, fast tests, so iterating is cheap	Weak or no checks, so the first answer matters more	Write the missing test before raising effort
Latency and cost budget	Interactive, high volume, tight budget	Offline or batch work with high value per task	Route per task (slide 14) instead of one global setting

How to use it: start each route at the level the provider’s guidance recommends for that kind of workload, or at the documented default. Raise effort only on routes where traced failures are mostly reasoning failures (slide 15) and the verifier is strong enough to show a difference. Lower it where verified success holds at the lower level. Then run the benchmark from slides 16 and 17, and let the numbers overrule the table.

Before Increasing Thinking Effort, Ask These Five Questions

Part V · Practical takeaways

Keep this as a reusable decision card: if you cannot answer yes to question 1 and question 5, you probably should not raise the setting yet.

The Best Agent Is Not the One That Thinks the Most. It Is the One That Allocates Effort Where It Matters.

Part V · Practical takeaways

1. Thinking effort can influence how a model processes a task. It does not guarantee correctness.
2. Reasoning, tool use, context retrieval and verification solve different problems.
3. The right configuration comes from evaluating your own workload.

Instead of asking “How can I make the AI think more?”, ask “How much reasoning does this task need to be completed reliably?”

References and further reading

All provider pages were last checked on October 22, 2026.

• OpenAI, Reasoning models: reasoning.effort, reasoning tokens and summaries. GPT-6 Astra model page: supported effort values.
• Anthropic, Effort and Steering thinking: effort levels and defaults per model, adaptive thinking, thinking tokens and cost.
• Google, Gemini thinking: thinking_level on the Interactions API, levels per model, thought summaries and thinking-token usage. generateContent reference: usageMetadata.thoughtsTokenCount.
• SDKs used for the type checks: openai 7.42.0, @anthropic-ai/sdk 1.1.0, @google/genai 2.30.0.
• Anthropic, Building effective agents (December 2024): workflows, agents and when to add complexity.
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022).
• Snell et al., Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (2024).
• Chen et al., Reasoning Models Don’t Always Say What They Think (2025).