Reasoning & extended thinking

Every major provider has shipped extended thinking, and every one of them shaped the API differently. OpenAI takes an effort enum. Anthropic takes a token budget — and changed the shape between model generations. Google takes an effort on one surface and a config object on another. Mistral has no dial at all.

Stav gives you one parameter and does the translation.

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Prove that √2 is irrational."}],
    reasoning_effort="high",
)

print(resp.choices[0].message.content)
print(resp.usage.completion_tokens_details.reasoning_tokens)

The parameter

reasoning_effort is canonical. reasoning and thinking are accepted aliases, so clients written against OpenRouter or Anthropic conventions work unchanged.

{ "reasoning_effort": "high" }
{ "reasoning": "high" }
{ "reasoning": { "effort": "high" } }
{ "thinking":  { "effort": "high" } }

Valid values: minimal, low, medium, high. Case and surrounding whitespace are normalised.

If more than one of the three fields is present, reasoning_effort wins, then reasoning, then thinking.

There is no "off" switch

Reasoning is on if one of the three fields is present, and off if none is. Sending {"thinking": {"type": "disabled"}} or {"reasoning": {"enabled": false}} is a 400, not a way to disable it — omit the field instead.

Likewise the native Anthropic budget shape is not accepted on /v1/chat/completions:

{ "thinking": { "type": "enabled", "budget_tokens": 4096 } }

{
  "error": {
    "message": "'thinking' must be one of ['minimal', 'low', 'medium', 'high'], or an object with a string 'effort' field, on /v1/chat/completions. Use 'reasoning_effort' for the simplest form.",
    "type": "invalid_request_error",
    "param": "reasoning_effort",
    "code": "invalid_value"
  }
}

If you want the native Anthropic contract, use /v1/messages — see below.

What each provider receives

The Anthropic budget mapping

On the models that take a token budget:

Anthropic requires the thinking budget to be strictly less than max_tokens, so Stav sizes max_tokens around the budget when you have not set one, and clamps the budget under your max_tokens when you have. If your max_tokens is too small to hold the minimum 1,024-token budget plus room for an answer, Stav raises it — the request would otherwise be rejected outright. Set a max_tokens of at least ~4,096 when you turn on high effort and you will never notice this.

Also on Anthropic: temperature and top_p are omitted while reasoning is active, because the provider rejects them in combination with extended thinking.

What comes back

{
  "usage": {
    "prompt_tokens": 42,
    "completion_tokens": 1310,
    "total_tokens": 1352,
    "completion_tokens_details": { "reasoning_tokens": 1180 }
  }
}

reasoning_tokens is a breakdown of completion_tokens, not an addition to it. In the example above, 1,180 of the 1,310 completion tokens were spent thinking and 130 became the visible answer. This matches the OpenAI convention.

The field appears only when the model actually spent thinking tokens. A reasoning-enabled request that turns out not to need any produces no completion_tokens_details at all.

The thinking text itself is not returned on /v1/chat/completions. You get the count, not the trace. include_reasoning is refused by name rather than silently ignored, so a client that expects the trace finds out immediately. If you need the reasoning content, use /v1/messages.

Streaming

Reasoning token counts arrive in the final usage chunk, which only appears if you asked for it:

stream = client.chat.completions.create(
    model="auto",
    messages=[…],
    reasoning_effort="medium",
    stream=True,
    stream_options={"include_usage": True},
)

Without include_usage, a streaming reasoning request reports no token counts at all — and reasoning tokens are usually most of the bill, so this is worth wiring up.

Native thinking on /v1/messages

claude-* models on /v1/messages are proxied verbatim to Anthropic. The native contract applies in full — including thinking blocks in the response content:

msg = anthropic_client.messages.create(
    model="claude-sonnet-4-5-20250929",
    max_tokens=8192,
    thinking={"type": "enabled", "budget_tokens": 4096},
    messages=[{"role": "user", "content": "Plan a migration from Postgres 14 to 17."}],
)

for block in msg.content:
    if block.type == "thinking":
        print("[thinking]", block.thinking[:

Thinking token counts from this path are captured on the request log too, so cost analysis is consistent across both surfaces.

Non-Claude models on /v1/messages are translated to the chat contract, and the native thinking object is refused there with an error naming the model that resolved.

Which models can reason

Support is a per-model flag in Stav's capability catalogue, validated by behavioural probing rather than inferred from family names — several "lite" models reason and several large ones do not.

info = client.models.retrieve("claude-sonnet-4-5-20250929")
info.model_extra["stav_supports_thinking"]   # True

The same flag is extended_thinking in stav_capabilities.

Pinning a model that cannot reason returns a 400 naming the capability, with alternatives:

{
  "error": {
    "error": "model_capability_error",
    "capability": "extended_thinking",
    "message": "Model 'gpt-4o-mini' does not support thinking mode.",
    "supported_models": ["…"]
  }
}

Reasoning and the router

Two things happen when you send reasoning_effort with model="auto" or a named router:

The pool is filtered to reasoning-capable models before scoring. Every failover candidate is reasoning-capable too, so a retry cannot land you on a model that would silently ignore the parameter.

The expected output length is scaled up by effort, which flows into both the cost estimate and the speed estimate. high effort assumes several times the token spend of a plain completion — so the router's idea of "cheapest" changes accordingly, which is the correct answer rather than a conservative one.

If nothing in your pool can reason, you get an error rather than a silent downgrade.

Cost

Reasoning tokens are counted inside completion_tokens and billed as output tokens. Some endpoints carry a distinct reasoning rate, which is used when set.

This is the practical consequence: effort is a cost dial, not a quality dial. Going from low to high on a hard problem is often worth it. Going from low to high on a summarisation task multiplies the bill for no benefit.

A pattern that works well:

def complete(prompt, hard=False):
    kwargs = {"reasoning_effort": "high"} if hard else {}
    return client.chat.completions.create(
        model="auto",
        messages=[{"role": "user", "content": prompt}],
        **kwargs,
    )

…and let the router pick a cheap fast model for the easy path, and a reasoning model for the hard one, from the same call site.

Use cases

Multi-step extraction from unstructured documents. medium effort plus a strict JSON schema. The thinking budget is what lets the model reconcile contradictions across a long document before committing to a value.

Code review and migration planning. high effort on /v1/messages with a Claude model, so you can show the reasoning trace to the engineer who has to approve the change. The trace is the deliverable as much as the answer.

Agent planning steps only. Turn effort on for the planner call and off for tool-result summarisation. Most agent loops spend the majority of their tokens on steps that do not benefit from thinking at all.

Do not enable reasoning globally as a quality setting. Measure it on your own evaluation set first — on many tasks the ranking does not change and the bill triples.

Next steps

• Request parameters — the full honour-or-refuse table
• Smart Router
• Anthropic SDK migration