Routing hints
Per-request fields that shape routing without changing your policy — a session id that keeps an agent loop on one model, and a provider filter that says where a request may run. Hints only narrow; none widens.
Your workspace policy and Routers decide what a request may use. Routing hints let one request add detail on top. They never widen what your policy allows.
No hint is forwarded to a provider. Stav reads them and strips them before the request goes upstream, so they cannot change the prompt or its cache prefix.
session_id — name the conversation
resp = client.chat.completions.create(
model="auto",
messages=[...],
extra_body={"session_id": "conv-8f21"},
)
Tells the Smart Router which conversation this request belongs to, so it can remember what it chose last turn. That is what lets an agent loop stay on the same model while tool calls are outstanding (see the continuity lock in Smart Router).
Accepted on /v1/chat/completions, /v1/messages and /v1/responses.
Stav resolves the conversation identity in this order and uses the first one present:
session_idin the body- the
X-Session-Idrequest header prompt_cache_keyin the body- a fingerprint of the conversation's opening
The value is hashed together with your workspace and API key, so two customers sending "default" never share state.
Responses may carry an X-Session-Id header. Send it back on the next request if you want session tracking to be exact rather than inferred.
provider — where a request may run, and who may serve it
provider narrows one request on top of your workspace policy. It filters Provider Endpoints, so the endpoint that serves and every failover stay inside it. It works on every inference surface, for a named model, auto and a Router alike:
client.chat.completions.create(
model="mistralai/mistral-small-4-119b-2603",
messages=[...],
extra_body={"provider": {
"runs_in": "eu",
"ignore": ["deepinfra"],
"max_price": {"prompt": 2.0, "completion": 6.0},
}},
)
| Field | Effect | Surfaces |
|---|---|---|
runs_in (alias region) | "eu", "eea", a country code, or a list of country codes | All |
only | Only endpoints of these Inference Providers serve | All |
ignore | No endpoint of these Inference Providers serves | All |
max_price | The highest price an endpoint may charge per million tokens, in your workspace's billing currency: prompt, completion. Every rate the request can be billed at counts, including a conditional rate for a long prompt or an image and the reasoning output rate. A service tier or cache write you ask for does not. An endpoint with an unknown price does not pass | Chat surfaces |
quantizations | Only endpoints serving one of these quantizations (fp8, bf16, …). Read from the endpoint, else the model. An endpoint with an unknown quantization does not pass | Chat surfaces |
sort | Order a named model's endpoints: "price" (cheapest for this request first), "latency" (fastest first token first) or "throughput" (most tokens per second first). Failover follows the same order | Chat surfaces, named model |
A client that can set fixed headers but not body fields can send runs_in as the X-Stav-Runs-In header instead (eu, eea, NO, NO, SE). Sent with provider.runs_in, both must hold.
If no endpoint passes, the request is refused rather than served elsewhere. See Choose where a request runs for every surface and every refusal.
sort only orders the endpoints the other fields and your policy admit; it never adds one. With model="auto" or a Router it is a 400 unsupported_parameter: there the Router's objective chooses. Without sort, Stav picks a named model's endpoint by live speed.
client.chat.completions.create(
model="deepseek-ai/deepseek-v4-1-flash",
messages=[...],
extra_body={"provider": {"sort": "price", "runs_in": "eea"}},
)
require_parameters, allow_fallbacks and any other key are not supported. Sending one is a 400 unsupported_parameter naming the field.
Seeing what happened
| Header | Says |
|---|---|
X-Stav-Model | the model that served |
X-Stav-Route-Reason | why it was chosen |
X-Stav-Route-Lock | the continuity lock that held the route (tool_loop, signed_thinking), or none |
X-Session-Id | the conversation session id, when one was resolved |