Concepts

Tools & structured output

Function calling and JSON Schema output across every dialect in the catalogue, with the router only picking models that can serve them.

Send the OpenAI tools array and response_format as you already do. Stav translates them for each Inference Provider, and when you route, only models that support them are considered.

Function calling

import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_invoice",
        "description": "Look up an invoice by number.",
        "parameters": {
            "type": "object",
            "properties": {"invoice_no": {"type": "string"}},

Loading the catalogue…

Tools & structured output · Stav Docs · Stav AI

messages.append(resp.choices[0].message)
messages.append({
    "role": "tool",
    "tool_call_id": call.id,
    "content": json.dumps(lookup_invoice(call.function.arguments)),
})
final = client.chat.completions.create(model="auto", messages=messages, tools=tools)

Parameter	Behaviour
`tools` (`type: function` or `custom`)	Translated for OpenAI-compatible, Anthropic, Google and Mistral models
`tool_choice`	Honoured on all dialects. On Anthropic models it travels with `tools`, so it does nothing on its own
`parallel_tool_calls`	OpenAI-compatible models only. Elsewhere: `400` naming the parameter
`allowed_tools`	OpenAI-compatible models only. Restricts which tools may be called without changing the `tools` array, so the prompt cache stays warm
Hosted tools (`web_search`, `file_search`, `code_interpreter`, …)	Left out and listed in `X-Stav-Dropped-Params`. No provider on this surface runs them

{"type": "text"}                                  # default
{"type": "json_object"}                           # any valid JSON
{"type": "json_schema", "json_schema": {...}}     # schema-conformant, strict only

resp = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": contract_text}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "clause_extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "governing_law": {"type": "string"},
                    "termination_days": {"type": "integer"},
                    "auto_renew": {"type": "boolean"},
                },
                "required": ["governing_law", "termination_days", "auto_renew"],
                "additionalProperties": False,
            },
        },
    },
)
data = json.loads(resp.choices[0].message.content)

	`json_object`	`json_schema`
OpenAI-compatible	Yes	Yes
Mistral	Yes	Yes
Google	Yes	Yes
Anthropic	No — `501`, use `json_schema`	Yes (native structured outputs)

resp = client.chat.completions.create(
    model="auto",
    messages=messages,
    tools=tools,
    response_format={"type": "json_schema", "json_schema": schema},
    reasoning_effort="medium",
)

Tools & structured output

Function calling

Tools & structured output

Function calling

Routing with tools

Structured output

JSON Schema

Support by dialect

Routing with schemas

`grammar`

Combining tools, schemas and reasoning

When a capability needs another API

Prompt caching

Other surfaces

What Stav does not provide

Next steps