Tool calling

Tool calling works the way it does with OpenAI: send tools, get tool_calls back, answer with a tool message. Coding agents that speak the OpenAI API run unchanged.

Katara sits between your client and a marketplace of providers running open models on their own hardware. The provider’s runtime is an inference engine, not the API. Every reply is checked and re-serialized before it reaches you, so what you receive is the same shape whichever provider served it.

What you receive

• One choice per reply. n above 1 is refused.
• tool_calls entries have an id starting with call_, type: "function", a non-empty function.name and function.arguments that parse as a JSON object.
• In a stream, each tool call arrives whole in a single chunk with its index. Text streams as it is produced; a call is held until it has been validated.
• finish_reason is tool_calls when a call was made, stop, length, or content_filter.
• usage is always present, on the final chunks of a stream too. The numbers are the settled receipt’s, not a runtime’s own report.
• Assistant text never contains a model’s raw tool syntax. If a model writes its call in its own markup instead of the structured form, Katara either recovers it into a proper tool_calls entry or refuses the reply. It is never passed through as text.

The tool contract

tools: true on a model means a certified contract, listed under capabilities.tool_contract on /v1/models:

Field	Meaning
tool_choice	The accepted tool_choice values. Most models accept auto only.
parallel_tool_calls	Whether more than one call per reply is certified.
strict_schemas	Whether strict: true on a tool is enforced by the runtime.
tool_streaming	whole-call: a call arrives in one chunk once validated.
max_argument_bytes	The largest argument document the certification exercised. 0 means untested.
certified_concurrency	How many tool requests at once were certified on the reference hardware.
agentic_certified	Whether the coding-agent regression flows passed.
metering_version	The billing rule the model is certified under.

Asking for something outside the contract returns 400 feature_not_supported with param naming the field (tool_choice, n, logprobs). Nothing is silently ignored.

Two flags are accepted whatever the contract says, because stock clients send them by default:

• parallel_tool_calls does not change how these models generate; the contract’s parallel_tool_calls tells you whether to expect several calls in one reply.
• strict: true on a tool is accepted but not enforced unless the contract’s strict_schemas is true. Schema enforcement needs constrained decoding in the runtime; without it a call arrives as the model wrote it, validated as a JSON object, and your client decides what to do with a missing optional parameter, as it would with any non-strict model.

When a model gets a call wrong

A small model can produce a call that cannot be executed: a truncated argument document, a tool that was never offered, arguments that are not JSON. Katara does not deliver those. The same applies to a model that has fallen into a loop and keeps repeating the same paragraph: the reply is cut off instead of running to the token cap.

• If nothing had been sent to you yet, the request is retried on another provider. You see one reply and pay for one reply.
• If text had already streamed, the stream ends with an error chunk with code provider_output_invalid. Nothing is charged for that attempt.
• If no provider produced a valid reply, the request fails with 502 provider_output_invalid and nothing is charged.

A provider whose model keeps producing calls that need recovery or refusal stops receiving tool requests, however cheap it is.

Example

from openai import OpenAI

client = OpenAI(base_url="https://api.staging.katara.com/v1", api_key=os.environ["KATARA_API_KEY"])
tools = [{"type": "function", "function": {
    "name": "write_file",
    "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}},
                   "required": ["path", "content"]}}}]
reply = client.chat.completions.create(
    model="katara/qwen3-coder-30b-a3b@1",
    messages=[{"role": "user", "content": "Write weather.py that prints Florida's weather."}],
    tools=tools,
)
call = reply.choices[0].message.tool_calls[0]
print(call.function.name, call.function.arguments)

Billing

A tool call is output you received, so it is billed: the tool name and the argument document count as output tokens, measured with the model’s own tokenizer. The wrapper syntax a model generates around a call is not billed, and neither is anything Katara refused. See Usage and cost.