Providers
Dispatch routes requests across these providers and fails over between them without changing the model. Restrict or order providers with team Directives. Prices are gateway list prices before the platform fee.
| Provider | Context | Max output | Latency | Throughput | Input | Output | Cache | Capabilities | ZDR | No training | Regions | Status |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
 Amazon Bedrock | 300K | 8K | 0.4 s | 160 tps | $0.060 | $0.24 | Read $0.015Write — | 4 features | Yes | Yes | EU | Certified |
Uptime
24 hoursRequest success rate per provider.
Amazon Bedrock100.00%Throughput
last hourP50 output tokens per second.
Amazon Bedrock160 tpsLatency
last hourP50 time to first token.
Amazon Bedrock0.4 sGetting started
Create a key on the API Keys page and set it as IB_API_KEY. See the quickstart.
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.inferencebureau.com/v1",
apiKey: process.env.IB_API_KEY,
});
const res = await client.chat.completions.create({
model: "amazon/nova-lite",
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(res.choices[0].message.content);Top-level parameters
Standard Chat Completions parameters work as documented. These have model-specific behavior.
| Parameter | Type | Required | Description |
|---|---|---|---|
model | string | Yes | Model ID in the form creator/model, e.g. amazon/nova-lite. Must be allowed by your team's Directives. |
max_tokens | number | No | Hard cap on generated tokens, and the basis for the balance reserved per request. Nova Lite supports up to 8,192 output tokens. |
tools | Tool[] | No | Function definitions the model can call. Tool calls are returned in choices[].message.tool_calls. |
Supported
max_tokenstemperaturestoptoolstool_choiceresponse_formatstructured_outputsInput limits
| Input | Sources | Limits |
|---|---|---|
| Text | — | Prompt and response share the 300K-token context window |
| Image | URL, base64 | Sent as image_url content parts; counts as input tokens |
| URL, base64 | Sent as file content parts; counts as input tokens |
More models by amazon
| Model | Context | Input | Output | Cache | Release |
|---|---|---|---|---|---|
 amazon/nova-2-lite | 1M | $0.33 | $2.75 | Read $0.083Write — | 12/02/2025 |
| amazon/nova-micro | 128K | $0.035 | $0.14 | Read $0.0087Write — | 12/03/2024 |
| amazon/nova-pro | 300K | $0.80 | $3.20 | Read $0.20Write — | 12/03/2024 |