SDK
Fallbacks
Two mechanisms. serverFallbackModels: the gateway retries inside one request, one bill. fallbackModels: the client retries with separate billed requests.
Server chain: serverFallbackModels
The gateway tries the chain in order when the requested model’s upstream call fails. One request, money held once, settled once against whichever member served, refunded in full if none did.
const answer = await conifer.chat({
model: "deepseek-v4-flash",
messages: [{ role: "user", content: "classify: refund request" }],
maxTokens: 200,
serverFallbackModels: ["glm-5.3-flash", "gemini-3.5-flash"],
});
answer.receipt.effectiveModel; // who answered
answer.receipt.reason; // "as_requested" | "provider_failover"answer = conifer.chat(ChatRequest(
model="deepseek-v4-flash",
messages=[{"role": "user", "content": "classify: refund request"}],
max_tokens=200,
server_fallback_models=["glm-5.3-flash", "gemini-3.5-flash"],
))
answer.receipt.effective_modelOn the wire it is one header:
curl https://api.conifer.build/v1/chat/completions \
-H "authorization: Bearer $CONIFER_API_KEY" \
-H "x-conifer-fallback-models: glm-5.3-flash,gemini-3.5-flash" \
-H "content-type: application/json" \
-d '{"model":"deepseek-v4-flash","messages":[{"role":"user","content":"hi"}],"max_tokens":200}'The gateway sees the provider’s own failure, so it advances on classes a client never observes: upstream 4xx/5xx, transport failures, dead streams. It also covers streamed turns, which a client chain cannot.
Validation
Members are validated before anything is spent. Each defect is refused with the member named:
| Refused | Reason |
|---|---|
| Unknown or unserved model | Fails at declaration time, not during the outage the fallback was for. |
| Duplicate member, or the primary repeated | That is a retry loop, not a fallback. |
| More than 3 members | The gateway holds worst-case money for the whole chain up front. |
A chain that de-duplicates to nothing sends no header; the request is just the primary.
Reading the receipt
| Outcome | effectiveModel | reason |
|---|---|---|
| Primary served | the model you named | as_requested |
| Fallback served | the member that answered | provider_failover |
Client chain: fallbackModels
The SDK re-sends the request to the next member after a retryable failure. Each attempt that reaches upstream is billed. allowClientFallback: true is required.
const answer = await conifer.chat({
model: "deepseek-v4-flash",
messages,
fallbackModels: ["glm-5.3-flash"],
allowClientFallback: true, // required: each member is a separate billed request
});
answer.fallbackIndex; // 0 = primary served, 1 = first fallback servedA 402 or malformed request answers identically on every member, so the chain stops instead of re-billing the same refusal.
One 400 does advance the chain: ConiferCapabilityError, the refusal for a request shape the model cannot serve (image content without the vision cap, tools on a no-tool model). It is unbilled and carries modelSwitchable = true.
// deepseek-v4-flash has no vision cap. The image turn raises
// ConiferCapabilityError (unbilled), the chain advances, glm-5.3-flash serves.
const answer = await conifer.chat({
model: "deepseek-v4-flash",
messages: [{ role: "user", content: [
{ type: "text", text: "what is in this image?" },
{ type: "image_url", image_url: { url } },
]}],
fallbackModels: ["glm-5.3-flash"],
allowClientFallback: true,
});Choosing
| Goal | Use |
|---|---|
| Outage resilience in production | serverFallbackModels |
| Absorb a capability mismatch (image on a text model) | fallbackModels + allowClientFallback |
| Your own backoff, logging, or per-attempt policy | client chain |
| Both | they compose; each client attempt can carry its own server chain |
Coming from another gateway
| You had | Becomes |
|---|---|
OpenRouter models + route: "fallback" | server chain (models without route stays a client chain) |
Helicone-Fallbacks | server chain. Helicone walked it in the proxy on one request; a client-side mapping would turn that into several bills. |