Reference

Fallbacks and caching

Most aliases are a short, ordered chain of models. When one fails, the gateway tries the next, within limits that keep data classes and costs where they belong.

Early accessBy invitationLast updated 5 October 2026

The fallback chain

1. Primary The request goes to the first model behind the alias.
2. Retry or move on Depending on the error, the gateway retries the same model or moves to the next one in the chain (table below).
3. Up to four hops A request makes at most four fallback hops. After that the error comes back to your app.
4. Cool-down A model that fails three times is rested for 30 seconds, so traffic stops piling onto it.

Retries

Your own retries stack on top of these. Keep client retries few and spaced, or one failure becomes many calls.

Special fallbacks

• Context window. If the input is too long for the primary model, the request moves to a model with a larger context window.
• Content policy. For some general aliases, a refusal on content-policy grounds can move to another model.
• Unsupported parameters are dropped for a fallback model that does not support them.
• No forced failovers. Test parameters that would let a client force a failover are rejected.

Where it never falls back

• Children’s aliases (askshiva-kids, askshiva-kids-fast): no fallback at all.
• Personal and public data-class aliases: a short, fixed chain that ends in a final step with no further fallback. A fallback never crosses a data class.
• Embeddings: never fall back to a different embedding model, because vectors from different models do not mix.
• Local aliases: no fallback.

Provider caps

Some providers have a spending cap at the gateway. When a cap is reached, that provider’s models are skipped and the next model in the chain answers. You may notice a different speed or style; your request still succeeds if the chain has another step.

Keep your own fallback

The gateway is one service. Our own products keep a direct path for when the gateway itself is down, and yours should handle that case too: a clear message to the user, a queued retry, or a smaller feature. Set client timeouts at least as long as the alias timeouts on the aliases page.

Caching

Response caching is off by default. You can opt in per request for an exact-match cache: an identical request within about an hour gets the stored answer.

curl "$ASKSHIVA_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $ASKSHIVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "askshiva-flash-public",
       "messages": [{"role": "user", "content": "Explain GST in one paragraph."}],
       "cache": {"use-cache": true}}'

reply = client.chat.completions.create(
    model="askshiva-flash-public",
    messages=[{"role": "user", "content": "Explain GST in one paragraph."}],
    extra_body={"cache": {"use-cache": True}},   # opt in for this call only
)

const reply = await client.chat.completions.create({
  model: "askshiva-flash-public",
  messages: [{ role: "user", content: "Explain GST in one paragraph." }],
  cache: { "use-cache": true }, // opt in for this call only
});