Policy layer

Today an app sends chat requests to aliases with its own key. The policy layer goes in front: apps run named tasks with a short-lived signed token, and the layer owns prompts, models and limits.

In developmentRunning in staging; not in front of live trafficLast updated 5 October 2026

Why a policy layer

A key per app gives cost per app. It does not say which customer or which feature spent the money, and it lets any caller send any prompt. The policy layer closes both gaps: a caller can run only the tasks its product is allowed, with typed input, and every run is booked against product, tenant and feature.

An endpoint that can only return, say, a list of steps in a fixed shape is a poor substitute for a free general chatbot, whatever the user types into it. That is the design argument for tasks over open chat.

What happens on a run

Verify the token A signed product token, valid for 5 to 15 minutes, with a cap on how many runs it can make.
Check the task The task must be open to the product, its feature must be on the token, and the product must be allowed the task’s data class.
Validate the input Against the task’s closed input schema, then render the task’s versioned prompt template.
Check the worst case Upper input estimate plus max_tokens, at the alias’s price. Above the task’s cost ceiling, the call is refused before it is made.
Call the gateway With the task’s alias and the product’s key for that data class and lane, never another class’s key.
Validate the answer Against the task’s output schema, if it has one.
Write one ledger row With pseudonyms instead of user ids, and no prompt, input or output.

Product tokens

Minted by the product’s own backend, never by a browser or a mobile app.
Short-lived (5 to 15 minutes) and limited to the features the product grants.
Identifiers are opaque ids. Email addresses and phone numbers must not be used as ids.
One product cannot mint tokens for another.

Named tasks

A task fixes everything a caller could otherwise abuse: the prompt, the model tier, the token limit, the data class and the cost ceiling. The caller sends only typed input. There is no way to send a system prompt, messages, a model name or a token limit.

yaml (illustrative)

task_id: askshiva.panchang_explain
feature: panchang
products: [askshiva]
model_tier: fast             # the registry maps each tier to an alias
lane: interactive            # interactive or batch
data_class: public           # public, personal or children
max_tokens: 500
cost_ceiling_inr: "…"        # worst-case cost is checked before the call
input_schema:                # closed: unknown fields are refused
  type: object
  additionalProperties: false
  properties:
    date: {type: string, maxLength: 10}
    city: {type: string, maxLength: 64}
    language: {type: string, enum: [en, hi]}
prompt_template:
  version: 1
  system: "…"
  user: "…"

Example

Illustrative only. The interface is in staging and may change.

http

POST /v1/tasks/askshiva.panchang_explain/run
Authorization: Bearer <short-lived product token>
Content-Type: application/json

{"input": {"date": "2026-10-04", "city": "Varanasi", "language": "hi"},
 "client_ref": "ui-1"}

json (shape only)

{
  "request_id": "req_…",
  "task": "askshiva.panchang_explain@1",
  "status": "succeeded",
  "output": {"summary": "…", "windows": [{"label": "…", "start": "…", "end": "…"}]},
  "alias": "askshiva-fast",
  "usage": {"input_tokens": 0, "output_tokens": 0, "cost_inr": "…"},
  "latency_ms": 0
}

Usage and cost in the answer are for the product backend’s own accounting. They are not prices and should never be shown to end users as prices.

Errors

Errors are RFC 9457 problem details with a stable code.

Status	Code	Meaning
401	`token_replayed` and others	Missing, expired or over-used token. Checked before the body is read.
403	`feature_not_allowed`, `data_class_not_allowed`	The token or product may not run this task.
413	`cost_ceiling_exceeded`	The worst-case cost is above the task’s ceiling.
422	`input_invalid`	The input does not match the task’s schema.
429	`rate_limited`, `budget_exhausted`	A gateway limit or budget was hit. `Retry-After` is sent for rate limits.
502	`output_invalid`	The model’s answer failed the output schema.
503 / 504	`upstream_unavailable`, `upstream_timeout`	The gateway or model could not answer in time.

Later: the full design

Planned Nested budgets per user (day and month), tenant (month), product (day) and the whole portfolio (day), with the estimated cost reserved before each call so parallel requests cannot overspend together.
Planned Lanes: interactive, deferred and batch, so work that can wait runs cheaper.
Planned Degrade, do not block. Suspected misuse gets a shorter answer, a smaller model, a slower pace or a human check, never a silent ban. Scores start in shadow mode and are tuned on real traffic.

Where it stands

In development The first version runs in staging in front of the real gateway, with its own ledger database and integration tests. Prices in its registry are placeholders. It does not yet carry live traffic, and it is not open to invited teams. Until it is, invited apps use the gateway directly with their virtual key.