Throttle

Throttle is Hypervibe's inference broker, credential custodian and economics ledger. This page explains how a model request is admitted and routed, where credentials live, and how capacity and cost are counted.

Throttle is the only path to a model #

Rocket! decides that a step needs a model and proposes the model, effort and context. Throttle admits the request against capacity and permissions, chooses the provider account, forwards only what that provider accepts and records the outcome. Nothing else in Hypervibe calls a model provider.

• Every answer records which requested controls, such as an effort level, were forwarded, refused or removed; a missing record reads as unknown.
• A refusal for lack of capacity names its cause: the pool, a concurrency limit or a budget. It gives a time to retry only when one is known.
• A refused request leaves a record and adds no usage.

Throttle is provider-neutral, and an unknown model is refused #

Rocket! keeps each turn in a neutral form, and Throttle renders it into each provider's own request format, the same way every time. Between them sits a versioned capability catalogue: what each model accepts, how its reasoning may be replayed, how its cache works and what it costs. The catalogue fails closed, so a model it does not list is refused.

The design covers every provider and model that Throttle's existing adapters serve, and the first release is planned to include them all, not one provider first. Its single-person record runs on Claude with an API key. Each other provider is planned to qualify by its own conformance tests and then one recorded live call. Until that call, every answer from it records that it has passed only its conformance tests.

Throttle alone decides what reasoning is replayed #

Some providers return items that only they can read back, such as the model's reasoning. Throttle keeps each one in its custody exactly as the provider sent it, with its provenance: the provider, the model that served it, the account and the catalogue revision. Rocket! journals a reference to it beside the turn it belongs to.

On each later call, Throttle alone decides whether to send each item back. An item it drops, for example because the account or the earlier conversation changed, is recorded with its reason, never dropped silently. Once a session holds an item bound to one account, Throttle keeps the session on that account. If that account runs out, the default is to move to another account and record every bound item as dropped; a setting makes it refuse instead.

Throttle holds every credential #

In this design, Rocket!, Lanes and agents never hold a credential. Throttle holds every credential value, and the other parties hold less:

A lease hands one value to one process for a stated purpose, bound to who asked and to an expiry. A lease lasts at most eight hours, and no longer than the provider allows; a request for a longer one is refused. Revealing a secret also hands out a time-bound lease, never the value itself. Each secret has exactly one scope: one workspace, only that workspace's coordinator lane, or only Fleet. The grant is checked where the effect happens, when the lease is issued. Revoking a grant stops future delivery; a value already delivered may need rotating, and Hypervibe says so. The one secret Throttle never holds is the key to the Hypervibe service's own store: the service keeps it in a key file only its owner can read or in the operating system's keychain.

The first release applies this to inference credentials, the model provider accounts you connect, including an API key for paid, metered use, and to the credentials of the managed GraphLake engine. That engine is the one designed exception to leases: it reads its credentials from files that Throttle writes and rotates, readable only by their owner. Connector tokens, deploy credentials and the other secrets join Throttle's custody after the first release.

Headroom and money are never one figure #

A pool is a set of provider accounts that Throttle can route between, and pools are never added together. An account is one attached provider credential. Providers measure use in windows with a reset time, such as five hours or seven days. Headroom is what is left of a window: capacity, never money.

38% left · pool claude · 7d · fresh 12s
paid route   paid share today 4% · 12 of 300
money today  settled $1.20 · held $0.40
             $1.60 of $50.00 budget · $98.40 left under the ceiling

Subscription capacity is tried first. The paid route is metered API use, and it stays off until you opt in to it: a request takes it when its pool is under pressure or no account can be selected, and only for a model with a list price. The ceiling is an emergency hard limit, and none is set until you set one: Throttle holds the most a paid request could cost against it until the request settles, so requests in flight together never pass it, and a ceiling of zero turns paid dispatch off. The daily budget you set is a soft limit: Throttle holds a realistic estimate against it, so paid use can go over the budget, and every overshoot is recorded. A ceiling, the emergency one or a project's, is what bounds that overshoot; with neither set, nothing caps paid use, and the soft budget is the only limit. An account that hit a provider limit is cooling until its known reset; without a known reset it reads unavailable.

Every request takes one recorded route #

Exactly one account serves each request, and the request's record names the route it took, or says the route is unknown. A served model that differs from the requested one is recorded as its own fact. Throttle owns the choice of account and provider, cooling, cache use, reservation, dispatch, retries and settlement. Retries and helper calls draw on the same budget as the work that caused them. If the same operation arrives again unchanged after Throttle has handled it, Throttle does not call the provider a second time: it answers with a reference to the first request and records no new usage or cost. Usage records keep short-lived and long-lived cache writes apart, so each can be priced at its own rate.

The aim is the lowest total cost per accepted result within the quality and latency you require, not the cheapest token or the highest cache-hit rate alone.

Held is not settled, and unknown cost is never zero #

A list price is the one in the catalogue revision that served the request, and the cost line names that revision, so a price change never rewrites an earlier line.

A hold reserves money for a request; it is not spend. A hold that a request has drawn on stays until settlement; only an unused hold can expire. An unknown price is never recorded as zero; it settles conservatively at the held amount. A timeout, cancellation or lost reply does not prove a request stopped, so its liability stays held until the outcome is observed.

Every attempt, retry, helper call and verification call is attributed to the run that caused it, and each physical attempt counts once. A run's receipt keeps settled and estimated amounts as separate sums and counts its unknown lines. A single total is shown only when no line is unknown. The first release's single-person record, the evidence that a whole run works for one person, reports the cost per accepted outcome and the share of cost still unknown, over a cohort of tasks fixed before the run.

A team shares a pool, never a credential #

In the first release's team record, the evidence that a whole run works for a team, the pool's credential lives only on the team server. Every request records the person, machine, team and lane. A per-person concurrency limit and the team's daily budget each refuse on their own, and nobody can replay another person's reasoning. A team's owners and administrators set the team's default model, daily budget and per-person concurrency limit through shared settings and can see usage per person.

Throttle never changes its own policy unprompted. A change to economics policy is filed as a request and applies only when it is approved. An approver that is not a person can only tighten a limit, and leaves a receipt when it does; loosening a limit needs an owner or administrator signed in as a person. Tightening never takes back what a running request has already been granted. Fleet is planned to file such requests under a grant after the first release.

Where to go next #

• Read a receipt for one run's verdict and cost.
• Reactor for builds, which receive credentials only as leases.
• Fleet for who proposes changes to economics policy.
• Choose a profile for the single-person and team records.