Rate Limits

ai& enforces rate limits per organization to keep the platform healthy under shared load. Limits depend on your tier and apply to inference endpoints; management APIs have their own (looser) limits.

Tiers

Tier	Description
Tier 0	Evaluation tier. Lower per-minute caps. Suitable for development and small projects. New orgs start here.
Tier 1	Production tier. Higher caps and access to higher-throughput models. Orgs are promoted on their first successful payment.

What’s measured

Each request is checked against six buckets — four per-model and two per-org global:

• Per-model RPM — requests per minute against one specific model.
• Per-model Input TPM — input tokens per minute (estimated up-front from the request body).
• Per-model Output TPM — output tokens per minute (charged after the response completes).
• Per-model Concurrency — max in-flight requests against one model.
• Global RPM — requests per minute across all models you call.
• Global Concurrency — max in-flight requests across all models.

Whichever bucket fills first triggers throttling.

Headers

Every authenticated inference request that reaches model admission reports each bucket that applies to you — including 200 responses, streams, and 429 responses from one of these six buckets. The first two rows use OpenAI’s header names and formats unchanged, so tools that already read them keep working; the rest extend the same pattern.

Bucket	Headers
Per-model RPM	x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests
Per-model Input TPM	x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, x-ratelimit-reset-tokens
Per-model Output TPM	x-ratelimit-limit-output-tokens, x-ratelimit-remaining-output-tokens, x-ratelimit-reset-output-tokens
Per-model Concurrency	x-ratelimit-limit-concurrency, x-ratelimit-remaining-concurrency
Global RPM	x-ratelimit-limit-global-requests, x-ratelimit-remaining-global-requests, x-ratelimit-reset-global-requests
Global Concurrency	x-ratelimit-limit-global-concurrency, x-ratelimit-remaining-global-concurrency

• limit-* is the bucket’s cap and remaining-* what is left in it right now. remaining-tokens counts input only — output tokens are charged after the response completes.
• reset-* is the time until that bucket is full again, as a duration like 45s or 6m0s. Concurrency buckets have no reset: they free up as in-flight requests finish.
• A bucket your tier has no cap on (an Input/Output TPM or global cap of 0) is omitted. A limit-requests or limit-concurrency of 0 means the opposite — that model is closed to your tier, and no wait will open it.

On a 429 Too Many Requests from one of these six model buckets, two more headers are added:

Header	Meaning
X-RateLimit-Policy	Which bucket denied the request: rpm, global_rpm, input_tpm, output_tpm, concurrency, or global_concurrency.
Retry-After	Seconds until this request fits — shorter than reset-*, which waits for the whole bucket. Omitted for concurrency and global_concurrency rejects — finish or cancel in-flight requests instead.

The earlier X-RateLimit-Limit and X-RateLimit-Remaining headers remain as deprecated compatibility aliases. New integrations should use the explicit headers above. Authentication throttles happen before model admission and carry Retry-After without X-RateLimit-Policy or bucket headers. Public catalog endpoints such as GET /v1/models are not metered this way and carry none of these headers.

See Response Headers for non-rate-limit headers.

429 responses

When throttled, ai& returns 429 Too Many Requests. The two kinds of bucket clear differently, so read X-RateLimit-Policy and handle the reject by its type — the same “back off and retry” is wrong for a concurrency reject.

Time-based rejects — rpm, global_rpm, input_tpm, output_tpm. These refill as the window advances. Retry-After gives the minimum safe delay: wait at least that long, then retry with exponential backoff and jitter.

Concurrency rejects — concurrency, global_concurrency. These clear only when in-flight requests finish or are cancelled, not with time, so no Retry-After is sent. Let existing requests drain, or lower your concurrency, before sending more.

Tip

Limits are per-org, not per-key. Multiple keys in the same org share the same buckets. Each model has its own per-model buckets; the global buckets are shared across every model you call.