# Rate Limits

Source: https://aiand-docs-staging.pages.dev/reference/rate-limits/

ai& enforces rate limits per organization to keep the platform healthy under shared load. Limits depend on your tier and apply to inference endpoints; management APIs have their own (looser) limits.

## Tiers

| Tier | Description |
|------|-------------|
| **Tier 0** | Evaluation tier. Lower per-minute caps. Suitable for development and small projects. New orgs start here. |
| **Tier 1** | Production tier. Higher caps and access to higher-throughput models. Orgs are promoted on their first successful payment. |

## What's measured

Each request is checked against six buckets — four per-model and two per-org global:

- **Per-model RPM** — requests per minute against one specific model.
- **Per-model Input TPM** — input tokens per minute (estimated up-front from the request body).
- **Per-model Output TPM** — output tokens per minute (charged after the response completes).
- **Per-model Concurrency** — max in-flight requests against one model.
- **Global RPM** — requests per minute across **all** models you call.
- **Global Concurrency** — max in-flight requests across all models.

Whichever bucket fills first triggers throttling.

## Headers

Every authenticated inference request that reaches model admission reports each bucket that applies to you — including `200` responses, streams, and `429` responses from one of these six buckets. The first two rows use OpenAI's header names and formats unchanged, so tools that already read them keep working; the rest extend the same pattern.

| Bucket | Headers |
|--------|---------|
| Per-model RPM | `x-ratelimit-limit-requests`, `x-ratelimit-remaining-requests`, `x-ratelimit-reset-requests` |
| Per-model Input TPM | `x-ratelimit-limit-tokens`, `x-ratelimit-remaining-tokens`, `x-ratelimit-reset-tokens` |
| Per-model Output TPM | `x-ratelimit-limit-output-tokens`, `x-ratelimit-remaining-output-tokens`, `x-ratelimit-reset-output-tokens` |
| Per-model Concurrency | `x-ratelimit-limit-concurrency`, `x-ratelimit-remaining-concurrency` |
| Global RPM | `x-ratelimit-limit-global-requests`, `x-ratelimit-remaining-global-requests`, `x-ratelimit-reset-global-requests` |
| Global Concurrency | `x-ratelimit-limit-global-concurrency`, `x-ratelimit-remaining-global-concurrency` |

- `limit-*` is the bucket's cap and `remaining-*` what is left in it right now. `remaining-tokens` counts input only — output tokens are charged after the response completes.
- `reset-*` is the time until that bucket is full again, as a duration like `45s` or `6m0s`. Concurrency buckets have no reset: they free up as in-flight requests finish.
- A bucket your tier has no cap on (an Input/Output TPM or global cap of `0`) is omitted. A `limit-requests` or `limit-concurrency` of `0` means the opposite — that model is closed to your tier, and no wait will open it.

On a `429 Too Many Requests` from one of these six model buckets, two more headers are added:

| Header | Meaning |
|--------|---------|
| `X-RateLimit-Policy` | Which bucket denied the request: `rpm`, `global_rpm`, `input_tpm`, `output_tpm`, `concurrency`, or `global_concurrency`. |
| `Retry-After` | Seconds until this request fits — shorter than `reset-*`, which waits for the whole bucket. Omitted for `concurrency` and `global_concurrency` rejects — finish or cancel in-flight requests instead. |

The earlier `X-RateLimit-Limit` and `X-RateLimit-Remaining` headers remain as deprecated compatibility aliases. New integrations should use the explicit headers above. Authentication throttles happen before model admission and carry `Retry-After` without `X-RateLimit-Policy` or bucket headers. Public catalog endpoints such as `GET /v1/models` are not metered this way and carry none of these headers.

See [Response Headers](/reference/response-headers/) for non-rate-limit headers.

## 429 responses

When throttled, ai& returns `429 Too Many Requests`. The two kinds of bucket clear differently, so read `X-RateLimit-Policy` and handle the reject by its type — the same "back off and retry" is wrong for a concurrency reject.

**Time-based rejects** — `rpm`, `global_rpm`, `input_tpm`, `output_tpm`. These refill as the window advances. `Retry-After` gives the minimum safe delay: wait at least that long, then retry with exponential backoff and jitter.

**Concurrency rejects** — `concurrency`, `global_concurrency`. These clear only when in-flight requests finish or are cancelled, not with time, so no `Retry-After` is sent. Let existing requests drain, or lower your concurrency, before sending more.

<Aside type="tip">
  Limits are per-org, not per-key. Multiple keys in the same org share the same buckets. Each model has its own per-model buckets; the global buckets are shared across every model you call.
</Aside>