Rate limits and quotas

Requests per minute#

Both are counted in fixed one-minute windows that reset on the minute. A request must fit under both; the first one exceeded answers 429 rate_limit_exceeded, with a message saying which. Requests refused for rate still count toward the minute, so retrying in a tight loop only extends the wait. Embeddings and rerank calls count like chat, one request each however many inputs they carry. GET /v1/models, GET /v1/key and count_tokens aren't rate limited.

Headers#

HTTP/1.1 200 OKx-ratelimit-limit-requests: 60x-ratelimit-remaining-requests: 59x-ratelimit-reset-requests: 29s

They come with every chat completion, message, embeddings and rerank call that gets past the account checks, on both APIs.

Retrying with backoff#

The OpenAI and Anthropic SDKs retry 429 and 5xx responses twice by default, honouring retry-after. Raise max_retries for batch jobs:

client = OpenAI(base_url="https://fullsendai.com/v1", api_key=os.environ["FULLSEND_API_KEY"], max_retries=5)

Without an SDK, wait for retry-after when it's there, else back off exponentially with jitter, and only retry what can succeed:

const RETRY = new Set([429, 502, 503, 504]);async function chat(body: object, attempts = 5) {  for (let attempt = 0; ; attempt++) {    const response = await fetch("https://fullsendai.com/v1/chat/completions", {      method: "POST",      headers: { Authorization: `Bearer ${process.env.FULLSEND_API_KEY}`, "Content-Type": "application/json" },      body: JSON.stringify(body),    });    if (response.ok) return response.json();    const error = await response.json();    // 429s for spend caps and budgets won't clear by waiting a minute.    const retryable = RETRY.has(response.status) && !["key_limit_reached", "project_budget_reached"].includes(error.error?.code);    if (!retryable || attempt + 1 >= attempts) throw new Error(`${response.status} ${error.error?.code}: ${error.error?.message}`);    const wait = Number(response.headers.get("retry-after")) * 1000 || Math.min(30_000, 500 * 2 ** attempt) * (0.5 + Math.random());    await new Promise((resolve) => setTimeout(resolve, wait));  }}

Spending limits#

Caps and budgets count calendar months in UTC and reset on the 1st. A request is checked before it runs, so the call that crosses a cap still completes; the next one is refused. A low-balance email (set under Billing) warns you before the balance runs out, and auto top-up keeps it from running out at all.

To check from code, GET /v1/key returns the key's balance, plan allowance, cap, spend this month and rate limits, so a job can stop before it's refused.

Size limits#

• Output: each model's maximum output (max_output_tokens in /v1/models). A higher max_tokens is lowered to it.
• Input: the model's context window (context_length), input and output together. A request over it is refused with 400 invalid_request_error.
• Time: a model must start answering, and keep sending, within 60 seconds; otherwise you get 504 upstream_timeout or an error event at the end of the stream. Long answers are fine as long as tokens keep coming.

Raising a limit#

Per-key limits, caps and budgets are yours to change on API keys and Projects. For a higher account or model limit, email [email protected] with your expected requests per minute and the models you use.