Rate Limits

Toolken enforces two types of limits at the gateway, before a request reaches your provider.

Request rate limit - how many requests your workspace can send per minute.
Concurrent stream limit - how many streaming responses your workspace can hold open at once.

Self-serve rate limit configuration is coming soon. Limits today are set by the Toolken team. Contact us to adjust limits for your account.

What you see when a limit is hit

Both limits return an OpenAI-compatible error body.

{
  "error": {
    "message": "Rate limit exceeded",
    "type": "proxy_error",
    "param": null,
    "code": "rate_limited"
  }
}

When the request rate limit fires, the response includes a Retry-After header with the number of seconds until your window resets.

How enforcement works

Both limits are checked at the edge before the request is forwarded. A request that exceeds either limit is rejected immediately with a 429. An open streaming response counts against your concurrent limit until it finishes.

If the enforcement backend is briefly unavailable, requests are allowed through rather than blocked. Toolken prioritizes availability over hard-blocking during transient disruptions.

Handling 429s in your code

A 429 from Toolken is safe to retry. Read Retry-After and wait that many seconds before the next attempt. For agent loops that fire many parallel requests, add a concurrency limit on the client side to stay within the per-workspace ceiling.

import time
import openai

def chat_with_retry(client, **kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(**kwargs)
        except openai.RateLimitError as e:
            retry_after = int(e.response.headers.get("Retry-After", 5))
            time.sleep(retry_after)
    raise RuntimeError("Rate limit retries exhausted")

Observability

Track request volume and spend before you hit a limit.

Anthropic & Gemini OpenClaw

Getting Started

Observability

Providers & Routing

Integrations

Use cases

Rate Limits

What you see when a limit is hit

How enforcement works

Handling 429s in your code

Next

Observability

​What you see when a limit is hit

​How enforcement works

​Handling 429s in your code

​Next

Observability

What you see when a limit is hit

How enforcement works

Handling 429s in your code

Next