Will I Hit My Rate Limit?
Enter your expected load to see if you'll hit limits
Rate Limits Across Providers
| Provider ↕ | Tier ↕ | Model | RPM ↕ | TPM ↕ | RPD ↕ | Rating |
|---|
Retry Strategies
Exponential Backoff
The industry standard. Double your wait time after each 429. Prevents thundering herd and gives the API time to recover.
# Python — exponential backoff
import time, random
def call_with_retry(fn, max_retries=5):
for attempt in range(max_retries):
try:
return fn()
except RateLimitError:
wait = (2 ** attempt) + random.random()
time.sleep(wait)
raise Exception("Max retries exceeded")
Token Bucket (Client-Side)
Pre-limit yourself before the API does. Track tokens spent per minute and queue requests when you approach the limit.
// JS — simple token bucket
class TokenBucket {
constructor(tpm) {
this.limit = tpm; this.tokens = tpm;
setInterval(() => this.tokens = tpm, 60000);
}
async consume(n) {
if (this.tokens < n)
await this.wait();
this.tokens -= n;
}
}
Request Queuing
Queue bursts instead of dropping them. Use a priority queue so critical requests go first when capacity frees up.
# Queue with concurrency limit
from asyncio import Queue, Semaphore
sem = Semaphore(10) # max concurrent
queue = Queue()
async def worker():
while True:
task = await queue.get()
async with sem:
await process(task)
queue.task_done()
Provider Fallback
When your primary provider is rate-limited, route to a fallback. Keep a priority list ordered by cost vs. speed.
// Fallback chain
const providers = [
{ name: 'openai', client: openai },
{ name: 'anthropic', client: anthropic },
{ name: 'groq', client: groq },
];
for (const p of providers) {
try { return await p.client.chat(msg) }
catch (e) { if (!isRateLimit(e)) throw e }
}