Multi-tenant rate limiting
For: SaaS and platform teams with per-tenant API keys
Per-key rate-limit counters kept in Python dicts blow up resident memory when you have thousands of unique keys. Fast LiteLLM's Rust rate limiter uses ~42× less memory on this high-cardinality case and is 1.6× faster, as a drop-in.
Rate limiterAutomatic fallbackCircuit breaker
The problem
- →Resident memory grows with the number of unique API keys
- →GC pauses appear as key cardinality rises
- →Rate-limit checks add latency on the request path
How Fast LiteLLM helps
Compact Rust counters
Atomic-counter rate limiting uses ~42× less memory than per-key Python dicts at high cardinality.
Faster checks
1.6× faster rate-limit evaluation keeps the check off your latency budget.
Drop-in
No schema or code changes — add one import and keep your existing limits.
See if it fits your workload
Open source, MIT, one import line. Reproduce the benchmarks on your own traffic before you commit.