Rate limit

A service policy that limits request, token, concurrency, or resource usage over a defined scope and interval.

Implementations use fixed or sliding windows, token buckets, concurrent-request caps, adaptive limits, and per-user, project, model, or endpoint scopes. A service may reject, delay, or queue excess work and may provide retry headers. A 429 response can also represent quota or capacity conditions, so clients should follow the documented semantics.

Builder example

Long-running work needs bounded concurrency, backpressure, queue state, retry classification, and progress persistence. Blindly waiting and retrying can duplicate non-idempotent effects or violate a server's Retry-After instruction.

Common confusion: Rate limits, quotas, and spending caps can overlap in provider APIs and need not reset on simple short versus billing-period schedules. Read the specific limit dimensions and error metadata.