Rate Limiting: The Control Your AI-Built App Almost Certainly Lacks
Written By
SprintX Team
AI & Product Engineering
August 01, 2026
7 min read
Why AI-built apps ship without rate limiting, what it exposes, and a layered approach to adding limits that protect your login, your database, and your API bill.
Ask an AI builder for a login page and you get a login page. Ask it for a chat endpoint and you get a chat endpoint. What you never get, unless you ask by name, is the thing that stops someone from calling either one forty thousand times an hour.
Rate limiting is invisible when it works, which is exactly why it does not get generated. It is not a feature anyone demos. It has no UI. It only exists because someone with production scars insisted on it, and an AI assistant has no scars.
The bill arrives in one of three forms: a credential-stuffing run against your login, a scraper walking your entire database through a public endpoint, or a single enthusiastic user driving your model provider spend into four figures overnight. All three are cheap to prevent and expensive to discover.
What "no rate limiting" actually exposes
It helps to be concrete about which endpoints are load-bearing.
Authentication. Login, password reset, and magic-link endpoints are the classic target. Without a limit, an attacker can test leaked credential pairs at machine speed against your users. Lockout on the account alone is not enough — you need a limit on the attempting IP and identity too, or you have handed anyone a way to lock out your customers on purpose.
Anything that costs money per call. Every LLM completion, transcription, image generation, SMS, or email your app sends is a small purchase made on your card by an anonymous request. This is the fastest way to a surprise invoice, and the mechanism behind most of the situations described in why your AI app is burning API credits.
Enumerable reads. A list endpoint that accepts a page parameter and does not limit request volume is a database export waiting for a for-loop. Combine it with the missing authorization checks that are common in AI-generated auth flows and it becomes an export of everyone's data, not just the caller's.
Writes that create rows. Signup, comment, upload, and webhook-receiver endpoints without limits become spam vectors and storage bills.
Limit in layers, not in one place
The instinct is to add one middleware and call it done. That produces either a limit so loose it never fires or one so tight it breaks legitimate users. Layers work better because each one has a different job.
Layer 1 — the edge. Your CDN or host (Cloudflare, Vercel, an ALB, an API gateway) can drop volumetric junk before it reaches your compute. This is the cheapest place to shed load, because rejected requests never wake up a function or open a database connection. Use it for crude, generous per-IP caps and known-bad patterns.
Layer 2 — the application. Per-route limits keyed to the authenticated user, applied in middleware. This is where nuance lives: five password resets an hour, sixty API reads a minute, ten uploads a day. It is the only layer that knows who the caller is.
Layer 3 — the resource. Quotas tied to the thing being consumed rather than the request: tokens per user per month, generations per plan tier, concurrent jobs per workspace. This is a product decision as much as a security one, and it is what turns runaway usage into an upsell prompt instead of a support ticket.
Layer 4 — the provider budget. Hard spending caps and alerts on your model provider and any usage-priced vendor. Assume every layer above will eventually fail; this is the one that keeps a failure from becoming a five-figure invoice. Pair it with the cost work in reducing OpenAI API costs.
Picking an algorithm without overthinking it
| Algorithm | Behavior | Good for | Watch out for |
|---|---|---|---|
| Fixed window | N requests per clock interval | Simple internal endpoints | Double burst at the window boundary |
| Sliding window | N requests across a rolling interval | General-purpose API limits | Slightly more state to track |
| Token bucket | Steady refill, allows short bursts | User-facing apps and SDKs | Tuning burst size takes a round or two |
| Concurrency cap | N in-flight requests at once | Long-running or expensive jobs | Needs reliable release on failure |
| Cost-based | Spend budget per unit of work | LLM and media endpoints | You must price each operation |
For most products, sliding window on cheap routes and token bucket on user-facing ones covers everything. Expensive AI endpoints are the exception worth extra thought: charging a request against a token or cost budget rather than a request count is the only approach that reflects reality, because one 60-second streaming completion is not equivalent to one autocomplete call.
Keying: the detail that breaks implementations
A limiter is only as good as the key it counts against. Get this wrong and you either block a whole office behind one NAT or let an attacker rotate past your limit for free.
Key on the authenticated user or API key wherever you have one. Fall back to IP only for unauthenticated routes, and treat it as a coarse signal — residential proxy pools make single-IP limits easy to evade, so combine IP with a second dimension like the submitted email address on a login route. For multi-tenant products, add a workspace-level key on top, so one enthusiastic account cannot consume a shared quota. That tenant dimension matters more than people expect once you are running a multi-tenant SaaS.
Never key on anything the client controls and you do not verify. A header the caller sets is not an identity.
Shared state, or your limits are fiction
Here is the failure we see most often in reviews: a perfectly reasonable limiter storing counters in a process-local variable, deployed to a serverless platform that runs a dozen concurrent instances. Each instance enforces the limit independently, so the real limit is twelve times what the code says, and it resets whenever the platform recycles an instance.
If your app runs on more than one process — and on modern hosting it always does — counters need to live in shared storage. Redis is the standard answer; a hosted key-value service with an atomic increment works equally well. Whatever you pick, decide what happens when it is unreachable: fail open and keep serving traffic, or fail closed and reject. For a login endpoint, closed. For a read endpoint, open, with an alarm.
Returning a limit users can work with
When you reject, do it properly. Respond with HTTP 429, include a Retry-After header, and put a plain-language message in the body that says what was limited and when it resets. Well-behaved clients and SDKs will back off automatically if you tell them how long to wait; they will hammer you forever if you return a generic 500.
On the front end, handle 429 explicitly. An AI-generated client typically treats every non-200 identically, which means a rate-limited user sees "Something went wrong" and clicks the button again — turning your limit into a self-inflicted retry storm. Show the wait, disable the control, and retry with exponential backoff plus jitter.
Finally, log every rejection with the key and route. Rate-limit logs are the cheapest attack-detection system you will ever run: a sudden spike of 429s on a password reset endpoint is a credential-stuffing run announcing itself.
Rolling it out without breaking real users
Do not guess your thresholds. Measure first, then set limits well above observed p99 usage, then tighten. The safe sequence is: log-only mode for a week (count what would have been blocked), review the top offenders to confirm they are bots and not your best customer, then enforce, then alert on rejection volume.
Start with the endpoints that hurt most — authentication and anything metered — and extend outward. A single afternoon covering login, password reset, signup, and your AI endpoints removes the overwhelming majority of the exposure, which makes this one of the highest-return items on any production readiness pass for an AI-built app.
Frequently asked questions
Where should rate limiting live: the edge or the application? Both, doing different jobs. The edge sheds volumetric abuse cheaply before it reaches your compute, using coarse per-IP caps. The application enforces the limits that require knowing who the caller is and what they are allowed to do. Neither one substitutes for the other, and only the application layer can protect a per-user quota.
What limits should I start with? Measure your real traffic first, then set thresholds above the p99 of legitimate usage. As a starting shape before you have data: a handful of authentication attempts per identity per hour, tens of reads per minute per user, and a hard monthly budget per account on anything that costs money per call. Run in log-only mode for a week before enforcing.
Does rate limiting stop DDoS attacks? Not on its own. Application-level limits protect against abuse, scraping, brute force, and cost blowouts from clients that reach your app. A genuine volumetric DDoS has to be absorbed upstream by your CDN or provider's protection layer, because the traffic never needs a valid response to hurt you.
If your app went to production without limits on login or on the endpoints that spend money, you are one script away from finding out what that costs. SprintX hardens AI-built products — limits, auth, quotas, and the monitoring that proves they work. Tell us what your app exposes and we will scope the fix.
