Rate Limit Behavior in Shared CI Token Pools

Most engineers reason about rate limits as though they were per-caller throttles, the API equivalent of a speed limit on a single road. That model breaks in CI the moment more than one pipeline authenticates under the same bot account. The budget is attached to the identity, and every consumer wearing that identity spends from it invisibly to the others.

How rate limits actually scope to identity, not to a pipeline or a person

Rate limits are identity-scoped buckets, not per-pipeline or per-process constructs. GitLab's 2026 change makes this explicit: limits attach per user and per top-level group, so every caller sharing one identity shares one hourly budget. A pipeline author watching a 429 in their own logs will naturally inspect what that pipeline requested, on the assumption that the throttle is measuring their pipeline's behavior. The accounting has no such granularity. The same bucket belongs to every integration, every scheduled job, and every tool that carries the same token, and none of them receives a statement of the others' withdrawals.

The Anthropic organization-scoped pooling case makes the same point from a different angle. Accounts under the same organizationUuid show identical utilization and reset times, so the pool is per-organization, not per-account. This was undocumented, and users who assumed their accounts were independent learned otherwise only by observing the shared reset times. The surprise itself is the diagnostic signal: engineers expect identity boundaries to follow the credentials they hold, but the provider draws those boundaries one level up.

The scope varies by provider, which makes the mental model harder to keep straight. OpenAI enforces at organization and project scopes, not user scope. Anthropic enforces at organization level with optional workspace sub-limits. Gemini enforces per project, not per API key. Atlassian's 2026 point-based system pools points across all tenants of an app against one shared hourly bucket, in what it labels the Tier 1 Global Pool, or gives each tenant its own hourly bucket under Tier 2 Per-Tenant Pool. The common thread is that the failure surface is the identity, not the volume. A budget-exhausting caller and an innocent pipeline are invisible to each other, but share consequences.

Why the bucket itself behaves in ways pipeline authors don't expect

Even engineers who understand that limits are identity-scoped are routinely surprised by how the bucket replenishes, counts, and punishes. Burst limits are not throughput targets. GitLab's 2026 announcement spells out the arithmetic: the per-minute burst ceiling is a spike allowance, and holding it exhausts the hourly allowance in roughly 50 minutes on Free and far faster on paid tiers. Upgrading the plan raises the ceiling, which a well-behaved workload reads as more room. A badly behaved client reads it as a faster route to exhaustion.

Failed retries consume the same scarce budget. Unsuccessful requests still contribute to per-minute limits on some providers. A client that retries on every failure, placed behind an SDK with automatic retry, behind a gateway that also retries, behind a workflow loop that retries the whole step, converts one overload event into many calls. Each layer is individually reasonable. Compounded, they spend the shared budget several times over while the original failure remains unresolved.

Hourly resets mean all-or-nothing recovery. Atlassian's 2026 point-based system uses an hourly token bucket, and when it is exhausted, all requests are denied until the next reset with no gradual throttling. A single tenant causing the app to exceed its limit creates an effective denial of service for every other customer for up to an hour. There is no partial degradation, no priority queue, no soft landing. The blast radius is total for the bucket's lifetime, and the problem compounds when multiple pipelines fire concurrently.

How concurrent pipelines collide inside a shared budget

The collision is not a configuration mistake that can be fixed by rate-limiting one pipeline. It is a structural consequence of multiple callers sharing a budget neither can see, and it produces failures that appear unrelated to their actual cause. A runaway nightly job throttles every other automated consumer at once, and the failure surfaces at an arbitrary time in an unrelated pipeline. The engineer on call investigates the pipeline that failed, finds nothing wrong with its request volume, and blames provider flakiness. The causal link is invisible without shared observability across all callers under the identity.

Unauthenticated callers silently compete on a separate, smaller budget. GitLab's 2026 limits cap unauthenticated traffic at a low per-IP ceiling regardless of the account's subscription tier. Documentation link checkers, status dashboards, curl calls in Makefiles, and internal tooling reading public project metadata all compete for the same small budget if they never authenticate. A shared office or VPN egress IP pools all of them together, so one developer's incidental curl loop counts against the same ceiling as the build server's health check.

Four patterns drain a shared budget fastest in multi-threaded CI contexts: short-interval polling, one token shared across threads, no TTL cache on repeated reads, and an agent loop with no per-turn call budget. Any combination of these across concurrent pipelines accelerates collision, because each multiplies the request rate against the same identity without giving any single caller more visibility into the remaining budget. The failure mode is not a single misbehaving pipeline. It is the arithmetic of many small consumers summing to an exhausted bucket.

Why AI coding agents make shared-token collisions dramatically worse

AI coding agents are a qualitatively different kind of API consumer: bursty, opaque about their call volume, and typically handed whatever credential is already available, which is usually the same shared token the pipelines use. The agent gets the most available credential, not the right one. The developer who sees the resulting 429 in their editor has no idea why, because the agent's consumption is invisible to the pipeline consuming the same identity's budget concurrently. Both are looking at the same bucket, and neither can see the other's spending.

One user request fans out into many provider calls. The 2026 AI agent rate-limit analysis documents that a single user request can fan out into planner calls, parallel subagents, retrieval, tool search, code execution, validation, and retries, each consuming from the same shared identity budget. What felt like one interaction in the editor is a distributed spend event on the provider side, and the multiplier is not visible from the request the user made.

Token accounting is provider-specific and harder to predict than request counts. Anthropic separates input and output limits and excludes cache-read tokens from ITPM. OpenAI exposes token-capacity headers and documents estimation behavior where the rate-limit calculation can use the greater of the configured maximum output and an estimate based on request size. An overstated output ceiling can reduce apparent throughput before those tokens are generated. The narrowest exhausted bucket wins: available requests do not imply available tokens, and available tokens do not imply spend headroom. A 429 from an agent workload may not be recoverable by backing off request rate alone.

What a readable budget actually requires: headers, scopes, and their limits

Provider-supplied rate-limit headers are necessary but structurally insufficient for shared CI pools. They report the state of the bucket from one caller's perspective, not the aggregate state across all callers sharing the identity. GitLab returns RateLimit-Limit and RateLimit-Remaining, and on throttled responses, Retry-After and RateLimit-ResetTime. OpenAI exposes limits, remaining capacity, reset timing, and Retry-After for temporary rate-limit errors. Anthropic publishes a programmatic Rate Limits API and documents acceleration limits and spend-cap error differences separately. Reading these headers is the beginning of observability, not the end of it.

What the headers cannot give is synchronization. The RateLimit-Remaining value a pipeline reads reflects only what that process has seen. Another pipeline consuming the same identity in parallel reduces the true remaining budget without updating any value the first process has already read. There is no cross-process coordination in the headers themselves. Each caller holds a stale snapshot, and the staleness widens precisely when consumption is fastest.

Forge cannot surface rate-limit headers to front-end code at all. Atlassian's developer community discussion documents that Forge is currently not capable of providing rate-limit headers to front-end code, which means part of the consuming surface is completely blind to the budget state. The gap is worse for the point-based system: the same discussion asks whether current headers expose points-based consumption, and the answer at publication was unclear. Object costs for many request types were still undetailed, so you cannot predict how many points a given call will consume before sending it. Instrumenting RateLimit-Remaining and alerting when any caller routinely burns toward zero is the minimum viable step, but it must be done at the identity level, aggregating across all callers, not per-pipeline.

The governance and architecture changes that stop pipelines from breaking each other

The fix is boring and worth doing: separate the identities, isolate the budgets, and put a shared admission controller in front of shared infrastructure. None of it requires waiting on the provider to change anything. Identity separation is the single highest-leverage change. GitLab's guidance is direct: use CI/CD job tokens for pipeline work, dedicated service accounts for integrations, and link agent workload credentials to the human who requested the action via composite identity, so both must be authorized before access is granted. Separating human and machine budgets turns rate limiting from an outage into a line on a dashboard.

Audit every automated caller before changing anything. An inventory of every automated caller, what it authenticates as, how many requests it makes at peak, and whether that identity is a human reliably surfaces integrations nobody remembered owning, including callers authenticating as people who have left the organization. Anonymous callers must authenticate or they share the smallest possible budget. Documentation link checkers, status dashboards, and curl calls in Makefiles that never authenticate compete for a very limited number of requests per hour per IP address, and giving each a dedicated identity is the only remedy.

A shared admission controller is required for any multi-replica or multi-pipeline deployment. An in-memory limiter only works for a single process. The PoliNetwork auth fix demonstrates the pattern: a single atomic INSERT... ON CONFLICT DO UPDATE on a shared backing store, PostgreSQL in that case and Redis the common alternative, so that two replicas cannot both read "budget remaining" and both allow a request that should have been rejected. The operation must be atomic. Checking the limit and incrementing the counter in two separate operations admits extra concurrent requests under load. A production admission controller needs to reserve across at least five dimensions: requests, estimated input tokens, maximum or predicted output tokens, concurrent work, and money. It must reconcile reservations against actual usage, propagating wait or reject decisions to the planner before they reach the provider.

For agent workloads, reduce speculative parallelism before routing to a fallback model. The safest response to overload is not immediate provider failover. A fallback model may have different context limits, tool behavior, safety properties, and independent or shared quotas. Classify the limit type first, honor provider reset guidance, and queue work within a deadline. Atlassian's tier escalation path is a governance dependency as well as a technical one. The developer community discussion surfaces the absence of a public escalation path for moving an app to a higher tier. When a single tenant can exhaust the hourly bucket and deny service to all other customers for up to an hour, having no fast-resolution SLA is itself a systemic risk that architecture alone cannot fully mitigate.

Why testing against a live shared token pool encodes the collision problem into your CI permanently

Running CI tests against a live API under a shared identity means more than just rate-limit failures. It means every test run is spending from a real budget that other callers depend on, making it impossible to write tests that reliably pass without coordinating with every other consumer of that identity. A test that passes at 09:00 on a quiet day and fails at 02:40 during a nightly batch run has not revealed a bug. It has revealed that the test depends on how much budget every other caller has consumed, which is not a property of the software under test.

Stateful flows cannot be correctly exercised against a stateless or shared live environment. A flow that creates a record, polls its status, and then cancels it requires the mock to remember what was created. A stateless fixture returns the same response regardless of prior calls, so the ID returned does not match the one created, the polling loop either never terminates or terminates immediately, and the conflict path cannot be reached at all. This is a testing correctness problem, not just a convenience issue. Mocks that are not continuously verified against the live API silently drift, because every change to the real API is a manual change to the stub if the stub is handwritten. Most teams discover drift in production, not in CI.

The brownout windows GitLab provides are the right mental model for what a good simulator should offer on demand. GitLab's two preview brownout windows on 7 and 14 October 2026, 15:00 to 19:00 UTC, function as a free scheduled load test. A locally hosted, stateful simulator that injects 429s and Retry-After headers on demand provides the same signal without depending on the provider's schedule or spending from a live budget. Fault injection is a first-class testing concern. Burst exhaustion, fixed-window boundary bursts, retry storms, and hourly hard resets are all injectable against a simulator that holds state across calls and can be reset between runs. They cannot be reliably triggered on demand against a live shared token pool without harming other callers. A stateful, continuously verified simulator is the architecture that decouples CI from shared identity budgets: each pipeline gets its own isolated instance with its own state, pre-seeded with realistic data, verifiable against the real API's behavior before every release. No shared budget, no invisible collision, no test that passes only when no one else is looking.