How-To

How to Benchmark Subagent Throughput Without Burning Your Allowance

A reproducible protocol for measuring multi-agent speed, accepted output, duplicated work, review burden, usage burn, recovery, and merge conflicts.

  • #AI agents
  • #subagents
  • #benchmarking
  • #developer productivity
  • #cost control

Adding workers to an AI coding task can make the terminal look busy without making the project finish sooner. One worker may produce a clean patch in forty minutes. Four workers may return in twelve minutes, then leave a maintainer with an hour of duplicate findings, incompatible edits, and failed integration. Eight workers may finish their individual assignments quickly while consuming enough tokens or credits to stop the next job.

The useful question is not “How many agents can the product spawn?” It is:

At what worker count does this workload produce the most independently accepted work per unit of wall-clock time, review effort, and controlled usage?

This article provides a benchmark protocol for answering that question. It deliberately uses 1, 2, 4, and 8 workers as experimental labels, not as a claim that any product supports, recommends, or guarantees those concurrency levels. No paid benchmark was run for this article, and the results table is intentionally blank.

The protocol was motivated by user reports of unexpectedly large subagent fan-out, rapid depletion of a weekly allowance, and a desire to choose the number and configuration of workers directly. Those reports are useful hypotheses, not official limits and not benchmark evidence. A social post cannot establish product capacity, billing behavior, or a generally reproducible burn rate. The controls below therefore rely on observable run logs and current vendor documentation rather than repeating anecdotal figures as specifications.

Official documentation was verified on 2026-07-24. Product behavior, beta status, metering, and plan rules can change, so recheck the linked pages immediately before running a benchmark.

What the official documentation establishes

OpenAI’s Agents SDK describes two broad orchestration choices: let an agent use handoffs or other agents as tools, or control the flow in code. Its orchestration guide says code-controlled parallel execution is useful when tasks do not depend on each other. That conditional matters: parallelism is a workload property, not a universal performance switch. The same documentation says a manager pattern is appropriate when one agent must combine specialist output or keep shared guardrails in one place. Source: OpenAI Agents SDK agent orchestration.

OpenAI also documents explicit run limits. max_turns caps model invocations in an Agents SDK run; exceeding it raises MaxTurnsExceeded, while setting it to None disables the turn limit. The SDK separately exposes a local function-tool concurrency cap through max_function_tool_concurrency. These are different controls: a turn ceiling constrains loop length, while a concurrency ceiling constrains how many eligible local tool calls execute together. Source: OpenAI Agents SDK running agents.

For metering, the OpenAI Agents SDK records requests, input tokens, output tokens, total tokens, cached-token details, reasoning-token details, and per-request usage. Its usage object aggregates model calls made during a run, including calls associated with tools and handoffs. That makes usage logs suitable for a benchmark, provided the chosen provider adapter actually reports the fields being compared. Source: OpenAI Agents SDK usage tracking.

OpenAI’s current model guide describes hosted multi-agent behavior as a beta and says it can reduce wall-clock time for complex tasks that divide cleanly into independent workstreams. This is a scoped statement, not a promise that more subagents always improve performance. Source: OpenAI API model guidance.

Anthropic documents a maxTurns frontmatter field for custom Claude Code subagents. The same page says subagents start in fresh contexts, can be restricted by tool allowlists and denylists, and cannot themselves spawn other subagents. It also distinguishes foreground execution from concurrent background execution and warns that many detailed subagent returns can consume significant context. Source: Claude Code custom subagents.

Claude Code’s agent-team documentation says teams are experimental and disabled by default. It says the user can specify an exact teammate count, that each teammate has its own context window, and that team token usage is higher than a single session. It recommends parallel work where workers can operate independently and warns against sequential, same-file, or dependency-heavy work. Source: Claude Code agent teams.

Anthropic’s cost guide is more explicit about scaling: token usage grows with the number of active teammates and how long they run, and it recommends small teams, focused spawn prompts, and shutting teammates down when their work is finished. Source: Claude Code cost management.

For non-interactive Claude Code runs, the CLI exposes both --max-turns and --max-budget-usd; the documented default is no turn limit unless the user supplies one. A benchmark harness should never inherit an unbounded default. Source: Claude Code CLI reference.

Finally, subscription allowance and paid continuation are separate concerns. Anthropic says Claude and Claude Code activity share limits on eligible Pro and Max plans, while API-key authentication can result in API charges. OpenAI says included plan usage is consumed first and eligible users may purchase credits or enable automatic top-up for supported agentic features. Those facts make authentication mode and paid-overage state mandatory preflight fields. Sources: Claude Code with Pro or Max and OpenAI flexible usage credits.

None of these pages publishes a universal “best worker count.” That number depends on the task graph, context duplication, model, tool latency, integration method, acceptance standard, and billing path. Benchmark the workflow you actually intend to operate.

Seven outcomes that must remain separate

A benchmark that reports only elapsed time rewards fast failure and shifts hidden work to reviewers. Record these seven outcome families independently before combining anything into a score.

1. Wall-clock speed

Wall-clock time starts when the controller releases the first worker and stops when the integrated candidate has completed all acceptance checks. It includes worker startup, queueing, retries, result collection, integration, and test execution. Also record “time to first worker output,” but do not substitute it for end-to-end completion time.

Use a monotonic clock where possible. Record timestamps at the controller, not from worker prose. A worker saying “done” is an event, not proof of completion.

2. Accepted artifacts

An artifact is a predefined unit of potentially useful output: a patch for one fixture, a test case, a structured research record, or a defect report with a reproducer. It becomes accepted only when an evaluator that did not produce it runs the predeclared checks and confirms that it is correct, scoped, non-duplicative, and integrable.

Generated files, claimed fixes, and completed worker tasks are not accepted artifacts. This distinction prevents a high worker count from winning merely by producing more material.

3. Duplicated work

Duplication includes identical findings, semantically equivalent patches, repeated repository exploration, and two workers implementing the same acceptance unit. Measure duplicate artifacts and, where telemetry permits, duplicated tool calls or repeated file reads. Context sent independently to every worker is overhead even when outputs differ.

Classify duplication as:

  • exact duplicate: same finding or materially identical change;
  • overlapping duplicate: same acceptance unit with different implementations;
  • redundant exploration: repeated work that does not improve the accepted result;
  • productive redundancy: independent work intentionally retained to compare hypotheses.

Productive redundancy belongs in the design only when declared before the run.

4. Review burden

Review burden is human or evaluator time spent triaging, validating, rejecting, reconciling, and explaining worker output. Measure active minutes, not the interval while a test suite runs unattended. Count review rounds, comments or correction messages, and artifacts that required manual investigation.

If an automated evaluator uses a model, record its usage separately. Moving review from a worker to a judge does not make review free.

5. Token or credit burn

Record the provider’s native usage units for every run: input tokens, cached input tokens, output tokens, reasoning tokens when exposed, request count, credits, or plan-usage units. Never infer subscription allowance consumption solely from token counts unless the vendor documents that conversion for the exact plan and surface.

Keep three columns:

  1. measured native usage;
  2. calculated cost under a versioned rate card, if applicable;
  3. observed change in a subscription usage meter.

Do not merge them. A token total, a purchased-credit charge, and a weekly plan meter are related but not interchangeable.

6. Failure recovery

Workers fail in ways that change the value of parallelism: timeout, rate limit, invalid output, permission denial, tool failure, context overflow, test failure, or process loss. Measure:

  • failures by class;
  • useful partial artifacts recovered;
  • retry count;
  • recovery wall time;
  • additional usage spent on recovery;
  • whether another worker had already covered the failed unit;
  • whether the entire run had to restart.

A design that is slightly slower in the happy path may be better if it resumes cleanly from checkpoints.

7. Merge conflicts

Count both textual and semantic conflicts. A textual conflict is reported by the version-control or patch application mechanism. A semantic conflict occurs when cleanly applied changes disagree about interfaces, schemas, assumptions, fixtures, or behavior.

Record:

  • files touched by more than one worker;
  • textual conflict count;
  • semantic conflict count;
  • minutes to resolve conflicts;
  • accepted artifacts invalidated by integration;
  • post-merge regression count.

Do not ask workers to edit a shared working tree concurrently during the benchmark. Give each worker an isolated workspace or make workers return patches without applying them. Otherwise, filesystem races contaminate the concurrency comparison.

Benchmark hypotheses

Write hypotheses before collecting results. A useful preregistration might be:

  • H1: Divisible tasks will show lower wall-clock time as worker count rises until startup and review overhead dominate.
  • H2: State-coupled tasks will show weaker speedup and more merge or semantic conflicts than divisible tasks.
  • H3: Native usage will increase with worker count because each worker loads context and reasons separately.
  • H4: Accepted-artifact throughput will peak before raw worker completion throughput.
  • H5: Recovery from one isolated worker failure will be cheaper than recovering from a failure that corrupts shared state.

These are hypotheses, not results. Preserve them even if the later data disagrees.

Build two task sets

Use both a divisible set and a state-coupled set. A benchmark containing only embarrassingly parallel work exaggerates the value of concurrency; a benchmark containing only same-file edits makes parallelism look irrational by construction.

Task set A: divisible work

Create at least eight independent work units so every treatment can receive the same total workload. Each unit should:

  • have its own input fixture and output path;
  • require similar but not identical effort;
  • have no dependency on another unit’s output;
  • be testable by a unit-specific deterministic command;
  • avoid shared mutable configuration;
  • produce one clearly defined acceptance artifact.

Examples include implementing eight independent parsers behind already-fixed interfaces, investigating eight unrelated failure fixtures, or producing eight structured records from separate source packets. Do not use eight copies of the same trivial task; the set should contain realistic variation in context size and difficulty.

Task set B: state-coupled work

Create at least eight units that share meaningful state. Examples include changes that depend on one evolving schema, modules that call a shared interface, migration steps with ordering constraints, or tests and implementation that touch a common fixture. Keep the total intended work comparable to the divisible set, but preserve the coupling that exists in real projects.

Declare the dependency graph in advance. Mark each edge as:

  • hard dependency: downstream work cannot be accepted before upstream output;
  • interface dependency: work can proceed against a frozen contract;
  • shared-file dependency: edits target the same file or generated artifact;
  • semantic dependency: separate files must agree on behavior.

For safe execution, isolate worker writes and merge through one controller. The state-coupled set should measure integration cost, not allow one worker to overwrite another.

Freeze the benchmark environment

Create a manifest before the first trial. It should contain:

  • repository or fixture commit hash;
  • operating system and relevant runtime versions;
  • product and client version;
  • model identifier and reasoning or effort setting;
  • worker role prompt hash;
  • controller prompt hash;
  • tool allowlist and denylist;
  • network policy;
  • permission mode;
  • context files supplied to each worker;
  • worker turn limit;
  • controller turn limit;
  • worker timeout and whole-run timeout;
  • maximum concurrent workers;
  • maximum total worker creations;
  • maximum retries per work unit;
  • native usage ceiling;
  • purchased-credit ceiling fixed at zero unless a later run receives separate authorization;
  • authentication path: subscription, API, enterprise gateway, or another explicit route;
  • auto-top-up and automatic overage status;
  • randomization seed;
  • acceptance-suite hash.

Do not silently change models, prompts, context, tools, or budgets between worker-count treatments. If a provider changes the service during the campaign, record the interruption and start a new benchmark version rather than pooling incompatible trials.

Treatment labels and task allocation

The treatments are W1, W2, W4, and W8, representing requested maximum active worker counts of 1, 2, 4, and 8. They are benchmark labels only.

Every treatment receives the same complete task set. Partition units deterministically after the randomized trial order is selected:

  • W1: one worker receives all units in the prescribed sequence;
  • W2: two workers receive balanced subsets;
  • W4: four workers receive balanced subsets;
  • W8: eight workers receive one initial unit each.

Balance subsets using a preregistered difficulty estimate derived from fixture properties, not from observing benchmark outcomes. For example, use source bytes, number of relevant files, and expected test duration. Preserve the allocation file for audit.

The controller may integrate outputs but must not perform unfinished worker tasks. Otherwise, controller work hides treatment failures. If the controller repairs an artifact, record the repair as review or recovery work and keep the original artifact rejected.

Randomization and repeated trials

Run treatments in a randomized order to reduce time-of-day, service-load, cache-warming, and operator-learning effects. Use blocked randomization:

  1. One block contains W1, W2, W4, and W8 exactly once.
  2. Shuffle the order within each block using a recorded pseudorandom seed.
  3. Use a fresh checkout or restored fixture state for every trial.
  4. Alternate the divisible and state-coupled task-set order across blocks.
  5. Do not reroll an inconvenient sequence.

Plan at least five completed trials per treatment per task set before interpreting medians or variability. This is a protocol target, not a claim that five trials guarantees statistical power. Determine a larger sample size from an expected effect and variance estimate if the decision justifies the expense.

Report every initiated trial, including stopped and failed trials. Excluding rate-limited or conflict-heavy runs after seeing them creates survivorship bias.

No paid run is authorized by this article. The protocol may be dry-run against mocks, local fixtures, or already-included capacity only after a human separately approves the exact execution. Do not convert this design into an API campaign automatically.

Independent acceptance

Write the acceptance suite before workers see the tasks. The evaluator must be independent of the workers and must not use worker self-assessments as evidence.

An artifact passes only if all applicable checks pass:

  1. output exists in the assigned scope;
  2. deterministic tests pass from a clean state;
  3. required schema or format validates;
  4. no forbidden files changed;
  5. no unexpected network, deployment, billing, or credential action occurred;
  6. the artifact is not a duplicate of an already accepted artifact;
  7. integration with previously accepted artifacts passes the full suite;
  8. a blinded reviewer accepts any criterion that cannot be automated.

Randomize artifact presentation to human reviewers and hide the worker-count label where practical. Use the same review rubric for all treatments. If reviewers disagree, record the disagreement and use a predeclared adjudication process.

Metrics and formulas

Let:

  • (n) = requested maximum active worker count;
  • (T_n) = end-to-end wall-clock hours for the trial;
  • (A_n) = number of independently accepted artifacts;
  • (G) = number of acceptance artifacts available in the fixed task set;
  • (D_n) = duplicate artifacts;
  • (R_n) = active review hours;
  • (U_n) = measured native usage units;
  • (F_n) = failed worker attempts;
  • (X_n) = textual plus semantic merge conflicts;
  • (M_n) = conflict-resolution hours.

First calculate coverage:

[ \text{Acceptance coverage}_n = \frac{A_n}{G} ]

Then calculate accepted-artifact throughput:

[ \text{Accepted throughput}_n = \frac{A_n}{T_n} ]

Raw speedup compares end-to-end time with the single-worker treatment:

[ \text{Speedup}_n = \frac{\operatorname{median}(T_1)}{\operatorname{median}(T_n)} ]

Classical concurrency efficiency is speedup divided by worker count:

[ \text{Concurrency efficiency}_n = \frac{\text{Speedup}_n}{n} ]

That formula assumes comparable completed work. When acceptance coverage differs, use an acceptance-adjusted efficiency:

[ \text{Acceptance-adjusted efficiency}_n = \frac{\operatorname{median}(A_n / T_n)} {n \times \operatorname{median}(A_1 / T_1)} ]

This ratio answers: “How much accepted throughput did each worker slot add relative to the one-worker baseline?” A value below 1 means per-slot efficiency fell, which is normal at some scale; the treatment may still be worthwhile if total accepted throughput rose enough and usage remained controlled.

Calculate duplication rate:

[ \text{Duplication rate}_n = \frac{D_n}{A_n + D_n} ]

Calculate review intensity and usage intensity:

[ \text{Review intensity}_n = \frac{R_n}{A_n} ]

[ \text{Usage intensity}_n = \frac{U_n}{A_n} ]

If (A_n = 0), report review and usage intensity as undefined, not zero. Zero would falsely imply efficiency.

For recovery and conflicts, report both counts and burden:

[ \text{Recovery failure rate}_n = \frac{F_n}{\text{worker attempts}_n} ]

[ \text{Conflict burden}_n = X_n + M_n ]

Do not treat the last expression as a dimensionless score; present the count and hours separately in the table. It is written compactly only to remind the analyst that conflict frequency and resolution time both matter.

Avoid collapsing everything into one weighted score until stakeholders declare the weights. A team that values rapid incident response may accept higher usage. A team with a fixed allowance may prefer slower completion with lower usage intensity.

Blank results table

Populate one row per trial. Do not replace stopped trials with averages.

Task setBlockTrial orderLabelRequested workersPeak active workersWall-clock minAccepted artifactsRejected artifactsDuplicate artifactsReview minInput tokensCached input tokensOutput tokensCredits or native unitsFailed attemptsRecovery minText conflictsSemantic conflictsConflict resolution minStop reason
W11
W22
W44
W88

After the trial-level table is complete, summarize medians and dispersion separately for each task set. Include the number of initiated, completed, failed, and stopped trials. Plot accepted throughput against usage intensity; a worker count that is faster but dramatically more expensive should be visible rather than hidden inside an average.

Stop conditions

The controller must enforce stop conditions mechanically where the product permits. A worker must not decide whether its own budget is “probably fine.”

Stop the affected worker when any of these occurs:

  • its maximum turns are reached;
  • its wall-clock timeout is reached;
  • it exceeds its assigned native-usage quota;
  • it attempts an unapproved tool, path, network destination, or side effect;
  • it produces repeated identical tool calls beyond the retry allowance;
  • it completes or irrecoverably fails its assigned unit;
  • it tries to create or delegate to another worker;
  • its output fails schema validation after the permitted correction attempt.

Stop the entire trial when any of these occurs:

  • the whole-run wall-clock limit is reached;
  • the maximum total worker creations is reached;
  • measured native usage reaches the trial ceiling;
  • the usage meter becomes unavailable or cannot be reconciled;
  • the authentication route changes unexpectedly;
  • an API key appears where subscription use was expected, or the reverse;
  • purchased usage, automatic top-up, deployment, publication, or an external write would be required;
  • two workers are observed writing to the same non-isolated workspace;
  • the fixture state or acceptance suite changes;
  • a credential, private data, or prohibited path is exposed;
  • the provider returns a rate-limit or safety response that the protocol did not preauthorize retrying;
  • the controller loses the ability to enumerate and stop active workers.

A stopped trial remains data. Record the trigger, last reliable usage sample, active workers, recoverable artifacts, and cleanup result. Do not automatically resume it under a higher budget.

Safeguards against recursive delegation

Recursive delegation turns a worker-count experiment into an uncontrolled branching process. Prevent it at three layers:

Configuration layer

  • Only the root controller receives a spawn capability.
  • Worker tool lists exclude agent creation, handoff creation, team creation, and generic skill or plugin entry points that could recreate delegation.
  • Set a hard maximum on total worker creations, not only active concurrency.
  • Give every worker a unique immutable ID and a parent ID.
  • Reject any spawn request whose parent is not the root controller.

Anthropic’s current subagent documentation says Claude Code subagents cannot spawn other subagents, which is a useful product boundary. Still verify the actual execution mode: agent teams, skills, plugins, an SDK wrapper, or external tools may introduce a different path. OpenAI’s SDK supports nested orchestration patterns, so an application using agents as tools must deliberately prevent workers from receiving further agent tools when recursion is outside the test.

Runtime layer

  • Maintain an atomic worker registry with created, running, stopping, and closed states.
  • Before every spawn, compare both active count and lifetime creation count with their ceilings.
  • Deny worker-supplied role definitions or rewritten prompts.
  • Emit an audit event for every accepted and rejected spawn.
  • Use controller-issued task IDs; workers cannot invent new benchmark units.
  • Shut down idle workers immediately after their assigned units resolve.

Acceptance layer

  • Reject artifacts produced by unregistered workers.
  • Reject results with missing lineage.
  • Count an attempted recursive spawn as a protocol violation, even if the platform blocked it.
  • Terminate the trial if lineage cannot be reconstructed.

The aim is not merely to avoid “too many agents.” It is to ensure that W4 always means no more than four active registered workers and that its total retry population remains bounded.

Safeguards against unbounded spend

Treat “included allowance,” “purchased credits,” and “API billing” as separate payment paths.

Before a trial:

  1. Confirm the authenticated account and product surface.
  2. Confirm whether an API key is present or selected.
  3. Capture the starting usage-meter state.
  4. Confirm purchased-credit balance and automatic top-up status.
  5. Set the trial’s paid-spend ceiling to zero unless a human has separately approved a fixed amount.
  6. Set both per-worker and whole-trial token, credit, turn, and time ceilings.
  7. Verify that exceeding a ceiling stops work rather than switching payment routes.

During a trial:

  • sample native usage after every model response where available;
  • reserve budget before launching a replacement worker;
  • stop launching new units when remaining headroom is smaller than the largest observed completed unit;
  • never interpret a delayed meter as free usage;
  • never retry a billing or rate-limit failure indefinitely;
  • keep auto-top-up disabled for an unpaid benchmark;
  • alert on authentication or billing-route changes.

After a trial:

  • wait for provider usage records to settle before finalizing cost fields;
  • reconcile request-level usage with the run total;
  • record missing or provider-estimated fields explicitly;
  • do not backfill unknown values from another model, plan, or surface.

This article authorizes no purchase, credit use, automatic top-up, API charge, or paid benchmark execution.

Decision tree: when not to parallelize

Use this decision tree before choosing a treatment for production work:

Does the job contain at least two independently acceptable work units?
|
+-- No --> Use one worker.
|
+-- Yes --> Can ownership be partitioned without concurrent writes to shared state?
            |
            +-- No --> Can each worker use an isolated workspace and a single controller merge?
            |          |
            |          +-- No --> Use one worker.
            |          |
            |          +-- Yes --> Continue.
            |
            +-- Yes --> Continue.
                        |
                        v
            Are interfaces and acceptance tests fixed before execution?
            |
            +-- No --> Use one worker to stabilize the contract first.
            |
            +-- Yes --> Is each unit large enough to repay startup and review overhead?
                        |
                        +-- No --> Batch the units under one worker.
                        |
                        +-- Yes --> Can usage, turns, retries, and total spawns be capped?
                                    |
                                    +-- No --> Do not parallelize.
                                    |
                                    +-- Yes --> Can the controller enumerate and stop every worker?
                                                |
                                                +-- No --> Do not parallelize.
                                                |
                                                +-- Yes --> Start with the smallest tested worker count
                                                            that meets the deadline and budget.

Even after the tree says parallelism is feasible, prefer one worker when the work is dominated by a single evolving design decision, a shared file, a fragile migration, high-bandwidth conversation, or sequential feedback. Parallel research may still help, but implementation should wait until the contract is stable.

How to interpret the completed benchmark

Choose the smallest worker count that satisfies the operational objective. Do not choose the treatment with the largest raw speedup automatically.

A higher count is justified when it:

  • improves median end-to-end wall-clock time by a practically meaningful amount;
  • maintains acceptance coverage;
  • keeps review intensity within the team’s capacity;
  • keeps usage intensity and total allowance burn within a fixed budget;
  • does not materially increase unrecoverable failures;
  • keeps conflict-resolution time below the time saved;
  • remains stable across repeated trials rather than winning through one outlier.

Reduce concurrency when accepted throughput flattens, duplication rises, reviewer queues grow, or recovery becomes coupled. If W8 returns worker outputs first but W4 reaches an accepted integrated state sooner, W4 is faster for the outcome that matters. If W2 is slightly slower than W4 but uses much less allowance and requires almost no conflict resolution, W2 may be the better default.

Keep task-set conclusions separate. It is reasonable for divisible work to favor more workers while state-coupled work favors one or two. A single organization may therefore need multiple concurrency profiles based on task classification rather than one global “agent count.”

Finally, repeat the benchmark after a material change to the model, agent runtime, prompt, repository architecture, tool set, metering rules, or acceptance suite. Old results describe the old system.

Reproducibility checklist

Before claiming a result is reproducible, confirm that the benchmark package contains:

  • the frozen fixture or repository revision;
  • task-set manifests and dependency graphs;
  • randomized treatment orders and seeds;
  • exact controller and worker configurations;
  • model and client versions;
  • tool and permission policies;
  • worker lineage and lifecycle logs;
  • start, finish, and integration timestamps;
  • request-level usage records;
  • raw worker artifacts, including rejected and duplicate outputs;
  • acceptance-suite source and machine output;
  • reviewer rubric and blinded decisions;
  • failure, retry, recovery, and conflict logs;
  • stopped-trial records;
  • a statement of the authentication and billing path;
  • the official documentation snapshot date and URLs.

If any of those are missing, describe the limitation. The honest outcome of a benchmark may be “inconclusive because usage metering was incomplete” or “not comparable because the service configuration changed.” That is more useful than a confident worker-count recommendation built on hidden variables.

Official sources checked

All sources below were checked on 2026-07-24:

  1. OpenAI Agents SDK, agent orchestration: https://openai.github.io/openai-agents-python/multi_agent/
  2. OpenAI Agents SDK, running agents and limits: https://openai.github.io/openai-agents-python/running_agents/
  3. OpenAI Agents SDK, usage tracking: https://openai.github.io/openai-agents-python/usage/
  4. OpenAI API, model guidance and multi-agent beta: https://developers.openai.com/api/docs/guides/latest-model
  5. OpenAI Help Center, flexible usage credits: https://help.openai.com/en/articles/12642688
  6. Claude Code, custom subagents: https://code.claude.com/docs/en/sub-agents
  7. Claude Code, agent teams: https://code.claude.com/docs/en/agent-teams
  8. Claude Code, cost management: https://code.claude.com/docs/en/costs
  9. Claude Code, CLI limits: https://code.claude.com/docs/en/cli-usage
  10. Claude Help Center, Claude Code with Pro or Max: https://support.claude.com/en/articles/11145838-use-claude-code-with-your-pro-or-max-plan

The benchmark should turn “more agents feels faster” into a falsifiable operational decision. Measure the integrated result, count only accepted work, expose the review queue, preserve failures, and cap the branching factor and payment path before execution. Parallelism is valuable when independent work outweighs coordination. The protocol’s job is to locate that boundary without discovering it through a depleted allowance.