---
title: "Study 8: sixteen AI coding agents, a regression, then 29% fewer after the fixes, 28.5% on the real server · Cork"
description: "Sixteen Claude Code agents at once on a new, harder repo, three rounds each way. Cork's unreleased hooks v75 build (356d349) used 2% more tokens than no Cork (37.6M vs 36.8M per run): gate holds that never released, broadcast messages and a stale pull notice cost more than landing saved. After fixing them, a rerun on the fix build (aff950e, also unreleased) used 25.7M: 29% fewer tokens (± 10), 18% lower cost, 46/48 tasks correct vs 48/48. A later run of hooks v76 on the real staging server used 29.3M: 20% fewer (± 5), with three regressions. Hooks v78 fixed them: 26.1M tokens on the real staging server, 28.5% fewer (± 9), 18% lower cost, 46/48 tasks correct."
canonical: "https://corkcli.com/benchmarks/study-8-sixteen-agents-a-regression"
last_updated: "2026-11-14"
---

# Sixteen agents on a harder repo: a regression, then 29% fewer after the fixes, 28.5% on the real server

We ran sixteen Claude Code agents at once on a new, harder repository, three rounds with Cork and three without: same tasks, same prompts, same model. Cork lost. It ran on hooks v75, a build that was never released, and this study is what caught the regression. Landing still did its job: no rejected pushes. But three new behaviours in v75 cost more than landing saved. This page says which, what each cost, and what is being fixed.

+2%
:   tokens with Cork (± 2), 37.6M vs 36.8M per run

61%
:   of tokens spent re-pushing without Cork

0 vs 37.3
:   rejected pushes per round, with and without

48/48
:   tasks correct, vs 48/48 without

Update · Oct 17, 2026 · latest

## v78 on the staging server: 28.5% fewer tokens

Hooks v78 fixes the three regressions found in v76, described in the next update. We ran it on the same sixteen tasks, against the same control rounds, on the real staging server. It saved 28.5%, about what the fix build saved on a local server (29.4%), and it kept v76's low conflict and pull counts. v78 is not released.

−28.5%
:   tokens (± 9), 26.1M vs 36.8M without

−18%
:   cost, $9.93 vs $12.15

7.7
:   merge conflicts per round, vs 47 without; 9.3 manual pulls

46/48
:   tasks correct, vs 48/48 without

|  | Without | v76, staging | v78, staging |
| --- | --- | --- | --- |
| Tokens | 36.8M | 29.3M | 26.1M |
| Saved |  | 20.0% | 28.5% |
| Cost | $12.15 | $10.56 | $9.93 |
| Tasks correct | 48/48 | 47/48 | 46/48 |
| Wall clock | 388s | 429s | 412s |
| Merge conflicts | 46.7 | 7.7 | 7.7 |
| Manual pulls |  | 10.7 | 9.3 |
| Rejected pushes | 37.3 | 0 | 0 |
| Agents at once, average | 6.8 | 7.2 | 6.9 |

Means per round; v78 saved 28.5 ± 9.2% (rounds: 38.9%, 21.4%, 25.1%). Its token spread is the tightest of any Cork run here (±0.9M).

### v76's regressions, on v78

- **The repo id is back.** Commands spent hunting for it fell from 16.3 per run to 0.
- **No more “wait for them to land”.** Those holds fell from 6.0 per run to 0; a teammate creating a module is now something to build against. Waiting at the gate fell from 5.0M to 2.4M per run.
- **Messages once.** Messages shown again fell from 26.3 per run to 0, and messages cost 0.6M per run against 0.9M. (Counted the current way; the update below, counted the way of its day, shows v76 at 1.0M.)
- **Fewer tool calls,** 14 per run down to 8.7.
- **Fix lines now fire,** about 6.7 per run (0 on v76): a notice shows the exact old and new line of a teammate's change.

### What is still left

Reactions to change notices are now the largest cost Cork adds, about 1.45M per run with alerts. Two tasks failed: one left discounts in code instead of config, a miss every Cork build made once; in the other, an agent saw the rename notice and still kept the old name as an alias. Rename notices will say not to keep the old name. Three rounds and a ±9-point spread make 28.5% an estimate.

Update · Oct 17, 2026

## v76 on the staging server: 20% fewer tokens

We ran hooks v76, which carries the fixes below plus new features, on the same sixteen tasks against the same control rounds. This time Cork ran on its real staging server, not a local copy. It saved less than the fix build did: 20% against 29%. It fixed the merge conflicts and brought three regressions of its own. v76 is not released.

−20%
:   tokens (± 5), 29.3M vs 36.8M without

7.7
:   merge conflicts per round, vs 23 on the fix build and 47 without

10.7
:   manual pulls per round, vs 22.7 on the fix build

47/48
:   tasks correct, vs 48/48 without

|  | Without | Fix build, local | v76, staging |
| --- | --- | --- | --- |
| Tokens | 36.8M | 25.7M | 29.3M |
| Saved |  | 29.4% | 20.0% |
| Cost | $12.15 | $9.93 | $10.56 |
| Tasks correct | 48/48 | 46/48 | 47/48 |
| Wall clock | 388s | 365s | 429s |
| Merge conflicts | 46.7 | 23.0 | 7.7 |
| Rejected pushes | 37.3 | 0 | 0 |
| Agents at once, average | 6.8 | 7.0 | 7.2 |

Means per round; v76 saved 20.0 ± 5.3% (rounds: 26.1%, 17.3%, 16.6%). Its cost was 13% lower than without Cork.

### What v76 fixed

- **Fewer manual pulls, far fewer conflicts.** Agents now hear that landing rebases for them and that a manual pull can conflict. Pulls per round halved, from 22.7 to 10.7, and merge conflicts fell from 23 to 7.7.
- **Smaller briefings.** The prompt briefing is about 25% smaller and the session start about 35% smaller.
- **Merge previews.** Landing now says when a teammate's in-flight change will conflict with yours, about 31 times per run.

### What regressed

Per run, by cause, the fix build against v76 (both counted the same way):

| Tokens per round | Fix build | v76 |
| --- | --- | --- |
| Waiting at the gate | 2.7M | 5.0M |
| Messages | 0.6M | 1.0M |
| Alerts and change notices | 1.3M | 1.6M |
| Git-sync notice | 0.4M | 0.3M |
| Cork tool calls | 0.6M | 0.8M |

1. 1.**Briefings dropped the repo id.** Every Cork tool needs it, so agents went looking for it in config files and status commands: about 18 commands and 0.85M tokens per run, mostly right after a gate block.
2. 2.**“Wait for them to land” holds doubled,** from 2.7 to 6.0 per run, including on a module a teammate was still creating. One agent deleted the half-built module and rebuilt around it. Waiting at the gate went from 2.7M to 5.0M per run with about the same number of blocks.
3. 3.**Old messages came back.** Messages already shown were delivered again 26 times per run, against 5 before.

Some new features never fired on this work: fix lines for renamed code and duplicate-task warnings did not appear at all. Branch notes, handoffs and flaky-test verdicts did not fire either, as expected here: every agent was on one branch, none took over another's work, and the tests are not flaky.

### What it does not show, and what is next

The server changed as well as the build: the real staging stack, remote and backed by a database, against a local process for the fix build. So the gap between 29% and 20% is not v76's alone. Three rounds is few. Hooks v78 is being built to fix the three regressions, and it will be measured next, the same way, and recorded here.

Update · Oct 16, 2026

## After the fixes: 29% fewer tokens

This page promised a rerun after the fixes. We fixed the problems below and ran the same sixteen tasks three more times on the fix build (aff950e, still unreleased), against the same control rounds. The original result is kept below, unchanged.

−29%
:   tokens with the fix build (± 10), 25.7M vs 36.8M

−18%
:   cost, $9.93 vs $12.15

365s
:   wall clock, vs 388s without and 660s on the first build

46/48
:   tasks correct, vs 48/48 without

|  | Without | First build | Fix build |
| --- | --- | --- | --- |
| Tokens | 36.8M | 37.6M | 25.7M |
| Saved |  | -2.3% | 29.4% |
| Cost | $12.15 | $13.55 | $9.93 |
| Tasks correct | 48/48 | 48/48 | 46/48 |
| Wall clock | 388s | 660s | 365s |
| Merge conflicts | 46.7 | 19.3 | 23.0 |
| Rejected pushes | 37.3 | 0 | 0 |
| Agents at once, average | 6.8 | 5.1 | 7.0 |

Means per round; the fix build saved 29.4 ± 9.6% (rounds: 36.4%, 18.4%, 33.4%). Three rounds and a ±10 point spread make 29% an estimate, not a fixed number.

### The causes, before and after

| Tokens per round | First build | Fix build |
| --- | --- | --- |
| Gate holds that never release | 7.9M | 2.7M |
| Broadcast messages reach every agent | 2.0M | 1.0M |
| Rename and signature alerts reach agents that never call the code | 2.6M | 1.3M |
| A git-sync notice still tells agents to pull | 0.9M | 0.4M |
| Cork tool calls | 2.4M | 0.6M |

- **The gate holds once now.** Blocks per round fell from 45 to 17, and the most on one agent and file from 16 to 3.
- **Fewer messages.** Messages sent fell from 19 to 7 per round, and agents stopped answering every broadcast.
- **Alerts reach callers.** Rename and signature alerts fell by about 90%. Notices about work already pushed did not shrink, and notices about teammates' test files grew.
- **Tool calls** fell from 41 to about 11 per round.

### What is still left

Agents still ran about 35 manual pull or rebase commands per round, as many as on the first build. The notice just before them was usually a list of teammates' changes (32 times over three rounds) or a landing (18 times). Those manual pulls cause the 23 merge conflicts per round that remain, against 47 without Cork. The next fix: a notice about pushed work says landing will rebase onto it, and notices about other agents' tests are dropped unless the agent edits the same test.

The two failed tasks were not caused by Cork: one report counted cancelled bookings, and one task left discounts in code instead of config as asked. Neither build is released yet.

The original result · first build (356d349)

## Every round

Tokens for all sixteen agents in each round, bars to scale.

with Cork (v75)without

Round 1 · 0.2% fewer

41.1M

41.2M

Round 2 · 4.4% more

34.5M

33.1M

Round 3 · 2.7% more

37.0M

36.0M

## The numbers

Mean ± standard deviation per round over three rounds. Tokens are input, cache writes, cache reads and output, as the provider reported them for each agent. Correct means the task's hidden acceptance tests pass on the final shared origin.

|  | With Cork | Without |
| --- | --- | --- |
| Tokens | 37.6M ± 3.3M | 36.8M ± 4.1M |
| Cost | $13.55 ± 1.22 | $12.15 ± 1.18 |
| Tasks correct | 48/48 | 48/48 |
| Wall clock | 660 ± 187s | 388 ± 28s |
| Merge conflicts | 19.3 ± 2.3 | 46.7 ± 8.0 |
| Rejected pushes | 0 | 37.3 ± 8.4 |
| Model calls | 636 ± 51 | 695 ± 38 |
| Agents working at once, average | 5.1 ± 1.2 | 6.8 ± 0.3 |
| Median agent, seconds | 193 | 161 |

Per round, Cork used 0.2% fewer, 4.4% more, 2.7% more tokens, cost 12% more in dollars and took 1.7× as long. All sixteen agents start together in both arms; on average about 7 of them were working at any moment without Cork and 5 with it, out of 16.

## What landing still did

Without Cork, agents spent 22.3M of their 36.8M tokens per round (61%) in loops of a rejected push, a pull, a rebase, a retest and another try. In [Study 7](https://corkcli.com/benchmarks/study-7-scaling-to-sixteen-agents), at sixteen agents on an easier repo, that share was 43%. Cork's landing removed every rejected push and cut merge conflicts from 47 to 19 per round. The conflicts that remain come from agents running `git pull --rebase` themselves, which is the fourth problem below.

So the opportunity was larger here than in Study 7. The loss is not landing; it is what the rest of v75 added.

## Why Cork lost

What Cork itself cost its agents, per round, as a share of their tokens, beside Study 7's sixteen agents on hooks v74. The repos and tasks differ, so the shares compare roughly, not exactly. Every one grew, and waiting plus reactions now take about 41% of the Cork arm.

|  | Study 8, v75 | Study 7, v74 |
| --- | --- | --- |
| Notices carried | 5.4% · 2.0M | 4.1% |
| Reactions to notices | 20.2% · 7.6M | 13.4% |
| Waiting at the gate | 21.2% · 7.9M | 8.5% |
| Cork tool calls | 6.5% · 2.4M | 5.6% |

A reaction is the next one or two model calls after a notice when they pull, inspect git, re-read the file the notice names or call a Cork tool. Waiting is every call from a gate block on a file until the next write to it. A call can count in both, so the rows overlap.

### The causes, by tokens per round

1. 1.**Gate holds that never release.** Waiting at the gate: holds on files a teammate is changing, on unpushed changes and on shared test files. about 7.9M
2. 2.**Broadcast messages reach every agent.** A new session-start tip made each agent announce its plan to all the others. about 2.0M
3. 3.**Rename and signature alerts reach agents that never call the code.** Reactions to semantic alerts and change notices. about 2.6M
4. 4.**A git-sync notice still tells agents to pull.** Reactions to it; it drives most of the manual pulls that remain. about 0.9M
5. 5.**Cork tool calls.** About 41 calls per run, many of them answering broadcasts. about 2.4M

The gate tells an agent to fit its edit to a teammate's change and retry. Here each retry carried different content, or another change arrived, so the agent was blocked again. One agent was blocked fifteen times on one file while resolving its own rebase conflict; it ran 1022 seconds and used 12.8M tokens, 31% of its round. That agent is most of round 1's spread.

## What is being fixed

Written before the rerun; the update at the top of this page has the result.

- **The gate holds once.** A hold releases per file and session whatever the next write contains, and the gate never blocks resolving a file that is in conflict.
- **Messages go to those they concern.** Broadcast becomes opt-in, or reaches only agents whose files overlap, instead of every agent reading every plan.
- **No more “pull” with landing on.** The git-sync notice says what the per-file notice already says in v75: landing will rebase onto it, carry on.
- **Alerts reach callers only.** A rename or signature change is announced to agents that use the code, not to everyone.

By these measurements, fixing the first three is worth about 6.0M–8.0M tokens per round. That is an estimate, not a result. v75 was not released. We will rerun this study after the fixes and record the result, either as an update to this page or as the next study. Either way, these numbers stay here.

## How it was run

- **The repo.** A freshly generated coworking-space booking app, about 527 Python files: bookings and members behind a request router, a job queue, billing, reports, an admin dashboard, email, plus decoy modules whose names match the task words. Nothing is shared with Study 7's repo.
- **The work.** 16 of 32 tasks, each a three- or four-part feature asked the way people ask, mostly without file or function names. They share hot spots (the router, the booking function, pricing, the admin views, the tests), and include two renames and four signature changes that other tasks depend on.
- **With Cork.** Hooks v75, an unreleased build, on a local copy of Cork's server with prompts matched by meaning, as in production. Every run got a fresh account, paid workspace and repo.
- **Without.** The same agents, prompts and flags with no Cork.
- **Both.** Claude Sonnet 5, all sixteen agents launched at once, at most 80 turns each, one identical wake-up for any agent whose work had not reached the origin. Cross-session messaging was switched off in both arms, and no run used it.
- **Discarded.** Runs on earlier versions of the tasks and seed (tasks too short to overlap, and a seed missing a function two tasks needed) are not in any number here.
- **Cost.** About $77 of model usage for the six runs kept.

## What this study does not show

- How a released build does on this repo. Every Cork number here is v75.
- Larger teams. Runs at 24 and 32 agents are planned after the fixes; on v75 they would mostly measure these bugs.
- Three rounds is few. The spread (± 2 points) is small next to the result, but one long agent shaped round 1.
- One person's machine per run, with every agent started at the same moment rather than over a working day.

## The data

Every number on this page is in [data.json](https://corkcli.com/benchmarks/study-8-sixteen-agents-a-regression/data.json), generated from the runs' recorded sessions. The harness that ran them is not public yet.

[All studies](https://corkcli.com/benchmarks)[Study 7](https://corkcli.com/benchmarks/study-7-scaling-to-sixteen-agents)

This page in HTML: https://corkcli.com/benchmarks/study-8-sixteen-agents-a-regression