---
title: "Study 3: four people, twelve AI coding agents, vague tickets, measured · Cork"
description: "Teams of Claude Code agents on one repo, from named tasks to vague tickets in a 654-file codebase. On the hardest tier the agents on Cork used 47% fewer tokens (7.82M vs 14.72M), all tasks correct both ways. What each agent was told, and what it did with it."
canonical: "https://corkcli.com/benchmarks/study-3-teams-easy-to-hard"
last_updated: "2026-09-28"
---

# Four people, twelve agents, vague tickets: what Cork changed, measured

Several people each launched several Claude Code agents at once, all pushing to one origin, in three tiers from named tasks to vague tickets in a 654-file repo. On the hardest tier, in the final round, the agents on Cork used 7.82M tokens and the agents without it 14.72M: 47% fewer, and 12 of 12 tasks correct against 12. The first round of that tier was a tie and exposed defects in Cork; later rounds added landing, which takes the push-rebase-retest loop out of the model, and a notice when a teammate starts the file your task is about. The last round also ran with the Ponytail plugin, alone and alongside Cork. This page shows every round, what we fixed between them, and what each agent was told and did.

47%
:   fewer tokens on the hard tier, final round, same run

5.4 vs 10.4
:   searches before the first edit, hard tier

14 vs 25
:   turns per agent, hard tier

50%
:   fewer tokens with Cork and the Ponytail plugin together

## The setup

Each person signs in to Cork the way a team does: the owner runs setup and commits it, each teammate connects with their own login. Every agent works in its own clone and must commit and push to the shared origin with the tests passing. The same prompts ran without Cork, with no MCP server and no hooks. Tasks were graded by hidden tests on the final origin, never by what the agents said they did. Model: Claude Sonnet 5.

Easy

2 people · 4 agents · 158 files

Two people, two agents each. The prompt names the file and the function.

In tillhouse/inventory/watch.py, shortfalls() skips items with on\_hand == 0. Flag every item with on\_hand < reorder\_point, including zero.

Medium

3 people · 6 agents · 158 files

Three people, two agents each. Vague tickets, written the way people write them; one agent renames a function two others depend on.

low stock alerts arent going out when something actually runs out completely. fix it

Hard

4 people · 12 agents · 654 files

Four people, three agents each, a 654-file repo where 480 modules are lookalike helpers nothing uses. Vague tickets, a rename others depend on, and one task that must build on a teammate's module that does not exist yet.

put total refunds on the dashboard as its own tile called refunds. someone on the team is writing a refunds report module right now, use it instead of rolling your own

## Results by tier, final build

Each table is one run: both conditions the same day, same prompts, same model. The Cork column is marked where it did better.

Easy · round 3

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 4/4 | 4/4 |
| Tokens, all agents | 1.16M | 2.52M |
| Cost | $0.63 | $0.97 |
| Turns per agent | 7.8 | 16.8 |
| Searches before first edit | 2.0 | 4.2 |
| Seconds per agent | 34 | 55 |
| Until every task was pushed | 41s | 76s |
| Agents that stopped and needed a nudge | 0 | 0 |

Medium · round 4

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 6/6 | 6/6 |
| Tokens, all agents | 3.35M | 5.58M |
| Cost | $1.60 | $2.21 |
| Turns per agent | 13.5 | 21.0 |
| Searches before first edit | 5.5 | 10.0 |
| Seconds per agent | 99 | 82 |
| Until every task was pushed | 173s | 157s |
| Agents that stopped and needed a nudge | 0 | 0 |

Hard · round 5

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 12/12 |
| Tokens, all agents | 7.82M | 14.72M |
| Cost | $3.48 | $5.39 |
| Turns per agent | 14.0 | 24.8 |
| Searches before first edit | 5.4 | 10.4 |
| Seconds per agent | 80 | 91 |
| Until every task was pushed | 193s | 237s |
| Agents that stopped and needed a nudge | 1 | 1 |

Where tasks name their files, there is little to find and little to save. The savings grow with vagueness and size, because what Cork removes is the searching: the agents without it averaged 10.4 searches before their first edit on the hard tier, the agents on Cork 5.4.

## The hard tier, round by round

Without Cork the hard tier cost 13.60M, 14.41M, 13.39M, 15.08M, 14.72M across 5 rounds: the spread one condition shows run to run. With Cork it cost 13.94M, 11.37M, 10.08M, 8.42M, 7.82M, because we fixed what each round exposed before running the next.

Round 1

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 12/12 |
| Tokens, all agents | 13.94M | 13.60M |
| Cost | $5.23 | $5.11 |
| Turns per agent | 19.1 | 23.6 |
| Searches before first edit | 8.8 | 9.8 |
| Seconds per agent | 132 | 87 |
| Until every task was pushed | 352s | 200s |
| Agents that stopped and needed a nudge | 1 | 1 |

Round 2

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 12/12 |
| Tokens, all agents | 11.37M | 14.41M |
| Cost | $4.49 | $5.30 |
| Turns per agent | 21.1 | 25.1 |
| Searches before first edit | 5.0 | 11.7 |
| Seconds per agent | 105 | 97 |
| Until every task was pushed | 242s | 231s |
| Agents that stopped and needed a nudge | 0 | 1 |

Round 3

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 12/12 |
| Tokens, all agents | 10.08M | 13.39M |
| Cost | $3.98 | $4.93 |
| Turns per agent | 21.0 | 22.5 |
| Searches before first edit | 6.2 | 8.8 |
| Seconds per agent | 96 | 86 |
| Until every task was pushed | 206s | 273s |
| Agents that stopped and needed a nudge | 0 | 1 |

Round 4

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 11/12 |
| Tokens, all agents | 8.42M | 15.08M |
| Cost | $3.69 | $5.59 |
| Turns per agent | 16.0 | 24.9 |
| Searches before first edit | 6.1 | 11.4 |
| Seconds per agent | 83 | 87 |
| Until every task was pushed | 189s | 200s |
| Agents that stopped and needed a nudge | 0 | 1 |

Round 5

| Measure | Cork | Without |
| --- | --- | --- |
| Tasks correct | 12/12 | 12/12 |
| Tokens, all agents | 7.82M | 14.72M |
| Cost | $3.48 | $5.39 |
| Turns per agent | 14.0 | 24.8 |
| Searches before first edit | 5.4 | 10.4 |
| Seconds per agent | 80 | 91 |
| Until every task was pushed | 193s | 237s |
| Agents that stopped and needed a nudge | 1 | 1 |

### What we fixed between rounds

1. A teammate's unpulled work is no rename

   One person's agents share a file record but not a checkout. Carol's h8 wrote a test file from a copy that never had bob's new test and added its own; the server called the difference a certain rename and blocked carol's h9 three times. Each agent's own copy is now remembered from its reads and writes; a name its copy never had is not one it renamed or removed.
2. Say origin moved only when it moved your files

   Agents stopped to pull on every "origin is N commits ahead" before they had edited anything; h4 pulled three times. The note now fires only when the incoming commits change a file the checkout changed, and names it. (Hooks v38.)
3. Rank meaning with the graph

   In the 654-file repo the prompt briefing named the file to change for 3 of 12 agents: "french customers get free shipping" sat closer to a lookalike helper nothing uses than to shipping/quote.py. A file wired into the rest of the code now outranks one wired to nothing. On the same prompts, locally: 3 → 11 of 12 (the twelfth file does not exist until its agent writes it); in rounds 2 to 4, 10 of 12 each.
4. No certain absence from a copy nobody saw

   After round 2: an agent that read a file with cat, which Cork never sees, wrote it without a teammate's new test, and bob's agent was told its own test "was removed". With no remembered copy, no rename or removal is certain. Deployed before round 3.
5. Tell an agent only what can break it

   After round 3: agents were told of every teammate addition and body edit and answered nearly all "unrelated, no action needed". That notice now carries only removals and new signatures. (Hooks v39.)
6. Land the push for the agent

   Round 3 showed where the rest of the session went: over half of every agent's tool calls, with or without Cork, were getting work into origin. Push, rejected, pull --rebase, rerun the tests, push, rejected again; bob's agent went round three times while carol's waited on his module. An agent's git push now runs collide-hook land: one push turn per repo at a time, a rebase onto teammates' commits, a teammate's landed rename applied to the agent's own commits, the tests when anything came in, and the push, in one step with one answer. (Hooks v40, round 4.)
7. Keep your name until the rename lands

   Told a teammate's rename was in flight, agents switched to the new name before it reached origin, then waited on failing tests. They are now told to keep the old name until it is in their checkout; landing applies it after the rebase. A landing also ends its in-flight notices, which had outlived the pushed work by minutes. (Round 4.)
8. Say when a teammate starts the file your task is about

   The hard tier finishes when carol's agent does: it must build on the refunds module bob's agent is writing. In round 4 it spent about 70 seconds concluding no such module existed. Now the server keeps each session's task as a vector (never its words) and checks every teammate's new, unpushed file against it; a close match is one notice with the interface. In round 5 carol's agent was told at 34 to 43 seconds that bob had just started refund\_report.py, and built on it.
9. A first write is compared against another copy too

   Medium tier, on landing's build: an agent's first write to the shared test file was compared against the map's copy, which was another agent's, so two different new tests read as a rename; the gate then held a third agent five times, because its up-to-date copy had both names. First writes now get the same rule as any other copy, and a copy that has the new name clears the gate. The medium tier was rerun after this fix.

## With Ponytail, a plugin that makes agents write less code

[Ponytail](https://github.com/DietrichGebert/ponytail) changes what one agent writes: before writing, it checks whether the code needs to exist, whether the codebase or the standard library already has it, and only then writes the minimum. Cork changes what each agent knows about the code and its teammates. They do different jobs, so round 5 ran the hard tier four ways, the same day: neither, Ponytail alone, Cork alone, and both.

| Round 5 | Tokens | Cost | Turns | All pushed | Lines added | Correct |
| --- | --- | --- | --- | --- | --- | --- |
| Neither | 14.72M | $5.39 | 24.8 | 237s | 64 | 12/12 |
| Ponytail | 13.20M | $4.72 | 24.6 | 159s | 70 | 12/12 |
| Cork | 7.82M | $3.48 | 14.0 | 193s | 53 | 12/12 |
| Both | 7.38M | $3.41 | 13.9 | 150s | 62 | 12/12 |

The savings stack: Ponytail alone saved 10% of tokens, Cork alone 47%, both 50%, and both finished soonest. Ponytail's own measure, lines of code, barely moved here, because these tickets are small bug fixes with nothing to over-build; its authors measure feature work. One run per arm: the runs without Cork varied by about 7% from round to round, so the gap between Cork and both is within that.

## Where each agent's tool calls went

Per agent, hard tier. Before landing, Cork had cut the searching but not the git loop, which took as many calls with Cork as without. With landing, the git calls fell to about a third.

| Calls per agent | Cork, round 3 | Cork, round 4 (landing) | Without, round 4 |
| --- | --- | --- | --- |
| Searching and reading | 6.4 | 7.0 | 10.4 |
| Git: commit, push, pull, rebase | 8.2 | 4.0 | 8.9 |
| Running tests | 2.5 | 1.5 | 2.5 |
| Editing | 1.5 | 1.9 | 3.1 |
| Cork tools | 1.0 | 0.6 | 0.0 |
| Waiting | 0.3 | 0.0 | 0.0 |

## Building on work that is not pushed yet

One hard task told carol's agent to put refunds on the dashboard using the refunds report bob's agent was writing at that moment. Without Cork there is nothing to find: the module exists only on bob's machine. In 5 of 5 rounds the agent without Cork searched, found nothing, and stopped to ask a person. On Cork it saw bob's module while bob was still writing it, with its interface, waited for the push, and built on it. In round 1 it stalled too, blocked by the false rename above. Since round 5 it is told the moment bob starts the file, within the first minute.

## What agents were told, and what they did

Cork's hooks put everything in the agents' context; none of it needs a tool call. In the final hard round the twelve agents received 10 prompt briefings that named the file to change, 14 exact answers attached to their searches (a symbol's signature and callers), 0 notices of a teammate's unpushed work they build on, 20 notices of teammates' changes, and 0 blocked writes. In round 1 there were 3 blocks, all false.

Hard, round 3, agent h9 (carol)

Told · Prompt briefing: the real modules for refunds and the dashboard; the lookalike ledger\_archive helpers ranked below them.

> Found it — s5-bob is actively working on `tillhouse/reports/refund\_report.py`, which didn't exist a minute ago. That's the real refunds report module the task meant; the `ledger\_archive` matches were decoys (dead code, "not the storefront path", zero callers).

It built its tile on bob's refunds\_by\_day once bob pushed, and finished without help.

Hard, round 3, agent h9 without Cork

Told · Nothing: the teammate's module was still on bob's machine.

> I checked before building anything, and I can't find a refunds report module anywhere … Before I write anything, could you point me to where it actually is?

It stopped and waited for a person. The same happened in every round without Cork.

Medium, round 1, agent m1 (alice)

Told · Gate: s5-bob renamed settled\_total in tillhouse/finance/daybook.py (now paid\_total). Pull that change and re-read the file.

> s5-bob just renamed `settled\_total` to `paid\_total` in that same file — likely fixing this exact refunds bug. Let me pull their change and adapt.

Both changes landed; neither was lost.

Hard, round 2, agent h2 (alice)

Told · Δ tests/test\_features.py: test\_checkout\_us\_has\_no\_levy body changed; test\_checkout\_uk\_has\_levy added — s5-bob.

> s5-bob just changed `test\_checkout\_us\_has\_no\_levy`'s body and added a new test `test\_checkout\_uk\_has\_levy` in the same file I edited. Let me check the merged result to make sure the rename applied consistently to bob's changes too.

A rename that reached a test written minutes earlier by someone else.

Hard, round 1, agent h9 (carol): a false block

Told · Gate: another session of yours renamed test\_refunds\_by\_day in tests/test\_features.py (now test\_welcome\_discount\_is\_one\_time\_only).

> Another session of mine renamed `test\_refunds\_by\_day` to `test\_welcome\_discount\_is\_one\_time\_only` in tests/test\_features.py — pulling that in before I retry my edit.

No such rename existed. Blocked three times, it spent 4.1M tokens against 1.1M without Cork. Fixed before round 2.

Hard, round 2, agent h1 (alice): noise

Told · Teammates changed code you may depend on: s5-dave modified dump (implementation changed, signature unchanged).

> s5-dave just changed `dump`'s implementation in tillhouse/exports/ledger\_dump.py (signature unchanged) — unrelated to this fix, so no action needed there.

Twenty notices like this in one round, nearly all answered this way. Cut in hooks v39.

Hard, round 4, agent h11 (dave): landing

Told · Its own git push, run as collide-hook land.

> Cork landed 1 commit(s) on origin/main (18e35db). Waited 2s for s5-dave's push. Rebased onto 4 new commit(s) from h2, h7, h12, h10. `'python3' -m pytest -q -p no:cacheprovider --no-header` passed on the rebased tree. It is pushed; nothing else to do.

One tool call. In round 3, bob's agent spent four pushes, three rebases and three test runs on the same situation.

## Where Cork still costs

- **Time, measured fairly.** An agent that stops to ask a person finishes its first run early, which made the runs without Cork look faster when we timed only first runs. Counted until every task was pushed, with the stalled agent restarted at once (a real person takes longer), the hard tier finished in 352s vs 200s, 242s vs 231s, 206s vs 273s, 189s vs 200s, 193s vs 237s across the rounds. The easy and medium tiers, where nobody stalls, finished in 41s vs 76s and 173s vs 157s.
- **Waiting for a teammate is still waiting.** The agent that builds on bob's module is the last to finish in every round with Cork; it cannot push before bob does. Without Cork the same agent stops and asks a person instead.
- **A real block costs time.** In the medium tier bob renamed settled\_total to paid\_total while fixing the same refund bug alice's agent was on. The gate stopped alice's write once; her agent pulled bob's version and built on his fix instead of writing a second one. That is what the gate is for, and it made hers the slowest agent in that run.
- **Searching.** About six searches before the first edit remain. Claude Code insists on reading a file before editing it, so part of that is a floor.

## Grading, disclosed

Two hidden tests were loosened after the first runs, for both conditions and every run alike, because they demanded more than the tickets said. France's shipping had to equal one exact rate the ticket never names; now any charge passes. The WELCOME10 ticket (“used over and over by the same person”) was graded as once per customer, while the code's own note says first order only; either fix now passes. All runs were regraded with the same file.

## Against the earlier studies

Study 2 measured 5.3× fewer tokens per agent, and this study measures 47% fewer (1.9×). They measure different things. In Study 2 ten agents did the same kind of change one after another, so every agent after the first replayed a verified recipe. Here every agent had a different task, at the same time, so there was nothing to replay: the savings come from finding the code, knowing what teammates are doing, and landing each push in one step. We report both as measured.

## What this study does not show

- The people are test logins on our staging server, not a real team, and the repos are generated.
- The hard repo's 480 decoy modules are adversarial by design; a real repo's dead code is less uniform.
- The Cork build changed between hard rounds; the rounds show the effect of the fixes and the run-to-run spread, not repeated samples of one build. Every tier's final round ran with landing; medium's ran after one more server fix, and the hard tier's final round also with the new-file notice, both described above.
- Agents that ended their turn waiting were resumed once with the same message in both conditions; their tokens count.

[All studies](https://corkcli.com/benchmarks)[Connect your agents](https://corkcli.com/docs)

This page in HTML: https://corkcli.com/benchmarks/study-3-teams-easy-to-hard