Signal SentrySearchUpdated Oct 28, 05:30 UTCAM Drop
Frontier watch · Orchestration & memory

Agent orchestration: does the harness beat the model?

Does the orchestration layer (decomposition, dispatch, verification, routing between agents) matter more than the base model, and which designs work?

Why it matters

If orchestration rather than the model decides whether long agent jobs finish, builders get more from how they split, check and route work than from switching models, and costs depend on how many agents a design spins up.

State of play

Newest first. Each claim is our summary of the linked primary source, with a short quote from it.

  1. ResearchOct 15, 2026Source

    A preprint reports that HTN-Orchestrate, which defers task decomposition until prerequisite results arrive, beat ReAct and Agent Team with the same base model: 89.73% on BrowseComp and 73.44% on SUPERChem.

    HTN-Orchestrate outperforms ReAct, Agent Team, and DeepSeek Harness using the same base model, achieving 89.73% accuracy on BrowseComp and 73.44% on SUPERChem, exceeding Agent Team by 2.45 and 4.69 percentage points, respectively.

    What it does not show: Preprint; margins over Agent Team are 2.45 and 4.69 points; results are the authors' own.

  2. ResearchOct 13, 2026Source

    Sakana AI researchers report in a preprint on The Agentic Scientist that their ablations suggest reliable scientific autonomy depends on the cross-agent orchestration layer as well as base model capability.

    Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-agent orchestration layer.

    What it does not show: Preprint; Sakana's project page also says the choice of coding harness (Codex, Claude Code, OpenCode, Pi) had a smaller effect than the underlying model.

  3. DocumentedOct 9, 2026Source

    Anthropic's release notes say Claude Managed Agents agents can now write dynamic workflows, in beta: a program that runs many agents in phases and combines their results, run by the server in the background.

    A workflow is a program that runs many agents in phases and combines their results.

    What it does not show: Beta. Anthropic's docs cap a run at 1,000 agents over its life and 64 working at once, so not 1,000 in parallel; no outcome data here.

  4. ResearchOct 9, 2026Source

    Researchers report in a preprint that sizing multi-agent collaboration to the backbone model's competence raised average accuracy from 87.22 to 89.97 for stronger models while cutting token cost by 36.26%.

    for higher-competence backbones, it improves average accuracy from 87.22 to 89.97 while reducing token cost by 36.26%; for lower-competence backbones, it yields a further average gain of 4.58 points over existing methods.

    What it does not show: Preprint; six reasoning benchmarks, not long-running agent tasks; self-reported.

  5. ResearchOct 7, 2026Source

    A preprint comparing matched Codex, Claude Code and Kimi Code runs with and without dynamic sub-agent spawning finds model capability drives short tasks, while orchestration becomes central on long-horizon work.

    Model capability largely determines outcomes on shorter tasks, whereas long horizon development makes orchestration central to task completion.

    What it does not show: Preprint; 354 tasks and 2,124 executions; it catalogues 13 concurrency-specific failure modes rather than reporting one win rate.

  6. ResearchSep 25, 2026Source

    Researchers report in a preprint that on AgentWorld, 100 tasks needing 3-20 agents over 50+ rounds, the best model tested reached 52.0% task success, with failures including communication breakdowns and lost shared plans.

    Experiments with Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, and DeepSeek R1-70B show that even the best model achieves only 52.0% task success, with systematic failure modes including communication breakdowns, role confusion, and inability to maintain shared plans across rounds.

    What it does not show: Preprint; four smaller models were tested (Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, DeepSeek R1-70B), not the labs' flagships.

What would change the picture

Signals we are watching for. Our own reading of where the thread could go: not predictions, and not sourced claims.

  • An independent benchmark scores the same base models under several orchestration designs and finds a consistent gap across labs' models Would suggest: Orchestration is a separable lever, not an effect of one vendor's model or harness
  • A flagship model clears 80% task success on a long-horizon multi-agent collaboration benchmark such as AgentWorld Would suggest: Coordination across many rounds and agents is becoming reliable rather than the main failure point
  • Anthropic or OpenAI moves its API multi-agent workflow feature from beta to general availability Would suggest: Fan-out orchestration becoming standard platform infrastructure rather than an experiment
  • A study shows multi-agent runs beating a single agent at an equal token budget on long tasks Would suggest: The gains come from the design, not just from spending more compute

From the Drops

AgentsResearch

Preprint: at Volvo, off-the-shelf agent harnesses cut monthly test costs 3-66%

Researchers compared Volvo Group's in-house GUI-testing harness with Claude Code and OpenCode, keeping MCP tools and models the same. On infotainment tests success matched, but inference cost per task was 2.6-10.7 times higher.

Why it matters: Counting engineering effort, a nightly regression suite cost 3-66% less a month, the authors report. Testers still wanted to inspect the agent's actions.

arXiv · Oct 27
From Drop #050 · Tue, Oct 27 · Evening
AgentsResearch

Preprint: DynaFlow trains a main agent to write its own multi-agent workflows

The main agent decides when to call sub-agents and generates programs that run them in sequence or in parallel, with verification. Only the main agent is trained; the sub-agents stay fixed.

Why it matters: With the same Qwen3.5-27B backbone, the authors report 88.08% overall accuracy across six benchmarks, 10.90 points above the strongest baseline they tested.

arXiv · Oct 27
From Drop #050 · Tue, Oct 27 · Evening
AgentsResearch

Preprint: the same coding model's success swung 44 points across harnesses

Researchers paired six open-source harnesses with three lightweight models on 100 SWE-bench Verified tasks, 1,800 graded runs. Harness rankings changed across models, the preprint reports.

Why it matters: Tokens per resolved task varied more than 20-fold even among similar harnesses, the authors say, making harness-model selection a central deployment decision.

arXiv · Oct 26
From Drop #048 · Mon, Oct 26 · Evening
AgentsResearch

Preprint: realistic MCP tool faults cut agent task scores by 60%, and scale didn't help

Researchers built MCP-Injector, with faults calibrated from 57,240 threads across 1,774 MCP repos. Task-completion score fell from 8.08 to 3.23 out of 10; a 120B gpt-oss kept no more of its score than a 20B.

Why it matters: The authors report agent stacks differed in absorbing faults, suggesting framework error handling matters more than model scale. Capability benchmarks can hide this.

arXiv · Oct 26
From Drop #047 · Mon, Oct 26 · Morning
AgentsResearch

Preprint: IF-MAP picks which agent in a multi-agent team to improve first

Researchers report that across six multi-agent workflows, tuning the prompts of the agents IF-MAP selects gained 3.81 percentage points on average, versus 1.85 when agents were picked uniformly.

Why it matters: The authors say it cut Qwen3-8B token use by 74.2% versus optimizing every agent: a cheaper way to aim improvement effort in agent teams.

Hugging Face Papers · Oct 17
From Drop #045 · Sun, Oct 25 · Morning
AgentsResearch

Preprint: a team of AI researchers with varied expertise beats GDN2 at 1.3B scale

Researchers report a team of persistent AI researchers from different disciplines, with private histories and shared evidence, found linear attention designs beating GDN2 at 1.3B scale; removing any piece weakened results.

Why it matters: The authors argue that who explores, what they remember and how effort is allocated drive autonomous discovery, not only stronger agents.

arXiv via Hugging Face · Oct 20
From Drop #044 · Sat, Oct 24 · Evening
AgentsResearch

Preprint: a shared memory buffer beats agent debate with 77.4% fewer tokens

Researchers report in a preprint that Muffin, where agents post findings to a shared buffer asynchronously, had the best accuracy in 17 of 20 settings and gained 19.2 points over multi-agent debate with 77.4% fewer tokens.

Why it matters: How agents share findings may matter more than how many agents you run; worth testing before adding more agents.

arXiv · Oct 23
From Drop #043 · Sat, Oct 24 · Morning
AgentsDocumented

AWS shares how its Deep Insight multi-agent analyst made it to production

AWS's Solutions Architecture team says eight agents split the work: a Haiku model routes requests, an Opus planner drafts steps for user approval, and a Validator re-runs each calculation. A code interpreter runs on AWS Fargate.

Why it matters: The lessons: route easy requests to small models, plan before acting, verify numbers. Note the web version auto-approves a plan after 300 seconds with no reply.

AWS ML Blog · Oct 21
From Drop #039 · Thu, Oct 22 · Morning
AgentsResearch

Preprint: multi-agent setups lost to single agents on τ²-bench's user-facing tasks

Across about 7,950 simulated conversations and six models, researchers report the multi-agent system was worse in 12 of 13 valid cells and tied in one, with a weighted mean gain of -28 points.

Why it matters: For agents that serve people, the authors say a single agent should be the default, and a multi-agent design must show compute-matched evidence from its own setting.

arXiv · Oct 21
From Drop #038 · Wed, Oct 21 · Evening
AgentsResearch

Preprint: evolving harness and policy together lifts a coding agent from 57.8% to 64.8%

Researchers report Self-HPO alternates harness changes the model proposes from its own failures with RL on the model. Across three coding benchmarks it raised Qwen3.5-35B-A3B's average resolve rate.

Why it matters: The authors say the joint gain beat the sum of harness-only and policy-only gains: a data point for teams deciding whether to tune the harness or the model.

Hugging Face Papers · Oct 16
From Drop #035 · Tue, Oct 20 · Morning
AgentsResearch

Cohere: a strong lead with cheap agent workers neared frontier results at 70% lower cost

On 200 ProgramBench tasks, Cohere says a Sol lead with 16 cheap Luna planning workers hit 61.8% reward vs 63.2% for an all-Sol team, cutting cost from $510 to $155. Teams gained up to ~10 points over solo agents at 3–7× the cost.

Why it matters: Cohere found where you put agents matters more than how many: delegating only planning matched delegating every stage (63.6% vs 63.2%) at half the cost.

Cohere · Oct 14
From Drop #034 · Mon, Oct 19 · Evening
AgentsResearch

Preprint: agent swarms that bid for turns learn to organize without a boss agent

Researchers report in a preprint that market-based post-training lets agents bid to decide who acts. Tested on crafting, robot navigation and deep research, learned self-orchestration beat other coordination strategies.

Why it matters: The authors report trained swarms adapted faster to related tasks and lost less performance when agents dropped out, a hint for decentralized agent designs.

Hugging Face Papers · Oct 15
From Drop #034 · Mon, Oct 19 · Evening
AgentsResearch

Preprint: MS-Eval finds sub-agent delegation often raises cost with uneven payoff

Testing four main-agent and sub-agent designs with eight frontier models on MCP-heavy tasks, researchers report specialized sub-agents work best. Delegation helped most open-source models but cost closed-source ones performance.

Why it matters: Gains came from context offloading and real parallelism, the authors say; unnecessary or redundant delegation turned the same cost into pure overhead.

arXiv · Oct 19
From Drop #033 · Mon, Oct 19 · Morning