The Agent Harness Should Be Open
Today, we are open sourcing TrueForge, the agent harness we use at TrueFoundry to build and run general-purpose agents. Because once agents ...
📝 摘要
TrueFoundry 宣布以 MIT 协议开源其 Agent Harness 框架 TrueForge。文章通过 DevRev Enterprise-Bench 的 14 项企业级任务基准测试证明：保持模型不变（Opus 4.8），TrueForge 比 Claude Managed Agents 节省约 30% 成本（仅用 40% 的 token）；切换到开源模型 GLM-5.2 后，成本降幅可达 75%，且解决率持平。核心原因在于 harness 层对上下文管理、工具调用效率和上下文压缩的精细控制，直接决定了 agent 的经济性。文章主张 Agent Harness 应该开源和厂商中立，使模型和基础设施成为可替换组件。
核心概念： Agent Harness 不是简单的编排层，而是 agent 经济学的重要组成部分——它决定了上下文如何管理、工具如何执行、以及多少模型预算在回答到达用户前就被消耗掉。
关键技术栈 / 关键要点：
- 上下文管理策略 — 更轻的初始载荷、更少更有针对性的工具调用、对旧上下文和大型工具响应的压缩（compaction）而非全部回放
- 厂商中立运行时 — 模型成为可替换组件，切换模型应是配置变更而非平台迁移项目
- MCP 工具集成 — 支持跨 CRM、Issue Tracker、文档库等多系统的 MCP 工具调用
- 沙箱与生态集成 — Daytona 代码沙箱、Exa/Tavily/Bright Data 检索、Together AI/Fireworks 推理、OpenUI 生成式 UI、Noma Guardrails
- 企业级治理 — 通过 TrueFoundry AI Gateway 统一模型访问、MCP 治理、Agent 控制、护栏、预算和可观测性
关键结论 / 实践建议： Agent Harness 层对行为和经济学影响太大，不应成为黑箱。团队应能检查、修改、自托管并替换其底层组件——模型应可替换，基础设施应可替换，Harness 应属于你自己。
Today, we are open sourcing TrueForge, the agent harness we use at TrueFoundry to build and run general-purpose agents. Because once agents do real work, the harness shapes context, tool use, execution, reliability, and cost.
It is MIT licensed, vendor-neutral, and runs on your infrastructure.
How much does the harness actually matter?
It is easy to think of an agent as a model with access to tools.
That description stops being useful pretty quickly.
Give an agent a task that takes 30 or 50 turns and the runtime around the model starts doing a lot of work. It decides what stays in context. What gets compacted. When to call a tool. What to do with a large tool response. When to delegate to a subagent. Where code gets executed. How state survives a long-running task.
Those choices affect whether the agent works.They also affect how much it costs.
Every tool call can create another model turn. Every large result added to context can get processed again on the next turn. A few thousand unnecessary tokens become much more expensive when they are carried through a long agent loop.
So before open sourcing TrueForge, we wanted to isolate one question:
If you keep the model and task exactly the same, how much can the harness change the economics?
Same agent. Same Accuracy. Up to 75% lower cost.
First, hold the model constant
We used 14 tasks from DevRev Enterprise-Bench.
These are cross-system enterprise tasks. The agent has to work across a CRM, an issue tracker, and a document store, call MCP tools, combine information from multiple systems, and produce an answer that satisfies a defined set of criteria.
Each harness got the same tasks, the same three MCP servers, and a fresh session for every task. The answers were graded by a blind LLM judge, which did not know which model or harness produced them.
The result:
Trueforge is 30% cheaper than Claude managed agents while keeping the model constant as Opus 4.8
The solve rates were same. The cost was not.
TrueForge reached roughly the same answers as Claude Managed Agents while using about 40% of the tokens and costing roughly 30% less per run.
The harness alone moved the bill.
Why?
Start with less. Every model turn re-processes instructions and tool definitions. A heavier starting payload gets paid for over and over again.
Make fewer tool calls. Every tool round trip means another model turn with a larger context. TrueForge made fewer, more targeted calls to reach the result.
Compact instead of replaying everything. Long-running agents accumulate a lot of history. TrueForge trims old context and large tool responses instead of feeding the entire history back into the model every time.
The harness is not just orchestration. It is part of the economics of the agent.
Then we changed the model
A leaner runtime gives you one cost lever.A vendor-neutral runtime gives you another.
We ran the same benchmark using GLM-5.2 with TrueForge.
The result:
Use open source models to save as much as 75% for same tasks while keeping solve rate same
Compared with Claude Managed Agents on Opus 4.8 at $11.8 per run, that is roughly 75% lower cost at a similar solve rate on this benchmark.
This is the part of vendor neutrality we care about most. It is about making the model a replaceable component.
A task that needs a frontier model can use one.A task that works just as well on a cheaper open model should not have to pay frontier-model prices.
And when a better model appears next month, switching should be a configuration change, not a re-platforming project.
Read detailed benchmarking blog here - https://www.truefoundry.com/blog/engineering/trueforge-vs-claude-managed-agents-benchmark/
That is why we think the harness should be open
The harness is becoming too important a part of the system to be a black box.
It determines how context is managed, how tools execute, how code is isolated, how subagents work, and how much of your model budget gets burned before an answer reaches the user.
If that layer affects the behavior and economics of your agents, you should be able to inspect it, modify it, self-host it, and replace the components underneath it.
Already being used in real agent systems
TrueForge already powers AskTFY, the AI copilot inside the TrueFoundry AI Gateway. AskTFY can reason across millions of production traces to investigate failures, understand application behavior, surface patterns, and recommend changes. It is the kind of long-running, tool-heavy workload TrueForge was built to handle.
And it is not just something we use internally. Teams including NetApp and Automatiq are already building agents with TrueForge.
We also believe an open harness should let you choose the best components around it, not force you to rebuild the entire stack.
That is why TrueForge plugs into a growing ecosystem:
Sandboxing: Daytona for isolated code execution.
Search & retrieval: Exa, Tavily, Parallel Web, and Bright Data.
Model inference: Together AI, Fireworks AI, Alibaba, plus any OpenAI-compatible endpoint.
Generative UI: OpenUI for dashboards, forms, tables, and interactive agent responses.
Guardrails: Noma Security through the TrueFoundry AI Gateway.
Own the agent. Own the stack.
The model landscape is moving too quickly to design an agent stack around today's winner.
Models will get better. Open models will get better.Inference will get cheaper.
New sandbox, retrieval, and tool providers will emerge.
Your agent runtime should give you the freedom to take advantage of all of it.
TrueForge stays open and vendor-neutral. For enterprise teams, it integrates with the TrueFoundry AI Gateway to centralize model access, MCP governance, agent controls, guardrails, budgets, credentials, and observability across the stack.
The model should be replaceable.
The infrastructure should be replaceable.
The harness should be yours.
TrueForge is open source today.