What an AI harness is and how harness engineering applies to enterprise coding agents
Published · updated
In AI, a harness is everything in an agent except the model: the instructions and documents it receives, the tools it may call, the rules and limits that apply, and the checks that test its work.
Harness engineering is the work of building the environment an AI agent operates in: the context it receives, the tools it can call, the rules that apply to it, and the point where its autonomy stops. For a coding agent, most of that environment lives in and around the repository: instruction files, linters, tests, permissions, and the human review that decides what reaches production. Harness engineering, autonomy limit and session record have short definitions in the glossary of software development with coding agents.
In 2024, a team at Princeton ran the same model, GPT-4 Turbo, on the same 300 real GitHub issues under two setups. Working through a plain Linux shell, it resolved 11% of them. Working through an interface the researchers built for it, it resolved 18%. That interface included a file viewer that shows 100 lines at a time, a search command that returns at most 50 results, and an edit command that rejects any change that fails a lint check for syntax errors and undefined names. Removing only that check brought the result down to 15%.
The term for that surrounding layer entered circulation in February 2026. OpenAI published a field report called Harness engineering that month, and Birgitta Böckeler, a Distinguished Engineer at Thoughtworks, posted her first memo on the term on martinfowler.com a few days later. By mid-2026, arXiv preprints were treating it as a discipline of its own. This page defines the term and looks at what changes when the codebase is an enterprise system with years of history behind it.
What is harness engineering
Practitioners sum it up as Agent = Model + Harness. Böckeler notes that the term has come to mean "everything in an AI agent except the model itself," then narrows it for coding agents. Part of the harness ships with the tool: the system prompt, the way it retrieves code, its orchestration. The rest is built by the team that uses the agent, for its own system. She calls that second part the outer harness, and it is the part an engineering organization owns.
A source-code study of eleven coding agents, among them Claude Code, Codex CLI, Gemini CLI, OpenHands and Aider (Barbaste et al., Wavestone AI Lab, a July 2026 preprint), maps every harness into seven subsystems: the agent loop, model integration, tools and actions, memory and context, safety and permissions, orchestration, and extensibility. In the paper's map, extensibility is the subsystem that lets users add capability, through configuration, hooks, skills, plugins and MCP servers. Our reading, which goes beyond what the paper states: together with permission settings, that layer is where most of a team's outer harness plugs in.
Böckeler sorts the outer harness into two kinds of control:
- Guides act before the agent writes anything: coding conventions in an
AGENTS.mdfile, skills, codemods. They raise the odds that the first attempt is right. - Sensors act afterward: tests, type checkers, linters, structural tests, an AI reviewer. They give the agent a signal it can correct against.
Either kind can be computational (a linter: deterministic, fast, cheap) or inferential (a model judging the output: slower, more expensive, less predictable). The two kinds need each other. With sensors alone, in her words, you get "an agent that keeps repeating the same mistakes"; with guides alone, one "that encodes rules but never finds out whether they worked."
Not everyone draws the boundary in the same place. Ralph Bean of Red Hat separates the harness, which adds knowledge and tools to make the agent more competent, from the sandbox, which removes capabilities to limit risk. Böckeler and the source-code study keep permissions inside the harness. An enterprise needs both, whatever it calls them.
How it differs from prompt and context engineering
A prompt is an input to one session. A harness persists across sessions, and much of it is enforced by code rather than by wording. Böckeler calls the user's harness a specific form of context engineering: context engineering supplies the means to put guides and sensors in front of the agent, and the harness is what a team builds with them for its codebase.
Research on where the gains come from points the same way. In a 2026 preprint by researchers at Fudan University and collaborators (Lin et al.), an agent rewrote a coding harness over ten rounds, and the pass rate on the Terminal-Bench 2 benchmark rose from 69.7% to 77.0%. The ablations traced the gain to tools, middleware and long-term memory rather than to the system prompt. OpenAI's field report points the same way. Their single large AGENTS.md "failed in predictable ways," so they cut it to roughly 100 lines that point into a versioned docs/ directory, and they moved rules into custom linters whose error messages tell the agent how to fix the violation.
Harness engineering vs. Harness.io
Harness (harness.io) is a software company. Its homepage describes "a unified AI software delivery platform to manage the SDLC using purpose-built AI agents," covering builds and deployments, security testing, runtime protection and cost, plus a newer agent that takes engineering work from ticket to code. Its documentation still covers continuous integration and continuous delivery. Harness engineering, in the sense used by OpenAI, Thoughtworks and the arXiv literature, is an engineering practice, not a product, and Ithaca makes no claim to the Harness name.
The two can still coexist in the same company. The agent harness governs what a coding agent may read, call and change; a delivery platform builds, tests and releases the result. When an RFP or a vendor deck says "harness" without a qualifier, ask which one is meant.
Harness engineering for enterprise coding agents
OpenAI's report is one of the most detailed public accounts of a harness, and its conditions matter. The team started from an empty repository in late August 2025 and, over five months, produced about a million lines with three to seven engineers and no hand-written code. Humans set priorities and validated outcomes, but they "may review pull requests, but aren't required to," and the repository runs "with minimal blocking merge gates." The authors warn that the end-to-end autonomy they reached "depends heavily on the specific structure and tooling of this repository and should not be assumed to generalize without similar investment," adding: "at least, not yet."
An enterprise codebase usually starts from the other end. Four differences change what the harness has to do.
The code already exists. Greenfield teams can build harnessability in from the first commit. Teams with legacy code, Böckeler writes, "face the harder problem: the harness is most needed where it is hardest to build." Strong types and clear module boundaries, which she lists among the properties that make a codebase easier to harness, are often missing from a ten-year-old system, and so is a test suite the team trusts. Building them is part of the harness work.
The blast radius crosses repositories. A Thoughtworks blog post by Jaya Simha Reddy Nandyala and Prabina Pani walks through a simplified case: an agent renames a field in one service, passes that service's tests, and silently breaks the two services that consume the field. Their answer is a shared harness that scans the dependency graph and stops at a confirmation gate when a change spans repositories. Human confirmation is reserved for "irreversible or high-blast-radius choices," such as schema migrations and public API changes; lower-risk work proceeds on defaults.
Someone has to answer for the change. Human developers, Böckeler notes, "know that our name is on the commit," while a coding agent has "no social accountability." In an audited company, that accountability has to sit somewhere in the process. A harness can make review cheaper and better aimed, but not unnecessary. Her own conclusion is that a good harness "should not necessarily aim to fully eliminate human input, but to direct it to where our input is most important."
The work spans many sessions. Anthropic's engineering team describes coding agents that, working across many context windows, tried to build a whole application at once or declared the job finished too early. In their harness, the first session writes a feature list with every item marked as failing, a progress file and an initial git commit, so each later session can read what happened and take one feature at a time. In an enterprise, those same artifacts are what a reviewer needs to reconstruct the work afterward.
In practice, an enterprise harness for coding agents has six parts:
| Part | What it means | Who decides |
|---|---|---|
| Context | What the agent reads: approved documentation, the relevant parts of the codebase, anonymized data | Your team |
| Rules | Conventions and prohibitions, moved from prose into linters and tests wherever they can be checked mechanically | Your team |
| Limits | What the agent may do without prior approval, and what needs a person's sign-off first | Your team |
| Tools | Which tools and MCP servers the agent may call, with least privilege | Your team |
| Mechanical feedback | Tests, type checks and structural checks that return an actionable failure to the session | Your team and the platform |
| Session record | The request, the steps, the model, the cost, the result and the authorship of each session | The platform, read by your team |
Examples of an AI harness in coding agents
One example for each part of the table above, from sources this page already cites. They are design examples, attributed to whoever described them, not accounts of Ithaca projects.
- Context. A short
AGENTS.mdfile that works as a map to a versioneddocs/directory, not the whole manual. OpenAI described that switch in February 2026, after a single large instruction file failed. - Rules. A convention moved out of prose and into a check: a custom linter that enforces which layers may depend on which, with an error message that tells the agent how to fix the violation. In the same report, when documentation falls short, the team promotes the rule into code.
- Limits. The agent stops and asks a person to confirm before a schema migration or a change that spans repositories. In the pattern Jaya Simha Reddy Nandyala and Prabina Pani describe on the Thoughtworks blog in September 2026, those gates are reserved for irreversible or high-blast-radius choices, and lower-risk decisions proceed with sensible defaults.
- Tools. The list of tools and MCP servers the agent may call, with least privilege. In the same post, an agent that only verifies code gets read-only tools and cannot write files.
- Mechanical feedback. Tests, type checkers, linters and structural tests that run after the agent acts and return a failure it can correct against. Birgitta Böckeler, on martinfowler.com, calls these checks sensors and what acts before them, such as the conventions in an
AGENTS.mdfile, guides. - Session record. For work that spans many context windows, Anthropic's engineering team has the first session write a feature list with every item marked as failing, a progress file and an initial git commit, so the next session can read where the work stands.
The harness prepares and limits the work; the session record shows what happened. On Ithaca's agentic development platform, that record is a separate layer, coding agent observability, and a gate at run time does not replace reading the code: people on the team read what goes to production.
The harness and the session record are separate layers
The harness decides what a session can do. Observability records what it did. A harness nobody can audit cannot be improved on evidence, and a detailed log of an unconstrained agent records the damage without limiting it.
The harness-evolution preprint cited above calls its method "observability-driven": each harness component had to be a file the agent could edit and revert, millions of tokens of trajectories were condensed into evidence it could read, and every edit came with a prediction checked against the next round. For a team improving its own harness, the equivalent evidence is the session record: what was asked, what the agent did, which model ran, what it cost and who the author was.
Where the harness sits in an agentic development platform
Ithaca operates an agentic development platform at level 4 of the autonomy scale Dan Shapiro proposed. Coding agents execute and humans supervise: the team writes the specification, sets the limits, decides what the agents may do without prior approval, and reads what goes to production. Every session is logged, with its authorship. Level 5, where nobody reads the code before it ships, is a dark factory, and the platform stops before it.
Harness engineering is a discipline inside that platform, and it does not remove risk: a failure is still possible, but the harness narrows what it can reach and makes its cause easier to find in the session. Context comes from the Knowledge base, which takes in the existing code, documentation and anonymized data before any new line is written. Rules and limits apply while the agents work. Human review stands between the agents and production, and the session record lives in the platform's observability. The same structure applies to legacy modernization without stopping the business, where the harness has to be built around a system that is still serving customers.
Ithaca also offers AI Harness and observability work for teams that want to govern their own coding agents. This page is a definition and a method, not a case study. The client case on this site, Dr. Consulta, is a deployment of the agentic development platform and is described on the legacy modernization page.
Frequently asked questions
What is an AI harness?
In AI, a harness is everything in an agent except the model. A common shorthand is Agent = Model + Harness. For a coding agent, the harness covers the instructions and documents the agent reads, the tools it may call, the rules and limits that apply, preferably enforced by code rather than by wording, and the checks on its work.
What is harness engineering?
Harness engineering is the design of the environment an AI agent works in: the context it receives, the tools it can call, the rules it must follow and the limits on its autonomy. For coding agents, it includes instruction files, linters, tests, permissions and the checks that decide what reaches production.
What are examples of an AI harness?
A short AGENTS.md file that points to versioned documentation, a custom linter whose error message tells the agent how to fix the violation, a confirmation gate before a schema migration, a least-privilege list of tools and MCP servers, tests and type checks that run after each change, and a feature list with a progress file for work that spans sessions. The section Examples of an AI harness in coding agents, on this page, names the source of each one.
What is the difference between the harness and the model?
The model generates text and code. The harness is what the team builds around it: what the agent sees, which tools it may use, which rules and limits apply, and which mechanical checks close the loop. In the SWE-agent study (NeurIPS 2024), the same model, GPT-4 Turbo, resolved 11% of SWE-bench Lite issues through a plain shell and 18% through an interface designed for it.
Is Claude Code a harness?
It depends on who draws the line. Ralph Bean of Red Hat calls Claude Code an agent runtime, the engine that runs the agent loop, and thinks it is incorrect, or at least imprecise, to call that runtime a harness; for him, the harness is the layer a team engineers around it. Böckeler splits the harness in two: part of the harness is built into the coding agent, and its users build an outer harness for their own system. The source-code study by Barbaste et al. counts Claude Code among the coding harnesses it analyzes.
Is harness engineering the same as Harness.io?
No. Harness.io is a company that sells an AI software delivery platform, with CI/CD and purpose-built AI agents for delivery, security and cost. Harness engineering is a practice for governing AI agents, coding agents in particular. A company can use both, because they answer different questions.
Does a harness replace human code review?
No. A harness catches what can be checked mechanically and sends the rest to a person. Böckeler lists misdiagnosed issues, unnecessary features and misunderstood instructions among the problems that neither deterministic checks nor AI reviewers catch reliably. On the agentic development platform, the team reads what goes to production.
Is this a dark factory?
No. A dark factory is level 5: code nobody reads before it goes into production. The platform runs at level 4, where coding agents execute, humans supervise and the team sets the limit.
Where does the platform run, and does our code leave the network?
The platform is deployed in the client's environment, on a cloud account the client controls or in its own data center, and the repository is not sent to an Ithaca SaaS. Model calls follow the client's policy for provider, region and contract; during inference, code excerpts may go to the chosen model provider under that policy.
Book a demo
The next step is a demo of the agentic development platform on one part of the system that can be checked end to end. If you are looking for a harness for your own team's coding agents, say so in the form, and the conversation starts from your codebase.
Sources
- John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan and Ofir Press, "SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering", NeurIPS 2024. The Princeton comparison of the same model through a plain shell and through an interface built for it, and the details of that interface.
- Ryan Lopopolo, "Harness engineering: leveraging Codex in an agent-first world", OpenAI, 11 February 2026. The field report: the empty repository, the short AGENTS.md as a map, rules moved into custom linters, and the conditions the authors attach to their results.
- Birgitta Böckeler, "Harness Engineering - first thoughts", martinfowler.com, 17 February 2026. Her first memo on the term.
- Birgitta Böckeler, "Harness engineering for coding agent users", martinfowler.com, 2 April 2026. Agent = Model + Harness, the outer harness, guides and sensors, legacy code, accountability, and what checks do not catch.
- Paul Barbaste, Tristan Darrigol, Germain Vu and Tom Wiltberger (Wavestone AI Lab), "Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents", arXiv preprint, 15 July 2026. The source-code study of eleven coding agents and its seven subsystems.
- Jiahang Lin, Shichun Liu, Chengjun Pan and others (Fudan University and collaborators), "Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses", arXiv preprint, 28 April 2026. The harness rewritten by an agent over ten rounds, the ablations, and the observability-driven method.
- Ralph Bean, "What even is the harness in AI?", Red Hat blog, 21 May 2026. The harness as an additive layer and the sandbox as a subtractive one, and Claude Code as an agent runtime.
- Jaya Simha Reddy Nandyala and Prabina Pani, "Engineering the harness: A practical pattern for reliable coding agents", Thoughtworks blog, 29 September 2026. The change that breaks two other services, the confirmation gate across repositories, and least-privilege tools.
- Justin Young, "Effective harnesses for long-running agents", Anthropic, 26 November 2025. The feature list, the progress file and the initial commit for work across many context windows.
- Harness homepage and its developer documentation for continuous integration and continuous delivery, consulted 20 October 2026. How the company describes its product.
- Dan Shapiro, "The Five Levels: from Spicy Autocomplete to the Dark Factory", 23 January 2026. The autonomy scale this page uses for its levels.
Related reading
- Agentic development platform: what it is and what it isn't
- Coding agent observability: what to record in each session
- Levels of AI coding autonomy, from level 0 to level 5
- Dark factory: what it is, and why level 4 is not one
- AI governance for coding agents in software development
- Agentic engineering vs. vibe coding: when each one fits
- Glossary: coding agents, harness engineering and level 4
