What an agent harness actually does
A model on its own is not an agent. This post covers the software around it and the parts that decide whether it works, including context, memory, durable execution, and evals.
Written · Published
A model on its own is not an agent. It takes text in and gives text back. What makes it an agent is the model plus a harness, the software around it that builds the prompt, decides which tool calls are allowed, runs them, records what happened, and decides when to stop.
The model predicts the next step. The harness decides whether the whole system is reliable.
The layers
There is no formal standard for how to divide a harness. Most write ups use similar names, and these are the six I use.
- Orchestration. The control loop. It decides what runs next, when to retry, and when to stop.
- Tool execution. It checks what the model asks for, confirms it is allowed, runs the tool, and returns the result.
- Context and memory. What the model sees each turn, including instructions, retrieved data, history, summaries, and what is remembered across sessions.
- State and persistence. A record of what has actually finished, so a crash does not lose or repeat work.
- Guardrails. Permissions, approvals, and limits on what the model can do.
- Observability and evals. Traces that show what happened, and tests that show whether a change helped.
Most of my time has gone into context and state, because an agent is only as good as the information around the model. Both turn out to be data problems.
Context is the part I trust least to chance
In the tagging work I presented at the Data + AI Summit, the model did not search the world for an answer. The harness narrowed the problem first.
- Extract. Pull candidate names, products, and keywords out of noisy text.
- Retrieve. Run a hybrid search over a reference list and return a handful of candidates.
- Choose. Ask the model to pick the best candidate from that list and apply business rules.
Choosing from a short list is a much easier job than naming a company from nothing. A model asked cold will give a confident answer that is sometimes wrong. A model given five candidates and the right context has far less room to be wrong.
The same lesson applies more strongly to smaller and older models. They forgive less. If the right information is not in the prompt, they do not make up for it. So when accuracy is low, my first move is to improve the data going in. I enrich each record with the known metadata that matters most, before I rewrite the prompt or switch models.
Memory is context with a time dimension
In a side project, I experimented with a memory layer. It reads past transcripts and pulls out facts, actions, user profiles, and projects. Each item is embedded and stored in a vector store. I built my own simple version of that store to understand how the pieces fit together, and I have used Qdrant briefly as a dedicated vector database. On the next turn, the harness retrieves the relevant memories and adds them to the prompt.
Storing text is the easy part. The harder questions are what is worth keeping, how to avoid saving the same fact twice, and which memories belong in the prompt right now. A memory layer that retrieves too much is just a longer prompt.
Durable execution keeps long runs from being lost
An agent run can take minutes or hours, and the process running it can die. Durable execution is how the harness survives that. It has two parts.
Ownership. A worker claims a task and gets a lease, which is a claim that runs out after a set time. While it works, it sends regular heartbeats, a signal that means it is still alive, to renew the lease. If the heartbeats stop, the lease runs out, the task is released, and another worker can claim it.
An expired lease only means the system stopped hearing from the worker. The worker may be slow or cut off, and it can come back and carry on as if nothing happened. Think of a hotel key card. When the hotel issues a new card for a room, the old card stops working. In the same way, when a new worker takes over, the old worker’s claim is cancelled, so anything it tries to save late is turned away. Without that rule, two workers could do the same step twice.
Progress. Ownership only decides who runs the task. The harness also needs a durable record of what has already happened, so the new worker picks up from the last completed step instead of starting over or repeating an action. For an agent, that means saving each tool call and its result before moving on, then reading those results back on recovery instead of asking the model to decide again. Anthropic’s long running coding example does this with a feature list, progress notes, and git history, so a fresh session can verify the state it inherits.
Anyone who has run data pipelines will recognize this. A retry has to know whether the last step finished, especially when that step wrote to another system.
Evals are how you know anything worked
I treat evals as the most important layer, because without them every other change is a guess.
In my experience the best eval sets are built by hand, by people who know the domain. They are slow to make, but the quality bar matters, because a weak eval set rewards the wrong behavior. I would rather spend a week building a small, careful set than an afternoon generating a large one.
An eval set also lets you measure each stage on its own. If the final answer is wrong, you can tell whether extraction missed the right name, retrieval left it out of the candidates, or the model chose badly. That tells you where to spend the next day of work.
Sources
- From Unstructured Data to Structured Insights: How YipitData Scales Data Enrichment with AI Agents, Data + AI Summit session page.
- Anthropic, Effective harnesses for long-running agents.