HEELO Packages Agent Capabilities into Verifiable Primitives for Real-World Tasks

Summary:The paper introduces Agent-Native Primitives and HEELO, a harness-engineering framework that turns resources such as tools, terminals, development environments, and navigation interfaces into reusable execution units equipped with resident LLMs. HEELO composes these units into task-specific graphs, verifies outcomes against environmental evidence, and revises or admits primitives only when failures expose a concrete gap or a candidate passes validation. Across more than 20 benchmarks, the repor

---

Core Idea: Make Correct Resource Usage a Reusable Asset

Conventional agent harnesses expose terminals, code runners, browsers, or business APIs to a model and rely on prompts to explain when to call them, how to compose arguments, and how to recover from errors. As a result, the same operational competence is rebuilt inside every harness. Models often fail after reaching a resource rather than successfully applying it.

HEELO introduces Agent-Native Primitives that move this competence into the resource itself. Each primitive contains three layers:

1. A resource adapter for the underlying interface, state, and side effects;
2. A resident LLM that applies the resource in the current task context;
3. An inter-primitive communication mechanism for exchanging intermediate state, dependencies, and failures.

The model therefore interacts with executable capability units rather than bare tools.

How HEELO Builds Task-Specific Execution Graphs

HEELO—Harness Engineering around Evolving Agent-Native Primitives—is a harness-engineering framework that organizes primitives according to each task. Its loop has four stages:

• Identify task requirements and select primitives from families such as development, tools, harness construction, and navigation;
• Compose a task-specific graph that defines inputs, outputs, ordering, and failure propagation;
• Verify against environmental evidence, including terminal state, file diffs, and business-system return values;
• Evolve selectively by revising a primitive implicated in a failure and admitting new candidates only after validation.

The optimization target is no longer a single prompt or conversation trace. It is a reusable, reviewable execution-capability library.

Benchmark Results

The paper evaluates eight primitive families across more than 20 benchmarks. Key results are summarized below:

| Evaluation | Baseline | With HEELO Primitives | Change | |---|---:|---:|---:| | Terminal-Bench 4.0 resolution | 37.3% with Codex | 55.5% on the same backbone | +18.2 points | | Terminal-Bench 2.0 Pass@1 | 60.0% with seed harness | 72.2% | +12.2 points | | ToolBench pass rate | 55.8% | 75.1% | +19.3 points | | GAIA-Text-103 accuracy | 45.6% | 58.3% | +12.7 points |

The Terminal-Bench 4.0 comparison is particularly informative because it keeps the backbone model fixed. The gain is therefore associated with primitive design and harness organization rather than a simple model substitution.

On 46 real-world platform tasks, Claude Opus 5.5 with Claude Code and GPT-6.1 Sol with Codex each completed nine tasks independently. After connecting them to HEELO through its MCP server, they completed 39 and 35 tasks. The paper reports 1.9-times and 2.3-times lower time, respectively, along with 1.55-times lower cost per completed task.

Engineering Trade-offs

Context and token overhead

Every primitive carries a resident LLM. In a multi-primitive graph, local inference, state synchronization, and evidence aggregation increase token consumption. The reported cost reduction relies on substantially higher completion counts; for trivial tasks, HEELO may be more expensive than a bare harness.

Latency accumulates along the critical path

Independent branches can execute concurrently, but dependent primitives must run sequentially. Environment calls, verification, and replanning add latency to the critical path. Better aggregate throughput does not necessarily imply lower per-request tail latency.

A shared primitive library can accumulate fragile combinations

A primitive revised for one task may behave incorrectly in another environment. Evidence-based validation reduces blind evolution, but cross-version regression, permission isolation, and failure atomicity still require production-grade evaluation.

Sharing expands the permission blast radius

Once code execution, file modification, and external APIs are packaged as resident capabilities, errors can propagate more efficiently through a compact call chain. Each primitive should still enforce least privilege, sandboxing, timeouts, spending limits, and auditable logs.

How Much Should We Trust the Evaluation?

The paper covers multiple benchmarks and includes same-backbone comparisons, which strengthens the evidence. Nevertheless, several limitations remain:

• Results from different primitive families cannot be directly aggregated because they correspond to different tasks;
• Terminal-Bench-style scores are sensitive to harness configuration, prompt templates, tool versions, and retry budgets;
• Cost and time comparisons depend on whether the evaluation normalizes model calls and tool usage per successful task;
• A library evolved using task feedback may overfit to benchmark-specific patterns.

The most defensible conclusion is that verified primitives can substantially reduce duplicated execution-harness engineering, not that the reported gains automatically transfer to every production workload.

Deployment Guidance

Teams adopting the HEELO approach should:

1. Start with frequent, well-bounded tasks such as code modification, terminal diagnosis, or web research;
2. Package execution competence into primitives with structured interfaces, evidence hooks, and explicit failure types;
3. Schedule the task graph with parallelism and track latency, tokens, cost, and side effects for every primitive;
4. Validate against environmental evidence, including tests for code and state or return records for business workflows;
5. Version primitives and maintain regression tests to prevent one failure-driven revision from breaking unrelated tasks;
6. Enable automatic primitive creation only after adding human or policy gates for code execution, outbound data, and credential access.

HEELO’s main contribution is a shift from writing longer prompts toward building composable, verifiable, and evolvable execution-capability libraries. The approach is especially relevant to terminal agents, software-engineering toolchains, and workflows spanning multiple external services.

---

Source: Hugging Face Daily Papers (2026-11-05)

— END —

Tags: #HEELO Follow #Agent-Native Primitives Follow #Agentic Workflow Follow #Harness Engineering Follow #MCP Follow