A cheaper model still does not know your company

When a small model gets cheap enough, the expensive part of an enterprise AI program stops being the model and becomes the memory it writes.

On October 7, Anthropic released the newest Claude Haiku. The company calls it the cheapest, fastest, and most capable small model it has released. It is the first Haiku-class model with an adjustable effort setting, and Anthropic points it at summaries, compaction, database queries, classification, and subagent work, with the larger Sonnet and Opus models kept for complex agentic coding.

Most coverage will compare prices and benchmarks. Those comparisons are fair, and your finance team will run them. This post is about the part of the release that no price list captures: what happens to a company's knowledge when the cheapest worker in the system is the one that rewrites it.

What the release says

Read the announcement for the jobs it names. Compaction. Summarization. Queries against a database. Classification. Subagent work inside a larger run. None of these is a decision. Each one is a reduction: a large pile of material goes in and a smaller, tidier one comes out.

That is the right job for a small model, and I am not arguing against it. A system that sends every ticket, every email thread and every call transcript to its largest model is wasting money and time. Routing narrow, high-volume work to a fast model and saving the expensive one for judgment is sound engineering, and Anthropic's own positioning says as much.

The consequence is easy to miss. Every reduction is a decision about what to drop. When a model compacts a long working session into a short note, it chooses which facts survive. When it classifies a thousand records, it chooses which ones look alike. When a subagent returns a summary to the model that called it, the parent never sees what the summary left out.

Cheap makes this happen more often. A step that cost enough to make someone think twice now runs on every record, every night, with no one thinking about it.

The reduction is where company knowledge goes missing

Think about what a company knows. It is rarely a clean fact. It is an exception granted last spring, a manager who overrode a rule for a reason that never reached the system of record, a customer segment that behaves differently in one region, a workaround that everyone in an office understands and no document mentions.

Those things survive in two places. They live in people's heads, and they live in the raw records and the decisions made against them. A summary erases them first, because they are the least typical thing in any pile of material. Compaction is built to keep the common pattern and compress the rest.

So the order of operations matters more than the model. If the summary is the thing the company keeps, then every cheaper, faster compaction pass makes the stored memory a little more average. Nobody notices, because each summary reads well. The loss only shows up months later, when a question depends on the exception that was compressed away.

This is the version of the "your people built it" argument I care about most. Your people built the intelligence by handling the exceptions. A pipeline that quietly smooths those exceptions into a tidy paragraph is returning less than it was given.

Keep the source, the judgment, and the outcome

The fix is to decide what the company keeps before the small model touches anything, and to keep it in a form no summary can replace.

A context graph is the company's knowledge graph connecting records, relationships, human judgment, decisions and outcomes. The point of that structure in this context is simple. The source records stay as they are. The relationships between them stay explicit. A summary can sit on top as a convenience for a reader, and it can be regenerated by any model at any time, because it is derived and the graph underneath is not.

The decision itself gets a separate record. A Decision Trace connects the evidence a person saw, the options compared, the rationale, who approved what, and what happened afterward. That record is written when the decision is made and is attached to the outcome when the outcome is measured. A compaction pass can read it. It cannot rewrite it.

Put those two rules together and a cheaper model becomes safe to use widely, because nothing it produces is the system of record. Its output is disposable by design. If a better small model arrives next quarter, you regenerate the summaries and keep everything else.

That is also the practical answer to who owns your agent's memory. If memory is the summary a vendor's model wrote inside a vendor's session, then the memory is a product feature, and its shape follows the vendor's choices. If memory is your graph and your decision history, and the model only reads and proposes, then you can change the model without changing what the company knows.

Where the small model belongs in a governed system

Here is a concrete way to place a model like this one. The jobs below are illustrative of a pattern. They are not a claim about any particular deployment.

A small, fast model is a good fit for work that is narrow, high-volume, reversible and checkable. Sorting incoming records into the categories a team already uses. Summarizing a long thread for a human who will read the source if the summary matters. Turning a plain-language question into a query against an agreed database. Compacting a working session so the larger model can continue without losing the thread.

It is a poor fit, on its own, for anything that changes a system of record, commits money, touches a person's employment or coverage, or reaches outside the company. Those steps sit behind an approval gate. A named person, working within current permissions, reviews the proposal, sees the sources, and can edit or decline it. Prior approval never expands what the system may do next time.

Notice that this split has nothing to do with which model is smartest. A very capable model proposing a consequential change still goes through the gate. A modest one summarizing a thread does not need to. The control follows the consequence of the step, and the model choice follows the cost and quality of the task. Mixing those two questions up is how companies end up either overpaying for trivial work or under-governing important work.

A model swap is an experiment on your own data

Every vendor now ships a faster, cheaper model every few months. A reasonable buyer reads that as an invitation to switch. The reasonable response is to treat each switch as an experiment on your own data.

For a candidate replacement, the checks are ordinary. Does it produce the same classification on a held-out sample of your own historical records? Does its summary preserve the exceptions your reviewers flagged as important? Is the latency acceptable for the step it sits in? What is the failure rate on the cases that matter most, and what does a failure cost? And the full cost of the work, counting the human time spent correcting it as well as the price per token.

A published benchmark answers none of those questions about your company. It tells you how the model does on someone else's tasks. Anthropic reports large benchmark gains over its previous small model, and I take that at face value. It still tells me nothing about whether the summaries of your claims files keep the detail your adjusters rely on.

This is why the stored history matters twice. It is the material the model reasons over, and it is also the test set. A company with linked decisions and measured outcomes can replay a candidate model against its own past and see where it would have differed. A company whose memory is a pile of summaries has nothing to test against, because the thing it kept is the thing that was already lossy.

Some deployments also need a model to run inside the customer's own boundary. Which models are available there, and what processing boundary applies, depends on the deployment and on the model connections the customer has approved. A cheaper hosted model is attractive. It is not automatically available, or permitted, in every environment, and the answer is established in diligence, not assumed from a launch post.

Where this argument breaks

There are fair objections.

The first is that retrieval over raw records is expensive too, and a good summary is often better than the source for a busy reader. That is true. Summaries are fine as derived views, labeled as such, with a path back to the source. A summary a reviewer can click through is a feature. A summary that replaced the original is a loss.

The second is that models are improving fast enough that today's compaction worry may be tomorrow's non-issue. Maybe. A better small model will drop fewer exceptions. It will not drop none, and you will not know which ones it dropped unless the source is still there. The cost of keeping the source is storage. The cost of not keeping it is permanent.

The third is that a graph and a decision-trace layer are more work than letting the vendor's tooling manage memory. Also true, at the start. The work is deciding which sources count, who may read and change what, and how often to check them. A company that skips those decisions has not avoided them. It has delegated them to whichever default shipped first.

There is one more limit worth stating plainly. A Decision Trace records the evidence, the rationale and the outcome. It does not give access to a model's hidden reasoning, and a favorable result does not by itself prove the recommendation caused it. The record makes the decision inspectable. It does not make it correct.

What I would ask this week

If your team is looking at a cheaper small model, ask five things before you move any workload.

Which steps does it replace, and are they reductions, such as summaries and classifications, or decisions? Where does each output get stored, and is it labeled as derived? Can a reviewer get from the summary back to the source record in one step? Which held-out cases from your own history will you run it against, and who reads the disagreements? And if you switch again next quarter, what survives the switch?

If the honest answer to the last question is "the summaries," the cost savings are real and the memory is thin.

The model will keep getting cheaper. Your company's record of what it knew, what it decided and what happened next only gets more valuable as everything around it gets cheaper to produce. Keep that part, and let the rest be swapped.

For how Nodes Engine holds that record across models, see the architecture overview.

Sources

Anthropic: introducing the newest Claude Haiku

Saad Bin Shafiq is the founder of Nodes, serving data-sensitive enterprises.