Synthetic data cannot replicate how your company decides

A synthetic dataset cannot contain a fact about your company that nobody put into it. Almost everything worth knowing about a synthetic data enterprise AI plan follows from that one constraint.

The hopeful version of the idea goes like this. The frontier labs are running short of real data and training on generated data instead, so a company might do the same and skip the slow work of connecting its own records. I understand the appeal. Those records are scattered, sensitive and awkward to reach, and a generator never asks anyone for access.

The labs are solving a different problem. They want a model that reasons well about anything. You want one that knows how this company decides, and generated data has very little to offer on that second job.

This comparison is for the CIO, chief data officer or head of AI weighing the two. I would hold any source of training or evaluation data to five questions. Does it contain facts specific to this company? Does it connect a decision to what happened afterward? Can a reviewer trace a claim to a real record? What does it expose? What does it cost to build and keep?

What the labs are doing with generated data

The scarcity story is partly true. At NeurIPS in December 2024, Ilya Sutskever said the field had reached peak data, compared training data to fossil fuel because the supply is finite, and called synthetic data a big challenge. That July, an audit of web domains found that consent restrictions had reached more than a quarter of the most actively maintained sources in a widely used training corpus within a single year. Against that, Thinking Machines pretrained Inkling, its July release, on 45 trillion tokens of text, images, audio and video. That does not look like a dry well, although the announcement never says where the tokens came from and never mentions scarcity.

What the labs' own reports do show is how much training now runs on generated material. Microsoft's Phi-4 report says synthetic data makes up the bulk of that model's training data, calls the practice increasingly common, and reports that the model surpassed its teacher, GPT-4, on STEM questions. Microsoft also rejects the idea that this is a cheap substitute. A human-written math solution may open with the final answer, while a generated one can walk through each step in order. Microsoft's pipeline still begins with curated real material: web content, licensed books and code.

Thinking Machines opened Inkling's post-training with supervised fine-tuning on data written by other open-weights models. Most of the compute then went to reinforcement learning in synthetic and human-created environments, over thirty million rollouts scored by automated graders. The company also suggests its small model for generating synthetic data for other models. Models now write homework for other models.

Look at what that post-training covered: math, agentic code and tool use, audio, image, chat and safety. Those are general skills. The same announcement says many real-world problems are not solved well by even the best generalist models, and that the gap closes through fine-tuning on "an organization's specialized knowledge." Thinking Machines sells fine-tuning, so weigh the sentence accordingly, and who ends up owning the result is its own question. Even so, the lab that published this pipeline looks outside it for the last stretch.

Why generated data teaches reasoning and stops there

Generated data works where a rule can grade the answer. A proof holds or it fails. Code passes its tests. A factual claim checks out, which is what Inkling's claims grader looks for. You can produce those problems endlessly because the right answer does not depend on who is asking.

Now take a question from inside a company. Should this office have granted that exception for that customer? No rubric exists. The answer depends on what the customer was told, on what the exception cost over the following year, and on whether the customer stayed. The only real grader is what happened next, which may sit in another system and arrive months later. To write the rubric, you would already need the knowledge you were hoping to generate.

The second limit is plain information. A generator's output is a function of its inputs and some randomness. Information theory calls the consequence the data processing inequality: processing a signal cannot add information about its source. A generator that has never seen your records knows about companies in general and will write fluent, plausible cases about nobody. A generator fitted to your records can produce more rows, and those rows cannot tell you anything the records did not already hold.

At best it keeps what was there. The third limit is that it may not. A 2024 paper in Nature found that indiscriminate training on model-generated content across generations makes the tails of the original distribution disappear first. Its authors conclude that tasks where the tails matter need real human-produced data. That result concerns recursive, unfiltered training, which is a long way from one careful pass.

A 2022 study of tabular data sits closer to what an enterprise would do with its own records. Synthetic data from the two generators it tested on this point did not retain an outlier's influence on predictions for similar records, and the authors argue you cannot predict which features a generator will keep. A later paper disputes that study's privacy test and reaches a kinder view of synthetic data against a traditional anonymization method. Neither one tested your records.

This matters because institutional knowledge is mostly tail. A renewal approved every year against the written rule because of a promise made long ago. The one broker whose submissions always get a second read. The region where the standard playbook keeps failing. A generator built to reproduce the typical case has every reason to treat those as noise.

The comparison

Two different things get called synthetic data here. Lab-generated training data is what a model provider creates to train its base model, and you receive it only as a model. Synthetic copies are rows from a generator fitted to your real records, meant to keep the statistical patterns without being anyone's actual record. A context graph is something else again: the company's knowledge graph connecting records, relationships, human judgment, decisions and outcomes.

Fine-tuning is missing from that table because it is a method, and it needs a source. Tune on generated examples and you reinforce the generator's assumptions. Tune on real decisions with their outcomes and expert review, and you are at least aiming at your own company.

Where synthetic data is the right tool

I would use it without apology in four places.

Your base model is the first, and you do not get a vote. If generated training gives you a model that reasons better for less money, take it and evaluate it on your own work.

Software testing is the second. Sandbox tests with synthetic or copied cases can exercise tool behavior, permission failures and retries, and nothing in those tests needs the record to be true.

Then there is rehearsing bad days: hand-written hostile or rare scenarios that show whether an approval gate fires or a model declines to answer. You are checking a control, so the scenario can be invented.

The last is sharing formats before access is agreed. An outside team can build against realistic rows while real access works through review. Check what the rows reveal first, because a generator fitted to real records can carry them along.

The line runs between rehearsal and learning. Generated cases can show whether a system behaves. They cannot show what is true about the company or whether a past decision was right. Model evaluation falls on the learning side. A test set written by a model grades a candidate against that model's idea of your business, so hold back real decisions for the test.

What one company's records showed

My best evidence comes from hiring at a Fortune 1000 insurance carrier. The Decision Traces study analyzed four years of production data covering 10,765 agents hired. It parsed 8,181 skills and tested the 3,597 with enough data. After Bonferroni correction, none predicted the production milestone, and 30 were anti-predictive. In the exact n=50 subset of top performers with the required text, an industry-experience filter would have excluded 80%, and the cumulative funnel 98%. Those last two figures are a retrospective counterfactual. They do not describe people the historical process rejected, and nothing here is a causal estimate.

Now ask a generalist model to write a few thousand plausible hiring decisions for an insurance carrier, and look at what it rewards. I would expect industry experience to do well, because public writing on hiring tends to treat it as common sense. That is a prediction, and anyone can test it in an afternoon. A model tuned on those cases would learn the filter and apply it with confidence.

The finding against that filter exists only because someone linked one company's applicant records to what the people it hired later produced. Another company's top performers may look nothing like these, so the result stays with this carrier. A fact this specific has no route into a generated dataset until someone has measured it. Once someone has, you no longer need the generator to know it.

What to ask before you buy

Put these to any vendor whose pitch leans on generated data.

1. Which numbers in this pitch were measured on real records, and which on generated ones?
2. If a generator is fitted to our records, where does it run, who holds the fitted model, and what happens to it when we leave? A fitted generator is a lossy copy of your data, so treat it as sensitive until a test says otherwise.
3. Is every generated record labeled, and what stops one from entering the record of what happened? A synthetic case that leaks into decision history documents something that never occurred.
4. Which held-out real decisions will a replacement model face, and who reads the disagreements? Score task quality, latency, risk, and the total AI and human cost.
5. What may you do with our prompts, evaluations and corrections? The Nature authors expected data about human interaction with models to grow more valuable as generated content spreads. Your reviewers' corrections are that kind of data, which puts intelligence exhaust on the contract review list: processing, retention, training use and ownership.

Where this argument breaks

Your records are incomplete. No system can recover a reason nobody wrote down, outcomes arrive late, and some never arrive. A context graph holds what was captured, which on the first day may be thin, and it is the most expensive column in that table.

Your history can also be wrong. A model that learns faithfully from old decisions can learn old habits, and the hiring filter is a warning about that. This is why the outcome has to travel with the decision, and why failed and inconclusive results stay in the history. Even then, a measured result does not prove on its own that the decision caused it.

A new line of business has no history to learn from. Expert-written scenarios may be all you have, and that is fine as long as they are labeled as assumptions and real decisions get recorded from the start.

Generators will improve, too. One fitted to real records may keep more of the tail than the ones tested in 2022, and rehearsal will be better for it. It still cannot report an outcome nobody measured.

Keep the part nobody can generate

Nodes Engine creates the context graph from authorized structured and unstructured sources the company chooses. Decision Traces connect the evidence, options, rationale, human judgment, authorized action and later outcome. Models stay swappable, so a stronger base model, however it was trained, can use what the business already knows once it has passed evaluation on that business's work. The processing boundary depends on the deployment (Nodes Cloud, a single-tenant customer VPC or customer-managed on premises) and the model connections the customer approved. Confidential customer memory is not pooled across tenants by default. The architecture overview shows how those pieces fit.

Expect the practice problems to keep getting better, and be glad of it. The next model you rent will probably have worked through more of them than anything before it. A generated case is still about a company that does not quite exist. Yours does, and its record either gets longer or gets lost each time someone decides something.

Sources

Thinking Machines Lab: Inkling, our open-weights model

The Decoder: OpenAI co-founder says AI is reaching peak data

Longpre et al.: Consent in Crisis

Microsoft Research: Phi-4 Technical Report

Shumailov et al.: AI models collapse when trained on recursively generated data (Nature)

Stadler, Oprisanu and Troncoso: Synthetic Data, Anonymisation Groundhog Day (USENIX Security)

Sarmin et al.: Synthetic Data, Revisiting the Privacy-Utility Trade-off

Saad Bin Shafiq: Decision Traces

Saad Bin Shafiq is the founder of Nodes, serving data-sensitive enterprises. Methodology: Decision Traces.