Synthetic data is training data a generator produces instead of a recording of something that happened. Every generator was fit to something, so the question behind what is synthetic data is how many generation steps sit between your training example and the last time a person actually did the thing. That count predicts where the data holds and where it quietly fails.
The seven rungs, by step count
- Recorded human performance. Step 0. The reference every other rung gets measured against.
- Augmented recordings. Step 1. A real event with its surface changed.
- Simulated environments. Step 1, but from a written model of the world rather than from a recording.
- Model generated data from a model trained on recordings. Step 1. This rung covers tabular generation for privacy as well as text and images.
- Distilled and self generated data. Step 2 or further.
- Scraped data of unrecorded origin. Step unknown.
- Verified synthetic data. Step reset, because a fresh real observation enters the chain.
Why the step count exists
Ask a vendor where a dataset came from and you get a process description. Ask how many generation steps sit between the file and a person doing the task, and the call usually goes quiet. That silence is not evasion. Most teams have never counted, because nothing in the buying process asks them to.
The count does not grade quality. It tells you what a dataset can possibly contain. A generator recombines what it was fit to and nothing else, so at every step away from a recording the rare cases thin out first while the common ones look sharper than before. Your eval numbers rise. Your failures move somewhere you stopped measuring. That is the part of what is synthetic data the definitional guides leave out, and it is the part your deployment will find for you.
One more thing worth saying out loud. You will see a figure quoted everywhere that 75 percent of businesses would be generating synthetic data by the end of 2026. That came from a 2022 forecast, not a measurement, and it says nothing at all about the step count of what those businesses generated. Adoption is not provenance.
Run the count on your own data in three moves
- Take one narrow slice, 200 to 300 items, from the part of your dataset you trust least.
- For every source feeding that slice, write the number of generation steps back to a recorded human action. Write unknown where you cannot trace it.
- Add up the items sitting at step 2 or at unknown. That number is your exposure, and it is the number to take into your next vendor call.
It takes an afternoon. Most teams are surprised by move two, not move three.
1. Recorded human performance (step 0)
Step 0 is not synthetic at all. It is a person performing the task in a real place, captured and verified, and it is the only rung that can add information the others do not already hold.
Everything above this line recombines what step 0 gave it. That matters more than it sounds. In June 2026 a research team trained a manipulation policy on roughly 800 generated demonstrations per task with no real demonstrations at all, then deployed it with no further tuning on a real robot. Average success came in at 35 percent, which is a genuine result and a genuine ceiling. Compare that with co training work where a policy trained on a tiny real set alone reached 10 percent on one task, while the same tiny real set mixed with simulation data reached 80 percent. Neither half wins alone. The real half is the half nobody can generate, and it is also where data yield rate decides how much of what you captured survives review.
Why it matters: your step 0 budget sets the ceiling on everything you generate afterwards, so cutting it to fund more generation lowers the ceiling and hides the fact.
2. Augmented recordings (step 1)
Augmentation changes the surface of a file you already own, which keeps you close to a real event without creating a new one.
You alter lighting, speed, background, noise, phrasing. The underlying event stays exactly what it was. That makes augmentation the cheapest honest step on the ladder and the easiest one to overread. If you hold 500 recordings and generate 50,000 variants, you still hold 500 events. Coverage of rare situations does not move. Coverage of environments you never recorded in does not move either. Augmentation earns its place when your model overfits to surface conditions that will change in deployment. It does nothing for a model that has never seen the situation at all, which is the failure most teams are actually trying to fix.
Why it matters: you are buying resistance to variation, not new coverage, so price it as insurance rather than as data.
3. Simulated environments (step 1 from a written model)
A simulator generates data from an explicit model of the world, so what comes back is a clean version of assumptions somebody wrote down in code.
The physics is explicit and reproducible, which is exactly why it misses. Contact, deformation, clutter, and the failure modes nobody thought to write down all sit outside the model. Domain randomization widens the range of what the model varies. It does not add anything the model was never told about. That is the simulation to reality gap, and it is a data problem before it is an algorithm problem. Simulation stays the right call for volume, for dangerous scenarios, and for events too rare to sit around waiting for. It is the wrong call when the thing you are missing is knowledge of what normal looks like. This guide covers where learned simulators break, and our write up on simulation to reality transfer shows how the gap lands on a training budget.
Why it matters: a simulator can only surprise you inside the rules it was handed, so treat every generated scenario as a restatement of your own assumptions.
4. Model generated data from a model trained on recordings (step 1)
Here a model that learned from real material writes new examples, which puts you one step from human work and inside the boundary of what that model already absorbed.
This rung is broad. It covers instruction data, reasoning traces, image generation, and the tabular generation that finance and health teams use to move analysis forward without exposing real records. The privacy case is real and it works. The catch is the same catch everywhere on the ladder. A condition that was rare in the source records stays rare in the generated rows, and a condition absent from the source stays absent, no matter how many rows you produce. Volume does not create coverage.
Verification is what changes the outcome. Setlur and colleagues found that reinforcement learning applied across generated reasoning traces, including the wrong ones, improved sample efficiency on math reasoning by roughly eight times against simply generating more correct traces. The lesson is not that generation scales. It is that verification scales what generation produces. Any answer to what is synthetic data that stops at the generator has skipped the expensive half.
Why it matters: budget for the checker, not just the generator, because the checker is what turns volume into coverage.
5. Distilled and self generated data (step 2 and beyond)
At step 2 the generator itself learned partly from generated material, so errors from the previous step arrive already treated as fact.
Shumailov and colleagues published the clearest evidence in Nature in 2024. Training generative models on recursively generated output causes what they named model collapse, and the tails of the original distribution disappear first. Rare cases go, then variety, and the defects are irreversible. The trap is that early collapse looks like progress. Average benchmark scores hold or improve while unusual inputs, minority cases, and edge conditions quietly stop being represented. Those are the inputs your deployment meets on day one. Distillation and self instruct loops are real techniques with real uses. Running them with no fresh human material entering is not one of them.
Why it matters: the first thing you lose is the rare case you were paying to cover, and your dashboard will not show it going.
6. Scraped data of unrecorded origin (step unknown)
Some data carries no step count you can verify, because nobody recorded where it came from.
Epoch AI puts the effectively usable stock of public text somewhere between 2026 and 2032, and a growing share of what remains was written by a model. A crawl from 2021 and a crawl from 2026 are different substances wearing the same name. You cannot audit what you cannot trace, and licensing exposure travels through the same gap. For a buyer weighing contamination risk, an unknown step count reads as worse than a known step 2, because a known 2 can at least be corrected. Read the full breakdown of what a defensible reference set has to prove before it goes near an eval.
Why it matters: unknown origin is a compliance question and a training question at the same time, and it is the one your legal team will ask about first.
7. Verified synthetic data (step reset)
Generated material that a qualified person checks against a real outcome resets the count, because a new real observation enters the chain at that point.
The research here is more interesting than the headlines suggest, and worth knowing before somebody quotes collapse at you in a review. Gerstgrasser and colleagues tested the two workflows directly in 2024. Replace the real data with each generation output and test error grows without bound, which is the collapse result. Accumulate the generated data alongside the original real data and test error stays inside a finite upper bound no matter how many iterations you run. Both findings are correct. They describe different workflows. One keeps a real anchor and the other throws it away, and that single choice is what separates a compounding asset from a compounding liability.
Human review is how you hold the anchor when no automatic checker exists for your task, which covers most physical work and most subjective judgement. It is also the only version of generated data you can defend in an audit, because somebody signed the check.
Why it matters: a review layer is the cheapest insurance available on a training run you have already funded.
The step count table
One row per rung. Read the second column first.
| Data type | Steps from a real event | What it preserves | What it cannot add | Best use |
|---|---|---|---|---|
| Recorded human performance | 0 | Everything that happened | Nothing, it is the source | New coverage and eval sets |
| Augmented recordings | 1 | The original event | A situation you never recorded | Resistance to surface variation |
| Simulated environments | 1 from a written model | Explicit rules and hazards | Anything outside those rules | Volume and dangerous scenarios |
| Model generated from recordings | 1 | Common patterns of the source | Rare cases the source underfits | Privacy work and instruction data |
| Distilled and self generated | 2 or more | The dominant mode only | Variety already lost upstream | Compression with a real anchor kept |
| Scraped, origin unrecorded | Unknown | Unknown | Cannot be audited at all | Nothing you have to defend |
| Verified synthetic | Reset to 0 at review | Generated volume plus a real check | Only what review can reach | Scaling once the anchor exists |
Where verified human data fits
Humyn Labs works at step 0 and at the reset. The full pipeline runs from sourcing through a verified network across more than 20 countries, then validation, multilayer quality control, annotation, and human in the loop review on generated output. That gives you two ways in. Commission the recordings your generator has no way to produce, or hand over generated material and get it reconciled against what people actually do. Humyn Labs physical AI data covers the first, and how the pipeline runs end to end covers the second. Every workable answer to what is synthetic data ends in the same place, which is a real observation that someone qualified is willing to stand behind. Send us the slice you cannot generate and we will scope the capture and the review around it.
Frequently asked questions
What is synthetic data in simple terms?
Synthetic data is information a program produces rather than records. A generator learns patterns from real material, or from a written model of the world, then outputs fresh examples that resemble it. The examples look real. They contain nothing the generator was not already holding.
Is synthetic data as good as real data?
For some jobs, yes. Generated data performs well when you already know what the real distribution looks like and only need more of it. It fails when the thing you are missing is the distribution itself, because a generator cannot produce a situation nobody ever recorded.
Does synthetic data cause model collapse?
Only in one specific workflow. Collapse happens when each generation replaces the real data instead of adding to it. Research published in 2024 showed that keeping the original real data in the mix and accumulating generated data alongside it holds test error inside a bounded range.
How much synthetic data is too much?
There is no fixed ratio, and any vendor quoting one is guessing. The limit is structural rather than numerical. Once no real observation is entering your pipeline, you are past the line no matter what the percentage says. Watch the anchor, not the ratio.
How do you check synthetic data quality?
Compare a narrow slice against real recordings of the same task and count the disagreements by hand. Aggregate similarity scores hide the gap. A qualified reviewer working through 200 to 300 items finds the missing cases faster than any distribution metric will.
Can synthetic data replace human annotation?
No, and the reason is structural. A generator produces labels with the same confidence for cases it understands and cases it does not. Human review supplies the outside signal that separates the two, which is exactly what resets the step count on generated material.
The close
Every generated dataset is a copy of something, and the copy is only ever as useful as the original it traces back to. Count the steps before you commit a training run, then put the money where the count resets. Talk to Humyn Labs about the slice you cannot generate.
