Most conversations about generative AI in manufacturing start with use cases — generative design, predictive quality, natural language search over engineering records. Fewer start with the question that determines whether any of those use cases are actually feasible: what compute and data infrastructure does this require, and does the plant or the organization actually have it?
This matters more in manufacturing than in most industries adopting generative AI, because manufacturing environments come with constraints — air-gapped networks, edge hardware, regulatory data residency rules — that a typical cloud-native AI deployment doesn’t have to think about. Engineers evaluating a generative AI initiative need a realistic picture of both sides of this equation before committing to an architecture.
Compute: Training vs. Inference Are Different Problems
The first distinction worth making clearly is that training a model and running inference on it have very different compute profiles, and manufacturing teams rarely need to do the former.
Training large models from scratch is almost never the right call. Foundation models (GPT-class, Claude-class) are trained on compute budgets far beyond what any single manufacturer would deploy for an internal tool. Almost all manufacturing use cases are better served by using a hosted foundation model via API, or fine-tuning/adapting a smaller open-weight model, rather than training from scratch.
Fine-tuning is a lighter but still non-trivial compute requirement. If a team fine-tunes an open-weight model (in the 7B–70B parameter range, for example) on proprietary manufacturing data, this typically requires GPU infrastructure — often a small cluster of A100s or H100s, or equivalent cloud instances — for hours to a few days depending on dataset size. This is a real cost and skills requirement, not something to take on casually.
Inference is the more common and more manageable compute question. Running a trained or hosted model to answer a query, classify an image, or generate a design variant is far cheaper than training, and is where most manufacturing AI budgets are actually spent. Inference costs scale with usage volume and model size, and can typically be handled through a hosted API without dedicated infrastructure at all.
Edge vs. cloud inference is a real architectural decision, not a formality. A quality inspection system on a production line that can’t tolerate network latency, or that runs in a facility without reliable connectivity, needs inference running locally — on edge GPUs (NVIDIA Jetson-class hardware, for example) or on-premises servers. A documentation search tool used by engineers at their desks has no such constraint and can comfortably call a cloud API. Matching the deployment pattern to the actual latency and connectivity requirement, rather than defaulting to cloud for everything, avoids a lot of downstream pain.
Data: What “Enough Data” Actually Means
Compute questions get most of the attention, but for Generative AI in Manufacturing specifically, data requirements are usually the tighter constraint.
Volume needs differ sharply by technique. A RAG-based system doesn’t need training data volume at all — it needs a well-organized corpus of documents and records to retrieve from, which can be a few hundred documents or several hundred thousand, depending on scope. Fine-tuning, by contrast, generally benefits from at least low thousands of high-quality labeled examples to meaningfully shift model behavior; fewer than that and few-shot prompting is usually more reliable than fine-tuning.
Quality matters more than volume in manufacturing specifically. A defect classification model trained on a large but inconsistently labeled dataset — where “defect” means something different across three different inspectors — will underperform a much smaller, consistently labeled dataset. Manufacturers evaluating whether they have “enough data” should audit labeling consistency before assuming volume is the bottleneck.
Structured and unstructured data need separate handling. Time-series sensor data from PLCs and historians, structured records from PLM and ERP, and unstructured text from inspection reports and manuals all need different preprocessing paths before they’re usable — there’s no single pipeline that handles all three well without dedicated work on each.
Data residency and access constraints shape what’s usable, not just what’s available. A dataset that technically exists but sits in a system a plant can’t export from due to IP or regulatory restrictions isn’t usable data for a given project, no matter how large it is. This is worth surfacing early, since it often changes the technical approach — for instance, ruling out a cloud-hosted fine-tuning workflow in favor of an on-premises one.
Estimating What a Given Use Case Actually Needs
A rough way engineers can size a project before committing to infrastructure:
| Use case | Compute requirement | Data requirement |
| Documentation / manual search (RAG) | Cloud API calls; no dedicated GPU needed | Document corpus, well-organized; volume flexible |
| Defect classification from images | GPU for training; cloud or edge for inference depending on latency needs | Thousands of consistently labeled images minimum |
| Predictive maintenance from sensor data | GPU for training; often edge inference for real-time response | Historical time-series data with labeled failure events |
| Generative design suggestions | Cloud API or fine-tuned model; inference-heavy, iterative | CAD history, simulation results, engineering constraints as structured data |
| Automated ECO drafting | Cloud API sufficient in most cases | Structured change records, BOM data, PLM metadata |
Most teams overestimate the compute they’ll need and underestimate the data cleanup work — the reverse of the intuition that shows up in initial project planning.
Common Mistakes Worth Avoiding
A few patterns show up repeatedly in manufacturing AI projects that stall or overrun budget:
- Assuming a hosted API can’t meet data residency requirements without checking. Many providers now offer enterprise agreements with specific data handling and residency guarantees that satisfy requirements teams initially assumed would force an on-premises build.
- Under-provisioning for fine-tuning, then being surprised by cost or timeline. Fine-tuning is treated as a small add-on step in some project plans when it’s closer in effort to standing up a small ML engineering function.
- Skipping a data audit before committing to an architecture. Committing to a fine-tuning approach, then discovering the labeled data is too inconsistent to be useful, is a common and avoidable source of stalled projects.
- Treating edge inference as a default rather than a specific requirement. Edge deployment adds real hardware and maintenance overhead; it should be chosen because latency or connectivity genuinely requires it, not as a precaution.
What This Means for Planning
The compute and data requirements for a generative AI initiative in manufacturing are almost always more modest on the compute side and more demanding on the data side than initial assumptions suggest. Most manufacturing use cases don’t require training a model from scratch or standing up a large GPU cluster — they require a hosted API or a modestly sized fine-tuning effort. What they do require, consistently, is data that’s been audited for consistency and structured well enough to be retrieved or trained on reliably.
Engineers scoping these projects will get more accurate estimates by starting with an honest data audit than by starting with a compute budget — the data work is usually the longer pole in the tent, and it’s the one that’s easiest to underestimate at the outset.
