Why foundation models and general-purpose AI don’t reconstruct CAD from drawings

Foundation models have changed what general-purpose AI can do with 3D shapes and engineering drawings. They have also revealed four limits that separate impressive demos from production-grade CAD output.

Trellis, SAM3D, and Hunyuan3D produce convincing meshes from a single photograph in seconds. GPT-4V, Gemini, and Claude can describe an engineering drawing in fluent technical language. For any engineering team evaluating these tools against a dedicated 2D-to-3D CAD pipeline, the question is what those capabilities can and cannot deliver.

This article walks through the four limits in detail. Some will close with more training data and better architectures. Others are structural: they are not artifacts of where the technology happens to sit today, but consequences of what foundation models are designed to do. Understanding which is which matters for anyone evaluating AI tools for engineering workflows.

Diagram showing the three levels of 3D representation: mesh, B-rep, and CAD feature sequence. Foundation models produce mesh output at the bottom of the ladder; manufacturing-grade CAD requires B-rep and above.

What “foundation models” actually means in this context

The term covers two distinct families that are sometimes conflated.

The first family is 3D generative models trained to produce 3D shapes from images or text. Trellis, SAM3D, Hunyuan3D, Meshy, Tripo, and similar tools sit here. They are trained on large libraries of 3D assets, typically game-ready meshes, sculpted models, or photogrammetry scans, and they learn to map from a 2D image (usually a photograph or rendering of an object) to a 3D representation. Their output is geometric: a mesh, a neural field, or a gaussian splat. None of these formats encodes feature-based design information; they describe the surface of an object, not its construction.

The second family is vision-language models and multimodal LLMs. GPT-4V, Gemini, and Claude sit here. These models are trained on broad mixtures of text, code, and images. When shown an engineering drawing, they can read dimension callouts, identify projection views, and discuss tolerances in natural language. They do not produce 3D output directly. Their contribution to a hypothetical CAD pipeline would be interpretation, not geometry generation: parsing the drawing into a structured description that some downstream system would convert into geometry.

Both families are foundation models in the technical sense: trained at scale on general-purpose data, then adapted to specific tasks. Both are sometimes proposed as alternatives to dedicated 2D-to-3D pipelines. The four limits below apply across both families, though each limit manifests differently depending on which family is being considered.

Limit 1 — Training data scale and realism

The training data argument is the most concrete of the four and the easiest to verify. Foundation models that work well on natural images do so because they were trained on billions of paired examples: image with caption, image with depth, image with segmentation mask. The scale is enormous and the diversity reflects the diversity of the open web, with all the messy variability that implies.

The equivalent dataset for 2D-to-3D CAD reconstruction does not exist at comparable scale. The largest open benchmarks contain under 200,000 paired examples of drawings and CAD models, and these are almost entirely synthetic: generated by rendering clean CAD models into idealized drawings, then asking a model to invert the rendering. Synthetic data is useful for early experimentation, but it omits the things that make real drawings hard. Scanner noise, photocopy artifacts, hand annotations, non-standard conventions, mixed projection styles, ambiguous dimensions, missing views, and the long tail of edge cases that any practitioner who has worked with a legacy archive recognizes immediately. The training distribution looks nothing like the operational distribution.

Academic benchmarks tend to report strong numbers on these synthetic datasets, which can give the impression that the problem is close to solved. It is not. A model that handles 95% of synthetic drawings can still fail on the majority of drawings that arrive in a real engineering inbox, because the distribution shift between rendered-CAD and real-archive is large. This is a well-understood phenomenon in machine learning. Performance on a held-out test set does not transfer to performance on a different distribution, especially when the test set was generated by the same process as the training set. The synthetic-to-real gap is particularly acute for engineering drawings because the artifacts of real drawings (the things synthetic data omits) are precisely the things that drive reconstruction failures.

The 3D generative family compounds the problem from a different angle. Their training data is rich in photographs of objects paired with 3D assets. A photograph of a bracket contains shading, texture, environmental context, and surface detail that the model leans on heavily. An engineering drawing contains none of that. It contains lines, dimensions, tolerances, hatching patterns, and view labels: a different visual language entirely, one these models did not learn. Asking a model trained on natural images to interpret an orthographic projection is closer to asking it to read a foreign script than to asking it to handle a new viewpoint of a familiar object.

There is also a quieter problem: the cost of producing the right kind of training data. Each real drawing in a training corpus needs a paired ground-truth CAD model, and producing those at scale requires the very engineering work that the AI is meant to automate. This is a difficult bootstrapping problem. The data does not exist in the wild in usable form; it has to be manufactured, slowly, by the same kind of engineering effort that makes manual reconstruction expensive in the first place.

Correctable or structural

Correctable. Larger and more realistic training corpora can be built, though the bottleneck is human effort rather than algorithmic capability. The structural component is narrower: the long tail of edge cases in legacy archives may always exceed what any finite training corpus can cover, which means the underlying need for human verification on hard cases is durable regardless of how much data accumulates.

Limit 2 — Output representation: from mesh to B-rep to CAD sequence

This is the limit most often glossed over in demos, and the one most consequential for downstream manufacturing. It is also the limit where naming the levels of representation precisely matters most, because the gap between what foundation models produce and what manufacturing pipelines need is in fact a stack of two distinct gaps, each with its own research literature.

A 3D generative model produces a mesh, a neural field, or a gaussian splat. These are visual representations of 3D shape: they render well, look right from any angle, and serve well for visualization, prototyping, AR, VR, gaming, and digital twin display. They are not sufficient for manufacturing, because manufacturing depends on a different family of representations entirely.

Production CAD operates at two levels that the literature usually distinguishes carefully but that consumer-facing discussions often collapse. The first level is the boundary representation (B-rep), which is what a STEP file encodes. The second level is the CAD feature sequence, sometimes called the CAD construction history or feature tree, which lives in the native CAD application file (SolidWorks .sldprt, CATIA .CATPart, NX .prt, etc.) and is lost when a part is exported to a neutral format like STEP. Foundation models do not produce either level. Understanding why requires looking at each gap separately.

Same engineering drawing reconstructed into three different 3D outputs: a mesh for visualization, a B-rep STEP file for manufacturing, and a CAD feature sequence for re-engineering.

Mesh vs B-rep: sampling vs exact analytical surfaces

A B-rep describes a part through its exact mathematical surfaces: cylinders defined by axis and radius, planes by point and normal, free-form surfaces by NURBS control points, edges by analytic curves, all stitched together by a topology of faces, edges, and vertices that defines the part’s solid volume precisely. The geometry is exact, not sampled. A hole is a face that is mathematically a cylinder, with the radius known to whatever precision the file was authored at, not a ring of polygons whose vertices approximate the circumference. A planar surface is a plane equation, not a triangulation. A fillet is a swept NURBS surface, not a strip of facets.

This exactness is what makes the file usable across the downstream tool chain. When a CAM system reads a STEP file for tool path generation, it identifies cylindrical faces, planar faces, and swept surfaces from the B-rep topology, then generates tool paths that machine those exact surfaces to their exact dimensions. When a metrology routine inspects a manufactured part, it compares the measurements against the analytical surface equations, not against an approximate vertex cloud.

A mesh, by contrast, samples the same geometry. A hole becomes a polygon ring of finite resolution; the radius is implicit in the average distance from each vertex to the axis, and depends on how finely the mesh was tessellated. A plane becomes a triangulation; its normal is approximated face by face. A swept surface becomes a strip of facets; its curvature is no longer continuous. Every measurement on the mesh is a measurement on the approximation, with an error that depends on the sampling density. For visualization this error is invisible; for manufacturing it is the difference between a part that fits and one that does not. The same applies to gaussian splats and neural fields, which encode appearance and shape statistically rather than analytically. For the deeper reading of why STEP file structure matters for manufacturing, see STEP file format best practices.

B-rep vs CAD sequence: exact geometry vs construction history

The second gap sits above the first. Even when a clean B-rep is available, it does not carry the CAD feature sequence: the ordered history of design operations (sketch this profile, extrude by this depth, fillet these edges by this radius, mirror across this plane, pattern around this axis, and so on) that the original engineer used to build the part. The feature sequence is what makes the model parametrically editable in the native CAD application: changing the depth of an extrusion propagates through everything that depends on it, including features added later in the history. A B-rep captures the final geometry; a CAD sequence captures how the engineer thought about it.

This distinction matters for two reasons. First, STEP files are sufficient for manufacturing precisely because manufacturing does not need the construction history: it needs the exact surfaces and their topology, which the B-rep provides. Second, STEP files are insufficient for downstream design work that involves re-editing the part, because reconstructing a feature sequence from a B-rep alone is a hard inverse problem (often called CAD reverse engineering or feature recognition). For tasks like re-engineering a legacy part for a new application, native-format files with the original feature sequence are far more useful than a STEP, even though the STEP captures the same final geometry.

Where foundation models sit on this ladder

Current 3D generative models (Trellis, SAM3D, Hunyuan3D, and their kin) produce mesh-level output. They are not at the B-rep level, let alone at the CAD sequence level. The research literature has explored several pathways toward closing the gap:

The mesh-to-B-rep pathway treats the problem as a surface-fitting task: take a generated mesh and fit analytical surfaces against it, then stitch them into a valid topology. This is feasible for restricted part families (primitive-rich, axis-aligned) but degrades quickly on the variety of real industrial parts, on curved surfaces, and on the imprecisions of generated meshes as input. The cascade of errors compounds: a mesh that looks acceptable for visualization is typically too noisy for reliable surface fitting, and any error in the surface fitting propagates downstream.

The mesh-to-CAD-sequence pathway tries to recover the construction history directly from the mesh, treating the problem as inverse CAD modeling. Research efforts here (the line of work around CAD generative models and CAD recovery from meshes) report that this direct route is harder than the two-stage mesh-to-B-rep-to-sequence approach, because reasoning about feature operations on a noisy mesh combines the difficulty of surface recovery with the difficulty of sequence inference into a single optimization. Progress is real, but the state of the art handles a narrow slice of part complexity.

None of these pathways is production-ready for real engineering drawings. The image-to-mesh family, which is what most foundation models actually deliver today, is several steps removed from the representation manufacturing pipelines need. In practice, translating a generated mesh into a clean STEP file requires fitting analytical surfaces against a noisy approximation, reconstructing the topology, and producing a B-rep that holds up to manufacturing scrutiny. The engineering effort required is comparable to the effort of reconstructing the STEP directly from the drawing, which is why dedicated 2D-to-3D pipelines bypass the mesh stage entirely.

Correctable or structural

Structural for image-to-3D foundation models as currently designed. The training objective of Trellis, SAM3D, Hunyuan3D and their kin is to produce photogrammetrically plausible 3D shape, sampled rather than analytical. Adding an exact-surface output head is not a fine-tuning task; it is a different problem, requiring different data (paired with B-rep ground truth rather than meshes), different architecture, different evaluation metrics, and a different deployment context.

Generating the CAD sequence above that is a further step that few systems have demonstrated even on simple parts, and the data problem becomes sharper at this level. Real CAD construction histories are proprietary to a handful of CAD software vendors and held in customer files that are not shareable; the open research datasets that exist rely on simplified synthetic sequences covering only a fraction of industrial part complexity. Recent work on richer synthetic sequence generation is promising, but remains far from the diversity of real industrial archives.

Until the field shifts its training objective from sampled appearance to analytical geometry and construction history, and acquires the data to train against at scale, no amount of scale at the mesh level will close the manufacturing gap. The 3D generative family is solving a related but distinct problem, and conflating the two leads to disappointed expectations downstream.

Limit 3 — Design intent that is invisible from drawing pixels

A foundation model, by design, learns patterns from what is visible in the training data. An engineering drawing is a hybrid document: some information is explicit on the page, but a large fraction is implicit, governed by conventions the reader is assumed to know.

A dimension callout of Ø12 H7 is explicit pixels: the number 12, the symbol Ø, the letters H7. What it means, that this is a hole, that the tolerance is +0.018 / +0 millimeters, that the hole is meant to mate with a shaft of corresponding tolerance, comes from ISO 286 and from decades of mechanical engineering convention. A foundation model can be trained to recognize the surface form. Recognizing the structural intent requires either explicit encoding of those conventions or enough training data to learn them implicitly. Neither is currently solved at the level engineering work demands, and the gap is not closing as fast as the surface-recognition gap.

The problem extends well beyond tolerances. Drawing conventions involve projection style (first-angle vs third-angle, a confusion that can flip a part’s orientation completely), sectioning conventions, hatching patterns indicating material, surface finish symbols, weld symbols, geometric dimensioning and tolerancing (GD&T) frame controls, datum references, and a long list of regional and industry-specific conventions. Each of these encodes engineering meaning that must be reconstructed correctly for the resulting CAD model to be production-grade. A model that fails on any of them produces output that looks plausible but is wrong in ways that only emerge downstream, often at the point where the CAD file meets a manufacturing process and a feature that should have been cylindrical turns out to be conical, or a hole that should have been positionally toleranced has no datum reference attached.

There is also the problem of missing information: what the drawing does not say. A drawing typically does not specify which features are functional and which are aesthetic. It does not state which surfaces matter for fit and which are clearance. It does not annotate which holes are critical for assembly and which are for weight reduction. Experienced engineers infer these distinctions from context: the type of part, the industry, the manufacturing process the part is destined for, the materials involved, and the operating conditions the part will see in service. Foundation models trained on broad visual data have little of this context. Foundation models trained on drawing data have little context beyond the drawing itself, and certainly not the engineering history that gives a real engineer the framing to make these judgments.

For a more focused analysis of how GD&T attachment to 3D geometry compounds this problem, see tolerances, GD&T, and reconstruction limits. The relevant point here is that recognizing a tolerance frame in a drawing (a surface-recognition task) is a different problem from correctly attaching that tolerance to a feature in the 3D reconstruction (a structural-inference task). The first is being progressively solved; the second is harder.

Correctable or structural

Structural at the core. The surface conventions (recognizing GD&T frames, projection style, common symbols) can be trained on enough labeled data, and progress here is steady. The deeper inference problem (separating functional from non-functional, identifying critical features, completing missing information) is structurally hard because the information needed is not in the drawing. A generalist foundation model cannot acquire it from training on drawings alone. A specialized system can mitigate it by combining drawing analysis with engineering review, which is the design choice Theia and similar dedicated systems make.

Limit 4 — Geometric precision and spatial reasoning gaps in VLMs and LLMs

This is the limit that surprises people most. The intuition that large multimodal models like GPT-4V, Gemini, or Claude could “just read the drawing and output the geometry” rests on the assumption that these models are precise enough at spatial reasoning to be useful for the task. They are not, at the level engineering work requires, and the gap between their qualitative competence and their quantitative reliability is not narrowing as fast as the scaling curves alone would suggest.

VLMs are excellent at qualitative visual reasoning. They can describe a drawing, identify its main features, recognize that a particular view is the front view, and answer general questions about the part’s likely function. They are weaker at quantitative spatial reasoning. Asking GPT-4V to count exactly how many holes are in a flange produces an answer that is often correct but unreliably so. Asking it to identify which view is the top view depends on whether the convention used in the drawing matches the convention most common in the training data. Asking it to estimate the angle between two features, or the perpendicular distance between two axes, yields a confident response that may or may not be accurate to within engineering tolerance. The error rate is low enough to be impressive in casual use, and high enough to be unacceptable in production engineering where dimensional errors propagate into manufacturing.

The deeper issue is that these models do not have a geometric representation of the scene they are looking at. They have learned correlations between visual patterns and language tokens. When asked about a precise relationship, “does the centerline of hole A pass through the axis of cylinder B?”, they generate a plausible-sounding answer rather than computing the relationship from a geometric model of the part. The output reads as confident technical analysis but reflects pattern matching, not measurement. The same query asked of a system with an actual 3D geometric model produces a determinate true-or-false answer; the VLM produces a probability-weighted natural-language response that may not even be self-consistent across rephrasings of the question.

This shows up in a particular way for 2D-to-3D reconstruction. Reconstructing a 3D model from orthographic projections requires constraint solving: each view contributes partial information about the part, and the 3D geometry is the unique (or constrained-set-of) solution that satisfies all views simultaneously. This is a precise mathematical operation that operates on geometric primitives, not a pattern-recognition task that operates on tokens. The front view tells you the width and the height of a feature; the top view tells you the width and the depth; the side view tells you the height and the depth. Reconciling these three partial measurements into a single feature with known location, orientation, and dimensions requires solving the constraint system the views collectively define. A small inconsistency between views, the kind that occurs routinely in real drawings, has to be resolved by choosing which view is the reference, which is an engineering judgment that depends on the part’s function rather than on the drawing alone. Current VLMs cannot perform this kind of constraint solving on geometric primitives reliably. They can describe what the solution would look like in natural language; they cannot produce it to the dimensional accuracy a manufacturing pipeline expects.

There is research on improving spatial reasoning in VLMs, and progress is real. The gap between qualitative description and quantitative reconstruction remains large, and closing it is not just a matter of scale. It requires either explicit geometric reasoning modules attached to the language model (at which point the system stops being a pure VLM and becomes a hybrid), or fundamental changes to how these models represent and manipulate spatial information. The current architectural family is excellent at the things it was designed for, and CAD-precision spatial reasoning is not among them.

Correctable or structural

Structural at the core. Surface-level spatial reasoning will improve with continued scaling and better training data; counting holes, identifying views, and approximate spatial description will become more reliable. Reliable geometric precision at engineering tolerance (sub-millimeter accuracy on dimensions, exact constraint satisfaction across views, deterministic feature identification) is structurally outside what current VLM architectures are designed to provide. Closing this gap likely requires moving from pure language modeling to systems that explicitly compute over geometric representations, which is a different category of system than what GPT-4V, Gemini, or Claude are today.

Correctable vs structural: which limits will close, which won’t

The four limits compound. Even if one is partially closed by future progress, the others remain. The table summarizes where each sits.

Four cards summarizing the four limits of foundation models for CAD reconstruction: training data (correctable), output representation, design intent, and geometric precision (all three structural).
LimitManifestationVerdict
Training data scale and realismModels trained on under 200K synthetic pairs do not transfer to real archivesCorrectable. Data engineering problem, not a research dead-end. Scales with effort, bounded only by the cost of producing real paired data.
Output representation: mesh → B-rep → CAD sequenceMesh / gaussian splat / neural field is sampled geometry, not the exact-surface B-rep manufacturing depends on, and even further from the CAD sequence needed for re-engineeringStructural. Wrong training objective. Closing the gap requires a different architecture trained on different data, not more scale on the same one.
Design intent invisible from pixelsFunctional vs aesthetic, critical vs clearance, missing contextStructural. The information needed is not in the drawing. No model trained on drawings alone can recover it; engineering judgment is required.
Geometric precision and spatial reasoning in VLMsQualitative description ≠ quantitative reconstructionStructural. Token-based reasoning cannot reach engineering tolerance without an external geometric engine. Scaling improves the qualitative layer, not the quantitative one.

A useful way to read this table: three of the four limits are structural, not transitional. Scaling and better data will help with the first limit, and with the surface-level aspects of the other three, but the core gaps (wrong output representation, irrecoverable design intent, token-based reasoning that cannot reach engineering precision) do not close with more scale. They close, if at all, with different architectures. The specific task of producing production-grade CAD output (exact B-rep geometry, ideally with feature recognition) from real engineering drawings, with the precision and traceability that manufacturing workflows demand, requires a system designed for that task. That is what dedicated pipelines like Theia are. The architectural decomposition is the differentiator, not the underlying capability of the AI components.

What this means for buyers evaluating AI CAD tools

For a Method Engineer or Engineering Lead evaluating tools, the practical implication is to separate two questions that vendor demos often conflate.

First question: does the tool produce engineering-grade output, or visually plausible output

A demo that shows a 3D model rendered next to the input drawing is impressive, but the rendered view is a poor proxy for the underlying representation. The follow-up that matters is what file format the tool actually exports. A STEP file is, by construction, an exact-surface B-rep representation that downstream CAD and CAM systems use natively. A mesh export (STL, OBJ, glTF, USD) is a sampled approximation that cannot be used for manufacturing without further reconstruction work. The separation between tools intended for visualization and tools intended for production is visible in the export menu, not in the demo render.

Second question: where does engineering judgment enter the workflow

The four limits above all reduce, in practice, to the same operational question: who catches the failure modes that no automated system catches reliably? A pipeline that produces output and ships it without review accepts the residual error rate as the user’s problem. A pipeline that includes engineering review as a designed step accepts the error rate as its own problem and resolves it before delivery. Both can be built on AI; only the second is honest about what AI alone cannot do, and only the second produces output that holds up when it reaches the manufacturing floor without the user having to validate every file manually.

For a pragmatic walk-through of the drawing-to-CAD reconstruction process and where validation fits, see how AI converts 2D drawings to 3D. For the comparison of approaches across in-house, outsourced, and AI-assisted workflows, see in-house vs outsourced vs AI.

Where Theia stands

Theia is built on the recognition that the four limits above are real and that no single AI architecture solves all of them. The pipeline is hybrid by design: AI does the work it does well, and engineering review does the work AI cannot do reliably. Each of the four limits is addressed by a specific layer of the system, and the resulting STEP files are validated by engineers before delivery rather than shipped on the model’s confidence alone.

For the training data and realism limit, Theia is trained on real industrial drawings, the kind that arrive as PDFs, scans, and photocopies, rather than on synthetic renderings of clean CAD. The training corpus is built incrementally from production work, which is slower than scraping the web but produces a model that generalizes to the operational reality of legacy archives. The trade-off is deliberate: a smaller training set drawn from real distributions outperforms a much larger synthetic set when the deployment target is real drawings.

For the output representation limit, Theia produces STEP files with exact analytical surfaces stitched into a clean B-rep topology. The internal reconstruction reasons about geometric features (cylinders, planes, swept surfaces, fillets) rather than about polygons, and the export preserves these as canonical surfaces that downstream CAD and CAM systems read natively. The intermediate mesh stage that image-to-3D models cannot avoid is not part of the pipeline. Native export to feature-bearing formats like SolidWorks and CATIA, which would also preserve a CAD feature sequence, is on the roadmap; today, exact-geometry STEP is the canonical output, and it integrates with the major CAD environments used in production engineering.

For the design intent limit, Theia separates what can be read from the drawing pixels (the geometric reconstruction) from what requires engineering judgment (the verification that the reconstruction respects functional intent). The geometric reconstruction is automated. The verification is performed by engineers on every delivered model. This is the validated-by-engineers commitment, and it exists precisely because no fully automated system can be trusted with the inference problem at the heart of design intent. The cost of human verification is real, but it is the cost of producing output that is actually usable downstream, not output that merely looks usable.

For the geometric precision and spatial reasoning limit, Theia combines two layers that work in concert. A heuristic geometric pipeline handles the parts of the reconstruction where exact computation over primitives is reliable: surface fitting, topology stitching, dimensional constraint solving across views. Above that, a learned component proposes plausible feature sequences in the regions where pure heuristic reverse-engineering struggles. Dense geometry zones are the canonical example: when several surfaces meet in a small region, recovering them from their final form alone is ambiguous, because many distinct construction sequences can produce visually similar results. A nut is a simple illustration: without knowing the sequence of operations that produced it, even a correct geometric reconstruction is hard to recover because the intent behind each feature is not visible in the geometry alone. The learned component proposes candidate sequences; the heuristic pipeline validates them against the actual geometry. Neither layer could do this work alone, and the combination is closer to the architectural decomposition the four limits demand than to any monolithic approach.

A foundation model alone could not produce Theia, not because foundation models are inadequate at what they do, but because the task requires a different decomposition. Building a production-grade 2D-to-3D CAD reconstruction pipeline means accepting that four cumulative limits each require different solutions, and that combining them under a single training objective is not how the problem yields. The architecture choice (AI for what AI does well, engineering review for the rest) falls out of taking the limits seriously rather than wishing them away. For a deeper analysis of why this hybrid architecture is structurally necessary, see hybrid AI for industrial CAD.

Frequently asked questions

Specific questions about foundation models and CAD reconstruction. For general Theia questions, see our main FAQ.

Could a much larger model trained specifically on engineering drawings solve this?

Scale alone does not solve the problem, for two reasons.

First — the right dataset does not exist at scale and is unlikely to. Real CAD feature sequences are held by a small number of CAD software vendors as proprietary customer data, with no research-accessible equivalent at industrial scale. Synthetic alternatives cover only a narrow slice of part complexity, and even substantial expansion would not reach the diversity of real industrial archives.

Second — the input side remains untouched. Even if such a dataset became available, it would not solve the upstream problem of vectorizing meaningful information from a real 2D raster: scanned, photocopied, or hand-annotated drawings with the noise, ambiguity, and convention variability that come with them. The drawing-to-vector step is a distinct problem from the vector-to-CAD step, and scaling the output side does not address the input side.

The architectural choice precedes the scaling choice.

What about CAD recovery from meshes, extracting features or sequences after generation?

CAD recovery from meshes splits into two research directions, both of which struggle on real industrial parts.

Mesh-to-B-rep — fits canonical surfaces against the mesh and stitches them into a clean topology. It works for restricted part families (primitive-rich, axis-aligned) and degrades on curved surfaces, complex geometry, and the noise level of generated meshes.

Mesh-to-CAD-sequence — tries to recover the construction history directly from the mesh. Research efforts report this is harder than going through the B-rep stage first, because reasoning about feature operations on a noisy mesh combines surface recovery and sequence inference into a single optimization.

In practice, recovering clean exact-surface CAD from a noisy mesh is comparable in effort to reconstructing the CAD directly from the drawing, which is why dedicated pipelines skip the mesh stage entirely. The chain “image to mesh to CAD” introduces an intermediate representation that adds error without removing work.

Can VLMs help as a component of a larger system, even if they cannot do reconstruction alone?

Yes, in narrow roles. VLMs are useful for tasks like classifying drawing type, extracting metadata, summarizing notes, or flagging non-standard conventions — auxiliary tasks where qualitative reasoning suffices and where the failure mode is a missed flag rather than a wrong dimension. They are not load-bearing components of the geometric reconstruction itself, which still requires a system designed for geometric work.

How does this affect AI image-to-3D tools that work on photographs?

Image-to-3D tools serve a different purpose than 2D-to-3D CAD reconstruction. Their target output is visualization, prototyping, AR/VR, content creation, or 3D-printable approximations — not manufacturing-grade CAD.

Within those domains, current tools (Trellis, SAM3D, Hunyuan3D, Meshy, and similar) are genuinely capable: they handle a wide range of input photographs and produce 3D shapes that look convincing from most angles.

The remaining limit even in their target domain is mesh quality: generated meshes often carry noise, uneven topology, surface artifacts, or holes that require cleaning before the mesh is usable downstream, even for non-manufacturing applications.

For the comparison in detail, see AI image-to-3D vs drawing-to-3D.

How do foundation models and dedicated pipelines fit together in a practical engineering workflow?

They sit at different points of the workflow, and serve different roles even when used inside the same pipeline.

General-purpose foundation models are useful upstream and downstream of the geometric reconstruction itself — classifying drawings, extracting metadata, flagging non-standard conventions, summarizing technical notes. They handle the qualitative tasks where their breadth is the right tool.

Learned components specialized for the task can sit inside the reconstruction pipeline, alongside heuristic engines, to handle the parts of the problem where pattern-based proposals are useful but exact geometric validation is still required. This is the role they play in Theia’s hybrid architecture.

Treating these as complementary layers, each doing the work it is best suited for, produces a more useful pipeline than asking a single general-purpose model to cover the whole stack.

Going further

For the foundational explanation of how AI converts drawings into 3D models, see how AI converts 2D drawings to 3D. For the architectural rationale behind hybrid AI in industrial CAD, see hybrid AI for industrial CAD. For the comparison of approaches across in-house, outsourced, and AI-assisted reconstruction, see in-house vs outsourced vs AI.

Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.