Teaching a Machine the Grammar of Geometry
On building an autoregressive model that generates mesh from CAD — and what it means to treat three-dimensional form as a language to be spoken, one token at a time.
Meshing is the unglamorous bottleneck of simulation, still dependent on expert intuition. If mesh generation is treated as a sequential problem — each face conditioned on the faces already laid down — an autoregressive transformer can learn it the way a language model learns grammar, inducing the tacit rules of good meshing from CAD-to-mesh data.
Mesh generation is sequential: a mesh is built element by element, each face conditioned on prior faces — exactly what an autoregressive model does.
The hard part is the vocabulary: geometry is continuous, so coordinates must be discretised into tokens, trading reconstruction fidelity against vocabulary size.
The model must induce meshing rules (refine at high curvature, respect feature edges) from data — none are written into the architecture.
There is a moment in language modelling — every practitioner knows it — when the model stops producing noise and starts producing sense. The loss has descended for hours, the validation curve is behaving, and then a generated sentence appears that is grammatically correct, contextually apt, and not in the training data. The machine has learned, in some approximate sense, how language works.
I spend my days chasing that moment. Except my substrate is not words. It is vertices, edges, faces — the raw geometry of a finite-element mesh.
The Problem Nobody Likes to Talk About
Meshing is the unglamorous bottleneck of simulation. Before a crash simulation can run, an engineer must transform a CAD model — a clean, parametric, manufacturer's description of a part — into a mesh: a discrete approximation of that surface as thousands of interconnected triangles and quadrilaterals. The quality of the mesh determines the quality of the simulation. A poor mesh produces a poor result; the solver cannot be blamed.
Traditionally, this step is part art, part engineering. Specialist tools exist. Rules of thumb have been accumulated over decades. But it remains slow, expensive, and stubbornly dependent on expert intuition. For a component geometry that has never been meshed before, there is no shortcut.
What if a model could do it the way a language model generates text?
Treating Geometry as a Sequence
The key insight — and the one that took the longest to get right — is that mesh generation is a sequential problem. A mesh is not produced all at once. It is built up, element by element, each new face conditioned on the faces already laid down. This is precisely what an autoregressive model does: predict the next token, given all the previous ones.
The engineering challenge is the vocabulary. In language, tokens are subwords — a finite, discrete set that covers everything speakable. In geometry, the space is continuous. Every vertex has exact floating-point coordinates; there is no clean discretisation.
Several approaches exist. You can quantise the coordinate space into bins, treating each bin index as a token. You can encode mesh elements as sequences of vertex indices and learn a codebook. You can work at the face level and encode local patch geometry as a learned embedding. Each choice involves trade-offs between reconstruction fidelity and the size of the vocabulary the model must maintain.
What the Model Sees
The input is a CAD model, represented as a boundary representation (B-rep): a collection of surfaces, edges, and vertices with their geometric and topological relationships. The model must learn to condition its output — the mesh — on this input.
This conditioning is the part I find most interesting. The model must learn, from data, the implicit rules that human meshing engineers apply tacitly: concentrate elements where curvature is high, coarsen where the surface is flat, respect feature edges, handle transitions between regions of different element density smoothly.
None of these rules are written into the architecture. The model has to induce them from the correspondence between input CAD geometry and output meshes seen during training.
The Moment
The first time a generated mesh held its shape — preserved the topology of the input, maintained manifold validity, produced elements of reasonable aspect ratio in the right places — was quiet. No alarm went off. The visualisation appeared in the viewport and I sat with it for a while.
It was a simple bracket. Not a complex part. But the transitions were clean, the refinement was in the right places, and the mesh was valid. The machine had, in some narrow and provisional sense, learned a piece of the grammar of geometry.
The work is ongoing. The vocabulary is still being refined. The model generalises to new part families imperfectly. But the direction is clear: treat geometry as language, train a model to speak it, and see how fluent it becomes.
The next note in this series will be on the failure modes — which are, as always, more instructive than the successes.
the reader’s voice, entering the same model the instrument ran the essay through