Why 3D Human Scan Registration Is Hard
A 3D body scan looks rich, but as data it is mostly a list of points in space. Each point has coordinates. From the scan alone, we cannot tell which points belong to the thigh, pelvis, shoulder, or abdomen. It does not encode left and right, front and back, joint locations, segment boundaries, or anatomical landmarks.
BODIESReg is an open-source pipeline for registering SMPL-family parametric body models to 3D human scans. Here, we first explain registration for 3D human scans, then show why distance-based fitting can fail and how BODIESReg makes the optimization less fragile.
What Is Registration?
Registration means assigning anatomical meaning to a raw scan by matching it to a body model whose anatomy is already known. A body model such as SMPL is also a surface, but its surface is organized. Its vertices have fixed identities. Its skeleton, joints, pose parameters, shape parameters, and body regions are defined in advance. If we can match this known template to an unknown scan, then labels from the template can be transferred to the scan.
That transfer is the practical goal of registration. We start with a target point cloud that has geometry but no semantic structure. We end with a scan whose vertices can be interpreted through template correspondence: this region is the upper leg, this vertex is near the hip, this vertex belongs to the torso, and so on. Scanning gives us geometry and registration gives that geometry anatomical meaning.
How Registration Assigns Meaning
The template is useful because we know what every part of it means. The target scan is useful because it is the person we measured. Registration connects the two.
Suppose the template has vertices
\[\mathcal{V}=\{\mathbf{v}_j\}_{j=1}^{N},\]and the scan has vertices
\[\mathcal{S}=\{\mathbf{u}_i\}_{i=1}^{N_S}.\]If template vertex $\mathbf{v}_j$ corresponds to scan vertex $\mathbf{u}_i$, then any anatomical label attached to $\mathbf{v}_j$ can be assigned to $\mathbf{u}_i$. This is how a plain array of 3D vertices becomes usable for landmark extraction, segmentation, and measurement.
How Correspondences Are Assigned
The hard part is finding the correspondence. We usually do not know in advance which scan vertex should match which template vertex. A common strategy is therefore indirect:
- Move and deform the template until it lies close to the scan.
- For each template vertex, choose a nearby scan vertex, often the nearest neighbor.
- Transfer labels from template vertices to their matched scan vertices.
This nearest-neighbor step is simple and useful, but it depends strongly on where the template is placed before matching. If the arm of the template starts near the torso in the scan, nearest-neighbor matching may connect arm vertices to torso vertices. The nearest-neighbor algorithm makes a geometrically close match, but not an anatomical one.
Fitting Pose And Shape
For SMPL-family models, the template is not moved vertex by vertex. Instead, the mesh is generated from pose and shape parameters. We can write this as
\[M: (\boldsymbol{\beta},\boldsymbol{\theta}) \mapsto M(\boldsymbol{\beta},\boldsymbol{\theta}),\]where $\boldsymbol{\beta}$ controls body shape and $\boldsymbol{\theta}$ controls pose. Registration then becomes an optimization problem: find the pose and shape parameters that bring the generated mesh close to the scan.
A common distance objective is the template-to-scan Chamfer distance,
\[E(\boldsymbol{\beta},\boldsymbol{\theta}) = \frac{1}{|\mathcal{V}|} \sum_{\mathbf{v}\in\mathcal{V}(\boldsymbol{\beta},\boldsymbol{\theta})} \min_{\mathbf{u}\in\mathcal{S}}\|\mathbf{v}-\mathbf{u}\|\]In words, this objective asks whether each template vertex has a nearby scan vertex. BODIESReg also uses regularization terms that keep the optimized pose and shape near the pose-initialized estimate, but the core idea is distance minimization.
The video below illustrates why distance is a poor metric for anatomical correctness. The blue point cloud is the input scan. The red point cloud is the template. We move the input scan relative to the fixed template using rotations, translations, and scale changes, and record the Chamfer distance along each transformation. The plotted curve is only one path through a much larger parameter space, but it already shows the main problem: several different arrangements can produce similar distances, while only one arrangement has meaningful anatomical correspondence.
Distance minimization is useful because it gives the optimizer a clear target. The difficulty is that the optimizer does not know anatomy unless anatomy is encoded in the objective, initialization, model, or constraints. It only sees the function we ask it to minimize.
What BODIESReg Adds
BODIESReg changes where surface fitting starts. Instead of beginning from a default template pose, it estimates an approximate scan pose first, builds a pose-aligned template, and then optimizes pose and shape against the scan.
The video below shows intermediate steps in BODIESReg. Before final pose-and-shape optimization, BODIESReg estimates a pose-aligned initialization that brings the red template close to the blue scan. The later optimization then refines the fit. The distance decreases while the anatomy stays in the intended configuration.
In the second video, we show the same pose-and-shape optimization without pose-aligned initialization. The optimizer can still reduce the distance, but it converges to a wrong anatomical arrangement. After convergence, the distance curve is close to the pose-aligned case, shown in gray, yet the registered body is upside down. This is the central failure mode: distance can look acceptable while correspondence is wrong.
Code is available at chaurasiavikash/BODIESReg. The paper is available on arXiv: 2607.15463.
Limitations
Pose-aligned initialization depends on keypoint detection. If the detected keypoints are wrong, the inverse-kinematics step can initialize the body model in the wrong pose, and the later surface-fitting steps can inherit that error. Difficult poses, occlusions, missing scan regions, clothing, or unusual scan views can all make this worse.
Because BODIESReg uses detected keypoints to initialize 3D pose, errors in those keypoints can propagate into later registration steps. In BODIESReg, the pose editor provides a manual correction path when automatic keypoint detection fails.
Closing
We built BODIESReg because we needed to batch-process optical and MR body scans, but could not find a published open-source framework for automatic registration without dataset-specific training. We also needed registrations that were not only numerically close, but anatomically meaningful.
BODIESReg runs locally, works on modest hardware, and includes manual correction tools for difficult cases.
Credits
Co-authors: Vikash Chaurasia, Judit Cueto Fernandez, J. Micah Prendergast, and Eline van der Kruk.
Related links: