Why is 3D scanning hard?

Marc Levoy
Stanford University

Improvements in optical rangefinding technology have made it easy to acquire
dense 3D samples of an object or scene. However, a fully automated procedure
for assembling this data to create a geometric model still eludes us. This is
especially true for large or complicated objects scanned under non-laboratory
conditions. For example, although Stanford's Digital Michelangelo Project
produced some nice models, it did not achieve many of its stated objectives.

In this talk, I briefly survey the unsolved problems of 3D scanning. For some
of these problems, well defined solutions exist, and steady progress can be
made on them. Examples of this type include calibration of scanning platforms,
multi-view registration, surface reconstruction, view planning, and the
handling of large datasets. For other problems, solutions exist but the
problem is usually badly conditioned due to noise. Examples of this type
include multi-view registration in the presence of scanner miscalibration,
estimation of surface reflectance, and the scanning of optically uncooperative
materials. In still other cases, the problem itself is ill-posed, admitting
multiple correct answers. Classic examples of this type are surface
reconstruction in the presence of missing data (i.e. holes), and estimation of
surface shape and reflectance in the presence of interreflections or subsurface
scattering. Finally, there are problems for which no good solutions exist,
such as scanning geometrically convoluted objects, and insuring safety for the
objects being scanned.

We end with a (mostly) upbeat assessement of the long-term prospects for 3D
scanning and some predictions concerning its likely impact on industry and
popular culture.