Piecewise-Smooth Surface Fitting onto Unstructured 3D Sketches
We propose a method to transform unstructured 3D sketches into piecewise smooth surfaces that preserve sketched geometric features. Immersive 3D drawing and sketch-based 3D modeling applications increasingly produce imperfect and unstructured collections of 3D strokes as design output. These 3D sketches are readily perceived as piecewise smooth surfaces by viewers, but are poorly handled by existing 3D surface techniques tailored to well- connected curve networks or sparse point sets. Our algorithm is aligned with human tendency to imagine the strokes as a small set of simple smooth surfaces joined along stroke boundaries. Starting with an initial proxy surface, we iteratively segment the surface into smooth patches joined sharply along some strokes, and optimize these patches to fit surrounding strokes. Our evaluation is fourfold: we demonstrate the impact of various algorithmic parameters, we evaluate our method on synthetic sketches with known ground truth surfaces, we compare to prior art, and we show compelling results on more than 50 designs from a diverse set of 3D sketch sources.
3D sketches done in VR are beautiful to behold and effectively perceived by viewers, yet are non-trivial to transform into 3D surface models for further design communication and processing. That is because they are sparse and unstructured, the strokes do not form a well-connected curve network with precise intersections (see Fig. 1c). Many on-surface details are also sketched (Fig. 1b), which help perceive the surface but make it challenging to structure the sketch into a curve network a posteriori.
Based on the observation that the artists explicitly depict sharp features of the surface with strokes (Fig. 1a inset, dashed green lines), we propose a method to transform an initial globally smooth “proxy” surface (Fig. 2b) – which can be obtained with previous methods or modelled easily by the artists – into a piecewise-smooth surface where sharp features align with sketched geometric features (Fig. 2d).
After projecting the sketch onto the proxy, our goal is to segment the proxy into regions, each associated with a smooth surface model, such that the resulting piecewise-smooth surface satisfies the following desiderata.
- Reproducing stroke geometry. The surface models should run close to the input strokes, both for strokes that depict sharp surface discontinuities and for strokes that depict details within smooth areas.
- Aligning patch boundaries with strokes. The boundary be- tween neighboring regions should lie along strokes that depict sharp surface discontinuities, yet not all strokes in the sketch depict a discontinuity.
- Keeping the reconstruction simple. The surface should be composed of a small number of smooth patches, rather than many intricate patches that would overfit to inaccuracy in the input strokes.
We formulate these competing requirements as energy terms in an optimization. We find a local optimum by alternating between optimizing the segmentation while keeping the surface model parameters fixed, and optimizing for the surface model parameters while keeping the segmentation fixed (Fig. 2c).
Here are some results:
Note that our method can fit piecewise-smooth surfaces even onto freehand imprecise sketches (author2_guitar) or sketches that present a strong amount of oversketching (swh_vr_controller).
Please have a look at our result webpage that presents all 50+ results, in the form of interactive 3D viewers.