Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features

The Problem

Reconstructing clean 3D shapes from noisy or sparse data is a long-standing challenge in computer graphics. Most deep learning approaches rely on small, curated datasets of 3D shapes, which limits their ability to generalize to real-world objects with complex geometry and appearance.

Our Approach

Instead of training on scarce 3D data, we tap into the vast visual knowledge already captured by image-based foundation models trained on billions of photographs. Our pipeline has two key components:

• Semantic-consistency regularization — We train a differentiable Gaussian-splatting renderer with an auxiliary loss that enforces consistent semantics across viewpoints, which stabilizes few-shot reconstruction.
• Diffusion-based denoising — We fine-tune Stable Diffusion to denoise rendered views of noisy point clouds. These learned 2D filters effectively remove 3D noise without any explicit 3D supervision.

Results

Evaluated on the Sketchfab Testset and the SCUT Dataset, our method achieves higher-quality 3D reconstruction compared to DSS and PointRadiance. By bridging 2D foundation models and 3D reconstruction, we show that large-scale image priors can be a powerful substitute for large-scale 3D datasets.