GS-RoadPatching: Inpainting Gaussians via
3D Searching and Placing for Driving Scenes

SIGGRAPH Asia 2025
Guo Chen1*, Jiarun Liu2,3*, Sicong Du2, Chenming Wu4,
Deqi Li1, Shi-Sheng Huang1†, Guofeng Zhang3, Sheng Yang2†
1Beijing Normal University    2Unmanned Vehicle Dept., Cainiao, Alibaba   
3State Key Lab of CAD&CG, Zhejiang University    4Baidu
Corresponding authors * denotes equal contribution

Abstract

This paper presents GS-RoadPatching, an inpainting method for driving scene completion by referring to completely reconstructed regions, which are represented by 3D Gaussian Splatting (3DGS). Unlike existing 3DGS inpainting methods that perform generative completion relying on 2D perspective-view-based diffusion or GAN models to predict limited appearance or depth cues for missing regions, our approach enables substitutional scene inpainting and editing directly through the 3DGS modality, extricating it from requiring spatial-temporal consistency of 2D cross-modals and eliminating the need for time-intensive retraining of Gaussians. Our key insight is that the highly repetitive patterns in driving scenes often share multi-modal similarities within the implicit 3DGS feature space and are particularly suitable for structural matching to enable effective 3DGS-based substitutional inpainting. Practically, we construct feature-embedded 3DGS scenes to incorporate a patch measurement method for abstracting local context at different scales and, subsequently, propose a structural search method to find candidate patches in 3D space effectively. Finally, we propose a simple yet effective substitution-and-fusion optimization for better visual harmony. We conduct extensive experiments on multiple publicly available datasets to demonstrate the effectiveness and efficiency of our proposed method in driving scenes, and the results validate that our method achieves state-of-the-art performance compared to the baseline methods in terms of both quality and interoperability. Additional experiments in general scenes also demonstrate the applicability of the proposed 3D inpainting strategy.

Pipeline

GS-RoadPatching pipeline diagram

Our method follows a four-stage pipeline: (a) automatic localization and segmentation of incomplete regions requiring inpainting; (b) 3D search and similarity measurement to identify optimal Gaussian candidates; (c) seamless pasting and fusion of selected primitives into target regions; (d) scalable editing through our proposed patch-anchor indexing mechanism for efficient region-based operations.

BibTeX

@inproceedings{chen2025gsroadpatching,
  title={GS-RoadPatching: Inpainting Gaussians via 3D Searching and Placing for Driving Scenes},
  author={Chen, Guo and Liu, Jiarun and Du, Sicong and Wu, Chenming and Li, Deqi and Huang, Shi-Sheng and Zhang, Guofeng and Yang, Sheng},
  booktitle={SIGGRAPH Asia 2025 Conference Papers},
  year={2025},
}