We propose a 3D Gaussian splatting-based framework for outdoor relighting that leverages intrinsic image decomposition to precisely integrate sunlight, sky radiance, and indirect lighting from unconstrained photo collections.
Unlike prior methods that compress the per-image global illumination into a single latent vector, our approach enables simultaneously diverse shading manipulation and the generation of dynamic shadow effects. This is achieved through three key innovations:
(1) a residual-based sun visibility extraction method to accurately separate direct sunlight effects, (2) a region-based supervision framework with a structural consistency loss for physically interpretable and coherent illumination decomposition, and (3) a ray-tracing-based technique for realistic shadow simulation.
Extensive experiments demonstrate that our framework synthesizes novel views with competitive fidelity against state-of-the-art relighting solutions and produces more natural and multifaceted illumination and shadow effects.
Method Pipeline
Building on the principles of intrinsic image decomposition, our relighting framework, termed GaRe, accurately separates reflectance and multiple physically interpretable shading components—sunlight, sky radiance, and indirect lighting—from unconstrained outdoor photo collections. This meticulous decomposition transforms these components into versatile lighting assets, enabling precise and highly adaptive relighting.
Results: Novel View Synthesis
Comparison with SOTA methods in outdoor NVS. Our method delivers high-quality texture reconstruction for landmarks and surpasses existing methods in synthesizing sunlight, closely matching ground truth with sharp and accurate boundaries between illuminated and shadowed regions.
Results: Illumination Decomposition
Sun shading captures the effects of direct sunlight, sky shading represents diffuse radiance from the sky, indirect shading models global illumination interactions, and reflectance preserves material properties independent of lighting. Combined with the sun visibility map, these components reconstruct the final illumination with high accuracy and physical plausibility
Results: Illumination Interpolation
Relighting through illumination interpolation. Visualization shows how scene appearance evolves when interpolating between two arbitrary illumination conditions. Each shading component—sun shading, sky shading, and indirect shading—is interpolated both individually and collectively (All), highlighting the ability to control lighting variations at a granular level.