Luce: Relightable Gaussians for 3D Asset Generation
Apple Machine Learning Research
Luce presented a voxelized multimodal Gaussian 3D representation that jointly generates geometry and PBR material outputs from a single image. It reports a 28% improvement in FID on Toys4K versus the strongest baseline and raises CLIP image-alignment to 0.8519 from 0.8299. The method changes single-image-to-3D generation by producing relightable PBR Gaussians (with optional textured meshes and normal maps) that better preserve appearance details like text and logos.
Why it matters
High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A…
Related stories
Black Forest Labs Releases FLUX 3: A Multimodal Flow Model for Image, Video, Audio and Robot Action Prediction
MarkTechPost · 1 month ago ·
15
FLUX 3 Expands Black Forest Labs' Image Generation Capabilities
Black Forest Labs · 1 month ago ·
14