Stability AI has just released Stable Video 3D (SV3D), a generative model that advances the field of 3D technology, delivering improved quality and view-consistency. Built upon the foundation of Stable Video Diffusion, SV3D delivers superior quality and multi-view consistency compared to its predecessors and open-source alternatives.
The SV3D release comes in two flavors: SV3D_u and SV3D_p. SV3D_u generates orbital videos from single image inputs without the need for camera conditioning. SV3D_p takes it a step further by supporting both single images and orbital views, enabling the creation of 3D video along specified camera paths. This level of flexibility and control opens up a world of possibilities for content creators and developers alike.
One of the key advantages of SV3D lies in its use of video diffusion models. Unlike image diffusion models used in Stable Zero123, video diffusion provides significant improvements in generalization and view-consistency of generated outputs. As Stability AI explains in their technical report, "By adapting our Stable Video Diffusion image-to-video diffusion model with the addition of camera path conditioning, Stable Video 3D is able to generate multi-view videos of an object."
SV3D's novel view synthesis capabilities are particularly impressive. It can generate coherent views from any given angle with exceptional generalization, ensuring consistent object appearance across multiple perspectives. This not only enhances pose-controllability but also contributes to more realistic and accurate 3D generations.
Under the hood, SV3D leverages its multi-view consistency to optimize 3D Neural Radiance Fields (NeRF) and mesh representations. This optimization is further enhanced by a masked score distillation sampling loss, designed to improve 3D quality in regions not visible in the predicted views. To tackle the issue of baked-in lighting, SV3D employs a disentangled illumination model that is jointly optimized with 3D shape and texture.
The potential applications for SV3D are vast. From virtual and augmented reality experiences to product visualization and beyond, this technology has the power to transform various industries. As Stability AI continues to push the envelope in AI-driven 3D generation, it will be exciting to see how creators and businesses harness the capabilities of SV3D.
Stable Video 3D is now available for commercial use with a Stability AI Membership. For non-commercial purposes, the model weights can be downloaded from Hugging Face, and the research paper is available here if you'd like to dive into the technical details.
With the release of Stable Video 3D, Stability AI has once again demonstrated its commitment to advancing the field of AI and empowering creators with cutting-edge tools. As the company continues to innovate, it's clear that the future of 3D generation is in capable hands.