Close

AI makes 3D images more realistic than ever: RayGauss innovation wins award at WACV 2025

Awards and distinctions Digital transformation Research Decoding
Published on 17 March 2025
Artificial intelligence and computer vision are making great strides, particularly in the ability to generate ultra-realistic 3D images from multiple points of view. The project RayGauss, led by Hugo Blanc, doctoral student, Jean-Emmanuel Deschaud, research fellow and AI Fellow within the excellence project PR[AI]RIE – Paris School of AI, and Alexis Paljic, professor, all three affiliated with the Center for Robotics (CAOR) of Mines Paris – PSL, illustrates this major advance. The project presents an innovative approach to generating photorealistic images from a set of images of a scene. Their method combines several advanced techniques to improve the quality of the rendering, while maintaining reduced computing times. Their work was recognized with the prestigious Best Paper Award – Algorithms at the international conference WACV 2025 (IEEE/CVF Winter Conference on Applications of Computer Vision), a distinction awarded from among nearly 1000 articles and recognizing a decisive contribution to computer vision.

Visual comparison between a 3D Gaussian filter and RayGauss

What is novel view synthesis and why is it important?

Novel view synthesis is a technique that allows the generation of images from angles that were not originally captured by a camera. This technology is essential for many applications, ranging from special effects in cinema to 3D mapping for autonomous navigation and virtual reality.

Recent approaches often rely on Neural Radiance Fields (NeRF), deep learning models that reconstruct a scene by simulating how light interacts with the environment. However, conventional NeRFs have limitations: they require very long computing times and can generate visual artifacts, impairing the quality of the final rendering.

The RayGauss innovation: a hybrid approach

RayGauss offers a new way of creating realistic 3D images using innovative rendering techniques. Rather than relying on ray tracing on a conventional textured mesh, RayGauss takes a different approach: it uses ray tracing with Gaussian-type primitives, which are mathematical ellipsoid shapes, to model both the density of matter and light. In concrete terms, instead of considering a scene as a simple set of surfaces, this method represents matter in the form of diffuse halos, which makes it possible to better capture the way in which light propagates and interacts with objects.

The approach is based on two major advances:

  1. More accurate modeling of light: The team has developed a model that breaks down the light emitted using ellipsoids associated with several mathematical components called Spherical Gaussians and Spherical Harmonics. These tools make it possible to better represent variations in color and light intensity according to the angle of observation, thus making the images more realistic.
  2. An optimized algorithm for accelerated rendering in successive layers: Instead of calculating the path of each light ray separately, which is time-consuming and computationally expensive, RayGauss segments the ray casting into several successive layers (slabs). It also uses a data structure called Bounding Volume Hierarchy (BVH), which allows it to quickly ignore the non-visible parts of a scene and focus only on the relevant areas. As a result, RayGauss is able to generate real-time images at a speed of 25 frames per second (FPS) on complex scenes from the Blender software, while offering image quality superior to current methods.

Volumetric rendering algorithm.

Superior performance and major potential impact

One of RayGauss’s great strengths is its balance between quality and efficiency. Unlike other techniques that require several hours of calculation, this method achieves exceptional image quality while maintaining a reasonable training time and real-time performance that can be exploited in concrete applications.

These advances open the way to a variety of uses, particularly in:

  • Video games and virtual reality, where realistic renderings must be calculated in real time.
  • Automotive and robotics, for a better perception of the environment in autonomous vehicles.
  • Archeology and heritage, by enabling the virtual reconstruction of historical sites from photographs.

International recognition for Mines Paris – PSL

The award obtained at WACV 2025 is a major recognition of the work accomplished by the Mines Paris – PSL team. This success illustrates the excellence of French research in artificial intelligence and computer vision, and highlights the role of the Robotics Center (CAOR) and the PR[AI]RIE-PSAI institute in scientific innovation.

The team has made its code publicly available on GitHub, thus promoting the sharing and continuous improvement of this promising technology. RayGauss marks a decisive breakthrough in the field of image synthesis and could well redefine the standards of photorealistic rendering in the years to come.

Comparaisons visuelles : ensemble de données Dex-NeRF

 

To go further: