In V-Ray 3.x we introduced our own denoising solution. It allows the user to render an image up to a certain point and then let V-Ray denoise it based on the info it has. This process runs very well on GPUs. One thing we mention in our Guide to GPU is that GPUs are excellent at massively parallel tasks. And denoising is one of those tasks. With GPUs, we get about a 20x speed boost, and the process can finish in just a few seconds.
But it could be faster. What if, instead of solving the denoising problem independently for each image, it could reference back to past denoising solutions to solve the problem faster?
Using neural networks to help denoising
Using this previously “learned” data is the basis of machine learning. In V-Ray, it can use the data learned during the light cache pass to help solve a variety of rendering problems much faster. For example, the Adaptive Sampler, Adaptive Lights and new Adaptive Dome Light all use this concept. But what if V-Ray could learn from other renderings too, not just the one it’s working on?
Right now, there’s a lot of buzz surrounding the topics of Deep Learning and Deep Neural Networks. (Really they’re the same thing). How deep the neural network is simply refers to the number of layers that the network contains. The idea is to build a computational network that learns how to solve specific problems, either from provided solutions to the problem, or by learning from it’s own tests. Once the network better understands how to solve a problem, such as denoising, it can solve it much faster.
Imagine if you didn’t know that 5+5=10, and you had to count on your fingers each time, it would be a much slower solution. But because you already know the answer, you can skip counting on your fingers, making it much faster.
In theory, by feeding the neural network thousands of different noisy renders along with the clean final versions, it could learn how to solve the noise problem using this image data, and then apply the solution to other cases.
That’s exactly what NVIDIA has introduced with their OptiX AI-accelerated denoiser. They built a neural network using thousands of images rendered in Iray, and this learned data can now be applied to other ray traced images. We decided to experiment with how this learned data might benefit V-Ray.
It’s all about speed
What’s the advantage of NVIDIA’s OptiX denoiser over V-Ray’s denoiser? While the V-Ray denoiser is very fast and can denoise an image in seconds on a GPU, the OptiX solution can denoise a render in real-time. But let’s keep in mind that a denoised image is never quite going to be accurate. By definition, it gives you the best guess for what it thinks the final image should be. At the same time, accuracy may not be the most important thing. If you can get a workable noise-free image in real-time, it could have an impact on your workflow, especially during lighting and look development.
How the NVIDIA OptiX denoiser works in V-Ray
It’s possible to use the learned data with V-Ray, even though the information was gathered using Iray renders. We could even retrain the network using V-Ray renders.
The more “real” information the denoiser knows about the image, as opposed to guessing, the better it can do its job. For example, let’s look at edge detection. Because edges are generally detected based on high contrast between neighboring pixels, a noisy image may not have enough information to detect the edges well. When you render a diffuse pass and a normals pass in V-Ray, it gathers enough information about a scene to determine where the edges are.
With the combination of learned data and render elements, the OptiX denoiser can give you a very good prediction of the final image, even with only a few samples. While this type of denoising will work on GPUs or CPUs, the biggest benefit for the user is when working interactively.
Some example results
In this example we are looking at a fairly complex scene with a lot of Global Illumination. We used both the diffuse and normal pass as part of the denoiser. We took snapshots during the render process to show both the original render and the denoised one.