Cars 3 and deferred texturing
I haven's spoken much about Cars 3 in the last few years. This time I will, because I think the film deserves attention, as it looks incredible, and it forced us to solve a lot of interesting problems. One of them is deferred texturing.
I am going to go back to the initial issue in Finding Dory, the show when we switched renderer to RIS. That gave us fantastic new potential for render quality, but brought with it the end of our classic Renderman Shading Language, along with all our traditional shading tools, including one, we called Picasso.
Using math figured out and later patented by my friend David Ryu in 2008, Picasso allowed a user in Maya to build a rig of cameras, and project and layer painted textures, onto arbitrary geometry, using poses (e.g. T-pose, exploded prims, or open mouth, so hard-to-reach parts could be painted with ease). The math is pretty specific about how much of each layer to see based on camera depth, incident normals and a number of parameters. At render time, when the objects deformed, the paint and its layering was calculated using reference points and normals, so the paint wouldn't "swim".
The origin of the name is no longer clear, but I think that Picasso should draw its name from the Spanish painter who, for some time in his life, adopted cubist techniques, where paint on the surface appeared to be coming from projections other than the point of view itself.
After being built on hierarchical Renderman attributes, the later incarnations of Picasso I wrote on Monsters University, progressively relied more on dynamically generated RSL co-shaders that looked at the camera setups and layered the paint appropriately.
So, why did we have to lose all this? It's not just about RSL. In RIS, the whole concept of co-shaders (hierarchically bound objects with an API that could be fetched by name) went away to help improve the new Renderman's performance. While those changes were needed, they had serious consequences on our traditional workflows: if we want to apply a single layer of paint on top of 100 object with 30 different shaders, we'd have to go into each one of those objects and splice some texture and projection nodes into their shading networks, and hope we don't have to change our minds or make mistakes, or else do it all again.
Fortunately, we didn't only switch to RIS, we also switched to Foundry's Katana, where deferred proceduralism is a solved problem, and at the core of the tool. In our shading networks we started adding nodes, named "Receivers", which only have a channel name (to tune in with a paint broadcaster) and a default RGBA, for when no paint is found at all.
We wrote a Katana "SuperTool" that looks somewhat analogous to the built-in "Gaffer". Instead of creating lights, it creates cameras and settings, and based on that set up, at render time, generates and wires in the appropriate textures, projections, and blending nodes, purely based on the channel name and their composition order and style.
The nice thing about this is that the shader didn't have to be authored knowing what paint would be available in the future, that part would be figured out later, even on a shot per shot basis. Instead of a texture node, we just put in a Receiver with channel "GraphicMask", or "MetalColor", or "Displacement". This can be extra important for Cars movies, because we use a ton of graphics, which are often finalized and approved long after the assets are shaded and painted. So there, deferred projections solved.
In fact, why even stop at projection painting? Why not make it be able to read paint no matter where it came from? That's why we picked a multi-talented scientist and artist to name this new tool: Leonardo.
Since the introduction of Mari in our pipeline on Monsters University, UDIM-based painting has taken a growing role in our workflow, especially in characters. This is notably later than a large part of the VFX community, and that is in part a consequence of how much Pixar cherishes the ability to make creative choices live, late in the pipeline, while paint sort of bakes it in. Leonardo allows us to switch techniques at any point, to cater for that need. So we added support for UVs, UDIMs and even Disney's Ptex to it - which came in handy with complex meshes that are typically bad to shade in UV space.
We didn't end up stopping there. By the time the show was wrapping, we were able to read point clouds, primvars, openVDB and even lights and interactive space-shapers in Katana, which we call Rods. That came in very useful when we had to drive complex, shared paint such as animated streaks of dirt and damage, shared between characters and environments.
I hope you get to see Cars 3. A lot of hard work and love went into it, and what came out is a surprisingly involving emotional piece, as well as a real eye candy.