3D Photos on Facebook and Substance Painter

Today I went tinkering with Substance Painter and the new 3D photos on Facebook. Turns out the feature is viewable, but not uploadable yet in France.

However, if you "like" the official Facebook 360 page, and restart your phone Facebook app, that feature will be enabled, as a "beta" before it becomes official.

I got it working, and it was quite fun! Here is my original Facebook post, using a sample scene provided with Substance Painter.

1. I had to write an openGL shader extension to output depth in a usable range. The code is pretty trivial, I am rerouting a function of gl_FragColor.z to _outputs.emissiveColor.xyz.

Here is my code snippet:

// change distance curve to something visually readable

float dist = pow(gl_FragCoord.z, 256.0);

// add contrast

dist = dist * 2.0 - 0.5;

// feed to the emission component of the pbr shader

outputs_.emissiveColor = vec3(dist);

2. Save screenshots with the color and the depth. There is no tool for that AFAICT, so I simply used my OS screenshot tool. The depth is quantized to 8 bits, but it's good enough for this.

3. Then, off to Gimp (photoshop, what not) to play with the depth contrast if needed and to crop it to 1200x800, or 800x1200 for Facebook to accept it. Layer the depth and color so they match. Export them separately. I suspect having different resolutions might in fact work, but I didn't test it.

4. Save to ICloud (There is definitely an android way too, but I used an IphoneX).

5. Import to your Iphone photos by saving it from ICloud.

6. In "Depth Cam", which I found available on the app store, load color and depth, then save photo.

7. Reload the combined photo in Depth Cam, and turn on "Portrait" mode. Save again to your Camera Roll, as "Facebook 3d Photo".

8. Upload from the Iphone facebook app. You need to specify you want to upload a 3d Photo, which will restrict the choice of valid photos to select from, to just "Portraits". Your depth images should be in there.

Again, if you are not in a region of the world with access to the early release of the 3d photos, you need to enter into the beta program. Or wait a couple of weeks, hopefully!

Finally, yes, there are artifacts. The extreme angles that expose what's behind the statue show some white areas.

This is due to the mesh, procedurally generated by the Facebook viewer. I think the issue is the discontinuity in depth at the edge of the statue, which is perhaps higher than most depths generated by photos.  Likely, this will be fixed soon by smart people in their group, like Ocean Quigley.

Finally... thanks Ocean and Daxx for the initial pointers!