https://www.blainebrezina.com/read-me daily 0.75 2019-01-21 https://www.blainebrezina.com/homereel daily 0.75 2019-02-01 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c53b8267817f73679bbe84b/1548990516496/HRLayersMov.gif Home w/ Reel https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c53ba6624a6946784bca251/1548991090324/teleport-space_small.gif Home w/ Reel https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c53b7f271c10b9ce577ec98/1548990462419/UpDown_small.gif Home w/ Reel https://www.blainebrezina.com/masterworks daily 0.75 2019-03-20 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c49ba19b8a0454731e1a10a/5c4c7180bba22374afa911cf/1548515948999/hologramProgressBarCombo.gif VR UI Shaders, MasterWorks https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c49ba19b8a0454731e1a10a/5c4a9df24d7a9c275daa5d1e/1548515948990/teleport-space_1.gif VR UI Shaders, MasterWorks https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c49ba19b8a0454731e1a10a/5c4a9de10ebbe892230c4324/1548515948980/DepthShader_1.gif VR UI Shaders, MasterWorks https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c4c5e9d70a6ad6d2abbf6ff/1548515949029/hologramProgressBarCombo.gif VR UI Shaders, MasterWorks - User Interfaces Left: “Hologram” effect for location selection menu to represent scenes without textures. Right: 3D representation of audio file playback progress. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c4c578c2b6a28b6ba0b926e/1548515949019/ VR UI Shaders, MasterWorks - Scene Change An experiment to focus users on a single object for closer inspection, and to move them between scenes without a sharp “pop”. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c4c571d40ec9a53af364409/1548515949010/teleport-space_1.gif VR UI Shaders, MasterWorks - Teleportation We wanted a teleportation UI that would not obscure the environment except when needed, and would apparate in a non-jarring way. https://www.blainebrezina.com/dither-transparency daily 0.75 2019-03-20 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c4e5035aa4a99273251c608/1548636232982/ZoomIn.gif Dither Transparency - Depth Unlike the usual method of rendering transparent objects in Unity, this shader still writes to the z-buffer, allowing for distance based fading. This way objects do not pop out when at clipping distance and clipping through an object is less jarring. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c4e500b575d1f05fb4832f9/1548636191460/UpDown.gif Dither Transparency - Discrete Levels Rather than having a smooth blend from 0 to 1, the dithering method has discrete steps. Transparency is achieved by providing a “chance” for a pixel to be colored based on a lookup matrix. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c4c7a6c21c67c8d50e72560/5c4e4fd51ae6cfaa21f7d72f/1548636128298/ZoomIn.gif Dither Transparency https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c4c7a6c21c67c8d50e72560/5c4e4fd58a922d08810b19f7/1548636138577/UpDown.gif Dither Transparency https://www.blainebrezina.com/weather-particles daily 0.75 2019-03-20 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c57821c1905f4a9009eb9da/1549648299902/LRAttributes.gif Weather Particles - Variability Using attributes in the PopcornFX editor this spawner can be controlled from code to use one .pkfx package to simulate multiple severities of cloud systems, from a light drizzle to a heavy storm. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c53bafb085229926930aea5/1549648299899/HRLayersMov.gif Weather Particles - Brewing a Storm The clouds use flipbooks of “slices” of a simulation created in Blender. These were layered on top of one another to approximate a volumetric effect. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c578cbaa4222f53ddf5cc13/1549648299905/SunMerc.gif Weather Particles - Other Systems Just two simple billboard textures pushed to emissive color levels. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c4dcbe34fa51a7413b4625d/5c53bad2104c7bd8f776696d/1549648299895/HRLayersMov.gif Weather Particles https://www.blainebrezina.com/about daily 0.75 2019-03-04 https://www.blainebrezina.com/vfx-explorations daily 0.75 2019-02-08 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c5dcad5f9619aa53dce14f8/1549650650726/StylizedWoodShingleRoof.png Explorations in VFX https://www.blainebrezina.com/style-substance daily 0.75 2019-06-25 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c65ff53b208fc777ac90d4d/5c65ff534785d3730c233024/1550188500301/StylizedWoodSiding_roundedCyl.png Style with Substance (Designer) - Wood Siding https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c65ff53b208fc777ac90d4d/5c65ff53e79c708baf1d22b1/1550188509779/StylizedShingles_roundedCyl.png Style with Substance (Designer) - Shingles https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c66066bf9619a89c1e1d8a3/1550190988808/StylizedPanelsInspiration.png Style with Substance (Designer) https://www.blainebrezina.com/old-home daily 0.75 2019-03-21 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e747817f7917adaf570/1550277459090/UpDown_small.gif The Portfolio of 3D Game Artist Matthew Blaine Brezina - Dither Transparency Shader for optimizing transparent rendering https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e61e4966bdcc0d0e4b3/1550277385378/LRAttributes.gif The Portfolio of 3D Game Artist Matthew Blaine Brezina - Weather Particle Systems PopcornFX for volumetric effects on mobile hardware https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e4a9140b74cd183d47b/1550276782211/teleport-space_1.gif The Portfolio of 3D Game Artist Matthew Blaine Brezina - VR UI Shaders Shaders for VR interfaces with a focus on user comfort and performance https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e25652deaaef0d99db9/1550793713148/StylizedShingles_roundedCyl.png The Portfolio of 3D Game Artist Matthew Blaine Brezina - Style with Substance Substance Designer materials to emulate hand-painted aesthetics https://www.blainebrezina.com/home-1 daily 0.75 2019-02-15 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c6751db9b747a5ab67f260c/1550193420036/teleport-space_1.gif Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e747817f7917adaf570/1550277459090/UpDown_small.gif Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina - Dither Transparency Shader for optimizing transparent rendering https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e61e4966bdcc0d0e4b3/1550277385378/LRAttributes.gif Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina - Weather Particle Systems PopcornFX for volumetric effects on mobile hardware https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e4a9140b74cd183d47b/1550276782211/teleport-space_1.gif Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina - VR UI Shaders Shaders for VR interfaces with a focus on user comfort and performance https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5c674e0fe79c70106efc2042/5c674e25652deaaef0d99db9/1550793713148/StylizedShingles_roundedCyl.png Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina - Style with Substance Substance Designer materials to emulate hand-painted aesthetics https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c6751db9b747a5ab67f260a/1550193227578/HRLayersMov.gif Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c6751db9b747a5ab67f2608/1550193134256/StylizedShingles_roundedCyl.png Copy of The Portfolio of 3D Game Artist Matthew Blaine Brezina https://www.blainebrezina.com/home daily 1.0 2020-11-17 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c89ed809b747a58e0557c62/1553209496491/teleport-space_1.gif New Home - UI Shaders - Mobile VR CG Shaders for VR UI with a focus on accessibility, performance and coolness https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c89fa74419202992164c20d/1561506823811/StylizedShingles_roundedCyl.jpg New Home - Style with Substance Materials made in Substance Designer to emulate a hand painted, stylized aesthetic https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c89fac58165f5cb8f5f6dfc/1552546657223/CloudLayer_movie.gif New Home - Weather FX - Mobile AR Particle systems made in PopcornFX https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5fb33b016a3c737902ddde7f/1605581570926/Cube+Map+SSS.png New Home - Skin Rendering Exploration in UE4 of multiple approaches and components of real-time skin rendering https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5c8b1788e79c70d357f01e48/1552619414463/UpDown_1.gif New Home - Dither Transparency CG Shader for optimizing transparent object rendering through a screen space effect https://www.blainebrezina.com/dragon-fight daily 0.75 2019-11-11 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5d12b4cb9a4cdb000123d9d1/1549648299899/HRLayersMov.gif Dragon Fight - The Star Made using sprite sheets, animated lights, and multiple layered particle systems. Besides the main clouds of fuel: the bright ignition point, the sparks, and the tongues of flame that appear when the fire hits all helped to sell the effect. The details count. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5d12b4cb9a4cdb000123d9d4/1549648299902/LRAttributes.gif Dragon Fight - The Supporting Cast Trail particles were important to many of the effects. Sparks, sword swooshes, and especially god rays all required this module of Unity’s particle systems. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5d12b4cb9a4cdb000123d9cc/5d12b4cb9a4cdb000123d9cd/1549648299895/HRLayersMov.gif Dragon Fight https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5d12b4cb9a4cdb000123d9d7/1549648299905/SunMerc.gif Dragon Fight - the Stage To give the game a more cinematic feel we used Unity’s post-processing stack for color grading and bloom. Bloom really helped to make those flame effects feel hot. But we had to be careful to balance the brightness of those emissive materials for player comfort, especially important in VR. https://www.blainebrezina.com/face-rendering daily 0.75 2020-01-13 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5df9cb6e2ef84a2cce137190/1578876943647/specularHomemade.gif Face Rendering 1 - Why Specularity? Besides its ubiquity, focusing on specularity first has the benefit of being platform agnostic. The highest end techniques, namely Screen Space Subsurface Scattering (SSSSS) require the use of separate buffers to render properly. This means that they generally only work in deferred renderers. And if you’re trying to deploy to a mobile device, you’re generally stuck in forward. So by focusing on specularity now we can add SSS later once a platform has been chosen. And if we use a forward renderer, Pre-integrated SSS can still look great, use the same assets and can then be in a single pixel shader, rather than split off into its own screen space effect. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5df9ad9f2ef84a2cce10decf/1578876943642/withSSS.gif Face Rendering 1 - Research First I downloaded The Wikihuman Project’s Digital Emily 2.0 mesh and texture pack. For use in engine I decided to downsample the textures to 4K from their standard 8K for ease and speed of iteration. Next I implemented them in UE4 using Epic’s tutorial for their photorealistic human skin pipeline. After completing this and reading through some of the associated papers I gained a much greater understanding of the techniques involved in skin rendering. After implementing the built-in pipeline I began the process of breaking it down into its component parts to later build in the different approaches that might work better in given situations. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5df9ce5ccda8467a3e7be67b/1578876943645/UE4BuiltInSkinMat.png Face Rendering 1 - Breakdown Click to view larger image https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1476c7f98bf1711b375956/5e15ed5d42505918ddb25568/1578876943607/preint+sphere.png Face Rendering 1 - Pre-integrated falloff Using the scene texture, notice the need for extreme amounts of blur https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1476c7f98bf1711b375956/5e14971d01a8570f0fb35ef1/1578876943603/Scene+Texture+SSS+1.png Face Rendering 1 - Scene Texture Based Pre-integrated SSS using Scene Texture. Requires Translucency which causes sorting problems. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5df9aa26ba6ef66e21ee38ef/5df9acdb2ef84a2cce10cb6a/1578876943639/withSSS.gif Face Rendering 1 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5df9aa26ba6ef66e21ee38ef/5df9acdb2ef84a2cce10cb69/1578876943635/noSSS.gif Face Rendering 1 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e147c8e3305720f1b9c8513/1578876943610/preintegrated_falloff_2D.png Face Rendering 1 - Pre-integrated SSS First up was scene texture sampling. I was advised to look at sampling from a reflection probe, environment map or other similar spatially placed entity. I began by sampling the scene texture. After getting the scene texture I grabbed the pixel normal to allow a fresnel-centric application of a pre-integrated penumbra texture. This texture is a computation of the diffusion of light in a scattering material at the edge of light falloff. More on that here under “Pre-integrated Subsurface Scattering”. However; this version of the technique is limited to light that you are looking at through the skin, not the whole scene. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df69dc85c857dfa96f12/1578876943630/Final+Luminance.png Face Rendering 1 - Apply to Texture https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df684de27a2740bebe49/1578876943627/5+gather+luminances+at+original+test+point.png Face Rendering 1 - Gather Luminances https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df68125da30c420626db/1578876943624/4+compute+luminance+at+intersections.png Face Rendering 1 - Compute Luminance at intersections with geometry https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df68dc85c857dfa96f01/1578876943620/3+radially+symmetric+rays.png Face Rendering 1 - Cast rays https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df6840c89a0354396800/1578876943617/2+Push.png Face Rendering 1 - Flip & Push https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e15df67dc85c857dfa96eff/5e15df679d5972667492da02/1578876943613/1+Normals.png Face Rendering 1 - Surface Normals https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e15f9dfa26eb612a582a30e/1578876943633/EAR-ray.jpg Face Rendering 1 - Proposal 1. Flip normal at pixel and push inside slightly 2. Use thickness map at pixel to determine length of ray 3. Ray march along that entire length for intersections 4. Compute luminance at those intersections 5. Gather luminance based on distance from original point and number of layers passed through (see right, that’s a real ear not a render by the way) 6. Choose color using curvature map and pre-integrated falloff This technique could possibly allow for: lighting of surfaces through others (see ear), and more physically correct scattering by taking into account the density/thickness of the material (which the publicized EA SEED method does not do. https://www.blainebrezina.com/face-rendering-1 daily 0.75 2020-01-13 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1bc00f22ffb72e97fc6029/1578498590162/EAR-ray.jpg Face Rendering 2 - Proposal 1. Flip normal at pixel and push inside slightly 2. Use thickness map at pixel to determine length of ray 3. Ray march along that entire length for intersections 4. Compute luminance at those intersections 5. Gather luminance based on distance from original point and number of layers passed through (see right, that’s a real ear not a render by the way) 6. Choose color using curvature map and pre-integrated falloff This technique could possibly allow for: lighting of surfaces through others (see ear), and more physically correct scattering by taking into account the density/thickness of the material (which the publicized EA SEED method does not do. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc602b/5e1bc00f22ffb72e97fc602e/1576643805932/withSSS.gif Face Rendering 2 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc602b/5e1bc00f22ffb72e97fc602c/1576643805845/noSSS.gif Face Rendering 2 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1bc00f22ffb72e97fc6039/1576651635222/specularHomemade.gif Face Rendering 2 - Why Specularity? Besides its ubiquity, focusing on specularity first has the benefit of being platform agnostic. The highest end techniques, namely Screen Space Subsurface Scattering (SSSSS) require the use of separate buffers to render properly. This means that they generally only work in deferred renderers. And if you’re trying to deploy to a mobile device, you’re generally stuck in forward. So by focusing on specularity now we can add SSS later once a platform has been chosen. And if we use a forward renderer, Pre-integrated SSS can still look great, use the same assets and can then be in a single pixel shader, rather than split off into its own screen space effect. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1bc00f22ffb72e97fc6017/1578880517790/preintegrated_falloff_2D.png Face Rendering 2 - Pre-integrated SSS First up was scene texture sampling. I was advised to look at sampling from a reflection probe, environment map or other similar spatially placed entity. I began by sampling the scene texture. After getting the scene texture I grabbed the pixel normal to allow a fresnel-centric application of a pre-integrated penumbra texture. This texture is a computation of the diffusion of light in a scattering material at the edge of light falloff. More on that here under “Pre-integrated Subsurface Scattering”. However; this version of the technique is limited to light that you are looking at through the skin, not the whole scene. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1c79dce07fd3395106e082/1581297411649/ezgif.com-crop%281%29.gif Face Rendering 2 - Cube-map to SSS Here we can see the SSS technique of sampling the cube-map. Usually used for calculating reflections, the map is captured from a node placed in world-space. From this map we can gather the lighting information behind an object. We can place this node at the head, for the most correct capture of the scene. From there we can approximate the light passing through that point by sampling a blurred cube-map in the opposite direction of the pixel normal. After gathering this sample of color behind the pixel normal, we can use the pre-integrated SSS LUT to control the effect the sample has based on how brightly lit that mesh point is on the outward side. As you can see this current iteration of the material does not take into account dynamic lights; they can only affect the SSS if reflected off another object into the cube-map. This technique also does not take into account the thickness at the mesh point behind that pixel normal. This causes significant light bleeding as seen in the eyelids especially. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1bc00f22ffb72e97fc6033/1576651380543/withSSS.gif Face Rendering 2 - Research First I downloaded The Wikihuman Project’s Digital Emily 2.0 mesh and texture pack. For use in engine I decided to downsample the textures to 4K from their standard 8K for ease and speed of iteration. Next I implemented them in UE4 using Epic’s tutorial for their photorealistic human skin pipeline. After completing this and reading through some of the associated papers I gained a much greater understanding of the techniques involved in skin rendering. After implementing the built-in pipeline I began the process of breaking it down into its component parts to later build in the different approaches that might work better in given situations. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e1bc00f22ffb72e97fc6036/1576653403391/UE4BuiltInSkinMat.png Face Rendering 2 - Breakdown Click to view larger image https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc600f/5e1c85084ea7cf33bb7feed5/1581297411645/Cube+Map+SSS.png Face Rendering 2 - Cube-Map SSS Material Graph Here is the Cube-map technique applied to a basis similar to the built-in UE4 SSS material from part 1 of this series. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc600f/5e1c824e12be5b2164ed1974/1581297411640/ezgif.com-crop%281%29.gif Face Rendering 2 - Cube-Map Sample SSS Samples a cube map in the opposite direction of a pixel normal. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc600f/5e1bc00f22ffb72e97fc6012/1578926475026/preint+sphere.png Face Rendering 2 - Pre-integrated falloff Using the scene texture, notice the need for extreme amounts of blur https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc600f/5e1bc00f22ffb72e97fc6010/1578926488706/Scene+Texture+SSS+1.png Face Rendering 2 - Scene Texture Based Pre-integrated SSS using Scene Texture. Requires Translucency which causes sorting problems. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc6024/1578492603180/Final+Luminance.png Face Rendering 2 - Copy of Apply to Texture https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc6022/1578492062832/5+gather+luminances+at+original+test+point.png Face Rendering 2 - Copy of Gather Luminances https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc6020/1578492041944/4+compute+luminance+at+intersections.png Face Rendering 2 - Copy of Compute Luminance at intersections with geometry https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc601e/1578491938531/3+radially+symmetric+rays.png Face Rendering 2 - Copy of Cast rays https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc601c/1578491908281/2+Push.png Face Rendering 2 - Copy of Flip & Push https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e1bc00f22ffb72e97fc6019/5e1bc00f22ffb72e97fc601a/1578491848557/1+Normals.png Face Rendering 2 - Copy of Surface Normals https://www.blainebrezina.com/face-rendering-dev daily 0.75 2020-11-10 https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430ad1/1578926192447/ezgif.com-crop%281%29.gif UE4 Face Rendering - Cube-map to SSS Here we can see the SSS technique of sampling the cube-map. Usually used for calculating reflections, the map is captured from a node placed in world-space. From this map we can gather the lighting information behind an object. We can place this node at the head, for the most correct capture of the scene. From there we can approximate the light passing through that point by sampling a blurred cube-map in the opposite direction of the pixel normal. After gathering this sample of color behind the pixel normal, we can use the pre-integrated SSS LUT to control the effect the sample has based on how brightly lit that mesh point is on the outward side. As you can see this current iteration of the material does not take into account dynamic lights; they can only affect the SSS if reflected off another object into the cube-map. This technique also does not take into account the thickness at the mesh point behind that pixel normal. This causes significant light bleeding as seen in the eyelids especially. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430adc/1581386875084/Final+Luminance.png UE4 Face Rendering - Apply to Texture https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430ad8/1581386853495/5+gather+luminances+at+original+test+point.png UE4 Face Rendering - Gather Luminances https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430ade/1581386842587/4+compute+luminance+at+intersections.png UE4 Face Rendering - Compute Luminance at intersections with geometry https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430ad6/1581383709308/3+radially+symmetric+rays.png UE4 Face Rendering - Cast Rays https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430ad4/1581383693700/2+Push.png UE4 Face Rendering - Flip & Push https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ad3/5e40af03c0b9914844430ada/1581383679360/1+Normals.png UE4 Face Rendering - Surface Normals https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ae5/5e40af03c0b9914844430ae8/1576643805932/withSSS.gif UE4 Face Rendering https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ae5/5e40af03c0b9914844430ae6/1576643805845/noSSS.gif UE4 Face Rendering https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40bcf32ed8a65672654421/1581330017881/Tracing2.gif UE4 Face Rendering - Ray marching Ray marching tests how long a ray travels through a medium that is contained within a domain. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ac2/5e40af03c0b9914844430ac9/1581386902289/Cube+Map+SSS.png UE4 Face Rendering - Cube-Map SSS Material Graph Here is the Cube-map technique applied to a basis similar to the built-in UE4 SSS material from part 1 of this series. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ac2/5e40af03c0b9914844430ac7/1581386895474/ezgif.com-crop%281%29.gif UE4 Face Rendering - Cube-Map Sample SSS Samples a cube map in the opposite direction of a pixel normal. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ac2/5e40af03c0b9914844430ac5/1581386913606/preint+sphere.png UE4 Face Rendering - Pre-integrated falloff Using the scene texture, notice the need for extreme amounts of blur https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/5e40af03c0b9914844430ac2/5e40af03c0b9914844430ac3/1581386936611/Scene+Texture+SSS+1.png UE4 Face Rendering - Scene Texture Based Pre-integrated SSS using Scene Texture. Requires Translucency which causes sorting problems. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e4214318185f139c7f96e44/1581388992266/emily_volume_texture.png UE4 Face Rendering - The wrinkles… This technique requires a volume texture of the head similar to this image that maps to the volume of the SDF. Although theoretically possible to recreate a volume texture procedurally the steps involved require closed geometry which is no easy task with our problem set. Additionally, it is difficult to calculate light at a point in space in UE4’s material editor as many of the lighting calculations are obfuscated behind what are called shading models. This structure has allowed Epic Games to develop a robust artist facing material editor at the cost of extensibility. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430ace/1578880517790/preintegrated_falloff_2D.png UE4 Face Rendering - Pre-integrated SSS First up was scene texture sampling. I was advised to look at sampling from a reflection probe, environment map or other similar spatially placed entity. I began by sampling the scene texture. After getting the scene texture I grabbed the pixel normal to allow a fresnel-centric application of a pre-integrated penumbra texture. This texture is a computation of the diffusion of light in a scattering material at the edge of light falloff. More on that here under “Pre-integrated Subsurface Scattering”. However; this version of the technique is limited to light that you are looking at through the skin, not the whole scene. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430ae3/1578498590162/EAR-ray.jpg UE4 Face Rendering - Proposal 1. Flip normal at pixel and push inside slightly 2. Use thickness map at pixel to determine length of ray 3. Ray march along that entire length for intersections 4. Compute luminance at those intersections 5. Gather luminance based on distance from original point and number of layers passed through (see right, that’s a real ear not a render by the way) 6. Choose color using curvature map and pre-integrated falloff This technique could possibly allow for: lighting of surfaces through others (see ear), and more physically correct scattering by taking into account the density/thickness of the material (which the publicized EA SEED method does not do. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430af3/1576651635222/specularHomemade.gif UE4 Face Rendering - Why Specularity? Besides its ubiquity, focusing on specularity first has the benefit of being platform agnostic. The highest end techniques, namely Screen Space Subsurface Scattering (SSSSS) require the use of separate buffers to render properly. This means that they generally only work in deferred renderers. And if you’re trying to deploy to a mobile device, you’re generally stuck in forward. So by focusing on specularity now we can add SSS later once a platform has been chosen. And if we use a forward renderer, Pre-integrated SSS can still look great, use the same assets and can then be in a single pixel shader, rather than split off into its own screen space effect. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e412e6fc0b991484456032f/1581389186756/pixelrender.gif UE4 Face Rendering - Ray tracing The common approach to ray tracing is to shoot a ray through each pixel of a frustum and if it hits something to get the color of the object at that point. For shadowing a ray is then sent from that point to each light which might be lighting that pixel. If something is in the way of that ray’s path to the light we know that that pixel will be shadowed and can affect the base color of the original hit accordingly. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430aed/1576651380543/withSSS.gif UE4 Face Rendering - Research First I downloaded The Wikihuman Project’s Digital Emily 2.0 mesh and texture pack. For use in engine I decided to downsample the textures to 4K from their standard 8K for ease and speed of iteration. Next I implemented them in UE4 using Epic’s tutorial for their photorealistic human skin pipeline. After completing this and reading through some of the associated papers I gained a much greater understanding of the techniques involved in skin rendering. After implementing the built-in pipeline I began the process of breaking it down into its component parts to later build in the different approaches that might work better in given situations. https://static1.squarespace.com/static/5ac65f81f8370a74ba2a10e9/t/5e40af03c0b9914844430af0/1576653403391/UE4BuiltInSkinMat.png UE4 Face Rendering - Breakdown Click to view larger image