UberShader

[LaidlawFX]

Hi all,

So for my thesis I'm shading my junkyard almost completely in Houdini Vops, and dabbling with some Renderman in Houdini. But since I'm pretty much 90% in Houdini I figured I would develop my own UberShader so I wouldn't have to keep on creating and modifying shaders for a 100 pieces of Junk. I choose hip file size over time management, though you can easily rip the stuff out if your scene gets as big as mine. I'll start a whip on the actual thesis after I present my final in a few weeks when I have some time, but I figured I would share and get some feedback on the shader I'm working with. It is a compilation of a bunch of Houdini shaders into one, with controlled on off switches, and good image planes/render passes. Also it locks out everything so you don't play with things that won't have an effect when testing the components of the shader. And you can roll over the on off switches and it tells you what the VEX variable for your image plane out put. Also SESI highlights there output nodes in green inside the VOP, for instance my diffuseExport doesn't have an output rollover reminder. I also have a Nuke script where the image planes / render passes are all combined back into the original beauty and then have z-depth / Pz, blooming, and post effect motion blur without vectors. The Nuke script is good for learning especially if you haven't done if before.

I hope to add some better slim/shaderman esq qualities to this shader, especially SSS, and point cloud based shading. I think in reality I just need to shade in the vop and not try and work at the interface level like slim and shaderman, but the simple shaders will carry you far especialy when you realize procedural shader only get so far and you need to texture things. Still working on that one myself.

I actually have to thank everyone from the forums for getting me this far along as I forum(dumpster) dived finding a bunch of gems. So thanks.

The attached files contain

UberShader.hip - a hip only with the shader

UberShader_BRL.otl - an otl version of it

Composite.rar - a zipped nuke script of the composited image planes

CokCanSingle080.part1.rar - The .exr the nuke script is based on

CokeCanSingle080.part2.rar - The .exr the nuke script is based on

CokeCan_Beauty.png - is the render and composite version. I apologise to Will I didn't work in Linear Light Flow and my image needs to be corrected with an inverse Gamma of .4545 to look properly. Will hopefully won't be seeing this post anytime soon, so that is why I am posting this and not correcting it. My apologies for the washed out nature of it, dowload it and correct it in mplay.

UberShader.hip

UberShader_BRL.otl

Composite.rar

CokeCanSingle080.part1.rar

CokeCanSingle080.part2.rar

[CeeGee]

Thanks.

[ykcosmo]

Looks nice.. Hope you can release starbucks soon -.-!

[LaidlawFX]

Updated shader... not tested

Now with Subsurface scattering... simpleSSS or multipleSSS with Point Cloud option. Make sure you create a point cloud with a SSS Scatter node for your geometry in order for the point cloud to work. Ripped from the 2 skin node types.

I wanna see if you can use this point cloud cache for multiple uses... like using my irradiance cache from my Ambient Occlusion instead

There is also a mantra node in the hip file with the different types of images planes in it, good for reference...

UberShader_BRL_v2.otl

UberShader_v2.hip

[LaidlawFX]

More info....

I just wanted to leave a heads up on using the shader. Turning everything on will make this a slow shader to render. The number one reason is irradiance, the number two is occlusion, the number three is bluring the reflection. To test these like all parts of the shader is to uncheck everything, shut those components off, and render with just the one thing to make sure the effect is worth it. This is a major thing all pre-built shaders need, and I would recommend installing the simple switches in any shader you create. Also since this is a fat shader 1mb file size along with long render times it would make this seem like a trash shader just so you know, all the options on aren't allways that good.

For alternative methods to my shader for Irradiance and Occlusion

For Irradiance; There are 7 combo's of different ways to do Irradiance / indirect lighting / color bleed in Houdini that I know of from the, cmiVFX Shading Tutorial, highly recommended.

1. (VEX GI Shader and Light Template) Add a VEX Global Illumination Shader at the Shop Level, turn on Full Irradiance; Add a Light Template at the Object Level, link it to the VEX Global Illumination Shader in the Light Shader Slot - for noise 1 way is to turn up number of samples in the VEX shader *Note Mantra always works faster when you deal with an enclosed area for irradiance gathering, a.k.a. a room or limiting the max distance the ray should bounce so anything but -1 which is infinity

2. (PBR Rendering) Turn on Physical Base Rendering in the Mantra Node under the render tab, turn up the # of Bounces on PBR Diffuse, - for Noise turn up the Mantra node samples from 3 by 3

3. (Photon Mapping #1 using PBR Rendering) Turn on Photon Map Generation in the Mantra Node under the render tab,, create your save directory for your .pmap,

For visual info under the statistics tab Turn on the Verbose Lebvel to 5, Check Alfred Style Progress, *Note Constant shaders do not receive photons,

To verify Photon Map create a geometry node at the OBJ level, at the SOP level use the file node to open your saved .pmap, and show points,

To use this .pmap turn on the Physical Based Render On in the Mantra Node, and link your .pmap to the secondary GI Cache

To speed this process up, you can create two mantra nodes, one that does the photon map generation and link into the second one that renders the picture *Note Pretty much if you don't have to, you do not want to generate photon maps for every frame, so make sure your not doing this, it all depends on the movement of your camera

4. (Photon Mapping #2 using VEX GI Shader and Light Template) Generate your photon map as stated above, Add a VEX Global Illumination Shader at the Shop Level, under the second tab link your .pmap to the photon Map tab; Add a Light Template at the Object Level, link it to the VEX Global Illumination Shader in the Light Shader Slot *Note not necessarily faster....but the more you know.... GI Joe!!!

5. (Shader Based - The method in my shader) Go into any shader at the Vop/VEX Level and just before the final color Output insert an Add node and link that with a Irradiance node. *Note, This is added on top because it is the additional bounces of indirect light on top of the shader that is already built. Also this is the best per object GI method, I would say the only, but it depends on how creative you want to be or what problem you need to have solved.

6. (Image Based Lighting #1) Create and Environment Light at the OBJ level, go to the area light option, check use Area Map and assign one *Note Mantra renders faster with .rat as opposed to any other image file format, a .rat is native to Mantra, and it is of the type of a maya mip map, or a .tex, I like importing my images into cops and converting them that way.

7. (Image Based Lighting #2 using VEX GI Shader and Light Template) Add a VEX Global Illumination Shader at the Shop Level, turn on Full Irradiance, Link a texture map; Add a Light Template at the Object Level, link it to the VEX Global Illumination Shader in the Light Shader Slot

....(Image Based Lighting #3) this a cheat more than anything, this is something I do to get to my goals faster, I just create a sphere and place my image on it, and make sure everything has reflections on with my ray limits high enough.... It doesn't work too badly especially if your doing reflection cards. It all just depends on what your trying to cheat.

For Occlusion,

Use the mantra nodes irradiance tab, once the info is cached it saves a hell of a lot of time. I had a bit of trouble getting it to work on 1080 plates with some buzzing, but most things smaller it works perfectly. A 1080HD still frame at 300 rays per pixel that took twenty minutes on a complex environment dropped down to 2 minutes once the appropriate info was cached, a.k.a. you still need to do the ray bounces at least once, but to do 720 frames after that simply amazing. *Note make sure you completely shut this off when you stop playing with it. I got the error a bunch of time, when I ran occlusion it would come up black since .pmap wasn't linked, or there wasn't point data for the appropriate frames. You'll understand if you view the .pmap with the file node. The one thing that does stink/is good is that this is a camera directional system where renderman will near automatically do omni-directional with it's point clouds.

As for slim like shading... just don't make an interface when your creating a shader then you have slim

I tested the sub surfaces scatter, it did not work 100% perfect all the time, but it did work most of the time. I had too many variables on my thesis project and not enough time to test between the controlled group and the barely controled group. The biggest dissapoint though was the shader ID system. It worked beautifully in the controlled can group, but horribly wrong and random on my thesis. It's pretty easy to rip the parts outs you don't want from this shader so I would go ahead and do that.

I'm going make a wip today on my thesis, after taking a month sanity sabbatical, for thos who want to reference out what my uncontrolled group was facing.

[LaidlawFX]

Oh here is what I tested my shader on.

Notes:

The sliding texture weren't locked down in the correct object space, the glass wasn't done through my shader(the lovely blackness) nor the water( my complements to SESI), and they all were just slapped down, with no finessing.

[LaidlawFX]

I attached a texture map channel to the occlusion tab... for those crazy bump people/people who like to save time through baking...

UberShader_v3.otl

[LaidlawFX]

Shader update:

I added the basic ogl component that comes with most of the houdini shaders

The UV or Cubic based texture mapping is on the displacement tab, so that all uv and texture components are ganged together since the displacement needs to run first. Also I did it to get rid of pointless options to save on node overhead.

All components have texture map layers.

There are a few error calls on the shader parameters, even though it renders fine. I tracked them down, but I couldn't figure out how to remove them, they seem to have disappeared

once I created a digital asset of it, figures.

The shader ID system works again, I had it glitch out for a while, while I was modifying it.

There is also a gamma shift option for the color layer, for linear light flow, nothing fancy but it gets the job done.

Ubershader_BRL_v4.otl

[mad cz math]

hi LaidlawFX,

One thing about your shader i really like is that your VOP network is very clear because u box them with color box and make clear title . Its easy for me to read because i am playing with the shader using VOP too. Thank you.

[LaidlawFX]

hi LaidlawFX,

One thing about your shader i really like is that your VOP network is very clear because u box them with color box and make clear title . Its easy for me to read because i am playing with the shader using VOP too. Thank you.

The funny thing is this time I didn't netbox everything before I loaded it up here this time, but I did pull everything out of the subnet if loops, like the spec, reflection, and displacement, so that everything was visible outside of the multi-tex node, but I ended up breaking that part too, from last time.

Your welcome and thanks

[LaidlawFX]

Starbucks

Re-added - Sub Surface Scattering, and it works!

Added - Sheen

Replaced - The standard sesi reflection shader with those of a glass shader. So you can also do glass and refraction. Optional concept to insert IOR based on two sided object or single sided. Added the attenuation option for the glass, too.

Fixed random little things and added the net boxes back in to organize it.

The Node references it's parts in the vop network, so all shop versions are instances saving the large over head of disk space using the shader multiple times.

I'm testing it on Jeremy Birn's lighting challenge, so far so good. Wishing I had a faster machine to test higher quality renders though.

DELETED UberShader_v5_BRL - It doesn't taste as good as it looks

Edited July 8, 2010 by LaidlawFX

[horizon1231]

v5.otl is broken,I have problem with it.

[LaidlawFX]

v5.otl is broken,I have problem with it.

Whats your exact brokeness? The only thing I know off hand that may be glitchy is that the first time you apply the node it the scene, it creates the vex network first then so therefore it won't allow you to use the first instance of the shader. The second instance of the shader works fine for me in my scene file.

[horizon1231]

Whats your exact brokeness? The only thing I know off hand that may be glitchy is that the first time you apply the node it the scene, it creates the vex network first then so therefore it won't allow you to use the first instance of the shader. The second instance of the shader works fine for me in my scene file.

below

[LaidlawFX]

below

Those are calls for the displacement nodes in the displacment shader to the surface shader in the VEX network. It says it at the top of the error readout that it's missing that node. I think the .otl does not like sharing the vex network nodes into digital assets. It ended up screwing up my default settings for my vex surface and displacement nodes. Apparently I ain't good with sharing. I'm gona upload a hip file just so everything is up here, and you can copy and paste it into your scene file until i can throw this thing against the rocks some more.

UberShader_v6_BRL.hip

Make sure you DELETE V5... It rewrote my vex surface and displacement default settings, so my ubershader nodes would perpetually haunt people. lol, that would be a good nano virus, one that screws up default houdini settings I'm sure sesi and users would love me for that. Thanks for noticing it, lol. Once the .otl is out it seems to work fine.

Mostly all those calls transfer info from the displacement shader to the Surface shader since the displacement shader runs first it was the only way I could gang the uv and cubic space options together. The g's are some random naming convention houdini uses upon the creation of an unknown call, that I haven't been able to figure out where they randomly generate from.

I don't have access to the labs anymore so it's hard to test on multiple machines, os, and projects concurrently.

Also added in a switch to negate the normals on the lambert for paper like stuff, not sure if it will be to helpful yet.

[papicrunch]

hello,

thank for this great stuff LaidlawFX. I'm learning a lot about shading network with your UbberShader !

papi

[LaidlawFX]

hello,

thank for this great stuff LaidlawFX. I'm learning a lot about shading network with your UbberShader !

papi

Your welcome, hopefully it saves a significant percentage of the time of what it took me to make.

Here is some more stuff that's hopefully helps.

The interface is completely designed from inside the shader network. I had to learn that on my own over several days, this should cut the learning curve down. If you look at how the the output node list everything all the parameters order except for the properties tab was designed there. That is why the Base Color is first on the list and you look at the shader interface and that is the first thing you see on the list(Under the Shader ID tab), then you can notice the little Brackets and red Xs those define folders. That way when you update it, promote material parameters it stays within it structure really robustly as oppose to promoting the materials at the interface level. The reason the properties tab hold true is because it is special and does not have any sub-networks outputs, so when you re-promote anything it locks it in. The different control icons like switches, toggles, and file directory are all created and designed from the parameter nodes in yellow. Look at say one control icon you wanna replicate go fine the control parameter in the vop and you can see the icons that are unlocked and therefore how the interface control is engaged.

If you work like that in the future all your digital assets can get pretty robust.

The Sesi coloration of nodes goes as;

Yellow = Parameter at interface level

Black = default constant

Green = Export Parameter, You can create any # you want. Just make sure you do not call it for one shader and have a simple default for the other, or else your render time will slow down significantly.

When there are three nodes right on top of each other,

Red = red channel

Green = green channel

Blue =blue channel

Helfpul for say ripping the red channel out for your displacement, or depending on some work environments the company might have a system to store, displacement, specular/reflection/displacement mask

Blue on the cubic shader parameters just signifies that I slashed and hacked itriix cubic shader. I'm not that smart, but slash and hack is def the way to go.

The reason the environment map parameteters are not ganaged together with everything else is, because the texture space is some special version of a vertical cross pattern. It must be designed for efficiency sake, because I can't figure it out. Hopefully I can learn how the .rat converter works in the prototype shader and use that same logic for the environment map.

Also parameters interface controls are locked out based on there parameter names {Dog = 0} So if the toggle for dog parameter is unchecked you can not use this shader (1 means it is checked, just double check you are using a switch node as opposed to a two way switch the defaults are backwards). In the end it's idiot proofing so you don't play with stuff that is not turned on. The MIA shader in Mental Ray, I spent hours with because of that stupid concept. I do not know if just disabling it does not make it run at render time, so that is one reason ubershader may be bad. Then again you usually just use them if you're in a hurry in production.

If you want you can pull up two vop screens in houdini and compare my crap to sesi stuff to further help. I'm still looking at the prototype shader to figure that stuff out, code and linguistics are not my strong suit, spatial is so thanks god for houdini for having a wonderful visual interface. lol, I'm not sure I would have liked prism.

The reason my stuff is all spread out is so that the node lines don't overlap when your playing with it so it doesn't cross connect(Not necessarily because I was being nice and wanted to share. It's funny how things work out). I love that feature, plus the wiggle, but all good things come with a price. Also the for each loops and if loops run a lot faster than some of the stuff I have because I ripped it open to see its guts. You can get a good idea of the efficiency amount if you look at the output statistics on full, or you bring the desire nodes into sops(you can't do this with all) you can test that with the windows - performance monitor, when you lock and unluck stuff. I found out copy stamps loops are inefficient to for each loops that way.

blah, blah... more stuff later, lol

[Mzigaib]

Cool stuff Laidlaw, very impressive and useful shader.

I would suggest a implementation on your shader on the raytrace part, it would be great if I could have a fade by distance, very useful on raytrace renders.

Anyway, thanks for the shader and keep the good work.

Edited July 26, 2010 by Mzigaib

[LaidlawFX]

Cool stuff Laidlaw, very impressive and useful shader.

I would suggest a implementation on your shader on the raytrace part, it would be great if I could have a fade by distance, very useful on raytrace renders.

Anyway, thanks for the shader and keep the good work.

I think I would have to hack some vex code from the occlusion node to put in a max ray distance on the reflect/refract nodes to do that. I'll play with it, not sure when I'll be able to get it implemented. I'm moving to LA then visiting family on the east coast which is gonna suck up some time in the near future not to mention Siggraph stealing last week.

A hack you can do in the mean time is to create a reflection ball for your object with reflection linking by rendering a polar view with a camera in scene. Then you can modify the reflection map, too in post.

[LaidlawFX]

Cool stuff Laidlaw, very impressive and useful shader.

I would suggest a implementation on your shader on the raytrace part, it would be great if I could have a fade by distance, very useful on raytrace renders.

Anyway, thanks for the shader and keep the good work.

Apparently I out smarted myself... Just play with the IOR and you can control fall off... It has a slightly different effect than what you were talking about with Petz Reflection Attenuation, but it gets the job done. It will cut it off in a visible line, but if you are blurring the heck out of it can work... I'll probably hack and slash some of Petz code into the ubershader(got to collect all the good stuff), but I'm focusing on converting my shader to Houdini 11 and optimizing for pbr(I think I'll save a lot more render time with the upgrades in 11), the surface material is pretty cool in there, too which is giving me some more ideas. I'll post when I get Petz stuff in there, for you.

Here is an example UberShader_02.hip

Petz

IOR