Shader construction workflow within mantra

[rob]

What is the work flow to observe for creating scripted shaders in mantra. I just look at the spaghetti of nodes in the vex editor and think it would be just far easier to script the shader from scratch from within the shader / vex context is this possible. I know for Renderman shaders their is a requirement to create an OTL from the complied code. Is this the same for creating shaders within mantra ?

for example the Renderman Code

/* Shader description goes here */

surface

constant_test(float Kfb = 1 /* fake brightness */)

{

color surfcolor = 1;

/* STEP 1 - set the apparent surface opacity */

Oi = Os;

/* STEP 2 - calculate the apparent surface color */

Ci = Oi * Cs * surfcolor * Kfb;

}

---------------------------------------------------------------------------------------------------------------------------------------------------------------------

Followed by the vex shader

---------------------------------------------------------------------------------------------------------------------------------------------------------------------

#pragma opname constant_rman_to_vex

#pragma oplabel "VEX Constant"

#pragma opicon SHOP_constant

#pragma label Kfb "Fake Brightness"

#pragma label opac "Opacity"

#pragma hint Kfb color

#pragma hint opac color

#pragma hint Cd hidden

surface

constant(vector Kfb=1; vector opac=1; vector Cd=1)

{

Of = opac;

Cf = Kfb * Of* Cd;

}

R

Edited April 1, 2008 by rob

[symek]

Sure! You think Mario makes his shaders in VOPs? (well who knows...) Most of the people here make their stuff in VEX code, not VOPs which get messy in bigger task.

Menu File -> New Operator Type -> Surface Shader -> Accept -> Code Tab -> and off you go with a code.

Check also Cutter Editor for working on code outside Houdini

http://www.fundza.com/

cheers,

sy.

[rob]

thanks ...

Out of interest where are all the global variables listed for surface shaders / displacement shaders ? help is not really forth coming with the info

[stevenong]

Hey Rob,

The online help has lots of information: http://www.sidefx.com/docs/houdini9.1/vex/ or you can use the Help Browser within Houdini.

Check out VEX Contexts and VEX Cookbook for a start. Each context has their own Global Variables so click through each to find out. Alternatively, you can go to VOPs, select what you want to create (SOPs, SHOPs etc) and put down a Global Variable VOP in that context.

Hope the above helps!

Cheers!

steven

[old school]

I usually write shaders outside of houdini using a shell with the houdini environment initialized (source houdini_setup).

The houdini command vcc is what you use to both compile your code in to .vex or .otl files and to get basic help on the various contexts. Just type vcc -h to get the help.

I use vim as my text editor btw.

To querry the global variables and all the functions supported by the Light context, you would type

vcc -x light

There are also a whole host of written shaders that ship with Houdini bundled up in the .otl file $HFS/houdini/otls/OPlibShopVMantra.otl. You can extract the shaders with htol.

[kumpa]

silly this question may be, but why does renderman shaders needs to be compiled first in order to make an OTL ?

[rob]

It could be to do with the compilers like slc are owned by Pixar and not Sidefx.

Thanks Steven for the links , I never seem to be able to find what I want .....im so blind

Jeff as ever thanks for the pointers. The subject could make some great Old School videos ! . You mention that you build your shaders outside of the houdini interface. So are you calling and test rendering your shaders much like you would do with prman / cutter

Edited April 1, 2008 by rob

[Jason]

Hi Rob,

I thoroughly believe that for 90% of general surfaces, you can fully achieve whatever you want using VOPs only.

However: You need to develop your own VOPs, as you would develop tried&tested functions from a VEX library .h file. The VOPs that are shipped with Houdini are not complete enough for a lighting pipeline. They're ok as low level building blocks - and only to a point because the underlying functions some of these VOPs represent often have variadic forms which are completely unsupported in VOPs and you'll find yourself digging and writing bits of VEX anyway.

So personally I believe if you have a decent strategy for what your pipeline would like to see from a shader builder, I'd spend a bit of time designing each unit as a VEX function which you can wrap in a VOP yourself. This way you'll get the power of pure VEX with the utility of VOPs to make it a fun, syntax-error-free environment with units of code you can fully trust - especially after you've fire-hardened them on a couple of shows.

For our work on Transformers at DD, we put together a great set of VOPs which fit in with our lighting paradigm and it turned out that we could build almost any custom shader with around 5-10 VOP nodes. Shaders for major characters like our shots of Megatron got so customized that it started at 10-15 and swelled to 50 or so, as we added layers of complexity due to him being frosted and encased in ice. We achieved our objective though - which was (myself and Phillip Prahl) wrapped up enough functionality into our VOP set that our hardest-core Look Development people didn't have to edit a single line of VEX code even once.

So, my conclusion: yes, VOPs can be powerful enough to achieve great results for regular surface rendering; but not out of the box.

Hope this helps,

Jason

[old school]

VOPs represent often have variadic forms which are completely unsupported in VOPs and you'll find yourself digging and writing bits of VEX anyway.

Variadic functions are supported through adding signatures when you build the VOP. It's not a limitation of VOPs. Just the author of the VOP that is missing this.

Jason, do you compose VOPs graphically (add VOP operator, start editing) or do you write them in a shell and use vcc to compile the vop?

Does anyone write VOPs outside of Houdini?

I am just wondering how hard core I need to go when demonstrating how to author a VOP.

[rob]

Anything you can cover in the correct workflow for this area would be a bonus. I must admit I am a big fan of cutter and Prman simply as the information is there in large amounts and with relative ease you can get up and running creating shaders. I am just not having a lot of luck with vops / vex > shaders as a none shader writer type person

Edited April 2, 2008 by rob

[old school]

This just shows you how daft I am! I didn't really comprehend what cutter is... Now I do. Thank-you. It has support for .vfl and Ifd too...

About Cutter

I just use vim. I compile the shader using !vcc <command options> from within vim to a .otl then press refresh assets in Houdini and re-render. Oh well...

Either that or write the shader inside a SHOP or VOP.

[Jason]

Variadic functions are supported through adding signatures when you build the VOP. It's not a limitation of VOPs. Just the author of the VOP that is missing this.

Ah yes, I knew with signatures it'd be possible to support variadics... although I must say that there are so many optional variadics (say the Ray Tracing Options) that you'd need a million signatures to cover the various permuations - if you were to do it right.

Any which way, the VOPs painfully undersell the power of VEX because it's easy to believe VOPs represent everything VEX can do.. which is not true.

Jason, do you compose VOPs graphically (add VOP operator, start editing) or do you write them in a shell and use vcc to compile the vop?

Does anyone write VOPs outside of Houdini?

I am just wondering how hard core I need to go when demonstrating how to author a VOP.

We found that maintaining header files outside in #include-ing them in the outer-code (like you do with oplibVOP.h) worked quite well for us for the most part. This way you get to build a function library that can be included in pure VEX shaders. The trickiest part was when you upgrade a VOP node to have more functions, (say a Diffuse Environment VOP v2 becomes a v3) that you have to bump up the .h file too - so we ended up having a .h file for every VOP node with a matching version number lest we get stuck in legacy function definitions. Unfortunately VEX itself doesn't allow users to overload functions so you can't just add a few parameters to an overloaded definition - you're forced to redefine the original function instead.

So; the short of it is that we had a .h for every (larger) VOP node, and another core library for utility functions which don't ever change. So not terribly portable, but it worked well for us - and you can still use an external editor... and working like this brought about the advent of the "Force Compile" option on the VOP VEX * SHOPs; because there was a lot of external editing going on and we need to be able to force a reload of the code.

[old school]

Perfect explanation Jason. Thank-you.

[shikung444]

Hi Jason and all,

I have a few questions on this topic...

(I apologize if some of these sound too basic, I'm not a programmer, just a user that can build vops shaders. I would like to ask these questions to gauge what I will need to learn in the near future to successfully make even better shaders in Mantra):

1. I'd be interested to see a list of things that are VEX-specific, not doable in VOPs shading alone. (So far, I only know of point clouds (edit: point clouds can be used in vops, but there are some functions missing right?)

2. What kind of production issues come up that require such complicated and multi-tiered approaches for shader writing?

3. About the .h header files, are those written in C++? If so, what type of functions are necessary to write in C++ that can't be achieved in VEX?

4. What is the relationship between the file "oplibVOP.h" and the current standard VOP operators shipping with Houdini?

5. Am I correct in saying that the .h files were implemented as a way to more easily support adding parameters to VEX definitions (as a workaround to a current limitation of VEX and Houdini)? Are there any other reasons for this setup?

6. What are signatures, and are they written into VEX or are a part of the operator definition?

7. The VEX code needed to be force compiled to reload into the VOPs nodes. C++ code also needs to be compiled too, right? Did all those .h files also need a compiling process before being able to be used?

Edited April 2, 2008 by shikung444

[Alanw]

3. About the .h header files, are those written in C++? If so, what type of functions are necessary to write in C++ that can't be achieved in VEX?

I am pretty sure this is the same case as Renderman's RSL in that .h header files are not true header files in the C/C++ sense of the definition, but more or less libraries of VEX/RSL functions for you to use in your current code. With RSL, you can actually name these files with any arbitrary extension you like and they'll still compile when included. I'm not sure if this is the case with VEX as I'm still very new to this myself.

For a usage example, RSL has plenty of built in noise functions, but the standard noise only outputs values from 0-1. If you wanted "signed noise" that ranges from -1 to 1 you could make yourself a function that achieves this and package it inside snoise.h or something similar. You wouldn't compile this file. Then you can refrence this file in the #include in your final shader and use the function as if it were "built in".

Sorry I can't comment on anything else. I'll leave that to the experts.

Edited April 3, 2008 by Alanw