Syntax for Writing Shader Operators

[Anti-Distinctlyminty]

Hi all,

I'm looking for learning resources to help me with writing a new operator type, but I'm also very interested in writing shaders in general.

For writing shaders I've essentially only found an old odforce article from the wayback machine. It could be out of date as far as I'm aware, but I cannot find anything else.

Anyway, rather than writing a whole shader I'd rather create a new operator, but I can find no resources on this at all. Is this in the documentation somewhere and I've missed it? For example, the $variable notation I've not seen anywhere else except when I cracked open an existing operator, along with essential looking syntax like the $isconnected_variable to determine if a variable is plugged in.

If someone could point me towards some sort of documentation I would be very grateful.

[rsd]

I've got everything you need.

$-subscription is used in Inline VOP and in Inner code in operators. It's needed to VOPNET code generator to variables have unique names.

Outer code is used for includes or functions and struct definitions, i.e. everything that is outside the main function.

Edited August 25, 2016 by rsd

[Anti-Distinctlyminty]

Hi rsd - and thanks for helping me out. I was wondering if I might ask a few more questions of you...

I've been trying to write a shader, and I've been having difficulty understanding the operator types and the various outputs they have and the contexts in which they reside. When I create a new operator by going to File > New Operator Type, I have these choices:

• VEX Type - Surface Shader Operator: This appears at SHOP level only with an output called 'surface' (see attached image) that seems to serve no purpose as this operator type is unavailable inside a Material Shader Builder.
• SHOP Type - SHOP Material: No Interface or anything. Is this an empty Material Shader Builder?
• VEX Builder Type: This only appears inside VOP Networks, but as I said before, there is no documentation 

Is there a reason why the VEX Type - Surface Shader Operator (and material nodes such as the cloth in the attached image) have an output that doesn't seem to be used?

I'm guessing for the documentation on the VEX Builder Type I can just refer to the 'Inline Code' documentation?

[rsd]

The Outputs of VEX Type operators are used to assemble shaders (usually Displacement and Surface) into materials  Try to subnet any shaders and you'll see what happened.

 

Yes, the Inline Code documentation is enough. The only differences is that in Inner code of VEX Builder Type you can also access parameters simply by its names with the same prefix $ and input names remain constant.

You can also make your VOP operator supporting different types with button New Signature in Input/Output type. The used functions in this case have to be overloaded. Take a look how Mix operator is done.

[Anti-Distinctlyminty]

Quote

The Outputs of VEX Type operators are used to assemble shaders (usually Displacement and Surface) into materials  Try to subnet any shaders and you'll see what happened.

Ah yes, if I put down a displacement shader in addition to my own, select both and move to a subnet, that in essence creates a 'material'? If so, I'm still a little confused as to why the cloth material output is shown as an output in the image above (i.e. the white output on the cloth node). 

Quote

Yes, the Inline Code documentation is enough. The only differences is that in Inner code of VEX Builder Type you can also access parameters simply by its names with the same prefix $ and input names remain constant.

So If I were to create a parameter called 'angle', I just refer to it in code as $angle? Makes sense.

The main thing I was confused about was that there seemed to be some special syntax. e.g. $isconnected_angle, I assume tells if that input is connected. Are there any more special cases like this? (I only knew of this one because it was in some nodes already).

Also, global parameters, such as position are still just referred to with P? (without a $)

Thank you so much for all the help rsd.

 

 

Edited August 26, 2016 by Anti-Distinctlyminty

[rsd]

On 26.08.2016 at 11:35 PM, Anti-Distinctlyminty said:

The main thing I was confused about was that there seemed to be some special syntax. e.g. $isconnected_angle, I assume tells if that input is connected. Are there any more special cases like this? (I only knew of this one because it was in some nodes already).

Also, global parameters, such as position are still just referred to with P? (without a $)

 

$isconnected_... seems to be the only special variable.

Yes, you can access global variables without without $-prefix.

As you noticed the common pattern for $isconnected usage is like this:

vector $_p = $isconnected_p ? $p : P;

where "p" is the name of the input.

There is also function isconnected(), but it takes input number, not a name.

[Anti-Distinctlyminty]

Thanks again rsd, I'm almost there with this shader now! There just one thing that is causing some problems is that when I multiply the output of my custom VOP with another colour mask, the result is way darker than it should be and I can see no reason for this. 

I've attached the hip file. If you have tme, take a look inside the shop network > cl_test. Change the input on switch1, this will render the custom VOP output (called ChanLum1), then the multiplication mask (from the trace_inside network part), then the multiplication of the two. You will see that the multiplication is not correct.

Shading_Tests.hipnc

[rsd]

The problem is due to the secondary bounces. Your 'trace inside' is incorrect. If you want to test the hit normal, you have to import it from the hit surface. Try to use gather loop instead of raytrace.

Two more advises: It's better to write routines in Outer code or in separate file and then include and call it from Inner Code section. Includes are easier to debug because vcc refers to line number when error found.

Illuminance loops can't handle area lights efficiently (tested it a couple of years ago, not sure for now), give preference to foreach loop with getlights().

[Anti-Distinctlyminty]

Thanks rsd, that was indeed the problem. I switched to a gather loop instead, which is working great, I was just using the raytrace as that was closest to the other 3D application I use. Most of the time I do not know what functions to actually use

Thanks for the tip. That code seems to be working now, but it I need to do anything more to it I'll refactor it. But I'm guessing that using external code means I'll have to change some environment variables so that Houdini knows where to find the files.

The illuminance loop seems like it's working, so that's a good thing, but I'll take a look into using a foreach combined with getlights() - again I just didn't know that function existed. 

I was reading the documentation trying and find something akin to a 'shader ID', (so inside the gather loop I could tell if I hit the same shader or a different one) and I came across a ray import node/fucntion, but was unclear on how this worked. How does one send information to be used by ray import? Does this have some common use that I'm unfamiliar with?

And finally, I'm having some issues with the ChanLum VOP...if I try to plug a non-parameter node into it (such as a noise or colourmix) I get an error, stating 'color input is not a parameter VOP'. I don't understand why there's an issue here, surely a colour is just a colour?

And thanks once again for all the assistance!

Shading_Tests.hipnc

Edited August 31, 2016 by Anti-Distinctlyminty

[rsd]

Mantra looks for includes in $HOME/houdini15.5/vex/include. Just drop your file there. There is also $HOUDINI_VEX_PATH environment variable, see doc.

Use vector type on inputs and outputs instead of color. The 'color' signature is defined for Renderman compatibility.

[Anti-Distinctlyminty]

Hmm, changing the node input/outputs to vector type still presents the same error :/

[rsd]

Reload the scene, it should work.

[Anti-Distinctlyminty]

Strange...it didn't work for me. But, it doesn't matter! I had to rebuild the shader in my commercial version of Houdini at work anyway, and it worked fine

There's a few more things I'd like to add to the shader:

1. There's a part that deals with reflection, is it possible to add this to the 'reflect' component?

2. This one may be harder, but it also more important...in the 'trace inside' within the gather loop, is it possible to tell if the gather is sampling this shader, or some other shader? Something like get a shader ID. I did search quite extensively, but couldn't find anything. The reason for this is that if the shader is sampling itself I'd like to handle things differently.

[rsd]

1. The component labels are used only inside illuminance or in BSDF's. If you need AOV's, just export a specified variable via Bind Export.

2. It's possible to set ID based on shader name using renderstate. Example:

# get shader name
string shader_name;
renderstate("shader:name", shader_name);
# hash it to int
int shader_ID = random_shash(shader_name);
# then export shader_ID to import it from other shaders using gather

 

[Anti-Distinctlyminty]

1 hour ago, rsd said:

1. The component labels are used only inside illuminance or in BSDF's. If you need AOV's, just export a specified variable via Bind Export.

2. It's possible to set ID based on shader name using renderstate. Example:

# get shader name
string shader_name;
renderstate("shader:name", shader_name);
# hash it to int
int shader_ID = random_shash(shader_name);
# then export shader_ID to import it from other shaders using gather

Ok, I assume that this has to be an inline VOP as there is no random_shash node.

I'm unsure on how to export/import the variables though and I couldn't get anything to work. I've attached my latest setup, just trying to successfully import the variable, but it always fails. I'm guessing it may be an issue with the vgather VOP itself as opposed to the VEX gather function?

Shading_Tests.hipnc

[rsd]

Rayimport is useless unless you send something, gather stores the imported value in the passed variable.

http://www.sidefx.com/docs/houdini15.5/vex/functions/gather

Shading_Tests.hipnc

[Anti-Distinctlyminty]

Excellent. Thanks again rsd...I shall try to leave you be for now!