Should you use Wrangle SOPs whenever possible?

[magneto]

Hi,

I think this seems to be a hot topic especially these days where more and more people hear about the Wrangle SOPs. I did notice two different groups though. On one hand you have people who use them at every opportunity, while others reserve them for the most complicated operations that might be harder to do otherwise. There are also people who barely use them or don't use them at all for many reasons such as preferring VOPSOPs, etc. We won't talk about them though

While I admire certain people valuing efficient networks over convenient methods such as using AttribWrangle SOP instead of Transform SOP, etc even at the expense of spending more time to set it up for the first time (since you can save it as a preset after that), I still wonder if there is any point in doing this?

Wrangle SOPs also take a little longer to understand since they are not as obvious as nodes that have a very specific purpose.

Also sometimes Wrangle SOPs seems like it takes more time when the data set is very small compared to other methods like AttribCreate which only slow down significantly when the data set is much larger.

It's true that certain nodes like Transform were slower than VOPs. But over time, i.e. now, they are much faster, so the performance gap is getting smaller, and in some cases negligible. It has to be done for each node separately, but SESI seems to speed these nodes as a long term goal, particularly when a major slowdown is reported.

You could probably replace most standard operations using the Wrangle SOPs. Is this a glimpse of the cutting-edge future workflows in Houdini, just like how all the POPs are replaced? Or is it a silver bullet that if you were to use it in your network, it would certainly be faster than the conventional methods. No questions asked

What's your take on it?

Thanks

[acey195]

Well if you go inside a point wrangle you can see that it is in essence a VopSOP I think.

pointwrangles are useful to replace point sops as they are almost always faster than a point sop,

but just as fast as a vopsop I think. It just depends on whether you like to type direct code or use VEX nodes inside a vopsop.

Additionally you can also compile a vopsop, which basically gives you a point wrangle, if you copy over the resulting VEX code.

They can be faster than native nodes, but only if you avoid using expressions that way.

if you use something like $PT or $PR in an expression, you could make it much faster using a pointwrangle /vopsop,

but with expressions that only need to be cooked one single time, like bbox() the gain would be minimal I guess...

I find that creating groups using vopsops is often much faster than using actual group nodes (+delete nodes), even if you are going to convert or delete the group afterwards.

in short, I'm in the vopsop camp

Edited November 8, 2013 by acey195

[magneto]

Thanks, I not just talking about the Point Wrangle though. There is also the AttribWrangle which is quite unique I must say Actually I feel AttribWrangle might be the future as far as VEX/VOPs go. But workflow wise I wonder if there is gonna be a drastic change because of their use.

[Pazuzu]

I use point Wrangle node every time I can, but I've noticed that if you use ramps or so much reference to spare parameters it slows down in comparison with a vopsop, for example if you want to colorize particles from a FLIP sim based on speed with a ramp the Point Wrangle node takes more time to cook than a vopsop, I've tested this in H12.5; The Attrib Wrangle node is way more slow to do this kind of things also, but if you don't use spare parameters seems to be much faster.

Just my two cents!!!

Edited November 8, 2013 by Pazuzu

[anim]

I use whatever feels faster to create so if I can type something faster I use wrangle otherwise VOP network

sometimes I mix them, like partially VOP with snippet inside to do some more complicated or VEX only stuff (like previously inline vop)

but anyway every wrangle node is just a wrapper for VOP network with snippet inside so you can always choose

and for execution speed difference, you can always use Performance Monitor and test by yourself and decide what to use for exact purpose if you really want the maximum speed

[fathom]

i'll use pointwrangles to create attributes since i'm usually going to generate them with some kind of expression that would likely be slowish outside of vops. if i'm doing anything that needs parameters, i'll switch to a full on vop sop since the wrangles don't really do parameters very nicely -- in fact, that might account for the slowdown.

what i'd like to see from the wrangles is automatic binding of parameters by name rather than having to do ch() functions. so if your ui had "foo" created as a float parm, you could use "foo" in your vex code just like P or id. if it did that, i would skip loads of vop sops since i find i generally use inline code in them 90% of the time.

[tjeeds]

I think the best way to break it down is to look at the Wrangle node purely as a replacement for hscript expressions. IMO, anything you need to build a parameter interface for is better off in VOPs. The Snippet is a great replacement for the Inline Code and pieces of your network that make more sense to write in VEX can now easily be embedded in your vopnet. This allows you the flexibility to code where you need to for clarity, while also preserving the interface building radness of VOPs.

I use wrangles pretty often, but I almost never go over 5 lines of code. They're great for creating attributes and setting random values, all that stuff that you learned to use expressions for before you knew how to use vops. I mean, do you really want to type out a point() function when you can use Import Attribute? Would you seriously prefer to type out a noise function rather than popping down a vop and creating parms for every input with two clicks?

[fathom]

wrangles are also a great way to learn vex. imo, vex > vops. vops are good for chaining together things like "color goes here" and "multiple this by some noise result". but once you get into logic operations, branching, looping, or even anything beyond simple math, i find vex is actually clearer.

[magneto]

I also prefer VEX, but for already made complex VOPs, VOPSOP is easier to build.

I noticed sometimes AttribWrangle to be faster than PointWrangle but didn't compare it to a VOPSOP. I guess there is more difference between VEX for POP/SOP context vs CVEX.

[magneto]

Do you use the new Gas Field Wrangle DOP instead of other specialized Gas DOPs? I am not familiar with them but after taking a quick look at some of them Gass Cross, Gas Calculate, Gas Linear Combination, etc seem like their VOP versions except they are DOPs. To me they seem too verbose.

Can Gas Field Wrangle DOP easily replace these?

Also since DOPs are a new area for me, are they inherently multi-threaded? If not, then surely Gas Field Wrangle is?

In any case I assume doing lots of operations in a Gas Field Wrangle DOP should be much faster than using several other DOPs that achieve the same result? Because here it's not a matter of comparing VEX to VOPSOP but VEX to DOPs, and there must be more overhead to using several DOPs instead of a single Wrangle DOP, right?

I am not a DOPs expert so feel free to correct me

[anim]

Gas Field Wrangle DOP is just a wrapper to Gas Field VOP with snippet inside, so it's not particularly new node as Gas Field VOP is in dops for ages and yes, it is multithreaded and can be used to replace many DOP nodes

but in my experience it's not necessarily faster than specialized DOP nodes, many times quite the opposite but I didn't benchmark it

I still use it a lot though, usually when it replaces more than one specialized node

[magneto]

Thanks anim, I thought it was a new thing. So it's not using CVEX then?

The specialized Gas DOPs seem like using Cross SOP, Add Values (arithmetic) SOP, etc which seem overkill to do at the higher level. So I thought if you had to do several operations, like add, multiply, normalize and what not, a single Gas Wrangle should be faster than using a dozen Gas DOPs which will pass the data from one another. I thought it would suffer if the data set is very large.

I will benchmark it as a test case then

[anim]

it is using CVEX

CVEX didn't come with wrangle nodes, Many "older" nodes are based on CVEX, like Volume VOP SOP, lots of Mantra Procedurals, Geometry VOP DOP, VOP Force DOP, ... so Gas Field VOP DOP was just one of them

I think benchmarking is the only way to be sure, the other purpose for specialized nodes may be Open CL support, there is no Open CL for Gas Field VOP DOP yet

[magneto]

Thanks anim. Do you use the performance monitor for this? Since this is DOPs, I am not sure how to benchmark it. Should I see how long it takes to complete say 100 frames using each method?

Also the OpenCL is a good point. But can you use OpenCL for certain nodes but not the entire simulation? I thought if the entire sim can't take advantage of OpenCL, then it won't help much. But I might be wrong

[anim]

yes, performance monitor

you can plug all desired nodes at once, all processing the same data, then just compare the cook times, and on top of that you can see heatmap which campares them all so you know immediately which one is the fastest for that job

and yes, it's a good idea to let it sim for a few frames so the ratios are more stable

[goldleaf]

Also the OpenCL is a good point. But can you use OpenCL for certain nodes but not the entire simulation? I thought if the entire sim can't take advantage of OpenCL, then it won't help much. But I might be wrong

Regarding OpenCL, I haven't had the time to play with it, but I've been intrigued by the existence of the Gas OpenCL DOP for a while now. It appears you just pass it OpenCL code in a file. No idea if you'd use HDK conventions to interact w/ DOPs data, but potentially a super useful tool. It'd be great if there was an example hip + opencl file, to understand what's expected.

http://www.sidefx.com/docs/houdini13.0/nodes/dop/gasopencl

[johner]

Regarding OpenCL, I haven't had the time to play with it, but I've been intrigued by the existence of the Gas OpenCL DOP for a while now. It appears you just pass it OpenCL code in a file. No idea if you'd use HDK conventions to interact w/ DOPs data, but potentially a super useful tool. It'd be great if there was an example hip + opencl file, to understand what's expected.

Probably the best example of using GasOpenCL for a custom Gas force is the Gas Stick on Collision DOP in H13. It shows using common OpenCL-supporting DOPs as much as possible, then has a switch between a GasFieldVOP for non-OpenCL and GasOpenCL DOP for the OpenCL path. The OpenCL kernel itself is a pretty straightforward conversion.

There's a few more GasOpenCL DOP uses in the Pyro solver, e.g. GasShred I believe. But StickOnCollision is the most self-contained.

[goldleaf]

Cool, thanks johner. I noticed the kernel file path shows sim/gasStickOnCollision.cl, but the file is in $HFS/houdini/sim; is that a bug that should be filed, or is there some black magic going on under the hood?

Edited November 19, 2013 by goldleaf

[johner]

Cool, thanks johner. I noticed the kernel file path shows sim/gasStickOnCollision.cl, but the file is in $HFS/houdini/sim; is that a bug that should be filed, or is there some black magic going on under the hood?

The kernel file is in $HFS/houdini/ocl/sim, so black (grey?) magic:

$hconfig -h HOUDINI_OCL_PATH
HOUDINI_OCL_PATH
The search path for OpenCL kernels. Setting this path will override the
default path of $HOUDINI_PATH/ocl/.
[/CODE]

Edited November 19, 2013 by johner

[magneto]

How much performance improvement does OpenCL provide though? If it's 30%, I shouldn't go crazy about it

Is there any chance of having the Gas Wrangle to support OpenCL completely?