vex multithreading problem ?

[eloop]

Hi again,

Further to my last posts regarding custom vex functions ... I've implemented a new vex function and it mostly works well, but I'm suffering crashes occasionally and I'm wondering if its related to multithreading. I'm developing on a multi-cpu machine, and I doing some stuff in the vex function that is definitely non reentrant. So is it the case that the "static void eval(int, void *argv[], void *data)" function may be called by multiple threads during cooking (I'm guessing yes on this one.) ?

If so does the hdk supply any functionality for doing mutex operations or should I implement it myself in an OS dependant fashion ? Something like the following is what I need.

if (first_time_called)
{
    lock_mutex(mutex);
    ..... do some time consuming setup that depends on the eval functions input paramaters ...
    unlock_mutex(mutex);
}

Thanks,

Drew

[edward]

Are you doing this from a VEX SOP? In VEX SOPs, they only use multi-threading only if you tell it to.

It simple enough to test if it's a multi-threading problem. Something like:

static bool entered = false;

if( entered )
   printf("multi-threading problem\n");
entered = true;

...

entered = false;

Take a look in UT_Lock for ways to deal with this.

static UT_Lock guard;
guard.lock();
...
guard.unlock();

[eloop]

Hi Edward, just wondering when you say "if you tell it to", do you mean setting multithreading via the houdini preferences or something at the hdk level. Ideally I would like to do something at the hdk level that tells houdini that a particular vex function may only be called by only one thread at a time. The function I've implemented is using some library code that is not re-entrant. But it seems a shame to have to turn off all threading so that this one function may be used safely.

-Drew

Are you doing this from a VEX SOP? In VEX SOPs, they only use multi-threading only if you tell it to.

....

16630[/snapback]

[edward]

No, the user chooses whether to use VEX multithreaded or not. If you put a VEX SOP in the network editor, one of its mandatory parameters is the Number of Threads. By default, this is set to No Threading. A user may choose to change this to say "1 per processor" which means that it will then execute the VEX code in parallel depending on the number of processors you have. For an example, append a VEX Mountain SOP to a Grid SOP.

To be safe, you might as well just add a lock anyhow as it's easy enough to do:

#include <UT/UT_Lock.h>

static UT_Lock theMyFuncLock;

void MyFunc()
{
    theMyFuncLock.lockLW();
    // the code
    theMyFuncLock.unlock();
}

[eloop]

Are you sure this is the case in the COP context where it appears that that the thread number is controlled from the preferences ?

I've got a vex function defined via c++ (ocean_uv) that I'm calling in a vex cop with a simple wrapper. Grabbing a lock for each pixel evaluated seems a bit course, but I'll give it a shot and see what the performance hit is. Thanks.

cop ocean_height(float time=0.0;
                 float height_scale=1;
                 int   x_res=6,z_res=6;
                 float x_size = 32,z_size=32;
                 float windspeed=30.0;
                 float smallest_wave=.2;
                 float winddirection=0.0;
                 float damp = 0.5;
                 float align = 2.0;
                 int   seed = 0;
                 string  name="";
                 )
                 
{
    float res =ocean_uv(time,height_scale,X,Y,
                                 x_res,z_res,x_size,z_size,
                                 windspeed,smallest_wave,
                                 winddirection,damp,align,seed);
    R  = G = B = clamp(0.5 + res/2.0,0.0,1.0);
}

-Drew

No, the user chooses whether to use VEX multithreaded or not. If you put a VEX SOP in the network editor, one of its mandatory parameters is the Number of Threads. By default, this is set to No Threading. A user may choose to change this to say "1 per processor" which means that it will then execute the VEX code in parallel depending on the number of processors you have. For an example, append a VEX Mountain SOP to a Grid SOP.

To be safe, you might as well just add a lock anyhow as it's easy enough to do:

#include <UT/UT_Lock.h>

static UT_Lock theMyFuncLock;

void MyFunc()
{
    theMyFuncLock.lockLW();
    // the code
    theMyFuncLock.unlock();
}

17913[/snapback]

[eloop]

Following up, using the lock per/pixel doesn't seem to be causing undue contention. With five threads running I can't see any appreciable (visible) difference in the cook speed, and thats grabbing the lock for the init,cook and cleanup callbacks. Of course the best solution is going to be getting the thing running properly multithreaded so I can get the benefit of multiple cpus.

-Drew

BTW I intend to release this code as open source so anyone interested in giving it a thrash and helping me clean it up, send me email (Drew.Whitehouse@anu.edu.au). Initially you should probably be of the tech persuasion, ie you can compile c++ code including installing 3rd party libraries. Here is a sample of the output.

[edward]

Oops, I forgot about COPs. Yeah, they're probably different. Note that COPs are multi-threaded by design. If you use lockLW(), it's a spin-lock (LW = Light Weight) which shouldn't hurt much. If you just did lock(), then that will probably incur a higher performance cost. But then again, you might want to check if it's actually calling your stuff in a multi-threaded manner as I've never looked at VEX COPs before.

PS. Nice image.

[malexander]

I've found that multithreading VEX only makes an impact if the amount of work that you're doing per point (or pixel) is really heavy (like 50+ lines). VEX batches data together in groups, and that overhead must be single threaded, while the excution of the code is multithreaded. So, if the work per-pixel or per-point is small, the execution phase will be a smaller percentage of the total cook time for a group, and you won't see as much of a benefit from the threading.