does this sound feasible for custom vex functions

[eloop]

Hi all,

I'd like to get your opinions about how to go about doing the following. I want to write a new SOP (or perhaps OBJ) that provides an instance of a physical simulation that has a reasonably expensive startup and per frame cook time, but once its cooked it caches the results to be made available for that time step via custom vex functions that may be called in any of the various vex contexts.

I've got it working at the moment where I can create a single instance of the simulation via a custom hscript command and my custom vex functions are returning the results but I'd like to do it in a way where I can have multiple copies of the simulation object each with different parameters (via the gui) and for the simulation objects to cook if the time has changed and I need the results from one of my vex functions.

Is this the way to go about this type of thing ? What is the fastest way to get access from a vex function back to the SOP_Node derived instance where I have a reference to my simulation object ? I imagine that there is a similar mechanism for vex functions to get access to textures from cop nodes. Any suggestions appreciated.

Thanks,

Drew

[hoknamahn]

Hi, Drew!

I think it's impossible to cache the data directly from the VEX and store it between two or more frames because of the VEX nature. And so you must use HScript to store your data in "temporary" environment variables... But as I know, VEX functions can't get access to environment variables... But where solution exists - you can save your data in geometry (maybe HScript+Expressions) and read them by the VEX functions.

Hope this helps.

[edward]

I'm quite sure what you have at the moment? Is it written using the HDK? The usual way is to write a system of custom SOPs which then expose various states of your simulation via attributes which can then be combined. Then all the regular houdini tools can used in conjunction.

I'm not sure where the tie in is with VEX. If you make your results available as geometry, then one can just pipe it into the input of a VEX Geometry operator to perform further modifications.

[eloop]

Yes, this is all using the hdk, the hscript commands and the vex functions are c++ dso's. I want the vex functions to be returning results that are based on the simulation that is computed in the custom SOP. So I would like to have the vex function (written in c++) get a parameter that identifies the simulation SOP so that it can get access to its internals, and force it to cook if neccessary. I can probably just maintain my own std::map<char*,SOP_Node*> in the custom SOP such that the vex function can look it up using a string parameter everytime it runs. But I'm guessing that there is already a safe and optimized mechanism provided for doing this via the hdk so I am hoping someone may me able to point in the right direction ?

-Drew

I'm quite sure what you have at the moment? Is it written using the HDK? The usual way is to write a system of custom SOPs which then expose various states of your simulation via attributes which can then be combined. Then all the regular houdini tools can used in conjunction.

I'm not sure where the tie in is with VEX. If you make your results available as geometry, then one can just pipe it into the input of a VEX Geometry operator to perform further modifications.

16395[/snapback]

[edward]

I think that would be a bit weird as it violates the encapsulation paradigm.

I think you can just take an operator path as a parameter of your vex function and get at the direct node pointer directly. But if you're doing this from VEX, be aware that it could be multi-threaded and so you would need to be careful and lock your accesses into SOPs.

[eloop]

Hopefully it will work though and I think as long as the methods I'm calling on my SOP are re-entrant the multithreading should't cause any problems, in fact I would like it to work that way.

I'm trying the following to find the operator from it's path name but it doesn't seem to be working as *op is being returned NULL.

    OP_Director *od = OPgetDirector();
    OP_OperatorTable *table;
    OP_Operator *op;
    const char *path= "/img/img1/in";

    od->getTableAndOperator(path,table,op);

update: the following method of OP_Node seems to be what I want

   OP_Node  *findNode(const char *path) const;

-Drew

I think that would be a bit weird as it violates the encapsulation paradigm.

I think you can just take an operator path as a parameter of your vex function and get at the direct node pointer directly. But if you're doing this from VEX, be aware that it could be multi-threaded and so you would need to be careful and lock your accesses into SOPs.

16410[/snapback]

[Jason]

I know that we've done a fairly similar thing recently; I'll see if I can get Josh to chime in on how he did it. BTW, did you know that you can add one line of code and your SOP can have SOPs inside of it?? If you want to keep it at least conceptually encapsulated, then you might be interested in having all the VexSOPs which might be accessing this data inside of your simulation SOP. You can't enforce this but it might be a cleaner/safer way of working.

Cheers,

Jason

EDIT: Just to explain a little more -we've done it for a COP to access noise fields generated in a SOP - for the purpose of visualizing that noise.

[edward]

If you know what type of operator you're dealing with, you can use findSOPNode(). Make sure you check for NULL pointer as the user can easily give you a path to something that's not a SOP. Finally, you would likely need to know if the operator can be casted to your type. So something like this (assuming op_path is an absolute op path)

SOP_Node *sop = OPgetDirector()->findSOPNode(op_path)

if (sop && sop->getOperator()->getName() == "my_sop_type")
{
   SOP_MyType *my_sop = reinterpret_cast<SOP_MyType *>(sop);

   // do stuff with my_sop
}

[eloop]

Which is pretty much what I'm doing as well. I'm doing a Tessendorf style ocean wave simulation and I'd like to be able to use the various associated computed quantities via vex to generate displacement/bump and foam and perhaps birth some particles as well. At the moment I'm just doing the geormetry displacement with a pic() expression that references a VexCOP.

I'd be interested in knowing how you make sure the sop has been cooked properly at the time when the vex function wants to access it.

Thanks,

Drew

...

EDIT: Just to explain a little more -we've done it for a COP to access noise fields generated in a SOP - for the purpose of visualizing that noise.

16416[/snapback]

[eloop]

Ok, the following works. Now I have to work out how to make the sop cook when needed ...

static void
ocean_eval_uv(int, void *argv[], void *)
{
    float *result = (float *)argv[0];

    // lookup tho simulation sop
    SOP_Node *node = OPgetDirector()->getSOPNode((char *)argv[1]);
    if (node && node->getOperator()->getName() == "ocean")
    {
        drwOcean::Ocean *o = reinterpret_cast<SOP_Ocean*>(node)->getOcean();
        float *u = (float *)argv[2];
        float *v = (float *)argv[3];
        *result = o->eval_uv(*u,*v);
    }
    else
    {
        *result = 0.5;
    }
}

[edward]

float t = OPgetDirector()->getChannelManager()->getEvaluateTime();
OP_Context context(t);
op->forceRecook();
op->cook(context);

[eloop]

Hmm, that one gets me a quick crash and it appears this would be cooking everytime the vex function is called whereas I really want a cook when we are at a different time.

Is there a way for the vex call to register interest in a SOP such that it will ony recook when needed ?

Perhaps I should just pass the time explicitly across to my sop in my eval_** calls and let the sop determine if it needs to run at for new timestep, in effect bypassing cooking ?

If any sidefx developers are reading this I would appreciate a bit of help here ... should I go to a system where I bypass Operators as containers for the simulation and just interact with it via hscript commands and vex functions ?

Thanks,

Drew

float t = OPgetDirector()->getChannelManager()->getEvaluateTime();
OP_Context context(t);
op->forceRecook();
op->cook(context);

16441[/snapback]

[edward]

Yes, that forces it to cook all the time. Sorry, I don't really have time to explain things.

Are you trying to recreate the Hydrous tools? The approach they seem to take is to wrap it all in VEX functions.

http://martian-labs.com/products.php?product=hydrous

[eloop]

Hmm, I guess they are a bit similar, but I'm having fun re-implementing the Tessendorf FFT stuff from the Siggraph notes. Exposing things via vex just seemed to be the most flexible way to go about this as you aren't forced to create any geometry or images unless you really need them and you can easily use the results in any or all contexts without stripping attributes off of a different data type via expressions which would probably also be slower. The fact that the martian labs guys and DD have done this means that there must be a right way to go about it...

A snapshot of the current state - ocean.jpg.

Thanks,

Drew

Yes, that forces it to cook all the time. Sorry, I don't really have time to explain things.

Are you trying to recreate the Hydrous tools? The approach they seem to take is to wrap it all in VEX functions.

http://martian-labs.com/products.php?product=hydrous

16444[/snapback]

[Jason]

The one thing about ours is that both contexts are always availble simultaneously - i.e. COPs and SOPs. When you're talking about VEX, you may very well not have the SOP context around when you're rendering displacements. The level of detail during rendering is dynamic (view-based) and as such, disconnected to any SOP solution.

The Hydrous Tools solution ONLY implements a VEX solution and in order to get the same "preview" result in SOPs, you must write a VEX SOP with the same displacement code and parameters exposed as the VEX Displacement Shader.

This is pretty easy to achieve actually since the Hydrous Tools are excellently implemented (and fast too), and would be even easier if the BakeVEX SOP was updated to be able to be applied to non-mesh surfaces. Sometimes using a mesh surface is an adequate representation though - but often not efficient for many technical purposes.

I can see the benefit of being able to link these, however - for accuracy and speed.

Any which way I hope it works out well.

Jason

[eloop]

I'm beginning to see the problems now, it looks like I need to be caching a bunch of simulation results for different time steps / contexts. Perhaps I could keep my own cache that holds results for a set of different time steps with mutex protection from threads etc its all a bit risky without a clear idea about how this stuff is setup.

I notice there is an OP_Cache class that may be useful. The same thing must be handled already for things like image cops that are being read in other contexts but its not obvious how it all hangs together at the hdk level and particularly when throwing vex into the mix as well. I think it's probably time to bail and just read in lots of floating point image files ... oh well. Time to step back and rethink.

If the martian labs guys are using fft's to sum the wave field rather than using a simple sum of the waves / per position enquiry I'm impressed

-Drew

The one thing about ours is that both contexts are always availble simultaneously - i.e. COPs and SOPs. When you're talking about VEX, you may very well not have the SOP context around when you're rendering displacements. The level of detail during rendering is dynamic (view-based) and as such, disconnected to any SOP solution. 

The Hydrous Tools solution ONLY implements a VEX solution and in order to get the same "preview" result in SOPs, you must write a VEX SOP with the same displacement code and parameters exposed as the VEX Displacement Shader.

...

Any which way I hope it works out well.

Jason

16448[/snapback]