Shared memory?

[hoknamahn]

Hi!

Is it possible and how to create a region of the shared memory available within the limits of the Houdini session?

For example... Some custom SOP creates the heap in one frame and saves a data in that heap and reads that data in the other frame. Or one custom vex function creates the heap and other custom vex function reads the data.

Heap must be located physically in the RAM.

[edward]

I don't see why not. Just make your data static within your .C file.

eg. outside of any function

class Foo {

...

};

static Foo theGlobalFoo;

[hoknamahn]

I don't see why not. Just make your data static within your .C file.

eg. outside of any function

class Foo {

...

};

static Foo theGlobalFoo;

17009[/snapback]

But what about dynamic data?

[AndrewVK]

Let me to explain what Hok meant.

He (and i, and probably somebody else) wants mimiс "array data type" for VEX.

I have offered 4 functions (array of "int" for now):

int handle = newarray(int size);

void setarray(int handle, int index, int data);

void getarray(int handle, int index, int &data);

void delarray(int &handle);

I cant declare new array as static because i want array for each sample.

The problem:

sop testarray()

{

int handle = newarry(10);

// at this point all handles becomes "wild" for OS ????

setarray(handle,5,13); //potentially dangerous operation;

int data = 0;

getarray(handle,5,data); // Not the fact, that data are equal 13 at this moment

delarray(handle); //No comments

}

The question is:

How to allocate memory from within "dll 1", use it from within "dll 2 & 3"

and free it from within "dll 4" ? Is it possible???

Sorry for probably stupid question...i`m still wander in the dark world of C++

[edward]

I think it will be ok as long as you use the handle correctly and be careful with not using allocated pointers over the 32-bit limit. If you want to be ultra-safe, then you create a mapping of handles to allocated memory blocks that your VEX dso functions return.

eg.

typedef int data_t;
static UT_PtrArray<data_t *> theAllocs;
static UT_IntArray theAllocSizes;

int vex_malloc(int size)
{
    data_t *ptr;
    int handle;

    ptr = (data_t *)::malloc( size*sizeof(data_t) );
    if (!ptr)
        return 0; // treat the 0 handle value as failure

    handle = theAllocs.find(NULL);
    if( handle < 0 ) {
        handle = theAllocs.append();    
        theAllocSizes.append();
    }

     theAllocs(handle) = ptr;
     theAllocSizes(handle) = size;

    return handle+1; // reserve 0 as special
}

bool vex_free(int handle)
{
    if( handle < 1 || handle > theAllocs.entries() )
       return false;
    if( theAllocs(handle - 1) == NULL )
       return false;

    ::free( theAllocs(handle-1) );
    theAllocs(handle-1) = NULL;
    theAllocSizes(handle-1) = 0;
}

bool vex_getvalue(int handle, int index, data_t &value)
{
    if( handle < 1 || handle > theAllocs.entries() )
       return false;
    if( index < 0 || index >= theAllocSizes(handle - 1) )
       return false;
    value = theAllocs(handle - 1)[index];    
    return true;
}

bool vex_setvalue(int handle, int index, data_t value)
{
    if( handle < 1 || handle > theAllocs.entries() )
       return false;
    if( index < 0 || index >= theAllocSizes(handle - 1) )
       return false;
    theAllocs(handle - 1)[index] = value;
    return true;
}

[edward]

Er, if you use the above, you also likely need to add a UT_Lock to make sure you synchronize access to the global variables.

[AndrewVK]

Hey Edward!

Thank You for almost finished solution!

[sibarrick]

Did this work? If it's this easy to add arrays to Vex how come it was never done officially?

[edward]

Who said it was easy to add arrays to vex? It's easy to add *functions* to vex.

[sibarrick]

Does this code not add access to arrays in vex then? I thought it was using a pointcloud type approach with handles etc. to get arrays working. What does it do then?

[edward]

Adding arrays to VEX proper would mean changing the language syntax.

[Mario Marengo]

I just noticed this thread and haven't had a chance to try out Edward's example yet (thanks Ed!! ), but I've always found it hard to make things work when the allocator controls the handle generation in a SIMD situation.. For example, in this vex shader fragment:

...

int handle = vex_malloc(size);

int i;

for(i=0;i<size;i++) vex_setvalue(handle,i,i);

...

the handle assignment (and allocation) is likely going to be called many times simultaneously (in pseudo-parallel) before the the following lines get a chance to execute, which usually results in multiple allocations (and handles that are not unique when used later on in vex_setvalue() for example). So I usually let the caller control the handle/ID, which in turn maps to a physical location via a std::map<> type of container (where the key/handle is usually a string or an instance ID for the caller OP).

In my case, this caller-centric approach is mostly a carry-over from stuff I've done in PRMan, but maybe the allocator-based handle generation works fine in VEX (never actually tried it).

So I'm really curious to hear if this is working for you guys. Please post your results!

Thanks.

[sibarrick]

Adding arrays to VEX proper would mean changing the language syntax.

24640[/snapback]

Thats what I thought, so what do these functions actually do then?

[edward]

@Simon:

They do it via the existing syntax, as functions.

@Mario:

I'm not sure I quite understand the problem you mention. Note that in the post immediately after, I corrected myself mentioning that a UT_Lock also needs to be added to the code. Handle id's are only re-used after they have been freed.

[sibarrick]

Maybe we are talking about two seperate things, what I mean is, yes it works via existing syntax but the functions you have written there aren't available in vex without compiling them through the hdk. Great if you have access to it, but why aren't they compiled by default into houdini so that everyone can use them?

Or am I missing something? I'm confused.

[edward]

Because no one would see that as a proper solution to arrays in VEX.

[Mario Marengo]

@Mario:

I'm not sure I quite understand the problem you mention. Note that in the post immediately after, I corrected myself mentioning that a UT_Lock also needs to be added to the code. Handle id's are only re-used after they have been freed.

24645[/snapback]

Yup. I was thinking of something completely different: storage that is persistent across multiple calls to a shader (i.e: storage whose lifespan is an entire Mantra frame), not a temporary thing during a run of a shader function.

But I'm thinking about this in the context of shading and I don't think that was Hok's original intent anyway, so never mind

Here's your code converted to a VEX dso in case somebody needs it. If you're on Linux, then just run make and open up the test hipfile. (and comment out all the print statements if you actually want to use it).

VexAlloc.zip