mantra "clusterThis" procedural geometry DSO

[Guest xionmark]

Wow! Are you compete with SESI upon the fastest development feedback? I wonder, wouldn't this technique had more sense if we were be able to shape mesh with points attributes (CVEX)? In general such dso should have at least as much control over mesh shape as particle surface SOP. Is it possible at all in case of general instancer dso? Just thinking. Thanks for another great tool, Mark!

Simon.

:-)

When I get the time to program, I really do enjoy it!

I'm not sure entirely how much "surfacing" work will be done in this particular project, I have something else in mind

As soon as I get a few more things figured out (primarily motion blur and shader string assignments) then I'll turn to implementing the CVEX hooks. There will soon be flow charts (oh my!) and UML design docs on the web page that will make the design and operation easier to understand, evolve and deploy. But, it's also important to keep the scope of this particular tool in mind before committing to too much work (which unfortunately have a tendency to do so ...); more to the point, there's all too many interesting things to explore ... music, music, music.

Here's another sample image, the first is creating metaballs, has a standard VEX glass shader and the second is the same except with spheres, and a render with the metaballs rendered as volumes.

[Aearon]

hey there,

really cool work!

i compiled it for 9.5.241 on win64. i didn't get around to testing it, but it loads fine in mantra.

it's compiled using an apprentice license, and i'm not sure if this taints the compiled files in anyway?

anyway, here you go

mantra_clusterThis_v1.3.2_win64.zip

Edited October 30, 2008 by Aearon

[Guest xionmark]

hey there,

really cool work!

i compiled it for 9.5.241 on win64. i didn't get around to testing it, but it loads fine in mantra.

it's compiled using an apprentice license, and i'm not sure if this taints the compiled files in anyway?

anyway, here you go

Thanks!!

I have simple deformation motion blur working and will release in a few hours but want to see if I can add just a few more features/fixes in this next version.

(BTW: Compiling with the apprentice version works just fine)

Mark

[Guest xionmark]

New version available:

http://www.digitalcinemaarts.com/dev/clust...s_v1.3.3.tar.gz

Now has metaball instancing (which you can also use to render volumes), simple deformation motion blur, a few odds and ends. The file instancing will kind of work, didn't have to finish everything for it, will have it done in the next few days. The shader strings are still a problem, need to iron that out ASAP as well as the velocity motion blur. I'll have more docs/help/examples in the next release too.

Please report bugs.

Enjoy,

Mark

[Marc]

Hey Mark

Do deepshadows work with this? I'm getting some nice results with the clustering, but shadows don't seem to be working for me.

[Guest xionmark]

Hi Marc,

Yes, deep shadows work but (for some reason) mantra calls the procedural for *every* light that requires a deep shadow map. This is odd to me as I would think the map could be generated without the additional geometry creation/instantiation because, well, the geo is already there. I can post an image with and without deep shadows if you'd like, it sure makes a big difference most of the time.

I've got a few questions pending on the SESI forum about this, no answers yet, maybe I should submit a support ticket. If in fact that's the way mantra has to work, I'll have an option to save the instanced geo to a temp location and simply read it in for the additional calls to the procedural, which will at least save time instead of re-running the entire function over and over again.

There's just a few more relatively minor tasks I have to finish then I can implement the more interesting tidbits.

Let me know if you have any questions or find bugs, or have suggestions.

Mark

[Marc]

I don't suppose you could post a hip with dshads working could you? I tried it today and couldn't get it working, so I assume it's a user error .

Thanks

Marc

[Guest xionmark]

Hi there,

Attached is a tar ball with an example hip file using deep shadows.

Below are 3 images from the test file, the first is with no shadows, second is with deep shadows, and third is with ray trace shadows (frame 45).

If you crank up the verbosity of mantra to 4 (mantra -V4) it will produce a message such as:

Generating deep shadow image: c:/DOCUME~1/mstory/MYDOCU~1/work/VRAY/hlight3.rat

when it's creating the deep shad maps.

Let me know if I've confused you more ... it's been known to happen! :-)

Mark

[Marc]

Thanks, I'll take a look. My deepshadows were rendering, but they never had any clusters in them. It's possible my supersampling on the shadow wasn't high enough for the small particles, I'll have to check my file on Monday.

Thanks again

Marc

[Guest xionmark]

Hola~

New version (1.3.7) available:

http://digitalcinemaarts.com/dev/clusterThis/index.html

Added point instancing, fixed the file instancing and implemented shader point assignment if "vm"surface" attribute is present in the incoming geometry (in deferred instancing mode). For the life of me I can't figure out how to get point based "vm_surface" attribute with the Material SOP, that's why the Shader SOP is still in the example file.

The example file has a handful of takes for various examples.

Be sure to replace the OTL as well as the DSO/DLL when upgrading.

Take care,

Mark

[Guest xionmark]

Greetings Earthlings,

New version (1.4.4) available:

http://digitalcinemaarts.com/dev/clusterThis/index.html

* Added an optimization for when mantra generates deep shadow maps. The DSO has an option to write the instanced data to a temp file which is then read for each subsequent call from mantra. This is only available for the "generate all the geometry at once" method, since in the deferred instancing mode the geo instancing is controlled by mantra.

* Added simple CVEX processing for the instanced geometry. Point positions of the instanced geometry is passed to the VEX engine for further processing.

Notes:

* CVEX: Presently the CVEX hook is just a simple call passing only point positions into the VEX code. The arrays that are used to call the CVEX code is currently a static array as this was a "proof of concept" test for the project. I need to put a little thought into the most efficient way to dynamically build the arrays to pass to the VEX code so as many attributes as desired to be processed in VEX can quickly be assembled and shipped off for some SIMD goodness. This will be coming along quickly as there's much to be gained by leveraging CVEX at render time.

* Deep Shadow passes: To optimize the process of generating deep shadow maps, the user can now choose to have a temporary geometry cache file stored which is then read by the DSO for each additional deep shadow map and the final pass that mantra calls before starting to render. You may also choose to not delete the file in case you would like to use the instanced geometry later in your pipeline.

From Mark Elendt regarding why mantra calls a geo procedural for each light that gens a deep shad map:

"Consider generating an iso-surface. One light

really close to the iso surface, the other really far away. If you

generate a single piece of geometry for both views, then you'll end up

with either way way way too much detail or not enough...

Since the geometry can change based on the view, then we need to

regenerate the geometry each time."

* Motion Blur: The motion blur is implemented as a deformation blur, duplicating the points and then moving them. For now the velocity based motion blur has to wait. I'm thinking that there should be an option to move the points with CVEX code so the user can adjust the motion blur in any sort of way they would like.

From Mark Elendt:

"That's a little tricky....

Right now, duplicate your geometry, then move one piece based on the velocity attribute."

Here's the current method for displacing the duplicated geometry to create motion blur for mantra:

	 
	  myPointAttributes.myMBPos[0] = myPointAttributes.myNewPos[0] - myPointAttributes.v.x();
	  myPointAttributes.myMBPos[1] = myPointAttributes.myNewPos[1] - myPointAttributes.v.y();
	  myPointAttributes.myMBPos[2] = myPointAttributes.myNewPos[2] - myPointAttributes.v.z();

The example file shows a few ways of how to use the DSO but because of some pesky license problems, I'm presently not able to save a non-NC hip file (don't want to distribute a .hipnc file); shouldn't be too much longer, than I'll have more examples using the CVEX feature.

Next pass I hope to have better CVEX code, the curve instancing, maybe true volume instancing (though you can currently instance meta balls and render them as volumes), a lot more examples, a few pretty pictures, and ...

It time for a cleanup/modular dissection and code review ...

Remember to be sure to replace the OTL as well as the DSO/DLL when upgrading.

Enjoy,

Mark

[vmuriel]

Hi Mark,

I am doing a HDK point party node too.

Only for your information, I use this simple function to get motion blur:

addVelocityBlurGeometry( GU_DETAIL, shutter );

maybe you know it, maybe no.

[Guest xionmark]

Hi Mark,

I am doing a HDK point party node too.

Only for your information, I use this simple function to get motion blur:

addVelocityBlurGeometry( GU_DETAIL, shutter );

maybe you know it, maybe no.

Hello,

Is this working in a mantra geometry procedural or in a SOP?

Mark

[Guest xionmark]

Hi Mark,

I am doing a HDK point party node too.

Only for your information, I use this simple function to get motion blur:

addVelocityBlurGeometry( GU_DETAIL, shutter );

maybe you know it, maybe no.

Crap! I just found it! I was using DOxygen HDK docs for 9.1 !!!!!! ARGGGGGHHHHH!!!! Forgot to update my copy on my laptop ... doh!

I wonder why Mark E. didn't mention this. I will try it.

Thanks!

Mark

[Guest xionmark]

Hi Mark,

I am doing a HDK point party node too.

Only for your information, I use this simple function to get motion blur:

addVelocityBlurGeometry( GU_DETAIL, shutter );

maybe you know it, maybe no.

Thanks! Works Great!

I even wrote to SESI support on this before asking Mark Elendt, something must have gotten dropped in the translation ... ;-(

This helps a lot because I want to have both options for motion blur.

I owe you one man!

Thanks,

Mark

P.S. I'll have another release in the next couple of days, would love to get more of the CVEX processing in place.

[Guest xionmark]

Oh oh ... this call only works for instancing points, not other primitives/geometry.

It will help for points instancing but I guess I'm back to deformation blur for everything else.

Thanks,

Mark

[vmuriel]

I owe you one man!

mmm ... interesting ... I note that! ; )

this call only works for instancing points

Ok, I am only working with points.

My node is very simple, only points, nothing special ... but hey, a lot of them.

Edited February 16, 2009 by vmuriel

[Guest xionmark]

Hello,

OK, I have a better mechanism for the CVEX support, and added the velocity based motion blur for point instancing.

Notes:

1) For point instancing P, as well as v, N, Cd, Alpha and pscale are passed to the VEX engine and returned; id is passed in too but not returned. Therefore you can alter any attribute including the point's position.

2) For primitive, metaball and file instancing P is not sent & returned to the VEX engine, (this is because I'm dealing with primitive attributes as opposed to point attributes). I need to figure out a way to handle this, because there can be differing array sizes needed for different types of primitives.

3) For metaball instancing the weight attribute is passed to VEX and returned.

4) The motion blur for point instancing is now calling addVelocityBlurGeometry() instead of using deformation blur. So, as usual, the velocity vectors and shutter time will determine blur.

After working with the CVEX API, it's got some serious promise for all kinds of applications, very nice to have this available in the HDK! I'm thinking that there could be a few places in this DSO that could take advantage of it, e.g. pre-processing incoming geo, processing the instanced geo (as is done now), and further processing of the motion blur copy of geo required for deformation blur. It could be used for filtering and attribute blending as well, not to mention the processing that could be applied to instanced volume primitives. Coolio!

http://www.digitalcinemaarts.com/dev/clusterThis/index.html

Have fun.

Mark

P.S. The example file is unfortunately a .hipnc file, my apologies, this will be remedied soon.

[niko3d]

Hi mark,

We have tried compiling your DSO for houdini 9.5 64bit windows. But we seem to be getting this error.

mantra: Cannot allocate procedural 'clusterThis' on object /obj/grid_object1

When ever I hit render.

Any idea whats causing this problem?

Cheers

Nick

[Guest xionmark]

Hi mark,

We have tried compiling your DSO for houdini 9.5 64bit windows. But we seem to be getting this error.

mantra: Cannot allocate procedural 'clusterThis' on object /obj/grid_object1

When ever I hit render.

Any idea whats causing this problem?

Cheers

Nick

Hi Nick,

That's weird, not sure what's causing that.

A couple of things. Try writing out an IFD file and then do something like:

mantra -V4 -f my_IFD_file.ifd

and see if you get the same error.

If you do, please post the messages in the console so I can try figure it out.

I don't have 64 bit Windows machine available, so I can't really test.

We'll figure it out. :-)

Mark