Rendering Tons of Points

[sanostol]

Hi, I have to render a large amount of points with mantra 200 million at least, and I'm looking for some tips too speed it up. Right now it works so far that I can render up to 400 million points fitting into my ram what is very cool. I tried different shadow types and still stick to shadow maps, as the render is way faster and less grainy. the reason why I tried different shadow types is that the preprocessing takes quite long and it seems to happen twice, one for shadow maps and one for the actual render. In addition the shadowmap generation takes some time, too.

All pointclouds are loaded as delayed loads, and You can follow the process of loading into mantra with the taskmanager, when all delayed loads are read, there is a long waiting time with only little cpu usage , I guess it is single threaded until the shadowmap generation is started same happens before the render starts. The render itself is really fast

Are the some mantra settings that can speed this up.

I attached a little demo with, just run the wedge and it generates some pointsclouds for the delayed loads

Martin

By the way, does anybody know how to activate verbose mode in rendering in h12 it seems to be very quiet

bunny_v001.hipnc

[magneto]

Verbose level is set using Mantra ROP > Statistics > Verbose Level, if that's what you mean

[rafaelfs]

I reduced the shading quality to 0.7 in the dicing tab and changed my render time from 32 to 20 minutes with no noticeable difference in the image.

I might be misunderstanding the parameters, but I figured since you have so many semi-transparent points, it's possible that past a certain number of them the opacity is already 1 and there'd be no reason to keep shading them at full quality...

Next I turned off stocastic transparency and that shaved off another 2.5 minutes.

I can't think of another way to get around the slow start rather then reducing the size of the caches. But even then I'm not 100% sure it'll help since it doesn't seem to be threaded anyway.

Cheers

[rafaelfs]

BTW, do you really need "share geometry" on in your delayed load procedurals? It seems to me that you have unique instances for each bunny, am I right?

Turning it off in this case should help with memory consumption a bit. This option is only good when you have multiple instances of the same geo.

Cheers

Edited July 17, 2012 by rafaelfs

[sanostol]

@magneto: that's what I meant, but here it keeps quiet

@rafaels: thanks for looking into the file and You suggestions. Unfortunately most time of the render is spent in the preprocessing.I hoped there would be a way to speed this up

Martin

[symek]

@magneto: that's what I meant, but here it keeps quiet

Here is your gotcha,...

Friday, October 28, 2011

Houdini 12.0.430: Houdini will now capture the output from renderers and add the information to the output driver info. There is a new SOHO parameter (soho_spoolrenderoutput) which controls this behaviour.

• off: SOHO won't capture the render output
• on: SOHO will capture the render output but only for graphical applications
• force: SOHO will capture the output for all applications

The previous behaviour (sending output to the terminal) can be restored by setting the parameter to "off". The default behaviour is "on", meaning that render output will be captured when running Houdini, but will be sent to stdout when running hbatch.

Edited July 17, 2012 by SYmek

[sanostol]

thanks, I had a preset in my folder that did not show me this option, great it's working again

[0quote name=SYmek' date='17 July 2012 - 11:24 AM' timestamp='1342549466' post='97162]

Here is your gotcha,...

[symek]

I really don't know who is making such a decisions (I presume no one), that a new feature changing common, long standing behavior of the software (in debugging department!), is deeply hidden and stays unnoticed. Lots of completely unnecessary confusion which is easy to avoid by installing introduced new property by default.

Interestingly whenever you use points as a point cloud or straight geometry, the load time is the same. I think this is mostly the cost of building acceleration structure (which was recently parallelized but apparently only for polygonal objects).

As to rendering, I would these days experiment with PBR... little slower rendering for a huge advance in shading, bouncing, emission etc. Good stuff. I can perfectly fine render your scene with PBR in a same amount of RAM (albeit slower), but for example with new physical hair diffuse - because it dismisses N for rendering giving me awesome looking the scattering-like look.

You could try different technique - with Volume Procedural. First you can see a slight performance increase while loading *.pc files with pcopen() in CVEX shader (plus *.pc takes 30% less on disk!), but it's a minor change like 35sec load time per *.pc file versus 42sec for bgeo (pc used as a normal geometry will be converted on-load to bgeo, so it's a regression to use it this way).

Secondly, you can convert point cloud into a i3d file. One time conversion cost causing no load time at all. So you end up with the Volume Procedural, executing CVEX shader per sample, which loads a i3d texture with density field generated from your original point cloud. Awkward, but might work and for a dense point clouds will look ~(the same).

I was thinking about simplifying this pipeline a couple of times. There are some Grid point acceleration structures, presumably less efficient than binary trees, but perhaps easier to stream from disk. There is one great working in HDK, but storing it in a file, is yet another topic.

Edited July 18, 2012 by SYmek

[Eitht]

In addition the shadowmap generation takes some time, too.

Incase you're not doing this - writing shadowmaps to disk will help speed up render time.

eitht.

[sanostol]

Hi SYmek, thanks again for You help.

The pbr render rocks, in addition I can use better lighting. Hope pointgeometry gets parallelized soon, too.

Martin

@Eitht:

Yes I know, thanks

I really don't know who is making such a decisions (I presume no one), that a new feature changing common, long standing behavior of the software (in debugging department!), is deeply hidden and stays unnoticed. Lots of completely unnecessary confusion which is easy to avoid by installing introduced new property by default.

Interestingly whenever you use points as a point cloud or straight geometry, the load time is the same. I think this is mostly the cost of building acceleration structure (which was recently parallelized but apparently only for polygonal objects).

As to rendering, I would these days experiment with PBR... little slower rendering for a huge advance in shading, bouncing, emission etc. Good stuff. I can perfectly fine render your scene with PBR in a same amount of RAM (albeit slower), but for example with new physical hair diffuse - because it dismisses N for rendering giving me awesome looking the scattering-like look.

You could try different technique - with Volume Procedural. First you can see a slight performance increase while loading *.pc files with pcopen() in CVEX shader (plus *.pc takes 30% less on disk!), but it's a minor change like 35sec load time per *.pc file versus 42sec for bgeo (pc used as a normal geometry will be converted on-load to bgeo, so it's a regression to use it this way).

Secondly, you can convert point cloud into a i3d file. One time conversion cost causing no load time at all. So you end up with the Volume Procedural, executing CVEX shader per sample, which loads a i3d texture with density field generated from your original point cloud. Awkward, but might work and for a dense point clouds will look ~(the same).

I was thinking about simplifying this pipeline a couple of times. There are some Grid point acceleration structures, presumably less efficient than binary trees, but perhaps easier to stream from disk. There is one great working in HDK, but storing it in a file, is yet another topic.

[Alanw]

Here is your gotcha,...

Friday, October 28, 2011

Houdini 12.0.430: Houdini will now capture the output from renderers and add the information to the output driver info. There is a new SOHO parameter (soho_spoolrenderoutput) which controls this behaviour.

off: SOHO won't capture the render output

on: SOHO will capture the render output but only for graphical applications

force: SOHO will capture the output for all applications

The previous behaviour (sending output to the terminal) can be restored by setting the parameter to "off". The default behaviour is "on", meaning that render output will be captured when running Houdini, but will be sent to stdout when running hbatch.

This was my "gotcha" as well. Thanks for mentioning this, I was wondering where my output went...

Edited July 19, 2012 by Alanw