How to import 2 million particles from realflow?

[Kevron]

Hi,

I am scratching my head around something I do not understand.

I have simulated a sculpted waterscene in RF5, and the particle count is just above 2 mil. (Total size for exportet bin files are 26.gb)

I have the newest plugin from Next Limit for both Houdini and Cinema 4d.

When I load the particles in Cinema 4D, the program thinks for a few seconds, and then I can see all the particles in my viewport. And the level of detail (LOD) is 100%. So C4D shows me all the particles, and is quite interactive when changing between frames.

Then I tries the same thing in houdini. The cooking time is 35 minutes, and it is still trying to import the particles. I can set the LOD to 1% in the RF particle import settings, and this reduces the cooking time. But I really want to render out all my particles, and not just 1%.

Do anybody knows what I am doing wrong here. Just seems that something is fishy. Is there a way to delayload Realflow particles upon rendertime?

I really like the way Mantra render particles, so that is why I would like it to work smoothly in Houdini.

My houdinibuild is the production one: 11.0.469

Tried also the current dailybuild (477), but the RealFlow plugin makes houdini crash upon execution.

Cheers,

Kevron

[Jim M]

Hi,

I am scratching my head around something I do not understand.

I have simulated a sculpted waterscene in RF5, and the particle count is just above 2 mil. (Total size for exportet bin files are 26.gb)

I have the newest plugin from Next Limit for both Houdini and Cinema 4d.

When I load the particles in Cinema 4D, the program thinks for a few seconds, and then I can see all the particles in my viewport. And the level of detail (LOD) is 100%. So C4D shows me all the particles, and is quite interactive when changing between frames.

Then I tries the same thing in houdini. The cooking time is 35 minutes, and it is still trying to import the particles. I can set the LOD to 1% in the RF particle import settings, and this reduces the cooking time. But I really want to render out all my particles, and not just 1%.

Do anybody knows what I am doing wrong here. Just seems that something is fishy. Is there a way to delayload Realflow particles upon rendertime?

I really like the way Mantra render particles, so that is why I would like it to work smoothly in Houdini.

My houdinibuild is the production one: 11.0.469

Tried also the current dailybuild (477), but the RealFlow plugin makes houdini crash upon execution.

Cheers,

Kevron

When/Where did you get your RF plug? There is one of 11th aug, that is outdated already.

There is another release .. 17th aug

https://portal.nextlimit.com/

[Kevron]

When/Where did you get your RF plug? There is one of 11th aug, that is outdated already.

There is another release .. 17th aug

https://portal.nextlimit.com/

Hi Jim,

I have made sure that I have used the lastest RF plug from Next Limit. I find it a bit strange that there are no forums that states the same problem.

I have searched through Odforce, Sidefx forum, official RealFlow forum and the "unofficial" realflow forum. But I can not seem to find any other Houdini user who experience the same problem.

My thoughts on this so far, it that it has something to be with how houdini imports the .bin sequence.

I have stored the .bin sequence localy on my harddrive, so there should not be some problems with transferspeed of data between different networks..

I am also a Mac user, but I can not see how that can affect the speed of things.. Its just strange.

Thanks,

Kevron

[Jim M]

I have stored the .bin sequence localy on my harddrive, so there should not be some problems with transferspeed of data between different networks..

I am also a Mac user, but I can not see how that can affect the speed of things.. Its just strange.

Hi Kevron,

Perhaps there is a cache memory setting somewhere. I know there are a few places in Houdini with settings for caching. I am just a beginner with Houdini so not the right person really. I have used the RF plugs in other apps, but only for meshes. I think there is a preload option in some versions, i.e you can import a range of frames either side of the current frame. Also if you are using the 2million imported particles in dops (good luck), then the default 500mb cache will probably be too small? I often set the dop cache to 4gb to get better performance. Not sure if my thoughts are relevant at all though...

LOD to 1% in the RF particle import settings

Isnt that for display rather than import? Like I said havent really used RF plugs to import lots of particles, but I know there is a setting in LWRF plugs to set a LOD which is for display, so you can play back at realtime-ish.

Certainly post this issue on realflowforum.com as there are more and more Houdini users on there these days.

[EDIT] I can confirm this is not viable. I have tried all caching options, and it takes far too long with 100k particles never mind 20 times that. I have 8gb so should be ok. It works fine in RF. I think there is somethign wrong with the caching code, it wont let me set to always, or when i do, I get zilch instead of particles....

Edited August 19, 2010 by Jim M

[Kevron]

Hi Jim,

I have a copy of this thread going at RF forum.

I tried as lukeiamyourfather suggested, to try it in Houdini 10. But it is still loading..

The LOD is also i bit strange, I think. It is supposed to be only the viewport display level of detail. But it seems that it controls how much of the particles beeing imported.

If I use a number of 1%, the loading is faster. But when I render out to disk, I ONLY get the 1% of the particles. It is possible I am wrong here, but it does not seem to work properly at the moment.

Any thoughts?

Thanks,

Kev

[Jim M]

Hi Jim,

I have a copy of this thread going at RF forum.

I tried as lukeiamyourfather suggested, to try it in Houdini 10. But it is still loading..

The LOD is also i bit strange, I think. It is supposed to be only the viewport display level of detail. But it seems that it controls how much of the particles beeing imported.

If I use a number of 1%, the loading is faster. But when I render out to disk, I ONLY get the 1% of the particles. It is possible I am wrong here, but it does not seem to work properly at the moment.

Any thoughts?

Thanks,

Kev

Well I it sounds like you have been reasonable in your efforts to sort it out yourself. I think the next step is contacting side FX and asking them. They might just say its a known bug and they will have it soted by X date. If you do that, let us know how you get on, as this is useful info for all.

Cheers,

Jim.

[Kevron]

Well I it sounds like you have been reasonable in your efforts to sort it out yourself. I think the next step is contacting side FX and asking them. They might just say its a known bug and they will have it soted by X date. If you do that, let us know how you get on, as this is useful info for all.

Cheers,

Jim.

I got an e-mail from David, one of the Houdini developers of the RF-plug. Here is the information he gave me (hope it is ok to copy/paste his answer, so more people can get some information on this subject):

Regarding your issue. As far as I can remember the importer plugins use data caching (per particle, per frame range etc.) in order to speed up the interactivity in the viewport. In this case since your particle file is too big you will run out of memory. This is a new issue since with RF5 it's possible to simulate a really big amount of particles, so you can possibly run out of memory.

The ideal solution for this problem will be RFRFK for Mantra which will be released very soon, so you can import and mesh your fluid in rendertime with much more control and there will be a new display node which will reduce the amount of data (number of attributes) to be loaded.

A workaround for your problem can be the new proxy type which only stores position and velocity data, so if you just want to use the particles this should work.

Anyway, the importer issue needs to be fixed, I will add a new cache mode which will create all the particles on the fly without storing the particle system data in memory.

This is still a good question how Houdini will handle this really massive frame in the viewport.

I hope we can release a fix for this during the next week.

So, they are on it

Cheers,

Kev

[Guest xionmark]

I don't use the plugins from Next Limit for this very reason, the caching is wonky. I have imported far more than 2 million RF particles into Houdini using my plugins, more on the order of 20 million particles.

And, if you want a render time solution, bake your RF particles out to disk as BGEOs, import them into my clusterThis mantra DSO, you'll have more points than you can imagine ... :-)

Oh, and no, I don't have H11 versions ready yet ... there's more changes to the HDK than usual, will take a little longer and also have more features in the new releases.

Mark

[Kevron]

I don't use the plugins from Next Limit for this very reason, the caching is wonky. I have imported far more than 2 million RF particles into Houdini using my plugins, more on the order of 20 million particles.

And, if you want a render time solution, bake your RF particles out to disk as BGEOs, import them into my clusterThis mantra DSO, you'll have more points than you can imagine ... :-)

Oh, and no, I don't have H11 versions ready yet ... there's more changes to the HDK than usual, will take a little longer and also have more features in the new releases.

Mark

Hi Mark,

your plugin seems like a nice addition. Do you have some builds for the Houdini 10 version which is public? I would really like to try it if you are sharing. But I also understand if its just for personal use.

Cheers,

Kev

[Guest xionmark]

http://www.digitalcinemaarts.com/dev/real_flow/index.html

You can use them in production, not only for personal use, they've been used in many for profit projects/films.

[anaka]

http://www.digitalcinemaarts.com/dev/real_flow/index.html

You can use them in production, not only for personal use, they've been used in many for profit projects/films.

Hi xionmark, beocuse I'm not so expert of Houdini can you explain how to install your useful stuf?!?

Does it work under Houdini 10 vc8 win32?!?

I'm copying the 2 dll files into DSO folder in Houdini 10.0.734 vc8, when I add the RF import SOP into a blank scene I get an error message and the application closes.

SOP_RF_Import_Exception::what() - Real Flow import exception: SOP_RF_Import::ReadRFParticleFile(): Can't open Real Flow Particle file for reading

many thanks

Edited August 27, 2010 by anaka

[Guest xionmark]

Hi,

It sounds like it's a mismatch of the version of Houdini vs. the version of Houdini used to compile the DSO. Fortunately I don't use Windows so I (unfortunately) wouldn't be able to tell ... but from your description, it's probably best to recompile with the version of Houdini that you are using.

Mark

[anaka]

Hi,

It sounds like it's a mismatch of the version of Houdini vs. the version of Houdini used to compile the DSO. Fortunately I don't use Windows so I (unfortunately) wouldn't be able to tell ... but from your description, it's probably best to recompile with the version of Houdini that you are using.

Mark

Hi xionmark,

I'll be glad to know when you'll update the versions available on your page, many thanks for your attention.

Edited September 1, 2010 by anaka