Flip simulation full my ram (but only after some frames: why?)

[rvb3n]

Hi, I'm pretty new with Houdini and I'm using it to play with Fluid simulation.

 

Unfortunately I have a problem: when doing a Flipbook of my simulation I can see in the task manager that my ram is full at 60%, but after some frame it get to 100% (of 32 GB) and the simulation slow down.

 

Is this normal? If the number of particles stay the same shouldn't every frame of the simulation need the same amount of ram?

 

Am I doing something wrong?

 

The simulation is just a flat tank with an object falling in it..

[Georgie]

Haven't played with flips yet but my best guess would be that it's simply being cached to your memory ;3 Your presumption would be correct if you were only dealing with just static attributes but that's not the case with simulations and even though I'm not that knowledgeable I think it's safe to assume that most attributes need to be cached at every single frame for a sim to ... well ... sim ;3 

 

Something else I came across a few months ago - http://www.sidefx.com/index.php?option=com_forum&Itemid=172&page=viewtopic&p=140520&sid=7f5c6967245088543fe862b3f8859ade

 

30 gigs still sounds a lot though. Maybe you can post a scene file for the technical guys to take a look ?

[rvb3n]

Hi Georgie and thank you for the quick answer!

 

The discussion you linked seems exactly what I'm experience!

 

Is there a way to make Houdini cache the simulation not in the ram but on the disk when doing test with a Flipbook (I don't want to make a ROP Output Driver)?

 

I read that a solution is to switch to Linux but I'm not very found of this option, anyway I guess I could give a try with Ubuntu (does Hudini work with 14.04?).

 

Tomorrow I'll clean the scene (I'm not allowed to share the model I'm using) and I'll share it here: hope someone has other suggestion!

[tricecold]

Go through this post, there are some hacks for windows,

 

http://www.sidefx.com/index.php?option=com_forum&Itemid=172&page=viewtopic&t=30531&highlight=memory+leak

 

The immediate workaround is to use the VC11 Houdini build on Windows and then copy/replace the tbbmalloc_proxy.dll over from a H12.5 install into H13.0. 

This will forcefully disable tbbmalloc_proxy.

 

 

Hi, I'm pretty new with Houdini and I'm using it to play with Fluid simulation.

 

Unfortunately I have a problem: when doing a Flipbook of my simulation I can see in the task manager that my ram is full at 60%, but after some frame it get to 100% (of 32 GB) and the simulation slow down.

 

Is this normal? If the number of particles stay the same shouldn't every frame of the simulation need the same amount of ram?

 

Am I doing something wrong?

 

The simulation is just a flat tank with an object falling in it..

[danw]

I'm pretty sure that memory leak got fixed ages ago, and doesn't affect any builds from at least the last 5-6 months.

 

One of the key things to keep memory usage in check for FLIP fluids - Go to the FLIP Object DOP, Creation tab, and switch off "Allow Caching"...  you'll still be able to playblast, and the sim will progress normally, you just won't be able to scrub the timeline.

 

Another HUGE memory hog can be if you happen to be using the Points from Volume SOP to generate your emission points to pass to the FLIP sim... the Points from Volume SOP is fairly horribly inefficient for generating large numbers of points.  If you are using it, try pre-caching your emission geometry to disc, and then using the disk cache to emit from instead.  This can potentially avoid those occasional huge memory spikes.

 

Other than that, I'm not sure.  If you can upload a scene file, we could get a better idea of what's up.

[rvb3n]

Hi danw,

 

thank you for your answer!

 

I'm trying what you suggested (disable "Allow Caching") and for the begins of the simulation it seems to go well! I will wait the end of the simulation and then upload the file!

 

Unfortunately I'm not sure what you mean in the second part of your answer.

 

I created the Flat tank with the shelf tool: I don't know if I'm using Points form Volume SOP

[rvb3n]

Ok, a little update:

 

I deleted the 3 nodes for the whiteawater and this together with disabling Allow Caching seems to keep the ram usage stable.

 

I've about 5 millions particles in scene and the ream usage is 16 GB at frame 50.

 

Now my question is: can I simulate the whitewater after the main fluid?

 

Thank you everyone for the help!

[tricecold]

yes you should do caching one after another, turn off everything you dont need, cache the flip first, cache the source after( I prefer to leave the minimum amount of work for the solvers so i cache everything prior) then run the whitewater sim.

 

typically for any kind of simulation I follow these steps

 

Cache the sources first this can be collisions, smoke, velocities etc , whatever needs to be fed into the sim.

 

in your case, just put a file cache node after white water source node, and cache that as well.

 

this way you free the cpu and ram from everything but the solvers.

[hall]

Definitely will performs better in linux/or mac. Using linux and running sim from console its the best deal. I could simulate 80kk flip sim that was impossible to run using win.

[rvb3n]

Hi tricecold, thank you for the answer, but what do you mean with "cache it"? How should I save it (and how should I read it) to perform the whitewater simulation?

 

I was going do save the the tank with a ROP Output Geometry to shade and render it but I don't know how to cache the particles: can someone point me to some learning material to do this?

 

claudiohick: yes I'm going to try Ubuntu to see if there is any difference for me!

[tricecold]

Hi again

 

I would write a step by step answer but that wont be any good to you, so you need to visualise what is fed into a DOP network, in this case it will be named AutoDopNetwork ( for the main body of water.

 

From the shelf tool for flat tank it will make 7 nodes, 2 for simulation.  AutoDopNetwork and whitewater_sim, then you will have two for feeding information in to these DOP networks wavetank_initial and whitewater_source respectively.

 

So firsty I would save the output of the wavetank_initial, if you dive inside this node you will see there is already a filecache node. after you are happy with your wave shapes etc you can use this node to cache out all the velocities and everything else that goes into the main body simulation, this is done from the Export to File tab and after its done activate Load from Disk.

 

after this step is complete , now you are ready to do your simulation for the main body. in which inside wavetank_fluid node you have import_wavetank node tyo do this. it is similar to the filecache node, use the Export to File tab and ran the sim and enable Load from Disk.

 

Now you are ready for the whitewater source caching.

inside the whitewater_source node, you already have a node called whitewatersource_cache, it is again the same thing.Its again the same thing then enable Load from Disk.

 

Now you are ready to do the actual whitewater sim. Which you will save from the import_whitewater node, same story here also cache it out using the import_whitewater node enable Load from disk.

 

Because it is like a chain linked events happening in Houdini, just to get the whitewater, whitewater simulation node needs a source which is created using the main water body, which is simulated by using several inputs like velocities from ocean spectrum, and all the other inputs.

 

We are just making sure during the simulation we have everything calculated but the simulation itself, this will cut sim time and probably the memory fragmentation quiet a lot, because all the Simulation Network needs to do is read everything off the HD instead of calculating them every frame.

 

You can and should follow this kind of workflow for everything in Houdini, I cache everything I can, you can simply put a file cache node anywhere in the network to basically save that stage out.

 

The only catch in the tank simulation is, you need to do your tests in low resolution first and gradually increase the particle count, because the caching along the way before the simulation will also take time. So its better to know if the initial settings are gonna work for you.

 

You should do the Go Procedural videos on Vimeo along with Peter Quint Videos

 

vimeo.com/goprocedural

 

vimeo.com/user2030228

 

 

Hi tricecold, thank you for the answer, but what do you mean with "cache it"? How should I save it (and how should I read it) to perform the whitewater simulation?

 

I was going do save the the tank with a ROP Output Geometry to shade and render it but I don't know how to cache the particles: can someone point me to some learning material to do this?

 

claudiohick: yes I'm going to try Ubuntu to see if there is any difference for me!

Edited August 12, 2014 by tricecold

[rvb3n]

Hi again and first of all thank you for taking the time to write all this!

 

Now I think I understand better the process I should follow and I'm already testing it.

 

So basicaly all I have to do is:

1. cache all the simulation for the flattank_initial
2. cache all the simulation for the flattank_fluid

Then I'll be ready to do the whitewater simulation!

 

I think it was a dumb mistake to not see that the caching node was already present inside the system but thank you anyway: I'll come back with my result!

 

P.S. I'm already following GoProcedural and Peter Quint on Vimeo: it's helping me a lot!

 

P.P.S Is there a reason why the increment for caching the flattank_initial is set to 0.5?

Edited August 12, 2014 by rvb3n