[novice] The when's and why's of caching

[art3mis]

Hi

Trying to get some clarity on the various caching mechanisms since its such an important part of Houdini's workflow.

There is the automatic caching that happens anytime you playback a sim in the Scene view as indicated by the blue progress bar and is limited by RAM unless cache to disk is enabled.  And then there are all the cache nodes where you can manually save files to disk.

In any application I've used, caching is something that is optionally done to help improve performance and or speed up production workflow. The key word here is optionally.

In Houdini, is this case? ie to complete any simulation is it optional or mandatory to save cache files at the various cache nodes?

2nd question> When would you normally cache your files, anytime there is a change upstream from a cache node and your network is taking a long time to cook?

3rd question> In the case of a FLIP simulation there are at least 2 places you can cache your files in the fluidtank_fluid node, the compressed_cache node and the surface_cache node.My understanding is the former is for caching the particle data and the latter for surface meshing. Would you normally always do both, ie after caching out new particle data, immediately cache out new surface mesh files?

And lastly, on all cache nodes there is a 'Reload Geometry' button and a 'Load from Disk' checkbox. Don't these duplicate functionality? ie if 'Load from Disk' is checked, doesn't this mean that cached files are loaded automatically and you don't have to click 'Reload Geometry'?

 

Thanks for help with any of the above?

[MirrorSword]

Caching is technically optional. You can turn off "cache simulation" on the DOP Network, and if you don't use any file caches you can have Houdini render the final image at the same time that you simulate it. However that is obviously a flawed workflow.

It sound's like your main question is when to use nodes like the file cache. Generally I cache anything that takes too long to calculate or is simulated. One thing you have to keep in mind is that with a file cache Houdini has to read a new file off disk each frame. This is a slow process so for some setups caching to disk might actually be slower than recalculating on the fly. Another thing you can do to speed things up is use a cache sop, which will temporarily cache the data to RAM. This can be great if you have enough RAM.

I don't have a ton of experience with FLIP simulations, but I guess that once you've dialed in the surface setting you would cache them both at the same time. However it is also possible to just cache the surface and not save the particles, and I've heard some people do this to save disk space.

Finally as to "Reload Geometry" and "Load from Disk". As the docs say "[Load from Disk] Switches between passing through the input geometry and loading the geometry from disk." so if it is unchecked then you will get live results. Reload Geometry works differently, and it requires that I explain a bit about how Houdini works. When Houdini loads a file off disk it keeps it in memory. So if for whatever reason that file on disk changes Houdini won't know about the change. In the case of file sequences it will reload the file when you change frames, but sometimes you want to tell Houdini to manually reload the file, and you can do that with the "Reload Geometry" button.

[art3mis]

That pretty well solves most of my confusion involving caching. Thanks for clearing things up.

[Yon]

Caching is a technique applied to any simulation ( Not just Houdini, or effects for entertainment industry) that is explained in this simple metaphor;

It is faster for your computer to run 1 mile, 10 times, with pounds 10 then it is to run 1 mile, 1 time, with 100 pounds. This also avoids breaking its back, or hitting memory limit.

This is because the more RAM that is in use, the slower the calculation will be. So as Mirror said simulating and rendering in one go can be done, But it will usually be slower, and you will have to run the simulation from the start if it crashes.

[art3mis]

Thats something I didn't know. "the more RAM that is in use, the slower the calculation will be." seems counterintuitive. I was wondering what the advantage to caching was since each cache step also requires a chunk of time.

 

Also came upon this thread

http://forums.odforce.net/topic/20694-flip-simulation-full-my-ram-but-only-after-some-frames-why/

 

Edited November 1, 2016 by eco_bach