cache How to cache a really heavy flip sim ?

[AlbusNolente 1]

Hello,

I have a really heavy flip sim of a river (around 101 millions of particles), I'm working on my final project for my master's degree. At my uni I can use only during the night as much computers as I want, each of them with 64gb ram. Since it's not possible to use the renderfarm for caches,I obviously thought to use more computers and render on each of them a specific frame range (470 frames total). The problem is that everytime a computer will start to cache, Houdini will always start to calculate the sim from the frame 1 and it will take the same huge amount of time, so what's the point to use more computers? Am I wrong?
What kind of procedure would you use?

p.s. keep in mind I cannot leave the computer caching after the night.

[Kaprolactam 1]

There are two options: One is doing a distributed fluid sim, which enables you to slice your sim into chunks that you can simulate on different computers at the same time. This should get you up to speed: https://www.sidefx.com/docs/houdini/dyno/distribute.html

If you just want to be able to continue simulations, have a look at checkpoints that you can set in the properties of your Dop network: https://www.sidefx.com/docs/houdini/nodes/obj/dopnet.html These will enable you to continue your sim from the last checkpoint, as opposed to having to redo the sim from the start.

Both techniques have their advantages and disadvantages, you'll have to see what works for you.

[AlbusNolente 1]

4 hours ago, Kaprolactam said:

There are two options: One is doing a distributed fluid sim, which enables you to slice your sim into chunks that you can simulate on different computers at the same time. This should get you up to speed: https://www.sidefx.com/docs/houdini/dyno/distribute.html

If you just want to be able to continue simulations, have a look at checkpoints that you can set in the properties of your Dop network: https://www.sidefx.com/docs/houdini/nodes/obj/dopnet.html These will enable you to continue your sim from the last checkpoint, as opposed to having to redo the sim from the start.

Both techniques have their advantages and disadvantages, you'll have to see what works for you.

Yes, I was reading about the checkpoints method, but I'm not sure how to use them. Tell me if I'm wrong about the procedure:

- I check the "save checkpoints" in the cache folder of the dop

- I specify the path for the .sim files

- I call these .sim files in the "Initial State" of the dop node (?)

I'm not sure about the last one, but I'm sure that the dop node must read the .sim from somewhere, right? Should I check/uncheck something else? 

[Kaprolactam 1]

You're correct. The sim files go in the initial state of your dop network. Also, be wary of the amount of sim files you dump, since they contain all the simulation data, they tend to be on the very large side. "Checkpoint trail length" (the number of checkpoint files to be kept simultaneously) and "checkpoint interval" are your friends here.