cache How to cache a really heavy flip sim ?

[AlbusNolente]

Hello,

I have a really heavy flip sim of a river (around 101 millions of particles), I'm working on my final project for my master's degree. At my uni I can use only during the night as much computers as I want, each of them with 64gb ram. Since it's not possible to use the renderfarm for caches,I obviously thought to use more computers and render on each of them a specific frame range (470 frames total). The problem is that everytime a computer will start to cache, Houdini will always start to calculate the sim from the frame 1 and it will take the same huge amount of time, so what's the point to use more computers? Am I wrong?
What kind of procedure would you use?

p.s. keep in mind I cannot leave the computer caching after the night.

[Kaprolactam]

There are two options: One is doing a distributed fluid sim, which enables you to slice your sim into chunks that you can simulate on different computers at the same time. This should get you up to speed: https://www.sidefx.com/docs/houdini/dyno/distribute.html

If you just want to be able to continue simulations, have a look at checkpoints that you can set in the properties of your Dop network: https://www.sidefx.com/docs/houdini/nodes/obj/dopnet.html These will enable you to continue your sim from the last checkpoint, as opposed to having to redo the sim from the start.

Both techniques have their advantages and disadvantages, you'll have to see what works for you.

[AlbusNolente]

4 hours ago, Kaprolactam said:

There are two options: One is doing a distributed fluid sim, which enables you to slice your sim into chunks that you can simulate on different computers at the same time. This should get you up to speed: https://www.sidefx.com/docs/houdini/dyno/distribute.html

If you just want to be able to continue simulations, have a look at checkpoints that you can set in the properties of your Dop network: https://www.sidefx.com/docs/houdini/nodes/obj/dopnet.html These will enable you to continue your sim from the last checkpoint, as opposed to having to redo the sim from the start.

Both techniques have their advantages and disadvantages, you'll have to see what works for you.

Yes, I was reading about the checkpoints method, but I'm not sure how to use them. Tell me if I'm wrong about the procedure:

- I check the "save checkpoints" in the cache folder of the dop

- I specify the path for the .sim files

- I call these .sim files in the "Initial State" of the dop node (?)

I'm not sure about the last one, but I'm sure that the dop node must read the .sim from somewhere, right? Should I check/uncheck something else? 

[Kaprolactam]

You're correct. The sim files go in the initial state of your dop network. Also, be wary of the amount of sim files you dump, since they contain all the simulation data, they tend to be on the very large side. "Checkpoint trail length" (the number of checkpoint files to be kept simultaneously) and "checkpoint interval" are your friends here.

[josuka]

Im a beginner in Houdini and now I started doing FLIPS sims, I dont know im im doing the Cache properly

In my DOP Network I activated the Save Checkpoints

Checkpoint file: $HIP/sim/checkpoint/cache.$OS.$SF.sim
Trail Length: 2
Checkpoint Interval: 20

Then I use a Dop I/O to read my DOPNET, click SAVE TO DISK to start the simulation and Save it

I made my SIM from frame 0 to 60, but my Sim folder has:
cache.DOPIO.80.sim
cache.DOPIO.100.sim

I didn´t Sim frames 61 to 100. And I dont have a cache.DOPIO.60.sim

To resume my simulation in going to the DOP Network node, in the Simulation Tab i insert $HIP/sim/checkpoint/cache.$OS.$SF.sim as Initial State. I change my timeline to f60-200 and press the Save to Disk on DOPIO

The lower part of Houdini says that is cooking the DOP from frame 51, when it reaches 60 the Render frame bar starts to move

This is the right way to resume a simulation using the Checkpoints?
 

Thanks

[Shantanu]

Is it working? If yes please let me know