Jump to content

Houdini Distributed Simulations Explanation

Recommended Posts

Hello Everyone, I am wondering can some one explain the details of a clustered simulation. I was wondering when I saw the clustering simulation why the individual clusters can't be saved as a scene and have it run on multiple computers since it would work on a renderfarm. So I'm wondering how the actual process works does the simulation have to work simultaneously where each cluster is still somehow referring to each other for calculations. 

Link to comment
Share on other sites

Clustered sims (a type of distributed simulation) are for when you do want the sims to run independently on a renderfarm and not look at each other. You need a script to iterate an env variable that changes something in your scene. This would switch the cluster the computer runs -- you could save out a hip file for each one of these, or the scripts should tell the farm to set a certain env variable. With pyro is usually means switching a bounding box, source, and cache file based on that env variable. 

I realize I am over-explaining in this next part, but figure it may be helpful for somebody else.

Distritubed sims are a bit different in that they do rely on each other for information, effectively simming one big sim over multiple machines. This requires slices to be set up where there is a tiny bit of overlap a so the grids and particles can transfer data to each other. Same thing goes here you need to set env variable on the farm machine so they know what slice they should be running. A sim tracker job has to be run as well and keeps tabs on the status of each of the sim jobs as they run so they know when to fetch data from the neighboring grids. Each grid has to wait for the other grids before it can start it's next frame though.

Keep in mind it may be faster to sim on one big machine than over 4 slower ones... it depends on the network overhead. It is going to be as slow as the slowest machine you sim on too. Speed is not linear as you increase machines either. I recently tested, and got some big sims (70million, and 135 million) done on 4 machines. They simmed about twice as fast as the same sim on a single machine. Ram use on each slice was about 1/2 vs the single sim too, so if you have a big sim that can't fit on your machines, distributing will help with that.

There is a masterclass on this that's pretty recent... he even shows how to set up an arbitrary particle sim for distribution at the end.


Edited by Solitude
  • Like 1
Link to comment
Share on other sites

  • 1 year later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...