Setting up a simulation farm

[Hazoc]

This is mainly for the people who have the privilige of using several machines to process their simulations and other heavy cooking.

How does this part of the pipeline actually work ? I know there's the batch license for Houdini but that's about it.

I'm sure someone here can shortly describe what kind of a setup this would be and how does a .hip file travel over the network to a machine which starts processing it's insanely heavy particle fluid simulation.

Thanks.

[ranxerox]

the simplest way to do this is to ssh into another machine and run hbatch with the name of the hip file, then render the appropriate rop output driver. The second simplest way is to write a .cmd file (text file) which opens your scene, and render the rop output driver.. something like

"myfile.cmd":

mread <myhipfile.hip>

render /out/geometry1

then run hbatch like so : hbatch -v myfile.cmd

if you want to split it onto multiple machines you're going to have to get much more involved and read the .pdf by Jeff Lait on splitting fluid simulations onto multiple machines: http://www.sidefx.com/index.php?option=com_content&task=view&id=1516&Itemid=216

(actually I don't think this applies to particle fluids.. just volume fluids .. not sure).

-ranxx

This is mainly for the people who have the privilige of using several machines to process their simulations and other heavy cooking.

How does this part of the pipeline actually work ? I know there's the batch license for Houdini but that's about it.

I'm sure someone here can shortly describe what kind of a setup this would be and how does a .hip file travel over the network to a machine which starts processing it's insanely heavy particle fluid simulation.

Thanks.

[lukeiamyourfather]

One way is to use HQueue as a queue manager. There's a ROP node for HQueue which lets you choose any other ROP node in the scene to render/process on a remote machine (like ROP geometry driver for a simulation cache). I guess it depends on how many machines you will have and if this is something for home use or something for a studio pipeline.

Edited March 9, 2011 by lukeiamyourfather

[Hazoc]

Ok thanks guys! Sounds like the SSH & batch method is suitable for us.

So you have machines running hbatch, you call those machines by using e.g. Putty(?) or can Command Line Tools do this too ? And then you point out a hip file path with various rendering options and the machine starts it's job ?

I'm pretty unfamiliar with this shell and network stuff:)

There's stuff written in the manual but I'm not quite getting the full picture.

Edited March 10, 2011 by Hazoc

[symek]

Ok thanks guys! Sounds like the SSH & batch method is suitable for us.

So you have machines running hbatch, you call those machines by using e.g. Putty(?) or can Command Line Tools do this too ? And then you point out a hip file path with various rendering options and the machine starts it's job ?

I'm pretty unfamiliar with this shell and network stuff:)

There's stuff written in the manual but I'm not quite getting the full picture.

Why don't you want to use HQueue? It's took 15 minute to setup on our render farm just for sim purpose (we manage render with another software). It's by far the simplest method to deal with sim distribution.

[Hazoc]

Why don't you want to use HQueue? It's took 15 minute to setup on our render farm just for sim purpose (we manage render with another software). It's by far the simplest method to deal with sim distribution.

We'll I really don't have any opinion about this since I haven't read about setting HQueue yet. But it sounded slightly overkill since there might be only one slave machine running batch for a start, what do you think ? If there will be more machines, a real distributor is certainly the way to go.

[symek]

Overkill? No, unless you'll encounter installing issues, which we hadn't (any). Well, you won't know unless you try

[Macha]

If you use HQueue, be careful with Windows machines. We had a lot of trouble with it and couldn't get it to work error-free, even with Side Effects support. Eventually we (and they) gave up on it.

I hear it works well with Linux.

Edited March 10, 2011 by Macha

[ranxerox]

mmmm sounds like you're on windows. In that case I'd use VNC or the remote desktop application which comes with windows. Hbatch is just a gui-less version of houdini. You could also configure a submission system using Hqueue or rush, qube, alfred etc.

cheers

-ranxx

Ok thanks guys! Sounds like the SSH & batch method is suitable for us.

So you have machines running hbatch, you call those machines by using e.g. Putty(?) or can Command Line Tools do this too ? And then you point out a hip file path with various rendering options and the machine starts it's job ?

I'm pretty unfamiliar with this shell and network stuff:)

There's stuff written in the manual but I'm not quite getting the full picture.

[Hazoc]

If you use HQueue, be careful with Windows machines. We had a lot of trouble with it and couldn't get it to work error-free, even with Side Effects support. Eventually we (and they) gave up on it.

I hear it works well with Linux.

Oh crap.