Best practices for cloud rendering

[jrockstad]

Hi guys, I'm looking to send my first job to the cloud and while it seems pretty straightforward I just wanted to clarify a couple things before pulling the trigger. 

 

My biggest concern is choosing a machine configuration. Is it better to distribute the job over a few less powerful machines or send everything to just one or two high performance machines? I've used remote renderfarms with other software in the past, but whereas most farms just have a few standard configurations, there's a lot of flexibility here and I'm not quite sure how to determine the best option. I'm rendering 240 frames at 720p, with tests on my laptop (Macbook Pro, 2.3 GHz i5) coming in at around an hour per frame at full resolution. Based on that information is there any way to get a rough idea of what type of configuration might be appropriate?

 

Also, since I'm using the Ocean FX tools, I figured I would save some render time by rendering all my maps locally and uploading them to the cloud for the final job, but with about 4 gigs worth of maps the tiny amount of render time saved doesn't seem worth it. If I let the cloud machine(s) render the maps, do I have to download them as well or can I choose to only grab my final renders? 

 

Cloud rendering seems very cool, just want to make sure I have everything in order. Thanks guys!

[mightcouldb1]

Questions to ask yourself:

How long does it take to generate the map + send it to the farm vs how much time does it take to compute on the fly? 

You can do simple math to figure out what this is saving you

 

Monitor your cpu and ram usage and go based on that.  Figure out how many cycles per second across each cpu and what will that amount to.  Is it more economical to go for the 2 power machines, or the 4 average ones?  Again simple math to solve this one!  $$$ per hour / total cycles per second is a good ratio for you to think about

[jrockstad]

Yeah, I've pretty much already decided to save the map rendering for the cloud given that the hassle of uploading 4 gigs worth of maps wouldn't really be worth it given that it's such a small portion of total render time. 

 

I was a bit confused by Amazon's "compute units" at first but I've been poking around their documentation and have found some useful information. Additionally, the help card for the HQueue Render node suggests a good default machine type to start with. Seems I should have just done a bit more reading before bugging you guys. 

 

In any event, it's probably best to just run a few simple test renders beforehand since every job is going to be different. Thanks for your reply Jason, I'll do a bit more investigating on my own, but of course I'm still happy to hear if anyone has had success with a particular configuration and would like to share their thoughts.

 

Update:

 

I've sent a couple jobs out but seem to have hit another snag. After I hit render I get the normal preflight / password dialogs, my browser opens to the Cloud Submit page, but after a while I get a dialog box with the message "The Houdini Cloud Submit daemon has terminated prematurely." There is a direction to a log file - this seems to be the most relevant passage:

 

ERROR: AuthFailure: Unable to launch image ami-73607f1a as the product code 5D6864EE is not supported for the specified region, instance type and operating system combination. Please contact the vendor for more information.

 

Total guess here, but might this be a Houdini Indie issue? Perhaps the Houdini build on EC2 isn't set up for .hiplc files?

Edited September 16, 2014 by jrockstad