McArt Posted September 18, 2014 Share Posted September 18, 2014 Have any of you used Amazon AWS to render your Houdini files? If so, I'd like to hear about the experience. what were the costs like, what was the set-up, any "gotchas" to avoid? It looks like it's got built-in support from the houdini side, but I've never tried it. Thought I'd ask you lot for your war stories, if any. TIA Martin Quote Link to comment Share on other sites More sharing options...
kev2 Posted September 19, 2014 Share Posted September 19, 2014 Highly recommend you give it a test. Most of the bugs are worked out now and for the most part its a compelling experience. The service has been hanging for me since I upgraded to Indie but I'm sure I'll get those sorted out soon. I did use it last year several times. There aren't a lot of gotchas but do set up notifications and alarms so you will know when you have instances running in the cloud. You can test your scenes using the Render > Preflight to see how the dependencies, etc work with your file system and workflow but for the most part, when you have a bear of a scene it is very cost-efficient. And its kind of fun to watch the frame boxes get filled up and checked off. Quote Link to comment Share on other sites More sharing options...
danylyon Posted September 22, 2014 Share Posted September 22, 2014 Hi Kevin I did want to give it a quick test recently, but I got an error in the log, something along the line "service not available without subscription". Do I need to pay a monthly subscription for Mantra to run? Is the free (on demand) cloud not sufficient? Quote Link to comment Share on other sites More sharing options...
kev2 Posted September 23, 2014 Share Posted September 23, 2014 No, there's no monthly fee. You set up the Amazon account, then add their EC2 service (Amazon has a tremendous number of cloud services such as S2 for storage. EC2 is their cloud cpu service). The EC2 fees Amazon charges you include a surcharge for the Mantra license. Here's a couple helpful SideFX pages: http://www.sidefx.com/docs/houdini13.0/render/cloudfarm http://www.sidefx.com/index.php?option=com_cloud&Itemid=213&page=faq but I would start here, the user guide: http://www.sidefx.com/images/stories/news/HQueue_Cloud/hqueue_cloud_guide_09nov12.pdf as I found it the most straightforward. For some reason SideFx dislikes using images in its help files (the occasional node tree set-up example would be a lovely thing SideFx, esp. in dynamics) but it walks you through it. It wasn't straightforward for me, getting it set up either so ping me if you can't get it, Kevin Quote Link to comment Share on other sites More sharing options...
SteveSayer Posted August 13, 2015 Share Posted August 13, 2015 Is there any more recent info on this topic? The pdf linked above doesn't exist anymore. I've been investigating using the Amazon Cloud to run some fluid simulations that my home machine isn't powerful enough for, then render the results. I've done successful tests rendering frames using the cloud service, but I haven't yet been able to accomplish a simulation/rendering combo. Here's the ideal process I'd like to be able to implement: Set up my simulation and do low-res tests on my home machine. Increase fluid resolution to production quality. Upload the scene to the cloud, and have one powerful machine with lots of RAM simulate it. Have a group of less-powerful cloud computers render out the resulting frames. Download the finished renders--but NOT the fluid cache files (which will be hundreds of gigs all told). Is this possible? Anyone know the ROP structure I should use? If this 'ideal' process isn't possible, I suppose the next best thing would be to have a single powerful cloud machine perform both the simulation and rendering tasks, but I haven't successfully set that up either. One of my tests resulted in the fluid cache being downloaded instead of the rendered frames (in fact the frames didn't render at all). Anyone have any experience with this sort of task? It seems like it must be a fairly common one, given the advantages of cloud computing. If there's a step-by-step guide somewhere it would be immensely helpful. Thanks, all. Quote Link to comment Share on other sites More sharing options...
pjackson Posted August 17, 2015 Share Posted August 17, 2015 (edited) Hey Steve I too am extremely interested in that workflow. I dont think its possible at the moment but it has got to be in the works.. It would be good to hear from someone at sidefx about this. I noticed in the documentation they have noted - Currently we do not support loading or saving to/from Amazon S3 storage service. S3 integration would make what you want possible, in theory you could just use the s3 web address like a local drive - sim to and load from http://s3.amazonaws.com/mybucket/myfluid.bgeo Edited August 17, 2015 by pjackson Quote Link to comment Share on other sites More sharing options...
kev2 Posted August 17, 2015 Share Posted August 17, 2015 Might ping the Gridworks folks and see if their workflow can handle it. https://www.sidefx.com/index.php?option=com_content&task=view&id=3121&Itemid=66 Quote Link to comment Share on other sites More sharing options...
paulh Posted August 17, 2015 Share Posted August 17, 2015 Is there any more recent info on this topic? The pdf linked above doesn't exist anymore. I've been investigating using the Amazon Cloud to run some fluid simulations that my home machine isn't powerful enough for, then render the results. I've done successful tests rendering frames using the cloud service, but I haven't yet been able to accomplish a simulation/rendering combo. Here's the ideal process I'd like to be able to implement: Set up my simulation and do low-res tests on my home machine. Increase fluid resolution to production quality. Upload the scene to the cloud, and have one powerful machine with lots of RAM simulate it. Have a group of less-powerful cloud computers render out the resulting frames. Download the finished renders--but NOT the fluid cache files (which will be hundreds of gigs all told). Is this possible? Anyone know the ROP structure I should use? If this 'ideal' process isn't possible, I suppose the next best thing would be to have a single powerful cloud machine perform both the simulation and rendering tasks, but I haven't successfully set that up either. One of my tests resulted in the fluid cache being downloaded instead of the rendered frames (in fact the frames didn't render at all). Anyone have any experience with this sort of task? It seems like it must be a fairly common one, given the advantages of cloud computing. If there's a step-by-step guide somewhere it would be immensely helpful. Thanks, all. I haven't done any fluid simulations on the Amazon cloud, but I've done a handful of Mantra renders. From my experience the cloud computers are not very powerful and I wouldn't trust them with any heavy duty simulations. The per-frame render time was far slower on the cloud machines than on our local farm/work stations. It did what I needed it to do, but it's not quite like having a real render farm at your fingertips. It would probably be more worth your while to upgrade your hardware. I hope the Gridworks solution will be more robust. Quote Link to comment Share on other sites More sharing options...
Kardonn Posted August 17, 2015 Share Posted August 17, 2015 I haven't done any fluid simulations on the Amazon cloud, but I've done a handful of Mantra renders. From my experience the cloud computers are not very powerful and I wouldn't trust them with any heavy duty simulations. The per-frame render time was far slower on the cloud machines than on our local farm/work stations. It did what I needed it to do, but it's not quite like having a real render farm at your fingertips. It would probably be more worth your while to upgrade your hardware. I hope the Gridworks solution will be more robust. They actually seem a lot better than before...I remember they used to be kinda mediocre 2-2.4GHz range chips, now it seems like a whole range, though nothing crazy like the Xeon 2699 v3: https://aws.amazon.com/ec2/instance-types/ Quote Link to comment Share on other sites More sharing options...
SteveSayer Posted August 18, 2015 Share Posted August 18, 2015 Thanks for all the replies and suggestions, folks. I'll keep digging and be sure to update if I find anything. Quote Link to comment Share on other sites More sharing options...
Sierra62 Posted August 18, 2015 Share Posted August 18, 2015 I just came across this today. https://vimeo.com/136587524 Quote Link to comment Share on other sites More sharing options...
SteveSayer Posted August 24, 2015 Share Posted August 24, 2015 (edited) Thanks, Kevin, Paul, and Cody. Sounds like they're trying to address all the main issues; definitely worth a look! Edited August 24, 2015 by SteveSayer Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.