Google Cloud Renderfarm with indie license.

[sebkaine]

Hi guys,

 

I would like to get your pov about the feasibility of seting up a small farm with 3 Houdini Indie Engine license.

I am actually debating beetween buy and set up 3 machines or use a cloud solution.

 

I am really not an expert on those subject so i would like to know if it's possible ?

 

Indie license are not floating so they are lock to one MAC adress.

I guess Google instance get a new MAC adress each time you start a new machine ?

I also guess that there are no way to emulate a fix MAC adress on random instance ?

 

There should be a way to setup a license server, and ask a license from your google instance throw a VPN ... but this

would be only possible in a floating license scenario.

 

So at the moment my conclusion is that it is impossible to do so ... but i would be happy to hear some expert advise just to be sure at 100%

 

Thanks for your help !

 

Cheers

 

E

Edited September 10, 2015 by sebkaine

[Mandrake0]

i think the amazon or the coming gridmarkers would make more sense for cloud rendering license fees are include.

a local system would make more sense when you want to have easy control and when you can feed it with data 24/7.

 

gridmarkers: http://forums.odforce.net/topic/23687-gridmarkets-houdinimantra-rendering-service/ 

 

it's a cost thing calculate when it make more sense to have a local farm or a cloud solution.

[PatriciaCornet]

Hi Sebkaine,

 

We may be able to help with your decision making.  

 

Sign up for GridMarkets' beta test (see gridmarkets.com/beta-signup) - and kick the tires on our cloud based Houdini rendering service at no cost, risk or obligation.  Mandrake0 is correct - Houdini licensing is included in the rendering price - which your free beta credits will cover.

 

Pat

[sebkaine]

Thanks for your answer guys !

 

I will definitly try GridMarkets beta .

 

My interesting in a custom home made cloud is better control of cost / power.

On the SESI announcement for EC2 offer you have :

 

a 64-bit machine with 7.5 GB of RAM is only $1.99 per machine per hour (and 2 VCPU).

 

If you setup your own houdini indie license on the cloud for $1.600 per machine per hour your have a 120GB Ram + 32 VCPU.

- so it's cheaper as you already own your indie license

- you can have 3 * 32 = 96 VPCU + 3 * 120 = 360 GB Ram

 

=> In other word you are not on the same planet of ratio for power / price

 

But again i will definitly check GridMarkets in depth !

 

Cheers

 

E

Edited September 11, 2015 by sebkaine

[PatriciaCornet]

Hi Sebkaine,

 

The referenced quote, which I believe may come from a Side Effects 2009 press release (http://www.sidefx.com/index.php?option=com_content&task=view&id=1631), is a bit old, I think, and refers to AWS machines. The GridMarkets service, just announced in partnership with Side Effects at SigGraph2015, makes use of "dark" CPU power (see our www) - and you will see that we can provide a much better power/price ratio.  Have a good weekend! Pat

[protozoan]

 

 There should be a way to setup a license server, and ask a license from your google instance throw a VPN ... but this

would be only possible in a floating license scenario.

 

A regular floating license won't cut it (due to license restrictions). 

The only type of license that allows your own setup off your premises (like in a google cloud environment) is the Global Access License. 

 

SESI has explicitly acknowledged that when using a GAL a Google setup is "legal", in licensing terms. 

[sebkaine]

Thanks for the extra info protozoan ... no google cloud for me then !

[tricecold]

Houdini Apprentice Sim and render on Google Cloud Instance. Works pretty awesome, testing on slow machines for now. If it turns out to be feasible, I plan to use this as my render and simulation farm for Houdini Indie. A single instance machine boots at 200mb ram  installed xfce for VNC, but its not needed. So İ would like to ask to sidefx if the licensing would cause any issues on virtual machines, single apprentice seems to work fine. also can I use the apprentice edition on google cloud, does that break the license agreement ?

[sebkaine]

Well Tim , that's very good to know.

 

I have just buy my indie license, and the license is generated according to your 

Machine name + Mac adress

 

Just the fact of changing the name of the server break the license.

I don't think you have a way to meet those 2 expectations by using virtual machines in indie Mode.

 

Like said protozoan the fact of calling a houdini license from machines with random ID's on a global network enter in the field of a Global access floating license (the most expensive one).

 

The only options i see for the moment for a indie renderfarm is 

- EC2

- GridMarket 

- Build your own (my choice ...)

 

Zync who was buy recently buy google offer renderfam services on google cloud.

But unfortunatly Houdini is not supported.

https://www.zyncrender.com/

 

But anyway i find that farm on the cloud really start to get interesting when you have full control on what you do.

You buy your license and you dispatch them on linux instance and from there you scale your computing needs in a free way.

And at the lowest possible price ... cause in the end when you concatenate computing price + file transfer price , it start to be exepensive pretty fast.

 

For fluid simulation that generate huge amount of data i'm not that sure that it's a great idea ... maybe for testing purposes ? 

Edited September 30, 2015 by sebkaine

[tricecold]

I wanted to do this on an option like google cloud, because it lets you start from scratch, and my test machine was using 200mb after boot , with a working version of Hbatch. This way you are still in full control of your shots etc. EC2 again is probably the least problematic version for Houdini but yet again someone else is in charge of your nodes. I wanted to make my own little farm also,but machines deprecate overtime, electric and cooling costs, etc is not feasible imho. I will check into GridMarket , thanks for the info.

[MCGrund]

When using GridMarket you don't have to pay for file transfer. Thought that might be of interest.

[kev2]

Hi Emmanuel (and anyone testing cloud renders too),

 

Wondering what you discovered? I've been using EC2. It's straightforward and fairly responsive, although not quite cheap enough that I would push most renders to it. I signed up for GridMarkets but haven't given it a try yet. What was your experience?

 

Also, I thought you could add render-only licensed machines by buying Houdini Engine licenses instead of full Indie licenses. Am I mistaken? 

 

Thanks for any thoughts, suggestions, etc. 

 

Kevin

[sebkaine]

Hi kevin well i finally settle on a not on the cloud solution.

 

basically i buy 3 machines with :

- shutlle SH87R6

- 32 Go Ram

- 4790K

- HDD 2T

the cost is around 2700 euros for all , so quite reasonable at the end.

I have link those with HQUEUE it does work good , but i need to accomplish true production test.

 

at the end you are restricted to max 5 machine on the cloud , 3 if you want to keep user workstation.

so i find that the cost of cloud render + file transfer was not that good.

the fact that you introduce a higher probability of bugs / problems external to your control was also a major factor

 

at the end i use 1 indie + 3 engine license , the cost for what you get is just unbreakable ...

when i will have enough cash to switch to full license the cloud on farm solution will have to be reinvestigated.

cause those infinite mantra token must be use one way or another ....

 

if you have any question feel free to ask.

 

Cheers

 

E

Edited October 29, 2015 by sebkaine

[CiaranM]

Hi kevin well i finally settle on a not on the cloud solution.

 

 

at the end you are restricted to max 5 machine on the cloud , 3 if you want to keep user workstation.

so i find that the cost of cloud render + file transfer was not that good.

the fact that you introduce a higher probability of bugs / problems external to your control was also a major factor

 

Cheers

 

E

 

Are you talking about the SESI EC2 setup? I'm pretty sure the machine limit was more like 20 machines last time I used it. Agreed, I wouldn't rely on it for constant use or simulation, but worked well for me in a tight spot!

[sebkaine]

You are right Ciaran, My answer is kinda misleading. The restriction is not 3 on EC2 , but 20 machines. I think that the number 20 doesn't come from SESI,

but it's an amazon restriction. If you call amazon by phone they can allocate you more machine but i'm not 100% sure it will work with H.

 

And if you are in deep trouble for delivery that is still an ace in your game.

 

I  must do some render test on EC2 with H indie also ! I realise that indie give you the ability to have a private farm of 5 machine + 20 EC2 instance !

Pretty cool fire power for indie guys !

Edited October 29, 2015 by sebkaine

[kev2]

Thanks Emmanuel. I think we're headed in the same direction. Nice setup.

Also, I tried EC2 with H15 last night - no luck. Support tells me that EC2 rendering won't be available until mid-Nov for 15. This happened with 14 too. Point for local render licenses. 

Kevin

[goldleaf]

GridMarkets is pretty nifty too; they're in beta still, but you can sign up for it.

[MCGrund]

I tried the Amazon EC2 cloud and have to say that I wasn't very thrilled with the experience. The instances were low powered and naturally rendering was really slow. The instances with more CPU-power caused problems and couldn't be started even though they are included as options in the submission ROP. That was H14 though, maybe this has changed with the current release. Another thing that really bothered me is that you are charged by the hour. With EC2 your renderings are downloaded automatically. While this is a nice feature I had the problem that my home machine crashed during the loooong rendertimes. Thereafter I couldn't reconnect and my renderings were gone. Take this experience with a pinch of salt though as I didn't look for a solution. Maybe there is the possibillity to reconnect, get your files via ftp or so.

 

I also tried out the Gridmarkets solution which I found much more user friendly. You are charged by the minute, you are not charged for up- and download times/bandwith, render machines are faster and you have ftp-access to your renderings. Another thing that happened to me was that I didn't set my output paths properly (because Gridmarkets didn't support $JOB at that time) and therefore rendered into 'nothingness'. Still I was able to access each frame through their web interface and download it manually. Not very professional but better than to rerender everything.

 

I didn't do any cost comparison between EC2 and Gridmarkets though, as Gridmarkets offered some free credits for beta testing

Edited October 30, 2015 by MCGrund

[kev2]

Thanks MC, I signed up for Gridmarkets a while back but haven't tried it yet. I'll try it out today. When EC2 rendering is back online, I'll try to do a side-by-side.

Kevin 

[Atom]

at the end i use 1 indie + 3 engine license

So what good are engine license for rendering?

I really have not looked into the Houdini Engine because i thought it was for game developers, not Mantra rendering.