openFirehawk - Houdini and Open Source Cloud Rendering Infrastructure As Code.

[AndrewG]

 

This is my first post in a long time here, I've shared this teaser with some of you on FB, Youtube, vimeo and Linkedin today.

So what is it? A path for us to use cloud providers directly for our craft at the lowest possible price.  I have been building modules to spin up infrastructure in AWS with a language called Terraform (also open source, and can talk to other providers).  My ongoing work for this I've decided to open source under the project name openFirehawk.

This work I've been doing over the last 9 months can benefit Houdini artists and I'm proud to start talking about it, before I launch on Patreon in a week from now for your support on this open source project.

I realised through my deep dive into all things cloud to solve my own CG life problems in Houdini, that actually this should just be "The Way"... 

Cloud rendering till now hurt my wallet too much, and I was yet to see it provide the power of a full blown pipeline, which I sorely wanted.

So I thought, 

"Wouldn't it be insane if we could just build our own modular open source infrastructure for VFX and directly pay the big services to use their machines at the lowest possible price?"

Also, I thought it would be cool to be able to say "Want a copy of my vfx infrastructure?"

So I've created openFirehawk to do that. Its purpose is to provide access to the cloud providers, at the base price of that resource, and give you full control with a killer combo - teradici software for remote workstation graphics if you want that (paid per minute), ssh access to nodes over vpn, render management with whatever render manager we want (I started to implement deadline, which is free on AWS by way of being credited to your account), and any storage solution we choose (currently softnas on demand consumption is the cheapest, best and easiest I've found).

Open source is a path to making the cloud as cheap as possible for our craft, and openFirehawk proves it is possible with this demo simulation I ran in AWS above.

The rendered effects here are just the beginning of what openFirehawk can do with CG rendering & virtual workstations for artists. The animation was simulated and rendered in Houdini on AWS instances for $1.20. I paid $0.01 per core hour - many times cheaper than the alternatives available to me. 100 cores for $1/hr sounds nice...

Now to be clear - for us, we still need floating engine licences to use this.  I can't propogate code that would violate the EULA and plonk Indie in the cloud yet...  But just 1 engine licence can do a lot with 96+ VCPUs, probably more compute than most of us could afford in a single box.  I would hope that in time, more doors open up for us as Side FX have already helped cloud users in response to some discussion I had with them about LAL Licensing in this video I did in the jungle a while back and I hope things continue down this path.

 

openFirehawk is currently in alpha, and its use has manual steps on the client side that I plan to eliminate. It is an on going project, and I want it to be a good modular and extendable foundation for other code contributors in our industry.

So, would you like a copy of my 3D rendering cloud infrastructure to produce CG like this? I will have a need for your support to take it far - into something that you can use more easily, and a tool that continues to improve.

Any support you want to provide by sharing my posts like this, talking about the work, and contributing through Patreon when it launches 19th February will help take openFirehawk to become something that everyone in CG can benefit alot from.

 

I'm stoked to share this work with the community, and I hope to see where it goes,

 

Big love!

Andrew Graham

Edited February 12, 2019 by AndrewG

[kleer001]

Congratulations, first of all! This seems like a lot of work.

I've never done any cloud rendering so I am curious as to the savings and look forward to more good news.

 

[AndrewG]

Cheers Clear!  I'd love to build it into something that becomes as easy for everyone to use as possible and automate the whole process.

It's been a rewarding journey, wormholes within wormholes of learning I'll get into more details next week with some video I've shot recently.

[AndrewG]

I shared this on Linked in and FB yesterday, those that saw the post, apologies for saying the same thing here.

I went through $40,000 of savings to create this VFX tool and I’m open sourcing it as I continue. I launched openFirehawk.com and a Patreon page now to support the work.  Companies have spent a lot on this problem, so I'm excited that there is an open source collective path towards a good solution for us in houdini.

 

I’m thinking to be free of the VFX dungeon, cloud rendering has to get WAY better. It has a long way to go and it needs to get VERY VERY CHEAP. The cheaper it is, the more effective remote CG work can be, and that excites me. If it’s expensive it doesn’t interest me because, I just want to use as many cores as I can afford.

I started to learn DevOps to make cloud infrastructure start doing what I wanted it to as an FX Artist. I wanted to have direct control over the compute resources at the base cost and efficient processes around data handling.

I’d like it to help other VFX artists and studios too, to render as affordably as possible when it's robust enough. It could enable a different ecosystem that is more remote work friendly.

If what openFirehawk enables would affect you, or your choices in the future, or perhaps other mates, let them and me know! If you have buddy’s or know studios similarly aligned, share the work or talk about it to help support it to grow.

Edited February 20, 2019 by AndrewG
can't figure out how to embed video