Houdini in Linux Production Advice

[RickW]

Hey guys & gals,

First post here..

I know that some folks here work in production @ D2 and other places. I'm wondering what is done on linux that you can't do on other platforms? I'm already aware of the costs issues.

I ask this question because I've seen a couple of job posts for houdini artists in the past and they require linux experience. To the best of my knowledge houdini for linux and windows is the same no?

I also read some things that say "our pipeline uses python on linux which helps with the production process."

Can someone shed anylight as to what python is used for in production? Like what outside of the 3D Application is python/perl/linux used for? Converting ascii files from 1 app to another?

Don't get me wrong, I'm a linux user myself. I'm just wondering if I'm not taking advantage of linux as I should. I'm running Gentoo Linux with 2.5.x kernel and this bad boy smokes!

Thanks for your input,

/Rick

[old school]

There are huge differences between the two OS'es and I don't want to start a flame war. They both have pros and cons.

I use both Windows and Linux daily. Linux mainly at work, although I have made my workstation dual-boot. I use windows at home for software comaptibility with software that my wife needs to use. No AutoCAD for linux.

The biggest difference is that Linux is a true multi-user environment where several users can access a linux system simultaneously through numerous methods: rlogin, ssh, rsh, etc. Windows is a single user environment. It doesn't support properly rsh although the latest cygwin tools are smoothing over alot of the issues with the latest release for windows.

Try to remotely connect to a windows box that is at the login screen. Forget it. Linux, no problem.

Second biggest difference: Linux comes pre-installed with all the shells and the associated shell tools. csh, tcsh, xsh, sh, bash, etc.

Linux allows several shells, each with it's own unique environment. Windows does not come with this but you can install cygwin tools to recreate this environment, almost.

Stability is the third major issue, perhaps the biggest if you are a systems guy. The up times for our linux boxes rival that of the SGI's at Side Effects these days. My home system is pretty clean and has an average up-time of two to three days. If I am doing a lot of printing and publishing stuff, my up-time drops to 1-2 hours before a re-boot. My only negative experiences with Linux boxes are with immature graphics card drivers. Not a big issue these days but a couple of years ago, it was a different story.

Fourth issue is cost. Windows is staggering in costs. Linux is free. I have heard the arguement where windows requires fewer system administrators than linux, but this is certainly not true. Chances are both OS'es require the same effort to keep up. Windows System Administrators are at the mercy of the Microsoft tools. Linux System Administrators get to blame their own tools. Guess who can turn around a fix faster... Let's put it this way, Microsoft won't issue a patch for a piddly little 400 person SFX company but a 40,000 person company will. Linux provides a much more flexible implementation strategy that can be tailored to each company's philosopy, workflow and pipeline.

Windows has real good memory usage but so does Linux now. Linux tends to get real unstable once you fill up physical memory and are running out of swap and crashes. Windows also crashes when you run out of swap but crashes nicer giving you a pretty blue screen. Linux just gives you a black screen. Both are bad in this respect.

From a developer's standpoint, Windows has the best debugging tools. Linux has decent tools, comparable to SGI.

Windows attracts viruses quite nicely. Linux developers create the windows viruses and would never consider writing bugs to directly attack other linux systems. Like shooting fish in a barrel. I unleashed a windows virus on Side Effects a few years back. That was enough for to move to Linux permanently and adopt Mutt as my main mail package. I have had zero issues with my mail since I moved to Mutt.

There are so many great software packages readily available for Windows that are just not available for linux. One such package is Microsoft Office (Word, Excel, PowerPoint, etc.) which is quite expensive. Linux now has OpenOffice which is free and duplicates the functionality of Microsoft's Office tools. It can even open up Microsoft files. I am waiting for Microsoft to somehow wrap up the Digital Millenium Copyright Act to all files saved from Microsoft Office. If you haven't heard about this scary piece of US law, then just do a google search. If you use MSN search, then you are the fool because your searh habbits are recorded very nicely, thank-you very much.

And that segways in to my last point for this post: Microsoft have alterior motives other than selling you an OS. Their current strategy is to make your computing experience as natural as breathing. Searching for things even before you start to look, automatically updating your OS as you work. Linux is the exact opposite to this in many respects. It really wants to be an OS that is free and infinitely expandable.

Even Microsoft has recently eluded to the fact that the automatic update process introduced in XP is dangerous as 3rd party vendors could render certain features of your OS inoperative. Basically royally screw up your system then demand fees to "release" the feature. Imagine if Side Effects delivered Houdini to you with a great new feature called PLOPS, you used it and couldn't live without it then it timebombed and poped up a message after you downloaded the latest build "Please contact Side Effects to enable this feature". Trust me, this will happen on Windows, if it hasn't already.

[RickW]

Thanks for your response Old School.

Yes no flame wars please I just wanted info as to if I was missing out on features. I hope nobody gets offended by this thread as it's not my intention.

You bring up some very good points.

After reading this article I've been more convinced on the directions of M$ and am a bit concerned as well:

http://news.com.com/2100-1016-1007528.html?tag=nl

I am releived that I switched over to linux. I'm a control phreak and like to setup things the way I like it, not how the OS thinks I want it. The auto-update of windows drove me mad! Red Hat's up2date feature reminded me of that as well. Now I'm using Gentoo and I update what I want.

Like I said in my original post, I'm seeing a greater movement towards linux and It's even clearer now why from the point in your thread.

I'm still a bit confused though as to what is done outside of the 3D Application on linux? Perhaps this is not a linux question but more of a development one.

Are studios just building middleware tools with perl/python for the pipeline?

I'm still fairly new to linux and have a lot to learn. So far I've managed to compile my enitre OS and apps from source (not commercial 3d apps of course). It may not seem like a big deal but after using windows for many years, it's like a big accomplishment for me.

I've used houdini in the past, I believe it was on irix that time (can't remember) and I'm glad to see that the unix roots are still in houdini with the current version.

You mentioned remote computing (rlogin, ssh, rsh, etc). Can you shed any light on which protocol/apps are good for the linux environment for interacting with the desktop?

What I'm looking for is to remotely login to a linux box and fireup renders. I tried the vnc solution and found it a bit slow.

Thanks!

/Rick

[TheUsualAlex]

FOr your auto update feature, I highly recommend Ximian's Red Carpet. It's free. I like it better than RH's update feature in a way that I don't have to bother setting up an RHN account.

[RickW]

I'll keep that in mind when I do another RH install. It's good to know that there are other options. I didn't really care to create an account for the up2date applet. I might install RH on my laptop and possibly even on my mom's computer. <-That should be interesting.

For now on Gentoo, when I want to update software I just type "emere world -u" in the console and it'll download and compile the updated apps as per my gcc CFLAGS that I have setup, it's pretty slick.

/Rick

[old school]

I'm still a bit confused though as to what is done outside of the 3D Application on linux? Perhaps this is not a linux question but more of a development one.

A lot can be done outside of Houdini. There is hscript and it's ability to wrap it in to scripts to form pipelines. Depending on the studios and what applications they have to cater for in their pipeline, tools are created to generate data in particular formats. Sometimes they are just shell scripts and can be maintained by the TD. I have seen perl, python, java, and compiled tools in use to support various pipelines.

An ever increasing issue is asset tracking and data management software. Many studios have tens if not hundreds of terabytes of on-line storage now. You need specialized tools to manage that amount of data. It is bad enough me managing my teeny 20 Gig or so of hip files, reference images and other stuff. Hardly any sequences in there.

Render farm tools are another important piece in a facilities pipeline. How to render remotely most efficiency. I have yet to see two facilities do this the same. Each has their own take on what is best. In reality, they are all good solutions tailored to each own's way of networking machines and how they store the mass data.

You mentioned remote computing (rlogin, ssh, rsh, etc). Can you shed any light on which protocol/apps are good for the linux environment for interacting with the desktop?

In unix/linux, it is trivial to create mount points. If you don't know what a mount point is, it is the ability to have a directory, say /n/machine1 where machine1 is another machine's mass storage device (hard drive) that can be directly accessed as though it were your own and /n would be just a directory through arbitrary convention assigned to put all remote mount points in. With this, you can use these mount pointsin your hip files to reference data, and if they are consistent across all machines in the network, you can always find your files. Good if you are doing remote renders.

What I'm looking for is to remotely login to a linux box and fireup renders. I tried the vnc solution and found it a bit slow.

First off, does the remote machine have your machine mounted? If not then you will have to pass off all the data to the remote machine, then render, perfectly recreating the same environment to pick up files.

Can you tell me exactly how you are doing this?

[RickW]

Hi Old School,

Thanks for the heads up. I have 2 workstations here with me in NY and I've networked them no problemo via nfs. 1 of the workstations is a file server so I just mounted the share and all is good.

The problem I'm having is with a network offsite. I have 2 more workstations in Montreal which are hooked up to a 4.5 mbs up/down line. I'd like to be able to interact with them. I don't think I'd want to mount the shares on these computers since file transfers can take some time. I'd rather ftp the files over there and login remotely and interactively control the desktop and fireup renders on that local network.

Well that's my thinking anyways...

Perhaps you know of a better solution?

Thanks,

/Rick

[cs00bren]

hello there.

Do you know if you can get photoshop for linux, if you can is it any good compared to the windows copy.

cheers bren

[Wolfwood]

No Photoshop for Linux, sorry. Sometimes I get the feeling that if Adobe would port their software to Linux, windows would lose half its user base.

I know a few people who use VMWare just so they can use Adobe apps on Linux. Its an added cost of~$300, with the additional hidden cost of a Windows license.

If you can live without Photoshop there is 2 commercial packages, Amazon Paint and Idruna's Photogenics. If you are doing it on the cheap then there is the Gimp and CinePaint.

Of course there is always ipaint.

jim.

[edward]

What about PhotoPaint on Linux?

[MG]

What about PhotoPaint on Linux?

I don't like it at all! The port that is, since the core of the application is win32 API (WINE) emulated rather than ported. But also Photogenics didn't really impress me, other than it's high dynamic range capabilities and the many, high-end, image fileformats it supports.

And at the moment I really like The GIMP and FilmGimp.

[RickW]

hello there.

Do you know if you can get photoshop for linux, if you can is it any good compared to the windows copy.

cheers bren

I used gimp and Photoshop via vmware. There's also a cross over plugin located at:

Codeweavers

http://www.codeweavers.com/home/

I hear it runs ok with this plugin but I dunno, never tried it.

As far as vmware goes, I have 1.5 gig of ram so I give 1 gig to the guest OS and everything works ok. Although these days, I'm getting most of my 2d work done in Gimp. I doubt I'll ever toss Photoshop though.

/Rick

[calin_casian]

hi

Well RickW you can transfer files via ftp but you can't render on all 4 workstations without a virtual private network (vnc), it's slow because of the internet connection, you may have 4.5M/s(I don't think that you have constantly 4.5M/s) but the normal lan 10/100 NIC adaptors works at at least 10M/s. You can render on the Montreal workstations from NY via ssh and viceversa. So if you want speed I think the best solution is to transfer the files localy (in NY or Montreal) and fire up 2 renders.

see ya

calin

[RickW]

Hi Calin,

So if you want speed I think the best solution is to transfer the files localy (in NY or Montreal) and fire up 2 renders.

I ended up doing just that. I wrote a script that:

-I submit my render jobs to the server(local)

-the server logs the info to the mysql database

-the server packages all the files and sends them to the client (montreal)

-the client(s) renders all the files locally

-once the jobs are done I have another script that collects all the frames

It actually works pretty good for my needs. I'm using it to render mi2 files without a hitch.

(I don't think that you have constantly 4.5M/s) but the normal lan 10/100 NIC adaptors works at at least 10M/s.

Interesting. Everytime I checked it was about 4-4.5 m/s. I could be wrong. I'll have to check again and run some benchmarks. Either way, it's a pretty fast line.

Thanks for your input,

/Rick