Welcome to od|forum

Register now to gain access to all of our features. Once registered and logged in, you will be able to contribute to this site by submitting your own content or replying to existing content. You'll be able to customize your profile, receive reputation points as a reward for submitting content, while also communicating with other members via your own private inbox, plus much more! This message will be removed once you have signed in.

sebkaine

Is HQUEUE for Indie a production ready tool under windows ?

Hi Guys,

 

I would like to know your experience about using HQUEUE under windows in real production scenario.

- do you suceed to make it work properly ?

- is it reliable tool in production ?

 

I need to be sure that the tool works properly before wasting more trial / error / googling time to make it work.

Does it worth this time or should i go straight to Deadline or Qube! ?

 

I ask this because i fall on a huge number of post on odforce+sesi forum of users struggling for hours to make things work.

I can keep struggling a day or two but not a week ... :)

 

Thanks for your feedback !

 

Cheers

 

E

Edited by sebkaine

Share this post


Link to post
Share on other sites

HQueue "works" but I run it directly with the batch script instead of the service because it has permissions issues left and right when run as a service. It has a lot of little quirks that I've had to email support about at least a dozen times over the years. I'm tempted to try an open source queue manager like Afanasy which is meant for render farms or maybe something generic like Grid Engine.

 

I've used Deadline and it was great up to about 100 nodes, at the time it wasn't designed to scale to many nodes and used text files and directories to manage jobs. This was Deadline 3 if I recall correctly. I've heard it uses a database now and can scale to more nodes and not choke.

Share this post


Link to post
Share on other sites

Thanks for your input Luke !

 

I finally make it work ... well at least on a small test render.

- in fact you are right , the service is not working properly on windows 

- you must start each client manually

 

I guess for a indie farm with 3 machines it's not a big deal but if you have a park of 100 machines it's not the same story ...

But i don't think there is an IT on earth who is crazy enough to build a big renderfarm under windows ...

 

When i will upgrade all the farm to H15 i will try to find the time to make a small video tutorial.

It could definitly help the brave soul that want to build a small indie farm under windows !

 

For the moment i'll stick with hqueue and see how it goes ...

 

Cheers

 

E

Edited by sebkaine
1 person likes this

Share this post


Link to post
Share on other sites

Hi Emmanuel,

 

Don't forget to wrap in Support when you can with Hqueue windows issues. Try submitting the Windows service HQUEUE start as an issue if you haven't already.

Share this post


Link to post
Share on other sites

Don't forget to wrap in Support when you can with Hqueue windows issues. Try submitting the Windows service HQUEUE start as an issue if you haven't already.

 

I suggest having someone who isn't involved with the development of HQueue independently try to setup a farm using HQueue and take note of all the issues they encounter. Try it on machines that have never had Houdini or HQueue installed.

 

In my experience HQueue doesn't work out of the box on any platform. On Linux the init.d script has broken paths, on Windows the service starts but is unable to actually do anything because it doesn't have permission to, and on OS X it uses localhost as the name for the client most of the time which breaks it with more than one client on the farm. I've found ways around these issues but it's very frustrating and time consuming. It feels as though nobody actually tests HQueue before it goes out because these are pretty obvious issues if you try to use the product.

 

Other things about HQueue that have caused much grief, sometimes IFD preparation takes longer than the actual renders so I made my own script to submit existing IFD jobs, cross platform rendering doesn't work with existing IFD because the path for the output is in the IFD itself (e.g. D:/projects/renders/frame_0001.ifd), randomly jobs will return as successful after one second and the logs clearly show errors and exit code of an error. I sound very critical of HQueue and I don't want it to be a negative thing. It's a great tool and I'm glad HQueue exists and works most of the time but it's an area of Houdini that could use some more attention. It's not up to the high standards found in other parts of Houdini (this is a good problem to have!).

Share this post


Link to post
Share on other sites

Thanks for your input jeff,

 

i will do a tutorial when i will upgrade to houdini 15, and i will highlight the windows services not able to get the proper right.

i will also transmit this to support to give them a feedback.

 

Thanks for your input Luke,

 

I think it highlight something about why the prerender process is pretty slow ...

At first i said to myself  "damm this sqlite database take ages to dispatch my simple test scene" ...

 

Actually i guess slowness as nothing to do with SQL read / write but more with .ifd creation and dispatching ... :)

Edited by sebkaine

Share this post


Link to post
Share on other sites

I am trying HQueue for the first time and I am having trouble setting it up on windows 7.

Does HQueue work with Apprentice client/serers installs?

I am running Apprentice and I have a couple other machines I would like to try to make a mini farm with.

So with Apprentice I don't actually have a license server locally I used the other license option. (Is this Ok?)

I have installed HServer on the same machine the I run Houdini Apprentice on and I see it running in the Windows Task Manager.

Do I need a separate machine just for the server?

I also have a machine on the network where I have installed the client, this machine also is a non-license server install. I see HClient running.

How do I point this HClient to the Hserver?

Do I need to install a license server?

I get the impression the interface is some kind of web page, I can't find any app to view jobs.

What is the URL I type into my browser to view the HQueue web page?

Edited by Atom

Share this post


Link to post
Share on other sites

i don't know if HQUEUE work with apprentice, but it does work fine with indie under windows 7/10.

 

if you have for exemple 3 machines i advise you to

- use the same os everywhere

- install houdini in license server mode NOT in local licensing everywhere (for indie user)

- install the client on one machine for exemple foo1

- install the server on the three machines foo1 foo2 foo2

 

You also have to check that you windows network work fine

- on one of your machines you had to build your share folder

- you must check that you can access / read / write from any machine of the network

- then you mount your share folder as a new drive in each machine as H:> for exemple

 

at the install of HQUEUE it will ask for your server name which is foo1

and for your share folder which is H:>

don't worry to much about this step it only write info to the .ini config file.

you can modify those after ...

 

So you now have

- a server foo1

- 3 clients foo1 foo2 foo3

- one share folder H:> mounted on each machine

- one install of houdini on 3 machines with the same os with a license server install for each one (at least in indie)

 

Now you have to configure your server .ini file you have to enter

- you server name foo1

- your share folder properties

 

Then on each machine you will have to start each client manually it's a .bat that you have to run located in your HQUEUE client folder.

This will force the permission restriction problem under windows.

 

Now type foo1:5000 in a browser to see the HQUEUE panel , go to clients and see if you have 3 green light

 

Now you can start to lunch a process, in the HQUEUE ROP

- with render you have to uncheck the generate ifd checkbox or you will get an error (indie are not allowed to write ifd)

 

Hope it will help and Happy new Year ! :)

 

Cheers

 

E

Edited by sebkaine
4 people like this

Share this post


Link to post
Share on other sites

Hey guys, i am working on my student reel and i tried using Hqueue to render my projects. In my opinion this service is only viable if you have small cache files size to upload, otherwise it will take forever to upload the files. I tried to upload 30gb of files and it failed about 3 times and started all over every time. I have also tried to use dependencies, but it failed. Maybe because their default machine has 7gb of memory only. The other machines that have higher memory don`t work in my region (europe). Finally i got very frustrated and i gave up. I just couldn`t find a way to render a project with only 30gb of cache size. I also tried GridMarkets and they have much faster machines than the default 20EC2 from amazon. Their service is also much easier to use and you get 100$ free credit now, since they are in beta. 

That's my experience with cloud rendering. I hope it helps.

Cheers,
Pavel

Share this post


Link to post
Share on other sites

@sebkaine: Thank you for the info, I am still trying to setup a local render farm.

You mention to install the server three times, once on each machine. This does not make sense to me. In traditional networking a server serves clients. Is that how HQUEUE works as well? I would assume you install the server on one machine and each node that is going to do the rendering becomes a client.

 

On my main machine I can see hqclientservice.exe, hqserverservice.exe and hserver running. I call this the server, it hosts the shared folder as well.

When I type localhost:5000 in a browser I do see the HQueue webpage. Jobs page reports Licenses Could Not Be Found.

 

On another machine I have installed Houdini and HQUEUE Client. I can see HoudiniLicenseServer, HoudiniServer and HQueueClient all running. HQueueServer also shows up in the list but it is stopped.

When I type localhost:5000 on the client machine I get no webpage at all?

 

I can see the shared hq folder through the network but it is not drive mapped to the letter H:

 

The goal I am looking to achieve is faster render times by using multiple machines.

Will HQueue allow me to render single frames faster or does it just assign full frame renders to each client?

 

When I try to issue a farm render from the client or server I get the same error as show in the image.

post-12295-0-83273800-1451922800_thumb.j

Edited by Atom

Share this post


Link to post
Share on other sites

I've been using HQUEUE successfully on windows for about a year now with good success. Some notes to make the process easier:

 

- Fixing the permissions is quite simple on WIndows. Open up your Services panel, and scroll dow until you see HQueueClient. Right Click --> Properties --> Log On Tab. Check the "This Account" box, and fill in your Windows username and password. Hit ok, and restart your client. Done!  HQueue Client will start up auromatically on boot-up, and it'll work!

 

- If you're serious about using a render-farm setup with Houdini, I'd definitely recommend getting A Synology Server or some other kind of networked hard drive. Some primary benefits:

   - Your shared folder can be located here. You install Houdini into this location, and each render node will use          this installation to do the work. So if a new production build is available, you install it just to this location, and all       the nodes benefit.

   - Your Houdini project files can be hosted here, for easy access by all the clients (also, your destination files can       be sent directly to your project folder here instead of going into a shared folder)

   - .bgeo files take up space, so lots of redundant HDD space is always a help!

 

 

- Instead of using localhost:5000 - set up the ini files to use the fixed IP of the server.

Edited by LukeLetellier
1 person likes this

Share this post


Link to post
Share on other sites

Ok, I keep seeing mention of this .INI file but where is it located?

What do I change in the .INI?

 

I am also still unclear about what gets installed where.

 

Does anyone know why I would not be able to connect, as shown in the image posted above?

I have complete network connectivity between the two machines but they just won't connect.

Edited by Atom

Share this post


Link to post
Share on other sites

Ok, I keep seeing mention of this .INI file but where is it located?

What do I change in the .INI?

 

 

C Drive -->HQueueClient --> hqnode.ini

C Drive -->HQueueServerClient --> hqserver.ini

 

Open them up in a free code-editing program like SublimeText. 

 

The client ini file is quite straightforward, as there are only a few lines of code: where it says, server, you input the IP of the machine hosting your server (192.168.X.X). For the sharedNetwork.mount, you put the IP address & name of the shared folder: \\192.168.x.x\MySharedFolder

 

The server.ini file is a bit more complicated; take a look at the HQueue documentation for help on this one, and if you're still having issues, send in a support email and have SESI help you out (I used both methods).

Share this post


Link to post
Share on other sites

OK after 1 day and a Half of fight i finally suceed to install Houdini 15 HQUEUE on windows 10.

I have assemble a detail pdf of all the process. It was supposed to be for SESI support , but i finally get it work.

 

Luke the service trick doesn't work on windows 10 see my .pdf for more detail.

 

I must confess that seting up HQUEUE is really a huge PITA. :) 

 

Cheers 

 

E

 

EDIT :

I have updated the file to simplify and add some correction.

hqueue_windows_10.pdf

Edited by sebkaine
5 people like this

Share this post


Link to post
Share on other sites

Luke the service trick doesn't work on windows 10 see my .pdf for more detail.

 

 

 It worked for me on Windows 10. *shrug* 

 

Glad that things are working for you! :)

Share this post


Link to post
Share on other sites

This thread saved my ass. Spent the day banging my head on a rock until I found this thread.  

SideFX really needs to make the install for this thing much more intuitive. Services should work.

The most annoying thing I have come across is that when  you submit a slice job , if one of your clients does not pick up and your slice number != client number ... the job just takes a dump and hangs. It should continue anyway.

When it works though .. it is a thing of beauty.

Edited by Mdonovan

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now