sebkaine Posted October 18, 2015 Share Posted October 18, 2015 (edited) Hi Guys, I would like to know your experience about using HQUEUE under windows in real production scenario. - do you suceed to make it work properly ? - is it reliable tool in production ? I need to be sure that the tool works properly before wasting more trial / error / googling time to make it work. Does it worth this time or should i go straight to Deadline or Qube! ? I ask this because i fall on a huge number of post on odforce+sesi forum of users struggling for hours to make things work. I can keep struggling a day or two but not a week ... Thanks for your feedback ! Cheers E Edited October 18, 2015 by sebkaine Quote Link to comment Share on other sites More sharing options...
lukeiamyourfather Posted October 19, 2015 Share Posted October 19, 2015 HQueue "works" but I run it directly with the batch script instead of the service because it has permissions issues left and right when run as a service. It has a lot of little quirks that I've had to email support about at least a dozen times over the years. I'm tempted to try an open source queue manager like Afanasy which is meant for render farms or maybe something generic like Grid Engine. I've used Deadline and it was great up to about 100 nodes, at the time it wasn't designed to scale to many nodes and used text files and directories to manage jobs. This was Deadline 3 if I recall correctly. I've heard it uses a database now and can scale to more nodes and not choke. Quote Link to comment Share on other sites More sharing options...
sebkaine Posted October 19, 2015 Author Share Posted October 19, 2015 (edited) Thanks for your input Luke ! I finally make it work ... well at least on a small test render. - in fact you are right , the service is not working properly on windows - you must start each client manually I guess for a indie farm with 3 machines it's not a big deal but if you have a park of 100 machines it's not the same story ... But i don't think there is an IT on earth who is crazy enough to build a big renderfarm under windows ... When i will upgrade all the farm to H15 i will try to find the time to make a small video tutorial. It could definitly help the brave soul that want to build a small indie farm under windows ! For the moment i'll stick with hqueue and see how it goes ... Cheers E Edited October 19, 2015 by sebkaine 1 Quote Link to comment Share on other sites More sharing options...
old school Posted October 19, 2015 Share Posted October 19, 2015 Hi Emmanuel, Don't forget to wrap in Support when you can with Hqueue windows issues. Try submitting the Windows service HQUEUE start as an issue if you haven't already. Quote Link to comment Share on other sites More sharing options...
lukeiamyourfather Posted October 19, 2015 Share Posted October 19, 2015 Don't forget to wrap in Support when you can with Hqueue windows issues. Try submitting the Windows service HQUEUE start as an issue if you haven't already. I suggest having someone who isn't involved with the development of HQueue independently try to setup a farm using HQueue and take note of all the issues they encounter. Try it on machines that have never had Houdini or HQueue installed. In my experience HQueue doesn't work out of the box on any platform. On Linux the init.d script has broken paths, on Windows the service starts but is unable to actually do anything because it doesn't have permission to, and on OS X it uses localhost as the name for the client most of the time which breaks it with more than one client on the farm. I've found ways around these issues but it's very frustrating and time consuming. It feels as though nobody actually tests HQueue before it goes out because these are pretty obvious issues if you try to use the product. Other things about HQueue that have caused much grief, sometimes IFD preparation takes longer than the actual renders so I made my own script to submit existing IFD jobs, cross platform rendering doesn't work with existing IFD because the path for the output is in the IFD itself (e.g. D:/projects/renders/frame_0001.ifd), randomly jobs will return as successful after one second and the logs clearly show errors and exit code of an error. I sound very critical of HQueue and I don't want it to be a negative thing. It's a great tool and I'm glad HQueue exists and works most of the time but it's an area of Houdini that could use some more attention. It's not up to the high standards found in other parts of Houdini (this is a good problem to have!). Quote Link to comment Share on other sites More sharing options...
sebkaine Posted October 20, 2015 Author Share Posted October 20, 2015 (edited) Thanks for your input jeff, i will do a tutorial when i will upgrade to houdini 15, and i will highlight the windows services not able to get the proper right. i will also transmit this to support to give them a feedback. Thanks for your input Luke, I think it highlight something about why the prerender process is pretty slow ... At first i said to myself "damm this sqlite database take ages to dispatch my simple test scene" ... Actually i guess slowness as nothing to do with SQL read / write but more with .ifd creation and dispatching ... Edited October 20, 2015 by sebkaine Quote Link to comment Share on other sites More sharing options...
Netvudu Posted October 20, 2015 Share Posted October 20, 2015 a quick tut on HQueue will always be well received Emmanuel Quote Link to comment Share on other sites More sharing options...
Atom Posted December 31, 2015 Share Posted December 31, 2015 (edited) I am trying HQueue for the first time and I am having trouble setting it up on windows 7. Does HQueue work with Apprentice client/serers installs? I am running Apprentice and I have a couple other machines I would like to try to make a mini farm with. So with Apprentice I don't actually have a license server locally I used the other license option. (Is this Ok?) I have installed HServer on the same machine the I run Houdini Apprentice on and I see it running in the Windows Task Manager. Do I need a separate machine just for the server? I also have a machine on the network where I have installed the client, this machine also is a non-license server install. I see HClient running. How do I point this HClient to the Hserver? Do I need to install a license server? I get the impression the interface is some kind of web page, I can't find any app to view jobs. What is the URL I type into my browser to view the HQueue web page? Edited December 31, 2015 by Atom Quote Link to comment Share on other sites More sharing options...
sebkaine Posted January 1, 2016 Author Share Posted January 1, 2016 (edited) i don't know if HQUEUE work with apprentice, but it does work fine with indie under windows 7/10. if you have for exemple 3 machines i advise you to - use the same os everywhere - install houdini in license server mode NOT in local licensing everywhere (for indie user) - install the client on one machine for exemple foo1 - install the server on the three machines foo1 foo2 foo2 You also have to check that you windows network work fine - on one of your machines you had to build your share folder - you must check that you can access / read / write from any machine of the network - then you mount your share folder as a new drive in each machine as H:> for exemple at the install of HQUEUE it will ask for your server name which is foo1 and for your share folder which is H:> don't worry to much about this step it only write info to the .ini config file. you can modify those after ... So you now have - a server foo1 - 3 clients foo1 foo2 foo3 - one share folder H:> mounted on each machine - one install of houdini on 3 machines with the same os with a license server install for each one (at least in indie) Now you have to configure your server .ini file you have to enter - you server name foo1 - your share folder properties Then on each machine you will have to start each client manually it's a .bat that you have to run located in your HQUEUE client folder. This will force the permission restriction problem under windows. Now type foo1:5000 in a browser to see the HQUEUE panel , go to clients and see if you have 3 green light Now you can start to lunch a process, in the HQUEUE ROP - with render you have to uncheck the generate ifd checkbox or you will get an error (indie are not allowed to write ifd) Hope it will help and Happy new Year ! Cheers E Edited January 2, 2016 by sebkaine 5 Quote Link to comment Share on other sites More sharing options...
Pavel Posted January 2, 2016 Share Posted January 2, 2016 Hey guys, i am working on my student reel and i tried using Hqueue to render my projects. In my opinion this service is only viable if you have small cache files size to upload, otherwise it will take forever to upload the files. I tried to upload 30gb of files and it failed about 3 times and started all over every time. I have also tried to use dependencies, but it failed. Maybe because their default machine has 7gb of memory only. The other machines that have higher memory don`t work in my region (europe). Finally i got very frustrated and i gave up. I just couldn`t find a way to render a project with only 30gb of cache size. I also tried GridMarkets and they have much faster machines than the default 20EC2 from amazon. Their service is also much easier to use and you get 100$ free credit now, since they are in beta. That's my experience with cloud rendering. I hope it helps.Cheers,Pavel Quote Link to comment Share on other sites More sharing options...
Atom Posted January 4, 2016 Share Posted January 4, 2016 (edited) @sebkaine: Thank you for the info, I am still trying to setup a local render farm. You mention to install the server three times, once on each machine. This does not make sense to me. In traditional networking a server serves clients. Is that how HQUEUE works as well? I would assume you install the server on one machine and each node that is going to do the rendering becomes a client. On my main machine I can see hqclientservice.exe, hqserverservice.exe and hserver running. I call this the server, it hosts the shared folder as well. When I type localhost:5000 in a browser I do see the HQueue webpage. Jobs page reports Licenses Could Not Be Found. On another machine I have installed Houdini and HQUEUE Client. I can see HoudiniLicenseServer, HoudiniServer and HQueueClient all running. HQueueServer also shows up in the list but it is stopped. When I type localhost:5000 on the client machine I get no webpage at all? I can see the shared hq folder through the network but it is not drive mapped to the letter H: The goal I am looking to achieve is faster render times by using multiple machines. Will HQueue allow me to render single frames faster or does it just assign full frame renders to each client? When I try to issue a farm render from the client or server I get the same error as show in the image. Edited January 4, 2016 by Atom Quote Link to comment Share on other sites More sharing options...
LukeLetellier Posted January 7, 2016 Share Posted January 7, 2016 (edited) I've been using HQUEUE successfully on windows for about a year now with good success. Some notes to make the process easier: - Fixing the permissions is quite simple on WIndows. Open up your Services panel, and scroll dow until you see HQueueClient. Right Click --> Properties --> Log On Tab. Check the "This Account" box, and fill in your Windows username and password. Hit ok, and restart your client. Done! HQueue Client will start up auromatically on boot-up, and it'll work! - If you're serious about using a render-farm setup with Houdini, I'd definitely recommend getting A Synology Server or some other kind of networked hard drive. Some primary benefits: - Your shared folder can be located here. You install Houdini into this location, and each render node will use this installation to do the work. So if a new production build is available, you install it just to this location, and all the nodes benefit. - Your Houdini project files can be hosted here, for easy access by all the clients (also, your destination files can be sent directly to your project folder here instead of going into a shared folder) - .bgeo files take up space, so lots of redundant HDD space is always a help! - Instead of using localhost:5000 - set up the ini files to use the fixed IP of the server. Edited January 7, 2016 by LukeLetellier 1 Quote Link to comment Share on other sites More sharing options...
Atom Posted January 8, 2016 Share Posted January 8, 2016 (edited) Ok, I keep seeing mention of this .INI file but where is it located? What do I change in the .INI? I am also still unclear about what gets installed where. Does anyone know why I would not be able to connect, as shown in the image posted above? I have complete network connectivity between the two machines but they just won't connect. Edited January 8, 2016 by Atom Quote Link to comment Share on other sites More sharing options...
LukeLetellier Posted January 9, 2016 Share Posted January 9, 2016 Ok, I keep seeing mention of this .INI file but where is it located? What do I change in the .INI? C Drive -->HQueueClient --> hqnode.ini C Drive -->HQueueServerClient --> hqserver.ini Open them up in a free code-editing program like SublimeText. The client ini file is quite straightforward, as there are only a few lines of code: where it says, server, you input the IP of the machine hosting your server (192.168.X.X). For the sharedNetwork.mount, you put the IP address & name of the shared folder: \\192.168.x.x\MySharedFolder The server.ini file is a bit more complicated; take a look at the HQueue documentation for help on this one, and if you're still having issues, send in a support email and have SESI help you out (I used both methods). Quote Link to comment Share on other sites More sharing options...
sebkaine Posted January 21, 2016 Author Share Posted January 21, 2016 (edited) OK after 1 day and a Half of fight i finally suceed to install Houdini 15 HQUEUE on windows 10. I have assemble a detail pdf of all the process. It was supposed to be for SESI support , but i finally get it work. Luke the service trick doesn't work on windows 10 see my .pdf for more detail. I must confess that seting up HQUEUE is really a huge PITA. Cheers E EDIT : I have updated the file to simplify and add some correction. hqueue_windows_10.pdf Edited January 21, 2016 by sebkaine 5 Quote Link to comment Share on other sites More sharing options...
LukeLetellier Posted January 23, 2016 Share Posted January 23, 2016 Luke the service trick doesn't work on windows 10 see my .pdf for more detail. It worked for me on Windows 10. *shrug* Glad that things are working for you! Quote Link to comment Share on other sites More sharing options...
Mdonovan Posted July 16, 2016 Share Posted July 16, 2016 (edited) This thread saved my ass. Spent the day banging my head on a rock until I found this thread. SideFX really needs to make the install for this thing much more intuitive. Services should work. The most annoying thing I have come across is that when you submit a slice job , if one of your clients does not pick up and your slice number != client number ... the job just takes a dump and hangs. It should continue anyway. When it works though .. it is a thing of beauty. Edited July 16, 2016 by Mdonovan Quote Link to comment Share on other sites More sharing options...
Paleo Pixels Posted July 20, 2017 Share Posted July 20, 2017 (edited) Hi Also stuck here, Windows 7 user, 2x Houdini Indie. Locally it works (client on server pc), but the second machine still gives "ERROR: Cannot open file etc" This is with Target HFS set to a local path (identical on both machines) If Target HFS is set to $HQROOT/houdini_distros/hfs.$HQCLIENTARCH the error is this "invalid username of wrong password". Users on both machines are the same name, but the passwords differ. Could that be it? Firewalls are off on both machines. No virusscanners. I tried DNSname of server pc and IP. Server ini: # The shared network. hqserver.sharedNetwork.host = Paleopixel1 hqserver.sharedNetwork.path.linux = %(here)s/shared hqserver.sharedNetwork.path.windows = hqueueprojects hqserver.sharedNetwork.path.macosx = %(here)s/HQShared hqserver.sharedNetwork.mount.linux = /mnt/hq hqserver.sharedNetwork.mount.windows = V: hqserver.sharedNetwork.mount.macosx = /Volumes/HQShared # Server port number. hqserver.port = 5000 Client ini [main] server = Paleopixel1 port = 5000 sharedNetwork.mount = \\Paleopixel1\hqueueprojects [job_environment] Client services on both machines are run with logons for current user. So,local client works fine, other pc client returns the 'cannot open file'. In the mount V: both machines can read/write. After a couple of days I'm really stuck, any advice would be much appreciated. Bjorn Edited July 20, 2017 by Paleo Pixels Quote Link to comment Share on other sites More sharing options...
lukeiamyourfather Posted July 20, 2017 Share Posted July 20, 2017 Try the "EnableLinkedConnections" registry edit on Windows. If you do a search for it there should be many results. Quote Link to comment Share on other sites More sharing options...
Paleo Pixels Posted July 21, 2017 Share Posted July 21, 2017 Thank you Luke, will try that tonight Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.