Jump to content
slayerk

ROP Operator Submit Houdini2Deadline

Recommended Posts

Advantages:

1. Can work with other ROP nodes. As Example it can be called from Wedge ROP for automatic multi job submission. (for different takes or for automatic parameter change).
2. May submit any ROP node. DYNAMIC_ROP or GEOMETRY_ROP. Not only Mantra ROP.
3. All Submit Settings Saved in scene file and are the attributes of the scene. You may open this scene after some yars and don't need remember some attributes which was rendered a scene in the last time. Also not needed to save submit setting in other places.

 

Installation:
1. put otl-file: houdini2deadline_v01.otl in standart user houdini otl storage:

on Windows: %USERPROFILE%/Documents/houdini13.0/otls)

on Linux: ~/houdini13.0/otls

2. save python script houdini2deadline.py anywhere

3. open  Houdini and navigate /out

4. open tab-menu and choose Farm->Houdini2Deadline

5. in Parameters of houdini2deadline node "Program" field set correct path to houdini2deadline.py python script (by default this field set to "$TOOLS/scripts/python/houdini2deadline.py")

6. May set this settings as default with "Gear" menu on node. Or change default values via RMB->Type Proprties...->Parameters Tab select desired parameter, go "Channels" tab and set default value.

 

This is beta version. Any bug report and suggestions are welcome.

houdini2deadline_v01.otl

houdini2deadline.rar

post-3849-0-68048600-1411980948_thumb.jp

Edited by slayerk
  • Like 1

Share this post


Link to post
Share on other sites

Hi Igor,

 

I tried it and it seems to error on the groups/pools fields and on submission (I'm on Linux).

 

Funny timing, I've been developing a houdini to deadline submitter for quite some time, it's called submitToDeadline and is going to be made available real soon (stay tuned!).

SubmitToDeadline supports all common rops, multiple submissions and complex dependencies, bypasses etc. It's quite similar in features to the proprietary solutions seen at highend vfx studios in London (but simpler to use):

http://simultools.com/submittodeadline

 

Video illustrating the main features can be found here:

https://www.youtube.com/watch?v=iJNoxD1hvI8

 

Thanks
L

Edited by Lucavfx

Share this post


Link to post
Share on other sites

Hi Igor,

 

I tried it and it seems to error on the groups/pools fields and on submission (I'm on Linux).

 

Funny timing, I've been developing a houdini to deadline submitter for quite some time, it's called submitToDeadline and is going to be made available real soon (stay tuned!).

SubmitToDeadline supports all common rops, multiple submissions and complex dependencies, bypasses etc. It's quite similar in features to the proprietary solutions seen at highend vfx studios in London (but simpler to use):

http://simultools.com/submittodeadline

 

Video illustrating the main features can be found here:

https://www.youtube.com/watch?v=iJNoxD1hvI8

 

Thanks

L

very-very powerful tool :))) that is what i'm search but not found. Only one flaw - NOT FREE. :)))

 

on linux problem: I don't test it on linux but think problem in calling "deadlinecommand". When i install deadline on windows it's automatically updates PATH variable. I think now i must add field in asset like: "Path to deadline command" with path to executables. if simply call "deadlinecommand <parameters>" not work then need to use this path by user. You may correct call to right way calling "deadlinecommand" houdini2deadline node->RMB->Type properties->Parameters Tab->Click on "Pool" parameter and go to Menu Tab. In menu script string: "stdout = os.popen( "deadlinecommand Pools" )". I think it's not work if path to "deadlinecommand" not in PATH variable.

post-3849-0-96984700-1412103252_thumb.jp

Edited by slayerk

Share this post


Link to post
Share on other sites

Thanks for the nice words, I think it is placed at a very competitive price for small/middle sized studios (the cost for the time of a pipelineTD building something similar would be many times higher).

BTW it is now live at the orbolt website https://www.orbolt.com/asset/Simultools::submitToDeadline

 

Have you tried using the environment variable DEADLINE_PATH to construct the path to the deadline executable?

that env variable should always be present whether you have a local installation of the deadline client or a remote installation. That is pretty much what I use in my olt.

 

 

very-very powerful tool :))) that is what i'm search but not found. Only one flaw - NOT FREE. :)))

 

on linux problem: I don't test it on linux but think problem in calling "deadlinecommand". When i install deadline on windows it's automatically updates PATH variable. I think now i must add field in asset like: "Path to deadline command" with path to executables. if simply call "deadlinecommand <parameters>" not work then need to use this path by user. You may correct call to right way calling "deadlinecommand" houdini2deadline node->RMB->Type properties->Parameters Tab->Click on "Pool" parameter and go to Menu Tab. In menu script string: "stdout = os.popen( "deadlinecommand Pools" )". I think it's not work if path to "deadlinecommand" not in PATH variable.

Edited by Lucavfx

Share this post


Link to post
Share on other sites

Thanks for the nice words, I think it is placed at a very competitive price for small/middle sized studios (the cost for the time of a pipelineTD building something similar would be many times higher).

BTW it is now live at the orbolt website https://www.orbolt.com/asset/Simultools::submitToDeadline

 

Have you tried using the environment variable DEADLINE_PATH to construct the path to the deadline executable?

that env variable should always be present whether you have a local installation of the deadline client or a remote installation. That is pretty much what I use in my olt.

No. Don't Use this variable. Thanks for tip.

Share this post


Link to post
Share on other sites

UPDATE:

 

1. Perfomance improvements: content of "Pools" and "Groups" menus creates once - right after "houdini2deadline" node creation via "OnCreated" callback Script. (in previous version this menus updates via menu callback script - spent a lot of time every time you interact with node ( selecting it and etc...).

2. Some submission script edits, based on working expirience.

3. Machine list now may be constructed from triangle menu button. menu of slaves updated by clicking "UPDATE SLAVES" button.

Dependences may be constructed from jobs by current user (

deadlinecommand getcurrentusername // getting name of user for jobs filtering
deadlinecommand -getjobidsfilter username="name_of_user" // getting current user jobs

). also list of jobs may be updated by "UPDATE MY JOBS" button.

4. Adapted for H15

 

Work in progress

Tested only under windows

Please report problems and bug reports

houdini2deadline.rar

houdini2deadline_v01.otl

post-3849-0-58766400-1450203632_thumb.pn

Edited by slayerk
  • Like 1

Share this post


Link to post
Share on other sites

When creating the ROP node:

 

Error running event handler:
Traceback (most recent call last):
  File "opdef:/Driver/houdini2deadline?OnCreated", line 6, in <module>
IndexError: list index out of range
 

Machine List does not update.

 

Image attached of error when attempting to Render.

 

Using Houdini 15.0.244.16.

 

As an aside, does this handling Sims in Deadline properly or does it still have the problem of Deadline simming every frame up to the current one rendering, reulsting in exponentially longer render times for each frame?

 

 

post-11063-0-42553900-1450761666_thumb.p

Edited by thestuntman

Share this post


Link to post
Share on other sites

When creating the ROP node:

 

Error running event handler:

Traceback (most recent call last):

  File "opdef:/Driver/houdini2deadline?OnCreated", line 6, in <module>

IndexError: list index out of range

 

Machine List does not update.

 

Image attached of error when attempting to Render.

 

Using Houdini 15.0.244.16.

It cannot find deadlinecommand executable. maybe path to it not added to PATH variable. 

Linux? Windows?

 

 

As an aside, does this handling Sims in Deadline properly or does it still have the problem of Deadline simming every frame up to the current one rendering, reulsting in exponentially longer render times for each frame?

This internal deadline Houdini plugin problem. need to edit houdini plugin in <Deadline Repository>/Plugins/Houdini/hrender_dl.py. I think in thinkbox spent a very tiny time to write this plugin. very raw and buggy. dont correctly proceed render and display render progress. 

Share this post


Link to post
Share on other sites

It cannot find deadlinecommand executable. maybe path to it not added to PATH variable. 

Linux? Windows?

This internal deadline Houdini plugin problem. need to edit houdini plugin in <Deadline Repository>/Plugins/Houdini/hrender_dl.py. I think in thinkbox spent a very tiny time to write this plugin. very raw and buggy. dont correctly proceed render and display render progress. 

Hi

You should definitely voice your concerns on the Thinkbox forum. I find the thinkbox-guys to be very open to suggestions regarding their submission scripts and rendering plugins. 

 

-b

Share this post


Link to post
Share on other sites

Hi

You should definitely voice your concerns on the Thinkbox forum. I find the thinkbox-guys to be very open to suggestions regarding their submission scripts and rendering plugins. 

 

-b

Yeah, yeah. Already done at a time when there was a relevant fifth version of Deadline. Only feature (one of more I requested) of 7-th deadline is to parse ALF_PROGRESS houdini output that operates with errors now  :(((((. Main conception of sending job via popup menu (a-la 3dsmax, not via node) still in use. Implementation via node  entails parsing dependences of ROP node networks, possibility of use WEDGE Rop and etc. I think it's problem and developer dont wants to spent a time for rarely used 3d software like Houdini.

Edited by slayerk

Share this post


Link to post
Share on other sites

I've had thinkbox adding a variable called IGNORE_INPUTS ( i think that's how it is called, I'm not in front of my PC) years ago to resolve this issue, no need to edit any deadline installation python files!

this is what I use in my submitToDeadline to prevent any rop from cooking any of its inputs.

obviously this would require you to handle the rop dependencies manually via code.

this is handy if you intend building your own dag algorythm for parsing the rop branch.

that's what submitToDeadline does internally.

L

Share this post


Link to post
Share on other sites

Update:

1. No need to install additional script. now It's incapsulated in otl.

2. If Asset cannot find executable of deadlinecommand.exe it will show a file chooser field called "Deadline Command" on top of parameters (see image below). and you may navigate to correct location of this executable. Also you may set correct value by editing defaults via RMB on asset ->Type Properties->Parameters->deadlineCommand->channels. When this parameter locates to correct executable it will hiding automatically

 

WIP. Any bugs may be present. Need feedback on it.

 

 

Planned:

1. Resolve dependencies of complex ROP Networks

2. Support for generating IFD and rendering via mantra.

3. Making custom qt widgets for selecting jobs and machines in blacklist/whitelist.

houdini2deadline_v01.otl

post-3849-0-76324200-1450913726_thumb.pn

Edited by slayerk
  • Like 1

Share this post


Link to post
Share on other sites

update:

1. New "Engeneering Tab" where you can test some Submitter Functions. And also disabled checkboxes show is deadlinecommand executable correct or not.

 

2. Add Support to Sending Mantra Jobs. If Mantra ROP in "ROP Path" Parameter setted to export IFD you have possibility to automatically send dependent Mantra job to render that IFD Files ("Automatically Send Mantra Job" Parameter).

 

post-3849-0-54653000-1451090015_thumb.pn

houdini2deadline_v02.hda

  • Like 2

Share this post


Link to post
Share on other sites

works for distributed sims?

Not tested.

As I remember you must start sim tracker for distributed sims. You Can start It manually.

Also You may use Houdini2Deadline in combination with Wedge ROP (to change SLICE Variable).

  • Like 1

Share this post


Link to post
Share on other sites

Hi, this is awesome. Just got it working on a mac enviroment.

 

One quick question, I am getting the date_time_ on the rendered files. Any ideas how to disable this?

Share this post


Link to post
Share on other sites

Hi,

 

I am using the HIPNAME token on the file output.

 

I think it's because it's making a copy of the hip file on the server, and adding a few details to hipname (date, time, jobnumber)

 

So if my HIPNAME is test01

 

when it gets to render it outputs:

 

test01_2016-04-13_11_21_45.455831

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×