Using Soho: Indigo Renderer Support

[Jason]

Day Six - Making an Indigo ROP

Up until now I've been using a repurposed Mantra ROP as the Indigo ROP - but now it's time to build one that comes in populated with all the parameters that I want on my default ROP. This is actually a quite awkward process unfortunately, as there a few inconvenient exceptions to the general rule with respect to ROP OTLs and their Creation scripts. And this is information I had to glean directly from SESI as it's undocumented or mis-documented and just plain silly (IMHO)

So, the general idea is that after the ROP is placed, a Creation Script gets run which populates all the Properties on the ROP. Quite simple idea, really, made complicated. This mechanism for normal nodes involves having a script on disk such as $HIH/scripts/obj/cam.cmd. For HDAs inside of OTLs, this is usually embedded inside as a CreateScript, editable in the Type Properties GUI. AFAIK, these are usually able to be overridden by any external .cmds in the appropriate location too (I could be wrong).

The documentation mentions that there is a $HIH/scripts/out/ifd.cmd which is meant to be the creation script for the Mantra (ifd) ROP. Don't look for it, it's not really there. Instead, this file has been sucked into the OPlibDriver.otl and renamed as the internal "CreateScript" probably when SESI build the Houdini distribution. You might now correctly expect to find this in the Type Properties of the Mantra ROP, but for some reason it is not visible. Where is it? Don't give up, there are more flaming hoops ahead!

In order to gain access to this script you must bust open the OTL from the commandline using "hotl -x <dir> <otlfile>" and then you'll see in that directory structure a file called "CreateScript". Ahh gold!

In here you'll find this line:

opproperty -f -F Properties $arg1 mantra9.1 default_output

.. and what this means is that it'll create Properties on the current ROP for mantra from the parameter list called "default_output". So where is this parameter list defined? It's defined in the SOHOparameters' Indigo.ds file. While looking at the distributed Mantra .ds files I saw that these are defined as special parameters with the tag "__spare_parmlist". Mimicking this, I could add my own for the Indigo default parms:

	parm {
		name	default_output
		label	"Default Output"
		parmtag	{ spare_category "__spare_parmlist" }
		parmtag	{ spare_parmlist "i_large_mutation_prob i_max_change i_russian_roulette_live_prob i_metropolis i_logging i_strata_width i_white_balance i_tonemapping_method i_tonemapping_linear_scale i_tonemapping_reinhard_prescale i_tonemapping_reinhard_postscale i_tonemapping_reinhard_burn" }
	}

Okay, after collapsing the OTL back from the component files (using hotl -C) - and the freshly edited CreateScript line to add indigo versus mantra9.1 parms, I now can create new Indigo ROPs with my own parameters added.

If I want to add or remove new default parms its just a matter of editing the Indigo.ds file and hopefully I won't have to ever break open the OTL again.

Note A word of warning - if you've modified the parameters on a ROP with the Type Properties GUI, it'll replace the nice opproperty line in the CreateScript with a huge honkin' opspareds line and it'll no longer listen to the default_output ever again and the only way to ever revert to that is you break open the OTL again and replacing the opspareds with the nice opproperty line.

So besides doing all this, I fixed some bad math of mine in the Camera export code and did some other stuff I can't remember right now.

I think I'll release an alpha of this thing tonight, but unfortunately I cannot get Indigo to run (under wine) on SUSE 9.3 so I cannot test any linux stuff. I wanted to make a wrapper that works for both platforms, but perhaps I'll do that after a round of an alpha release.

Cheers,

Jason

PS. I really hope that SESI could see to make this part of the process simpler and more consistent with the rest of Houdini's workflow

[edward]

Note A word of warning - if you've modified the parameters on a ROP with the Type Properties GUI, it'll replace the nice opproperty line in the CreateScript with a huge honkin' opspareds line and it'll no longer listen to the default_output ever again and the only way to ever revert to that is you break open the OTL again and replacing the opspareds with the nice opproperty line.

I think this is only done if you have "Save Spare Parameters" enabled on the first page of the Type Properties dialog. There's been some discussion that this should be off by default.

PS. I really hope that SESI could see to make this part of the process simpler and more consistent with the rest of Houdini's workflow

Recall that these internal sections were exposed in the (now removed) Edit Contents... dialog from the Operator Type Manager window prior to H9. The problem with the Edit Contents... dialog was that people would constantly open it up to add their own user-defined sections to it. However, that dialog _and_ the Type Properties dialog both to resaved to the same .otl file (writing to the same internal sections). Thus, users frequently lost data due when they had both the dialogs opened and then saving from one could lose the data saved from the other dialog. To address this error-prone UI issue, the Edit Contents... dialog was incorporated (more or less) into the Type Properties dialog in H9 (the new Extra Files tab). However, since the Type Properties dialog directly creates/modifies those internal sections as well, the semantics weren't clear with them exposed. I'm on your side though, I think that there should still be some way for advanced users to modify them in the UI but then advanced users also are more trusted to use hotl.

BTW, "SESI" doesn't officially read the forums. If you want something that "SESI" should notice (eg. log an RFE), it should be sent to support. Same goes for the General/Technical Discussion forums at sidefx.com as well. Bugs can be logged directly through the Submit H9 bugs link there as well.

[Jason]

I think this is only done if you have "Save Spare Parameters" enabled on the first page of the Type Properties dialog. There's been some discussion that this should be off by default.

Yes, I agree - it really plays havoc when building OTLs because there is no GUI indication of whether a parameter is a Spare parameter or a bona-fide parameter. Perhaps if they were tinted differently or marked (like the little Default check-mark) or something it's be clearer - but they aren't and it just serves to confuse right now.

To address this error-prone UI issue, the Edit Contents... dialog was incorporated (more or less) into the Type Properties dialog in H9 (the new Extra Files tab). However, since the Type Properties dialog directly creates/modifies those internal sections as well, the semantics weren't clear with them exposed.

Exactly right - it seems to me that this CreateScript should appear in the Extra Files section. Do you know why it doesn't? All these other scripts show up without a problem. Is it filtered out like the DialogScript section? It seems that the CreateScript doesn't have an alternate way of being edited in the same way that the DialogScript has the Parameters tab. (And the OnCreated event script doesn't do it.)

BTW, "SESI" doesn't officially read the forums. If you want something that "SESI" should notice (eg. log an RFE), it should be sent to support. Same goes for the General/Technical Discussion forums at sidefx.com as well. Bugs can be logged directly through the Submit H9 bugs link there as well.

Well, it's good to post here and get feedback to check my thinking before raising it to SESI - but point taken, and thanks for responding.

Cheers,

Jason

[edward]

Exactly right - it seems to me that this CreateScript should appear in the Extra Files section. Do you know why it doesn't?

The CreateScript is created to restore the parameter values currently on the saved node when Save Defaults as Initial Parameters is disabled. As you already noticed, it also contains the opspareds command for saving spare parameters.

[Jason]

Ok, here it is - my 0.1 alpha test it-might-blow-up version of Indigo4Houdini. I would love to have anyone and everyone play with it and pose questions, suggest changes, add features and make bug-fixes to this pipeline.

Please download: Indigo4Houdini on the Exchange

INDIGO RENDERER for Houdini

===========================

Author: Jason Iversen, jason@odforce.net

Indigo Renderer, v1.0.7+ required, http://www.indigorenderer.com

Houdini 9.1+ required, http://www.sidefx.com

For reference on the development of this stuff, and to report bugs or lavish praise, please see:

http://forums.odforce.net/index.php?showtopic=6405

http://www.indigorenderer.com/joomla/forum...opic.php?t=3119

Installing Indigo for Houdini.

------------------------------

You are required to install the indigo soho components in a location along the $HOUDINI_PATH.

It looks like $HOME/houdini9.1 is guaranteed for simple Windows installations of Houdini.

Unzip the .zip archive into your $HOME/houdini9.1 directory.

Warning: if you have the file $HOME/houdini9.1/soho/parameters/SOHOparameters from

another 3rd render pipeline, these will have to be merged.

As for Indigo setup, install Indigo anywhere you like and you can either set an environment variable

called "$INDIGO_PATH" to the Indigo.exe executable, or you can edit the run_indigo.py and customize

the executable location. If so, open the file $HOME/houdini9.1/scripts/python/run_indigo.py and edit it.

The $TEMP environment variable should be set already but this is just a location to write the temporary

.igs scene file to.

You must install the indigo.otl Operator Type Library, and the easiest GUI method of doing this is to go

to the File > Install Digital Asset Library and browse for it and click Accept.

Using Indigo

------------

There are several new operators defined in the indigo.otl library:

Objects > Indigo Light : the native Indigo light.

SHOPs > Indigo * : all the shaders and their settings.

ROPS > Indigo : the render settings exist here.

Rendering Parameters:

Look under 'Edit Rendering Parameters' on any node and browse through the available rendering switches

and settings that can be added to customize your output.

Geometry:

Indigo can also only render polygon triangles. NURBs, Primitive, Quads, etc will not render.

Either make your Generator SOPs (Sphere/Torus/etc) output Triangles or Alternating Triangles or

follow all your geometry with a Convert and a Divide. The SOHO pipeline cannot perform conversions

or tessellation of geometry for you.

Shaders:

There are two types of shaders: "shaders" and "mediums". All the Mediums start with "Indigo Medium: ..."

Mediums can be referred to by Shaders. If you go to your network Display Options ('d' over the

Network), you can turn on Dependency > Show Local Dependency Links and you'll see some nice

wires connecting these nodes.

Motion Blur:

Indigo cannot render motion blurred images, so don't worry about it:)

Cameras:

By default the camera will auto-focus on the object in the center of the frame. The 'fstop' parameter on the

Camera will control the depth of field still. You cannot shut off depth of field rendering in Indigo, only

reduce it as you would if you were shooting with a real camera. To set your focus distance manually, add

the Rendering Parameters under the "Indigo > Camera" folder. Here you will see more optional parameters that

can be defined per Camera, such as "Autofocus", "Bloom", etc.

Lights:

The Indigo Light is a fully defined light source which should support all the options that Indigo can. Be

aware that if you use an HDR, it must be an EXR format file and must be in a latlong representation. Indigo can

also support .float spherical files but the exporter doesn't handle this currently. The pipeline will also try

to convert (wrangle) normal Houdini Lights into Indigo too, mapping the parameters as best it can.

KNOWN ISSUES YOU SHOULD KNOW ABOUT and things TODO:

---------------------------------------------------

* The pipeline does not support Materials

* The pipeline does not yet support shaders applied by the Shader SOP.

Geometry needs to split into seperate Models for Indigo. This is called "partitioning" in SOHO.

* Currently we invoke Indigo.exe, which will render forever until you terminate it. The ROP needs to

support invoking Indigo.exe or indigo_console.exe with a time cutoff and output image name.

* Need to support the "background" light type.

* Need to support .float files for Environment Maps.

Please make lots of suggestions, fix bugs and that's it!

Enjoy!

Jason Iversen

[Jason]

Updated to Houdini4Indigo v0.2.

This new version supports almost all the features in the newly released Indigo v1.0.9 (stable) version.

Get it here: Indigo4Houdini v02 on the Exchange

0.2 - 27 March 2008,

Updated to Indigo 1.0.9 (stable) release.

Added support for new renderer settings: Halt Time, Halt Samples, AntiAliasing, Aperture Diffraction, lots of other things etc.

Added support for new camera capabilities: aperture design stuff, obstacles.

Added Oren-Nayar material.

0.1 - Initial Release

* The pipeline does not support Houdini-style Materials, only SHOPs.

* The pipeline does not yet support shaders applied by the Shader SOP.

Geometry needs to split into seperate Models for Indigo. This is called "partitioning" in SOHO.

* Currently we invoke Indigo.exe with an output image name.

* Need to support the "background" light type.

* Maybe need to support .float files for Environment Maps.

* Need to layout the Rendering Parameters better, with appropriate disablewhen's.

* Need to add exit_portals

* Need to add mesh_lights

* Need to add support for directly reading .obj and .3ds. Auto-archiving would be useful for this.

* Need to complete the downsize/splat_filter parameters - these are only half-defined.

* Need to add override_res from the ROP

[vmuriel]

Great work, Jason !!!!

cheer up!

[Jason]

Great work, Jason !!!!

cheer up!

Thank you; I hope someone finds it useful and produces a good looking image:)

[mtucker]

I must say I was pretty excited about the idea of reporting bugs to Jason instead of the other way around. So I had to download it and give it a try

1. I'd suggest making the renderer exec path a parameter on the ROP (or a parameter passed to the python script). It would be nice to be able to render without having to edit the .py file or set an env var.

2. The Temp file calculation (tempdir + "temp.igs") doesn't work if $TEMP doesn't end with a slash. And the hardcoded temp.igs prevents multiple simultaneous renders. Maybe you should just use "os.tempnam()" or "os.tmpfile()" to create the temporary file?

3. Does the os.system() call block? If so, you can delete the temp file when you're done...

4. This is no doubt a problem specific to my setup, but I kept getting errors from python about being unable to "import os". I eventually changed the render command to use hython instead of python. Maybe you should change this to be the default because you know without a doubt that all Houdini users have hython, but plain python may or may not be there...

5. I tried rendering before reading the README, so my geometry had quads. Nothing rendered, but I didn't get any error messages (from indigo or the ROP). I would think SOHO could detect and complain if your polygons aren't triangles?

And last (but perhaps most important) I would love to have one of the hip files for the sample images you posted to this thread included in the download. It would provide a great jump start. I'm particularly confused about the material stuff right now - I can't find the shader/parameter that simply says "make my teapot blue"... But then I'm happy when I can make mantra render my teapots blue so I'm probably not your target audience for this

Mark

P.S. I assume you know this, but there is a link to your stuff on the exchange from the Indigo web site.

[Jason]

I must say I was pretty excited about the idea of reporting bugs to Jason instead of the other way around. So I had to download it and give it a try

What goes around comes around, I guess, ha!

Points 1 to 4 - this would be all nicely solved if (1) Indigo could accept the IGS piped in, and/or (2) SOHO natively supported the writing of an external file then launching a formatted commandline afterward (like "indigo.exe %s"). I'll definitely make these changes you suggest, though -good ideas!

Point 5, unfortunately it seems as if SOHO doesn't have a soho.warning() function, only a soho.error(). I didn't want to error out because, believe it or not, sometimes it can be hard to ensure you are only sending triangles. Using the Divide SOP for triangulation has two problems: (1) It won't always create triangles! If two points of a quad are coincident, it'll skip the quad - I suppose to avoid generating zero-area triangles, or it can't decide which way to split it. You have to first consolidate points and then triangulate... which is an unexpected step for users. (2) If a mesh is animating, it'll triangulate the mesh differently frame to frame. I suppose it's making a decision based on the angles in the polygon, but this can causing twitching in the surface. In the past, to have reliable/stable topology frame to frame I've had to triangulate a rest frame and snap this mesh to the animated point positions- something that could be avoided I think. It'd be good to have the Divide SOP have a "Stable triangulate" option which would handle both of these cases I mention.

If I use soho.error(), the exporter will then be too fussy I fear. Any chance of adding a soho.warning() to yellow-flag the ROP? Currently I just print a warning into a comment in the IGS file (which is obviously useless).

1. I'd suggest making the renderer exec path a parameter on the ROP (or a parameter passed to the python script). It would be nice to be able to render without having to edit the .py file or set an env var.

2. The Temp file calculation (tempdir + "temp.igs") doesn't work if $TEMP doesn't end with a slash. And the hardcoded temp.igs prevents multiple simultaneous renders. Maybe you should just use "os.tempnam()" or "os.tmpfile()" to create the temporary file?

3. Does the os.system() call block? If so, you can delete the temp file when you're done...

4. This is no doubt a problem specific to my setup, but I kept getting errors from python about being unable to "import os". I eventually changed the render command to use hython instead of python. Maybe you should change this to be the default because you know without a doubt that all Houdini users have hython, but plain python may or may not be there...

5. I tried rendering before reading the README, so my geometry had quads. Nothing rendered, but I didn't get any error messages (from indigo or the ROP). I would think SOHO could detect and complain if your polygons aren't triangles?

And last (but perhaps most important) I would love to have one of the hip files for the sample images you posted to this thread included in the download. It would provide a great jump start. I'm particularly confused about the material stuff right now - I can't find the shader/parameter that simply says "make my teapot blue"... But then I'm happy when I can make mantra render my teapots blue so I'm probably not your target audience for this

Mark

P.S. I assume you know this, but there is a link to your stuff on the exchange from the Indigo web site.

[edward]

(2) If a mesh is animating, it'll triangulate the mesh differently frame to frame. I suppose it's making a decision based on the angles in the polygon, but this can causing twitching in the surface.

I think a PolyReduce SOP set to 100% with Use Original Points enabled and a reference input could solve this case.

[Jason]

I think a PolyReduce SOP set to 100% with Use Original Points enabled and a reference input could solve this case.

Sounds risky, but I'll try it. I still vote for a Stable Triangulate (and a Quadrangulate;)) option though, as this sounds like triangulation is almost a side-effect of PolyReduce.

[kubabuk]

Hey Jason,

Thanks for the great exporter and useful SOHO example? I am slightly confused with the direction of skylight in you your exporter at the moment. Which axis in the exporter is vertical, Y or Z? Indigo uses Z as vertical and doesn't allow to go below 0, so does it mean if the houdini exporter uses the same layout? If yes do we have to rotate all model to that directions (rotate 90 on X), or is it done internally? It sounds simple but I am completely lost at the moment. Also to get different directions should I play between -1 and 1 values?

I am also experimenting with the exit_portals (which I really miss this feature in maxwell) and igi files, I am planning to add this settings to the plugin if you don't mind?.

Don't know if any of you use Violet at all, but recetly I used it to remove a noise from igi rendered files (using median filter). It works wonderfull much better then the median on exr files in Halo. Could somene explain me what might be the reason of such a difference? Would it be 32/16 bit differnce between these files?

thanks

kuba

[vmuriel]

For linux testers, there is a native Indigo beta version for linux 64 bits,

http://www.indigorenderer.com/joomla/index...p;limitstart=10

My early tests with this beta and Indigo4Houdini in Linux are very goods.

Attachment rendered in ten minutes.

Edited April 1, 2008 by vmuriel

[Jason]

Thanks for the great exporter and useful SOHO example? I am slightly confused with the direction of skylight in you your exporter at the moment. Which axis in the exporter is vertical, Y or Z? Indigo uses Z as vertical and doesn't allow to go below 0, so does it mean if the houdini exporter uses the same layout? If yes do we have to rotate all model to that directions (rotate 90 on X), or is it done internally? It sounds simple but I am completely lost at the moment. Also to get different directions should I play between -1 and 1 values?

Currently I just swap the Z and Y components of the Sky Light direction parameters, but I was thinking of using the Indigo Light's rotations instead (optionally) as you would a Directional Light but the sun would start vertically above scene at {0,1,0} instead of down -Z as with regular lights. I'll update the guide geometry so that it reflects the sun directional angle properly in the next release.

I am also experimenting with the exit_portals (which I really miss this feature in maxwell) and igi files, I am planning to add this settings to the plugin if you don't mind?.

That would be just fine:) I was imagining an "exit_portal" rendering parameter, if existent on an object would cause the exporter to output an exit_portal block instead? There should also be an "Indigo Exit Portal" Object which would just have this parameter set on by default and have good representative geometry in it (a correctly oriented grid).

I was about to add a "meshlight" RenderingParameter and look at rounding out some of the other parms a bit - and I should write a little summary on "How to add further parameters to Indigo4Houdini".

If you get it working, please mail it back to me and I'll merge the changes:) In fact, we should start maintaining a little set of hipfiles which demo such things.

Don't know if any of you use Violet at all, but recetly I used it to remove a noise from igi rendered files (using median filter). It works wonderfull much better then the median on exr files in Halo. Could somene explain me what might be the reason of such a difference? Would it be 32/16 bit differnce between these files?

Oh, whats "Violet"? I haven't heard of it. Where do you find it?

[Jason]

For linux testers, there is a native Indigo beta version for linux 64 bits,

http://www.indigorenderer.com/joomla/index...p;limitstart=10

My early tests with this beta and Indigo4Houdini in Linux are very goods.

Attachment rendered in ten minutes.

Cute scene:) I'm also curious how Mantra PBR handles something like this..

[stevenong]

Cute scene:) I'm also curious how Mantra PBR handles something like this..

Yes, I'm curious how PBR compares to the render too.

Victor, do you mind rendering with PBR and compare the time difference?

Thanks!

Cheers!

steven

[Jason]

Updated to v0.3: Indigo v0.3 on the Exchange

0.3 - 1 April 2008,

Added "Indigo Executable" parameter on ROP which, if pointed to an indigo.exe executable will use

it instead the one set in the env var.

Uses hython instead of python everywhere. It's more reliably installed.

Writes truly random temporary IGS filenames so that you can launch many simultaneous renders.

Added background light. The internal name of rectangle light changed so in old scenes, you need to

pick "Rectangle" on the Light again to make this work. Shuffled efficacy_scaling parms

to the Rectangle geometry tab.

Allowed user to override Camera render resolution from the ROP

Added "meshlight". To use this, add a "Light > Geometry Is A Meshlight" Rendering Parameter to

an Object, along with the light color controls (called the Spectrum).

[kumpa]

my first image

[kubabuk]

Oh, whats "Violet"? I haven't heard of it. Where do you find it?

That's a indigo, tone mapping package, althoug it only supports indigo native igi files it's very interesting, you can apply nice post effects stuff to the image as well.

http://www.indigorenderer.com/joomla/index...9&Itemid=62

kuba