Using Soho: Indigo Renderer Support

[Jason]

Hi all,

I'm going to spend a little time fashioning Houdini 9.x support for the hobbyist renderer Indigo (home site).

I am writing this support using SOHO and I will attempt to journal my progress as I go through it.

Wish me luck!
Jason

See: SOHO, RendererDirectory



EDIT: Download it here - Indigo4Houdini On the Exchange

[Jason]

Day One

I downloaded the Windows 32bit version of Indigo and installed it.

I'm a little familiar with how the soho code for the pipelines distributed with Houdini are layed out - it's basically:
* IFD.py - the starting point
* IFDframe.py - all the entities which are written out per frame
* IFDgeo.py - called by IFDframe generally - defines the geometry output
* IFDapi.py - ad hoc functions imported into every module
* wranglers/HoudiniLightIFD.py - functions called to define how lights get exported in IFD

My tactic is always to hack up a working pipeline to a point, and then rebuild it in the name of optimization.

I copied $HIH/soho/IFD.py to a new directory and changed it's name to indigo.py. I axed a lot of 2nd half of the code specific to Mantra - just commenting some general points. I surgically removed any other mantra-specific references.

In Houdini, used the Type Manager on the Mantra ROP to duplicate the HDA and changed the name and label to Indigo. Create an Indigo node and in the Type Properties change the name of the soho_program to point at my indigo.py instead of IFD.py. Also changed the program from "mantra" to "indigo" and such.

Hitting Render now, I am using my indigo.py script; I import hou put in some hou.ui.displayMessage()'s to see what I'm doing.

The crux of what SOHO is meant to do is it will catch anything printed to stdout and redirect it into a file. This file is then gets submitted to the the program of your choice. SOHO also assist you with a couple of other things: it determines which objects are to be output into your scene and builds a nice list so you're not responsible for re-doing this for every exporter you write - and with crazy subnets and so on, it's not an easy task. It also has a mechanism for quickly querying larges batches of parameters in the hierarchical Rendering Properties way.

Now I examine the Indigo doc pdf and load up all their test scenes (.igs) and examine the format. Its XML and seems simple enough: define a "camera", define lights of various kinds, define "material"s, define "mesh" instances and "model" which instantiate those defined by "mesh" entities. All good - nothing too confusing.

So I begin by wanting to set up the simplest scene: a triangle mesh of a box.

From the original IFD.py I see how to loop over objects and get the camera and stuff and set up quick procedures for outputting a simple camera and mesh and model definition. Unlike the IFD pipeline, I just put all the my code into the one indigo.py file until I get to a point that I might want to split it out.

I set what I can (inspecting my output .igs file) and when I'm happy I have what looks like a valid XML file, I launch Indigo against it. It complains about all these missing entities one by one and so I substituted some hardcoded prints in there temporarily just to squeeze through Indigo's parser and start a render. It turns out that although there are defaults for many of the entities/tags, Indigo needs them defined anyway (which seems to contradict having a default in the first place, doesn't it?). This bit takes a time, hammering the output into parsable shape.

Finally, success! Something renders... a bit of the sky! Perfect. And all I could expect given the crappy cut'n'pasted entities from various examples.


What have I learned about Indigo now:

• Triangle meshes only - no quads, polys. Boo!
  
• No motion-blur blocks or anything in any example or documentation.
  
• Needs an exhaustively set up Camera.
  
• Can instance geometry.
  
• XML format - might get large on big geo
  
• Can reference external .3ds, .obj and .ply geometry.
  
• Can read/write EXR files - goody!

[Jason]

Day Two


So now I want to get my geometry defined properly and I can read the SOP geometry something like this:

	soppath = obj.getDefaultedString("object:soppath", now, [""])[0]
	geo = SohoGeometry(soppath, now)
	npts = geo.globalValue('geo:pointcount')[0]
	nprims = geo.globalValue('geo:primcount')[0]
	atrP = geo.attribute('geo:point',  'P')
	for pts in xrange(npts):
		P = geo.value(atrP, pts)
		print "vertex pos=\"%f %f %f\" "%(P[0], P[1], P[2] )
...		
	atrVertexCount = geo.attribute('geo:prim', 'geo:vertexcount')
	atrPointref	= geo.attribute('geo:vertex', 'geo:pointref')
	for prim in xrange( nprims ):
		nvtx =  geo.value(atrVertexCount, prim)[0]
		print "tri"+"".join([" %d "%geo.vertex(atrPointref,prim, vtx)[0]  for vtx in xrange(nvtx)])

Loops through all points and writes them out, and then follow with a loop through all primitives and print all the points they reference.


Looking at the transforms in the model entity, I can only specify a position, and a 3x3 rotation matrix. Didn't figure out the conversion from hou.Matrix4 to a Matrix3 so I'll just translate stuff and expect rotations to be weird.

I put the camera at the origin and move some objects around and render away...

LATER:

Ok, so I convert the Matrix4 transform to a Matrix3 - which is rotation + scale Matrix - and Indigo seems to respect scales in the 3x3 (unexpectedly, but luckily) because the <scale> tag is only a Uniform scale and I was worried about how to represent non-uniform scales for instances.

I have now decided that I should start to set up some of the basic parameters available to Houdini and SOHO, just to start it off and get it running. All the Rendering Properties are stored in $HIH/soho/parameters. I have started one called Indigo.ds and started to populate it with Indigo's parameters. (See attached image). There is some awkwardness in a casual environment like my Windows setup at home with respect to the .ds files being included from the right location. Without having a HOUDINI_PATH setup with a fallback searchpath from $HOME/houdini9.1 to $HFS, I have to copy all the .ds files manually over to the houdini9.1 location or they won't be found. Perhaps I can figure out a better way of doing this later.

Attached thumbnail(s)

• 
• 
• 

[rdg]

That's cool and explained me a lot about SOHO.
But where did you get the information about all those: *.getDefaulted* things?
Reverse engineering?

Georg

[Jason]

rdg, on Jan 29 2008, 12:51 AM, said:

That's cool and explained me a lot about SOHO.
But where did you get the information about all those: *.getDefaulted* things?
Reverse engineering?

Afraid so A lot of SOHO learning is reverse engineering - at least for me. SESI have made attempts to train people in SOHO but it's quite intricate in many ways.

[SYmek]

Thank you Jason. You shorten a way significantly for successors .

sy.

[kodiak]

Thanks Jason for doing all this for us..these journals turn out to be great.

Oh, and Good luck!

[Mario Marengo]

Great stuff, Jason!
It'll be great to have easy access to a research-type renderer after all your hard work.
Many thanks, and please keep it coming

Cheers.

[SYmek]

Jason,
could you tell a bit what library you're using to deal with XML files. I must say that this is my biggest problem for now. Dealing with XML tags. Minidom or Element Tree simply don't fit. Sometimes I have a feeling like XML was an idea of a real evil...

[Jason]

SYmek, on Jan 29 2008, 02:58 PM, said:

Jason,
could you tell a bit what library you're using to deal with XML files. I must say that this is my biggest problem for now. Dealing with XML tags. Minidom or Element Tree simply don't fit. Sometimes I have a feeling like XML was an idea of a real evil...

Since I'm just writing XML, I'm cutting down the complexity and RAM use and I'm just printing tags myself; which seems to me to be an acceptable trade-off.

I just make a couple of helper functions like:

def i_entity_start( entity ):
  print "<%s>"%entity
  soho.indent(1,None)

def i_entity_end( entity ):
  soho.indent(-1,None)
  print "</%s>"%entity

[SYmek]

yes, this makes sense.

thanks,
sy.

This post has been edited by SYmek: 30 January 2008 - 03:47 AM

[Jason]

Hi all,

I'm going to have to take about a week off this project - I'm moving house on Friday and there is a lot of Real Life ™ to attend to in the evenings. It's fun though - so I'll be back