Alembic workflow in production environment

[wandersonp]

Hey guys, I like to hear about your workflow, in regards alembic assets, material assignment and organization.

 

my main concern is applying shaders inside alembic root node for pieces, when I rebuild/update the asset, I lost all my shaders connections.

 

How u handle that? What´s the suitable approach in day production basis when assets still changing or refining?

 

Thanks

[narbuckl]

Ive been using alembic in production since they first added it to Houdini and there are several ways to keep things procedural, and like everything in Houdini, they all work but its not always clear which one to use.

 

So using the alembic archive:

 

All that the alembic archive does is extract your transform hierarchy into Alembic transform nodes, the actual geo is inside a houdini geo object and is being read with an alembic sop with transform to worldspace set to off (Because you want it to be in local space prior to the xforms applied to it).

 

[Alembic Archive] > [Alembic Transform] > [Houdini Geo Object] > [Alembic Sop Reading Geo]

 

 

If you make sure that 'Build Hierarchy With Channel References' is on, then all the corresponding alembic xforms and sops will be linked to the top alembic archives file name path and frame/fps. Meaning that if you change paths, it should update, but all you have to do in order to update without rebuilding is link to the reload geometry inside the alembic sops. Technically it should update when you change paths, but we all know that can be a little hit and miss considering how it loads into memory.

 

Something to note about motion blur. If you were to say import or object merge from your alembic archive into a fresh sop and do not specify the correct transform or in most cases use 'into this object' (Which i have done plenty of times for various reasons) keep in mind that you lose the transform hierarchy so your motion blur will change. This also applies to just reading in an alembic through sops. Although this really only matters for objects that have transform 

 

Now if you are lighting something like a character and its just a bunch of deformation motion blur anyways then that doesn't really matter because you will need to geo sample more than once anyways.

 

An example where it would matter would be something like this:

 

Say you have a cg ball that is animated globally and has a local rotation on it as well. If you can work it out so that you just use alembic xform nodes then you can make a really efficient pipeline where you have the default position or skinned position of the geo saved out as a packed prim or alembic or .obj or w/e and you are just applying the exported xform data to it. However for things like a character it would probably all be deformation in local space anyways so the transform stuff is irrelevant.

 

Materials:

 

When it comes to material assignments you can use the data tree 'object appearance' pane to assign materials quickly to the houdini geo nodes, but if you want it to be more procedural you will need to use a packededit or do something like i mentioned before where you build a subnet asset that has a reload button for just the alembic sops so that you don't need to 'rebuild'. My experience is that the alembic xforms update fine on their one once you change the path. As a third option you could always create a python script that generates the link to all the reloads on the alembic archive itself. Keep in mind the transform hierarchy naming convention needs to be perfect.

 

 

Now if your less worried about being able to use transform data, such would be the case with a character, or if your planning to render with high geo samples anyways then you can just use an alembic sop and assign materials via packed edit or material or w/e.

 

Note about groups: by default the alembic sop does not create any groups for you (which may be confusing, especially if someone spent a long time making a nice structure in maya or wherever the alembic is coming from). Go down to primitive groups and you can decide how you like the alembic to partition your groups out. Typically I use the 'Transform' method because that is typically what people decide to use in maya, and I the shape method also would generate much longer names.

 

This is all from experience using alembic, so if I said something wrong and somebody else knows better I would appreciate the correction!  

 

Hope that helps.

 

-Nathan

[wandersonp]

Thanks a lot narbuckl

 

Thats information is very helpful 

 

[Whatsinaname]

How do you handle rendering complex geometry inside Alembic coming from other packages like Maya? I've got some very detailed models in my scene, and when it comes to rendering, Mantra loads each and every modeled nut, bolt and screw (saying: everything that once was a shape node inside Maya), one after one, out of the Alembic scene. Doing so, preparing a frame for rendering takes a very long time. That's why I started baking out the Alembics into BGEOs, as they are read quickly by Mantra (Alembic ~10mins, Bgeo ~1min to a couple of seconds).

 

Creating a cache of a cache feels a bit useless to me, so I was wondering if there was a way I could set up or prepare the Alembics inside Houdini so Mantra can handle them as quickly as the BGEOs?

[narbuckl]

I would be curious as to what version of houdini/alembic you are using and if you are using the alembic archive or the alembic sop to read in your geometry. Houdini loads alembics as 'Packed Primitives', so unless you are doing things to them after you import them that require them to be cached again it should just be loading the alembic from disk at render time. Check to make sure your alembic is set to read in as Delayed Load Primitives, the other thing you can look at doing is checking to make sure you are exporting ogawa alembics and not the legacy HDF5. I would also see if there is a difference if you uncheck 'Build Hierarchy using subnets' on the alembic archive so that it just creates the final nodes all under 1 transform.

 

The other thing I would recommend is creating a script on the 3rd party side (maya for example) that only exports the geometry in world space at its final transform/deform instead of a huge hierarchy of transforms that you may not need. 

 

On another note I will say that if I create an alembic with houdini it appears to run smoother than an alembic created from maya.

 

If you have an example scene I will gladly take a look.

 

-Nathan

[Whatsinaname]

Well, thank you so far. We've only got Maya 2014, so Ogawa unfortunately isn't an option. Our rigger tells me it wasn't possible to bake out the whole car into one combined object that doesn't consist of loads and loads of individual pieces, but I highly doubt that. Do you know of a script or an example where something like that is done?

 

I load the ABC as delayed load primitives and building without subnetworks seems to be accelerating the process, the only problem is it seems like it's very difficult to clean up the intrinsics of the ABC, then (like doing away with the engine etc.).

[michael]

I wrote a script (sorry can't share it) that

finds all the alembic nodes (inside the Houdini geo node)

object merges them into a new /obj/object

then inside each Houdini geo node lays down a network to bring back the group from that new object into the Houdini geo node that ends in a null with the name of the original alembic node

this way I can export the whole alembic hierarchy back to Maya

[narbuckl]

Something that I sometimes do on objects that are rigid ('Like a car') is Create my own heirarchy cleanly in houdini of the static geo then in the alembic you just export the transforms for the moving parts and use an alembic xform node to import those then just put the appropriate pieces under the appropriate xform. This is however what should be happening already inside an alembic archive. If it is still really slow I would ask your rigger if he skinned the objects or if he is using transform constraints, because if he skinned the geometry then its reading as deformation not transformation in which case it will be much slower since it has to read the vert position on every frame instead of reading a single transform and then moving static objects under that transform.

 

I have also made export scripts for shows ('For maya') where I Duplicate and constrain the geometry so that its clean and all on world level transforms then export those pieces in the actual alembic.

 

If there is a way for you to give me an example of the file your trying to export and read in I could have a look to see if I notice any specific red flags that could help you fix.

 

Also just to note, In my experience I have come across geometry before that alembics just simply don't like, usually in complex models where someone made bad decisions and used booleans and made some crazy shit geo, 5 sides, non manifold, zero area faces, etc.

 

Hope this helps, but if you can post a file I will have a look, if you don't want to post it on the forum, pm me and we can work something else out.

 

-Nathan

[narbuckl]

I wrote a script (sorry can't share it) that

finds all the alembic nodes (inside the Houdini geo node)

object merges them into a new /obj/object

then inside each Houdini geo node lays down a network to bring back the group from that new object into the Houdini geo node that ends in a null with the name of the original alembic node

this way I can export the whole alembic hierarchy back to Maya

Sounds very familiar to a script I've made myself to create happy objects for maya's transform > transformShape structure.

[michael]

it's pretty simple...

I have another one that will load locators/joints exported from Maya and place a null in the right spot - for getting their transforms, it's good for deforming geometry, so you can attach stuff to characters or recreate the joint hierarchy for additional deformations (muscles etc)

[tricecold]

how do you guys render alembic with point instancing using instancefile string. I can do this with an instance sop ( after changing the file node with an alembic one) and it does loads the abc and copies on the abc to the points, thanks to Tomas Slancik, I can even offset the caches of each individual abc by modfying abcframe intrinsic.

 

but

 

how can I render this the same way i normally would with bgeos and a point procedural (vm_geo point instance)

 

any ideas

[3__]

convert to bgeo, hose the alembic nodes, carry-on as usual...

[murathan altunsoy]

I wrote a script (sorry can't share it) that

finds all the alembic nodes (inside the Houdini geo node)

object merges them into a new /obj/object

then inside each Houdini geo node lays down a network to bring back the group from that new object into the Houdini geo node that ends in a null with the name of the original alembic node

this way I can export the whole alembic hierarchy back to Maya

genel="sukulle"
aciklama="select alembic node"

selected_node_name=hou.selectedNodes()
size_node=len(selected_node_name)
if size_node == 0:
    hou.ui.displayMessage(aciklama,title=genel)
    pass 
else :
    alembic_node=hou.node('obj').createNode('geo',node_name="alembic_merge_",run_init_scripts=True)
    full_merge=alembic_node.createNode('merge')
    for i in selected_node_name[0].children():
        for m in i.children():
            import_obje=alembic_node.createNode('object_merge')
            import_obje.parm('xformtype').set(1)
            path_al=m.path()
            import_obje.parm('objpath1').set(path_al)
            import_obje.moveToGoodPosition()
            full_merge.setNextInput(import_obje)
            full_merge.moveToGoodPosition()

    transforme_node=alembic_node.createNode('xform')
    transforme_node.setFirstInput(full_merge)
    #transforme_node.parm('scale').set(0.1)
    transforme_node.moveToGoodPosition()

its for shelf tool

Edited November 25, 2015 by murathan altunsoy

[martinkindl83]

the script seems to work (it will recreate the connection), but im having errors on object merge: invalid SOP specified.

 

Should i be able to see the same mesh in the viewport when selecting the alembic_merge or it should be blank?

 

 

[murathan altunsoy]

first  code is just for alembic import

the script seems to work (it will recreate the connection), but im having errors on object merge: invalid SOP specified.

 

Should i be able to see the same mesh in the viewport when selecting the alembic_merge or it should be blank?

 

genel="sukulle"
aciklama="select obj node "

selected_node_name=hou.selectedNodes()

size_node=len(selected_node_name)

if size_node == 0:
    hou.ui.displayMessage(aciklama,title=genel)
    pass 
else :
    alembic_node=hou.node('obj').createNode('geo',node_name="file_merge_",run_init_scripts=True)
    full_merge=alembic_node.createNode('merge')
    for i in selected_node_name:
            import_obje=alembic_node.createNode('object_merge')
            import_obje.parm('xformtype').set(1)
            path_al=i.path()
            import_obje.parm('objpath1').set(path_al)
            import_obje.moveToGoodPosition()
            full_merge.setNextInput(import_obje)
            full_merge.moveToGoodPosition()
transforme_node=alembic_node.createNode('xform')
transforme_node.setFirstInput(full_merge)
#transforme_node.parm('scale').set(0.1)
transforme_node.moveToGoodPosition()

this one  merges  together for selected obj

[adrianr]

first  code is just for alembic import

genel="sukulle"
aciklama="select obj node "

selected_node_name=hou.selectedNodes()

size_node=len(selected_node_name)

if size_node == 0:
    hou.ui.displayMessage(aciklama,title=genel)
    pass 
else :
    alembic_node=hou.node('obj').createNode('geo',node_name="file_merge_",run_init_scripts=True)
    full_merge=alembic_node.createNode('merge')
    for i in selected_node_name:
            import_obje=alembic_node.createNode('object_merge')
            import_obje.parm('xformtype').set(1)
            path_al=i.path()
            import_obje.parm('objpath1').set(path_al)
            import_obje.moveToGoodPosition()
            full_merge.setNextInput(import_obje)
            full_merge.moveToGoodPosition()
transforme_node=alembic_node.createNode('xform')
transforme_node.setFirstInput(full_merge)
#transforme_node.parm('scale').set(0.1)
transforme_node.moveToGoodPosition()

this one  merges  together for selected obj

 

Bump to say thank you - Just saved me some time!