Osipa lip sync tutorials for Houdini?

[Gurbo]

I know this is a little particular but I have to ask..

Has anyone posted or found any tutorials on the Jason Osipa method of rigging for facial animation and lipsync as applied in Houdini?

Jason Osipa wrote a book 'Stop Staring: Facial Modelling and Animation Done Right'. It was largely Maya based.

Thanks

Gurbo

[Mario Marengo]

I wasn't aware of this book. But judging by all the reviews, it seems like it's worth checking out.

Are the concepts hard to implement in Houdini? (I would be surpriced if that was the case, but then I don't own the book so... (?)).

Thanks for the reference

Cheers!

[Gurbo]

Are the concepts hard to implement in Houdini? (I would be surpriced if that was the case, but then I don't own the book so... (?)).

I haven't worked through the rigging part to try his technique. It sounds quite reasonable though. His facial animation is based on visimes - visual landmarks rather than audio based phonemes. Many worthwhile tricks with the modelling and applying displacement maps to achieve forehead wrinkles when frowning.

I'm hoping to put together a tutorial for Houdini as part of my research for college.

Stephen Gurban

[edward]

I'd say go for it.

[sibarrick]

I say go for it too. The more character stuff out there for Houdini the better. I'm not a character animator but it seems to me the way to get the package more widely accepted is to get cool character stuff done with it, once people believe it can do that then the rest follow.

[Gurbo]

I'm well under way with the head modelling and I was simply going to use Blendshapes thinking that like other software I could bring up my audio waveform in CHOPs and animate to that.

However, over at the 3D Buzz Houdini forum jjstanley mentioned the Phoneme, VoiceSplit and Voice Synch CHOPs. I found some legacy PDFs at the SideFX site, but I am having trouble annotating text to the audio waveform in the CHOPs editor. The help browser mentions it but has a dead link.

Could someone please help me out. I know there's a little bitty button somewhere...

Gurbo

[sibarrick]

In the viewer pane for chops click on the button that looks like a recyling sign, two arrows chasing each other top right on the viewer, then you can add notes to your waveform by typing in the field to the left of the drop down where it says Notes off and changing that to the chop you want to add notes too. The note appears next to the frame you are currently on.

[Jim Su]

I'd say go for it.

16771[/snapback]

I'm familiar with it, having worked with him over at Mainframe. I'm sure his method has evolved since then, I haven't read his book. I plan on setting up a gui interface for the blendshapes that I'm working on in Houdini so I'll let you know the progress.

Jim

[djorzgul]

hello.

I have the book, and I did few facial setups based on it, in maya. In general, the point of whole osipa's-face-rigging idea is software indipendant. Based on visimes, as said, and rigging part is simple network of connected attributes and blendshapes, so supose it wouldn't be that hard to recreate it in houdini.

[vrljc]

A majority of the setup in Maya is done by using expressions. I think doing this in Houdini would be even simpler since Houdini is so expression based. I have used this method on a number of characters and I have found it to be the easiest way to do facial animation. It would be great to see this in Houdini.

Howeverm, one downfall of using this method in Houdini is with the eye lids. Jason outlined a simple way of skin weighting the eyelids to eyelid joints to make the eyelids move along with the movement of the eyeballs. I think achieving this in Houdini would be a bit trickier.

I might give this a try too.

-jon

[exel]

Howeverm, one downfall of using this method in Houdini is with the eye lids. Jason outlined a simple way of skin weighting the eyelids to eyelid joints to make the eyelids move along with the movement of the eyeballs. I think achieving this in Houdini would be a bit trickier.

I might give this a try too.

-jon

20914[/snapback]

It may not be that bad, we've done quite a bit of that sort of thing here on our current job, creative use of bones... if you can keep the bones centered on the eyeballs and just rotate them up and down, that's a start... then use expressions to add or mix in movement from the eyeball, with some very careful point weighting it should work fairly well...

[vrljc]

It may not be that bad, we've done quite a bit of that sort of thing here on our current job, creative use of bones... if you can keep the bones centered on the eyeballs and just rotate them up and down, that's a start... then use expressions to add or mix in movement from the eyeball, with some very careful point weighting it should work fairly well...

20916[/snapback]

I figured that there was a method to do the same thing in Houdini, but I have never skinned a character in Houdini before. What is the SOP that is the equivelant of skin weighting from maya? I was looking at the CaptureProximity SOP. Is this it?

[edward]

son outlined a simple way of skin weighting the eyelids to eyelid joints to make the eyelids move along with the movement of the eyeballs.

20914[/snapback]

In H8.0, the WireDeform SOP has the ability to use not only the tangent of the curves, but also their N (normal) point attribute as well. It should help with eyelids.

[exel]

I figured that there was a method to do the same thing in Houdini, but I have never skinned a character in Houdini before. What is the SOP that is the equivelant of skin weighting from maya? I was looking at the CaptureProximity SOP. Is this it?

20917[/snapback]

There's a buncha tools to use:

Capture SOP : assigns initial point weighting based on bones' "capture regions" (a pill shape)

Capture Proximity : assigns initial point weighting based on distance from bone(s)

Capture Paint : (...)

Capture Correct : adjust your bones' positions without having to bind your skin again (this is great)

there are several others as well (CaptureOverride, CaptureEdit, Capture Mirror), I think it's a very flexible system, it's come a real long way in the past few years...

The trick is to learn these tools properly, several of them really have to be called from the geometry viewport to work properly, and I find that others work better if I just tweak them from the parameters window...

Check out the Houdini Exchange for a few downloadable rig demos to help get you started...

http://www.sidefx.com/index.php?option=com...pper&Itemid=145

Woops I kinda got off-topic there -- about the eyelid stuff, yeah what you'll need to do is use a Capture SOP or CaptureProximity (which I never really liked) to perform your initial skin binding, and then follow that with some Capture Paint operations to tweak the weighting.

[Gurbo]

Hello,

I glad that this topic has been revisited since I last posted the initial questions.

At the time I was scoping out possible topics for a Master's thesis. And as Jason O states in his book that the techniques are not software specific, I'm comparing the approach between maya and Houdini to see if the principles of visimes is actually a new development in character animation.

I hope to post the finished paper and associated workflow in Houdini when I'm done. I'm still plugging away at it.

Thanks for the info about the wireDeform, Edward. So far I have found it easier to carry out the task in Houdini - but I did have a bit of giref with weighting the eyelid points.

Thanks for the feedback

Gurbo

[MatrixNAN]

Hey,

You probably want to checkout the new Houdini Video at www.thegnomonworkshop.com. Also you probably want to implement this with ZBrush for defining the wrinkle details with displacement maps generated from the modeling process and to use the morph button in ZBrush to switch between poses with the model which you need to do in Houdini then export OBJ to ZBrush and then Export from ZBrush the Displacement Map to Houdini and link up the Blend Node with the Shading System you make in VOPS and link the animation to the SHOPS that you generate from VOPS. Blend between your displacement maps and then you can sync the displacement maps and the Blend Nodes together and make an interface that allows you to control the face and while doing so will auto animated the face. You can then tie this into the CHOPS audio file to drive these controls to auto animate these controls. If you like you can then go in and edit the results of the auto animated controls and hand key them if you like. Still very powerful.

Cheers,

Nate Nesler

[Gurbo]

Also you probably want to implement this with ZBrush for defining the wrinkle details with displacement maps generated from the modeling process..

21698[/snapback]

Thanks for the reply MatrixNAN,

I've tried using ZBrush and I must say that I found the interface for 3D tumbling to be strange. Great for experimentation. I found the Z spheres cumbersome but the painting tools were nice. Their concept of edgeloops seems to be different to other 3d apps(?).

I decided not to pursue it because I found managing the poly count a little difficult, the gallery pics I've seen tend to look hand painted and people seem to go so nuts with the displacement modelling that the models look like they are blistering.

Other than that a lovely tool and I'll check it out a bit more thoroughly.

Cheers

[old school]

There are three SOPs that bind the skin to bones: Capture SOP, Capture Proximity SOP and Capture Layer Paint SOP. Either does a good job. It is expected that paint weighting is required after either of them.

It is best to set all this up in the object viewport though. I will unlink the viewport and keep it at the object level while looking at the SOP network for the character skin object. You can see what is exactly happening.

You can slap both the Capture SOP and Capture Proximity SOPs if you want and see which one gives you the better initial results.

Capture Proximity is the "quick let's get this thing going" tool. The Capture SOP is at it's best when you fastidiously manipulate all the bones' capture regions but you really don't have to if you will be doing a lot of painting.

I generally use the Capture SOP and it's cregions. Somehow I always get better initial joint bending with cregions. Takes a bit of practice. I will gladly trade off a 2 minute cregion adjust to 10 minutes of paint weighting.

Now the third option is to use the Capture Layer Paint SOP to bind bones to skin. Yes this is the hairy-scarey approach. In the object viewport, just enter the Paint Capture Weights tool, select your new bone along with the skin and the bind is done implicitly for you.

Why is this scarey? I myself like the control the Capture and Capture Proximity SOPs give you. You can actually see the list of bones that are being used. You can add and remove bones with care of course. With the Capture Layer Paint SOP you can't see what bones have been mysteriously added. In a complex binding skin network that you are traversing up and down, it can get quite confusing. Best to simply add a new Capture SOP and add the bones, then continue painting. I literally name the Capture SOPs down the line like "capt_eye_bones", "capt_ear_bones", "capt_lip_bones", etc. You know, those bones that get added for the miniscule control that exel spoke about.

This feature was added to Capture Layer Paint as a safe-guard for some experienced riggers who simply did not care for the control that the sop networks gave you and would constantly break the SOP bind skin network but then again I am a tweakey control-freak so I don't really know fer sure what's right.

[Gurbo]

Best to simply add a new Capture SOP and add the bones, then continue painting. I literally name the Capture SOPs down the line like "capt_eye_bones", "capt_ear_bones", "capt_lip_bones", etc.  You know, those bones that get added for the miniscule control that exel spoke about.

21702[/snapback]

Thanks old school,

I'm not that familiar with the Capture Proximity SOP, thanks for explaining it. I am coming up to painting weights so I appreciate your tips - I'll have your post up for reference as I go through. The quote above, I take it that the bones have already been made but that you are just (selecting and) adding them to the Capture SOPs.

I got confused with this last time I was skinning. I ended up having different Paint Layer Capture SOPs all over the place and not sure what any of them did.

Thanks

Gurb