Rigging Experiment

[sibarrick]

Hi all, I've been experimenting again, and now I'd like to see if anyone has any feedback. This little experiment has basically been the reason I haven't posted any hair stuff yet.... hair is on the way but I got distracted by the whole idea of intergrating bones and wire capture.

I've always wanted to find a more organic way to manipulate meshes other than just with bones, to my mind the limitation of bones is that they don't give you enough control nor are they very procedural. Not only that but they don't really mimick what is going on very well. Skin is driven by muscles not by bones. Problem is that simulating a whole muscle rig just to get some skin deformation is all rather heavy. Ok with me so far?

Next thing, skin captured by proximity gives very nice smooth deformation in stretchy skin areas but is no good where you need tighter control, this tends to be in areas where bones do more of the work than fat or muscle.

Ok, so what I wanted was something that is similar to bones in how you set it up, is more procedural, is more flexible and organic looking, and combines proximity weighting and nearest bone weighting. Plus I don't want to have to fiddle about painting weights to get the results I want.

Still with me, if this has tweak your interest read on....

So I think I have a way to do this, but I don't know if

A. someone has already done this?

or

B. is it a good idea to anyone else but me?

This is hopefully where you come in. Please read the following information and then take a look at the attached file. Be sure to read the following first though or it won't work.

The method

In the attached hip file I have prototyped my idea in vex, so it's not really a real-time solution. It's surprisingly good concidering but bare that in mind, it's the idea I'm interested in not the implimentation.

The way it works is this:

1. You setup your ik/fk bones as normal, parenting however you like just as normal. (The rigging in the attached file is something of a mess because it is a mish mash of all sorts of old files and stuff I had lying around - don't take any notice of it)

2. Because this is a prototype it is all done inside one object called skin, and is all implimented with sops. For a real system this wouldn't need to be the case but in order to get it to work I did it in sops. Anyway, the next stage is to draw lines using curve sops onto your skin, these lines follow muscles or tendons or just general areas where you want control.

They can be easily added or removed just using a merge sop.

In the test file you will find these curves inside the skin object in the network boxes. When you first open the file the solution won't cook this is because of a bug in chops, to fix it just dump down a sop any old sop should do, this seems to kick things into life.

3. Once your lines have been drawn you parent them to the bones you want to control them, where a line traverses multiple bones you parent the curve sop to each bone in turn and then blend between the solutions.

I have a special parent sop that calculates a transformation matrix that does the parenting and then stores this matrix with the points in the curve. The blending operation interpolates these matrices to give a final solution that stores for each a point a matrix that will later maniulate the points in the skin mesh. At this point you can previz the deformation directly on the curves.

4. All the matrices are combine together and transfered onto the combined set of lines in the capture pose.

5. The points in the skin are given attributes to determine the radius in which to search for curve points, this will determine how many animating curves will affect each point in the skin mesh.

6. Finally the skin is marked up to show how areas that are close by each other are to be affected by the animating curves. In the case of this example which is a hand it is designed to stop fingers that are next to each other from influencing fingers other than themselves.

Put simply I have painted the fingers red blue red blue red and made the rest of the hand green. The meanings of the colours are as follows: Points that are red or blue can only effect other points of the same colour or that are green, whereas green points can affect themselves plus red or blue points.

That's about it, all the above information is fed into a sop that does the deforming, and I use point clouds to deal with the calculations as efficiently as I can given that it is all done in vex.

I've only rigged one hand and I've so far only tested it with the animation that you will find in the file. I haven't done anything about testing how the wrist area holds up but I'm pretty sure that the method will work for anywhere.

So as I say I'm very interested in feedback on this concept, is it something worth pursuing or am I barking up the wrong tree, or a tree already barked up

The only other thing to note is hidden away in some of the network boxes are controls for added muscle type deformations on top of what is naturally produced by the bones moving, at the moment that includes controls for tension and translation (ie bulging) So in theory muscle deformation could all be created via the one setup with very little overhead.

Finally, my thoughts are if this were properly implimented one could add weight painting and other rigging style ops and improve efficiency by doing a full capture first and the deformation second, rather than doing the whole thing in one op.

flex_test2.zip

Oh I nearly forgot, you like pretty pictures....

[Jason]

Hey Simon,

This is an interesting setup, for sure:) Unfortunately the VEX sop kinda explodes the geometry but I can look at the fundamental structure and concept. Is it working for anyone else?

I'll write more tomorrow - I'm just a little hungov- .. tired right now.

[edward]

I haven't gone through the file too closely. So the whole point is to drive deformations via better wire deforms?

[sibarrick]

That's really the basics of it Edward yes, wire deformers that can act as bones or muscles, or put another way they can act as rigid deformers or soft ones, all in one setup. This is really how I'd like to see wire deformers develop, that plus taking them out of just the sop world and have them available like bones are at the moment. In a way I want this to show Sesi how it could work, just in case they don't believe in the concept, this I hope proves it can work and it would be useful.

Edward not sure why the vex sop is blowing up, except for the reason I state the chops don't cook until you either put down a new sop, I usually just stick down a sphere, or if you go into one of the network boxes and find a parent sop then press the recook button, that sorts it out too, other than that I'm not sure why it wouldn't work.

When you've sobered up, let me know if any of it works, maybe I can repost something else.

[TheDunadan]

Just wanted to note: The file loads up fine over here (after having placed some SOP) and the deformations overall look pretty good to me. A few small deformations don't look quite as if they were intended to look like that, but overall it seems as if the hand when deforming keeps it's volume and with softbinding this is usually a tough cookie and involves many additional deformers to get it look somewhat correctly.

If a setup process like this doesn't take ages to ages to do, it looks to me like a very promising technique. The network as it is seems somewhat complex at first sight and is sure good enough to scare anyone away from my desktop

Anyhow, thanks for sharing this concept setup with us

Jens

P.S. Can't wait to see something of your hair modeling system for Houdini

[sibarrick]

If the setup process like this doesn't take ages to ages to do it looks to me like a very promising technique. The network is seems somewhat complex at first sight  and is sure good enough to scare anyone away from my desktop

14933[/snapback]

I know the setup looks complex but it's really just because I kept everything in seperate sops whilst I was figuring out what was needed. I can vastly simplify things by combining certain sops together.

Also, if sesi did this they could cut out a lot because they wouldn't be confined by vex.

I think the setup time is quick, certainly transferring it to different files should be, once it is setup you don't need to paint weights so it should be pretty close to working on any model straight away.

Thanks for the feedback.

[Mario Marengo]

Hey Simon,

This strikes me as a very cool idea. I can only watch this from the sidelines at the moment, but if I understand it correctly, the intent is to guide surface deformations from a few structural curves -- but with better control than what the existing wire deform/capture give you...

The only thing that is not very clear for me right now (and I'm not up on all the minutiae of character rigging, so please excuse the dumb question), is exactly in what way does your proposed method improve upon the existing wire tools? -- from your description I see that you go from the skin to the wire, whereas the existing method goes the other way, but I'm sure there's much more going on here. It would help people like me if you explained this difference a little more so we can get a better understanding (at least at a conceptual level) of what you're up to.

BTW: I only had a very cursory look, but some pretty impressive stuff going on in that there hip file of yours!

Cheers!

[sibarrick]

Hi Mario, the main difference is that I store the matices that actually do the parenting of the curves to their parent bones, and I do a quaternion interpolation to create the blend between them. The deformation is then created by directly using these matrices to transform the points. At the moment if you try and do the same deformation with a wire deformer the geometry will mess up because there is no proper frame of reference used.

I found this out when I was messing about with the whole skin wrap thing.

The other thing is that this method shouldn't be seen as very far away from wire deform, it's just that wire deformers as they are won't do this, but I would like them to. It's really just a big hint to Sesi to say extend wire deformers.... the only reason I do the capturing the way I do is because it's really the best way to do it in VEX, if it were written in hdk I'd do it as normal.

Apart from that I wanted to suggest how you could animate a bunch of curves so that they create nice natural deformation, whilst being driven by a simple bone setup. This has lots of advantages to just using pills as capture regions, the main one being there is a much smaller requirement for painting weights all over the place, thus I would hope it would be quicker to set up. And at the same time you can use them to do muscle deformation.

Also because the deforming blended curves match quite closely the effect you get on the skin you get a very quick previz of the final effect.

Finally, I tried to rig the hand in that example file before with traditional bones and I just couldn't get a nice deformation at the base of the fingers as they bend into the palm. I think this is because it is based on real anatomy and the metacarpal bones rotate correctly around the knuckles, this makes the palm compress a lot and normal bones just don't handle this very well. At least not that I could get working, I spent hours and hours painting weights trying to get the effect I wanted and i just kept thinking I bet wire deformers could do this better, but if I use wire deformers I lose all the other stuff I get with bones, so in the end I just thought heck I'm sure I can put something together in vex to do this and maybe if I prove it works Sesi will extend wire deformers to be more useful.

Oh yeah nearly forgot, I've now also added a more normal capture method into the mix, whereby you specify a capture radius on the curve rather than the skin, this allows for additional deformations to be built on top of the ones created by the primary deformers, or it can be used to completely replace the need for adding them to the skin. I won't post this though as it will probably just confuse things more.... it's handy though

[edward]

Simon, doing better deformations is totally feasible within WireDeform as then we just need a new blending method. What I'm not clear about is how you want the capturing to go about. In your initial posts, you mention that the weighting is done be basing a radius per point as opposed to per wire. But then you mention about doing it per wire. which way should it be done? What are the capture controls that should be given?

For deformations, using quaternions is something that we've thought about and but it's another one of those many things we've thought about but never had the time to focus on. It's always nice to have someone else prove the concept.

While the philosophy for the character tools has always been to make them accessible at the Object level, ultimately, everything has be driven done to the SOP level. I'm not sure how you would envision the wiredeform tools to be at the object level. In your example, it seems to me that you went through a lot of trouble to create those capture curves. If that's the way that it's going to be, then there's little point in making object-level tools for them.

[edward]

Some more added notes to using quaternions:

- They're quite a bit slower

- There's no hardware support for them right now as opposed to the default bones method. We don't do hardware acceleration right now even for regular bones but we at least conceivably could.

- Blending quaternions is quite a bit trickier for points weighted by more than two influences. There doesn't seem to be a good (and fast) way for blending more than two quaternions.

[sibarrick]

Hi Edward,

Ok here are my thoughts on your questions.

The reason I use blended quaterions was just an experiment, the main point of it though was to be able to have curves that could be parented to a bone system that then had very well defined transformation matrices for each point on the curve, in my mind this made it simpler in the long run to intergrate it with the way I imagined bones were implimented. (How are bones implimented by the way?)

My initial idea was that you could extend bones to include flexible elements as well as rigid ones. Maybe this is still an idea. That way you could do most of your rigging with normal bones and add in a flexible one in areas like the webbing between thumb and ring finger. If there is another way to blend matrices that is better then that could be used, the method isn't important it's the result that counts.

I tried to keep the blending of quaterions to a minimum by just performing it on the curves, in theory I you are only doing it on a very small number of points in the blend region, and you only ever need to blend between two curves at a time. The really slow bit in my setup seems to be the chops that calculate the parenting matrix, I'm sure this could be picked up much more efficiently in hdk.

In terms of capturing I'm only doing it the way I am because it seemed the most efficient way to use point clouds, my thoughts were only grab the minimum number of points at a time as this will be more speedy. I've just added an option to do it the other way round just so I can see what other effects I can get out of the system. The way I would see it done for real would be much as you have it now, the capturing is done first and you store all the capture info and then the deform sop is written to be as efficient as possible, my problem was you can't do this in vex and I can't write hdk so I thought do what you can to prove the point and then see if you can get Edward interested

The thing is that now I've moved away from the intial idea of integrating this curve method with bones, and thought hell it could do the whole thing, plus I'm now beginning to think it could do other nice things. For example I starting to see a way of having the effect of muscle and bone moving around under the skin without the skin necessarily being rigidly stuck to the things do the moving. This is important in areas like the shoulder where the scapula moves around under the skin making it bulge but isn't actually attached to the surface. If I can get this to work then I already have a way of doing muscle bulging so you could avoid the whole need for inflate, I think in the end this would be a much more efficient solution than raying, if it works

As to the need for it to be at object level, all I meant here was that although my solution gets you 95% of the way to a complete rig there will always be the need to tweak and so having te ability to add painted capture weights at object level like you can with bones would I'm sure be very useful. Also, I was thinking that these capture curves could be created as individual objects like bones are and would have a special parent object that allows for multiple parents to be created with all the transforms passed onto the curves, these would be looked for by the capture sop just as bones are at the moment. The reasoning here was that everyone else I speak to about this seems to want rigging tools to be at object level, personally I tend to do most of my work at sop level ( being a modeller at heart) and so I don't have a problem with it all being inside one object. At the end of the day it's about what is most efficient, can you pick up all the necessary transformations as easily at sop level as you can at object level, and will people understand the concept of parenting a sop to an object

If you are poking around with the file have a look at the colouring for preventing one area from influencing another, I haven't tried a whole body yet but the idea here is that you can be much more free and easy with your capture regions if the skin it's self has attributes on it that tell the system what areas should be influencing other areas. I added it originally just because it was the only way I could get the thing to work without having the ability to paint per curve weights and store them, but as it turned out I rather like the idea and is maybe worth more of a look.

It's very quick to set up.

I think with some more experimenting it might be possible to find an almost ideal set of capture curves that work for a human skin, these could then be very quicky transferred to almost any biped very quickly.

That's the hope anyway

[thekenny]

All very interesting. I had also thought about doing cartoony characters with bones and wires to help keep the line of action in their limbs and torso. Think about ElasticGirl in the Incrediables.

It sounds very cool and I'm glad you have taken the time to try it out. I still have to look at your file... silly work and all.

I think I planted an good seed in that coordinate space idea:)

-k

[Mario Marengo]

Thanks for the explanation, Simon. This is making a lot of sense now.

It strikes me that one of the key contributions here, is taking the concept of wire deformers away from parametric, 1D components (with 3D influence, but still in "free" space, isolated and independent from other components), and putting them to work instead as "structural glue" -- where the binding (and here's the key) is done within the blended spaces of all the solid (3D) components (bones) within a neighborhood. And yes; this is *not* the same as a bone's capture region (or a wire's capture region); it's "inter-bone" bridging -- a directed, cross-spatial component. It augments the concept of capture regions by being able to work *between* the implicit spaces of all (local) capture regions. I like it!

The visual that immediately popped into my head was that of a "path integral" -- a curve inside a vector field, where the curve is Simon's wire deformer, and the vector field is the influence of all the bones in the local neighborhood. As the curve deforms, it injects new force into the field. I know it's not quite the same, but it could be a useful abstraction....

In terms of implementation... hmmmm.... I guess it depends on how you define/interpret this device. In the way that I'm currently thinking about it, it would seem most natural as some kind of an extension to bones... but one layer *above* bones, somehow... but that's just a result of how I'm visualizing it... (what other cross-object components do we currently have in Houdini?). I'll leave that discussion to you guys... I'll just listen in, if you don't mind

Good stuff, Simon!

[sibarrick]

I think I planted an good seed in that coordinate space idea:)

-k

14957[/snapback]

Actually something I still have to try is pumping this same rig through my skinwrap implimention, to do this I can use the stored transforms to manipulate the surface normals and this will give me the reference frame I need to do the deformation... hmmmmm worth a try

[sibarrick]

It strikes me that one of the key contributions here, is taking the concept of wire deformers away from parametric, 1D components (with 3D influence, but still in "free" space, isolated and independent from other components), and putting them to work instead as "structural glue" -- where the binding (and here's the key) is done within the blended spaces of all the solid (3D) components (bones) within a neighborhood. And yes; this is *not* the same as a bone's capture region (or a wire's capture region); it's "inter-bone" bridging -- a directed, cross-spatial component. It augments the concept of capture regions by being able to work *between* the implicit spaces of all (local) capture regions. I like it!

The visual that immediately popped into my head was that of a "path integral" -- a curve inside a vector field, where the curve is Simon's wire deformer, and the vector field is the influence of all the bones in the local neighborhood. As the curve deforms, it injects new force into the field. I know it's not quite the same, but it could be a useful abstraction....

14959[/snapback]

Wow, I think you get it better than I do now!

I think you've encapsulated exactly what I'm trying to explain, but in lovely tech speak that programmers can understand..

Do you have any ideas on better/other ways of blending matrices, interpolating quaternions is all very well but it's a little limited in how easy it is to control the shape of the blend, but I'm guessing there must be nicer ways to do this that can be leveraged from animation techniques. In essence all I'm doing is animating a curve from one parent to another like the blend node does at object level, only I don't do it over time I do it over parametric space. I have one or two ideas on this but they add rather too much overhead for my liking.

[Mario Marengo]

Do you have any ideas on better/other ways of blending matrices, interpolating quaternions is all very well but it's a little limited in how easy it is to control the shape of the blend

14962[/snapback]

Well; I'd really need to spend some quality time thinking about this, but maybe (just maybe), matrices and quat slerping is not the only possible model in this case. I'm just thinking out loud here so please bear with me, but my gut says that those constructs are perhaps better suited to rigid transformations, whereas your wires seem to exist in a place that more closely resemble fluids/cloth (again; this comes from my initial reaction of visualizing them inside vector fields, so my thinking is now biased in that direction).

In your current interpretation, the wire's effect is the result of a transformation based on yet another space -- a space resulting from a blend of local spaces, yes, but a *single* new space nonetheless. Spaces (matrices) allow for rigid transformations (possibly weighed, but still rigid) that concatenate themselves to other transformations in the region of influence (those induced by neighboring bones, etc.). And in this scenario, I think what you're already doing (using quat blending), is probably the best choice. And perhaps any shortcomings you mention are simply a result of this choice of interpretation.

An alternative interpretation may be to think of the wire as a *set* of implicit spaces (one per point), or just a collection of force vectors -- for each point, the implicit basis would be: the tangent to the curve, the normal to the surface, and the cross of those two. Their potential starts off balanced, or null (capture pose); the influence of each transforming bone is then expressed as a set of force vectors in the local curve-point coords, and the curve is then solved based on a set of structural constraints. The resulting curve has a new position and a set of differential vectors that satisfy the constraints -- these are the deformation vectors that are applied to the region of influence. In this model then, the overall behaviour is controlled through constraints (one could, for example, only allow deformations along the normal and/or maintain segment length, etc.). IOW; not a "cloth", but a "string"... maybe worth waiting around for DOPs for this kind'a thing though...

Like I said; just thinking out loud here...

[sibarrick]

An alternative interpretation may be to think of the wire as a *set* of implicit spaces (one per point), or just a collection of force vectors -- for each point, the implicit basis would be: the tangent to the curve, the normal to the surface, and the cross of those two. Their potential starts off balanced, or null (capture pose); the influence of each transforming bone is then expressed as a set of force vectors in the local curve-point coords, and the curve is then solved based on a set of structural constraints. The resulting curve has a new position and a set of differential vectors that satisfy the constraints -- these are the deformation vectors that are applied to the region of influence. In this model then, the overall behaviour is controlled through constraints (one could, for example, only allow deformations along the normal and/or maintain segment length, etc.). IOW; not a "cloth", but a "string"... maybe worth waiting around for DOPs for this kind'a thing though...

Like I said; just thinking out loud here...

14971[/snapback]

Ok now you lost me...

I see what you mean in theory but I think you are right I'll wait for DOPs to come along and see if they have anything to offer. I can't see me hacking this in vex.....

Sounds interesting though, I've never come across differential vectors, I'll have to do some reading.

[TheDunadan]

I think Mario is only referring to vectors that indicate the differential change of the magnitude of velocity vectors on these points of the curve. I'm not quite sure on the correct english math terms, but it's somewhat an ordinary vector but they are called differential vectors since they hold the results of this constrained force system. If you look at the force system in math terms this would be expressed as differential equation system. And since we look at the problem from the point perspective it would be called a lagrangian solution

I hope I'm not all wrong .. but I can edit my comment once Mario has given us a few further insights

Jens

[Mario Marengo]

I think Mario is only referring to vectors that indicate the differential change of the magnitude of velocity vectors on these points of the curve.

14985[/snapback]

Yes; that's exactly what I meant: vectors that represent the amount (and direction) of the correction necessary to satisfy the constraints. Sorry for the confusion... language always gets in the way of these things

Actually; just throw away all the verbiage and keep one thing: you'd be treating the wire as though it were a "string" (i.e: a 1D chunk'o cloth).

Hope that makes more sense

But to directly answer your original question: "no; I personally can't think of a better way to blend transform matrices" -- the only thing I can come up with is a possibly different way to interpret the whole thing (which may or may not be an improvement in the end...only one way to find out). That's what I was going on about.

Cheers!

[sibarrick]

I think Mario is only referring to vectors that indicate the differential change of the magnitude of velocity vectors on these points of the curve. I'm not quite sure on the correct english math terms, but it's somewhat an ordinary vector but they are called differential vectors since they hold the results of  this constrained force system. If you look at the force system in math terms this would be expressed as differential equation system. And since we look at the problem from the point perspective it would be called a lagrangian solution  

14985[/snapback]

Ok now you both lost me...

I should never have given up maths at school

I was actually thinking of a much more mundane solution, I'll leave these other ideas for Sesi or you guys. My other approach is somewhat simpler and goes something like this.

Let the user "design" the cure they want using something like the way the path object does, with CVs and handles, then match the "parented" curves to this solution by calculating a matrix that transforms each point in the blend region to the point on the desired curve. This would then mean I could still maintain a space for doing the deformation but it could be abitarily controlled. At the moment the only control I have is the pivot point and the size of the parametric space of the blend.

Either way I guess the simple answers is there is no other simple way to get more control, quat slerping is as close as I can get to what I'd call simple.