Epic MetaHuman sculpt workflow.

[kroaton]

I feel a bit outside of my comfort zone on this one (and probably biting off more than I can chew), so I thought I'd ask for some advice from some of you more knowledgeable folk: 

As of right now, there doesn't seem to be a well put together sculpt workflow for the Epic (UE4) MetaHuman Creator rigs. 
Exporting the head to Zbrush and back to UE4 (while maintaining the same topology) breaks the rig, as the bones will not be in the proper positions anymore and the skin weights will be messed up.
Houdini seems to be well suited for dealing with this, but not being a character rigger by trade, I thought it might be worth asking if I'm missing something obvious and if this sounds like a fool's errand.

The current idea/ progress:

Export a MetaHuman from the Quixel Bridge -> Load in UE4 -> Export the head FBX (with all LODs)
Import the MetaHuman head mesh in Houdini, blast the LODs, rig, groom, all attributes, etc.
Send to Zbrush -> sculpt while keeping the same topology, prim and point count, etc.
Import back to Houdini -> attribute copy back everything from the original mesh (apart from the boneCapture) -> attribute transfer the boneCapture points from the original mesh to the sculpt (this should work well due to both heads having the same point/prim/vert count).

Current problem to solve: 
Move (or lengthen maybe?) the bones based on vertex ID to fit the new sculpt from Zbrush

This could be done via VEX: find closest point to the mesh on the bone -> move the pivot to that point -> lerp() from old to new mesh -> move bones based on distance from the closest point's former location to the new location

https://drive.google.com/file/d/1DnuNlSOiJZZJ0RmSv5_g_BqSR-VMRxZO - File if anyone wants to play around with this.

Questions: 
Would it be better to move or lengthen each bone? 
Would using KineFX's IK solver to pull on the entire bone hierarchy as I move the bones closer to the new target mesh location be better/worse than just moving or lengthening them?

edit: Will on the Discord suggested using "the old and new geo to point deform the rig. The only trouble would be the bind pose" as a possible option

Any other thoughts or ideas?

Cheers.
/rant

[Trilec]

Hi kroaton ,

Ive also started looking at this problem, I'm at the beginning stages but will Post my thoughts and considerations.

Some discoveries:

*Adjusting the corrective blend shapes particular around the eyes and lips. (and re-baking these) are problematic.
(hoping for procedural techniques that can be implement it here on the correctives.

*adjusting/positioning the bones to match the new sculpt. (as you mentioned)

*readjusting displacement maps for micro wrinkles etc, to the new sculpt locations.(again thinking possible procedural methodologies)

*Pay attention to the whole Z-up /Y-Up in unreal/Maya/Houdini.

My Observations.

The BODY

*Uses an adapted (RBF, Radial Basis Function ) solver node plugin for use on the BODY leaf nodes. (one would need to be developed for Houdini)
    ie. "clavicle_r_UE4RBFSolver" makes it easier to control correctives or blend shapes (orientation) and I assume to maintain compatibility with UE/DCC apps
*The driver skeleton "root_drv" is the control rig in maya.
*animation is exported from skeleton to unreal these curves can use "rig-inversion" to target different body types (within the meta-human rig setup)

The HEAD

*Separate workflow from BODY

*The Maya facial animation interface board is the same as the one in unreal (one-to-one curve representation).
*The interface board creates the input curves ( and with a little bit of rig-logic) creates the facial expressions ( a mixture of joint movement, blend shapes and textures)

*ideally any new sculpted head shape should use Delta based blend shapes to minimize the existing corrective blend shapes.
*still not 100% sure about the bone placement but ideally they should be close to the skin. (Looking at several meta human facial rigs this is fairly consistent)

(lengthening rather than moving?)

 

so far the workflow seems to be:

Extract into maya from bridge (maya 2020, currently supported by plug-in)

export to fbx or abc

import to Houdini

...do stuff (if applying animation correctives and RBFs won't correctly work, however the animation curves /model adjustments can be applied back in maya

... next

 

Anyway, these are just some of my thoughts in the early stages and hopefully helps others.

 

Curt

 

 

 

 

Edited May 5, 2022 by Trilec
Updated

[Keshaw singh]

looks complicated