Blendshape and joint Conversion

Use the Blendshape Conversion widget to convert the output Audio2Face animation into a blendshape-driven animation. To use - Set your input animation mesh (The mesh driven by audio2face) and then Set the Blendshape Mesh to connect to and click “Setup Blendshape Solve.”

The Data Conversion tool's Blendshape Conversion widget.

Element

Description

Input anim mesh

The mesh you want to convert into a blendshape animation. It is usually the mesh from the A2F pipeline.

Blendshape mesh

The neutral mesh with blendshape prims and this should be based on the usdSkel schema.

Set up blendshape solve

Setting up an interactive blendshape solve pipeline in the stage.

The Data Conversion tool's Blendsolve nodes in OmniGraph.

Element

Description

Format

Export the resulting blendshape weight values as .json file or USDskelanim.

Batch

Check this option to batch export multiple Blendshape Solves.

Additional notes

When exporting animation, you will use the information in “common settings” for the file path and the playback fps.

MetaHuman

Now supports Unreal Engine’s MetaHuman via the Omniverse Unreal Engine Connector version 103.1. The exporter creates custom “animation curves” that are imported as an animation sequence to align with the Metahuman skeleton.

Two arrays

custom:mh_curveNames

This token array contains the names of the control curves, like CTRL_expressions_eyeLookUpR.

custom:mh_curveValues

this float array contains the time sampled curve value data.

Advanced Export Options

  • Joint - L Eye: Allows for the Export of Left Eye Joint transform data.

  • Joint - R Eye: Allows for the Export of Right Eye Joint transform data.

  • Joint - Jaw: Allows for the export of the Jaw’s pivot as joint transform data.

  • Tuned Weights: If you have tuned the weights using the Blendshape Weight Tuning Node - add the node here to include in the export.

For additional setup options - Select the “BlendShapeSolve” Omnigraph node under the Audio2Face Main Graph node.

../../_images/a2f_BlendSolveProperties.png

Pose & pose option table

When you set up blendshape solve, you can access the pose options:

Option

Effect

Pose

Name of the blendshape poses.

Use

whether we want to include or exclude the pose in the blendshape solve.

Cancel pair

tagging 2 poses that cancel each other. They cannot be used together at a frame, only one of them can get a non-zero weight.

Symmetry

tagging 2 poses that are symmetric. They try to have the same weights during the blendshape conversion.

Select All, Selection options

(Right mouse click menu): Handy pose selection options.

The Pose Options table.

Toggle use

Toggle the selected “use” checkboxes in the table.

Tag/untag cancel pair

Tag/untag the selected 2 poses as a cancel pair.

Tag/untag symmetry pair

Tag/untag the selected 2 poses as a symmetry pair.

Weight regularization

Stronger value prevents the solve to compute strong weight values. Weight regularization.

Temporal smoothing

Stronger value prevents pops between consecutive frames. Temporal weight regularization.

Weight Sparsity

Stronger value generates sparse blendshape weights, which is preferable for animator’s edit. L1 weight regularization.

Symmetric Pose

Stronger value makes the symmetry poses to have more similar weight values.

Load preset

Load preset blendshape options from a .json file.

Save preset

Save blendshape options as preset in a .json file.

Blendshape Weights Tuner

Each Pose on a given mesh has a different range of motion. The Blendshape Weight Tuner provides the ability to modify the blendshape weight using gain & offset, so you can amplify/reduce/offset some pose weights as you want. This tuning process can greatly improve the animation result on the target face. The Blendshape Weight Tuner node can be found within the Audio2face Graph. Select the node and view it’s properties to adjust the offsets.

../../_images/a2F_blendshapesWeightTuner.png

Set Selected Gains

Set the gain values for the selected weights on the list. 0 sets the selected gains to 0. 1 sets the selected gains to 1. Reset sets the gain value to the default value stored in the input Gains attribute.

Set Selected Offset

Set the offset values for the selected weights on the list. 0 sets the selected gains to 0. 1 sets the selected gains to 1. Reset sets the offsets value to the default value stored in the input Offsets attribute.

Save Preset

Saves the current mapping to a json file.

Load Preset

Sets the mapping from a json file.

BlendShape Source files

  • You can find the required blendshape mesh assets in the Samples Tab under BlendShapes. Provided are BlendShape solutions for both Mark and Claire Models. Follow the Same Setup procedure for each mesh you which to generate data for.

  • To Setup - Drag and drop the _tongue_bs file into your stage window. The Blendshape assets are blue for easy identification from the driven mesh.

The Pose Options table.