Renders an animated glTF model with vertex skinning. The sample is based on the [glTF scene](../gltfscene) sample, and adds data structures, functions and shaders required to apply vertex skinning to a mesh.
## Description
This example demonstrates how to load and use the data structures required for animating a mesh with vertex skinning.
Vertex skinning is a technique that uses per-vertex weights based on the current pose of a skeleton made up of bones.
Animations then are applied to those bones instead and during rendering the matrices of those bones along with the vertex weights are used to calculate the final vertex positions.
A good glTF skinning tutorial can be found [here](https://github.com/KhronosGroup/glTF-Tutorials/blob/master/gltfTutorial/gltfTutorial_020_Skins.md), so this readme only gives a coarse overview on the actual implementation instead of going into full detail on how vertex skinning works.
Note that this is not a full glTF implementation as this would be beyond the scope of a simple example. For a complete glTF Vulkan implementation see https://github.com/SaschaWillems/Vulkan-glTF-PBR/.
Several new data structures are required for doing animations with vertex skinning. The [official glTF spec](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#skinned-mesh-attributes) has the details on those.
To calculate the final matrix to be applied to the vertex we now pass the indices of the joints (see below) and the weights of those, which determines how strongly this vertex is influenced by the joint. glTF support at max. four indices and weights per joint, so we pass them as four-component vectors.
#### Skin
[glTF spec chapter on skins](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#skins)
This struct stores all information required for applying a skin to a mesh. Most important are the ```inverseBindMatrices``` used to transform the geometry into the space of the accompanying joint node. The ```joints``` vector contains the nodes used as joints in this skin.
We will pass the actual joint matrices for the current animation frame using a shader storage buffer object, so each skin also get's it's own ```ssboo``` along with a descriptor set to be bound at render time.
The animation sampler contains the key frame data read from a buffer using an accessor and the way the key frame is interpolated. This can be ```LINEAR```, which is just a simple linear interpolation over time, ```STEP```, which remains constant until the next key frame is reached, and ```CUBICSPLINE``` which uses a cubic spline with tangents for calculating the interpolated key frames. This is a bit more complex and separately documented in this [glTF spec chapter](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#appendix-c-spline-interpolation).
**Note:** For simplicity, this sample only implements ```LINEAR``` interpolation.
The animation channel connects the node with a key frame specified by an animation sampler with the ```path``` member specifying the node property to animate, which is either ```translation```, ```rotation```, ```scale``` or ```weights```. The latter one refers to morph targets and not vertex weights (for skinning) and is not used in this sample.
The animation itself then contains the animation samplers and channels along with timing information.
#### Loading and passing the data
##### Vertex attributes
This samples adds two new vertex attributes for passing per-vertex `joints` and `weights` information. As with other per-vertex attributes, these need to be loaded using glTF accessors in `VulkanglTFModel::loadNode`:
```cpp
// Get vertex joint indices
if (glTFPrimitive.attributes.find("JOINTS_0") != glTFPrimitive.attributes.end())
As usual we check if those attributes actually exist inside the glTF file, and if so we use the accessor and buffer view to get a pointer to the required vertex attributes.
The new attributes are added to the vertex input state of our pipeline in `VulkanExample::preparePipelines` at locations 4 and 5:
Loading skins is done in `VulkanglTFModel::loadSkin` and aside from getting the required data from the glTF sources into our own structures, this method also creates a buffer for uploading the inverse bind matrices:
As with e.g. vertex attributes we retrieve the inverse bind matrices from the glTF accessor and buffer view. These are used at a later point for generating the actual animation matrices.
We then create a shader storage buffer object (for each skin) big enough to hold all animation matrices. See [Updating the animations](#UpdatingAnimation) for how these are calculated and updated.
**Note**: For simplicity we create a host visible SSBO. This makes code easier to read. In a real-world application you'd use a device local SSBO instead.
##### Animations
Loading the animation data is done in ```VulkanglTFModel::loadAnimations``` and is also mostly about getting the required data from glTF into our own sources, including sampler and channel data.
The most interesting part is getting the input and output pairs for the animation's samplers, which are loaded using glTF's accessors.
We first load the **sampler's input values**, containing keyframe time values. These are also used to determine the start and end time of the animation:
So if the animation's channel's path is set to ```translation```, the keyframe output values contain translation values that are applied to the channel's node depending on the current animation time.
#### <a name="UpdatingAnimation"></a>Updating the animation
With all required structures loaded, the next step is updating the actual animation data. This is done inside the ```VulkanglTFModel::updateAnimation``` function, where the data from the animation's samplers and channels is applied to the animation targets of the destination node.
We first update the active animation's current timestamp and also check if we need to restart it:
Next we go through all the channels that are applied to this animation (translation, rotation, scale) and try to find the input keyframe values for the current timestamp:
for (size_t i = 0; i <sampler.inputs.size()-1;i++)
{
// Get the input keyframe values for the current time stamp
if ((animation.currentTime >= sampler.inputs[i]) && (animation.currentTime <= sampler.inputs[i + 1]))
{
// Calculate interpolation value based on timestamp, Update node, see next paragraph
}
}
}
```
Inside the above loop we then calculate the interpolation value based on the animation's current time and the sampler's input keyframes for the current and next frame:
Note the use of the ```getNodeMatrix``` function which will return the current matrix of a given node calculated from the node hierarchy and the node's current translate/rotate/scale values updated earlier. This is the actual matrix that's updated by the current animation state.
After this we copy the new joint matrices to the shader storage buffer object of the current skin to make it available to the shader.
#### Rendering the model
With all the matrices calculated and made available to the shaders, we can now finally render our animated model using vertex skinning.
Rendering the glTF model is done in ```VulkanglTFModel::draw``` which is called at command buffer creation. Since glTF has a hierarchical node structure this function recursively calls ```VulkanglTFModel::drawNode``` for rendering a give node with it's children:
There are two points-of-interest in this code related to vertex skinning.
First is passing the (fixed) model matrix via a push constant, which is not directly related to the animation itself but required later on on the shader:
And we also bind the shader storage buffer object of the skin so the vertex shader get's access to the current joint matrices for the skin to be applied to that particular node:
The skin matrix is a linear combination of the joint matrices. The indices of the joint matrices to be applied are taken from the ```inJointIndices``` vertex attribute, with each component (xyzw) storing one index, and those matrices are then weighted by the ```inJointWeights``` vertex attribute to calculate the final skin matrix that is applied to this vertex.
