Coding 3D Animation #4: managing multiple actions

In the previous posts in this series, we’ve looked at getting an animated model out of Blender, and then recreating that animation in Codea by interpolating between a set of keyframes with a Catmull-Rom spline in the vertex shader. We ended up with some code that loaded up a model and then animated a continuous walking loop cycle.

Chances are though, that for most of the objects in the game, we’re not just going to want them to animate in a continuous loop; rather we’ll want them to respond to various inputs and triggers around them, and respond accordingly, In this post then, we’re going to look at how to add multiple actions to a rig: from how to manage your increasingly complex rig in Blender, through to the changes we’ll need to make to our code to handle multiple actions.

Organising multiple actions in Blender

The blend file is available here. It is based on this public domain model by Teh Joran. I used Blender’s Rigify feature to automatically add the skeleton. It was my first time using Rigify, and I didn’t spend much time lining up the bones correctly. I didn’t adjust the bone weightings either. So all things considered, the rig came out looking OK!

As well as a walk cycle, I’ve added a karate-esque kick, and a two-frame stand cycle to the Blender model. Rather than string these altogether on the same timeline, I’ve separated them out into different actions. When you select the Animation layout in Blender (from the screen layout at the top of the page), it defaults to showing you the Dope Sheet. Change the context to the Action Editor, and this will expose a filter menu for your different actions. When you hit the plus sign to create a new action, it will copy the currently selected action to the new action.

A crucial tip: when creating a new action, make sure you hit the “F” button next to the action selector (this will make an F appear next to the action’s name in the selection menu when the action is not selected). If you don’t do this, then the action will NOT be saved if it is not the currently selected action (don’t you just love the Blender UI?).

I’ve highlighted these areas in the screenshot below:

Blender Action Editor Blender’s Action Editor

For a more detailed description of the action editor, see this video.

When using the Action Editor, the export functions will only export the currently active animation, so you’ll need to do an export for each action. I recommend including the name of the action in the filename, ie CaptainCodeaWalk or whatever. See part 2 of this series for details of how to export.

Organising multiple actions in Codea

Link to the new source code

We’re going to need to make some changes to the rig class in order to store all of these actions, and to the way the animation is loaded into the shader. In the Assets tab, the urls for the .obj files are now nested in sub-tables that are named after the action they contain, default (for the default, at rest, standing pose), walk, and kick:


The Rig class then loads these frames into the self.obj table, except now the table has an extra layer to store the actions. The identifier keys for the action sub-arrays are taken from the table in Assets, so that, for example, the second frame in the walk cycle, will be stored at self.obj.walk[2].

We also need to change the way that we load the keyframes into the shader. The reason is that the way that Open GLES 2.0 is implemented in iOS, the number of vertex attributes that you can place in a shader is limited to sixteen. That means there is only enough space for seven keyframes (7 position vectors + 7 normal vectors + 1 colour + 1 texCoord = 16 vertex attributes). If we want out model to have multiple actions, we are probably going to exceed this limit. For our Captain Codea model, I have four walk key frames, three kick key frames, and a stand frame, so that’s already eight frames.

So here’s how we are going to work around this restriction. We will keep the default stand frame in the default position attribute, as when any action ends we will want the model to interpolate back to the default pose. As arrays in Open GL are indexed from zero, not one, this default resting pose will be position zero. Then, as each action is activated, we will load the required frames into position1, position2 etc. Currently, the code just loads all of the frames at once, at the beginning of the action. There are ways this could be improved,1 but for now this works well.

Changes to the shader

I’ve also changed the way that the shader accesses the various frames. As I discussed in part 3, vertex attributes cannot be arrays in Open GLES 2.0. My previous solution to this was to write a hash look-up function for the position and normal pseudo-arrays:

    vec3 getPos(int no) //home-made hash, ho hum.  
        if (no==1) return position;
        if (no==2) return position2;
        if (no==3) return position3;
        if (no==4) return position4;

This time round I simply load all of the values into a local table in the main function. In more recent versions of GLES you can initiate a table when you declare it, like this:

vec3 pos[5] = vec3[](position, position1, position2, position3, position4)

But this isn’t possible in GLES 2.0. So instead we have to go for the some what more verbose:

vec3 pos[5];
pos[0] = position;
pos[1] = position1;
pos[2] = position2;
pos[3] = position3;
pos[4] = position4;

I haven’t profiled this, but it should be quicker than repeatedly calling a look-up function with a set of branching if statements.


Well, it just goes to show that you should profile not presume: it turns out that the hash look-up method is faster. I’ve updated the source code on Github to restore the former look-up method

Transitioning between animations

Finally, we need a way to smoothly interpolate between different animations. To keep things simple, I assume that the model has a default “at rest” state (in our case, just standing), and any interpolation between actions is going to go via the default pose. So at the beginning of an action we need to interpolate from the resting state into the action cycle; and when an animation ends, we need to interpolate back to the resting state, and then stop the animation.2 Currently, if you are mid-walk at the moment that you hit the kick button, the kick animation actually starts from the standing position rather than from a frame in the walk cycle. You could interpolate between the two via the resting pose, although this would increase the delay between the kick button being hit and the action completing. Games like the original Prince of Persia prioritise realistic and smooth rotoscoped action over quick player-response,3 whereas more arcade-y interpretations have traditionally prioritised instantaneous player response over this kind of realism/ smoothness. So how much effort you want to put into smoothly switching between animations depends in part on the kind of “feel” you’re going for. In this code, the kick animation is so quick that I don’t find the transition too jarring.

So, we cue up a new animation like this:

model:cueAnim("walk", {0,1,2,3,4})

First we have the name of the action as a string, "walk". This is the same as how the action is named in the Assets tab. Then we have the frames. We want this action to interpolate from the standing position, so the first frame is 0, the standing position. Then comes the walk cycle, 1,2,3,4. The leading zero is automatically stripped away by a tween.delay in Rig:cueAnim (so that the character doesn’t come to a stand-still each cycle).

To bring the character back to a stand-still, two zeros are placed at the end of a series of frames. Rig:anim detects this and clears the frame buffer, bringing the animation to a halt. For a once-only action like a kick, we can add the two zeros at the end when we cue the animation. Here is the kick animation being cued (note that it is a “ping-pong” animation, not a loop):

model:cueAnim("kick", {0,1,2,3,2,1,0,0}, 0.1)

The 0.1 at the end is the speed, expressed as the amount of key frames to advance by each program cycle. Here, 0.1 key frames per cycle means that there’ll be a new key frame every 10 ticks. The default is a new keyframe every 20 ticks, so the kick is a much faster action than the walk. A more sophisticated system would be able to vary the frame-rate between keyframes (eg to make the aftermath of the kick slower than the initial release), but for now, a fixed frame-rate per action will have to suffice.

Other extras

I’ve added a simple UI so we can check our animation. Hold and drag anywhere on the left side of the screen to activate a virtual stick that makes our character walk, and controls his direction (so that you can check the model from various angles). Tap anywhere on the right half of the screen to make our hero kick. I’ve also added a floor, so that we can check the speed of the walk cycle, and to give the model some context.

Given that the loader could now be loading up many more frames, I’ve placed the rig loader into a coroutine, so that we can give some feedback to the user (a “loading” status bar) as to how the load is progressing. In future, I should perhaps investigate saving each keyframe’s data in an interim format (perhaps just a json table dump), so that the obj file does not have to be parsed every time we run the program. That should significantly speed up load times, and perhaps obviate the need for a loading status bar. That’s for a future post though.

Up next

If you look at the image at the top of this post, you can see a funky effect: a toon shader, perhaps most associated with the game Borderlands. I’ll share the code in the next post, so subscribe with one of the options (WordPress, RSS, email) in the sidebar.

  1. I should at least check whether the new action is the same as the last one that just competed, if so, there is no need to reload the frames. A still more sophisticated system would gradually introduce the new animation into the shader, one frame at a time as each frame is needed. This would distribute the frame loading over several program cycles, and would allow for smoother spline interpolation between the frames of various actions. 
  2. I have a second “at rest” keyframe, in case I we want to animate the resting state. I haven’t implemented this yet. 
  3. In the original Prince of Persia series, if you stopped running for instance, but your character was mid-run-cycle, he wouldn’t stop immediately. His foot would have to come down, and then he’d gradually come to a halt. By this point he might have overshot the ledge he was on and plunged to his death in a spike-filled pit. This realism was all part of the “fun” of course. 

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s