Coding 3D Animation #5: ‘toon shaders

Who’s up for a ‘toon shader? 3D graphics on a computer screen is of course an illusion of depth on a flat surface. Sometimes though, it’s fun, or useful, to lay that illusion bare: to prod the user by pointing out that the painstakingly rendered 3D scene they’re looking at is in fact, a depthless image. Or, perhaps you want to integrate 3D elements in with hand-drawn 2D-style elements, without the former looking too far out of place. Whatever the reason, we see many many 3D visualisations that eschew photorealistic rendering in favour of something a little more, well, flat.

One of the most well-known of these effects is the ‘toon shader, or cel shader. The main element of the toon shader, what makes the image look like it might have been hand-drawn, is a black outline going around the shape. The varying thickness of the line is suggestive of a pen-stroke, and it doesn’t just run around the outer-most edge of the object, but also its internal edges. This for me is the most beautiful part of the effect, and is what suggests that the 3D image might in fact be sitting on a “page”. Perhaps the most famous use of this effect in gaming is in the Borderlands series, while in animated films, I’m sure something like this is used in Belleville Rendezvous (The Triplets of Belleville/ Les triplettes de Belleville) to integrate the 2D and 3D elements.

A secondary, optional part of the effect, is a posterization effect: a drastic reduction in the colour depth of the image, so that only four or five shades (or intensities) are represented. Perhaps the effect here is closer to a cheaply printed comic book. This is an easier effect to achieve, but is less impressive: it looks to my eye more like a 1980s music video than a cel-shaded ‘toon.1

Implementing the ‘toon shader

Source code is here

I got the idea for how to do this from this e-book on Unity:

There are various techniques to achieve this effect in a shader. Unity 3.3 is shipped with a toon shader in the standard assets that renders these outlines by rendering the back faces of an enlarged model in the color of the outlines (enlarged by moving the vertex positions in the direction of the surface normal vectors) and then rendering the front faces on top of them.

What follows is my take on this description. First of all, the black outline. Unfortunately, this is a two-pass process, involving drawing the outline first, and then everything else. So we need a boolean in the shader:

     uniform bool outline; //outline pass of shader?

This indicates whether this is the first pass (where we draw the outline) or the second. We toggle this switch on and off inbetween our draw calls, like this:


On the first pass, we achieve the varying thickness of the outline first of all by extending each vertex out along its normal in the vertex shader:

    if (outline) framePos += frameNorm * 0.005; //expand verts along their normal

If this were the only change we were making, this would have the effect of “inflating” our character slightly, so that they start to resemble a michelin man. This, however, is not going to be an ordinary draw operation.

In the fragment shader, we invert the regular face-culling operation: instead of culling rear-facing fragments (which, unless the object has transparent faces, would not be visible, and would be culled by OpenGL… I think), we render these rear-facing fragments in solid black (this is our outline); and we instead discard the front-facing fragments (as these are now “inflated” along the point’s normal):

            if (gl_FrontFacing) discard; //discard front-facing
            else gl_FragColor=vec4(0.,0.,0.,1.); //draw rest black

On the second pass, the vertex shader acts as normal. In the fragment shader, we achieve the posterization effect by multiplying our varying intensity float by the number of posterization layers we want, and then ceiling-ing the value (the loss of the fractional element is what creates the radical drop in colour-depth, producing the posterization effect). Finally we convert it back to the range 0.0 to 1.0:

            gl_FragColor=vAmbient + vColor * ceil(vIntensity * posterize)*unposterize; //posterize colours

We can determine how many “layers” of posterization we have with a couple of constants defined in the shader’s preamble:

    const float posterize = 5.; //layers of posterization
    const float unposterize = 1./posterize;

If you want to turn the posterization off, then just replace the gl_FragColor line with:

            gl_FragColor=vAmbient + vColor * vIntensity; //non-posterized colours

In case you’re wondering what the intensity variable is, it is identical to the direct diffuse value from our regular lighting shader, but stripped of its colour value. For performance purposes, I’m doing the lighting calculation on the vertex shader, and interpolating the value on the fragment shader. For more details on lighting shaders in Codea, be sure to check out Ignatz’s series of posts on the topic, as well as his e-Books.

That wraps this post up, grab the source, and enjoy! In future posts I’m planning on exploring more examples of playing games with depth, by looking at different projection systems.

Photo-realism is so over-rated.

  1. Perhaps a more interesting effect would be representing the different intensities with a Roy Lichtenstein-style dot-stippling effect. Hmmm, something for a future post I feel. 

One thought on “Coding 3D Animation #5: ‘toon shaders

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s