O'Reilly logo

iPhone 3D Programming by Philip Rideout

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 7. Sprites and Text

My god, it’s full of stars.

Dave Bowman, 2001: A Space Odyssey

Even though OpenGL ES is designed for 3D graphics, you’ll often find the need to render visual elements that are 2D. OpenGL is actually quite well-suited to rendering a flat world; several popular 2D libraries, such as cocos2d, use OpenGL as their rendering engines.

The most common type of 2D rendering is text rendering. The OpenGL API lives too close to the metal to treat text as a first-class citizen, but it’s easy to render a pregenerated glyph (a character’s shape in a given font) using a textured quad, and that’s the approach we’ll take in this chapter.


Computing the points along the outline of a glyph can be surprisingly complex. For example, the TrueType file format specifies a unique programming language—complete with loops and if statements—solely for the purpose of tweaking the curves in a glyph.

In this chapter, we won’t attempt to go over kerning algorithms, ligatures, or line wrapping; simple text layout is good enough for our purposes. (Check out the popular pango library if you need a full-featured layout engine.)

Another common 2D concept is the sprite, which is a rather generic term for any bitmap that gets composited into a scene. Sprites often contain transparent regions, so their texture format contains alpha. Sprites are often animated in some way. There are two ways of animating a sprite: its screen position can change (imagine a bouncing ball), or its source image can change (imagine a ball that’s spinning in place).

The iPhone supports two extensions to OpenGL ES 1.1 that make it easy to render sprites: GL_OES_draw_texture and GL_OES_point_sprite. We’ll make good use of both these extensions throughout the chapter, and we’ll wrap up the chapter with a fun sample application that renders a spring system with sprites.

Text Rendering 101: Drawing an FPS Counter

Rather than demonstrating text rendering with yet another goofy toy application, let’s do something useful for a change. Overlaying a frames-per-second counter in one corner of the iPhone screen provides a quick and easy way of evaluating graphics performance; see Figure 7-1.

FPS counter

Figure 7-1. FPS counter


For more sophisticated runtime analysis of graphics performance, Apple provides an excellent free tool called Instruments, which we’ll cover in a subsequent chapter.

Before writing any application code, you’d need to generate an image that contains bitmaps for the numbers zero through nine, as depicted in Figure 7-2. (Don’t bother trying to create this yet; you’ll see a way to automate this shortly.)

Numerals in a 128x32 luminance texture

Figure 7-2. Numerals in a 128x32 luminance texture

You probably already guessed that you need to store off the bounding box of each glyph in order to compute the appropriate texture coordinates. Thinking about this a bit more, you’ll realize a mere bounding box is not enough. When you’re writing a sentence on ruled paper, some parts of letters extend below the baseline, like the descender of the lowercase p. And, in the case of the rather artsy font shown in Figure 7-3, the type designer wants the 9 numeral to be vertically offset from the other letters. Further complicating matters is the fact that the bounding boxes of glyphs can overlap in the destination image. In Figure 7-3, observe how the descender of the letter p extends right below the letter i.

Simple text layout with baseline

Figure 7-3. Simple text layout with baseline

It turns out that associating a specific set of glyph metrics with each character supplies enough information to achieve the simple text layout shown in Figure 7-3. A popular naming convention for these metrics is described in Figure 7-4; in this diagram, the origin represents the current pen position.

Glyph metrics: bearing and advance vectors; width and height lengths

Figure 7-4. Glyph metrics: bearing and advance vectors; width and height lengths

To summarize, the four glyph metrics are as follows:

Bearing vector

2D vector describing the offset from the pen position.

Advance vector

2D vector describing how to advance the pen to the next position after rendering the current glyph. The y component is always zero for Latin-based alphabets.


The horizontal length of the glyph.


The vertical length of the glyph.

Using these metrics, Example 7-1 the pseudocode for a simple text layout algorithm.

Example 7-1. Simple text layout algorithm

void RenderText(string s, vec2 p)
    for each character c in s
        metrics m = GlyphTable[c].Metrics
        vec2 topLeft = GlyphTable[c].Position
        box source = box(topLeft, m.Width, m.Height)
        box dest = box(p + m.BearingVector, m.Width, m.Height)
        Blit(source, dest)
        p += m.AdvanceVector

Generating a Glyphs Texture with Python

Before writing any application code, we need to choose a way of generating a glyphs texture and a set of metrics for each glyph.

Leveraging Quartz is perhaps the most obvious way of generating a glyphs texture (see Generating and Transforming OpenGL Textures with Quartz). This can be done at runtime when your application first starts up. This might slow down your startup time by a tiny amount, but it has the benefit of shrinking the size of the application bundle.

My preference is to generate the glyphs texture as a build step, mostly because it simplifies my application code. Build steps take place in Xcode rather than the iPhone execution environment, which brings a much richer tool set to the table. This is a perfect use case for a scripting language, and Python comes to mind first.


There are many ways of generating a glyphs texture; here I’m giving an overview of my favorite. Take it only as a high-level example.

Given that we’re using Python in a build step, we need to find some useful Python modules for image manipulation and image generation. At the time of this writing, the Python Imaging Library (PIL) is the most popular imaging manipulation module, and it provides excellent support for manipulating PNG images at a low level. However, it’s not quite enough on its own because it doesn’t provide direct access to the glyph metrics that we need. Another popular library is Cairo, which has a well-maintained Python binding called pycairo. Cairo is robust and fast, and it’s used as the rendering backend in Firefox, Mono, and GTK. So, let’s go with PIL (http://www.pythonware.com/products/pil/) and pycairo (http://www.cairographics.org/pycairo/).

Rather than packaging the glyphs texture as a PNG or PVR file, let’s serialize the data to a C header file. Since it’s a single-channel texture, the amount of data is relatively small. The header file will also provide a convenient place to store the glyph metrics. We’ll simply have our Python script spawn PVRTexTool (see the section entitled The PowerVR SDK and Low-Precision Textures) for generating the header file from the image. We’ll still generate a PNG file for preview purposes, but we won’t include it in the application bundle. See Example 7-2 for the complete Python script that generates Figure 7-2.

Example 7-2. CreateNumerals.py

import cairo
import os
from PIL import Image

# Create a Cairo image surface:
imagesize = (256,32)
surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, *imagesize) 1
cr = cairo.Context(surface)
padding = 3

# Choose a font (look in /Library/Fonts) and set up the transforms.
cr.select_font_face("Apple Chancery", 
                    cairo.FONT_WEIGHT_BOLD) 2

# Create a string for appending the glyph metrics to the texture file:
glyphs = '''4
struct GlyphPosition {
    int X;
    int Y;
struct GlyphMetrics {
    int XBearing;
    int YBearing;
    int Width;
    int Height;
    int XAdvance;
    int YAdvance;
struct Glyph {
    GlyphPosition Position;
    GlyphMetrics Metrics;
static const Glyph NumeralGlyphs[] = {\n'''

# Render glyphs '0' through '9' and write out their extents:
x, y = 0, 0 5
for character in '0123456789': 6
    extents = cr.text_extents(character) 7
    x_bearing, y_bearing, width, height, x_advance, y_advance = extents
    glyphs += '    {{ %d, %d }, ' % (x, y) 8
    glyphs += '{ %d, %d, %d, %d, %d, %d }},\n' % extents 9
    cr.translate(x, -y_bearing)
    cr.fill() 10
    x += width + padding 11
glyphs += '};\n'

# Extract the alpha channel and open it up for a quick preview:
surface.write_to_png("NumeralsTexture.png") 12
image = Image.open("NumeralsTexture.png") 13
image.split()[3].save("NumeralsTexture.png") 14
os.system("open NumeralsTexture.png")

# Serialize the image data to a C header file:
os.system('PVRTexTool -h -yflip1 -fOGL8 -iNumeralsTexture.png') 15

# Write to the header file:            
headerFile = open('NumeralsTexture.h', 'a') (16)


For this to work, you must either put the location of PVRTexTool (The PowerVR SDK and Low-Precision Textures) into your shell’s PATH environment variable or copy PVRTexTool into one of your PATH entries, such as /usr/local/bin. If you’ve extracted the Khronos SDK into your current directory, you could copy the file and mark it executable with these commands:

cd SDKPackage/Utilities/PVRTexTool/PVRTexToolCL/MacOS
sudo cp PVRTexTool /usr/local/bin
sudo chmod +x /usr/local/bin/PVRTexTool
cd -

Cairo is a fairly extensive library and is beyond the scope of this book, but here’s a brief explanation of Example 7-2:


Create a 256×32 image surface with Cairo, and then create a context associated with the surface.


Select a TrueType font file, and then choose its size. In this case, I’m selecting the Apple Chancery font found in /Library/Fonts.


Set Cairo’s current draw color to white.


Initialize a string that we’ll later append to the header file. For starters, define some structs for the glyphs table.


Initialize the pen position to (0, 0).


Iterate over glyphs 0 through 9.


Obtain the metrics for the glyph.


Populate the GlyphPosition structure that we’re defining in the generated header file.


Populate the GlyphMetrics structure that we’re defining in the generated header file.


Tell Cairo to fill in the glyph shape.


Advance the pen position with some padding.


Save the Cairo surface to a PNG file.


Load the PNG image into PIL.


Use PIL to extract only the alpha channel, and then overwrite the PNG file.


Use PVRTexTool to serialize the image data to a C header file. (At this point, the PNG is no longer needed.)


Append the metrics data to the same header file that defines the image data.

If you’d like, you can add the number-generation script to your Xcode project and make it into a build step, similar to what we did for texture compression (Texture Compression with PVRTC). For simplicity’s sake, I chose not to do this in the sample project that you can download from this book’s website.

Rendering the FPS Text

Now that we’re past the grunt work of generating the glyphs texture, we can move on to the actual rendering code. A frames-per-second counter is much more useful than our other toy demos, so this time let’s strive to make the rendering code very self-contained and easy to integrate into any project. We can do this by creating a C++ class wholly implemented within a single header file. Example 7-3 shows the basic outline for this class.

Example 7-3. FpsRenderer.h skeleton

#include <OpenGLES/ES1/gl.h>
#include <OpenGLES/ES1/glext.h>
#include <mach/mach.h>
#include <mach/mach_time.h>
#include "../Textures/NumeralsTexture.h"

typedef unsigned int PVRTuint32;

struct PVR_Texture_Header {
    // ...see PVRTTexture.h in the PowerVR SDK...

class FpsRenderer {
    FpsRenderer(vec2 windowSize)
    void RenderFps()

    static const int MaxNumDigits = 3;
    static const int VertsPerDigit = 6;
    static const int FloatsPerVert = 4;
    static const int FloatsPerDigit = VertsPerDigit * FloatsPerVert;
    static const int TexCoordOffset = sizeof(float) * 2;
    static const int BytesPerVert = sizeof(float) * FloatsPerVert;
    static const int BytesPerDigit = sizeof(float) * FloatsPerDigit;
    uint64_t GetElapsedNanoseconds()
        uint64_t current = mach_absolute_time();
        uint64_t duration = current - m_previousTime;
        m_previousTime = current;
        mach_timebase_info_data_t info;
        duration *= info.numer;
        duration /= info.denom;
        return duration;

    float* WriteGlyphVertex(const Glyph& glyph, vec2 pos, int corner, float* vertex)1
    double m_filterConstant;2
    double m_fps;3
    uint64_t m_previousTime;4
    vec2 m_windowSize;5
    vec2 m_textureSize;6
    GLuint m_textureHandle;7
    GLuint m_vbo;8

Private method that generates the vertex and texture coordinates for one of the corners in a glyph rectangle.


Smoothing factor for the low-pass filter; this is explained further in the next section.


Exponentially weighted moving average of the frame rate (again, this is explained in the next section).


Timestamp in nanoseconds of the most recent call to RenderFps().


Width and height of the viewport (usually 320×480).


Width and height of the glyphs texture.


The OpenGL ID of the glyphs texture object.


The OpenGL ID of the vertex buffer object used for rendering the glyphs.

Stabilizing the counter with a low-pass filter

To prevent the FPS counter from fluctuating wildly, we’ll using a low-pass filter similar to the one we used for the accelerometer (see Adding accelerometer support). The application can compute a constant called the smoothing factor, which is always between zero and one. Here’s one way of doing so:

double ComputeSmoothingFactor(double sampleRate, double cutoffFrequency)
   double dt = 1.0 / sampleRate;
   double RC = 1.0 / cutoffFrequency;
   return dt / (dt + RC);

In the previous listing, cutoffFrequency and sampleRate help define what constitutes “noise” in the signal. However, for our purposes, computing a smoothing factor like this is a bit pedantic; pragmatically speaking, it’s perfectly fine to come up with a reasonable number through experimentation. I find that a value of 0.1 works well for a frame rate counter. A higher smoothing factor would result in a more spastic counter.

Fleshing out the FpsRenderer class

Let’s go ahead and implement the constructor of the FpsRenderer class; see Example 7-4. It’s responsible for loading up the glyphs texture and creating the empty VBO for rendering up to three digits.

Example 7-4. FpsRenderer constructor

FpsRenderer(vec2 windowSize)
    m_filterConstant = 0.1;
    m_fps = 0;
    m_windowSize = windowSize;
    m_previousTime = mach_absolute_time();
    glGenTextures(1, &m_textureHandle);
    glBindTexture(GL_TEXTURE_2D, m_textureHandle);
    PVR_Texture_Header* header = (PVR_Texture_Header*) NumeralsTexture;
    const unsigned char* bytes = (unsigned char*) NumeralsTexture;
    const unsigned char* imageData = bytes + header->dwHeaderSize;
    GLenum type = GL_UNSIGNED_BYTE;
    GLenum format = GL_ALPHA;
    int w = header->dwWidth;
    int h = header->dwHeight;
    m_textureSize = vec2(w, h);
    glTexImage2D(GL_TEXTURE_2D, 0, format, w, h,
                 0, format, type, imageData);
    glGenBuffers(1, &m_vbo);
    glBindBuffer(GL_ARRAY_BUFFER, m_vbo);
    int totalSize = BytesPerDigit * MaxNumDigits;
    glBufferData(GL_ARRAY_BUFFER, totalSize, 0, GL_DYNAMIC_DRAW);

The FpsRenderer class has only one public method; see Example 7-5. This method is responsible for updating the moving average and rendering the digits. Note that updating the VBO is quite a hassle; we’ll demonstrate a much simpler way of rendering textured rectangles in the next section.

Example 7-5. RenderFps() method

void RenderFps()
    uint64_t deltaTime = GetElapsedNanoseconds();
    double fps = 1000000000.0 / deltaTime;
    double alpha = m_filterConstant;
    m_fps = fps * alpha + m_fps * (1.0 - alpha);
    fps = round(m_fps);
    char digits[MaxNumDigits + 1] = {0};
    sprintf(digits, "%d", (int) fps);
    int numDigits = strlen(digits);
    vec2 pos(5, 10);
    vector<float> vbo(numDigits * FloatsPerDigit);
    float* vertex = &vbo[0];
    for (char* digit = &digits[0]; *digit; ++digit) {
        int glyphIndex = *digit - '0';
        const Glyph& glyph = NumeralGlyphs[glyphIndex];
        vertex = WriteGlyphVertex(glyph, pos, 0, vertex);
        vertex = WriteGlyphVertex(glyph, pos, 1, vertex);
        vertex = WriteGlyphVertex(glyph, pos, 2, vertex);
        vertex = WriteGlyphVertex(glyph, pos, 2, vertex);
        vertex = WriteGlyphVertex(glyph, pos, 3, vertex);
        vertex = WriteGlyphVertex(glyph, pos, 1, vertex);
        pos.x += glyph.Metrics.XAdvance;
    glBindBuffer(GL_ARRAY_BUFFER, m_vbo);
    glBufferSubData(GL_ARRAY_BUFFER, 0, 
                    BytesPerDigit * numDigits, &vbo[0]);
    glBindTexture(GL_TEXTURE_2D, m_textureHandle);
    glVertexPointer(2, GL_FLOAT, BytesPerVert, 0);
    glTexCoordPointer(2, GL_FLOAT, BytesPerVert, 
                      (GLvoid*) TexCoordOffset);
    glOrthof(0, m_windowSize.x, 0, m_windowSize.y, 0, 1);
    glColor4f(1, 1, 1, 1);
    glDrawArrays(GL_TRIANGLES, 0, numDigits * VertsPerDigit);

Next we need to implement the private WriteGlyphVertex method, which generates the VBO data for a given corner of a glyph rectangle. It takes a pointer-to-float for input, advances it after writing out each value, and then returns it to the caller (see Example 7-6).

Example 7-6. WriteGlyphVertex() method

float* WriteGlyphVertex(const Glyph& glyph, vec2 position, 
                        int corner, float* vertex)
    vec2 texcoord;
    texcoord.x = glyph.Position.X;
    texcoord.y = glyph.Position.Y + glyph.Metrics.Height;
    position.y -= glyph.Metrics.Height + glyph.Metrics.YBearing;
    if (corner % 2) {
        position.x += glyph.Metrics.Width;
        texcoord.x += glyph.Metrics.Width;
    if (corner / 2) {
        position.y += glyph.Metrics.Height;
        texcoord.y -= glyph.Metrics.Height;
    *vertex++ = position.x;
    *vertex++ = position.y;
    *vertex++ = (1 + texcoord.x) / m_textureSize.x;
    *vertex++ = 1 - (1 + texcoord.y) / m_textureSize.y;
    return vertex;

That’s it for the frame rate counter! It’s pretty easy to use the class from within the rendering engine class; see Example 7-7.

Example 7-7. Using the FpsRenderer class

#include "FpsRenderer.h"

class RenderingEngine : public IRenderingEngine {
    RenderingEngine(IResourceManager* resourceManager);
    void Initialize();
    void Render(float objectTheta, float fboTheta) const;
    FpsRenderer* m_fpsRenderer;

void RenderingEngine::Initialize()
    m_fpsRenderer = new FpsRenderer(m_screenSize);

void RenderingEngine::Render(float objectTheta, float fboTheta) const


With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required