How FreeType's rasterizer work | |

by David Turner | |

Revised 2007-Feb-01 | |

This file is an attempt to explain the internals of the FreeType | |

rasterizer. The rasterizer is of quite general purpose and could | |

easily be integrated into other programs. | |

I. Introduction | |

II. Rendering Technology | |

1. Requirements | |

2. Profiles and Spans | |

a. Sweeping the Shape | |

b. Decomposing Outlines into Profiles | |

c. The Render Pool | |

d. Computing Profiles Extents | |

e. Computing Profiles Coordinates | |

f. Sweeping and Sorting the Spans | |

I. Introduction | |

=============== | |

A rasterizer is a library in charge of converting a vectorial | |

representation of a shape into a bitmap. The FreeType rasterizer | |

has been originally developed to render the glyphs found in | |

TrueType files, made up of segments and second-order Béziers. | |

Meanwhile it has been extended to render third-order Bézier curves | |

also. This document is an explanation of its design and | |

implementation. | |

While these explanations start from the basics, a knowledge of | |

common rasterization techniques is assumed. | |

II. Rendering Technology | |

======================== | |

1. Requirements | |

--------------- | |

We assume that all scaling, rotating, hinting, etc., has been | |

already done. The glyph is thus described by a list of points in | |

the device space. | |

- All point coordinates are in the 26.6 fixed float format. The | |

used orientation is: | |

^ y | |

| reference orientation | |

| | |

*----> x | |

0 | |

`26.6' means that 26 bits are used for the integer part of a | |

value and 6 bits are used for the fractional part. | |

Consequently, the `distance' between two neighbouring pixels is | |

64 `units' (1 unit = 1/64th of a pixel). | |

Note that, for the rasterizer, pixel centers are located at | |

integer coordinates. The TrueType bytecode interpreter, | |

however, assumes that the lower left edge of a pixel (which is | |

taken to be a square with a length of 1 unit) has integer | |

coordinates. | |

^ y ^ y | |

| | | |

| (1,1) | (0.5,0.5) | |

+-----------+ +-----+-----+ | |

| | | | | | |

| | | | | | |

| | | o-----+-----> x | |

| | | (0,0) | | |

| | | | | |

o-----------+-----> x +-----------+ | |

(0,0) (-0.5,-0.5) | |

TrueType bytecode interpreter FreeType rasterizer | |

A pixel line in the target bitmap is called a `scanline'. | |

- A glyph is usually made of several contours, also called | |

`outlines'. A contour is simply a closed curve that delimits an | |

outer or inner region of the glyph. It is described by a series | |

of successive points of the points table. | |

Each point of the glyph has an associated flag that indicates | |

whether it is `on' or `off' the curve. Two successive `on' | |

points indicate a line segment joining the two points. | |

One `off' point amidst two `on' points indicates a second-degree | |

(conic) Bézier parametric arc, defined by these three points | |

(the `off' point being the control point, and the `on' ones the | |

start and end points). Similarly, a third-degree (cubic) Bézier | |

curve is described by four points (two `off' control points | |

between two `on' points). | |

Finally, for second-order curves only, two successive `off' | |

points forces the rasterizer to create, during rendering, an | |

`on' point amidst them, at their exact middle. This greatly | |

facilitates the definition of successive Bézier arcs. | |

The parametric form of a second-order Bézier curve is: | |

P(t) = (1-t)^2*P1 + 2*t*(1-t)*P2 + t^2*P3 | |

(P1 and P3 are the end points, P2 the control point.) | |

The parametric form of a third-order Bézier curve is: | |

P(t) = (1-t)^3*P1 + 3*t*(1-t)^2*P2 + 3*t^2*(1-t)*P3 + t^3*P4 | |

(P1 and P4 are the end points, P2 and P3 the control points.) | |

For both formulae, t is a real number in the range [0..1]. | |

Note that the rasterizer does not use these formulae directly. | |

They exhibit, however, one very useful property of Bézier arcs: | |

Each point of the curve is a weighted average of the control | |

points. | |

As all weights are positive and always sum up to 1, whatever the | |

value of t, each arc point lies within the triangle (polygon) | |

defined by the arc's three (four) control points. | |

In the following, only second-order curves are discussed since | |

rasterization of third-order curves is completely identical. | |

Here some samples for second-order curves. | |

* # on curve | |

* off curve | |

__---__ | |

#-__ _-- -_ | |

--__ _- - | |

--__ # \ | |

--__ # | |

-# | |

Two `on' points | |

Two `on' points and one `off' point | |

between them | |

* | |

# __ Two `on' points with two `off' | |

\ - - points between them. The point | |

\ / \ marked `0' is the middle of the | |

- 0 \ `off' points, and is a `virtual | |

-_ _- # on' point where the curve passes. | |

-- It does not appear in the point | |

* list. | |

2. Profiles and Spans | |

--------------------- | |

The following is a basic explanation of the _kind_ of computations | |

made by the rasterizer to build a bitmap from a vector | |

representation. Note that the actual implementation is slightly | |

different, due to performance tuning and other factors. | |

However, the following ideas remain in the same category, and are | |

more convenient to understand. | |

a. Sweeping the Shape | |

The best way to fill a shape is to decompose it into a number of | |

simple horizontal segments, then turn them on in the target | |

bitmap. These segments are called `spans'. | |

__---__ | |

_-- -_ | |

_- - | |

- \ | |

/ \ | |

/ \ | |

| \ | |

__---__ Example: filling a shape | |

_----------_ with spans. | |

_-------------- | |

----------------\ | |

/-----------------\ This is typically done from the top | |

/ \ to the bottom of the shape, in a | |

| | \ movement called a `sweep'. | |

V | |

__---__ | |

_----------_ | |

_-------------- | |

----------------\ | |

/-----------------\ | |

/-------------------\ | |

|---------------------\ | |

In order to draw a span, the rasterizer must compute its | |

coordinates, which are simply the x coordinates of the shape's | |

contours, taken on the y scanlines. | |

/---/ |---| Note that there are usually | |

/---/ |---| several spans per scanline. | |

| /---/ |---| | |

| /---/_______|---| When rendering this shape to the | |

V /----------------| current scanline y, we must | |

/-----------------| compute the x values of the | |

a /----| |---| points a, b, c, and d. | |

- - - * * - - - - * * - - y - | |

/ / b c| |d | |

/---/ |---| | |

/---/ |---| And then turn on the spans a-b | |

/---/ |---| and c-d. | |

/---/_______|---| | |

/----------------| | |

/-----------------| | |

a /----| |---| | |

- - - ####### - - - - ##### - - y - | |

/ / b c| |d | |

b. Decomposing Outlines into Profiles | |

For each scanline during the sweep, we need the following | |

information: | |

o The number of spans on the current scanline, given by the | |

number of shape points intersecting the scanline (these are | |

the points a, b, c, and d in the above example). | |

o The x coordinates of these points. | |

x coordinates are computed before the sweep, in a phase called | |

`decomposition' which converts the glyph into *profiles*. | |

Put it simply, a `profile' is a contour's portion that can only | |

be either ascending or descending, i.e., it is monotonic in the | |

vertical direction (we also say y-monotonic). There is no such | |

thing as a horizontal profile, as we shall see. | |

Here are a few examples: | |

this square | |

1 2 | |

---->---- is made of two | |

| | | | | |

| | profiles | | | |

^ v ^ + v | |

| | | | | |

| | | | | |

----<---- | |

up down | |

this triangle | |

P2 1 2 | |

|\ is made of two | \ | |

^ | \ \ | \ | |

| | \ \ profiles | \ | | |

| | \ v ^ | \ | | |

| \ | | + \ v | |

| \ | | \ | |

P1 ---___ \ ---___ \ | |

---_\ ---_ \ | |

<--__ P3 up down | |

A more general contour can be made of more than two profiles: | |

__ ^ | |

/ | / ___ / | | |

/ | / | / | / | | |

| | / / => | v / / | |

| | | | | | ^ | | |

^ | |___| | | ^ + | + | + v | |

| | | v | | | |

| | | up | | |

|___________| | down | | |

<-- up down | |

Successive profiles are always joined by horizontal segments | |

that are not part of the profiles themselves. | |

For the rasterizer, a profile is simply an *array* that | |

associates one horizontal *pixel* coordinate to each bitmap | |

*scanline* crossed by the contour's section containing the | |

profile. Note that profiles are *oriented* up or down along the | |

glyph's original flow orientation. | |

In other graphics libraries, profiles are also called `edges' or | |

`edgelists'. | |

c. The Render Pool | |

FreeType has been designed to be able to run well on _very_ | |

light systems, including embedded systems with very few memory. | |

A render pool will be allocated once; the rasterizer uses this | |

pool for all its needs by managing this memory directly in it. | |

The algorithms that are used for profile computation make it | |

possible to use the pool as a simple growing heap. This means | |

that this memory management is actually quite easy and faster | |

than any kind of malloc()/free() combination. | |

Moreover, we'll see later that the rasterizer is able, when | |

dealing with profiles too large and numerous to lie all at once | |

in the render pool, to immediately decompose recursively the | |

rendering process into independent sub-tasks, each taking less | |

memory to be performed (see `sub-banding' below). | |

The render pool doesn't need to be large. A 4KByte pool is | |

enough for nearly all renditions, though nearly 100% slower than | |

a more comfortable 16KByte or 32KByte pool (that was tested with | |

complex glyphs at sizes over 500 pixels). | |

d. Computing Profiles Extents | |

Remember that a profile is an array, associating a _scanline_ to | |

the x pixel coordinate of its intersection with a contour. | |

Though it's not exactly how the FreeType rasterizer works, it is | |

convenient to think that we need a profile's height before | |

allocating it in the pool and computing its coordinates. | |

The profile's height is the number of scanlines crossed by the | |

y-monotonic section of a contour. We thus need to compute these | |

sections from the vectorial description. In order to do that, | |

we are obliged to compute all (local and global) y extrema of | |

the glyph (minima and maxima). | |

P2 For instance, this triangle has only | |

two y-extrema, which are simply | |

|\ | |

| \ P2.y as a vertical maximum | |

| \ P3.y as a vertical minimum | |

| \ | |

| \ P1.y is not a vertical extremum (though | |

| \ it is a horizontal minimum, which we | |

P1 ---___ \ don't need). | |

---_\ | |

P3 | |

Note that the extrema are expressed in pixel units, not in | |

scanlines. The triangle's height is certainly (P3.y-P2.y+1) | |

pixel units, but its profiles' heights are computed in | |

scanlines. The exact conversion is simple: | |

- min scanline = FLOOR ( min y ) | |

- max scanline = CEILING( max y ) | |

A problem arises with Bézier Arcs. While a segment is always | |

necessarily y-monotonic (i.e., flat, ascending, or descending), | |

which makes extrema computations easy, the ascent of an arc can | |

vary between its control points. | |

P2 | |

* | |

# on curve | |

* off curve | |

__-x--_ | |

_-- -_ | |

P1 _- - A non y-monotonic Bézier arc. | |

# \ | |

- The arc goes from P1 to P3. | |

\ | |

\ P3 | |

# | |

We first need to be able to easily detect non-monotonic arcs, | |

according to their control points. I will state here, without | |

proof, that the monotony condition can be expressed as: | |

P1.y <= P2.y <= P3.y for an ever-ascending arc | |

P1.y >= P2.y >= P3.y for an ever-descending arc | |

with the special case of | |

P1.y = P2.y = P3.y where the arc is said to be `flat'. | |

As you can see, these conditions can be very easily tested. | |

They are, however, extremely important, as any arc that does not | |

satisfy them necessarily contains an extremum. | |

Note also that a monotonic arc can contain an extremum too, | |

which is then one of its `on' points: | |

P1 P2 | |

#---__ * P1P2P3 is ever-descending, but P1 | |

-_ is an y-extremum. | |

- | |

---_ \ | |

-> \ | |

\ P3 | |

# | |

Let's go back to our previous example: | |

P2 | |

* | |

# on curve | |

* off curve | |

__-x--_ | |

_-- -_ | |

P1 _- - A non-y-monotonic Bézier arc. | |

# \ | |

- Here we have | |

\ P2.y >= P1.y && | |

\ P3 P2.y >= P3.y (!) | |

# | |

We need to compute the vertical maximum of this arc to be able | |

to compute a profile's height (the point marked by an `x'). The | |

arc's equation indicates that a direct computation is possible, | |

but we rely on a different technique, which use will become | |

apparent soon. | |

Bézier arcs have the special property of being very easily | |

decomposed into two sub-arcs, which are themselves Bézier arcs. | |

Moreover, it is easy to prove that there is at most one vertical | |

extremum on each Bézier arc (for second-degree curves; similar | |

conditions can be found for third-order arcs). | |

For instance, the following arc P1P2P3 can be decomposed into | |

two sub-arcs Q1Q2Q3 and R1R2R3: | |

P2 | |

* | |

# on curve | |

* off curve | |

original Bézier arc P1P2P3. | |

__---__ | |

_-- --_ | |

_- -_ | |

- - | |

/ \ | |

/ \ | |

# # | |

P1 P3 | |

P2 | |

* | |

Q3 Decomposed into two subarcs | |

Q2 R2 Q1Q2Q3 and R1R2R3 | |

* __-#-__ * | |

_-- --_ | |

_- R1 -_ Q1 = P1 R3 = P3 | |

- - Q2 = (P1+P2)/2 R2 = (P2+P3)/2 | |

/ \ | |

/ \ Q3 = R1 = (Q2+R2)/2 | |

# # | |

Q1 R3 Note that Q2, R2, and Q3=R1 | |

are on a single line which is | |

tangent to the curve. | |

We have then decomposed a non-y-monotonic Bézier curve into two | |

smaller sub-arcs. Note that in the above drawing, both sub-arcs | |

are monotonic, and that the extremum is then Q3=R1. However, in | |

a more general case, only one sub-arc is guaranteed to be | |

monotonic. Getting back to our former example: | |

Q2 | |

* | |

__-x--_ R1 | |

_-- #_ | |

Q1 _- Q3 - R2 | |

# \ * | |

- | |

\ | |

\ R3 | |

# | |

Here, we see that, though Q1Q2Q3 is still non-monotonic, R1R2R3 | |

is ever descending: We thus know that it doesn't contain the | |

extremum. We can then re-subdivide Q1Q2Q3 into two sub-arcs and | |

go on recursively, stopping when we encounter two monotonic | |

subarcs, or when the subarcs become simply too small. | |

We will finally find the vertical extremum. Note that the | |

iterative process of finding an extremum is called `flattening'. | |

e. Computing Profiles Coordinates | |

Once we have the height of each profile, we are able to allocate | |

it in the render pool. The next task is to compute coordinates | |

for each scanline. | |

In the case of segments, the computation is straightforward, | |

using the Euclidean algorithm (also known as Bresenham). | |

However, for Bézier arcs, the job is a little more complicated. | |

We assume that all Béziers that are part of a profile are the | |

result of flattening the curve, which means that they are all | |

y-monotonic (ascending or descending, and never flat). We now | |

have to compute the intersections of arcs with the profile's | |

scanlines. One way is to use a similar scheme to flattening | |

called `stepping'. | |

Consider this arc, going from P1 to | |

--------------------- P3. Suppose that we need to | |

compute its intersections with the | |

drawn scanlines. As already | |

--------------------- mentioned this can be done | |

directly, but the involved | |

* P2 _---# P3 algorithm is far too slow. | |

------------- _-- -- | |

_- | |

_/ Instead, it is still possible to | |

---------/----------- use the decomposition property in | |

/ the same recursive way, i.e., | |

| subdivide the arc into subarcs | |

------|-------------- until these get too small to cross | |

| more than one scanline! | |

| | |

-----|--------------- This is very easily done using a | |

| rasterizer-managed stack of | |

| subarcs. | |

# P1 | |

f. Sweeping and Sorting the Spans | |

Once all our profiles have been computed, we begin the sweep to | |

build (and fill) the spans. | |

As both the TrueType and Type 1 specifications use the winding | |

fill rule (but with opposite directions), we place, on each | |

scanline, the present profiles in two separate lists. | |

One list, called the `left' one, only contains ascending | |

profiles, while the other `right' list contains the descending | |

profiles. | |

As each glyph is made of closed curves, a simple geometric | |

property ensures that the two lists contain the same number of | |

elements. | |

Creating spans is thus straightforward: | |

1. We sort each list in increasing horizontal order. | |

2. We pair each value of the left list with its corresponding | |

value in the right list. | |

/ / | | For example, we have here | |

/ / | | four profiles. Two of | |

>/ / | | | them are ascending (1 & | |

1// / ^ | | | 2 3), while the two others | |

// // 3| | | v are descending (2 & 4). | |

/ //4 | | | On the given scanline, | |

a / /< | | the left list is (1,3), | |

- - - *-----* - - - - *---* - - y - and the right one is | |

/ / b c| |d (4,2) (sorted). | |

There are then two spans, joining | |

1 to 4 (i.e. a-b) and 3 to 2 | |

(i.e. c-d)! | |

Sorting doesn't necessarily take much time, as in 99 cases out | |

of 100, the lists' order is kept from one scanline to the next. | |

We can thus implement it with two simple singly-linked lists, | |

sorted by a classic bubble-sort, which takes a minimum amount of | |

time when the lists are already sorted. | |

A previous version of the rasterizer used more elaborate | |

structures, like arrays to perform `faster' sorting. It turned | |

out that this old scheme is not faster than the one described | |

above. | |

Once the spans have been `created', we can simply draw them in | |

the target bitmap. | |

------------------------------------------------------------------------ | |

Copyright (C) 2003-2020 by | |

David Turner, Robert Wilhelm, and Werner Lemberg. | |

This file is part of the FreeType project, and may only be used, | |

modified, and distributed under the terms of the FreeType project | |

license, LICENSE.TXT. By continuing to use, modify, or distribute this | |

file you indicate that you have read the license and understand and | |

accept it fully. | |

--- end of raster.txt --- | |

Local Variables: | |

coding: utf-8 | |

End: |