7. Matrix Transformations
In the previous renderers we used a Vector to position a Model in
a Scene. This limits how we can move a model. For example, we cannot
easily do rotations.
This renderer introduces Matrix transformations. We modify the Position
class to refer to a Matrix object and we modify the Model2World
transformation stage to use matrix multiplication.
This gives us a better way to position models in a scene.
7.1 Renderer source code
The Java source code for this renderer is publicly available as a zip file.
Here is a link to the renderer's source code.
Download and unzip the source code to any convenient location in your computer's file system. The renderer does not have any dependencies other than the Java 11 (or later) JDK. Once you have downloaded and unzipped the distribution, you are ready to compile the renderer and run the renderer's (interactive) example programs.
This renderer adds three new files to the scene package.
Matrix.javaOrthographicNormalizeMatrix.javaPerspectiveNormalizeMatrix.java
In the scene package, both
Vertex.javaVector.java
are modified to hold homogeneous coordinates.
In the pipeline package, both
Model2World.javaView2Camera.java
are modified to use a Matrix transformation.
The client programs are modified to make use of this renderer's rotation feature.
Interesting examples of what this renderer can do are the "robot arm"
programs in renderer_7/clients_r7/robot_arms_R7 and the "solar system"
programs in renderer_7/clients_r7/solar_system_R7. These programs are long
and complex, even though they seem to do something simple. In the next
renderer we will se how to make these examples shorter and more elegant.
7.2 Pipeline
The pipeline has the same eight stages as the previous renderer.
v_0 ... v_n A Model's list of (homogeneous) Vertex objects
\ /
\ /
|
| model coordinates (of v_0 ... v_n)
|
+-------+
| |
| P1 | Model-to-world (matrix) transformation (of the vertices)
| |
+-------+
|
| world coordinates (of v_0 ... v_n)
|
+-------+
| |
| P2 | World-to-view (vector) transformation (of the vertices)
| |
+-------+
|
| view coordinates (of v_0 ... v_n) relative to an arbitrary view volume
|
+-------+
| |
| P3 | View-to-camera (normalization matrix) transformation (of the vertices)
| |
+-------+
|
| camera coordinates (of v_0 ... v_n) relative to the standard view volume
|
/ \
/ \
/ \
| P4 | Near Clipping (of each line segment)
\ /
\ /
\ /
|
| camera coordinates (of the near-clipped v_0 ... v_n)
|
+-------+
| |
| P5 | Projection transformation (of the vertices)
| |
+-------+
|
| image plane coordinates (of v_0 ... v_n)
|
/ \
/ \
/ \
| P6 | Clipping (of each line segment)
\ /
\ /
\ /
|
| image plane coordinates (of the clipped vertices)
|
+-------+
| |
| P7 | Viewport transformation (of the clipped vertices)
| |
+-------+
|
| pixel-plane coordinates (of the clipped vertices)
|
/ \
/ \
/ \
| P8 | Rasterization & anti-aliasing (of each clipped line segment)
\ /
\ /
\ /
|
| shaded pixels (for each clipped, anti-aliased line segment)
|
\|/
FrameBuffer.ViewPort
7.3 Matrix Class
This renderer defines a Matrix class and modifies the Position class in
the scene package. A Position object now holds a reference to a Matrix
object (instead of a Vector object). The Matrix object determines the
position (and orientation) of the Model object in the world coordinate
system.
The Matrix in a Position is used by the Model2World stage to compute the
model's location in world coordinates. For each Position in the Scene, the
Model2World stage transforms every Vertex object in the Position object's
Model by multiplying the Vertex by the position's Matrix.
// Model2World pipeline stage
for (Position position : scene.positionList)
{
// A new vertex list to hold the transformed vertices.
final List<Vertex> newVertexList = new ArrayList<>();
for (Vertex v : position.model.vertexList)
{
newVertexList.add( modelMatrix.times(v) );
}
}
The matrix multiplication changes the coordinates in the vertex from the model's own model coordinate system to the world coordinate system. The world coordinate system is shared by all the models in the scene. So the position's matrix places the position's model at a location in the scene.
Here is what a Scene graph looks like now. Each Position object holds
a reference to a Matrix object instead of a Vector object.
Scene
/ \
/ \
Camera List<Position>
/ / | \
/ / | \
Vector / | \
Position Position Position
/ \ / \ / \
/ \ / \ / \
Matrix Model Matrix Model Matrix Model
/ | \
/---/ | \---\
/ | \
/ | \
List<Vertex> List<Color> List<Primitive>
/ | \ / | \ / | \
| | |
Vertex Color LineSegment
/ | \ / | \ / \
x y z r g b / \
List<Integer> List<Integer>
(vertices) (colors)
/ \ / \
/ \ / \
Integer Integer Integer Integer
To read about matrices and matrix transformations, download and read Chapters 1 and 2 from the following reference.
- Geometry for Computer Graphics.pdf
- https://www.diag.uniroma1.it/~degiacom/didattica/informatica_grafica/geom_sn.pdf
The following sections assume that you are familiar with the ideas from this reading assignment.
7.3 Model-to-World Transformation
Here is how we think about "model coordinates", "world coordinates" and the
model-to-world transformation. Take a Model and pair it with a Matrix M
in a Position object. Let v0 = (x0, y0, z0) be a Vertex in the Model.
The coordinates (x0, y0, z0) are the coordinates of the vertex in the model's
own model coordinate system. When we multiply v0 by the Position's matrix
M, we get a new vertex v1 = (x1, y1, z1).
v1 = M * v0
We use the new coordinates (x1, y1, z1) as the coordinates of a vertex in
the world coordinate system. We think of the position's matrix M as "placing"
the vertex v0 at the location v1 in world space. Why is this useful? If the
model has millions of vertices (which is not uncommon) and we want to change
where the model is placed in world space, we can do that by changing just
the Position's matrix M instead of having to move the model by changing
every one of its vertices. The actual moving of the millions of vertices will
still need to be done, but it will be done for us automatically by the
renderer's first pipeline stage. We only need to change the position's matrix.
It's important to realize that while every model has its own model coordinate system, there is only one world coordinate system. While every model has it own "origin", the vertex (0, 0, 0), there is a single origin in world space. Every model picks an origin for its model coordinates that is convenient for that particular model. For example, for a model of a cube, the origin of model coordinates might be the center of the cube, or it might be a corner of the cube; both work well. For a model of a sphere, you would almost always make the origin of the model space the center of the sphere, but you do not have to (the south pole of the sphere might be a convenient origin for some applications). For many models there is no obvious best place to put the origin of model space (think of a cow), so an arbitrary choice gets made.
7.4 Matrix Concatenation
There are many reasons for using a model matrix to place and move models
around in world space. Pairing every Model object in a Position object
with its own Matrix object makes many aspects of graphics programming
work better. For example, consider "composing transformations".
Suppose we want to implement the following transformation of every vertex in some model.
v = m2 * (m1 * v)
This applies matrix m2 to the result of applying matrix m1 to every
vertex v in the model (for example, m1 might be a rotation and m2 a
translation). This is what is meant by concatenating (or "composing")
two transformations. We can implement this transformation of a model with
the following loop.
for (Vertex v : model.vertexList)
{
v = m2 * (m1 * v); // two matrix-times-vector operations
}
But we can use the associative law for matrix multiplication.
m2 * (m1 * v) = (m2 * m1) * v // associative law
So the loop can be rewritten this way.
for (Vertex v : model.vertexList)
{
v = (m2 * m1) * v; // one matrix-times-matrix operation & one matrix-times-vector operation
}
Notice that m2 * m1 is a constant as far as the loop is concerned. Suppose
we let m = m2 * m1. That is, we pre-multiply 'm2' and 'm1' before entering
the loop.
Matrix m = m2 * m1; // a single matrix-times-matrix operation
for (Vertex v : model.vertexList)
{
v = m * v; // a single matrix-times-vector operation
}
Notice that this loop does half the work of the previous loops. We pay a (very) small price to pre-multiply the two matrices before the loop, and then the loop itself is twice as fast. This speedup is magnified if, as is common, there are more than two matrix transformations to apply to the vertices of a model. Take this loop,
for (Vertex v : model.vertexList)
{
Vertex v2 = m1 * v;
Vertex v3 = m2 * v2;
Vertex v4 = m3 * v3;
Vertex v5 = m4 * v4;
v = m5 * v5; // five matrix-times-vector operations
}
which can also be written this way,
for (Vertex v : model.vertexList)
{
v = m5 * (m4 * (m3 * (m2 * (m1 * v)))); // five matrix-times-vector operations
}
or this way,
for (Vertex v : model.vertexList)
{
v = (m5 * m4 * m3 * m2 * m1) * v; // four matrix-times-matrix and one matrix-times-vector
}
and compare them with the following loop.
Matrix m = m5 * m4 * m3 * m2 * m1;
for (Vertex v : model.vertexList)
{
v = m * v; // a single matrix-times-vector operation
}
Remember that translation of a vertex can only be implemented as a matrix multiplication when we use 4-dimensional homogeneous coordinates in the vertex and 4 x 4 matrices (otherwise we have to implement translation of a vertex using vector addition). Here is why it is a good idea to implement translation using matrix multiplication instead of vector addition, even though matrix multiplication of a vertex needs many more operations than vector addition (16 multiplications and 12 additions vs. just three additions!).
Suppose we have five transformations, T1, T2, T3, T4, T5, where
T1, T3 and T5 are rotations and T2 and T4 are translations. Suppose
we want to apply them as T5( T4( T3( T2( T1(v) ) ) ) ) for every vertex v
in some model. If we implement the translations as vector addition and the
rotations as matrix multiplications, then this would lead to code like the
following, where vec2 and vec4 are the 3-dimensional vectors that
represent T2 and T4, and m1, m3, m5 are the 3x3 matrices that
represent T1, T3, T5.
for (Vertex v : model.vertexList)
{
Vertex v2 = m1 * v; // 3x3 matrix multiplication
Vertex v3 = vec2 + v2; // vector addition
Vertex v4 = m3 * v3; // 3x3 matrix multiplication
Vertex v5 = vec4 + v4; // vector addition
v = m5 * v5; // 3x3 matrix multiplication
}
On the other hand, if we represent all of the transformations as 4x4 matrices, then the code can be written in this far more efficient way.
Matrix m = m5 * m4 * m3 * m2 * m1;
for (Vertex v : model.vertexList)
{
v = m * v; // one 4x4 matrix multiplication
}
The key ideas here are that,
(i) all the possible transformations (translation, rotation, scaling, even projection) can be written as 4x4 homogeneous matrices, (ii) the associative law of matrix multiplication lets us pre-multiply successive transformations into a single matrix, (iii) matrix multiplication can be implemented in hardware very efficiently.
These three ideas combine together to make matrix multiplication the heart of all graphics rendering pipelines.
7.5 Object-Oriented Matrix Operations
In our Java code, we cannot write the product of matrices like this,
Matrix m6 = m5 * m4 * m3 * m2 * m1;
since Java does not let us overload the * operator (like C++ does).
Similarly, we cannot write the product of a matrix and a vertex like this.
Vertex v2 = m * v1;
And we cannot write
v1 = m * v1;
to denote the updating (mutating) of the coordinates in the vertex v1
since Java does not let us overload the = operator (like C++ does).
In object-oriented Java, we should implement matrix multiplication as a
method in a Matrix class. We can replace this matrix-times-matrix notation,
m1 * m2
with this method call,
m1.times(m2) // m1 * m2
or maybe this method call.
m1.mult(m2) // m1 * m2
We can in fact use both a times() method and a mult() method in a
Matrix class. The mult() method can be a method that mutates its calling
object so that it holds the resulting product matrix. The times() method
can return a new Matrix object that holds the resulting product matrix.
Both a mult() and a times() method have advantages and disadvantages.
Some matrix libraries have one, some have the other, and some have both.
The Matrix class in the scene package from this renderer only has the
(non-mutating) times() method. We will explain why below.
In a Matrix class, we can implement the matrix-times-vertex operation
m * v
with a method call,
m.times(v)
where the (overloaded) times(Vertex) method returns a new Vertex object
that holds the resulting vertex.
Notice that a mutating matrix-times-vertex method,
m.mult(v)
does not make sense. The calling object, matrix m, is not supposed to be
mutated. Mutating the parameter object, vertex v, is very bad style. So
how can we implement a mutating matrix-times-vertex operation? We want an
operation that implements this idea.
v = m * v
Think of this as similar to Java's "times-equals" operator.
x *= 5; // x = 5 * x
So we want something like this,
v *= m; // v = m * v
which we can implement in the Vertex class with a (mutating) method like the
following.
v.timesEquals(m);
We can also implement
v = m * v;
as
v = m.times(v);
but this does not mutate v. It replaces the Vertex object that v refers
to with a new Vertex object (and the previous Vertex object might be
garbage collected).
Let's assume for a while that our Matrix class has both the mult() and
times() methods. Let us see how we can use each of them.
Here is an informal, pseudo code, loop that transforms a model.
Matrix m = m5 * m4 * m3 * m2 * m1;
for (Vertex v : model.vertexList)
{
v = m * v; // one matrix-times-vertex operation
}
We can implement this pseudo code the following way. This mutates the vertices in the model.
Matrix m = m5.times(m4).times(m3).times(m2).times(m1);
for (Vertex v : model.vertexList)
{
v.timesEquals(m); // v = m * v
}
This code is an abbreviation of the following code.
Matrix m = m5; // m = m5
m = m.times(m4); // m = m * m4
m = m.times(m3); // m = m * m3
m = m.times(m2); // m = m * m2
m = m.times(m1); // m = m * m1
for (Vertex v : model.vertexList)
{
v.timesEquals(m); // v = m * v
}
We can also write the code this way, which uses mutation of Matrix objects.
Matrix m = Matrix.identity();
m.mult(m5); // m = I * m5
m.mult(m4); // m = m * m4
m.mult(m3); // m = m * m3
m.mult(m2); // m = m * m2
m.mult(m1); // m = m * m1
for (Vertex v : model.vertexList)
{
v.timesEquals(m); // v = m * v
}
We can also write our code this way, which uses chaining of the mult() method.
Matrix m = Matrix.identity().mult(m5).mult(m4).mult(m3).mult(m2).mult(m1);
for (Vertex v : model.vertexList)
{
v.timesEquals(m); // v = m * v
}
Exercise: Explain why the following loop does not transform the model. (This is an important exercise. Make sure that you can explain what this loop is doing.)
for (Vertex v : model.vertexList)
{
v = m.times(v); // v = m * v ?
}
The following code transforms a model without mutating the model's vertices.
In order for the code to have an effect, we must mutate something (there must
be something that "remembers", or "records", what we do). We mutate an empty
List<Vertex>.
// A new vertex list to hold the transformed vertices.
final List<Vertex> newVertexList = new ArrayList<>();
for (Vertex v : model.vertexList)
{
newVertexList.add( m.times(v) );
}
We now have a vertex list holding transformed versions of the vertices from
the original model, but we do not yet have a transformed model. We use the
new vertex list, along with other data from the original model, to create
a new, transformed, Model object.
Model m2 = new Model(newVertexList,
m.colorList,
m.primitiveList,
m.name,
m.visible);
Notice that a line of code like this
m = m.times(m2);
or like this
m.mult(m2);
is implemented as matrix multiplication of m2 on the right of the matrix m.
m = m * m5
Whenever we multiply a position's model matrix by some other matrix, we do so by multiplying the other matrix on the right of the model matrix. So each line in the following sequence of code
position.getMatrix().mult(Matrix.translate(a, b, c)); // m = m * T(a,b,c)
position.getMatrix().mult(Matrix.scale(s, t, u)); // m = m * S(s,t,u)
position.getMatrix().mult(Matrix.rotateZ(r)); // m = m * Rz(r)
multiplies the appropriate transformation matrix on right of the position's model matrix. The result is
m = m * T(a,b,c) * S(s,t,u) * Rz(r)
A code sequence like the three matrix multiplications above can be written
several ways using times() and 'mult()`. Here are four other ways to
write it.
position.transform( position.getMatrix().times(Matrix.translate(a, b, c)) );
position.transform( position.getMatrix().times(Matrix.scale(s, t, u)) );
position.transform( position.getMatrix().times(Matrix.rotateZ(r)) );
position.transform( position.getMatrix().times(Matrix.translate(a, b, c)
.times(Matrix.scale(s, t, u)
.times(Matrix.rotateZ(r)))) );
position.transform( position.getMatrix().times(Matrix.translate(a, b, c))
.times(Matrix.scale(s, t, u))
.times(Matrix.rotateZ(r)) );
position.getMatrix().mult(Matrix.translate(a, b, c))
.mult(Matrix.scale(s, t, u))
.mult(Matrix.rotateZ(r));
As we mentioned above, the Matrix class in this renderer's scene package
only has the times() method. We do nut use the mutating version of matrix
(or vertex) multiplication because mutation often leads to buggy code. Code
that mutates is often faster than non-mutating code, but the speed increase
comes at a very high cost in code complexity. Since this renderer is not
meant to compete with real renderers (GPUs), it does not need to be as fast
as possible. So we try to use modern programming techniques that avoid
mutation as much as possible. In modern functional programming languages, it
is possible to avoid mutation entirely. In modern versions of Java, we can
minimize the use of mutation, but some forms of mutation are hard to avoid
(like inserting or removing items from a List, which is mutation of the
list).
7.6 Instancing
Besides making it easier to move a model in a scene, a position's model matrix also helps us solve the "instancing problem" where we want to have multiple instances of a model in a scene. Say we want three identical chairs to be in a scene. If the chair is represented by a Chair class (that sub classes the Model class), then we can easily get three chairs in the scene by instantiating Chair three times, instantiating Position three times (one Position object for each Chair object), and giving each Position object a different model matrix.
Scene
/ \
/ \
Camera List<Position>
/ | \
/ | \
/ | \
Position Position Position
/ \ / \ / \
/ \ / \ / \
Matrix Chair Matrix Chair Matrix Chair
But why should we need three instances of the Chair class when the Chair objects are identical? Why not let the three Position objects each refer to the same Chair object?
Scene
/ \
/ \
Camera List<Position>
/ | \
/ | \
/ | \
Position Position Position
/ \ / | / /
/ \ / | / /
Matrix \ Matrix | Matrix /
\ | /
\-----\ | /-----/
\ | /
Chair
Since each Position object represents a different position in camera space,
the single Chair object will appear in three different places in the final
rendered scene. So we need only one instance of Chair in order to have three
identical chairs in the scene. Also, notice how what was a tree data structure
became a graph data structure (a directed acyclic graph (DAG)).
7.7 View2Camera Transformation Revisited
In Section 5.5 (from the renderer_5 readme) we derived the formulas for
the View2Camera pipeline stage.
We can now use a Matrix to implement the View2Camera stage instead of
using a sequence of linear equations.