5. Cameras with Arbitrary View Volumes
This renderer modifies the Camera class to give us more control over the
renderer's virtual camera.
In the previous renderers, the camera is fixed at the origin, looking down the negative z-axis and has a fixed view volume (either a fixed projective view volume or a fixed orthographic view volume). This is not very convenient. In order to render a complex scene, we have to position each object relative to the (fixed) camera. It would be better to have the camera act like a real movie camera, where objects are positioned in a scene and then the camera is positioned in whatever location seems best. In fact, we should be able to move the camera around the scene, independently of moving the objects in the scene. In addition, a real movie camera can zoom in and out to view less or more of the objects in the scene.
In this renderer we give the Camera class the ability to set the camera's
view volume. This gives us two abilities that we did not have before. It lets
us set the aspect ratio of the camera's view rectangle. And it lets us "zoom"
the camera in and out of a scene.
In the next renderer we will allow the camera to translate in space and in a later renderer we will allow the camera to rotate in space and point in any direction.
5.1 Renderer source code
The Java source code for this renderer is publicly available as a zip file.
Here is a link to the renderer's source code.
Download and unzip the source code to any convenient location in your computer's file system. The renderer does not have any dependencies other than the Java 11 (or later) JDK. Once you have downloaded and unzipped the distribution, you are ready to compile the renderer and run the renderer's (interactive) example programs.
This renderer has one new file in the pipeline package,
View2Camera.java,
and the previous Model2Camera.java pipeline stage is renamed
Model2View.java.
The files Pipeline.java and Pipeline2.java are modified to put the
new View2Camera stage into the rendering pipeline after the Model2View
stage. In the scene package, the file
Camera.java
is updated to hold the camera's view volume parameters. The client programs are modified to make use of the camera's view volume feature.
5.2 Pipeline
The pipeline has the following seven stages.
v_0 ... v_n A Model's list of Vertex objects
\ /
\ /
|
| model coordinates (of v_0 ... v_n)
|
+-------+
| |
| P1 | Model-to-view transformation (of the vertices)
| |
+-------+
|
| view coordinates (of v_0 ... v_n) relative to an arbitrary view volume
|
+-------+
| |
| P2 | View-to-camera (normalization) transformation (of the vertices)
| |
+-------+
|
| camera coordinates (of v_0 ... v_n) relative to the standard view volume
|
/ \
/ \
/ \
| P3 | Near Plane Clipping (of each primitive)
\ /
\ /
\ /
|
| camera coordinates (of the near-clipped v_0 ... v_n)
|
+-------+
| |
| P4 | Projection transformation (of the vertices)
| |
+-------+
|
| image plane coordinates (of v_0 ... v_n)
|
/ \
/ \
/ \
| P5 | Clipping (of each primitive)
\ /
\ /
\ /
|
| image plane coordinates (of the clipped vertices)
|
+-------+
| |
| P6 | Viewport transformation (of the clipped vertices)
| |
+-------+
|
| pixel-plane coordinates (of the clipped vertices)
|
/ \
/ \
/ \
| P7 | Rasterization & anti-aliasing (of each clipped primitive)
\ /
\ /
\ /
|
| shaded pixels (for each clipped, anti-aliased primitive)
|
\|/
FrameBuffer.ViewPort
Exercise: Rewrite Pipeline.java so that the normalization pipeline
stage, View2Camera, comes after the Projection pipeline stage.
5.3 View Volumes
In this renderer the view volume for a Camera object is determined by four
parameters,
left,right,bottom,top.
The left and right parameters are coordinates on the x-axis. The top
and bottom parameters are coordinates on the y-axis.
Each of the two kinds of projection has its own view volume shape. For
the perspective projection, the camera's view volume is an infinitely long
frustum that is formed from the pyramid with its apex at the origin and its
base in the plane z = -1 with edges x = left, x = right, y = bottom,
and y = top.
For the orthographic projection, the camera's view volume is an infinitely
long parallelepiped that is formed from the rectangle in the z = 0 plane
with edges x = left, x = right, y = bottom, and y = top.
Here are some online demo programs that let you play with the shape of a camera's view volume.
- https://webgl2fundamentals.org/webgl/frustum-diagram.html
- https://webgl2fundamentals.org/webgl/webgl-visualize-camera-with-frustum.html
- https://webgl2fundamentals.org/webgl/webgl-visualize-camera-with-orthographic.html
- https://cs.wellesley.edu/~cs307/threejs/demos-s21-r95/Camera/camera-api.shtml
- https://cs.wellesley.edu/~cs307/threejs/demos-s21-r95/Camera/frustum.shtml
The Camera class has static factory methods for constructing Camera
objects of the appropriate type and shape.
public static Camera projPerspective(final double left,
final double right,
final double bottom,
final double top)
public static Camera projOrtho(final double left,
final double right,
final double bottom,
final double top)
If you look in the code for the Camera class you will see that it has
only one constructor and that constructor is private. A private
constructor can only be called by code within the class file. Any external
code that wants to construct a Camera object cannot call the Camera
constructor. Code external to the Camera class must call one of the
static factory methods in the Camera class. A static factory method
is another object-oriented design pattern. Static factory methods are more
versatile than traditional constructors. They solve many problems that
often come up when using constructors. Many modern Java designs emphasize
static factory methods over constructors.
- https://www.baeldung.com/java-constructors-vs-static-factory-methods
- https://en.wikipedia.org/wiki/Factory_method_pattern
5.4 Normalized View Volumes
Recall that in previous renderers the camera used fixed view volumes, one view volume for perspective projection and another view volume for orthographic projection.
The fixed perspective view volume was the infinitely long pyramid formed from the pyramid with its apex at the origin and its base in the plane z = -1 with edges x = -1, x = +1, y = -1, and y = +1.
The fixed orthographic view volume was the infinitely long parallelepiped formed from the square in the plane z = 0 with edges x = -1, x = +1, y = -1, and y = +1.
These two fixed view volumes are still important. We shall call them the normalized view volumes (or sometimes the "standard view volumes").
5.5 Aspect Ratio
You probably have noticed that many of the screens that you look at have slightly different "shapes". Some are more square (old TVs and old CRT monitors) and some are more rectangular. Some of the more rectangular screens are looked at with their long edge horizontal (movie screens and laptop screens) and some are looked at with their long edge vertical (tablets and smart phones). This shape of a screen is called the screen's "aspect ratio".
The aspect ratio of a rectangle is defined as the ratio of its horizontal length to its vertical length (or, width/height). We define the aspect ratio of a screen to be the ratio of its horizontal length to its vertical length, both measured in pixels.
We will define the aspect ratio of a view volume to be the aspect ratio of its view rectangle.
For both perspective and orthographic projection, the aspect ratio of the view volume's view rectangle is
right - left width
aspect ratio = --------------- = ------
top - bottom height
Notice that a square screen has an aspect ration of 1. A rectangular screen with its long edge horizontal has an aspect ratio > 1. A rectangular screen with its long edge vertical has an aspect ratio < 1.
An aspect ratio > 1 is called landscape mode. An aspect ratio < 1 is called portrait mode.
aspect ratio > 1 aspect ratio < 1
(landscape) (portrait)
+----------------------+ +------------+
| | | |
| | | |
| | | |
| | | |
+----------------------+ | |
| |
| |
| |
| |
+------------+
Some devices, like tablets and smart phones, can easily switch between portrait and landscape modes. Other devices, like a movie screen, can't.
5.6 View Coordinate System and Camera Coordinate System
In previous renderers, the clipping stage of the pipeline assumed that the vertices of every Model were in a coordinate system where the camera had the standard view volume. But in this renderer, we are assuming that the vertices of every Model are in a coordinate system where the camera has the view volume determined by the data in the Scene's Camera object. So the vertex data in our pipeline is not in the coordinate system that our clipping stage expects.
We could rewrite the clipping stage to use the new view volumes, but we would rather not do that (the clipping stage would need too many parameters, which would just slow it down). So we use a trick. We define a new pipeline stage, View2Camera.java, that transforms the coordinates of every Model's vertices from the coordinate system of the camera's view volume to the coordinate system of the standard view volume. Another way to put this is that the new stage transforms the camera's given view volume into the standard (or "normalized") view volume.
After this transformation, the previous clipping stage works just as it did in the previous renderers. (NOTE: We could put the new transformation stage either before or after the projection stage. We put it before the projection stage because that is where it needs to be in future renderers.)
We call the coordinate system relative to the camera's view volume the "view coordinate system". We call the coordinate system relative to the standard view volume the "camera coordinate system". So the new pipeline stage converts vertices from view to camera coordinates.
Here is a brief description of the formulas that transform an arbitrary perspective view volume into the standard perspective view volume (and transform view coordinates into camera coordinates). We use two steps to transform the camera's arbitrary perspective view volume into the standard perspective view volume. The first step "skews" (or "shears") the arbitrary view volume so that its center line is the negative z-axis (that is, we skew an asymmetric view volume into a symmetric one). The second step scales the skewed view volume so that it intersects the plane z = -1 with corners (-1, -1, -1) and (+1, +1, -1) (that is, we scale the symmetric view volume so that it has a 90 degree field-of-view).
Let R, L, t, b denote the right, left, top, and bottom fields
of the Camera. These numbers do not have to define a symmetric view volume.
We are not assuming that R = -L, or t = -b.
Let v = (x_v, y_v, z_v) be a vertex in view coordinates. Skew the perspective
view volume in each of the x and y directions so that the negative z-axis is the
center of the transformed view volume.
x' = x_v - z_v * (R + L)/(2 * near)
y' = y_v - z_v * (t + b)/(2 * near)
z' = z_v
Next, scale the skewed view volume in each of the x and y directions so that it intersects the image plane z = -1 with corners (-1, -1, -1) and (+1, +1, -1).
x_c = (2 * near)/(R - L) * x'
y_c = (2 * near)/(t - b) * y'
z_c = z'
Here is a derivation of the above formulas for the x-coordinate (the derivation for the y-coordinate is similar). We need to skew the xz-plane of the arbitrary view volume in the x-direction so that the center line of the arbitrary view volume skews to the negative z-axis. For some skew factor s, the skew equations are
x' = x_v + s * z_v
z' = z_v
The point (x_v, z_v) = ( (R + L)/2, -near) from the arbitrary view
volume's center line should skew to the point (0, -near) on the
negative z-axis. So
0 = (R + L)/2 + s * -near
so
s = (R + L)/(2*near).
Using this skew factor, we can compute that the point (x_v, z_v) = (R, -near) from the arbitrary view volume skews to the point (x', z') = ((R - L)/2, -near) and the point (L, -near) skews to the symmetric point (-(R - L)/2, -near).
After skewing the arbitrary view volume, we need to scale the xz-plane of the new symmetric view volume in the x-direction so that it has a 90 degree field-of-view. For some scale factor s, the scaling equations are
x_c = s * x'
z_c = z'.
We know that the point (x', z') = ((R - L)/2, -near) should scale to the point (x_c, z_c) = (near, -near) so
near = s * (R - L)/2
so
s = 2*near/(R - L).
We have a similar set of equations for transforming an arbitrary orthographic view volume into the standard orthographic view volume.
We use two steps to transform the camera's arbitrary orthographic view volume into the standard orthographic view volume. The first step translates the arbitrary view volume so that its center line is the z-axis. The second step scales the translated view volume so that it intersects the plane z = 0 with corners (-1, -1, 0) and (+1, +1, 0).
Let v = (x_v, y_v, z_v) be a vertex in view coordinates. Translate the
orthographic view volume in each of the x and y directions so that the
z-axis is the center of the translated view volume.
x' = x_v - (R + L)/2
y' = y_v - (t + b)/2
z' = z_v
Next, scale the translated view volume in each of the x and y directions so that it intersects the plane z = 0 with corners (-1, -1, 0) and (1, 1, 0).
x_c = (2 * x')/(R - L)
y_c = (2 * y')/(t - b)
z_c = z'
5.7 View2Camera vs. Viewport Transformations
The Viewport transformation is, in a way, the opposite of the
View2Camera transformation. The View2Camera transformation takes an
arbitrarily shaped view volume, and its arbitrarily shaped view rectangle,
and transforms them into the standard view volume and view rectangle so
that the clipping stage can do its work in the standard view rectangle.
Then the Viewport transformation transforms the standard view rectangle
into an arbitrarily shaped logical viewport in the pixel-plane. If the
aspect ratio of the camera's view rectangle matches the aspect ratio of
the framebuffer's viewport, then the Viewport transformation will undo
the distortion caused by the View2Camera transformation.
Suppose we have a sphere positioned in camera space and the camera's view
rectangle is not a square. Then the View2camera transformation will distort
the sphere into an ellipsoid shape when the transformation transforms the
camera's non-square view rectangle into the standard, square, view rectangle.
But, if the framebuffer's viewport has the same aspect ratio as the camera's
view rectangle, then the Viewport transformation will distort the ellipsoid
back into a sphere, undoing the distortion caused by the View2Camera
transformation.
If the aspect ratios of the camera's view rectangle and the framebuffer's
viewport are not equal to each other, then the distortions caused by the
View2Camera and Viewport transformations do not "cancel" each other
out. In that case, we are left with a distorted image in the framebuffer's
viewport. The next section is about how we can deal with this situation.
5.8 Mismatched View Volume & Viewport Aspect Ratios
When the aspect ratios of the view volume's view rectangle and the framebuffer's viewport are not equal, one of three possible compromises must be made when rendering the contents of the view volume into the viewport.
1.) The image in the viewport is a distorted representation of the scene in the view volume.
2.) Part of the viewport cannot be used (in film, this is called letterboxing).
3.) Part of the scene in the view rectangle cannot be drawn into the viewport (in film, this is called cropping).
Another way to put this is that when the aspect ratios are not equal:
a.) you can draw all of the contents from the view rectangle using all of the space in the viewport, but the resulting image must be a distorted representation of what is in the view rectangle,
b.) you can draw an undistorted representation of all of the contents of the view rectangle but on only part of the viewport,
c.) you can use all of the viewport to draw an undistorted representation of just part of what's in the view rectangle.
So what can we do when the aspect ratio of the viewport is not the same as the aspect ratio of the view rectangle?
1) Distort the contents of the view rectangle to match the aspect ratio of the viewport.
2) Find a "maximal" sub-rectangle within the viewport that has the same aspect ratio as the view rectangle, and then display the whole view rectangle within that viewport sub-rectangle. How you position the sub-rectangle within the viewport is entirely arbitrary. It is most often either centered in the viewport, or attached to one edge, or one corner, of the viewport.
3) Find a "maximal" sub-rectangle of the view rectangle that has the same aspect ratio as the viewport and then display that sub-rectangle of the view rectangle in the viewport. In other words, crop the contents of the view rectangle so that what is left has the aspect ratio of the viewport. How you position the sub-rectangle within the view rectangle is entirely arbitrary. It is most often chosen so that the most important content of the view rectangle is centered in the sub-rectangle. If the most important content of the view rectangle is moving around, the chosen sub-rectangle may need to move around also. In film, this is called "pan and scan".
Choice 1 is more or less the default choice in most renderers. If the aspect ratios of the view rectangle and the viewport do not agree, then the renderer will do the distorting.
You make choice 2 by resizing (and possibly repositioning) the framebuffer's viewport, or by creating a new viewport within the previous viewport.
You make choice 3 by resizing (and possibly repositioning) the camera's view volume.
5.9 Scale, Crop, Letterbox, Pan
When a "source" rectangle must be displayed within a "destination" rectangle, if the two rectangles do not have the same aspect ratio, then we must use one of these effects.
a) distort, b) crop, c) letter-box, d) pan (must be interactive or an animation).
We implement these effects with a combination of the following actions.
1) scale in the x-direction 2) crop in the x-direction 3) letter-box in the x-direction 4) pan (scroll) in the x-direction 5) scale in the y-direction 6) crop in the y-direction 7) letter-box in the y-direction 8) pan (scroll) in the y-direction
Example 1. Portrait to landscape:
+----------+
| | #===================#
| | # #
| | # #
| source | # destination #
| | # #
| | # #
| | #===================#
+----------+ aspect ratio > 1
aspect ratio < 1 (landscape)
(portrait)
Example 2. Landscape to portrait:
#===========#
# #
+-------------------+ # #
| | # #
| | # #
| source | #destination#
| | # #
| | # #
+-------------------+ # #
aspect ratio > 1 # #
(landscape) #===========#
aspect ratio < 1
(portrait)
Example 3. Both landscape, but still larger aspect ratio to smaller:
+------------------------+ #===============#
| | # #
| | # #
| source | # destination #
| | # #
| | # #
+------------------------+ #===============#
aspect ratio >> 1 aspect ratio > 1
(landscape) (landscape)
Example 4. Both portrait, but still larger aspect ratio to smaller:
#===========#
# #
+-----------+ # #
| | # #
| | # #
| | # #
| source | #destination#
| | # #
| | # #
| | # #
+-----------+ # #
aspect ratio < 1 # #
(portrait) #===========#
aspect ratio << 1
(portrait)
The demo programs demonstrate several ways that a graphics program can react to a user changing the shape of the program's window. Each demo program is displaying a rotating wheel. The rotating wheel image has an aspect ratio of 1 and it starts out with dimensions 512 pixels by 512 pixels. When the user changes the dimension of the program's window (which is the same thing as changing the dimensions of the program's framebuffer) the issue is how the program should display the 512-by-512 wheel image in the new window. The new window might be larger or smaller than the original 512 pixels by 512 pixels. Should the wheel be magnified, or maybe letterboxed, into a larger window? Should the original image be scaled down, or maybe cropped, into a smaller window? If you crop the original image, which part of it is cropped off? If the image is letterboxed, where in the framebuffer is the letterboxing viewport placed (that is, where should the "empty" spaces go)?
To see some answers to the last few questions, look at the source code to the examples Circle_v2_Letterbox.java and Circle_v3_Crop.java. In particular, notice that when we letterbox the viewport within the framebuffer, there are nine locations in the framebuffer where we can place the viewport (think of a tic-tack-toe grid placed on the framebuffer). Similarly, if we use the view-rectangle to crop a source image, the view- rectangle can be situated at nine different locations within the source image (think of the same tic-tac-toe grid but placed this time on the source image). If we are both cropping and letterboxing, then we could even choose different grid locations for the cropping of the view-rectangle and the letterboxing of the viewport (see Circle_v3_Crop_Scale.java).
In general, letterboxing an image into a framebuffer is done by setting the viewport using the
setViewport(int x, int y, int wVP, int hVP)
method of the renderer's FrameBuffer class.
Cropping a source image is done by changing the camera's view rectangle using the
projOrtho(double left, double right, double bottom, double top)
method in the renderer's Camera class.
Scaling an image is done automatically by the renderer when it maps the whole of the camera's view-rectangle onto the framebuffer's viewport. If the aspect ratios of the camera's view-rectangle and the framebuffer's viewport do not agree, then the renderer will distort the final image in the viewport. Most of the time, we want to make sure that the view-rectangle and the viewport have the same aspect ratio so that we get scaling without distortion.
There are two illustrations in this folder that try to visualize the
relationship between cropping, letterboxing, and the parameters to the
projOrtho() and setViewport() methods.