This page runs from a web server, not from a file: in the repository run python3 -m http.server 8000 and open http://localhost:8000/tutorial/.

Part I · Seeing with rays · Chapter 1

Rays make a picture

ShadyTracer is a program that makes pictures of 3D scenes in your web browser. A computer picture is a grid of coloured dots, and to colour one dot ShadyTracer asks one question: if I look out through this dot, what do I see? This chapter is about how it asks.

1The question

You see the ball because light from the lamp landed on it, and some of that light bounced off into your eye. A camera works the same way. A picture is a record of how much light arrived from each direction, one pixel per direction.

So to make a picture, a program needs to know, for every pixel: what is out there in that pixel's direction, and how much light is it sending back?

Each line from the eye in the figure is a ray: a starting point and a direction. It stops at the first thing it meets, and its pixel takes that thing's colour: brighter where the lamp shines on it, dark on the faces turned away from the lamp (section 5 is why). A ray that meets nothing shows the sky, the pale blue.

The strip at the top is the picture the eye would see. It is the screen turned on its side, so its left end is the top of the screen. Drag the eye, the lamp, the ball and the block by their circled crosses (⊕), and watch the picture change.

A flat room seen from the side. Each pixel of the screen sends one ray out into the room; the pixel takes the colour of what the ray hits, or of the sky.

2Follow the light, or follow the eye?

The obvious way to simulate light is to do what light does: send rays out of the lamp in every direction and see which ones end up in the eye. Switch the figure to From the lamp. A ray that meets something bounces off it in a new direction. A matte surface like these scatters light every way, not neatly like a mirror (Diffuse and mirror), so each bounce here goes in a random direction. Of every hundred rays, only one or two pass through the eye's pupil, and the pupil is drawn far bigger than a real one. Everything else was work for nothing. The picture stays empty: a pixel needs many rays to arrive before it has a value, and almost none do.

So ray tracers run it backwards. Start at the eye, send a ray out through each pixel, and see what it hits. Every one of these rays is useful, because every one belongs to a pixel. This works because light paths can be walked in either direction: if light can go from the lamp, off the ball, into the eye, then a ray from the eye can find the ball and then the lamp.

Why is one ray from the eye enough, when the lamp needed so many? The ray from the eye only has to find what the pixel sees. How brightly that point is lit, it then asks the lamp directly, with a second ray aimed at it (section 5). From the lamp's side nothing is aimed: a pixel gets light only when rays happen to arrive.

The ray from the eye is called a camera ray (or primary ray). It is where every picture ShadyTracer makes begins. How a ray finds the first thing in its way is the whole of the next chapter (Shapes as distances); for now, take it that it can.

The same room. From the lamp, almost no light reaches the eye; from the eye, every ray finds something.

3One ray per pixel

Put a screen a little way in front of the eye and divide it into pixels. Each pixel's ray starts at the eye and goes through the middle of its pixel. More pixels means more rays, finer detail and more work: the picture's resolution. Move the pixels slider.

How wide the screen is, compared with how far away it is, sets the field of view: how much of the world fits in the picture. A wide field of view fits more in, and stretches what is near the ends of the picture: the screen is flat, so a pixel near its end sees a narrower slice of angle than one in the middle, and anything there covers more pixels.

All the rays start at one point, the eye. That is a pinhole camera: everything is sharp, near and far. A real lens blurs what is out of focus; ShadyTracer can do that too (The camera's lens).

The mathsThe direction of a pixel's ray

Number the pixels across the screen from 0, and give each one a position x from −1 at one end of the screen to +1 at the other (a place on the screen, not a coordinate of the room; in the figure, −1 is the top). Pixel i of N, at a point u inside it (u from 0 to 1, ½ its middle), sits at

x = 2 · i + uN − 1

so pixel 0's middle is just inside −1 and pixel N−1's just inside +1. The camera has a forward direction (where it looks) and a direction across the screen, each an arrow one unit long. In 3D that is right; in the side view of the figures the screen runs top to bottom, but the sum is the same. The ray's direction is forward plus a bit of the across direction, then made one unit long:

d = normalise( forward + right · x · tan(fov2) )

tan(fov/2): how far the screen reaches either side, if it is one unit in front of the eye: the ray to the screen's end makes the angle fov/2 with forward, and tan is the side across over the side along
normalise: divide an arrow by its length, √(a² + b²) for an arrow (a, b), so it is one unit long and points the same way

An example, the figure's first: N = 12, a field of view of 52°, pixel 0 at its middle. Then x = 2 · 0.5 / 12 − 1 = −0.917 and tan 26° = 0.488, so the direction is forward plus 0.917 × 0.488 = 0.447 of the across direction: 24° from straight ahead, just inside the edge of the 26° half-view.

In 3D there is also an up direction and a second position y for the pixel's row: d = normalise(forward + right · x · f_x + up · y · f_y), where f_x and f_y are the half-widths of the screen. ShadyTracer's field of view is the vertical one, top to bottom; across, the screen is wider by the picture's width over its height. That is the line you will find in ShadyTracer's code below, where u is the random point rnd and N the number of tiles across.

More pixels, more rays. A wider field of view spreads the same pixels over more of the room.

4In 3D: a grid, not a row

The flat world has a row of pixels. A real picture has rows of them, so in 3D the screen becomes a grid, and the eye sends one ray through each cell. This figure is three-dimensional: drag the background to turn it and look at the rays from the side. Each cell of the screen is painted with what its ray found, and laid flat the cells are the picture in the corner.

Switch on Every ray to see all of them at once: a fan spreading out from one point, the eye, through the grid. Drag the circled cross on the screen to follow one pixel's ray, and change pixels across: the picture gets finer, and the number of rays grows with the square of it. Double the pixels across and down, and there are four times as many rays.

Everything else in this chapter is the same in 3D. The figures stay flat wherever a slice shows the idea more clearly, and turn three-dimensional where the third dimension is the point.

The camera in 3D: the eye, a screen of pixels in front of it (wider than it is tall, like a monitor: 8 across, 5 down), one ray per pixel into a small scene, each cell painted with what its ray hit, lit by the lamp or in shadow. Drag to turn it.

5What does the ray see?

When a ray hits something, the pixel needs to know how bright that point is. The simplest answer asks one more question: can this point see the lamp? Here the chosen pixel's ray meets the ball. Send a second ray from that point towards the lamp. If it gets there without hitting anything, the point is lit. This second ray is a shadow ray. The arrow standing straight out of the surface is its normal, and θ is the angle between it and the lamp's direction.

Now a pixel near the bottom of the screen: its ray meets the floor just in front of the block, on the side away from the lamp. The shadow ray from there is stopped by the block, crossed out where it is blocked. The lamp cannot see this point, so it is in shadow. A face turned away from the lamp is in shadow too: the lamp is behind it. Drag this pixel along the screen to try others.

How much light a lit point gets depends on two things. Distance: further from the lamp, less light. And angle: light arriving at a slant spreads over more surface, so it lights each bit of it less. The next figure shows why.

One pixel's ray and the shadow ray from where it lands: towards the lamp, and either reaching it (lit) or stopped by something in the way (in shadow).

6Light at a slant

Here is a beam of light b wide, from a lamp far enough away that its rays are parallel. Straight down onto the floor, it lights a patch b long.

Tip the beam (drag the lamp, or the slider). The same beam, with the same light in it, now lands on a longer patch: b / cos θ. The same light shared over more floor gives each bit of it less: cos θ as much. At 60°, cos θ is ½, so each bit gets half. That is the cosine law, and it is why the parts of the ball that face the lamp are the brightest.

The mathsHow bright a lit point is

A lamp of intensity I (how strongly it shines in the point's direction), at distance r from the point, seen at angle θ from the normal, gives the point this much light, its irradiance E:

E = I · cos θr in the flat world, E = I · cos θr² in 3D

The cos θ is the figure's: the light a beam brings, shared over a patch 1 / cos θ times longer. The r is the same idea at a larger scale. In 3D, light from a lamp spreads over a sphere whose area grows with r²; in a flat world it spreads around a circle whose length grows with r. The figures use the flat world's version, because that is the world they draw.

A surface's colour is three numbers between 0 and 1: how much of the red, the green and the blue light falling on it it sends back. The pixel is each of them times E. Measuring light explains the units ShadyTracer gives these numbers.

Rays, a hit, a shadow ray to the lamp, the cosine law: that is already a working ray tracer, and the oldest kind. Its pictures look hard and dark, because it only counts light that comes straight from the lamp. A real room is lit just as much by light bouncing off its walls. The rest of this tutorial is about counting that light too (Following a path), and chapter 14 shows why the shadow ray is still the most useful trick in the whole renderer (Aim at the light).

A beam meeting the floor at θ from the normal. The thick line is the patch it lights: longer the more the beam slants, so each bit of it gets less.

7Edges, and jitter

Section 1 said a pixel is one direction. That was a simplification: a pixel covers a small wedge of directions (the two faint lines). But a ray through its middle looks in one direction only, and sees either the ball or the sky, never a bit of both. In the figure the ray goes through the middle every time, and just misses: however many times you try, the pixel reads 0, though the ball covers 40% of it. In a picture with rows and columns, a slanted edge then flips whole pixels from ball to sky, a row at a time, and comes out as stair steps: aliasing. Look back at the picture in the corner of the 3D figure in section 4: the edge of its ball is a staircase.

Now turn on Jitter. Each try (each sample) sends its ray through a random point inside the pixel. Some hit the ball and some miss, and the average of all of them settles on the share of the pixel the ball really covers. It wobbles on its way and takes many samples to get close: after 35 it can still be well off, and the figure stops at 400, near. The edge comes out smooth.

ShadyTracer jitters every camera ray, every sample. That is why its picture starts speckled and gets better while you watch: each frame adds more samples to each pixel's average. The speckle is called noise. How fast an average settles, and how ShadyTracer knows how settled it is, is Dice and averages.

Move the edge slider to slide the ball's edge across the pixel, and press Start again.

One pixel at the ball's edge, close up: its last few rays (blue ones hit the ball, grey ones miss it and see the sky), the pixel's colour so far, and how its average approaches the share of the pixel the ball covers.

8How ShadyTracer does it

ShadyTracer's camera rays are made by a kernel called genrays: a small program the graphics card (the GPU) runs once for every ray, thousands at the same time. It is written in WGSL, inside a JavaScript file. Here is the part that makes one ray. It is the flat world's recipe with one more direction (up) and a jitter:

Line by line, the parts that need saying:

The kernel gets a number, not a pixel. Each copy of it is handed its own number 0, 1, 2 … (rayindex, and from it tile). The rays of a sample are laid over the picture as a grid of tiles, tilesX across and tilesY down, and the number says which tile: column tile % tilesX (the remainder after dividing) and row tile / tilesX (rounded down). With one ray a pixel, the tiles are the pixels. Section 10 is what happened when they were not.
UB holds the kernel's uniforms: the values that are the same for every ray, such as the camera's position, the picture's size and the number of tiles. invTX is 1 / tilesX, worked out once beforehand so no ray has to divide. UB.rayFirst is where this batch starts: a sample can be sent to the GPU in several batches, and a ray's tile is its place in the whole sample. max(1u, …) guards against a grid of 0 tiles (1u is the number 1, as a whole number without a sign), and clamp keeps the jittered point inside the picture at its edges.
The random numbers come from PRNG(), ShadyTracer's random number generator. Every sample starts it from a fresh seed (), so every sample jitters differently. rnd is four random numbers; .xy means its first two.
The camera's directions come from invView, a matrix the camera keeps: four columns of numbers, of which the first three are exactly the arrows right, up and backward (.xyz: the first three numbers of each). Backward, because the maths cameras are built with puts a camera's own third axis out of its back, towards you; so the variable fwd, despite its name, points from what the camera looks at to the camera, and the direction it looks is minus fwd. That is why the ray's direction has −1.0 as its third number: one unit along fwd, the other way. fx and fy come from the camera's other matrix, invProjection ([0][0] is its first number): the comment's a is the picture's width over its height.
isPersp says which camera this is. There is a second kind, orthographic, where every ray points the same way and they start side by side instead (). Technical drawings use it: sizes do not shrink with distance.

The comments in the code (// Stable method only) are its authors' notes to each other; the caption above says what each line does. VEC2, u32 and f32 are WGSL's kinds of number: a pair of decimals, a whole number, a decimal. A name like genrays.js and a number after it, in brackets below, is a file and a line in it: follow it to see that line of ShadyTracer's code.

9Tricks and trade-offs

Each of these is stamped with how it is known: read in code means the code does it and the page links the line. Point at a stamp for what it means; other chapters have stamps for what was measured and for what is only a guess.

One ray per pixel per sample, then average. Instead of tracing a perfect pixel once, ShadyTracer traces a rough one many times and keeps the average. The picture is usable after one sample and better after each one, and the camera can move at any time: the average just starts again.

The kernel that makes camera rays does not know the scene. Many of ShadyTracer's kernels are written anew for each scene, with its shapes in them. This one never reads the shapes, so its program text is the same for every scene. Before the GPU can run a kernel it has to compile it, translate it into the card's own instructions, which can take seconds; the compiled copy is kept and reused when you switch scenes (). Faster compiles is about this idea.

The lens costs nothing when it is off. With a lens radius of 0 the kernel makes exactly the pinhole's ray. The lens's two random numbers are the other half of the four that one PRNG() call already gives (rnd.zw, beside the jitter's rnd.xy), so switching it on or off changes no other random choice ().

10What went wrong before

Before any ray is made, ShadyTracer decides how to lay a sample's rays over the picture: as a grid of tiles, tilesX across and tilesY down, one ray in each. Each ray goes through a random point of its tile, and counts for whichever pixel that point lands in:

A sample has N rays, and for a picture w pixels wide and h high there is one ray a pixel, so N = w × h. The grid should have the picture's shape: tilesX × tilesY = N, and tilesX / tilesY = w / h. Multiply the two and tilesY cancels: tilesX² = N × w / h, so tilesX = √(N × w / h), which with N = w × h is √(w × w) = w: one tile per column of pixels. tilesY is then N / tilesX, rounded up so that every ray has a tile.

The code rounded tilesX down. A computer keeps a number like w / h with a limited number of digits, so it is stored a tiny bit off, and the product can land a hair under the exact answer. For a window 809 pixels wide and 395 high, it comes out as 654,480.9999999999 instead of 809² = 654,481; its square root is 808.9999999999999, and rounded down that is 808 tiles across for 809 pixels. (Type Math.sqrt(809 * 395 * (809 / 395)) into a browser's console to see it.) Then tilesY became 396 rows for 395 pixels.

Why that gives a pixel two rays, or none: take a row 4 pixels wide covered by 3 tiles, each 1⅓ pixels wide. Tile 0 covers pixel 0 and a third of pixel 1; tile 1 the rest of pixel 1 and two thirds of pixel 2; tile 2 the rest. Each tile's one ray lands at a random point inside it, so in one sample pixel 1 may catch the rays of tiles 0 and 1, two, and in the next neither. With 808 tiles for 809 pixels the drift is smaller, but it builds up across the row, and the same happens down the 396 rows. It happened for about one window size in forty:

The picture looked fine: over many samples the jitter still covered every pixel. What was wrong was the noise. A pixel that gets two rays a sample settles faster than one that gets one, so how speckled a picture was depended on the window's size, which nobody comparing the noise of two pictures would think to keep the same. Rounding instead of rounding down gives exactly one ray per pixel at every size. The comment says what happened, because the next person to touch the line needs to know why it is written that way.

11Try it

In the first figure, set pixels to 3, then to 40. Which details of the room appear first?
Drag the eye close to the ball and set the field of view to 130°. Why does the ball take up most of the picture, and why do things near the ends of the strip look stretched?
Switch to From the lamp and press Throw 400 more a few times. Roughly what share of the lamp's rays reaches the eye? What would happen with a smaller pupil?
In the slant figure, at what angle does each bit of floor get a quarter of the light it gets straight on?
In the edge figure, slide the edge to 50% with jitter off. Why is that the one setting where the middle ray cannot decide?
In ShadyTracer itself: open any example and watch the first second. The picture starts speckled and settles. Each frame is more jittered samples added to every pixel.