Part VI · The machine · Chapter 21
A GPU for beginners
Everything so far could run on any processor, one ray after another. ShadyTracer runs on the graphics card, which does not work one ray after another: it runs one small program on thousands of rays at once. That changes what is fast and what is slow, what can go wrong, and how a program is handed its work. This chapter is the minimum needed to read the next three: what the card does with a kernel, how its memory is laid out, and how work is given to it, each with the trouble it caused ShadyTracer.
1Thousands of copies of one program
A kernel is one function. The card runs a copy of it for every item of work, each copy a thread told which item is its own, and here every pixel's ray is one. The threads are not independent: the card runs them in groups that take every instruction together, in lock-step, 32 at a time on NVIDIA's cards and 32 or 64 on AMD's. Each thread is one lane of its group. When the rays of a group need different amounts of work, the group takes as long as its slowest ray, and the lanes that finished early wait, idle. Here 128 rays march through a flat scene (chapter 2: each step moves a ray on by the distance to the nearest surface, so a ray passing close to an edge crawls). Each bar is one ray's march steps, and each box the time its group of 32 takes, as tall as its slowest ray. A lane-step is one lane held for one step: the boxes hold 4,000 of them, and the rays' 1,791 steps fill 45%. Point at a group to see its rays above.
Neighbouring pixels see neighbouring things and need similar work, so a group of neighbours wastes less. Shuffle the pixels and each group gathers rays from all over the picture, a slow one in nearly every group: 39%. ShadyTracer numbers its camera rays along the picture's rows (thread k makes pixel k's ray, at one ray a pixel), so a group of its threads is a run of neighbouring pixels.
The narrower the group, the less one slow ray holds up: in groups of 8 the lanes are busy 70% of the time. The card decides the width, not the program: the figure lets you choose it only to show what it does. What a renderer can do is keep the rays in a group alike, and stop a long loop before its slowest ray holds everyone up, which is what chapter 22 is about.
2Memory, laid out
A kernel reads and writes buffers, blocks of the card's memory.
Two kinds matter here. A storage buffer is a large array the threads read and write: the rays, the picture. A
uniform buffer holds the values every thread reads alike, the camera and the settings, read
by a path the card keeps for exactly that; one may be at most 64 KiB (65,536 bytes). Its values are a struct, a
list of named fields, each a type: f32 a decimal number, u32 a whole one
from 0 up, i32 a whole one that may be negative, vec2, vec3 and
vec4 two, three and four of one of those (vec3<f32> three f32s,
vec3<u32> three u32s), and a matrix a small table of f32s. WGSL lays them out by a rule:
each field starts at an offset (its first byte, counted from the struct's start) that is a multiple of its
alignment, and the gap left before it is padding, never used.
| type | bytes | starts at a multiple of |
|---|---|---|
| f32, u32, i32 (one number) | 4 | 4 |
| vec2 | 8 | 8 |
| vec3 | 12 | 16 |
| vec4 | 16 | 16 |
| the whole struct | rounded up to a multiple of 16 | |
So a vec3 takes 12 bytes but starts on a 16-byte boundary, and a single number after it fits in its row's last
4 bytes. This struct, in this order, takes 64 bytes, 8 of them padding: 4 before col, which must
start at 48, and 4 at the end.
The same seven fields in another order, 80 bytes, 24 of them padding: t first
leaves 12 bytes empty before pos, which must start at 16; uv after col
leaves 4, since it must start at a multiple of 8; and its 8 bytes leave the last row half empty. Nothing is wrong, only wasted, and the kernel and the
code that fills the buffer must agree on every offset. ShadyTracer does not write its uniforms' offsets by hand:
tinywebgpu, its small WebGPU library, works them out from the field types (the second excerpt), and the tutorial's
own test holds this figure's arithmetic to it. (A ray's size it does write by hand, 112 bytes, and a test holds that to the WGSL
compiler's own.)
ShadyTracer's own uniform block, UB, before a scene adds its fields: 109 fields in 768 bytes, 36 of them padding. Its rows are drawn in eight columns to fit: read down the first column, then the next. Point at a box for its field. The scene's own values go on top of these, and the scene's records, when the scene is sent as data, have a uniform buffer of their own: 4096 vec4s, exactly the 64 KiB every WebGPU device allows (chapter 24).
3Giving it work
The program on the main processor asks the card to run a kernel by a dispatch: how many workgroups to run, each of a size fixed in the kernel (64 threads in ShadyTracer's ray kernels, 64 along x and 1 along y), counted along up to three dimensions. The graphics card runs a workgroup of 64 as two of its lock-step groups of 32 on NVIDIA's cards: the workgroup is the program's choice, the lock-step group the card's. Each thread works out its item from its group's place and its own: the thread at place 5 of workgroup 3 handles item 3 × 64 + 5 = 197. The simplest way over a picture is one long count: pixel 0, 1, 2 … along the rows, 64 to a group. At 1920 × 1080 that is 32,400 groups.
Every dimension of a dispatch may count at most 65,535 groups. 2048 × 2048 pixels is 65,536 groups of 64: one over. That is not a slow frame; it is a validation error, and nothing is drawn. The kernel that adds each sample into the picture (chapter 23's resolve) was dispatched this way, so it worked up to 65,535 × 64 = 4,194,240 pixels and no further: not at 2048 × 2048, one workgroup over, nor at a 4K picture (3840 × 2160, 8.3 million pixels), nor at a render sized to a 4K monitor.
One row per y instead: along x, the groups across one row (60 for a 4K row); along y, the rows (2160). A thread's pixel is then x = its group's x × 64 + its place, y = its group's y. Both far under the limit, at any picture a card can hold; the bars are drawn too short to see beside it, so each carries its number.
Dispatches are recorded and sent to the card together, a submission. If one runs too long, about 2 seconds on Windows, the system decides the card has hung and resets it, and the page loses its device (its handle on the card, which every buffer and kernel belongs to) and everything on it. ShadyTracer once sent a whole sample as one submission, and the Noise library, an example scene of shapes with noisy surfaces, hung (2026-09-29). Its sample had been timed at over a second, under the limit; that week a change to the march may have made it slower, which is ShadyTracer's own guess, not a measurement. A heavy frame is now cut into submissions of about 150 ms (the fourth excerpt). Since version 0.81.0 each submission is timed as well, and the longest one of the last frame decides how fine the next is cut: a 3840 × 2160 render lost its device "after a few batches" (2026-10-31), the cause inferred to be one submission holding the costly start of a band of the picture where the frame's average said there was room.
4How ShadyTracer does it
A kernel in tinywebgpu is given as two pieces of WGSL: body, the helpers and
structs, and main, the entry point's statements. The library writes the entry point around them, with
the thread's three places, each a vec3<u32> (an x, a y and a z): gid its place
among all threads, lid within its workgroup, wid its workgroup's:
Its packing rule, section 2's table as code, which the figure mirrors (the sizes in 4-byte units). The regular
expressions test the type's name: vec3<f32> matches the vec3 case and gets [3, 4, 3, 0],
3 units of size (12 bytes), aligned to 4 units (16 bytes), 3 numbers; a matrix is its columns, each a vector:
The dispatch that ran out of x:
No submission long enough to be killed:
5Tricks and trade-offs
Each is stamped with how it is known; point at a stamp to see what it means.
Compiling is slow. The browser (through WebGPU)
hands each kernel's text to the graphics driver, whose compiler, running on the main processor, turns it into
the card's own instructions; ShadyTracer's kernels are large. The user has two machines: a newer one with an
NVIDIA Ampere card, and an older GTX 1080 Ti (a Pascal card). Built after an edit on the Ampere machine, the
kernels were ready at:
shade 18,546 ms, occlusion 3,365, march 1,765, the camera rays 507 (docs/PERFORMANCE.md, 2026-09-29).
The benchmark's compile of every kernel at once read 21.2 and 22.7 s on the user's Ampere card and 7.9 to 8.1 s
on the GTX 1080 Ti, for the same scene (the same document). Why the newer machine read slower is not known:
the same document has the same kernels compiling in about 4 s there when timed one by one, and leaves the two
numbers unexplained. Chapter 27 is how that was
shortened.
Eight storage buffers a kernel. WebGPU promises every device at least 8 storage buffers to one kernel; many cards allow more, but a kernel that needs a ninth will not run on one that does not. The shading kernel uses exactly 8, so what it needs from the kernel before it (the surface kernel, chapter 22) is written beside each ray's hit, in the list of hits it reads (the shade queue), instead of in a buffer of its own ().
Rays counted as memory. How many rays a sample may hold in flight is set as the memory they take, 1 GiB (2³⁰ bytes, about 1.07 billion), not as a count: a ray is 112 bytes in each of two ping-pong buffers (one read while the other is written, chapter 22), plus a 288-byte item in the shade queue, 512 bytes, so about 2.10 million rays, enough for a 1080p picture (1920 × 1080, 2.07 million pixels) at one ray a pixel. Even so the shade queue alone is about 600 MB, over the 256 MB a device need allow one buffer, so the program still asks the card for its largest buffers when it starts. A sample with more rays and photons than that is traced in batches, chapter 23 ().
Compiled while the page goes on. Making a pipeline (a compiled kernel ready to dispatch) the plain way holds the browser's main thread, the one that also draws the page (not one of the GPU's threads), until the compile ends; tinywebgpu asks for it asynchronously, so the page goes on while it compiles, and the renderer refuses to trace until every kernel of a build is ready (the fourth story below). How long a compile takes is the measured stamp above.
6What went wrong before
- A barrier not every thread reached. Threads of a workgroup can wait for each other at a
barrier: none goes on until all have arrived. So every thread of the group must reach it, or the rest would wait
for one that left, and WGSL refuses a kernel where that can happen (its uniformity rule). A guard,
if (gid.x == N) { return; }, put before one let some threads leave first. The checker ShadyTracer runs without a GPU passed it, and the browser's compiler refused it on the user's machine (). - "24 modules parse." ShadyTracer's checker, a script run before every change that reads each JavaScript file (a module) without running it, said so on the day a kernel shipped with an empty string for its code, which failed to compile for everyone, and nothing noticed: it ran only with one setting on, and its compile failed long after the call that caused it (). Now every kernel is built against every scene by a stand-in device that records instead of compiling.
- A buffer bound and then dropped. tinywebgpu binds what a kernel's text names (hands the card a
buffer for each name); the compiled pipeline keeps only the names its entry point,
main, reaches through the functions it calls. One kernel's text named the uniform buffer in a helper it never called, so the binding and the pipeline disagreed, and every frame with light tracing on was rejected (). - A dispatch while compiling. A redraw asked for during the first build dispatched a kernel still compiling: "No resources bound", the frame's commands rejected ().
- Four million rays at every size. Before the 1 GiB, ShadyTracer allocated 4 million rays in three ray buffers and the hit buffer, about 1.7 GB with the ray's and hit's sizes of the time (both have grown since), whether the picture was 4K or 256 pixels. It worked only because the program asks the card for its largest buffers when it starts; on a device that gives the specification's defaults (256 MB a buffer), every one of those allocations failed ().
- Out of x. Section 3, as it happened.
7Try it
- In the first figure, which group is slowest, and what are its rays doing? Try each width with both orders.
- In the second, could the small struct take less than 64 bytes in any order? Count its fields' bytes.
- In the third, find the largest square picture a flat dispatch could cover. (Its pixels must be at most 65,535 × 64.)
- Open ShadyTracer with the browser's developer tools (F12) and look in the console for the line that says when its kernels compiled, and how long each took.