Part VIII · Getting it right · Chapter 29
Noise, bias and checking
A picture made of random samples is wrong in two ways at once. Its noise fades as samples add up, and every pixel carries an error bar that says how much is left (chapter 12). Its bias does not fade, and no error bar sees it. This chapter is how to tell the two apart, how ShadyTracer checks itself against answers known in advance, why every such check needs a row that must fail, and how the error bar itself is checked. The checks are in the program; which of them have been run on a real card, and which have not, is said for each.
1Two ways to be wrong
One pixel, looking at a floor in a corner lit by a small lamp near the ceiling, in the flat world these pages draw in (chapter 0: the renderer's rules in a slice, two dimensions instead of three). Each sample follows one path from the eye (chapter 13), bouncing off the matte walls, and never aims a ray at the lamp on purpose (no NEE, chapter 14), so the lamp is found only by chance: most paths bring back nothing, a few bring back a lot. The lamp's brightness is 10 and each bounce keeps 0.7 of the light (the walls' albedo), so a path that reaches the lamp straight off the floor brings 0.7 × 10 = 7, one that bounces once more first 0.7 × 0.7 × 10 = 4.9. The pixel's reading is the average of its samples, and the dashed lines either side of it are two error bars away. Its true value, 1.644, is the long-run average over a million paths (computed). The reading wanders early, then closes in, and the dashed lines hold the true value between them nearly all the way: an unbiased estimate, whose only error is noise. (One error bar either side holds the true value about two times in three, chapter 12; two, about 95 times in 100. The samples are counted on a log scale, each mark four times the last, so the first few samples and the last thousands both fit.)
The same paths, each cut after its first bounce, as a renderer with a bounce cap of 1 would trace them: the light that reaches the floor straight from the lamp, and none that comes off the walls first. The reading settles just as neatly, its error bars close in just as fast, and it settles on 0.94, about 19 of its own error bars under the true value (its own long-run value is 0.926, computed). It is precise and wrong. Nothing in the picture says so: the error bar measures how much the samples disagree with each other, and these all agree on the wrong answer.
A firefly clamp is the other common bias: every sample brighter than a limit is cut to it, here 4, to take the bright speckles out of an early picture. Every path that finds the lamp in one or two bounces brings 7 or 4.9 and is cut, so the reading settles at 1.20, quieter and darker, 17 error bars under (1.183 in the long run). ShadyTracer has one (off by default: "a choice rather than a guard", its comment says), and it clamps only light that has bounced, never what the camera sees directly.
The live view's moving window is not a bias. Averaged over a window of 10 (chapter 23: each new sample weighs 1/11, and old ones fade), the reading follows the true value on average, but it never settles: it is worth about 21 samples however long it runs (section 3), so its error bar stops shrinking.
Between unbiased and biased there is a third kind, consistent: wrong on average after any given number of samples, but by less the more there are, and right in the limit. The photon map of chapter 19 is one: its radius shrinks with every sample, and so does its blur.
measure.js keeps them; the other three readings faint;
the true value solid. The paths' legs that end on the lamp are drawn as the chosen reading counts them. Choose
the reading; Other paths draws new ones.2A check that cannot fail
A check needs an answer known in advance. ShadyTracer's white furnace (chapter 8) is one: a ball of a material that reflects all the light it gets, in a sky that is equally bright every way. No light can be made or lost on the way to the eye, so the whole picture must read the sky's brightness, and the ratio of a row of such a material (the picture's average over the sky's) must be 1. Here it is in the flat world, one disc filling three quarters of the view, each pixel the average of 32 samples, read the way ShadyTracer's test bench reads its pictures: the ratio, and the noise, how much neighbouring pixels differ, over the average. The white rows pass: within 2% of 1, about five times this little picture's own spread (0.4% from one run to the next, computed over 200; ShadyTracer's pictures, a million pixels each, are held far closer). A grey disc reads less, and it must: the quarter of the picture that is sky reads 1 and the disc 0.8, so 0.25 + 0.75 × 0.8 = 0.85. It shows the test can see a loss. The last row is a control: a black disc, which must read well below 1, and does, 0.25, the sky's quarter alone. Its noise is not zero: the edges between black disc and white sky are real detail, which the noise number counts too (section 5).
Now the disc is never drawn: a scene that did not load. Every pixel sees the sky, and every white row reads 1.0000 and passes. They would pass whatever the material did, because an empty picture reads exactly what a perfect one does. The grey row fails, and the control reads 1.0000 too: a row that must fail, passing, is the sign that nothing above it was measured. This happened. On 2026-09-29 a page that mixed old cached files with new ones read 1000.000244, against a sky of brightness 1000 (the last digits are 32-bit rounding: inferred): a ratio of 1.000000244, "with zero noise on every row, rough glass included" (a row that, like the grey here, reads below 1 when it works). The control row's comment tells that story as its reason.
And the camera turned away from the disc: the same readings, from another cause. The first time the test of dust in the air was run (the user, 2026-10-16), its camera was aimed at a point given in the wrong unit, 40 millimetres read as 40 metres, so it looked at empty sky: "every row 1000.000244, noise 0, the CONTROL with them". Noise 0 is a clue but not a proof: here the NEE-off row and the mirror read exactly 1 even when the page works, because every one of their samples is exactly 1. The control is the proof.
raytrace_main.js computes the bench's, and whether it reads what it must; point at
a pixel for its value. Matte rows with NEE on aim a ray at the sky as well as
bouncing, and weigh the two by how likely each was to find that direction (MIS, chapter 15),
so their samples differ a little: noise. Rows read what they must within 2% of 1 (the white), under 0.95 (the
grey) and under 0.5 (the control). Choose how the page is working.3Is the error bar right?
Chapter 12 showed how resolve, the kernel that
adds each sample into the picture, keeps every pixel's error bar from two numbers: V, how much its
samples scatter, and S, the sum of the squares of their weights in the average (the fold-out below
recaps both: in a plain average every sample weighs the same, 1/n, and in a moving one the newer
samples weigh more). The error bar is a claim, and it can be checked:
here a thousand pixels, all of the same true brightness 1, each the average of 32 samples drawn as a path
tracer's are (mostly small, a few large: an exponential distribution, as the engine's own test of the error bar
draws them). Their averages really scatter by 0.175 (their standard deviation, the typical distance of one from the average of
all, chapter 12); the error bars they report say 0.176.
256 samples a pixel: both shrink to about 0.065, 2.7 times smaller, as 1/√n says for eight times the samples (√8 is 2.8). The check the engine can run on a real picture uses only that: on a flat, evenly lit floor, two neighbouring pixels differ only by noise, so the average squared difference between neighbours must equal the sum of their squared error bars. seRatio is the square root of the one over the other, and it must read 1: here 1.08. seRatio is noisy too: from one set of a thousand pixels to the next it spreads by 3% (computed, over 100 sets), so this set is an unlucky one, about 2.6 of those spreads high. Other pixels draws another.
The moving window of 10 (section 1): after 256 samples each new one weighs 1/11, and the average is worth 21 samples, not 10 and not 256. The kernel does not need to be told: S, the sum of squared weights, works its way to 1/21 by itself, and the error bar follows, 0.218 reported against 0.216 real. Why 21 and not 11: the oldest samples still count a little, so the weight is spread over more of them than the window's 10 (the sum of squared weights of a sample weighing w and the rest fading is w/(2 − w), with w = 1/11 that is 1/21: the fold-out works it out).
Now the mistake the engine's comment warns of, "the window's real sample count, which
'N' is not": the error bar worked out as if the window's average were worth its count, 11. It reports 0.308, too
wide, and nothing in a picture would look wrong. seRatio sees it: 0.69. The check exists in ShadyTracer, as a
row of its test bench, error-bar, with a window-10 row for exactly this; it has been built and never
run on a real card, so no seRatio has been recorded anywhere (docs/TESTING.md).
The mathsThe error bar, the window's worth, and why neighbours check it
The error bar from V and S (chapter 12). A pixel's average is a sum of its samples, each times a weight wi, the weights adding up to 1. If one sample scatters by s (its variance is s²), the average scatters by √(Σ wi² · s²) = s √S: a sum of independent things has a variance that is the sum of their variances, and a thing times w has w² times the variance. V estimates s² once divided by (1 − S), so the error bar is √(S · V / (1 − S)). Why the 1 − S: V measures the scatter about the samples' own average, which sits a little closer to them than the true value does, so it comes out a little small; dividing by 1 − S puts that back. For a plain average of n, S = 1/n, the divisor is (n − 1)/n, and the error bar is √(V / (n − 1)): chapter 12's s/√n with the statisticians' n − 1.
Two rules used above. A spread is measured in the number's own units, its variance in their square: multiply a number by w and its spread is w times as big, its variance w² times. And two things that do not depend on each other: the variance of their sum is the sum of their variances (the cross terms average to zero); so is the variance of their difference, since −b scatters exactly as b does.
1/S counts samples. n equal weights of 1/n give S = n × (1/n)² = 1/n, so an average with any weights is as good as 1/S equal samples.
The window's worth. Each new sample weighs w = 1/11, and every older weight is multiplied by (1 − w). So S becomes (1 − w)² S + w², the update resolve uses. Once it stops changing, S = (1 − w)² S + w², so S (1 − (1 − w)²) = w², and since 1 − (1 − w)² = w(2 − w), S = w / (2 − w) = (1/11) / (21/11) = 1/21.
Why neighbours check it. Two neighbouring pixels of an evenly lit floor have the same true value, so their difference La − Lb is noise alone, and the variance of a difference of two independent things is again the sum of theirs: on average (La − Lb)² = sea² + seb². seRatio is √ of the first averaged over all neighbours, over the second: 1 when every error bar is right, below 1 when they are too wide, above when too narrow, or when the picture has real detail.
measure.js stepMoments and standardError, the kernel's update), and the true value dashed; on the
right, how much the averages really scatter against the error bars' typical size, and seRatio over neighbours,
as raytrace_main.js computes it. Choose the samples, the average, and in a window how its error bar is
worked out.4How ShadyTracer does it
The furnace's control, in the test bench:
The noise, and the error bar checked, as the bench reads a picture:
The error bar's arithmetic, on the host, with the window as an argument:
Another check that could pass without checking: the shader check once only made sure the code could be read, and said so on a day a kernel was empty. Why it now builds every kernel, from its own header:
5Tricks and trade-offs
Each is stamped with how it is known; point at a stamp to see what it means.
A known answer for every kind of light. The bench's image tests each compare a picture with
arithmetic done in advance: a white floor under the sun and sky reads ρE/π, its albedo times the light falling on it, over π (chapter 5); a tilted
plane E sin(elevation + tilt); a glowing disc's light on the floor, worked out numerically to match
the formula; the furnace 1 for every white material, a little under for rough glass (lib/pathtracer/tests.js).
A verdict on every row. Each test carries a rule, and each row reads PASS within three of its
own error bars or the allowance its test's note gives, whichever is wider; CHECK out to five, or twice the
allowance, where a rerun comes first; FAIL past that. A row with no answer to hold it to reads INFO, and a test
that measured nothing (the emitter test on a scene with no emitter) VOID, never a pass. The allowances are
choices, not measurements, and the error bar of a picture's mean assumes its pixels' errors independent, which
they are not where pixels share light (lib/pathtracer/verdict.js, docs/TESTING.md).
A trimmed mean for comparing two builds. Two renderers converging to the same picture can
differ by percent in a plain mean at a few samples: a handful of fireflies move it. So the bench also reports the
mean without the brightest 0.1% of pixels; a difference that survives that is real. The NEE test compares plain
means, because the brightest pixels are exactly what NEE changes (raytrace_main.js,
docs/TESTING.md).
Noise compared only like with like. The noise number counts real detail in the picture as
well as noise, so it is compared only between two pictures of the same scene at the same sample count: halving
it there is worth four times the samples (raytrace_main.js).
The furnace, passed, nearly. Its first run: every material but rough glass 1.000 to 0.04%,
the NEE-off rows equal to their partners, nothing above 1 (the user's Pascal card, 1188 × 969, 32 samples,
docs/TESTING.md). Rough glass must read a little under 1, since its model loses the light that
would bounce between its tiny facets (chapter 9); it does (section 6), at values not yet explained between
runs. And later runs found the metal rows 0.02 to 0.09% above 1, light made, each about one
error bar: a bias at the edge of the noise, the largest traced to one table's coarse steps
(roadmap/SURFACES.md).
The kernels' arithmetic held to a copy. Where the shader's arithmetic matters (the error bar,
the photon map's gates, the dust's free flight), a JavaScript copy is tested against known answers, and sixteen
*.gpu.mjs tests run the shader's own text on a software graphics card in node to hold the two
together. Without that software they say SKIPPED, loudly, rather than pass; and they say what a kernel
computes, never how fast (docs/TESTING.md).
Built is not run. Several of the bench's rows have been built and never run on a card:
error-bar, render-modes, draft-lamps, the firefly clamp's rows
(docs/TESTING.md). Until they are, what they check is unknown, and section 3's figure is a model
of what the error bar should do, not a measurement of what it does. What would settle it: one press of
error-bar.
6What went wrong before
- Every row passing, the control included. Section 2's two stories: old cached files, and a camera target in the wrong unit ().
- The mean wrong, the median right. The first run of the floor's test with the real sun, 0.53°
across (chapter 5), read 142 to 352 times the right brightness on average, and about right (1.01 to 1.04) in its median pixel, the
one with half the pixels brighter and half darker. A light sample
(a direction aimed at the sun, chapter 14) and the chance of drawing it were worked out by two different tests of whether a direction is inside the sun's disc,
which disagreed in a thin band at its rim, so a rare sample there counted about 100,000 times too much: a firefly
made by a bug. A firefly clamp would have hidden much of it (inferred: it cuts exactly such samples)
(
docs/TESTING.md). - Rough glass losing 8%. The furnace's rough glass read 0.9231: the code chose whether to reflect by
Fresnel's law (chapter 10) for the surface as a whole, which past the critical angle, where light can no longer
leave glass, says reflect always; but a rough surface is tilted facets (chapter 9), some of which would let the
light out, and that was never tried. Choosing by the facet first took it to 0.9587, what its model predicts
(
docs/TESTING.md). - A bias that was not there. Russian roulette (chapter 13: ending some paths at random, and
weighting the survivors up) seemed to brighten pictures by 6 to
8%; a test of it found every arm within 0.15%, and the brightening was another setting changing between the two
runs compared (
docs/TESTING.md). - "24 modules parse". The shader check said so on the day a kernel shipped with an empty text, and failed to compile for everyone; section 4's excerpt is why it now builds every kernel against every scene.
7Try it
- In the first figure, press Other paths ten times on the whole path, and count how often the true value is outside the dashed lines at the end. About once in twenty?
- Do the same with cut after 1 bounce. Is the true value ever inside?
- In the error-bar figure, choose the moving window and slide the samples from 2 to 512. Where does the real spread stop shrinking, and why there?