This page runs from a web server, not from a file: in the repository run python3 -m http.server 8000 and open http://localhost:8000/tutorial/.

Part II · What light is · Chapter 7

From light to pixels

ShadyTracer's picture is light, in candela per square metre, from a dark corner's 0.6 to the sun's 1.5 billion. A screen shows numbers from 0 to 1, in 256 steps a channel. Between the two is what a camera does: choose how bright to show the scene (its exposure), decide what happens to light brighter than white (a tone curve), and store the result so that 256 steps are enough for the eye (sRGB).

1How bright to show it

The exposure is a single number: the luminance that will show as white. Everything is divided by it, and 1 is white. ShadyTracer gives it as a camera does, by an EV (exposure value): each step of EV, called a stop, halves the light the camera lets in, so white is 1.2 × 2^EV cd/m², and a higher EV is a darker picture. A sunny scene is about EV 15, an office about 7.

A word first, since light runs over so many doublings: log₂ x is how many times 1 must be doubled to make x. log₂ 8 = 3, log₂ 1 = 0, log₂ ¼ = −2. On a log₂ scale every doubling is one equal step, which is how both a camera and the eye treat light.

Which EV? A camera's light meter answers by averaging the scene. ShadyTracer does it on the graphics card every frame: it sorts the pixels' luminances into a histogram, 128 bins over 2⁻¹⁰ to 2³⁴ cd/m², a third of a stop each; it drops the darkest 10% and the brightest 10% (deep shadow, the sun, a few very bright noisy pixels), and averages the rest on the log scale: the average of their logarithms, so that 1 and 100 average to 10, not 50.5 (every ×10 counts the same; this is the geometric mean). The figure is a sunny scene, with chapter 5's floor in it: the middle 80% average 7,460 cd/m², which a meter calls EV 15.87, and white is 71,600. The sunlit floor shows at 0.3 of white; the sun, at the far right of the scale near 2^30.5, is clipped whatever happens. (The swatches show the scene's parts through PBR Neutral, the default tone curve of section 2.)

A lamp-lit room is harder. Most of it is dim walls and a dark corner, and the lamp lights a small desk. The average, 3.6 cd/m², would set EV 4.86, and the desk, at 600, would be 17 times white: blown out. So the meter has a second rule, about the brightness where the brightest 5% of the pixels begin: it may be at most 2 stops (4 times) above white (the headroom, a setting). Here the brightest 5% begin at 541 cd/m², so white must be at least 541 / 4: EV 6.82, white at 135, and the meter takes the darker of the two, the higher EV. The desk's brightest parts are still over white, 4.4 times (section 2 is what happens to light over white), but no longer 17 times; the corner goes darker. (Why 10% for the cuts and 5% here: they are the meter's settings, read in its code, not laws; the 5% is the part of the picture the rule protects.)

With a headroom of 12 stops the rule never binds, and the average alone decides: EV 4.86, the desk white.

Black is not part of the scene. A pixel that sees nothing (behind every object, a sky that is switched off) is below the histogram's floor, and does not vote. It used to count, in the darkest bin. Here 60% of the picture is black and voting, as it used to: past the 10% cut, black is in the average, which falls to 0.019 cd/m², and only the highlight rule holds the exposure at EV 1.07, white at 2.5, everything lit blown out (an EV can be below 0: white is then under 1.2 cd/m²). Even that rule is fooled: the desk, 7.4% of the room, is only 3.0% of a picture that is 60% black, so "the brightest 5%" now reaches down into the walls (2^3.33 cd/m²), and it is the walls the rule keeps within 2 stops of white. Switch Black votes off to see the meter as it is.

The mathsEV, the meter, and where the average lands

White, the luminance that just clips at an EV of v (for a camera set to ISO 100, its film or sensor's standard sensitivity):

L_white = 1.2 × 2^v cd/m²

A reflected-light meter reading an average luminance L (12.5 is the meter's calibration constant, from the standard ISO 2720 that light meters follow):

v = log₂(L × 100 / 12.5)

Put that v back in the first (2^{log₂ x} is x again): L_white = 1.2 × L × 100 / 12.5 = 9.6 L, so the average shows at 1 / 9.6 = 0.104 of white, a tenth. The sRGB step of section 3 lifts 0.104 to 0.356 on the screen: a mid-grey. Photographers call the tone a meter aims for "18% grey", after a grey card that reflects 18% of the light; by these two constants a meter puts the average at 10.4% of white, darker by about 0.8 of a stop. The average is on the log scale, so a pixel a hundred times brighter than the rest pulls it up exactly as far as one a hundred times darker pulls it down.

The highlight rule in the same terms: if the brightest 5% begin at 2^h cd/m² and may be at most H stops over white, white is at least 2^{h − H}, and 1.2 × 2^v = 2^{h − H} gives v = h − log₂ 1.2 − H: the code's evHi.

Top: a scene's pixels by luminance, on a scale where each mark is 32 times the last (2⁵), with the meter's 10% and 90% cuts (dotted), the average of what lies between them, the brightest 5%, and white at the EV chosen. Bottom: the scene's parts as the display shows them at that EV. Choose the scene, how much of it is black, the headroom, and whether black votes; point at the histogram to read a bin.

2Above white

Divided by white, a scene still has light above 1: a lamp, a sunlit sheet of paper, a caustic. The plainest display clips it: everything over white is white, and a bright colour loses its shape, a flat patch at full strength. That is the first tone curve, none: grey light 3 stops under white shows 99 of 255, and from white up, 255.

Under none, the curve still bends: the screen's numbers are not the light but its sRGB code (section 3), drawn against stops, a log scale. And a colour clips one channel at a time: the orange light's red reaches 1 first and stops, while its green goes on rising, so it turns yellow before it turns white.

ShadyTracer's default is Khronos's PBR Neutral, a curve made for showing products and materials as they are: up to 0.76 of white it changes light only a little (the dark tones the most, next paragraph), and above that compresses the brightest channel smoothly towards 1, keeping the hue and desaturating only a little where it compresses. Grey at white shows 240 of 255, at 4 stops over, 255. A colour keeps its colour until it is very bright.

Below 0.76 the change is small but not nothing: the curve takes 0.04 of white off every channel, less off the darkest tones, whose toe bends smoothly to 0. So greys come out a little darker than with no curve: 3 stops under white, 82 of 255 against 99.

AgX, a filmic curve, starts its shoulder (where a curve starts to bend over towards white, as its toe is where it bends up from black) far below white, and turns bright saturated colours towards white, as film and camera sensors do: the orange light on the right turns pale well before white, and is nearly white by 4 stops over. It is a look, chosen for a picture. ShadyTracer offers six curves besides none (the figure leaves out AgX Punchy, a stronger AgX), and the picture's light in candela per square metre is the same under each.

Left: what the screen shows (0 to 1) for grey light from 8 stops under white to 4 over, under each tone curve of ShadyTracer's display (the chosen one bold); then the sRGB step of section 3. Right: grey, an orange and a blue light, from 6 stops under white to 4 over, through the chosen curve. Choose the curve; point at the plot to read it.

3Eight bits, spent where the eye looks

A screen's pixel keeps each channel in 8 bits, 256 steps. Kept in proportion to the light, the darkest tenth of the range would get 27 of them (codes 0 to 26), and the eye, which tells dark tones apart far more finely than bright ones, sees the steps as bands. sRGB, the standard every screen and image file assumes, stores light through a curve that spends more steps on the dark: the same tenth gets 90. Both strips are the light from black to a tenth of white, stored the two ways and shown as it would then be displayed.

Over the whole range both have all 256 steps to use; only where they put them differs. ShadyTracer's display ends with the exact sRGB curve, for a channel's light c between 0 and 1: c × 12.92 for the darkest light, up to 0.0031308, and 1.055 c^1/2.4 − 0.055 above it. A power below 1 lifts small numbers most: 0.1 becomes 0.349.

The same ramp of light, from black up to the share of white the slider sets, stored in 8 bits in proportion to the light (top) and through the sRGB curve (bottom), each shown as it would be displayed, with how many of the 256 codes fall in that range.

4How ShadyTracer does it

The two relations of the fold-out, in the file where every light setting becomes a number, and written into the graphics card's code from the same constants:

The meter's last step, on the graphics card, after the histogram's middle has been averaged:

And every pixel on its way to the screen, in this order: exposure, into the gamut (chapter 6), the tone curve, sRGB, then the contrast, saturation and brightness settings on the encoded picture ():

A number in the comment, checked. photometry.js says the meter puts the average "at about 18% of the clipping point". By its own two constants it is 12.5 / 120, 10.4%, as its test says, which sRGB shows at 0.356; 18% would show at 0.46. And the comment above the exposure setting in uniforms.schema.js lists tone curves 0 to 2, while the setting offers 0 to 6, and calls Neutral "linear up to 0.76 of white", which its 0.04 off every channel (section 2) makes not quite so.

5Tricks and trade-offs

Each is stamped with how it is known; point at a stamp to see what it means.

A picture kept as light. A picture rendered to be saved is kept as light, 16 bytes a pixel (four 32-bit numbers, each good to about seven digits, not 256 steps, so it needs no sRGB: there are no steps to spend), and developed into 8-bit sRGB afterwards with its own exposure and tone curve, through the same function the display uses (displayPixel): the exposure can be chosen after a render that took minutes, and a picture saved untouched cannot differ from the view (lib/pathtracer/develop.js).

The same picture, to the bit. A 480 × 270 render developed untouched equals the display's own output pixel for pixel with a manual exposure; with auto-exposure, 4 of its 518,400 values (129,600 pixels, four numbers each) are 1/255 off, the EV turned into a multiplier on the main processor rather than the graphics card (run with no window, on a GTX 1080 Ti, the user's Pascal card, CHANGELOG.md 0.27.0).

Adjusted before the curve. A saved picture can also be changed on the light itself, before its tone curve. A white balance, for one, says which colour of light should look white, so a room lit by an orange lamp comes out neutral, as a camera makes it; and there is a version of the Hable curve whose toe, shoulder and white point (the brightest light that still shows as white) you set yourself. These run in their own copy of the display's code, used only once one of them is moved, so an untouched picture still goes through displayPixel itself and matches the view (lib/pathtracer/darkroom/grade.js).

A cap in the exposure's own units. An optional firefly clamp caps what one path may add to a pixel, in multiples of display white, not in candela per square metre, and never below white (uniforms.schema.js, the setting's note). Off by default: capped, the picture is no longer a measurement.

6What went wrong before

A picture tuned by eye. The display was a multiplier of 2.2 and a "gamma" of 1.0: no sRGB curve at all, set by eye against a sky of no particular brightness. EV and the exact curve replaced it (the comment above the exposure settings, ).
Black voted. A scene lit only by a glowing object, more than half of it black (the emitter benchmark's median pixel is 0), metered at the histogram's floor once the black passed the 10% cut, and the exposure blew out everything lit (the comment in the histogram kernel, autoexposure.js).
A firefly clamp that darkened the sun. The clamp was in cd/m². A sunlit white floor is about 20,000 cd/m², from one shadow ray a path, so any cap low enough to touch the speckles flattened every sunlit surface: "the user saw the whole scene go dark and low in contrast at every value they tried" (wavefront.js, above the deposit). Earlier still it clamped each channel to 10,000 separately, which changed the hue of whatever it touched.
An exposure fixed before the render. A save wrote the 8-bit picture, so the exposure and the tone curve had to be right before a render that could take minutes. Now a save keeps the light.

7Try it

In the first figure, choose the room and slide the headroom from 0 to 12. Which part of the room do you lose each way?
Add black pixels with Black votes on, then off. How much black does it take to fool the old meter, and why that much?
In the second figure, compare the orange light under PBR Neutral and AgX. Which keeps its colour longer?
In ShadyTracer, open the exposure settings, switch auto-exposure off, and set EV 15 outdoors, then EV 7: what happens to the sky, and to a lamp? Then try each tone curve on a lit glass object.