The Science Inside the RAW Pipeline

Most photographers interact with their RAW processor as a collection of sliders. Move the temperature slider right, the image warms up. Increase highlight recovery, the blown sky comes back. Choose "AMaZE" in the demosaicing dropdown because someone on a forum said it was better. It works. You get good results. But you don't know why it works, and that means you don't know when it will fail, or how to push it further.

This article is the course I wish I'd had when I started building RAW processors. It covers every major stage of the pipeline, from the raw sensor data to the final rendered pixel, with the mathematics and physics explained in enough depth that you'll understand not just what each slider does, but the science behind it. If you've ever wondered what "correlated color temperature" actually means, why your demosaicing algorithm matters, or how a denoiser can remove noise without destroying detail, this is where you'll find out.

I've organized this around the nine stages of a typical RAW pipeline, and the principles apply to any RAW processor. Lightroom, DxO, RawTherapee, darktable — they all implement variations of the same fundamental operations, rooted in the same physics and the same century of color science.

Part I — The Sensor: Where Light Becomes Numbers

The Bayer Mosaic

A digital camera sensor is a grid of photosites, each of which measures how many photons struck it during the exposure. That's all it records: a count of photons, stored as an integer. A 45-megapixel sensor produces 45 million integers. No color information whatsoever.

Color comes from a mosaic of tiny colored filters placed over the photosites, called a Color Filter Array (CFA). The most common pattern, invented by Bryce Bayer at Kodak in 1976, arranges red, green, and blue filters in a repeating 2×2 block: one red, one blue, and two green. The green gets two because human vision is most sensitive to green wavelengths, so more green samples give better luminance resolution.

Row 0:  R  G  R  G  R  G  R  G  ...
Row 1:  G  B  G  B  G  B  G  B  ...
Row 2:  R  G  R  G  R  G  R  G  ...
Row 3:  G  B  G  B  G  B  G  B  ...

When you open a RAW file, the first thing the software reads is this mosaic of single-channel values. At this point, the data is just a grid of integers — typically 12-bit (0 to 4,095) or 14-bit (0 to 16,383) depending on the camera. The entire purpose of the RAW pipeline is to transform this grid of integers into a full-color (or B&W) image that looks like what you saw through the viewfinder.

Black Level and White Level

The first operation in the pipeline is deceptively simple but critically important. Every sensor has a black level: an offset voltage that the ADC (analog-to-digital converter) adds to every reading. Even with the shutter closed, pixels don't read zero. They read some small positive value, typically 256 or 512 on a 14-bit scale. This offset must be subtracted before any processing, or your shadows will be lifted and your contrast will be wrong.

The black level varies by CFA color. Red pixels might have a black level of 512, green pixels 510, blue pixels 514. The camera stores these values in the RAW file metadata. Some cameras use different black levels for the top and bottom halves of the sensor, or even store a full row of optically-shielded pixels along the sensor edge for real-time dark current measurement.

After subtracting the black level, the values are normalized by the white level — the maximum value the ADC can produce (e.g., 16,383 for 14-bit). The result is a floating-point value between 0.0 (black) and 1.0 (maximum sensor output) for each pixel. Values above 1.0 can occur after white balance multiplication and represent clipped (saturated) channels.

pixel_normalized = (raw_ADC - black_level[color]) / (white_level - black_level[color])

Part II — White Balance: The Physics of Color Temperature

White balance is the step that brings us into the deep end of color science. To understand what your temperature slider actually does, you need to understand how physicists describe light sources, which takes us back to 1900 and Max Planck's work on thermal radiation.

Black-Body Radiation and the Planckian Locus

A "black body" is an idealized object that absorbs all radiation and, when heated, emits light with a spectrum determined entirely by its temperature. Planck's law gives the spectral power distribution of this radiation at any temperature. Heat a black body to 1,800 K (kelvin) and it glows deep red. At 2,856 K, it produces the warm yellow of an incandescent bulb. At 5,500 K, it approximates noon sunlight. At 10,000 K and above, it appears blue-white.

The critical insight: the color of a black body at a given temperature can be plotted as a single point in a two-dimensional color space. When you plot the color of a black body for all temperatures from 1,667 K to 25,000 K, you get a smooth curve called the Planckian locus. This curve is the backbone of color temperature. When your camera says "5500K," it means the scene's illumination has the same chromaticity as a black body heated to 5,500 kelvin.

CIE 1931: The Map of All Colors

In 1931, the Commission Internationale de l'Éclairage (CIE) defined a mathematical framework for describing color that is still the foundation of all color science. The CIE 1931 color space maps every visible color to a pair of coordinates, x and y, called chromaticity coordinates. The full diagram — the famous horseshoe shape you've seen in textbook illustrations — contains every color the human eye can perceive.

The Planckian locus is a curve within this diagram, starting in the orange-red region (low temperatures) and sweeping through white toward blue (high temperatures). Every point on this curve corresponds to the color of a heated black body at a specific temperature.

To plot a specific temperature on this curve, the standard method uses the Kim et al. (2002) approximation, a set of polynomial equations that compute the x chromaticity from the temperature, then the y chromaticity from x:

For T < 4000 K:
  x = -0.2661239×10&sup9;/T³ - 0.2343589×10&sup6;/T² + 0.8776956×10³/T + 0.179910

For T ≥ 4000 K:
  x = -3.0258469×10&sup9;/T³ + 2.1070379×10&sup6;/T² + 0.2226347×10³/T + 0.240390

For T < 2222 K:
  y = -1.1063814 x³ - 1.34811020 x² + 2.18555832 x - 0.20219683

For 2222 K ≤ T < 4000 K:
  y = -0.9549476 x³ - 1.37418593 x² + 2.09137015 x - 0.16748867

For T ≥ 4000 K:
  y =  3.0817580 x³ - 5.87338670 x² + 3.75112997 x - 0.37001483

The old approach — and the one you'll still find in many open-source tools — uses the Tanner Helland approximation, a simpler set of equations that were designed for display rendering, not color science. The Helland formulas convert temperature directly to RGB values, skipping the CIE framework entirely. This works adequately for on-screen color previews, but it's inaccurate for white balance correction because it doesn't account for the geometry of the chromaticity diagram. The errors are small at mid-range temperatures (4000–7000 K) but grow significantly at the extremes, exactly where creative B&W photographers like to work.

From Chromaticity to WB Multipliers

Once we have the CIE (x, y) coordinates of the target illuminant, we need to convert them into the red, green, and blue multipliers that the pipeline applies to each pixel. The conversion follows a precise chain:

The result is a set of three numbers. At 3200 K (tungsten), you get something like [0.52, 1.0, 2.66] — red is suppressed by half, blue is boosted nearly three-fold. At 10000 K (shade), you get [1.14, 1.0, 0.70] — red boosted, blue suppressed. The pipeline multiplies every Bayer pixel by the multiplier for its CFA color. This is why white balance happens before demosaicing: it operates on the raw sensor data, one pixel at a time, before any color reconstruction.

The Tint Axis: Moving Perpendicular to the Locus

The temperature slider moves your white point along the Planckian locus. But real-world light sources don't sit perfectly on this curve. Fluorescent lights, LEDs, and mixed lighting conditions produce chromaticities that fall off the locus, displaced toward green or magenta. The tint slider corrects for this displacement.

To understand how tint works properly, we need a second CIE color space: CIE 1960 UCS (Uniform Chromaticity Scale). The 1931 xy diagram has a significant flaw: equal distances in the diagram don't correspond to equal perceptual differences. A shift of 0.01 in the green region looks like a huge color change, while the same shift in the blue region is barely noticeable. The UCS space corrects this by warping the coordinates to be more perceptually uniform.

The conversion from CIE 1931 (x, y) to CIE 1960 (u, v) is straightforward:

Forward:
  u = 4x / (-2x + 12y + 3)
  v = 6y / (-2x + 12y + 3)

Inverse:
  x = 3u / (4 + 2u - 8v)
  y = 2v / (4 + 2u - 8v)

In the UCS space, the Planckian locus is a smooth curve, and at any temperature, the perpendicular direction (the isothermal line) points along the green–magenta axis. Shifting in one direction adds green; shifting in the other adds magenta. This is geometrically precise and perceptually uniform, unlike the naive approach of just multiplying the green channel by a factor.

The implementation computes the tangent direction of the locus at the current temperature using a finite difference (evaluating the locus at T−50 K and T+50 K), then rotates it 90° to get the perpendicular. The tint value scales the shift along this perpendicular, with ±100 corresponding to approximately ±0.02 Δuv — a range calibrated to match the behavior of Adobe Camera Raw and DxO.

"As Shot" Mode: Reading the Camera's WB

When you select "As Shot" white balance, the processor reads the WB coefficients that the camera stored in the RAW file at capture time. These coefficients are the camera's own red, green, and blue multipliers, determined by the camera's auto white balance algorithm (or the manually selected WB preset) at the moment of exposure.

These camera-provided coefficients are usually stored as four values [R, G1, B, G2] in the RAW metadata. The processor normalizes them by dividing by the green coefficient, giving multipliers relative to green = 1.0. This is the most "faithful" starting point because it reflects the camera's assessment of the scene illumination, but for B&W work it's rarely the optimal choice because the camera was optimizing for neutral color, not tonal separation.

Part III — Denoising on the Bayer Mosaic

Denoising is one of the most consequential steps in the pipeline, and where it's placed matters enormously. Most consumer software denoises after demosaicing, operating on the fully reconstructed RGB image. Professional-grade processors denoise before demosaicing, directly on the Bayer data. The difference is significant.

When you denoise after demosaicing, the noise has already been mixed between channels by the interpolation algorithm. Red noise has bled into the green estimate, blue noise into the red, and the denoiser must try to untangle this cross-contamination. When you denoise on the Bayer data, each pixel's noise is independent and channel-pure. The denoiser works on cleaner input and produces cleaner output.

The catch is that Bayer denoising is technically harder. You can't use standard 2D filters because adjacent pixels are different colors. A red pixel at position (0,0) has green neighbors at (0,1) and (1,0) and a blue neighbor at (1,1). Averaging these would be nonsensical. Instead, you must restrict the filter to pixels of the same CFA color, which are spaced two pixels apart in each direction.

Bilateral Filtering on the Bayer Mosaic

The bilateral filter is the workhorse of edge-preserving denoising. It was introduced by Tomasi and Manduchi in 1998 and remains one of the most practical denoising tools because of its simplicity, speed, and tunable behavior.

A conventional Gaussian blur averages nearby pixels weighted by distance: close pixels contribute more than far ones. This smooths noise but also destroys edges. The bilateral filter adds a second weighting term: pixels contribute more if they have similar intensity to the center pixel. At an edge, the pixels on the other side of the boundary have very different intensities, so they receive near-zero weight and don't blur across the edge.

                    Σ w_s(||p-q||) · w_r(|I(p) - I(q)|) · I(q)
BF[I](p) = ─────────────────────────────────────────────────
                    Σ w_s(||p-q||) · w_r(|I(p) - I(q)|)

where:
  w_s(d) = exp(-d² / 2σ_s²)     spatial weight (Gaussian in distance)
  w_r(d) = exp(-d² / 2σ_r²)     range weight (Gaussian in intensity)
  σ_s = spatial sigma         controls blur radius
  σ_r = range sigma           controls edge sensitivity

On the Bayer mosaic, the filter only considers pixels of the same CFA color. For a red pixel at position (row, col), the neighbors are at (row±2, col), (row, col±2), (row±2, col±2), etc. — always stepping by 2 in each direction. The spatial sigma is typically set to 1.5 (in same-color pixel spacing), and the range sigma is adjusted by the strength parameter: σ_r = 0.02 + strength × 0.08, where strength ranges from 0 to 1.

The bilateral filter is fast (O(n × r²) where r is the kernel radius in same-color spacing) and predictable. Its weakness is that it can produce staircasing artifacts on smooth gradients at high strength settings, because the range weighting creates discrete steps rather than smooth transitions.

Non-Local Means (NLM): Patch-Based Denoising

Non-Local Means, introduced by Buades, Coll, and Morel in 2005, takes a fundamentally different approach. Instead of comparing individual pixel intensities, it compares patches — small rectangular neighborhoods around each pixel. Two pixels that have similar patches are considered "related" even if they're far apart spatially. This allows the algorithm to find and average repeated patterns across the entire image, producing remarkably clean results while preserving texture.

                  Σ w(p, q) · I(q)
NLM[I](p) = ────────────────────────
                  Σ w(p, q)

where:
  w(p, q) = exp(-||P(p) - P(q)||² / h²)

  P(p) = the patch of pixels around p (e.g. 7×7 neighborhood)
  ||P(p) - P(q)||² = sum of squared differences between patches
  h = filtering parameter (controls strength)

On the Bayer mosaic, NLM operates on sub-images. The four CFA positions (R, G1, G2, B) are extracted into separate quarter-resolution images. NLM runs independently on each sub-image, then the results are written back into the full-resolution mosaic. This ensures that each color plane is denoised without cross-channel contamination.

The computational cost is substantial: O(n × patch² × search²) per pixel. For a 45-megapixel image with 7×7 patches and 11×11 search windows, this is roughly 600 operations per pixel per sub-image, or about 27 billion operations total. This is why NLM is typically 5–10× slower than the bilateral filter. On a modern multi-core CPU using parallel processing, a 24-megapixel RAW file takes 2–5 seconds for NLM versus 0.3–0.8 seconds for bilateral.

When to Use Which

Part IV — Highlight Recovery

When a photosite receives too many photons, its charge well saturates and the ADC reads the maximum value. The pixel is "clipped." In a single-channel sensor, this would mean total information loss. But in a Bayer mosaic, clipping rarely hits all three channels simultaneously. Blue sky, for instance, might clip the blue channel while red and green remain below saturation. A white shirt might clip red and green while blue barely touches the ceiling.

Highlight recovery exploits this partial clipping. When one or two channels are saturated but others aren't, the algorithm estimates the missing values from the surviving channels using the ratios established by the white balance. If the WB multipliers tell us that R should be 0.85× the green value in this part of the scene, and green isn't clipped, we can reconstruct a plausible red value.

The strength parameter controls how aggressively the algorithm blends the recovered values back into the image. At 0%, clipped channels are simply clamped to white. At 100%, the algorithm reconstructs as much as it can. In practice, recovery above 60–70% tends to produce flat, grey-looking highlights that have lost their luminous quality. For B&W work, where the tonal character of highlights is critical, moderate recovery (40–60%) usually produces the best results.

Part V — Demosaicing: Reconstructing What the Sensor Didn't Capture

Demosaicing is the most computationally intensive and algorithmically complex step in the pipeline. Its job: take a mosaic where each pixel has only one color, and produce an image where every pixel has all three. Two-thirds of the data in your final image is fabricated by this algorithm. The quality of that fabrication determines whether your image has clean edges or zipper artifacts, whether fine detail is preserved or smeared, and whether you see false color moiré on high-frequency textures.

Bilinear Interpolation: The Baseline

The simplest approach: for each missing color at each pixel, average the nearest neighbors of that color. To find the green value at a red pixel, average the four surrounding green pixels (above, below, left, right). To find the red value at a blue pixel, average the four surrounding red pixels (at the diagonals).

Bilinear interpolation is fast and easy to understand, but it's fundamentally flawed. It assumes that color changes smoothly across the image, which is true in uniform areas but catastrophically wrong at edges. At a sharp boundary between a red object and a green background, the bilinear filter happily averages red pixels on one side with green pixels on the other, producing a smeared, color-fringed edge. On high-frequency patterns (fabric weaves, window screens, brick walls), it generates moiré: false patterns that don't exist in the original scene.

Despite its flaws, bilinear interpolation serves as the speed baseline. When you need to process thousands of images quickly and absolute quality isn't critical (contact sheets, quick previews, machine learning training data), bilinear gets the job done in minimal time.

PPG: Patterned Pixel Grouping

PPG, described by Chuan-Kai Lin in 2009, improves on bilinear by making the interpolation direction-aware. Instead of blindly averaging all neighbors, it estimates the gradient (rate of change) in the horizontal and vertical directions and interpolates along the direction with the smaller gradient — the direction where the image is smoother.

Horizontal gradient:
  grad_H = |G_left - G_right| + |2·R_center - R_left2 - R_right2|

Vertical gradient:
  grad_V = |G_up - G_down| + |2·R_center - R_up2 - R_down2|

If grad_H < grad_V:
  G = (G_left + G_right)/2 + (2·R_center - R_left2 - R_right2)/4

If grad_V < grad_H:
  G = (G_up + G_down)/2 + (2·R_center - R_up2 - R_down2)/4

If grad_H == grad_V:
  G = average of both estimates

The gradient test is PPG's key innovation. At a vertical edge, the horizontal gradient is large (the colors change rapidly left-to-right) but the vertical gradient is small (the colors are similar top-to-bottom). PPG chooses the vertical direction, interpolating along the edge rather than across it. This preserves edge sharpness at modest computational cost.

PPG typically runs 1.5–2× slower than bilinear and produces noticeably better results on edges. Its weakness is that it only considers two directions (horizontal and vertical), which means it struggles with diagonal edges and fine textures where the optimal interpolation direction is neither H nor V.

AMaZE: The State of the Art

AMaZE (Aliasing Minimization and Zipper Elimination) was developed for RawTherapee by Emil Martinec and represents the current state of the art in open-source demosaicing. It extends the directional approach to four directions and uses a sophisticated fusion strategy based on local homogeneity analysis.

The homogeneity analysis in step 2 is what separates AMaZE from simpler directional methods. Consider a pixel at a nearly-diagonal edge. The horizontal and vertical interpolations both cross the edge and produce inconsistent color differences (high variance). The diagonal interpolation aligned with the edge stays on one side and produces consistent results (low variance). AMaZE automatically selects the diagonal direction in this case, something PPG cannot do.

The computational cost is roughly 3–5× that of PPG, due to the four interpolation passes and the homogeneity analysis. On a modern 8-core CPU processing a 24-megapixel RAW file, AMaZE takes approximately 500–800 ms versus 150–250 ms for PPG. For single images, this difference is negligible. For batch processing thousands of files, it becomes a meaningful factor.

Choosing Your Algorithm

For B&W photography specifically, the choice matters less than you might expect. Much of what demosaicing fights against — false color, chroma moiré — disappears when you convert to monochrome. What remains is luminance resolution and edge definition, where AMaZE still holds an advantage but PPG is often sufficient. If you're printing at 20×30 inches from a 45-megapixel sensor, the difference between AMaZE and PPG may be visible in fine fabric textures. At 8×10 or on screen, it's nearly impossible to see.

Part VI — Color Matrix: Speaking the Sensor's Language

After demosaicing, you have a full RGB image. But the R, G, B values are in the camera's native color space — determined by the spectral response of the sensor's CFA dyes. A Canon sensor's "red" isn't the same as a Nikon sensor's "red." Their red filters pass different wavelengths, so the same scene produces different RGB numbers on different cameras.

The color matrix transforms these camera-native RGB values into a standard color space (sRGB, via CIE XYZ as an intermediate). The transformation is a 3×3 matrix multiply:

| X |   | m00  m01  m02 |   | R_cam |
| Y | = | m10  m11  m12 | · | G_cam |
| Z |   | m20  m21  m22 |   | B_cam |

then:

| R_sRGB |   |  3.2405  -1.5372  -0.4986 |   | X |
| G_sRGB | = | -0.9693   1.8760   0.0416 | · | Y |
| B_sRGB |   |  0.0556  -0.2040   1.0572 |   | Z |

For B&W work, the color matrix step is still important even though the output will be monochrome. The matrix affects the relative brightness of different colors, which directly controls how they translate to grey tones. Skipping it would mean your B&W conversion is based on the camera's spectral response rather than standardized colorimetry, which makes the result camera-dependent and unpredictable.

Part VII — Exposure, B&W Mixing, and Gamma

Exposure Compensation

Exposure compensation in the digital domain is a simple multiplication. One stop of exposure compensation doubles the pixel values; minus one stop halves them. The formula is:

pixel_corrected = pixel_linear × 2^EV

where EV is the exposure compensation in stops.

+1.0 EV = ×2.0    (one stop brighter)
-1.0 EV = ×0.5    (one stop darker)
+0.5 EV = ×1.414  (half a stop brighter)

This happens in linear light space, before gamma correction. The exponential relationship matches the physics of exposure: each stop represents a doubling or halving of light. Because the data is still linear at this point, the multiplication is physically accurate — it's equivalent to having used a different shutter speed or aperture at capture time, within the limits of the sensor's dynamic range.

B&W Channel Mixing

The channel mixer converts the color image to monochrome by computing a weighted sum of the three channels. This is the digital equivalent of placing a colored filter in front of the lens when shooting B&W film. A red filter (high red weight) brightens red subjects and darkens blue ones. A blue filter does the opposite.

luminance = R × w_red + G × w_green + B × w_blue

Standard luminance (Rec. 709):    0.2126 R + 0.7152 G + 0.0722 B
Yellow filter equivalent:         0.35 R   + 0.45 G   + 0.20 B
Orange filter equivalent:         0.50 R   + 0.35 G   + 0.15 B
Red filter equivalent:            0.60 R   + 0.30 G   + 0.10 B
Green filter equivalent:          0.20 R   + 0.55 G   + 0.25 B
Blue filter equivalent:           0.10 R   + 0.25 G   + 0.65 B

The critical insight for B&W photographers: white balance and channel mixing are complementary tools, not alternatives. White balance shifts the raw channel gains before demosaicing, changing the fundamental tonal relationships in the data. The channel mixer then selects how those modified channels contribute to the final luminance. Used together, they give you a two-dimensional control space that's far more powerful than either alone.

sRGB Gamma

The final step transforms the pixel values from linear light to the sRGB gamma curve. This is necessary because human perception of brightness is nonlinear: we're much more sensitive to differences in dark tones than in bright ones. A linear-encoded image displayed directly on a monitor looks extremely dark, with all the detail crushed into the shadows.

If C_linear ≤ 0.0031308:
  C_sRGB = 12.92 × C_linear

If C_linear > 0.0031308:
  C_sRGB = 1.055 × C_linear^(1/2.4) - 0.055

Skipping the gamma step (using the --linear flag in dusk-raw) produces a file in linear light space. This is useful when the output will be processed further in software that expects linear data, or when you want to analyze the actual photometric values. But for viewing on screen or printing, gamma encoding is essential.

Part VIII — The Complete Pipeline

Here's the complete pipeline, with every step, its input, its output, and what it changes in the image:

The order is not arbitrary. White balance must happen before demosaicing because the WB multipliers change the effective sensitivity of each CFA color, which affects the demosaicing algorithm's gradient calculations. Denoising must happen before demosaicing because the noise is still channel-pure on the Bayer data. The color matrix must happen after demosaicing because it needs full RGB triplets. Exposure must happen before B&W mixing because the mixing weights are calibrated for specific brightness levels. Gamma must be last because every prior step operates in linear light.

Every step can be individually bypassed using the --skip-* flags in the dusk-raw CLI. This lets you inspect the intermediate results at any point in the pipeline. Want to see what the image looks like after demosaicing but before color correction? Use --raw-demosaic-only. Want linear-encoded color output for external processing? Use --color --linear. This kind of step-by-step inspection is invaluable for understanding what each stage contributes and for diagnosing processing problems.

Afterword: What DxO Does That We Don't (Yet)

DxO PhotoLab is the benchmark for RAW processing quality, and honesty about the gap is important. Their advantages fall into a few categories:

Per-camera optical profiles. DxO calibrates every camera-lens combination they sell profiles for, measuring distortion, vignetting, chromatic aberration, and sharpness across the entire field. Their corrections are not generic algorithms but measured lookup tables specific to your exact equipment. Building this kind of profile database requires a laboratory and years of measurement.

DxO DeepPRIME. Their neural-network denoiser, trained on pairs of noisy and clean images from specific cameras, produces results that handcrafted algorithms like NLM cannot match at high ISO. The network has learned the specific noise characteristics of each sensor and can separate signal from noise with remarkable precision.

Lens softness correction. DxO's profiling data includes measured MTF (Modulation Transfer Function) for each lens at each aperture and focal length. They can deconvolve the optical softness, effectively "un-blurring" the lens. This is something no generic sharpening algorithm can replicate because it requires knowledge of the specific blur kernel.

These are hard advantages built on proprietary data. Our approach is to match or exceed on the algorithmic side — CIE-correct white balance, AMaZE demosaicing, proper Bayer-domain denoising — where the physics and mathematics are public and well-understood. The profile-based corrections are a longer-term goal that requires either community contribution or partnership with calibration data providers.

Written by

Eric K'DUAL

Photographer & Writer

Eric K'DUAL is a French photographer and digital artist based in France. Passionate about code and black & white photography, he bridges traditional darkroom craft with modern computational imaging, building his own tools and chasing the decisive moment in monochrome.

Home All Articles