Video Tone Mapping on GPU

Author: Fyodor Serzhenko
Tags: HDR, Tone Mapping, ALTMapper, CUDA, Video Processing, Camera ISP on GPU

Video tone mapping is essential because modern cameras capture high dynamic range (HDR) scenes that far exceed the display capabilities of standard dynamic range (SDR) screens, and even most HDR displays. The core challenge is compressing this massive range of brightness (often 10,000+ nits) into the 100-400 nits of a typical monitor without destroying detail, color, and temporal stability.

Original raw frame captured with XIMEA 8.9 MPix camera
The result of tone mapping is on the left, the result of the same pipeline without tonemapper (gamma sRGB is included) is on the right

The Core Problem: HDR Capture vs. LDR Display

Image sensors in professional digital cinema cameras, machine vision cameras, drones, and even smartphones capture 12-16 stops of dynamic range linearly. This linear data is scientifically accurate but visually unviewable — it appears flat, washed out, or with crushed blacks and clipped highlights. Tone mapping applies a non-linear transform to make this data visually appealing. However, video is not just a sequence of images. Processing each frame independently could create catastrophic artifacts unique to motion.

Why Video Tone Mapping is Different (and Harder)

1. The Flickering Problem (Temporal Instability)

This is the #1 killer of video quality. Frame-by-frame tone mapping causes brightness and color to fluctuate unnaturally:

A gray wall might pulse between light and dark across frames
Shadows crawl and breath even when lighting is static
Noise gets amplified differently each frame, creating "dancing" artifacts

Why it happens: Local tone mapping algorithms (like CLAHE or bilateral grid) compute parameters per-frame. Small changes in camera position or sensor noise cause these parameters to oscillate, creating temporal incoherence.

Without temporal coherence:

Image quality is unusable for various applications
Compression efficiency collapses — codecs waste bits on flickering instead of detail
Viewer fatigue increases dramatically

2. Motion Artifacts (Ghosting and Halos)

Local methods that work great on photos create halos that move with objects:

A bright window glows and the glow lags behind when the camera pans
Object edges shimmer as local contrast changes per-frame
Temporal ghosting: Moving objects leave "trails" of incorrect exposure

This occurs because local operators don't respect motion — they treat each frame as isolated.

3. Real-Time Performance Constraints

Unlike photo processing (which can take seconds on the CPU), video often requires real-time performance or even faster:

Camera apps: ISP for RAW processing faster than real-time, including tone mapping
Cinema: 24-48 fps, but needs highest quality (offline rendering acceptable)
Broadcast: Live events need real-time performance and minimum latency

This forces algorithm choices that balance quality vs. speed, often sacrificing spatial quality (algorithm strength) for temporal stability.

What Happens Without Proper Video Tone Mapping?

Clipping: Details lost in skies and shadows (looks "video-like")
Flatness: Low contrast, "log-like" look without artistic intent
Flicker: Unwatchable even to untrained viewers
Color Shifts: Saturation changes between frames (especially in skin tones)
Compression Artifacts: Flicker masquerades as film grain, breaking codecs

Side by side video with/without ALTMapper

Video from XIMEA 8.9 MPix camera with GPU-based ISP, with tone mapping (on the left) and without tone mapping (on the right)

Summary: Why We Need It

Without video tone mapping:

HDR footage is unviewable (linear/log data looks wrong)
SDR displays clip and crush detail
Flicker and motion artifacts make content unwatchable
Compression becomes inefficient
Viewer experience is poor

With proper video tone mapping:

Detail is preserved in highlights and shadows
Colors remain natural and stable
Motion is smooth without artifacts
Displays of all types can show the content correctly
Artistic intent is maintained from capture to display

The need is not just technical — it's foundational to modern visual storytelling across cinema, streaming, gaming, and communication.

ALTMapper Performance Demonstration

We evaluated the performance by running tests on the NVIDIA GeForce RTX 3060. We used the XIMEA MC089CG-SY camera, which has an output of 8.9 megapixels at 40 frames per second (12-bit mode). We used the full image processing pipeline (ISP) for RAW processing on the GPU, which includes tone mapping. ALTMapper on that GPU takes about 2.5 milliseconds to process. This means that ALTMapper can process around 4 gigapixels per second on that hardware.

Speed isn't just a feature — it's a requirement. On the NVIDIA GeForce RTX 4090, ALTMapper processes 62-megapixel frames in just 3.86 milliseconds. That's over 16 Gigapixels per second, enabling real-time 8K video tone mapping with minimal GPU memory footprint.

Use Cases

We've included ALTMapper algorithm on the NVIDIA GPU to the following software:

Some of our customers have already embedded that tonemapper into their solutions for high performance raw processing.