|
|
GPU RAW Processor for Camera Applications
We offer high quality GPU-accelerated solution for RAW image/video processing with very high performance - much faster than realtime. Now it's possible by implementing the whole RAW image processing pipeline on NVIDIA GPU. This is applicable both for realtime tasks for machine vision camera applications and for offline RAW-to-RGB processing in digital cinema applications.
Advantages of RAW image processing on GPU
- High speed RAW rendering (see benchmark below)
- High quality debayering
- Realtime adaptive denoising
- Flexible image processing pipeline with 16/32-bit workflow
- Smooth RAW video playback with full resolution (no screen tearing)
- Realtime resize before OpenGL output to monitor (no aliasing)
- Online RGB Parade and Histogram output
- Software outputs: processed image to monitor, 8-bit JPEG or 16-bit TIFF for each frame
- h264/h265 stream with desired resolution to SSD
We have implemented full image processing pipeline on GPU to speed up every stage of RAW data workflow. That gives us opportunity to significantly decrease data transfer time over PCI-E bus and to improve total performance. At the same time, output image quality is comparable with Adobe Camera Raw (ACR), LightRoom and Raw Therapee software.
Features of GPU RAW Processor
- Input images: 8-16 bits per pixel RAW, PGM, byte array in CPU/GPU memory
- Output formats for image series: JPEG (24-bit), PPM (48-bit), TIFF (24/48-bit), J2K (24/48-bit), EXR (48-bit)
- Output formats for video: AVI (Motion JPEG), MP4, MOV (Apple ProRes)
- Output color space: sRGB, Rec.709, Rec.2020, AdobeRGB, ProPhoto, BMD Film, BMD Film 4K, BMD Film 4.6K
- Fast demosaicing algorithms with 16-bit pipeline: L7, DFPD, MG
- Denoise algorithms for luma and chroma (float workflow): wavelets, bilateral, NLM
- Resize algorithm: Lanczos (float workflow)
- Sharpening with USM (float workflow)
- Realtime playback of processed RAW images via OpenGL
Compatibility
- Windows-10 (64-bit)
- CUDA-12.3 (64-bit)
- NVIDIA GPU (Pascal, Volta, Turing, Ampere, Ada) for mobile, laptop, desktop, server
- Compatibility with FFmpeg to read/write MJPEG streams (FFmpeg is under LGPLv2.1)
Camera support for RAW processing workflow on GPU
- Blackmagic Design Pocket/Production 4K/Cinema, URSA and URSA mini cameras
- Kinefinity cameras: KINEMINI 4K, KINEMAX 6K, KINERAW-MINI (Full HD), KINERAW-S35 (Full HD)
- AJA CION 4K/UHD and 2K/HD Production Camera
- DJI Zenmuse X5R camera for Mavic, Phantom, Inspire, Matrice drones
- Digital Bolex D16 and D16M
- Atomos Ninja Shogun
- Ikonoskop
- XIMEA 8/12/20/50/65 MPix
- Basler
- IO Industries Redwood with Euresys grabber
- Emergent Vision 10/25/50/100-GigE cameras
- Daheng Imaging
- Mindvision
- Hikrobot
Image Processing Pipeline for RAW processor on GPU
- Ingestion of RAW images from HDD/SSD
- Multithreaded reading of RAW frames from SSD to CPU memory
- RAW parsing
- Opional RAW unpacking or decoding
- Data transfer from CPU to GPU
- Data linearization and transform to 16-bit representation
- Dark frame subtraction and Flat-Field Correction
- RAW historgam
- White/Black points
- White Balance (R, G1, G2, B)
- Exposure compensation
- Raw Tone Curve for each channel of Bayer data
- Raw denoising
- Demosaicing
- Adaptive noise suppression
- Color Correlated Temperature and Tint control
- Color Correction with CCM matrix or DCP profile
- Tone mapping (RGB/HSV Curves for each channel)
- Histogram and RGB Parade for processed image
- Monitor output
- Crop
- Resize to specified width and height of output window
- Sharp with USM
- Transform with monitor profile
- LUT 16-bit to 8/10-bit for each color channel
- Copy data to OpenGL texture
- Realtime monitor output via OpenGL (RAW video player)
- Histogram output for RAW and processed data
- RGB Parade output
- Vectorscope output
- Compressed video output
- Crop
- Resize to Full HD
- Sharp with USM
- LUT 16-bit to 8/10-bit for each color channel
- Native MJPEG compression
- Optional native encoding to Apple ProRes 422 LT, 422, 422 HQ, 4444, 4444 XQ
- External compression via FFmpeg to MP4
- Data transfer from GPU to CPU memory
- Save compressed video to SSD/HDD
- Image sequence output
- Data transform to 16-bit TIFF or to 8-bit JPEG for each frame
- Data transfer from GPU to CPU memory
- Save image sequence to SSD/HDD
Benchmarks for NVIDIA GeForce RTX 4090
Standard image processing pipeline for 12-bit RAW images with 4K resolution on GPU GeForce RTX 4090 could be done at frame rates up to 60 fps (raw and processed data reside in CPU memory, without denoising and USM). Here you can find more info about the benchmarks.
Options for realtime output compression on GPU
- JPEG/MJPEG codec
- JPEG2000/MJ2K codec
Standard tasks to solve with GPU RAW Processor
- Batch image processing for RAW image sequences
- Realtime full frame rendering, compression, preview and playback for video streams with RAW data
- Full image processing pipeline for industrial cameras with 8-16 bits Bayer format
Workflow example for GPU RAW Processor
- Upload video footage to computer (PC) from camera or card reader
- Create new project and import RAW images
- RAW image processing with GPU RAW Processor
- RAW video player
- Export JPEG (8/24-bit) or TIFF (16-bit) files
- Export to Motion JPEG or H.264/H.265/AV1
Useful links:
Roadmap for GPU RAW Processor
- Highlights recovery algorithm - done
- Bilateral denoiser on GPU - done
- NLM denoiser on GPU - done
- Chromatic aberrations removal in RAW domain - in progress
|
