Fast RAW Image Encoding on NVIDIA GPUPerformance of image processing in camera applications could be much higher if we apply image compression for raw frames without using debayer. That approach is usually called Raw Bayer Compression. Right after image acquisition we can split RAW image to four planes according to colors of Bayer pattern (RGGB). Then we can do JPEG image compression for each plane with high quality and store compressed image to SSD. It could be done very fast on GPU and without introducing significant image artifacts. In this way we can temporary exclude debayer from image processing pipeline and increase performance for realtime applications. This is the main idea - to store RAW image with visually lossless encoding. It is the way to preserve original raw frames for further white balance, exposure correction, debayering and color grading at post-production. In the task of realtime data acquisition one should set JPEG quality at least 90% or more. This is so called set of parameters for visually lossless compression. JPEG quality 100% or lossy JPEG2000 are preferrable, but in that case compression ratio and performance could be not high enough. Optimal compression quality is very important, because strong JPEG image compression can introduce additional image artifacts, which will lead to image artifacts at RAW-to-RGB conversion and at post processing.
Pipeline for RAW image encoding on the GPU
After such image acquisition we finally get compressed RAW frames at SSD. Actually we are encoding 1-channel image which leads to good compression ratio and fast performance. Then we need to solve the task of image processing and visualizing for compressed RAW images. GPU pipeline for RAW decoding, processing and visualization
The above pipeline is computationally demanding and it's not easy to achieve high performance for that task. For visualizing we need to reach up to 60 fps, and for most resolutions this could be done on NVIDIA GPU in realtime. We utilize our image processing SDK for the above pipeline and it can handle data rates up to 4 GPix/s on GeForce RTX 4090 for the case with built-in JPEG compression on NVIDIA GPU. Such a solution is available for all NVIDIA GPUs: mobile, laptop, desktop, server. We can supply the software for each stage of the task
To improve image quality for that application, we could run camera calibration procedures, including FFC, color/tona calibration, undistortion, etc. Optional methods for 12-bit RAW encoding on the GPU
We also have more sofisticated methods for RAW image compression which are applicable for 14-bit and 16-bit RAW frames from photo and video cameras. These methods also take into account camera noise to offer high image quality at the restoration stage. Performance benchmarks for 12-bit RAW encoding on the GPU
Useful links on the subject |