Bayer Codec on NVIDIA GPUPerformance of image processing in camera applications could be much higher is we apply image compression for raw data without using debayer. That approach is usually called Raw Bayer Compression. Right after image acquisition we can split RAW image to four planes according to colors of Bayer pattern (RGGB). Then we can do JPEG image compression for each plane with high quality. It could be done very fast on GPU and without introducing significant image artifacts. In this way we can temporary exclude debayer from image processing pipeline and increase performance for realtime applications. This is the main idea - to store RAW image with visually lossless encoding. It is the way to preserve original data for further white balance, exposure correction, debayering and color grading in post-production. In the task of realtime data acquisition one should set JPEG quality at least 90% or more. This is so called set of parameters for visually lossless compression. JPEG quality 100% or lossy JPEG2000 are preferrable, but in that case compression ratio and performance could be not high enough. Optimal compression quality is very important, because strong JPEG image compression can introduce additional image artifacts, which will be boosted further, at debayer stage. Pipeline for RAW image acquisition on GPU
After such image acquisition we finally get compressed RAW data at hdd/ssd. Then we need to solve the task of image processing and visualizing for compressed RAW images. GPU pipeline for RAW image decoding, processing and visualization
The above pipeline is computationally demanding and it's not easy to achieve high performance for that task. For visualizing we need to reach up to 60 fps, and for most resolutions this could be done on NVIDIA GPU in realtime. We utilize our image processing SDK for the above pipeline and it can handle data rates up to 2 GPix/s on GeForce RTX 2080 for the case with built-in JPEG compression and decompression on NVIDIA GPU. Such a solution is available for all NVIDIA GPUs: mobile, laptop, desktop, server. Useful links on the subject |