Bayer codec on CUDA

Performance of image processing in camera applications could be much higher is we apply image compression for RAW data without using debayer. That approach is usually called Raw Bayer Compression. Right after image acquisition (after some preprocessing) we can create four planes according to colors of Bayer pattern (RGGB). Then we can do JPEG/JPEG2000 data compression for each plane with high quality. It could be done very fast and without introducing significant image artifacts. In this way we can temporary exclude debayer from image processing pipeline and increase performance for realtime applications.

This is the main idea - to store RAW image for visually lossless encoding. This is the way to preserve original data for further white balance, exposure correction, debayering and color grading in post-production.

In the task of realtime data acquisition one should set JPEG quality at least 90% or more together with subsampling 4:4:4. This is so called visually lossless compression. JPEG quality 100% or lossy JPEG2000 are preferrable, but in that case compression ratio and performance could be not high enough. Right choice of compression quality is very important, because strong JPEG/JPEG2000 compression can introduce additional image artifacts, which will be boosted further, at debayer stage.

This is pipeline description for image acquisition stage:

  • Image acquisition and unpacking of raw data
  • Image preprocessing: dark frame (black offset) and FFC
  • Decomposition and alignment of real bayer pattern to 4 separate color planes
  • JPEG compression (quality >90%, subsampling 4:4:4), optionally JPEG2000 (CDF 5/3 or 9/7)
  • Data storage to SSD/HDD/RAID

After image acquisition we finally get compressed raw data at hdd/ssd. Here comes a task of image processing and visualizing. To accomplish that task, we need to do the following:

  • Bayer image decompression (JPEG decoding)
  • Image composition with real bayer pattern from 4 color planes
  • Debayer
  • Denoiser
  • Color correction
  • LUT (gamma)
  • Crop/Rotate/Resize/Sharp
  • OpenGL output
  • Optional output MJPEG compression to AVI

The above pipeline is computationally demanding and it's not easy to achieve high performance for that task. For visualizing we need to get up to 60 fps and for most resolutions this could be done on CUDA in realtime. We utilize our image processing SDK for the above pipeline and it can handle data rates up to 2 GPix/s on GeForce GTX 980 or Titan for the case with built-in CUDA JPEG compression and decompression.

     Home                   Contacts                 Site Map
GPU Image Processing