GPU Image Processing for XIMEA xiQ cameras with USB-3.0 interface

There are quite a lot of image sensors with high resolution and high speed capabilities at the same time. All of them can generate significant amounts of raw data and it's a complicated problem how to perform image processing and save recorded data in realtime. One can choose simple solution - just use fast SSD drive or RAID to record and save all raw data. In that approach one could quickly ran out of disk space and you will need much time to do image processing to get high quality images from raw data.

Image sensors of color cameras are covered with Bayer filter arrays to recover color information later at post processing stage. So after demosaicing (debayering) the amount of data will be tripled. Trying to do that even on the fastest CPU we will see that performance for image processing would be far from realtime due to high frame rate or high resolution of image sensors. Below we list some frequently used image sensors to deal with:

CMOSIS image sensors

• CMV300: 640×480, 500 fps (8-bit), 250 fps (10-bit)
• CMV2000: 2048×1088, 340 fps (10-bit), 70 fps (12-bit)
• CMV4000: 2048×2048, 180 fps (10-bit), 37 fps (12-bit)

Data path for realtime image processing on GPU

• Driver gets raw data from the camera
• Driver writes data to CPU memory buffer
• Application sends raw data from CPU to GPU
• Unpacking module for various camera formats
• Dark image subtraction
• Flat-Field Correction
• White Balance
• Image demosaicing
• Base Color Correction
• Denoising
• Curves and Levels with 1D LUT
• Data transform with 3D LUT
• Crop/Rotate/Resize/Remap/Sharp
• Gamma
• OpenGL output to monitor
• JPEG/JPEG2000 compression
• MJPEG/MJ2K stream storage to SSD/HDD/RAID in AVI/MXF

Advantages of GPU image processing for xiQ cameras

• Per-pixel offset and shading correction
• High quality demosaicing
• Low latency output to monitor via OpenGL
• Fast image processing for the whole pipeline
• Realtime image processing for multiple camera systems
• Low CPU usage