|
Image Processing SDK for NVIDIA GPU
Fastvideo has designed high performance SDK for image and video processing on NVIDIA GPU. That SDK is a set of software components which correspond to standard image processing pipeline for camera applications. We cover all image processing stages starting from raw image acquisition from the camera to JPEG compression with storage to RAM or SSD. All image processing is done completely on GPU and this leads to realtime performance or even faster for the full pipeline. We can also offer high speed imaging SDK for non-camera applications on GPU: offline raw processing, web, digital cinema, video walls, video transcoding, FFmpeg codecs and filters, 3D, AR/VR, AI, etc.
Values of Fastvideo SDK
- High Performance
- Low latency
- Excellent image quality
- Reduced TCO for final applications
- Reliability
- Compatibility with third-party SDKs on CPU and GPU
- Integration with FFmpeg and UltraGrid
- Ease of maintenance, updates and upgrades
- Availability for full range of NVIDIA GPUs including Jetson
Image & Video Processing SDK Features
- Image ingestion from a camera, frame grabber, HDD/SSD/RAM or GPU memory (PGM, BMP, PPM, JPG, YCbCr, RAW, DNG, byte array, OpenGL texture or PBO, etc.)
- Raw data linearization
- Dark frame subtraction (FPN)
- Flat-Field Correction (Shading Correction)
- Dynamic Bad Pixel Correction in raw domain
- Binning (for grayscale images)
- White Balance
- RAW Histograms
- Raw Bayer Curves (master and RGB)
- Raw Denoising with wavelets
- Demosaicing with Binning (B2, B4, B8), HQLI, L7, DFPD, MG algorithms
- Highlight recovery
- Exposure correction
- Denoising: wavelet-based, NLM, bilateral
- Color Correction with matrix profile
- Color Transforms: RGB->YCbCr, YCbCr->RGB, RGB->HSV, HSV->RGB, etc.
- Composite or per-color component 8/10/12/14/16-bit 1D LUTs for tone mapping in RGB or HSV (Hue->Hue, Hue->Saturation, etc.)
- 3D LUT for RGB and HSV
- RGB Histograms
- RGB Parade (waveform monitor)
- Crop
- Rotate to 90/180/270 degrees and flip/flop
- Resize (downscale or upscale)
- Standard Remap (rotation to an arbitrary angle, affine transform, perspective transform, projection, arbitrary mapping)
- Fast Remap for undistortion applications
- Defringe
- Transforms to Rec.601 (SD), Rec.709 (HD), Rec.2020 (4K) for broadcasting, etc.
- Gamma transform (composite or per-color component)
- Sharpening with Unsharp Mask filter (USM)
- Realtime output via OpenGL
- JPEG codec (8/12 bits per channel)
- Raw Bayer codec for lossy compression or decompression (8/12 bits per channel)
- JPEG2000 encoder and decoder (8-16 bits per channel, tiles, batch, multithreading, multithreaded batch)
- Trace
- Multiplexor
- Stream-per-thread support for better performance
- NVIDIA Jetson NANO, TX2, TX2i, NX and AGX Xavier, AGX Orin support
- CUDA MPS support for Linux
- FFmpeg filters and codecs on GPU for high performance video processing
Image Processing SDK Options
- Unpacking module for various raw formats
- Image tiling
- Pyramid images
- Image split into separate planes
- Auto White Balance
- RGB to Gray transform
- MinMax
- HDR for high bit depth image sensors (automotive applications)
- Discrete Cosine Transform
- Discrete Wavelet Transforms: CDF 5/3 and 9/7, Haar, UHaar, UCDF
- Time measurements for all SDK modules
- Encoding and decoding for BMD RAW 3:1, 4:1 and 5:1 formats
- Support for input data from SD/HD-SDI and 3G/6G/12G-SDI grabbers: Blackmagic, Bluefish, Deltacast, Imperx
- Multiple camera solutions for realtime image processing
- H.264 codeс (NVENC and NVDEC)
- H.265 codeс (NVENC and NVDEC)
- RAW and DNG (CinemaDNG) image and video processing on GPU
- MLV raw video processing on GPU
- OME-TIFF, BigTIFF, GeoTIFF support (tiles, pyramids, encoding)
- J2K codec on GPU for FFmpeg
We are developing and optimizing software for the latest NVIDIA GPUs as well and we are designing new image and video processing algorithms to get high image quality and realtime performance. Parallel computing is the key to achieve realtime performance for imaging applications.
Image Processing Benchmarks: 2K vs 4K
*OS Windows-10 (64-bit), CUDA-10 (64-bit). Timing includes GPU computations only, single frame image processing at one GPU (no batch mode, no streaming). JPEG encoding with compression ratio ~10:1 (24-bit data, jpeg quality 90%) and subsampling 4:2:0. Debayer algorithm is DFPD. Resizer algorithm is Lanczos (2x downscaling). Denoising with CDF97 wavelet. Here you can find other benchmarks for our SDK.
Compatibility
- OS Windows-10, Linux Ubuntu/CentOS, Linux4Tegra (L4T)
- CUDA-12.3, MSVS 2022, sample C++ applications with source code, video processing API, manual
- NVIDIA GPUs (Maxwell, Pascal, Volta, Turing, Ampere, Ada) for mobile, laptop, desktop, server
- NVIDIA Jetson Nano, TX2, NX, AGX Xavier, Orin NX/AGX
- Motion JPEG integration with FFmpeg (reader/writer), FFmpeg is under LGPLv2.1
SDK sample applications with source codes
- CUDA JPEG Codec
- Debayer
- Debayer + JPEG
- Raw Bayer Codec
- Denoiser
- Resizer
- J2K Codec
- SDI converter
- JPEG Resize (Jpeg2Jpeg app): jpeg decoding, crop, resize, sharp, jpeg encoding
- Camera application on C++ and CUDA: raw data acquisition and unpacking, dark frame subtraction, shading correction, white balance, exposure correction, debayer, luma and chroma denoising, color correction, tone mapping with LUT (curves and levels), rotate, crop, resize, remap, undistortion, sharp, OpenGL output, JPEG or JPEG2000 compression, writer to AVI (MJPEG) or MXF (JPEG2000)
- AVI player (MJPEG) via GLFW
- CUDA Streams sample application
- FFmpeg Remap filter
- FFmpeg J2K Decoder
Open source projects from Fastvideo on GitHub
Download our GPU-based demo applications
Custom GPU Accelerated Image & Video Processing SDK
We can offer services to integrate your software with our image and video processing SDK. To achieve high performance on GPU for specific imaging applications, Fastvideo is also developing custom image and video processing solutions according to agreed specification. If you can't find at that document your anticipated image processing pipeline, please call us to discuss that possibility. You can also have a look at our projects in development. We are ready to accelerate your video processing application.
Licensing for CUDA Image & Video Processing SDK
We license GPU Image & Video Processing SDK and its components for Windows, Linux and L4T to software developers, camera manufacturers and resellers, internet providers, software integrators, startups, etc. That SDK is utilized in wide range of realtime imaging applications. Demo version of SDK, documentation, licensing info and quotation are available upon request. We are also offering custom software design according to agreed specification. If you need to get significant speedup or to improve quality of your image processing application, don't hesitate to contact us.
Roadmap for CUDA Image Processing SDK
- Fast VCR software for XIMEA cameras (realtime raw image processing on Jetson with integrated camera control) - done
- Highlights recovery - done
- HDR processing - done
- CUDA-12.3 support - done
- Interoperability with neural network libraries and applications - done
- High performance NLM denoiser on the GPU - done
- High quality colour calibration software - done
- CUDA-12.2 support for Jetson Orin - in progress
- Accelerated Remap - in progress
- GPU memory manager - in progress
|