FastVCR software on GPU for Hikrobot cameras

HikRobot is a global product and solution provider specializing in machine vision and mobile robots. The machine vision products are widely used in industrial automation sectors such as consumer electronics, semiconductors and logistics, as a part of the vision applications like positioning guidance, measurement, quality inspection, code reading, OCR, etc. They help users to greatly improve productivity, accuracy and stability.

FastVCR Software Hikrobot

You can test any Hikrobot camera with their original MVS software, but it's not intended for high fps applications. To overcome that limitation we've developed a GPU-based software to work with Hikrobot cameras to allow maximum performance and the highest image quality at realtime. You can download demo software from that link and test it with available RAW/PGM frames from Hikrobot cameras (8/10/12-bit raw monochrome or bayer).

Why do we need high performance software for Hikrobot cameras?

If we have a look at the latest SONY Pregius image sensors of the 4th generation, most of them have "high speed" version which means that these image sensors have not only high resolution, but also high frame rate, which is doubled in comparison with the second generation on SONY image sensors. Hikrobot cameras with these image sensors are supplied with high speed USB3 or GigE interfaces, so they have capabilities to capture acquired frames at the maximum bandwidth.

That's true not only for SONY image sensors, the situation is the same with Gpixel and AMS (CMOSIS) as well. Modern machine vision cameras offer both high resolution and high fps, so it's quite a complicated task to perform real time processing with high quality and without dropped frames. That is exactly what we can do with FastVCR software which is running on NVIDIA GPU (mobile/laptop/desktop/server). In our pipeline we've included high quality demosaicing together with other image processing modules to take into account color science and high performance requirements.

Machine vision cameras capture high quality images at realtime, so they are utilized in industrial and medical applications, streaming and broadcasting, digital cinema, 3D, AR and VR.

FastVCR software for Hikrobot camera control

  • Camera initialization (camera model, default values for frame resolution, fps, etc.)
  • Input bit depth and pixel format
  • Image resolution and region of interest (ROI)
  • Frame rate (FPS)
  • Exposure time
  • Gain
  • Camera bandwidth
  • Automatic exposure/gain control with "remove overexposure" feature
  • Software presets for fast change of image sensor parameters
  • Optional modes: binning, decimation, dual gain, dual adc, etc.

FastVCR software: image processing pipeline on GPU

  • Image acquisition
  • Frame unpacking for 10-bit and 12-bit modes
  • Image linearization
  • Dark frame subtraction (FPN)
  • Flat-Field Correction (shading correction)
  • Bad pixel removal
  • White Balance / AWB
  • Adaptive Exposure and Gain control
  • High quality demosaicing with MG algorithm
  • Color correction with matrix profile or DCP profile
  • Highlight recovery
  • Exposure correction (brightness control)
  • Denoisers: wavelet-based, Bilateral, NLM
  • Curves and Levels
  • Rotation to 90/180/270 degrees and flip/flop
  • Crop
  • Resize (downscale and upscale)
  • Rotation to an arbitrary angle
  • Undistortion via LCP or via calibrated maps
  • Sharpening (local contrast)
  • Gamma transform
  • JPEG compression and storage on SSD
  • Optional conversion to NV12 and h264/h265/av1 encoding
  • Automatic realtime partitioning of AVI/MP4 video files to the specified file size
  • Built-in RTSP server for low latency video streaming and broadcasting
  • Realtime output to monitor

FastVCR software outputs

  • Video output to monitor via OpenGL in real time
  • Camera statistics
  • Histogram, Parade, Vectorscope
  • Realtime processing and JPEG compression with image storage on SSD
  • Video encoding to MJPEG (AVI), H.264/H.265/AV1 (MP4) and storage to video container on SSD
  • Low-latency video streaming via RTSP (both player and server are included)
  • RTSP streaming is compatible with VLC and with our own video player
  • Real time RAW/PGM data storage on SSD
  • Interoperability with third-party GPU-based SDKs
  • Interoperability with GPU-based AI libraries and applications at the GPU level

FastVCR performance benchmarks

That software was built with the idea of GPU image processing to be capable of working in high performance camera applications. This is exactly what Hikrobot is offering for their products with USB3 and GigE interfaces. The software can work with multicamera systems to process raw frames from all cameras in real time without frame drops. Still, the maximum performance could be achieved with the fastest 10-GigE cameras due to their bandwidth. In general, the performance depends on the complexity of image processing pipeline and utilized hardware (CPU/GPU/SSD, etc). You can download the demo software from the link below to test your Hikrobot camera and to evaluate image quality, latency and performance.

That software can work at camera emulator mode to check image processing performance on GPU without having a camera connected to a PC. User can load arbitrary RAW image in RAW/PGM/TIFF format to work with. This is very useful to check time measurements which are available for each image processing module on GPU.

Optional image processing modules on GPU

  • RAW capture, preview and storage to SSD
  • Histograms for RAW and processed data
  • Curves and levels for RAW, RGB, HSV
  • DCP support and calibration (CCM, HSV LUT, Tone Curve)
  • 3D LUT support
  • Geometric and affine transforms
  • Undistortion via accelerated remap - in progress
  • Chromatic aberration suppression at RAW domain - in progress
  • LCP support
  • HDR support
  • Local and global tonemapping
  • SDI output support to work with Blackmagic grabbers
  • Multiple camera support
  • Fast replay (synchronized viewer) solution for multicamera systems
  • High performance JPEG2000 and Raw-JPEG codecs
  • Interoperability with FFmpeg
  • Camera and lens calibration: FPN, FFC, DCP, undistortion
  • GenICam (GenTL) support to work with Basler, Imperx, Flir, Lucid, XIMEA, Emergent Vision, IO Industries, Daheng Imaging, Mindvision and other cameras
  • Much more...

FastVCR command-line application

Quite often we need to run a software without any GUI, and it could happen in different situations. This is the case for drone applications or for any other task with remote camera control. This is also the case for any long-term unsupervised video recording and streaming.

To meet these requirements, we've developed a CLI application which has all the above features of FastVCR software and it could work without GUI. We are still able to have full control on image sensor and image processing parameters at real time. For video preview we supply our own player with RTSP client or one can use VLC instead. The software is compatible with Windows/Linux/L4T and all image processing is done on the GPU.

Why could we need FastVCR software instead of MVS?

Here we point out the most important features which are essential for camera applications.


1. Fast RAW image processing on the GPU
Performance of FastVCR software is limited by utilized GPU and by complexity of the utilized ISP.
FastVCR has 16/32-bit pipeline on the GPU with high quality demosaicing.

2. Fast RGB encoding
FastVCR software can encode RGB frames to MJPEG at almost any fps and resolution, which could be more than the max bandwidth of H.264/H.265 in terms of frame rate and image resolution. We use MJPEG codec on GPU to achieve high frame rates.

3. Testing
FastVCR software can work not only with cameras, but also with RAW/PGM/TIFF frames from SSD, which is useful for testing if the customer doesn't have a camera at the moment. FastVCR can work with folders of images to emulate camera by processing RAW frames consequtively.

4. Denoising
FastVCR software has bilateral, wavelet and NLM denoisers for monochrome and color images.

5. Low latency solution
Due to fast RAW processing we can achieve low latency with fast data transfer via RTSP.
The software has built-in G2G (glass-to-glass) test to measure the latency.

6. CLI application
We supply both GUI and CLI applications, plus RTSP server and player.
CLI application is important for situations when there is no GUI. That app could be controlled remotely.

7. Realtime historgam, parade, vectorscope monitor.

8. Detailed statistics about camera operation at realtime.

cuda isp sdk

All the above features are designed on the basis of our CUDA image processing SDK. Now our customers have an opportunity to utilize GPU-accelerated software components in their applications as a part of video processing pipeline. Here you can find other benchmarks for our SDK.

Apart from FastVCR software, we are offering custom software design to build a solution according to customers' requirements. We can implement application-specific modes of operation to solve the task you need. Please fill the contact form below with your project info and send it to us.

Compatibility

  • CUDA-12.6 for Windows/Linux, NVIDIA GeForce, Quadro, Tesla
  • CUDA-11.4 for NVIDIA Jetson NX, AGX Xavier, Orin
  • CUDA-10.2 for NVIDIA Jetson TX2, NX, AGX Xavier
  • Hikrobot USB3 and GigE cameras
  • CUDA processing of 8-16-bit RAW/PGM/TIFF images from SSD

Software downloads

Applications

  • Ophthalmology and retinal surgery, medical imaging
  • Machine vision and industrial applications
  • Media and entertainment
  • Broadcasting and streaming
  • Drones and UAVs
  • 3D scanning and robotic arms
  • Aerial imaging
  • Various AI applications with machine vision cameras

FastVCR roadmap

  • NLM denoiser on the GPU - done
  • Bilateral denoiser on the GPU - done
  • High quality color calibration for camera applications - done
  • CUDA-12.2 support for Jetson Orin - in progress
  • Accelerated GPU kernels to get better performance on Jetson TX2 and NX - in progress
  • Compatibility with YOLO - in progress
  • Lossy RAW compression and decompression in real time - soon

Contact Form

This form collects your name and email. Check out our Privacy Policy on how we protect and manage your personal data.