FFmpeg Remap Filter on GPURemap filter is intended to copy a source image to a target image according to two maps (ymap/xmap) which are usually supplied in two files. Map files are passed as a parameters and they are usually in binary PGM format, where the values are y(rows) and x(columns) coordinates of the source frame. The target image dimension is based on mapfile dimensions which are specified in the header of the mapfile. Dimensions of ymap and xmap must be equal, data values must be positive or zero. These coordinates are in floating point format, so fraction pixel positions can be used. 
Remap chooses source pixels by using pixel coordinates explicitely supplied in two 2D device memory image arrays pointed to by the pXMap and pYMap pointers. The pXMap array contains the X coordinate and the pYMap array contains the Y coordinate of the corresponding source image pixel to use as input. In a standard FFmpeg we can find several filters for remapping: v360 (11.213) and remap (11.164). Usually FFmpeg Remap Filter on CPU is quite slow an it could be easily checked by benchmarking of these filters. Nevertheless, remap filter could be performed much faster on GPU. We've implemented that filter on NVIDIA GPU with Fastvideo SDK and it's working very fast even on laptop GPUs. On professional GPUs we can decode, remap and encode multiple video streams in realtime with high performance and minimum latency. How to remap video on GPU via FFmpegThe task of video remapping on GPU is quite standard and it could be utilized to create 360 movies with equirectangular projection. We can run our GPU-based FFmpeg filter to remap video file or to apply it to available video stream, which is also possible due to high performance of that algorithm on GPU. Below you can see pipeline description for such a task of video file remapping.
Example command for FFmpeg Remap FilterCommand line below shows how to run FFmpeg Remap filter on GPU for h264-encoded video in mp4 container: ffmpeg -y -vsync 0 -hwaccel cuvid -c:v h264_cuvid -i source.mp4 -vf remap_fastvideo=mapX=x.pgm:mapY=y.pgm -c:a copy -c:v h264_nvenc -b:v 5M target.remap.mp4 Why we can remap mp4 much faster than standard FFmpeg?
It's worth mentioning that any stage of the above image processing pipeline on CPU instead of GPU will bring us slower results and it makes sense to implement full pipeline on GPU. That's exactly what we've done to speedup FFmpeg Remap Filter on GPU. Benchmarks for FFmpeg Remap Filter on GPUWe've done several tests for remapping of live video which was originally compressed with h.264 (mp4 format). For a video with frame resolution around 5 MPix we've got the performance 160 fps on NVIDIA GeForce RTX 2070S GPU, though on Intel Core i9-9960X CPU we've got much less for the same test. This is just an example to show how fast FFmpeg could be for video decoding, remapping and encoding on desktop NVIDIA GPU. We can actually do the same on any NVIDIA GPU: mobile, laptop, desktop, server and for multiple GPU setup. As soon as NVENC and NVDEC are hardware-based encoder and decoder, then video encoding and decoding are performed on NVIDIA GPU in parallel with CUDA code. This is very important to get additional acceleration for GPU-based remap filter. Actually, this is the way to create even more complicated remap filter by adding additional image processing features on GPU to the pipeline. We would suggest to try Fastvideo SDK with sample application for FFmpeg Remap Filter to check the performance and quality for your video on your GPU. That filter is available for Windows, Linux and L4T. We've also implemented GPU-based remap feature at Fast CinemaDNG Processor software to solve various tasks of lens undistortion and image rotation to an arbitrary angle for digital cinema applications. In that software there is a benchmark window and user can see how much time it's necessary for particular Remap transform on CUDA. Applications for FFmpeg RemapFilter
Standard Remap ProcedureTo apply FFmpeg Remap filter, user needs to create two maps first. This is usually done via calibration.
Other GPU software for FFmpeg
|