Embedded devices like the Nvidia Tegra X1/2 offer tremendous video processing capabilities. But often there are bottlenecks hindering you from taking advantage of their full potential. One solution to this problem is to employ the general purpose compute capabilities of the GPU (GPGPU). For this purpose, we have developed a GStreamer Plug-In that lets you add a customized video processing functionality to any pipeline with full GPU support.
A possible application is shown in the image below. Two video inputs are combined to a single video output as a picture-in-picture video stream. A 4k image is depicted in the background and on top of it a downscaled FullHD input is streamed.
In order to cope with the huge amount of data, the video processing is outsourced to the GPU. The use of CUDA allows you to create new algorithms from scratch or integrate existing libraries. The plugin enables you to benefit of the unique architecture of the TX1/2, where CPU and GPU share access to the same memory. Therefore, memory access time is reduced and unnecessary copies are avoided. The next image shows a pipeline of the example mentioned above.
At the beginning of the pipeline, where the data rates are the highest, the GPU and internal Hardware encoders are used. The CPU can then handle the compressed data easily and gives access to the huge number of existing GStreamer Plug-Ins. For example it is capable of preparing a live video stream for clients.
The GStreamer Plug-In can also serve as a basis for other applications like format conversion, debayering or video filters.
Feel free to contact us on this topic.
The recording of the presentation held at the Gsteamer conference can be found here:
https://gstconf.ubicast.tv/videos/efficient-video-processing-on-embedded-gpu/
This can be very helpful for the user and by this, a user has the GPU plugin and make the GStreamer pipeline very strong and boost it also. By this also we can have the understanding of the 4k images and how it has been done.