{"id":122,"date":"2017-10-24T16:31:34","date_gmt":"2017-10-24T14:31:34","guid":{"rendered":"http:\/\/blog.zhaw.ch\/high-performance\/?p=122"},"modified":"2017-10-24T19:30:57","modified_gmt":"2017-10-24T17:30:57","slug":"boost-your-gstreamer-pipeline-with-the-gpu-plugin","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/","title":{"rendered":"Boost your GStreamer pipeline with the GPU plugin"},"content":{"rendered":"<p>Embedded devices like the Nvidia Tegra X1\/2 offer tremendous video processing capabilities. But often there are bottlenecks hindering you from taking advantage of their full potential. One solution to this problem is to employ the general purpose compute capabilities of the GPU (GPGPU). For this purpose, we have developed a GStreamer Plug-In that lets you add a customized video processing functionality to any pipeline with full GPU support.<\/p>\n<p>A possible application is shown in the image below. Two video inputs are combined to a single video output as a picture-in-picture video stream. A 4k image is depicted in the background and on top of it a downscaled FullHD input is streamed.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-123\" src=\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\" alt=\"\" width=\"888\" height=\"343\" srcset=\"https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png 300w, https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-768x298.png 768w, https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-1024x397.png 1024w, https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-676x262.png 676w\" sizes=\"auto, (max-width: 888px) 100vw, 888px\" \/><\/p>\n<p>In order to cope with the huge amount of data, the video processing is outsourced to the GPU. The use of CUDA allows you to create new algorithms from scratch or integrate existing libraries.&nbsp;The plugin enables you to benefit of the unique architecture of the TX1\/2, where CPU and GPU share access to the same memory. Therefore, memory access time is reduced&nbsp; and unnecessary copies are avoided.&nbsp;The next image shows a pipeline of the example mentioned above.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-124\" src=\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/pipeline-300x77.png\" alt=\"\" width=\"892\" height=\"229\" srcset=\"https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/pipeline-300x77.png 300w, https:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/pipeline-768x196.png 768w\" sizes=\"auto, (max-width: 892px) 100vw, 892px\" \/><\/p>\n<p>At the beginning of the pipeline, where the data rates are the highest, the GPU and internal Hardware encoders are used.&nbsp;The CPU can then handle the compressed data easily and gives access to the huge number of existing GStreamer Plug-Ins. For example it is capable of preparing a live video stream for clients.<\/p>\n<p>The GStreamer Plug-In can also serve as a basis for other applications like format conversion, debayering or video filters.<\/p>\n<p>Feel free to contact us on this topic.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Embedded devices like the Nvidia Tegra X1\/2 offer tremendous video processing capabilities. But often there are bottlenecks hindering you from taking advantage of their full potential. One solution to this problem is to employ the general purpose compute capabilities of the GPU (GPGPU). For this purpose, we have developed a GStreamer Plug-In that lets you [&hellip;]<\/p>\n","protected":false},"author":270,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1,2],"tags":[],"features":[],"class_list":["post-122","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-hdmi2csi"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Boost your GStreamer pipeline with the GPU plugin - Embedded High Performance Multimedia Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Boost your GStreamer pipeline with the GPU plugin\" \/>\n<meta property=\"og:description\" content=\"Embedded devices like the Nvidia Tegra X1\/2 offer tremendous video processing capabilities. But often there are bottlenecks hindering you from taking advantage of their full potential. One solution to this problem is to employ the general purpose compute capabilities of the GPU (GPGPU). For this purpose, we have developed a GStreamer Plug-In that lets you [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\" \/>\n<meta property=\"og:site_name\" content=\"Embedded High Performance Multimedia Blog\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-24T14:31:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-10-24T17:30:57+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\" \/>\n<meta name=\"author\" content=\"rosn\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"rosn\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\"},\"author\":{\"name\":\"rosn\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9\"},\"headline\":\"Boost your GStreamer pipeline with the GPU plugin\",\"datePublished\":\"2017-10-24T14:31:34+00:00\",\"dateModified\":\"2017-10-24T17:30:57+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\"},\"wordCount\":283,\"commentCount\":2,\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\",\"articleSection\":[\"Allgemein\",\"HDMI2CSI\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\",\"url\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\",\"name\":\"Boost your GStreamer pipeline with the GPU plugin - Embedded High Performance Multimedia Blog\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\",\"datePublished\":\"2017-10-24T14:31:34+00:00\",\"dateModified\":\"2017-10-24T17:30:57+00:00\",\"author\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage\",\"url\":\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\",\"contentUrl\":\"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/blog.zhaw.ch\/high-performance\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Boost your GStreamer pipeline with the GPU plugin\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/#website\",\"url\":\"https:\/\/blog.zhaw.ch\/high-performance\/\",\"name\":\"Embedded High Performance Multimedia Blog\",\"description\":\"A Blog of the ZHAW Zurich University of Applied Sciences\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.zhaw.ch\/high-performance\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9\",\"name\":\"rosn\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g\",\"caption\":\"rosn\"},\"url\":\"https:\/\/blog.zhaw.ch\/high-performance\/author\/rosn\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Boost your GStreamer pipeline with the GPU plugin - Embedded High Performance Multimedia Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/","og_locale":"en_GB","og_type":"article","og_title":"Boost your GStreamer pipeline with the GPU plugin","og_description":"Embedded devices like the Nvidia Tegra X1\/2 offer tremendous video processing capabilities. But often there are bottlenecks hindering you from taking advantage of their full potential. One solution to this problem is to employ the general purpose compute capabilities of the GPU (GPGPU). For this purpose, we have developed a GStreamer Plug-In that lets you [&hellip;]","og_url":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/","og_site_name":"Embedded High Performance Multimedia Blog","article_published_time":"2017-10-24T14:31:34+00:00","article_modified_time":"2017-10-24T17:30:57+00:00","og_image":[{"url":"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png","type":"","width":"","height":""}],"author":"rosn","twitter_card":"summary_large_image","twitter_misc":{"Written by":"rosn","Estimated reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#article","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/"},"author":{"name":"rosn","@id":"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9"},"headline":"Boost your GStreamer pipeline with the GPU plugin","datePublished":"2017-10-24T14:31:34+00:00","dateModified":"2017-10-24T17:30:57+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/"},"wordCount":283,"commentCount":2,"image":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png","articleSection":["Allgemein","HDMI2CSI"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/","url":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/","name":"Boost your GStreamer pipeline with the GPU plugin - Embedded High Performance Multimedia Blog","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage"},"image":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png","datePublished":"2017-10-24T14:31:34+00:00","dateModified":"2017-10-24T17:30:57+00:00","author":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9"},"breadcrumb":{"@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#primaryimage","url":"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png","contentUrl":"http:\/\/blog.zhaw.ch\/high-performance\/files\/2017\/10\/PipCase-300x116.png"},{"@type":"BreadcrumbList","@id":"https:\/\/blog.zhaw.ch\/high-performance\/2017\/10\/24\/boost-your-gstreamer-pipeline-with-the-gpu-plugin\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/blog.zhaw.ch\/high-performance\/"},{"@type":"ListItem","position":2,"name":"Boost your GStreamer pipeline with the GPU plugin"}]},{"@type":"WebSite","@id":"https:\/\/blog.zhaw.ch\/high-performance\/#website","url":"https:\/\/blog.zhaw.ch\/high-performance\/","name":"Embedded High Performance Multimedia Blog","description":"A Blog of the ZHAW Zurich University of Applied Sciences","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.zhaw.ch\/high-performance\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blog.zhaw.ch\/high-performance\/#\/schema\/person\/e1a77329f74257615afa71bc883106c9","name":"rosn","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/eec853590b797a7cdaea9a010f9e4424a21362148edf1b8d8d51fc8c5bced337?s=96&d=mm&r=g","caption":"rosn"},"url":"https:\/\/blog.zhaw.ch\/high-performance\/author\/rosn\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/posts\/122","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/users\/270"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/comments?post=122"}],"version-history":[{"count":2,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/posts\/122\/revisions"}],"predecessor-version":[{"id":126,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/posts\/122\/revisions\/126"}],"wp:attachment":[{"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/media?parent=122"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/categories?post=122"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/tags?post=122"},{"taxonomy":"features","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/high-performance\/wp-json\/wp\/v2\/features?post=122"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}