{"id":1351,"date":"2020-11-04T16:29:51","date_gmt":"2020-11-04T15:29:51","guid":{"rendered":"http:\/\/blog.zhaw.ch\/splab\/?p=1351"},"modified":"2020-11-04T16:29:51","modified_gmt":"2020-11-04T15:29:51","slug":"leveraging-docker-and-reproducible-workflows-for-portable-research-environments","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/","title":{"rendered":"Leveraging Docker and reproducible workflows for portable research environments"},"content":{"rendered":"\n

Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a \u2018best practices\u2019 approach for research.<\/p>\n\n\n\n

However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are hard to replicate. In cases such as these, the added workload acts as an inhibitor on ensuring reproducibility. The result is that many experiments, data pipelines and results are hard to replicate. According to our preliminary insights, based on interviews within the ZHAW, on how experiments are conducted, such complexity is indeed commonplace in our current research activities. <\/p>\n\n\n\n\n\n\n\n

It would thus be great, if the entire complex environment could be set up within a portable container, that anyone can simply download and use with no additional steps. Enter Docker<\/a>. By no means \u2018new\u2019 technology, it has been around for several years and has been proven in many fields. Its well-documented advantages in software portability can also be leveraged to assist in creating portable and reproducible research software. A Dockerized research tool can run on any host that meets the hardware requirements with minimal setup, thus enabling a much simpler process of replicating the experiments and results of the original authors. There are already examples of leveraging Docker to improve reproducibility in research software, both internationally<\/a> and within Switzerland<\/a>. That said, eliminating the software setup is only one part of the equation.<\/p>\n\n\n\n

In addition to making the software itself portable, the entire workflow of using it can be made reproducible with novel tools that enhance reproducibility. Examples include OSF<\/a> and SDSC\u2019s Renku<\/a>. Renku is a tool designed entirely with the FAIR principles in mind, that augments the versioning functionality of git with the added benefits of a knowledge graph and other key features. Researchers can set up their repository much like with traditional git, with the added advantage of being able to record their entire workflow as they run it. This is stored in the repository\u2019s knowledge graph and can be rerun on demand when new data or code changes are committed.<\/p>\n\n\n\n

Both of these elements can be argued to provide an approach for increased reproducibility of research in isolation, nevertheless, we believe there are key ways in which they can complement each other to combat each other\u2019s limitations.<\/p>\n\n\n\n

Docker can make even some of the more complex tools reproducible, but researchers still need to learn how to operate the tool correctly to replicate a result.<\/p>\n\n\n\n

Renku can record such a workflow and \u2018replay\u2019 it on command, assigning a much easier role to the human operator, but is restricted in the complexity of tools that it can natively manage.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

We thus see the benefit in a combined approach, with the development of customized \u2018Renkurized\u2019 images that contain the necessary software stack as part of the Docker image, while also giving access to the reproducible workflow functionality and knowledge graph of Renku. Such images would also become available in a dedicated catalogue, thus making the tools easily findable and accessible to all researchers and students. Public catalogues, such as Docker Hub<\/a> are also viable and indeed there are interesting tools available there already, but we argue that to guarantee long-term availability to researchers even if the public portal were to move or change, a solution dedicated to our research is still the better choice.<\/p>\n\n\n\n

Questions to be answered include how feasible it is to transform complex research setups into these custom container images, and whether this solution can be sustainable, with researchers within the ZHAW willing to shift to this approach for reproducibility in the future. We expect to answer these questions with an extended study of individual use-cases within the ZHAW.<\/p>\n

Schlagw\u00f6rter: docker<\/a>, fair<\/a>, portability<\/a>, reproducibility<\/a>
<\/div>","protected":false},"excerpt":{"rendered":"

Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a \u2018best practices\u2019 approach for research. However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are […]<\/p>\n","protected":false},"author":438,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[4],"tags":[44,178,179,155],"features":[],"class_list":["post-1351","post","type-post","status-publish","format-standard","hentry","category-research","tag-docker","tag-fair","tag-portability","tag-reproducibility"],"yoast_head":"\nLeveraging Docker and reproducible workflows for portable research environments - Service Prototyping Lab<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Leveraging Docker and reproducible workflows for portable research environments\" \/>\n<meta property=\"og:description\" content=\"Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a \u2018best practices\u2019 approach for research. However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\" \/>\n<meta property=\"og:site_name\" content=\"Service Prototyping Lab\" \/>\n<meta property=\"article:published_time\" content=\"2020-11-04T15:29:51+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png\" \/>\n<meta name=\"author\" content=\"Panagiotis Gkikopoulos\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Panagiotis Gkikopoulos\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\"},\"author\":{\"name\":\"Panagiotis Gkikopoulos\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11\"},\"headline\":\"Leveraging Docker and reproducible workflows for portable research environments\",\"datePublished\":\"2020-11-04T15:29:51+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\"},\"wordCount\":638,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png\",\"keywords\":[\"docker\",\"fair\",\"portability\",\"reproducibility\"],\"articleSection\":[\"Research\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\",\"url\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\",\"name\":\"Leveraging Docker and reproducible workflows for portable research environments - Service Prototyping Lab\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png\",\"datePublished\":\"2020-11-04T15:29:51+00:00\",\"author\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage\",\"url\":\"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png\",\"contentUrl\":\"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/blog.zhaw.ch\/splab\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Leveraging Docker and reproducible workflows for portable research environments\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/#website\",\"url\":\"https:\/\/blog.zhaw.ch\/splab\/\",\"name\":\"Service Prototyping Lab\",\"description\":\"A Blog of the ZHAW Zurich University of Applied Sciences\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.zhaw.ch\/splab\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11\",\"name\":\"Panagiotis Gkikopoulos\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g\",\"caption\":\"Panagiotis Gkikopoulos\"},\"url\":\"https:\/\/blog.zhaw.ch\/splab\/author\/pang\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Leveraging Docker and reproducible workflows for portable research environments - Service Prototyping Lab","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/","og_locale":"en_GB","og_type":"article","og_title":"Leveraging Docker and reproducible workflows for portable research environments","og_description":"Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a \u2018best practices\u2019 approach for research. However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are […]","og_url":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/","og_site_name":"Service Prototyping Lab","article_published_time":"2020-11-04T15:29:51+00:00","og_image":[{"url":"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png","type":"","width":"","height":""}],"author":"Panagiotis Gkikopoulos","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Panagiotis Gkikopoulos","Estimated reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#article","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/"},"author":{"name":"Panagiotis Gkikopoulos","@id":"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11"},"headline":"Leveraging Docker and reproducible workflows for portable research environments","datePublished":"2020-11-04T15:29:51+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/"},"wordCount":638,"commentCount":0,"image":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png","keywords":["docker","fair","portability","reproducibility"],"articleSection":["Research"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/","url":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/","name":"Leveraging Docker and reproducible workflows for portable research environments - Service Prototyping Lab","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/splab\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage"},"image":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png","datePublished":"2020-11-04T15:29:51+00:00","author":{"@id":"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11"},"breadcrumb":{"@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#primaryimage","url":"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png","contentUrl":"https:\/\/blog.zhaw.ch\/splab\/files\/2020\/11\/container.png"},{"@type":"BreadcrumbList","@id":"https:\/\/blog.zhaw.ch\/splab\/2020\/11\/04\/leveraging-docker-and-reproducible-workflows-for-portable-research-environments\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/blog.zhaw.ch\/splab\/"},{"@type":"ListItem","position":2,"name":"Leveraging Docker and reproducible workflows for portable research environments"}]},{"@type":"WebSite","@id":"https:\/\/blog.zhaw.ch\/splab\/#website","url":"https:\/\/blog.zhaw.ch\/splab\/","name":"Service Prototyping Lab","description":"A Blog of the ZHAW Zurich University of Applied Sciences","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.zhaw.ch\/splab\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/blog.zhaw.ch\/splab\/#\/schema\/person\/6a362b2982b686a44a453206cb3cee11","name":"Panagiotis Gkikopoulos","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/88f238fea03fa1a0fe3ee06ef74194f1b2f1ebde2deecb56364cf02b2eb18ec2?s=96&d=mm&r=g","caption":"Panagiotis Gkikopoulos"},"url":"https:\/\/blog.zhaw.ch\/splab\/author\/pang\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/posts\/1351","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/users\/438"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/comments?post=1351"}],"version-history":[{"count":4,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/posts\/1351\/revisions"}],"predecessor-version":[{"id":1357,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/posts\/1351\/revisions\/1357"}],"wp:attachment":[{"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/media?parent=1351"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/categories?post=1351"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/tags?post=1351"},{"taxonomy":"features","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/splab\/wp-json\/wp\/v2\/features?post=1351"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}