{"id":3076,"date":"2013-08-02T10:54:12","date_gmt":"2013-08-02T10:54:12","guid":{"rendered":"http:\/\/www.cloudcomp.ch\/?p=3076"},"modified":"2013-08-02T10:54:12","modified_gmt":"2013-08-02T10:54:12","slug":"distributed-file-system-series-glusterfs-introduction","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/","title":{"rendered":"Distributed File System Series: GlusterFS Introduction"},"content":{"rendered":"<p>In the <a title=\"Distributed File Systems Series: Ceph Introduction\" href=\"http:\/\/www.cloudcomp.ch\/2013\/07\/distributed-file-systems-series-ceph-introduction\/\">first part<\/a> of the Distributed File Systems Series we gave an introduction to Ceph. Today we will explore another file system: GlusterFS.<\/p>\n<p><a href=\"http:\/\/www.gluster.org\/\">GlusterFS<\/a>\u00a0is a clustered network filesystem\u00a0that uses FUSE and that allows to aggregate different storage devices, <em>bricks<\/em> in GlusterFS terminology, into a single storage pool. The storage in each brick is formatted using a local file system, e.g. XFS or ext4, and then exposed to GlusterFS for storing data.\u00a0The user space approach gives GlusterFS flexible release cycles and easiness of use without taking tolls on the performance of the file system.<\/p>\n<p>Two of the main concepts of GlusterFS are <em>volumes<\/em> and <em>translators<\/em>.<\/p>\n<p>A volume is a collection of one or more bricks.\u00a0There are three different type of volumes in GlusterFS that differentiate in how the volume stores the data in the bricks:<\/p>\n<ul>\n<li>Distribute Volume.\u00a0In this type of volume all the data is distributed throughout all the bricks. The data distribution is based on an algorithm that takes into account the size available in each brick. This is the default volume type.<\/li>\n<li>Replicate Volume.\u00a0In a replicate volume the data is duplicated, hence replicate, over every brick in the volume. The number of bricks must be a multiple of the replica count.<\/li>\n<li>Stripe Volume.\u00a0In a stripe volume the data is striped into units of a given size among the bricks. The default unit size is 128KB and the number of bricks should be a multiple of the stripe count.<\/li>\n<\/ul>\n<p>A translator is a GlusterFS component\u00a0that has a very specific function; some of the main functionalities of GlusterFS are implemented as translators, e.g. I\/O scheduling, striping and replication, load balancing, failover of volumes and\u00a0disk caching. A translator connects to one or more volumes, perform it specific function and offers a volume connection; therefore translators can be hooked together to provide a file system tailored to certain needs. The whole set of translators linked together is called a <em>graph<\/em>.<\/p>\n<figure id=\"attachment_3146\" aria-describedby=\"caption-attachment-3146\" style=\"width: 486px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-3146 \" alt=\"\" src=\"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\" width=\"486\" height=\"214\" srcset=\"https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png 486w, https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph-300x132.png 300w\" sizes=\"auto, (max-width: 486px) 100vw, 486px\" \/><figcaption id=\"caption-attachment-3146\" class=\"wp-caption-text\"><strong>GlusterFS Translators graph (Source: <a href=\"http:\/\/www.gluster.org\/community\/documentation\/index.php\/GlusterFS_Concepts\">gluster.org<\/a>)<\/strong><\/figcaption><\/figure>\n<p>GlusterFS offers a fairly long list of translators for different needs:<\/p>\n<ul>\n<li>Storage Translators. These translators define the behaviour of the back-end storage for GlusterFS. A storage translator is typically the first translator in a chain.\n<ul>\n<li>POSIX: tells GlusterFS to use a normal POSIX file system as the backend, e.g. ext4.<\/li>\n<li>BDB: tells GlusterFS to use the Berkeley DB as the backend storage mechanism. This translators uses key-value pairs to store data and POSIX directories to store directories.<\/li>\n<\/ul>\n<\/li>\n<li>Clustering Translators. These translators are used to allow GlusterFS to use multiple servers to create a cluster. These translators are used to define the basic behaviour of GlusterFS.\n<ul>\n<li>Unify: with this translator all the subvolumes from the storage servers will appear as a single volume. An important feature of this translator is that a given file can exist on only one of the subvolumes in the cluster. The translator uses a scheduler to determine where a file resides, e.g. Adaptive Least Usage, Round Robin, random.<\/li>\n<li>Distribute: this translator aggregate storage from several storage servers.<\/li>\n<li>Replicate: this scheduler replicates files and directories across the subvolumes. If there are two subvolumes then a copy of each file will be on each subvolume.<\/li>\n<li>Stripe: with this translator the content of a file is distributed across subvolumes.<\/li>\n<\/ul>\n<\/li>\n<li>Performance Translators. These translator are used to improve he performance of GlusterFS.\n<ul>\n<li>Read ahead: this translator pre-fetches data before it&#8217;s required, usually data that appears next in the file.<\/li>\n<li>Write behind: this allows the write operation to return even if the operation has not completed.<\/li>\n<li>Booster: using this translator applications are allowed to skip using FUSE and access the GlusterFS directly.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>More translators can be written according to specific requirements.<\/p>\n<p>For the next part of the Distributed File Systems Series we will be looking at XtreemFS.<\/p>\n<div class=\"pt-sm\">Schlagw\u00f6rter: <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/distributed-file-systems-2\/\">Distributed File systems<\/a>, <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/glusterfs\/\">GlusterFS<\/a><br><\/div>","protected":false},"excerpt":{"rendered":"<p>In the first part of the Distributed File Systems Series we gave an introduction to Ceph. Today we will explore another file system: GlusterFS. GlusterFS\u00a0is a clustered network filesystem\u00a0that uses FUSE and that allows to aggregate different storage devices, bricks in GlusterFS terminology, into a single storage pool. The storage in each brick is formatted [&hellip;]<\/p>\n","protected":false},"author":73,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[5,20],"tags":[115,164],"features":[],"class_list":["post-3076","post","type-post","status-publish","format-standard","hentry","category-articles","category-open-source","tag-distributed-file-systems-2","tag-glusterfs"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Distributed File System Series: GlusterFS Introduction - Service Engineering (ICCLab &amp; SPLab)<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Distributed File System Series: GlusterFS Introduction\" \/>\n<meta property=\"og:description\" content=\"In the first part of the Distributed File Systems Series we gave an introduction to Ceph. Today we will explore another file system: GlusterFS. GlusterFS\u00a0is a clustered network filesystem\u00a0that uses FUSE and that allows to aggregate different storage devices, bricks in GlusterFS terminology, into a single storage pool. The storage in each brick is formatted [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\" \/>\n<meta property=\"og:site_name\" content=\"Service Engineering (ICCLab &amp; SPLab)\" \/>\n<meta property=\"article:published_time\" content=\"2013-08-02T10:54:12+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\" \/>\n<meta name=\"author\" content=\"strp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"strp\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\"},\"author\":{\"name\":\"strp\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a\"},\"headline\":\"Distributed File System Series: GlusterFS Introduction\",\"datePublished\":\"2013-08-02T10:54:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\"},\"wordCount\":648,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\",\"keywords\":[\"Distributed File systems\",\"GlusterFS\"],\"articleSection\":[\"Articles\",\"Open Source\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\",\"name\":\"Distributed File System Series: GlusterFS Introduction - Service Engineering (ICCLab &amp; SPLab)\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\",\"datePublished\":\"2013-08-02T10:54:12+00:00\",\"author\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\",\"contentUrl\":\"https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png\",\"width\":486,\"height\":214},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/blog.zhaw.ch\/icclab\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Distributed File System Series: GlusterFS Introduction\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/\",\"name\":\"Service Engineering (ICCLab &amp; SPLab)\",\"description\":\"A Blog of the ZHAW Zurich University of Applied Sciences\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a\",\"name\":\"strp\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g\",\"caption\":\"strp\"},\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/author\/strp\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Distributed File System Series: GlusterFS Introduction - Service Engineering (ICCLab &amp; SPLab)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/","og_locale":"en_US","og_type":"article","og_title":"Distributed File System Series: GlusterFS Introduction","og_description":"In the first part of the Distributed File Systems Series we gave an introduction to Ceph. Today we will explore another file system: GlusterFS. GlusterFS\u00a0is a clustered network filesystem\u00a0that uses FUSE and that allows to aggregate different storage devices, bricks in GlusterFS terminology, into a single storage pool. The storage in each brick is formatted [&hellip;]","og_url":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/","og_site_name":"Service Engineering (ICCLab &amp; SPLab)","article_published_time":"2013-08-02T10:54:12+00:00","og_image":[{"url":"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png","type":"","width":"","height":""}],"author":"strp","twitter_card":"summary_large_image","twitter_misc":{"Written by":"strp","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#article","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/"},"author":{"name":"strp","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a"},"headline":"Distributed File System Series: GlusterFS Introduction","datePublished":"2013-08-02T10:54:12+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/"},"wordCount":648,"commentCount":0,"image":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png","keywords":["Distributed File systems","GlusterFS"],"articleSection":["Articles","Open Source"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/","url":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/","name":"Distributed File System Series: GlusterFS Introduction - Service Engineering (ICCLab &amp; SPLab)","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage"},"image":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png","datePublished":"2013-08-02T10:54:12+00:00","author":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a"},"breadcrumb":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#primaryimage","url":"https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png","contentUrl":"https:\/\/blog.zhaw.ch\/icclab\/files\/2013\/07\/Graph.png","width":486,"height":214},{"@type":"BreadcrumbList","@id":"https:\/\/blog.zhaw.ch\/icclab\/distributed-file-system-series-glusterfs-introduction\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/blog.zhaw.ch\/icclab\/"},{"@type":"ListItem","position":2,"name":"Distributed File System Series: GlusterFS Introduction"}]},{"@type":"WebSite","@id":"https:\/\/blog.zhaw.ch\/icclab\/#website","url":"https:\/\/blog.zhaw.ch\/icclab\/","name":"Service Engineering (ICCLab &amp; SPLab)","description":"A Blog of the ZHAW Zurich University of Applied Sciences","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/d7955ea228a04e754bad7f721febf73a","name":"strp","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4bd7dfc3a47734d2255bff73fdc2f40d8899db7726711977f95ae35e0ee3d2ac?s=96&d=mm&r=g","caption":"strp"},"url":"https:\/\/blog.zhaw.ch\/icclab\/author\/strp\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/3076","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/users\/73"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/comments?post=3076"}],"version-history":[{"count":0,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/3076\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/media?parent=3076"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/categories?post=3076"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/tags?post=3076"},{"taxonomy":"features","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/features?post=3076"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}