{"id":5391,"date":"2014-07-09T12:03:18","date_gmt":"2014-07-09T10:03:18","guid":{"rendered":"http:\/\/blog.zhaw.ch\/icclab\/?p=5391"},"modified":"2014-07-09T17:47:43","modified_gmt":"2014-07-09T15:47:43","slug":"advanced-queries-to-ceilometer-with-a-mongo-backend","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/","title":{"rendered":"Advanced Queries to Ceilometer with a mongo backend"},"content":{"rendered":"<p>We are changing our ceilometer backend from mysql to <a href=\"http:\/\/www.mongodb.org\/\">mongodb<\/a> in one of our experimental <a href=\"https:\/\/www.openstack.org\/\">Openstack<\/a> installations. The reason for this change is that mongo seems to deliver better ceilometer performance than <a href=\"http:\/\/www.mysql.com\/\">mysql<\/a>; further ceilometer data structures are a more natural fit with mongo (and indeed, this is largely where they came from). This can be seen in a typical record below which is clearly hierarchical and contains so-called embedded documents (in mongo terminology):<\/p>\n<pre>\"_id\" : ObjectId(\"53bbe7ea926fc4597b42aafc\"),\r\n\"counter_name\" : \"instance\",\r\n\"resource_metadata\" : {\r\n    \"status\" : \"active\",\r\n    \"display_name\" : \"Test\",\r\n    \"name\" : \"instance-00000001\",\r\n    \"image\" : {\r\n        \"id\" : \"bdfaab74-6542-4cbb-94f1-5306662208a7\",\r\n        \"name\" : \"cirros-0.3.2-x86_64-uec\"\r\n    },\r\n    \"host\" : \"7c261f3a33c099538d448be797e5ce0c0d8cf8bf9f75dd59ce04df86\",\r\n}\r\n...\r\n<\/pre>\n<p>Mongo natively provides support for queries of data structured in this fashion. More specifically, mongo enables data at different levels of the hierarchy to be queried &#8211; something which is difficult in SQL.<\/p>\n<p>In <a href=\"https:\/\/www.python.org\/\">python<\/a>, this can be done simply as follows:<\/p>\n<pre><code>\r\nquery = [{'field': 'timestamp', 'op': 'gt', 'value': <i>date<\/i>},\r\n{'field': 'metadata.status', 'op': 'eq', 'value': 'active'}]\r\n\r\nsample_list = ceilometer.samples.list(meter_name='instance', q=query)\r\n<\/code><\/pre>\n<p>The interesting point, which we did not understand clearly until now, is that ceilometer with a mongo backend supports exactly this type of query. Thus, the following command line query can obtain all instances that were active for a certain period of time:<\/p>\n<pre><code>\r\nceilometer sample-list -m instance -q \u201ctimestamp&gt;<i>date<\/i>; metadata.status=\u2019active\u2019\u201d \r\n<\/code><\/pre>\n<p>Then ceilometer will return all the samples of instances active in this time range.<\/p>\n<pre>+--------------------------------------+-----------+-----------+--------------------+\r\n| Resource ID \u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0  \u00a0\u00a0 \u00a0 | Name\u00a0 \u00a0 \u00a0 | Type\u00a0\u00a0\u00a0\u00a0 | Timestamp  \u00a0 \u00a0 \u00a0 \u00a0  |\r\n+--------------------------------------------------+----------+---------------------+\r\n| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance\u00a0 | instance | 2014-07-09T09:30:05 |\r\n| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance\u00a0 | instance | 2014-07-09T09:20:05 |\r\n| 7535b9f6-01a6-410e-980d-338031e7a2c4 | instance\u00a0 | instance | 2014-07-09T09:10:04 |\r\n+--------------------------------------+-----------+----------+---------------------+\r\n<\/pre>\n<p>Querying ceilometer with a mysql database in this fashion results in an error.<\/p>\n<div class=\"pt-sm\">Schlagw\u00f6rter: <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/ceilometer\/\">Ceilometer<\/a>, <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/mongo\/\">mongo<\/a>, <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/mongodb\/\">mongodb<\/a>, <a href=\"https:\/\/blog.zhaw.ch\/icclab\/tag\/openstack\/\">openstack<\/a><br><\/div>","protected":false},"excerpt":{"rendered":"<p>We are changing our ceilometer backend from mysql to mongodb in one of our experimental Openstack installations. The reason for this change is that mongo seems to deliver better ceilometer performance than mysql; further ceilometer data structures are a more natural fit with mongo (and indeed, this is largely where they came from). This can [&hellip;]<\/p>\n","protected":false},"author":101,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1,5,21],"tags":[65,432,433,240],"features":[],"class_list":["post-5391","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-articles","category-openstack-2","tag-ceilometer","tag-mongo","tag-mongodb","tag-openstack"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Advanced Queries to Ceilometer with a mongo backend - Service Engineering (ICCLab &amp; SPLab)<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Advanced Queries to Ceilometer with a mongo backend\" \/>\n<meta property=\"og:description\" content=\"We are changing our ceilometer backend from mysql to mongodb in one of our experimental Openstack installations. The reason for this change is that mongo seems to deliver better ceilometer performance than mysql; further ceilometer data structures are a more natural fit with mongo (and indeed, this is largely where they came from). This can [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\" \/>\n<meta property=\"og:site_name\" content=\"Service Engineering (ICCLab &amp; SPLab)\" \/>\n<meta property=\"article:published_time\" content=\"2014-07-09T10:03:18+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2014-07-09T15:47:43+00:00\" \/>\n<meta name=\"author\" content=\"Bruno Grazioli\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bruno Grazioli\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\"},\"author\":{\"name\":\"Bruno Grazioli\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba\"},\"headline\":\"Advanced Queries to Ceilometer with a mongo backend\",\"datePublished\":\"2014-07-09T10:03:18+00:00\",\"dateModified\":\"2014-07-09T15:47:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\"},\"wordCount\":194,\"commentCount\":0,\"keywords\":[\"Ceilometer\",\"mongo\",\"mongodb\",\"openstack\"],\"articleSection\":[\"*.*\",\"Articles\",\"OpenStack\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\",\"name\":\"Advanced Queries to Ceilometer with a mongo backend - Service Engineering (ICCLab &amp; SPLab)\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\"},\"datePublished\":\"2014-07-09T10:03:18+00:00\",\"dateModified\":\"2014-07-09T15:47:43+00:00\",\"author\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/blog.zhaw.ch\/icclab\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Advanced Queries to Ceilometer with a mongo backend\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/\",\"name\":\"Service Engineering (ICCLab &amp; SPLab)\",\"description\":\"A Blog of the ZHAW Zurich University of Applied Sciences\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba\",\"name\":\"Bruno Grazioli\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g\",\"caption\":\"Bruno Grazioli\"},\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/author\/gaea\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Advanced Queries to Ceilometer with a mongo backend - Service Engineering (ICCLab &amp; SPLab)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/","og_locale":"en_US","og_type":"article","og_title":"Advanced Queries to Ceilometer with a mongo backend","og_description":"We are changing our ceilometer backend from mysql to mongodb in one of our experimental Openstack installations. The reason for this change is that mongo seems to deliver better ceilometer performance than mysql; further ceilometer data structures are a more natural fit with mongo (and indeed, this is largely where they came from). This can [&hellip;]","og_url":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/","og_site_name":"Service Engineering (ICCLab &amp; SPLab)","article_published_time":"2014-07-09T10:03:18+00:00","article_modified_time":"2014-07-09T15:47:43+00:00","author":"Bruno Grazioli","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Bruno Grazioli","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#article","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/"},"author":{"name":"Bruno Grazioli","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba"},"headline":"Advanced Queries to Ceilometer with a mongo backend","datePublished":"2014-07-09T10:03:18+00:00","dateModified":"2014-07-09T15:47:43+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/"},"wordCount":194,"commentCount":0,"keywords":["Ceilometer","mongo","mongodb","openstack"],"articleSection":["*.*","Articles","OpenStack"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/","url":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/","name":"Advanced Queries to Ceilometer with a mongo backend - Service Engineering (ICCLab &amp; SPLab)","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#website"},"datePublished":"2014-07-09T10:03:18+00:00","dateModified":"2014-07-09T15:47:43+00:00","author":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba"},"breadcrumb":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blog.zhaw.ch\/icclab\/advanced-queries-to-ceilometer-with-a-mongo-backend\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/blog.zhaw.ch\/icclab\/"},{"@type":"ListItem","position":2,"name":"Advanced Queries to Ceilometer with a mongo backend"}]},{"@type":"WebSite","@id":"https:\/\/blog.zhaw.ch\/icclab\/#website","url":"https:\/\/blog.zhaw.ch\/icclab\/","name":"Service Engineering (ICCLab &amp; SPLab)","description":"A Blog of the ZHAW Zurich University of Applied Sciences","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/769a2455bfa3cbcc87857218f19abeba","name":"Bruno Grazioli","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/07d475be0415d5914aa49599b3295e4d972f971e46f5d7ab89d474327ab7b5f0?s=96&d=mm&r=g","caption":"Bruno Grazioli"},"url":"https:\/\/blog.zhaw.ch\/icclab\/author\/gaea\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/5391","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/users\/101"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/comments?post=5391"}],"version-history":[{"count":9,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/5391\/revisions"}],"predecessor-version":[{"id":5403,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/5391\/revisions\/5403"}],"wp:attachment":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/media?parent=5391"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/categories?post=5391"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/tags?post=5391"},{"taxonomy":"features","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/features?post=5391"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}