{"id":9054,"date":"2015-10-23T13:30:03","date_gmt":"2015-10-23T11:30:03","guid":{"rendered":"http:\/\/blog.zhaw.ch\/icclab\/?p=9054"},"modified":"2015-10-23T13:34:13","modified_gmt":"2015-10-23T11:34:13","slug":"reactive-orchestration-in-hurtle-with-monasca","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/","title":{"rendered":"Reactive Orchestration in Hurtle with Monasca"},"content":{"rendered":"

In a previous series of blog posts (1<\/a>,\u00a02<\/a>,\u00a03<\/a>), we have discussed how to install Monasca<\/a> to monitor OpenStack, how to create alarms based on specific events happening in the monitored system, and how to setup\u00a0notifications when any of these alarms are triggered.<\/p>\n

Going further, in the context of the Cloud Orchestration<\/a> initiative and the Hurtle<\/a> framework, we go further by using Monasca to detect events in orchestrated applications and perform callbacks to the orchestrator so it can react to events. The motivation behind this is provide hurtle with processes able to perform\u00a0continuous health management<\/em> of any orchestrated application.<\/p>\n

While initially designed to monitor the Cloud itself, it is easy to install the monasca agent on any platform, making it simple to monitor deployed VMs behaviour.<\/p>\n

Concepts<\/h1>\n

Below is the logical architecture of the reactive orchestration in Hurtle:<\/p>\n

\"alarm_handling\"<\/a><\/p>\n

Each VM comprising an orchestrated application (see the logical\u00a0architecture<\/a>\u00a0of hurtle to learn about how hurtle orchestrates services) hosts a Monasca Agent which pushes metrics to Monasca.<\/p>\n

Monasca aggregates these metrics and performs a periodic alarm check process, checking all alarms against the metrics it receives.<\/p>\n

If a threshold is reached, Monasca issues an alarm, which in turns sends a notification as an OCCI request to the Service Orchestrator controlling the deployed application. This orchestrator can then perform corrective action as required. Multiple alarms can be registered for the same systems, so it is easy to build a service orchestrator able to react to multiple types of events and in each case issue a specific action.<\/p>\n

Implementation<\/h1>\n

Creating a service orchestrator able to react to triggered alarms requires a few modifications to the methods triggered at the deployment and dispose phases of the lifecycle. It also requires the implementation of a notify method which is called when a service orchestrator instance receives a notification callback.<\/p>\n

Note that you can find a complete example of a runtime SO can be found in the sample_so<\/a>\u00a0hurtle repository, with a detailed explanation of each method in the notify function\u00a0README<\/a>\u00a0file. Below is a summary of what happens at each step of the lifecycle.<\/p>\n

Deploy phase<\/h2>\n

At this phase, the Heat template<\/a> of a service is deployed on an underlying infrastructure using the Hurtle SDK<\/a>.<\/p>\n

Setting up Monasca Agent on VMs<\/h3>\n

Remember that we said that a Monasca agent needs to be present on each virtual machine of the application, so the Heat template must contain a configuration script able to install this agent dynamically on each virtual machine as it is created. The agent is setup using the\u00a0monasca-setup<\/em> utility using a command similar to:<\/p>\n

monasca-setup --username monasca_username --password monasca_password --project_name monasca_tenant --service service_id --keystone_url http:\/\/bart.cloudcomplab.ch:35357\/v3<\/pre>\n
Every metric sent to Monasca is authenticated through a keystone token, which is why it is necessary to provide authentication information the agent, so it can use a valid token at all times.<\/p>\n
This setup can be done in the user_data section of each Server resources in a Heat template.<\/p>\n
Setting up alarms and notifications<\/h3>\n
The SDK provides helper functions to create alarms and notifications in a single command. First one needs to get familiar with the alarm model used by Monasca, found in their API documentation.<\/a>\u00a0An alarm is defined as a threshold based on a specific set of metrics, and has three states: OK (threshold has not been reached), ALARM (threshold has been reached) an UNDETERMINED (no metrics received over a period of time). Here are a few examples:<\/p>\n
avg(cpu.user_perc{service=monitoring,hostname=mini-mon})>60<\/code><\/pre>\n
avg(cpu.system_perc{hostname=host1}, 120) > 95 times 3<\/code><\/pre>\n
The first alarm is triggered when the average of all cpu.user_perc metrics\u00a0tagged by the service\u00a0monitoring<\/em> and the\u00a0hostname\u00a0mini-mon<\/em>\u00a0received by Monasca over one period (one period is hardcoded to 60 seconds in Monasca) is above 60.<\/p>\n
The second alarm is triggered when the average of cpu.system_perc metrics tagged by the hostname\u00a0host1<\/em> over a period of 120 seconds is above 95, three times consecutively.<\/p>\n
A commonly used alarm is one which is always OK unless no metrics are received, in which case it goes to UNDETERMINED. This is useful as there is no other way to detect that a monitored system is offline (besides external checks by another system), as suddenly no metrics are sent, it is not possible to trigger an ALARM state, which can only happen if metrics are received by Monasca. The alarm used in our example is:<\/p>\n
avg(cpu.user_perc{<\/span>hostname=host1}) > 200)<\/span><\/pre>\n
Notifications are separate from alarms, by default nothing happens when an alarm is triggered. If the user wants to be notified, he needs to configure his\u00a0alarm definition with notifications on\u00a0state changes<\/em>, for instance when an alarm goes from OK to UNDETERMINED or from OK to ALARM.<\/p>\n
A SO developer can use the SDK which provides the function\u00a0notify<\/em> to create an alarm and an associated notification in one call. Here is an example:<\/p>\n
n_name, n_id = rt.notify('(avg(cpu.user_perc{hostname=rcb_serv_1}) > 200)', 'http:\/\/inst1.rcb.cloudcomplab.ch\/orchestrator\/default', runtime.ACTION_UNDETERMINED)<\/pre>\n
This creates the alarm and sets up a notification to be sent to the orchestrator if the state of the alarm goes from OK (or ALARM) to UNDETERMINED. The alarm name needs to be unique across all Monasca alarms, so it is automatically generated by the SDK. The SO developer can then store this name and associate it with a specific action. This needs to be saved as it will be used later when the alarm is triggered.<\/p>\n
The notification is set to be sent to\u00a0‘http:\/\/inst1.rcb.cloudcomplab.ch\/orchestrator\/default’, which in this example is the hardcoded address of the SO. In a real implementation, the SO developer needs to retrieve the dynamic address of the deployed SO, which differs based on if it is deployed on OpenShift v2 or OpenShift v3.<\/p>\n
Notification<\/h2>\n
When an alarm set up during the deploy phase is triggered, a OCCI<\/a>-formatted POST is sent by Monasca to the SO instance, with a special attribute alarm_name\u00a0<\/em>displaying the name of the triggered alarm. This reaches the SO\u00a0notify<\/em> method, which can then act based on which alarm name it receives and can choose which action to perform.<\/p>\n
Restart a resource with Heat<\/h3>\n
Using the SDK, the only mean of the SO to modify the deployed resources is through Heat. It can only issue update<\/em> commands to this end. The issue is that if a resource has disappeared, Heat will not know it as by default it includes no health management capabilities (besides using specific resources); this means that if an update is sent with resources using the same resource names, Heat will simply do nothing. A workaround to that is to create a new resource with a different name, and simply link it to the previous Neutron Port used by the old resource.<\/p>\n
For example, take this<\/a> initial template, to restart the server\u00a0rcb_si<\/em>\u00a0while using the same port, Heat can receive a template similar to this one<\/a>. The only difference is that the Nova Server resource name has been changed to rcb_si_1<\/em>.<\/p>\n
Dispose phase<\/h2>\n
In this phase, on top of destroying the deployed Heat templates, all previously created alarms and notifications have to be deleted. The SDK provides the\u00a0dispose_monasca<\/em> function for that purpose.<\/p>\n
Conclusion<\/h1>\n
This addition to Hurtle provides what the health management functionality that is required of a full-scale orchestrator, and is very flexible as users have full control over which metrics they send (the monasca agent is completely configurable, and can send any metrics so long as they conform to the common Monasca formatting and are accompanied with a valid token, so any application-specific metrics can be configured on top of typical resource metrics like CPU usage), and can create very diverse alarms.<\/p>\n
Current limitations are due to the fact that Monasca is relatively slow to react to events and can not easily be configured with a period below 60 seconds, meaning that alarms are slow to trigger. This is even worse with UNDETERMINED\u00a0triggers, which can only happen when no metrics have been received for three consecutive periods (180 seconds!).<\/p>\n
Schlagw\u00f6rter: cloud incident management<\/a>, Cloud Orchestration<\/a>, hurtle<\/a>, monasca<\/a>, openstack<\/a>
<\/div>","protected":false},"excerpt":{"rendered":"
In a previous series of blog posts (1,\u00a02,\u00a03), we have discussed how to install Monasca to monitor OpenStack, how to create alarms based on specific events happening in the monitored system, and how to setup\u00a0notifications when any of these alarms are triggered. Going further, in the context of the Cloud Orchestration initiative and the Hurtle […]<\/p>\n","protected":false},"author":94,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[1],"tags":[509,585,592,510,240],"features":[],"class_list":["post-9054","post","type-post","status-publish","format-standard","hentry","category-allgemein","tag-cloud-incident-management-2","tag-cloud-orchestration","tag-hurtle","tag-monasca","tag-openstack"],"yoast_head":"\nReactive Orchestration in Hurtle with Monasca - Service Engineering (ICCLab & SPLab)<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Reactive Orchestration in Hurtle with Monasca\" \/>\n<meta property=\"og:description\" content=\"In a previous series of blog posts (1,\u00a02,\u00a03), we have discussed how to install Monasca to monitor OpenStack, how to create alarms based on specific events happening in the monitored system, and how to setup\u00a0notifications when any of these alarms are triggered. Going further, in the context of the Cloud Orchestration initiative and the Hurtle […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\" \/>\n<meta property=\"og:site_name\" content=\"Service Engineering (ICCLab & SPLab)\" \/>\n<meta property=\"article:published_time\" content=\"2015-10-23T11:30:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2015-10-23T11:34:13+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png\" \/>\n<meta name=\"author\" content=\"dudo\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"dudo\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\"},\"author\":{\"name\":\"dudo\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a\"},\"headline\":\"Reactive Orchestration in Hurtle with Monasca\",\"datePublished\":\"2015-10-23T11:30:03+00:00\",\"dateModified\":\"2015-10-23T11:34:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\"},\"wordCount\":1304,\"commentCount\":0,\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png\",\"keywords\":[\"cloud incident management\",\"Cloud Orchestration\",\"hurtle\",\"monasca\",\"openstack\"],\"articleSection\":[\"*.*\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\",\"name\":\"Reactive Orchestration in Hurtle with Monasca - Service Engineering (ICCLab & SPLab)\",\"isPartOf\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png\",\"datePublished\":\"2015-10-23T11:30:03+00:00\",\"dateModified\":\"2015-10-23T11:34:13+00:00\",\"author\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a\"},\"breadcrumb\":{\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1.png\",\"contentUrl\":\"https:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1.png\",\"width\":1454,\"height\":798},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\/\/blog.zhaw.ch\/icclab\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Reactive Orchestration in Hurtle with Monasca\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#website\",\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/\",\"name\":\"Service Engineering (ICCLab & SPLab)\",\"description\":\"A Blog of the ZHAW Zurich University of Applied Sciences\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a\",\"name\":\"dudo\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g\",\"caption\":\"dudo\"},\"url\":\"https:\/\/blog.zhaw.ch\/icclab\/author\/dudo\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Reactive Orchestration in Hurtle with Monasca - Service Engineering (ICCLab & SPLab)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/","og_locale":"en_US","og_type":"article","og_title":"Reactive Orchestration in Hurtle with Monasca","og_description":"In a previous series of blog posts (1,\u00a02,\u00a03), we have discussed how to install Monasca to monitor OpenStack, how to create alarms based on specific events happening in the monitored system, and how to setup\u00a0notifications when any of these alarms are triggered. Going further, in the context of the Cloud Orchestration initiative and the Hurtle […]","og_url":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/","og_site_name":"Service Engineering (ICCLab & SPLab)","article_published_time":"2015-10-23T11:30:03+00:00","article_modified_time":"2015-10-23T11:34:13+00:00","og_image":[{"url":"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png","type":"","width":"","height":""}],"author":"dudo","twitter_card":"summary_large_image","twitter_misc":{"Written by":"dudo","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#article","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/"},"author":{"name":"dudo","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a"},"headline":"Reactive Orchestration in Hurtle with Monasca","datePublished":"2015-10-23T11:30:03+00:00","dateModified":"2015-10-23T11:34:13+00:00","mainEntityOfPage":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/"},"wordCount":1304,"commentCount":0,"image":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png","keywords":["cloud incident management","Cloud Orchestration","hurtle","monasca","openstack"],"articleSection":["*.*"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/","url":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/","name":"Reactive Orchestration in Hurtle with Monasca - Service Engineering (ICCLab & SPLab)","isPartOf":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#website"},"primaryImageOfPage":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage"},"image":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage"},"thumbnailUrl":"http:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1-300x165.png","datePublished":"2015-10-23T11:30:03+00:00","dateModified":"2015-10-23T11:34:13+00:00","author":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a"},"breadcrumb":{"@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#primaryimage","url":"https:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1.png","contentUrl":"https:\/\/blog.zhaw.ch\/icclab\/files\/2015\/10\/alarm_handling1.png","width":1454,"height":798},{"@type":"BreadcrumbList","@id":"https:\/\/blog.zhaw.ch\/icclab\/reactive-orchestration-in-hurtle-with-monasca\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/blog.zhaw.ch\/icclab\/"},{"@type":"ListItem","position":2,"name":"Reactive Orchestration in Hurtle with Monasca"}]},{"@type":"WebSite","@id":"https:\/\/blog.zhaw.ch\/icclab\/#website","url":"https:\/\/blog.zhaw.ch\/icclab\/","name":"Service Engineering (ICCLab & SPLab)","description":"A Blog of the ZHAW Zurich University of Applied Sciences","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blog.zhaw.ch\/icclab\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/blog.zhaw.ch\/icclab\/#\/schema\/person\/aba82692aa266eab256d8df5ea04b06a","name":"dudo","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0df0ad477b78c3b2ef455cd33a99c552c77c6b80c18289b27b7036a0c7cff025?s=96&d=mm&r=g","caption":"dudo"},"url":"https:\/\/blog.zhaw.ch\/icclab\/author\/dudo\/"}]}},"_links":{"self":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/9054","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/users\/94"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/comments?post=9054"}],"version-history":[{"count":16,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/9054\/revisions"}],"predecessor-version":[{"id":9077,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/posts\/9054\/revisions\/9077"}],"wp:attachment":[{"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/media?parent=9054"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/categories?post=9054"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/tags?post=9054"},{"taxonomy":"features","embeddable":true,"href":"https:\/\/blog.zhaw.ch\/icclab\/wp-json\/wp\/v2\/features?post=9054"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}