A Web Application to Monitor and Understand Energy Consumption in an Openstack Cloud

In one of our projects we need to understand the energy consumption of our servers. Our initial work in this direction involved collecting energy consumption data using Kwapi and storing it in Ceilometer for further study. The data stored in Ceilometer is valuable; however, it is insufficient to really understand energy consumption in detail. Consequently, we are developing a web application which gives a much greater insight into energy consumption in our cloud resources. This is very much a work in progress, so this post just highlights a few points relating to the application as well as a video which shows the current version of the application.

The tool was developed to be totally integrated with Openstack. Users log in with their Openstack credentials (using Keystone authentication) and are  redirected to the overview page where they can see  the total energy consumed by the VMs in their projects for the the previous month as well as some  general information regarding virtual machines; a line chart displays how energy consumed varies over time.

Continue reading

ICCLab Talk at the “INTEL European Research and Innovation Conference 2012”

Header Image

The ICCLab was invited to speak at the Intel European Research & Innovation Conference incorporating Research@Intel Europe, 22 October 2012 – 23 October 2012

Andy Edmonds was talking about “Open Cloud Standards” and Thomas M. Bohnert was reviewing “Dependability in the World of Clouds” (slides are available here).

About ERIC 2012

The 2012 Intel European Research and Innovation Conference will take place at the UPC campus/Princesa Sofia Gran Hotel in Barcelona, Spain, from October 22-23. The theme of this year’s conference is ‘Building a Smart, Sustainable and Inclusive society through Research and Innovation partnership’ and there will be a number of distinct focus areas included within the event. Additionally we celebrate the 10th anniversary of Intel Labs Barcelona.

Conference Goals

Promote a world class research and innovation ecosystem in Europe through focused research and innovation between enterprise, academia and policy makers and advance further understanding of applied strategic research in the following areas;

    • Computer Architecture – New trends in computer architecture on a broad set of topics including Processor, Memory, Interconnects, SoC, Dependability, Parallelism, Power and Performance Modeling among others.
    • Social Innovation – A new mindset and a new strategic direction which holds a great potential to transform economies, to fuel innovation and to improve the lives of people around the world.
    • Securing Clouds and Mobility – The convergence of mobile, automotive, and the cloud from a security and risk perspective.
    • Sustainable Connected Cities – Advancing compute, communication and social constructs to deliver breakthrough innovations in system architecture, algorithms, and societal participation

Agenda

Day One

  • 11.00-12.00 – Registration & Lunch
  • 12.00-14.00 – ERIC Opening Plenary Sessions
  • 14.15-17.30 – Track Sessions Part 1:
    • Computer Architecture
    • Social Innovation
    • Securing Clouds & Mobility
    • Sustainable Connected Cities
  • 19.00 onwards – 10th Anniversary Intel Labs Barcelona Celebration Dinner

Day Two

  • 09.00-10.30 – Keynotes
  • 10.30-10.45 – Coffee Break
  • 10.45-12.00 – Research@Intel Europe Showcase
  • 12.00-13.00 – Lunch
  • 13.00-16.30 – Track Sessions Part 2
    • Computer Architecture
    • Social Innovation
    • Securing Clouds & Mobility
    • Sustainable Connected Cities

ICCLab Invited to European Commission Cloud Expert Group

The ICCLab (Andy and Thomas) is invited to participate to the next meeting of the Cloud Expert Group which will take place in October 29-30, 2012 in Brussels.

The previous work of the Cloud Expert Group (“Advances in Clouds”) has clearly shown that Cloud Computing still requires research and development work in multiple domains (e.g. software & services, networks, security, complex systems, etc.).

The aim of the workshop is to refine the research topics identified in the above mentioned report, provide more details and develop a roadmap with priorities and actions.

To better shape the workshop discussions, position papers (no longer than 2 pages) are submitted on any of the following topics:

  • Data management, communications & networks
  • Resource description & usage, resource management
  • Programmability & usability
  • Federation, interoperability, portability
  • Security
  • Business and cost models, expertise & usability

Andy will provide a position paper on cloud standards, federation, and interoperability from the ICCLab’s perspective. The slides will be online after the event.

ICCLabs Present on OCCI at OWF/CloudCamp Paris

ICCLab is just back from Open World Forum in Paris after being part of the organisation of the “Open Cloud, Open Standards” session that was run in conjunction with CloudCamp. It was an excellent session and Simon Wardley was the master of MC’ing. There was a number of presentations to kick off the day and that was followed by very interesting and energetic round table discussions. The ICCLab presented on OCCI and you can see that presentation here (PDF here):

The topics of discussion during the day revolved around:

  • “Does the Cloud need standards?”
  • “Government role in Cloud, beneficial or dangerous intervention?”
  • “What do we mean by standards anyway – specification or reference model?”
  • “Whose standards – de jeure or defacto?”
  • “Should those standards be open?”
  • “Is there any such thing as an open API or are all APIs open?”

Stay tuned for an upcoming blog post where we discuss the out comes of the session.

Upcoming Events: Swiss OpenStack User Group and OCCI Working Group Meetings

In coordination with ZH-Geeks, the ICCLab will be hosting the very first Swiss OpenStack user group meeting on the 15th of Novemeber. In addition to Tim Bell there will be a number of other speakers who will present on various OpenStack topics. Tim Bell runs the infrastructure team at CERN where they run a very large installation of OpenStack. Tim is also a member of the OpenStack foundation management board and one of the main drivers of the OpenStack-based IaaS needed to analyse the 25PB of data a year from the LHC. If anyone wants to submit a lightening talk (talk with slides of no more than 10 mins) on their work with OpenStack please let Andy Edmonds know. We would encourage as many of you to attend! Please sign up at the following site.

Also, on the 13th of Novemeber the Open Cloud Computing Interface working group will be having a day long meeting to discuss and improve the current specification. This event as an excellent means to influence one of the most prominent cloud standards today. If you wish to attend please get in touch with Andy Edmonds and also add your name to the list of attendees.

Updated OCCI OpenStack Install Guide

Recently, the [OCCI implementation](http://www.github.com/dizz/nova) for [OpenStack](http://www.openstack.org) was made available by [work done by Intel Labs Europe](http://wiki.openstack.org/occi) as part of the [FI-ware project](http://www.fi-ware.eu). Some of the install instructions are now somewhat out of date. In this post we’ll outline the steps necessary to get the OCCI implementation up and running. This updated install guide is also now reflected on the [OpenStack OCCI wiki](http://wiki.openstack.org/occi). A big thanks goes out to Piotr Kasprzak at [GDWG](http://www.gwdg.de) for some of the updates!

# Steps

Create a fresh VM. Ubuntu 12.04 is a good baseline.

Get devstack:

[gist id=2917429]

Install some necessaries:

[gist id=2917431]

Fix the `prettytable` issue:

[gist id=2917434]

Edit `~/devstack/stackrc` so it checks out the OCCI branch of OpenStack:

[gist id=2917441]

Create/Edit `~/devstack/localrc` with the following content:

[gist id=2917445]

Now execute devstack:

[gist id=2917453]

You will be now asked for a number of service admin passwords. Once the devstack process has completed you should see the following:

[gist id=2917462]

Once the stack is running you can then issue the commands that are on the [OpenStack OCCI wiki page](http://wiki.openstack.org/occi).

A virtual machine with both OpenStack and OCCI can be [downloaded from here](http://www.cloudcomp.ch/wp-content/uploads/2012/06/OCCI-OS.ova). The user name and password is `occi` and `occi`. It is in an OVA export format and you can easily import into VirtualBox. In `~/devstack/localrc` the `OFFLINE` parameter is set to `True`. If you want to update the devstack installation change this to `False`.

## Potential Issues

1. If the OCCI API blocks and does not return a response then please check your `/etc/nova/api-paste.ini` configuration. Ensure that the `[filter:authtoken]` section has the correct `service_host` and `auth_host` values.

OCCI and ICCLab

We here in the ICCLab are very much interested in all activities related to standardisation in the Cloud. Standardisation forms part of our [overall strategy in bringing impact](http://www.cloudcomp.ch/research/) to the communities we take part in. One of these activities is our participation in the [Open Cloud Computing Interface (OCCI)](http://occi-wg.org/). OCCI is a standardisation activity ran out of the [Open Grid Forum](http://www.ogf.org). Briefly OCCI is:

> a Protocol and API for all kinds of Management tasks. OCCI was originally initiated to create a remote management API for IaaS model based Services, allowing for the development of interoperable tools for common tasks including deployment, autonomic scaling and monitoring. It has since evolved into a flexible API with a strong focus on integration, portability, interoperability and innovation while still offering a high degree of extensibility. The current release of the Open Cloud Computing Interface is suitable to serve many other models in addition to IaaS, including e.g. PaaS and SaaS.

Apart from contributing to the OCCI specifications, the ICCLab is contributing to the OCCI implementation for OpenStack and it is with this implementation that we take part in the various [Cloud Plugfest](http://www.cloudplugfest.org/) events, which “*is a co-operative community project designed to promote interoperability efforts on cloud-based software, frameworks, and standards among vendors, products, projects and implementations*”.