In our previous blog post we showed a web application to monitor and understand energy consumption in an Openstack cluster; the main goal of this tool is to understand how energy consumption relates to activity on the cloud resources. In general, this can be quite complex as there are many resources to take in account and Ceilometer doesn’t report all the information required at this time. In this blog post we describe one task we needed to perform in this work: we needed to retrieve from Ceilometer the set of VMs running on a given physical server together with their CPU utilization over some period of time. We can envisage other contexts in which it might be useful to do this, so we present the solution here. (Note that it is of course possible to obtain the set of VMs that are currently running on a given physical host using Nova, but this does not offer the historical perspective).
Before getting the CPU utilization data, it is first necessary to understand on which physical machine the instance is running. Ceilometer instance sample data provides the information shown below (note that in our system Icehouse is installed – the record below shows a subset of the information relating to each instance in Icehouse/Ceilometer).
{
u'counter_name': u'instance',
u'user_id': u'00d9efc38ce6455cb3cf3431a51a7b83',
u'resource_id': u'6831e904-cf78-4832-982a-b5e283197b80',
u'timestamp': u'2014-11-19T09:27:06',
u'counter_volume': 1.0,
u'resource_metadata': {
u'display_name': u'Arcus-Ubuntu',
u'status': u'active',
u'host': u'88cc2806f303e1f8dad5ca6c14eca1d446bc92e6e9f273391a14153c',
u'image.name': u'Ubuntu 14.04 Cloud',
u'memory_mb': u'2048',
u'vcpus': u'1'},
u'source': u'openstack',
u'counter_unit': u'instance',
u'project_id': u'd663ec7a8be942ba97cf1b0f904647ed',
u'counter_type': u'gauge'
}
The host parameter provides some information regarding the physical node; according to the documentation:
“The compute provisioning algorithm has an anti-affinity property that attempts to spread customer VMs across hosts. Under certain situations, VMs from the same customer might be placed on the same host. host Id represents the host your server runs on and can be used to determine this scenario if it is relevant to your application. Host Id is unique per account and is not globally unique”.
A little more digging on this revealed that there is some tension between the cloud provider and the customer: the cloud provider does not always want to offer full transparency to the customer regarding the infrastructure, while the customer often wants to have some understanding of how her work is distributed on physical nodes (for reliability and other reasons). The Openstack community devised a reasonable compromise in which a host parameter (previously hostid) is associated with each instance which is a hash of the project_id and the hypervisor name. It limits what the customer can infer about the infrastructure, but enables her to understand when two VMs are running on the same physical host.
The code snippet below shows how this host parameter is generated:
import hashlib sha_hash = hashlib.sha224(project_id + hypervisor_name) print sha_hash.hexdigest()
With the snippet above we could then write a simple script which extracts the information from Ceilometer and write the result to a CSV file. The script iterates over all known instances recorded by Ceilometer and determines if they were running on the given physical host within the time window specified. Then the script extracts the CPU utilization from each instance found previously and writes to the CSV file; for each row in the CSV file a subset of fields are written and they can be specified in the script.
To run the script it is necessary to set your environment variables, also modify server_name, meter, start_date, end_date variables.
server_name Hypervisor name, it can be found using nova-manage host list on the controller node
meter Name of the meter. eg cpu_util, vcpus, memory
start_date, end_date Dates in “%Y-%m-%dT%H:%M:%S” string format
The output files are created in the same folder as the script is and their names are formatted in meter-instance_id-server_name.csv.
This script helps us to have a better understanding of what is happening inside the servers, automating the task of extracting the relevant information from Ceilometer. With the results collected is possible to correlate the activity of a server with its energy consumption. This is very much a work in progress the results acquired and corresponding analysis will be shown in the next blog post.