Energy in general and energy consumption in particular is a major issue for the large cloud providers today. Smaller cloud providers – both private and public – also have an interest in reducing their energy consumption, although it is often not their most important concern. With increasing competition and decreasing margins in the IaaS sector, management of energy costs will become increasingly important.
A basic prerequisite of advanced energy management solutions is a good understanding of energy consumption. This is increasingly available in multiple ways as energy meters proliferate: as well as having energy meters on racks, energy meters typically exist in modern hardware and even at subsystem level within today’s hardware. That said, energy metering is something that is commonly coupled to proprietary management systems.
The focus of this initiative is to develop an understanding of cloud energy consumption through measurement and analysis of usage.
Objectives
The objectives of the energy monitoring initiative are:
- to develop a tool to visualize how energy is being consumed within the cloud resources;
- to understand the correlation between usage of cloud resources and energy consumption;
- to understand what level of granularity is appropriate for capturing energy data;
- to devise mechanisms to disaggregate energy consumption amongst users of cloud platforms.
Research Challenges
Understanding cloud energy consumption does not give rise to fundamental research challenges – indeed, it is more of an enabler for a more advanced energy management system. However, to have a comprehensive understanding of cloud energy consumption, some research effort is required. The following research challenges arise in this context:
- How to consolidate energy consumption from disparate sources to realize a clear understanding of energy consumption within the cloud environment
- How to correlate energy consumption with revenue generating services at a fine-grained level (compute, storage and networking)
Relevance to current and future markets
Understanding energy consumption is essential for the large cloud providers as well as for today’s Data Centre providers. Consequently, there are already solutions available which support monitoring of energy consumption of IT resources. Today’s solutions typically do not have specific knowledge of cloud resource utilization and consequently, there is an opportunity for new tools which correlate cloud usage with energy monitoring.
In the Gartner Hype Cycle for Green IT 2014, there are some related technologies which have growth potential over the coming years. Specifically, these are:
- DCIM Tools
- Server Digital Power Management Module
- Demand Response Management Tools
As such, there are future market opportunities for such energy related work. However, we are still evaluating its commercial potential.
Impact
- Code
- Blog posts
- A Web Application to Monitor and Understand Energy Consumption in an Openstack Cloud, Oct 2014
- Understanding the relationship between ceilometer processor utilisation and system energy consumption for a basic scenario in Openstack, Sept 2014
- Collecting energy consumption data using Kwapi in Openstack, August 2014
- Profiling the Ceilometer API to Identify Performance Bottlenecks, August 2014
- Understanding how server energy consumption varies with CPU-bound workload, July 2014
- Advanced Queries to Ceilometer with a mongo backend, July 2014
- Migration of Ceilometer Energy Consumption Data from Havana/MySQL to Icehouse/MongoDB, July 2014
- Authenticating the python ceilometer client against the Openstack APIs – bloody lambda functions!, May 2014
- Related projects
Architecture
TBA.
Implementation Roadmap
This work has largely resulted in a live demonstrator. At present, there is not a significant effort to add more features and capabilities.
The current tasks on the roadmap are:
- Ensure system is live – maintenance task
- Periodically review energy consumption
- Review usage of cloud resources and determine the amount of resources necessary to support this amount of utilization; thus the potential energy saving can be determined.
- Promote the tool somewhat
- Presentation at next Openstack Meetup
- Investigate deployment opportunities
Contact
- Seán Murphy (murp@zhaw.ch)