Author: Piyush Harsh (page 2 of 2)

Most suitable language for SDK development for SDN?

We at ICCLab are embarking upon an exciting project to make a software development kit to enable SDN researchers develop exciting products and innovative protocols overcoming the challenges and drawbacks of decades old network protocols in use today. We had a huge debate internally to decide which programming language to use for this development. Since, internally we had quite strong and vocal supporters of both Java and Python, it led to stalemate. So how did we resolve it?

Continue reading

Managing hosts in a running OpenStack environment

How does one remove a faulty/un/re-provisioned physical machine from the list of managed physical nodes in OpenStack nova? Recently we had to remove a compute node in our cluster for management reasons (read, it went dead on us). But nova perpetually maintains the host entry hoping at some point in time, it will come back online and start reporting its willingness to host new jobs.

Normally, things will not break if you simply leave the dead node entry in place. But it will mess up the overall view of the cluster if you wish to do some capacity planning. The resources once reported by the dead node will continue to show up in the statistics and things will look all ”blue” when in fact they should be ”red”.

There is no straight forward command to fix this problem, so here is a quick and dirty fix.

  1. log on as administrator on the controller node
  2. locate the nova configuration file, typically found at /etc/nova/nova.conf
  3. location the ”connection” parameter – this will tell you the database nova service uses

Depending on whether the database is mysql or sqlite endpoint, modify your queries. The one shown next are for mysql endpoint.

# mysql -u root
mysql> use nova;
mysql> show tables;

The tables of interest to us are ”compute_nodes” and ”services”. Next find the ”host” entry of the dead node from ”services” table.

mysql> select * from services;
| created_at          | updated_at          | deleted_at | id | host              | binary           | topic       | report_count | disabled | deleted | disabled_reason |
| 2013-11-15 14:25:48 | 2014-04-29 06:20:10 | NULL       |  1 | stable-controller | nova-consoleauth | consoleauth |      1421475 |        0 |       0 | NULL            |
| 2013-11-15 14:25:49 | 2014-04-29 06:20:05 | NULL       |  2 | stable-controller | nova-scheduler   | scheduler   |      1421421 |        0 |       0 | NULL            |
| 2013-11-15 14:25:49 | 2014-04-29 06:20:06 | NULL       |  3 | stable-controller | nova-conductor   | conductor   |      1422189 |        0 |       0 | NULL            |
| 2013-11-15 14:25:52 | 2014-04-29 06:20:05 | NULL       |  4 | stable-compute-1  | nova-compute     | compute     |      1393171 |        0 |       0 | NULL            |
| 2013-11-15 14:25:54 | 2014-04-29 06:20:06 | NULL       |  5 | stable-compute-2  | nova-compute     | compute     |      1393167 |        0 |       0 | NULL            |
| 2013-11-15 14:25:56 | 2014-04-29 06:20:05 | NULL       |  6 | stable-compute-4  | nova-compute     | compute     |      1392495 |        0 |       0 | NULL            |
| 2013-11-15 14:26:34 | 2013-11-15 15:06:09 | NULL       |  7 | 002590628c0c      | nova-compute     | compute     |          219 |        0 |       0 | NULL            |
| 2013-11-15 14:27:14 | 2014-04-29 06:20:10 | NULL       |  8 | stable-controller | nova-cert        | cert        |      1421467 |        0 |       0 | NULL            |
| 2013-11-15 15:48:53 | 2014-04-29 06:20:05 | NULL       |  9 | stable-compute-3  | nova-compute     | compute     |      1392736 |        0 |       0 | NULL            |

The output for one of our test cloud is shown above, clearly the node that we want to remove is ”002590628c0c”. Note down the corresponding id for the erring host entry. This ”id” value will be used for ”service_id” in the following queries. Modify the example case with your own specific data. It is important that you first remove the corresponding entry from the ”compute_nodes” table and then in the ”services” table, otherwise due to foreign_key dependencies, the deletion will fail.

mysql> delete from compute_nodes where service_id=7;
mysql> delete from services where host='002590628c0c';

Change the values above with corresponding values in your case. Voila! The erring compute entries are gone in the dashboard view and also from the resource consumed metrics.

Floating IPs management in Openstack

Openstack is generally well suited for typical use cases and there is hardly reasons to tinker with advance options and features available. Normally you would plan your public IP addresses usage and management well in advance, but if you are an experimental lab like ours, many a times things are handled in an ad-hoc manner. Recently, we ran into a unique problem which took us some time to figure out a solution.

We manage a full block of 255 public IP addresses. Due to an underestimated user demand forecast, in our external cloud we ended up with a floating-ip pool that was woefully inadequate. One solution was to remove the external network altogether and recreate a new one with the larger floating-ip pool. The challenge was – we had real users, with experiments running on our cloud and destroying the external network was not an option.

So here is what we did to add more floating ips to the pool without even stopping or restarting any of the neutron services –

  1. Log onto your openstack controller node
  2. Read the neutron configuration file (usually located at /etc/neutron/neutron.conf
  3. Locate the connection string – this will tell you where the neutron database in located
  4. Depending on the database type (mysql, sqlite) use appropriate database managers (ours was using sqlite)

I will next show you what to do to add more IPs to the floating pool for sqlite3, this can be easily adapted for mysql.

$ sqlite3 /var/lib/neutron/ovs.sqlite
SQLite version 3.7.9 2011-11-01 00:52:41
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .tables

The list of tables used by neutron dumped by the previous command will be similar to –

agents ovs_tunnel_endpoints
allowedaddresspairs ovs_vlan_allocations
dnsnameservers portbindingports
externalnetworks ports
extradhcpopts quotas
floatingips routerl3agentbindings
ipallocationpools routerroutes
ipallocations routers
ipavailabilityranges securitygroupportbindings
networkdhcpagentbindings securitygrouprules
networks securitygroups
ovs_network_bindings subnetroutes
ovs_tunnel_allocations subnets

The tables that are of interest to us are –

  • ipallocationpools
  • ipavailabilityranges

Next look into the schema of these tables, this will shed more light into what needs to be modified –

sqlite> .schema ipavailabilityranges
CREATE TABLE ipavailabilityranges (
allocation_pool_id VARCHAR(36) NOT NULL,
first_ip VARCHAR(64) NOT NULL,
last_ip VARCHAR(64) NOT NULL,
PRIMARY KEY (allocation_pool_id, first_ip, last_ip),
FOREIGN KEY(allocation_pool_id) REFERENCES ipallocationpools (id) ON DELETE CASCADE
sqlite> .schema ipallocationpools
CREATE TABLE ipallocationpools (
subnet_id VARCHAR(36),
first_ip VARCHAR(64) NOT NULL,
last_ip VARCHAR(64) NOT NULL,

Next look into the content of these tables, for brevity only partial outputs are shown below. Also I have masked some of the IP addresses with xxx, replace these with real values when using this guide.

sqlite> select * from ipallocationpools;
sqlite> select * from ipavailabilityranges;

Looking at the above two outputs, it is immediately clear what needs to be done next in order to add more IPs to the floating-ip range.

  1. modify the floating-ip record in the ipallocationpools table, extend the first_ip and/or last_ip value(s)
  2. for each new ip address to be added in the pool, create an entry in the ipavailabilityranges table with first_ip same as last_ip value (set to the actual IP address)

An an example, say I want to extend my pool from to, this is what I would do

sqlite> update ipallocationpools set last_ip='' where first_ip='';
sqlite> insert into ipavailabilityranges values ('7bca3261-e578-4cfa-bba1-51ba6eae7791', '', '');
sqlite> insert into ipavailabilityranges values ('7bca3261-e578-4cfa-bba1-51ba6eae7791', '', '');
sqlite> .exit

And that’s all, you have 2 additional IPs available for use from your floating-ip pool. And you don’t even need to restart any of the neutron services. make sure that the subnet id is the same as in the ipallocationpools table entry.

Rating, Charging, and Billing for the Clouds

Cloud has revolutionized the way we think of computing now. Now everything is on-demand, self-service, pay-as-you-go, and scalable. Although, these are welcome features that tremendously reduces the CAP-Ex and OP-Ex for any business, but the true potential of the clouds in regards to novel rating-charging and billing potentials has yet to be realized. Infrastructure clouds are being treated as commodity now. Much of the innovation is now shifting towards platform and software services over infrastructure clouds.

Telecom domain has always seen lots of innovations in this regard – different tiers of pricing, bundled services, numerous packages, and what not. And think about it – they only offer in reality just 1 type of service namly voice traffic. They have standardized their protocols, they even have a standard to facilitate charging and billing a.k.a Diameter. Not downplaying the significance of their innovation, the standards are needed as the consumers are mobile and they roam from one business domain into another – and unless their interfaces, user-equipment radios, accounting is streamlined (read standardized) it would be almost impossible to support the demands of a modern telco consumer.

So now the question to ask is – is there a need to replicate what has been done in the telecom world for the cloud services? I tend to lean towards a NO. The needs of the cloud consumers are not same as a telco consumers. By keeping things manageable and simple – cloud providers can keep costs low which further reinforces the USP of the clouds – simplicity and lower costs. The computations in the cloud are generally not mobile – there is typically no need. By virtue of the Internet – a computation could take place in any part of the world and there is a complete delinking of customers’ actual location and where the services are being offered from, as long as certain broad-ranging SLAs are satisfied. Therefore there is no real need for cloud hardware systems to implement complex plethora of standards. And that could be argued for the charging and billing strategies too (there is a need for standards in the consumer facing management interfaces for mitigating vendor lock-ins, but I reserve it for another blog post for another day).

A unified and simple billing strategy would work wonders for the consumers. But one also should be mindful of willy-nilly application of such a strategy. There is a need to justify the cost to the consumer as well as the cost to the provider. And hence a proper rating-charging engine is desired. And that is where there is still a lot of room for innovation in the world of clouds. Modern infrastructure management stacks including CloudStack and OpenStack already include monitoring models and metering functions in their core offering. These must be utilized by the provider in determining the real cost of operating their cloud infrastructure, and then tie this to the cost offered to the consumers.

A proper rating-charging engines would really help the providers make a sound judgement in this aspect. There are already numerous open source products including jBilling,,, etc. in the market. The “open” part is a severely crippled offering either providing simple billing interfaces, or features without ability to compute and process “usage data records” (UDRs). There is a real need for a true open-source platform that would allow numerous cloud services to accurately undertake rating-charging activity for their customer so that they can keep the billing model accurate, simple but “no simpler”.

If we now consider PaaS services – there is a scope to do lot more with a unified RCB engine. The umbrella of metrics of interest would be dependent on the platform services and therefore the RCB engine must be adaptive for such systems. The built-in measurement systems have to be evaluated and proper metering mechanisms enabled / provided.

In summary, rating-charging-billing innovations are necessary for several reasons (money, money, and more money …), and the innovations in the world of the clouds are just starting.

ICCLab presents Interoperability and APIs in OpenStack @ EGI Technical Forum, Cloud Interoperability Week

The ICCLab was invited to give a talk on Interoperability and APIs in OpenStack at EGI Technical Forum which was co-located with Cloud Interoperability Week. The workshop and hands-on tutorial sessions took place between September 18-20, 2013 in the beautiful city of Madrid.

The presentation sessions were followed by panel discussion where all the speakers entertained several questions from the audience. There was substantial interest in the audience with respect to OCCI development roadmap and questions were also raised on suitability of one cloud standard against another.

In the tutorial sessions that followed the workshop, there were several projects that demonstrated their use of the OCCI standard. Notable among them were OpenStack, OpenNebula, and CompatibleOne.

[slideshare id=26782772&doc=interoperabilityandapisinopenstack-131002072248-phpapp01]

Custom branding your OpenStack testbed

Of all the IaaS software stack available, the one gaining rapidly in popularity is OpenStack. It has a huge community behind it and it enjoys support from several large IT vendors therefore ensuring a clear and long development and support roadmap ahead. Naturally it makes a logical deployment choice by numerous cloud vendors across the globe.

If you have your own OpenStack deployment, in this guide I will show you how to quickly perform a custom rebranding using some minor configuration tweaks.

If you have a OpenStack Grizzly testbed, then most likely your Horizon dashboard looks something like this:
Default OpenStack Grizzly dashboard on Ubuntu

First of all you should remove the default Ubuntu theme. Perform the steps below as superuser.

sudo su
dpkg –purge openstack-dashboard-ubuntu-theme

Next you should prepare two transparent logos, ideally of dimensions 200×27 and 365×50, lets call them logo_small.png and logo_medium.png.

Upload these images to the OpenStack images folder on the node where Horizon is installed.


Next you should create a small custom css file that would bypass the OpenStack default rendering. You should put the custom css file under this directory on the Horizon node. Lets say that you name the css file as custom.css.


Here is the example css file content that I have used:

* New theme colors for dashboard that override the defaults:
* By Preston Lee
h1.brand {
background: #DAEDE9 repeat-x top left;
border-bottom: 2px solid #BAD3E1;
h1.brand a {
background: url(../img/logo_small.png) top left no-repeat;
#splash .login {
background: #DAEDE9 url(../img/logo_medium.png) no-repeat center 35px;
#splash .login .modal-header {
border-top: 1px solid #BAD3E1;
.btn-primary {
background-image: none !important;
background-color: #7EC4B7 !important;
border: none !important;
box-shadow: none;
.btn-primary:active {
border: none;
box-shadow: none;
background-color: #66AB9F !important;
text-decoration: none;

Next edit the file using the text editor of your choice:


Simply add this line at the end of the file


And finally restart your httpd service and see the change in the dashboard page. For ubuntu use this command:

service apache2 restart; service memcached restart

Here is how the transformed dashboard page looks for our lab’s testbed setup:

New Horizon dashboard page

Adapted from a blog post by Preston Lee.

Making money from your cloud services – Part 2

A thoughtful pricing strategy goes a long way!

In my previous post, I tried to explain in brief the various steps that makes (or ought to make) up a general business accounting workflow. The process described in that post was adapted for the cloud service business context. Staying with the same theme, in this post I will try to show some light on the next logical step in the service monetization process.

The holy grail – the optimal pricing strategy

It is very important to have the correct pricing strategy in place. In an arena which is getting increasingly crowded, it is very important to stand out from the rest. You can either create a cloud service offering which is truly outstanding in nature, but in this era of fast technology changes and cutthroat competition, very soon you will find your innovative space being encroached by copycats and other competitors. You will have to keep on innovating, and in step – you must also differentiate your service in terms of an attractive pricing for end users. This does not necessarily imply offering lower prices, but having a smarter pricing model in place for the full spectrum of consumers you cater to (enterprises, individuals, public organizations, governments, not-for-profit groups, etc.). The design of your optimal pricing strategy will be very much service model dependent, so I will not try to prophesize a universal pricing strategy. If I dare say that I have a universal strategy in place – I would be lying. Therefore in this post I will cover the various pricing strategies to help you design a custom strategy most suitable for your cloud service offerings.

Types of pricing strategies

A carefully thought out pricing strategy would help a provider differentiate themselves from their competitors. One must conduct periodic reviews of their pricing model to take into account the ever changing business environment and changing customer expectations. With that in mind I will list the different strategies that you could adopt in your financial processes.

  • Time based pricing – you charge your consumers based on how long they have been using your services
  • Volume based pricing – you charge your consumers based on the volume of the metric that has been consumed (for ex. amount of bytes sent/received over the network)
  • Quality of Service (QoS) based pricing – if you offer differentiated services to your end users, then you could have differentiated pricing based on the different QoS offered
  • Flat rate pricing – you offer your services at a fixed rate for a fixed duration of time regardless of the volume of consumption.
  • Paris-metro pricing – in this pricing strategy you offer two or more different services offering the exact same service configuration and exact same QoS guarantees but with different pricing levels. But you reserve dedicated resources for the two offerings. The idea behind the strategy is that the higher priced service will be used less and therefore whoever is using the higher priced service will have a better experience due to low load.
  • Priority pricing – in this you charge differently depending on the levels of priority assigned to end-users’ tasks. Normally a higher priority request will get charged at an elevated rate.
  • Smart market pricing – in this model you let the market demand and supply decide the prices. In other words – your services will be priced on auctioning. This model is similar to Amazon spot pricing.
  • Edge pricing – the service pricing depends on the distance of the user from the service.
  • Responsive pricing – in this model, the pricing is activated only when there is congestion in the system.
  • Proportional fairness pricing – in this model the service prices are set keeping in mind the users’ willingness to pay and costs involved in service optimization.
  • Cumulus pricing – in this model, the user agrees to pay a flat rate for service consumption upto a certain limit, over which an added charge is levied on the extra consumption. In some sense it is similar to a dynamic pricing using a credit point system.
  • Session oriented pricing – the price is determined based on the nature of the task being conducted in a particular session.
  • One-off pricing – in this model, you charge a small amount for each session, for example – this could include the cost to perform inter-domain network orchestration for each service instantiation/setup.
  • Time of day based pricing – in this model, you set a different price for your service depending on different hours the service is being consumed (for example – daytime vs. nighttime rates charged by various utility companies around the world).

Of course, there is no one correct model. You should feel free to devise your own pricing strategy using any one or even a combination of several models described above. Since pricing is an important piece in the overall puzzle, this decision is not to be made hastingly. Let me also list the various variations in the pricing strategies that I have already listed above. Many of these are self-explanatory so I will simply list those.

  • Free of charge model
  • Periodical fees model
  • Discounts
  • Pre-paid pricing model – in fact implementing a pre-paid pricing scheme for a cloud services can prove a bit challenging (more on that will be covered in a future blog post).
  • Post-paid pricing model
  • Online accounting – in this model, you do customer accounting in a real-time basis, this could be beneficial in the sense that as a provider, you have a real time view of your resources utilization pattern and you can react to demand variations quickly.
  • Offline accounting – You store your consumers usage metrics and process them periodically at a later time. This style of operation is simpler but you lose the ability to view your users’ real time consumption behavior.
  • Static pricing – in this model, you fix the pricing function (refer to my previous blog) for the computation of charge records.
  • Dynamic pricing – as the name suggest, the pricing function is not static but changes depending on the operating environment trigger conditions and is more suitable to react to congestion in your system as the prices of your service will adjust accordingly.

So where do we go from here? Now that you are aware of the basic accounting process and various pricing strategies / models, hopefully you are more equipped now to differentiate your cloud services from your competition. Of course – you must keep yourself aware of the market situations and your consumers’ expectations. And as a general rule – never stop innovating!

So moving ahead I will try to put together a general architecture that will throw some light on how to provision a system to bill your end users and have a cash-inflow to support your business. Until then – ciao!

Making money from your cloud services

Step 1 – Careful financial planning and understanding the financial process

Here at ICCLab, we do applied research covering the full spectrum of cloud services stack – Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and also Software as a Service (SaaS). Our researchers have been doing extensive research in automating the cloud stack installation and management, making your services highly available and dependable by identifying critical services in your offering. At the end of the day – everyone requires sufficient financial flow to keep their services up and running. All of us love doing ground-breaking, challenging applied research involving cutting-edge technologies, but it is also important to differentiate our service offerings from potential competition. Given the huge interest that cloud computing has generated among all stakeholders in any economic setting – it is not farfetched to assume that any novel cloud service you can come up with, very soon you will have to deal with potential competition in your service space. So the question naturally arise – how do I differentiate my service from my competitors? In this first post of a several part series we will try to take a baby step towards just that!

Understanding the business accounting process

Since most of us are pure technologists, we tend to overlook the importance of careful financial planning. Having a general understanding of the accounting process could help us create a sustainable business model for the cloud service offerings. So let us very briefly take a peek into a typical accounting process.

A typical financial process can be nicely explained using the flow diagram shown below.
Accounting process

The key elements of the accounting process are –

  • Metering – this refers to the process of collecting resource usage metrics of your customers. This step is critical and must be planned carefully as these metrics provides you the basis on which you bill your end users.
  • Mediation – it refers to the process of assimilating different metering records from various meters into a general meter-agnostic accounting records. This step helps transform disparate data into a form which is somewhat uniform and makes the rest of the elements of the accounting process metering strategy agnostic.
  • Accounting – within the overall accounting process, the accounting box task includes long term storage of the accounting records. It also houses the logic to combine several accounting records into a session record – which would entail identifying all accounting records belonging to the same customer session and then aggregating them into one or several session records.
  • Pricing – it refers to the various strategies that in the end generates the appropriate pricing function that would be applied to different session records. The pricing module needs a bit of thought to implement, and we will cover a bit more about it later in the series.
  • Charging – this is simply the process of applying the pricing functions to the session records in order to generate the charge records. This is the process that ultimately assigns the monetary values to end users’ consumption of your cloud services.
  • Cloud Bursting – in case you have agreements with other cloud service providers to migrate their peak load into our own environment, you will at some point in time bill them for the use of your resources. Depending on special agreements that could be in place between the two providers, the interdomain billing process will be different. The translation of session records into an interdomain format and handling of inter-domain bills is done in this module.
  • Billing – this refers to the process of combining the various charge units of the consumer since the last billing cycle into an easily comprehensible bill that will be sent to the users for payment. The final bill format could depend on the billing preferences such as consolidated view vs. itemized view and possibly other parameters.
  • Financial Clearing – it refers to the actual money collection from the customers, it could include processes such as handling of automatic clearing house (ACH), credit card transactions, direct banking channels, etc. Once the customer pays the bill, the billing cycle is considered closed.

Now that we have learned the high level view of a general accounting process, in the next post of the series we will focus on one of the more critical elements of the whole process – pricing!

Rating, Charging, Billing

This page is kept for archiving. Please navigate to our new site:


Financial accounting is a very critical process in the monetization process of any service. In the telecommunication world, these processes have long been documented, used, and standardized. Cloud computing being a relatively new paradigm, is still undergoing a transition phase. Many new services are being defined and there is still a huge untapped potential to be exploited.

Rating, Charging, and Billing (RCB) are key activities that allows a service provider to fix monetary values for the resources and services it offers, and allows it to bill the customers consuming the services offered.

Problem Statement

Given a general service scenario, how can the key metrics be identified. The identification of measurable metrics is essential for determining a useful pricing function to be attached to the metric. The challenges we are trying to address under this initiative are multi-dimensional. Is it possible to come up with a general enough RCB model that can address the needs of multiple cloud services – IaaS, PaaS, SaaS, and many more that would be defined in the future?

Where is the correct boundary between real-time charging strategy, which could be very resource intensive, versus a periodic strategy which has the risk of over-utilization of resources by the consumers between two cycles? Can a viable middle-path strategy be established for cloud based services. Can pre-paid pricing model be adapted for the cloud?

Simplified workflow



User Data Records
Rating & Charging


  • rule engine and pricing strategies
  • prediction engine and alarming
  • revenue sharing and SLAs
  • usage collectors
  • scalability


  • vBrownBag Talk, OpenStack Summit, Paris, 2014

  • Swiss Open Cloud Day, Bern, 2014

  • CYCLOPS Demo


  • OpenStack Meetup, Winterthur, 2014

Articles and Info

Research publications

Technology transfer

Research Approach

Following the ICCLab research approach



  • icclab-rcb-cyclops[at]dornbirn[dot]zhaw[dot]ch


Piyush Harsh

This page is kept for archiving. Please navigate to our new site:

Image of Dr. Piyush Harsh Piyush Harsh is a researcher in ZHAW InIT Cloud Computing Lab. His research interest includes distributed self-* capable systems, nature inspired computing, authentication protocols, cloud computing, to name a few. Before joining ICCLab, Piyush was the development coordinator of WP5 in FP7 project Contrail and was instrumental in the development of cloud-middleware tool Virtual Execution Platform.

Piyush received his bachelors in Computer Science from Indian Institute of Technology and then received his MS and PhD degrees in Computer Engineering from Department of Computer and Information Science and Engineering at University of Florida. In the past Piyush worked as research engineer at INRIA Rennes Bretagne-Atlantique research center.

At ICCLab, Piyush is involved in FP7 IPs Mobile Cloud Networking and T-Nova projects.

Contact: piyush.harsh /at/
More Details:

Newer posts »