{"id":8946,"date":"2015-09-25T17:54:42","date_gmt":"2015-09-25T15:54:42","guid":{"rendered":"http:\/\/blog.zhaw.ch\/icclab\/?p=8946"},"modified":"2015-09-25T18:04:57","modified_gmt":"2015-09-25T16:04:57","slug":"initial-experience-with-openstack-nova-cells","status":"publish","type":"post","link":"https:\/\/blog.zhaw.ch\/icclab\/initial-experience-with-openstack-nova-cells\/","title":{"rendered":"Initial Experience with Openstack Nova Cells"},"content":{"rendered":"

In the GEYSER project, we are examining suitable Openstack architectures for our pilot deployments. In an earlier blog post<\/a> we described different ways to <\/a>architect \u00a0an Openstack deployment mostly focusing on AZ (Availability Zone) and Cells (those were the only options available back in 2013). Much has changed since then and new concepts were added such as regions and host aggregates. Even though Cells have been available since Grizzly they are still considered experimental due to lack of maturity and instability. In this blog post we describe our experience enabling Cells in an experimental Openstack deployment.<\/p>\n

Why Cells?<\/strong><\/h4>\n

The documentation<\/a> says that \u201cCells functionality enables you to scale an OpenStack Compute cloud in a more distributed fashion without having to use complicated technologies like database and message queue clustering. It supports very large deployments\u201d. Although we don\u2019t have a large deployment this is pretty much in line with our requirements for our pilot – a distributed system with a single public API exposed. Comparing with other architectural approaches currently available the one which gets closer to this design are regions, but even then is not desirable as it exposes a public API for each region.<\/p>\n

Cells Architecture<\/strong><\/h4>\n

Understanding Nova Cells architecture was somewhat tricky, as the documentation does not make it clear where each service should run and new concepts are introduced – \u00a0the API Cell and the Compute Cell – which do not map obviously to the standard controller\/compute node architectures. While there are a number of articles on getting started with Nova cells (listed below) the clearest picture of its design is presented in the prescriptive example<\/a> of CERN architecture.<\/p>\n

It is important to understand that the API Cell operates as a thin layer managing and redirecting queries to the right Compute Cell; it sits on top of each Compute Cell being the top-level cell of the hierarchical tree exposing a single public API to users.<\/p>\n

Compute Cells are very similar to a classical Openstack deployments comprised of controller and compute nodes and associated resource management. However, instead of running the nova-api service on every controller node a new service – nova-cells – is added which connects its nova-* services to the API Cell via a message bus. This is shown below (source<\/a>).<\/p>\n

\"Generic_CERN_Example\"<\/a><\/p>\n

Converting to Openstack Cells<\/strong><\/h4>\n

For our experimentation we actually didn\u2019t want to redeploy a full environment; we wanted to convert from a typical small controller\/compute nodes configuration to a cell architecture – note that we have installed Juno version following the installation guide<\/a> and all packages and services were installed from the main ubuntu repository. The instructions in this blog post<\/a> provide a clear idea of what has to be done to convert a running Openstack deployment to use nova cells. Our biggest issue was to think that the controller node was supposed to be converted to the API cell, which was the source of some misunderstanding which resulted in reinstalling nova services of a controller node in a compute node(much more work). Rather than converting the controller to an API cell, it is simpler to get a new server and use it as API Cell, or take a compute node and change the installed Openstack services. The picture below illustrates key services running on both API and Compute cell and Controller and Compute nodes. We haven\u2019t worked with other services as yet like ceilometer, neutron and cinder and hence these are not shown (for networking, we used nova-network in each of the cells).<\/p>\n

\"Cells\"<\/a><\/p>\n

Configuring the \u00a0API Cell<\/strong><\/h5>\n

To configure the API call, we had to add the following lines to \/etc\/nova\/nova.conf:
\n
\n[default]<\/code><\/p>\n

rabbit_host=api_rabbit_ip
\nrabbit_port=api_rabbit_port
\nrabbit_username=api_rabbit_user
\nrabbit_password=api_rabbit_password
\nrabbit_virtual_host=api_rabbit_vhost<\/p>\n

[cells]<\/p>\n

enable=True
\nname=API_CELL_NAME
\ncell_type=api<\/p>\n

Configuring the Compute Cells<\/h5>\n

To configure the compute cells we had to add the following lines to \/etc\/nova\/nova.conf:
\n
\n[default]<\/code><\/p>\n

quota_driver=nova.quota.NoopQuotaDriver
\nrabbit_host=cellX_rabbit_ip
\nrabbit_port=cellX_rabbit_port
\nrabbit_username=cellX_rabbit_user
\nrabbit_password=cellX_rabbit_password
\nrabbit_virtual_host=cellX_rabbit_vhost<\/p>\n

[cells]<\/p>\n

enable=True
\nname=CELLX_NAME
\ncell_type=compute<\/p>\n

Finally run this command on the API Cell for each child (Compute) and in each child to connect to the API Cell.<\/p>\n

nova-manage cell create --name cell_name --cell_type child\/parent \\
\n--username cell_user --password cell_passwd --hostname cell_rabbit_host \\
\n--port cell_rabbit_port --virtual_host cell_rabbit_vhost --woffset 1.0 --wscale 1.0<\/code><\/p>\n

Installing from scratch<\/strong><\/h5>\n

Installing a Cell based system from scratch is easier than converting an existing deployment in terms of placing the right services on the right systems, although it probably requires more work setting up the configuration files.<\/p>\n

Conclusions<\/strong><\/h4>\n

In this blog post we presented an overview of Nova Cells and how a basic configuration can be set up. Our initial experimentation shows the system to be quite stable and the design offers much greater flexibility and elasticity to an Openstack deployment – cells can be easily added, changes to the cell hierarchy are straightforward – and also looked reasonably robust in terms of standard operations such as start and remove vms, etc. We are still \u00a0investigating whether Nova Cells meets the requirements of our pilot deployment, so stay tuned as we investigate it further!<\/p>\n

Useful readings:<\/p>\n