Ice, Ice, Baby…installing Icehouse with Mirantis OS 5.0

The release of Icehouse brought a few enhancements that were of particular interest in our work – notably Ceilometer/Telemetry developments both in terms of data models  as well as performance and improved support for live VM migration. As we had a positive experience with installing Openstack using Mirantis Openstack 4.x (based on Fuel 4.x), we thought it would be worth having a go at upgrading to Icehouse with Mirantis Openstack 5.0 (of which Fuel 5.0 is a key component) on a small set of servers. Here’s a short note on how it worked out.

First off, we did not read the release notes, which we probably should have. Fuel 5.0 has been quite significantly rearchitected with many of the services (eg rabbitmq, postgresql, nginx etc) now running on Docker as opposed to directly within the Fuel Master CentOS itself.

This meant that it took a very long time for the Fuel Master to boot (abt 30 minutes). To be fair, the boot process indicates that it may take some time, but it was very surprising to us that it took so long. It sequences through all the services that are being put in containers at quite a leisurely pace, giving many notifications that filesystems have been mounted with ordered data mode.

As well as the installation issues with Docker, we found that the Fuel Master (in the default configuration) seemed a little unstable/unreliable: we rebooted the it a few times and we noticed that quite a few services ended up in some strange states, ultimately resulting in a quite ugly shutdown (services not terminating cleanly and having to be killed). Also, we noticed at one point that the Fuel Master DHCP server was not responding, but we did not track specifically when this broke.

Of course, it is a very complex system and for sure such a rearchitecting requires time to mature. In our case, things were probably not helped by doing the installation on a system which was completely isolated and had no DNS server nor NTP server.

After a couple of tries, we did get it working. We ran the configuration tool which is part of the initial boot – it was more explicit in Fuel 4.x and we didn’t notice it the first time we booted Fuel 5.0. We moved the PXE boot off docker and onto the eth0 interface; similarly, we moved the DHCP service off docker onto eth0. Then, it ran through the installation process without any problem. (We’re not sure if we had to do this, but given the issues noted above, we felt it better to use this approach).

We also had a couple of issues with switch configuration (of course) and timeouts in the installation – deploying CentOS to the machines under Fuel control timed out on occasion: we just tried again and it worked. We are running this on an older switch, so perhaps that is giving rise to some of the problems.

Overall, the experience was not as straightforward as with version 4.x, but we did do more of the background prep when we tried v4 for the first time. Generally, it’s still much easier than doing it from scratch.

Now to have a look at what this baby can do…this ice, ice, baby.