In FIWARELab, we recently upgraded from Openstack Icehouse to a Kilo High Availability (HA) deployment. Our approach involved having two simultaneous active Openstack deployments so VMs and volumes could be migrated from one deployment to another with a minimal downtime. More specifically, VMs and volumes were snapshotted and transferred to the Kilo deployment as images where they could be recreated by their users. During this process, we found that, VMs originally created from  of CentOS images did not boot properly – their network interface did not come up correctly and the VM was unable to fetch user-metadata.

The root of the issue lies in a standard configuration of CentOS: its device manager (udev) saves a mapping between MAC address and network for security reasons and ensures that the network interface only boots up on that specific MAC address. This mapping is stored in a file which is created when a VM boots for the first time. The file is located in /etc/udev/rules.d/70-persistent-ipoib.rules.

Continue reading