Rancher is a container management platform focused on delivering containers on any infrastructure. It has support for multiple environments which could make use of one of the multiple container orchestrators available (at the time of this writing) – Cattle, Kubernetes, Swarm and Mesos. In our previous blog post we showed how we built rancher-agent containers for arm64 systems in a Cattle environment. We made a few improvements since then, mostly on porting a Swarm environment to arm64, and these are documented in this blog post.
In our setup we had a Swarm environment that was configured to have a single manager, the manager was already set up in an x86 virtual machine running in the cloud. We then only had to add arm64 workers to the swarm.
Note that swarm-manager could also run on arm but we decided to use a vm on Openstack to make sure we have a heterogenous environment in Rancher.
Just like in a Cattle environment, a set of infrastructure containers are started after a host is created in a Swarm environment – healthcheck, net, dns and plugin-manager (network manager) – there is one important additional container that’s added on top of those, the swarmkit container. Since most of the work was done a priori, we expected that it would be straightforward to build and run another image for arm64. The main issue we faced was not in building the swarmkit container image itself but in running it and joining the host to the Swarm cluster. As a result, once the container started at its entry point it attempted to join the Swarm, the swarm-manager received the request adding the host to the cluster, however it also immediately removed it.
After a bit of investigation we found out that the swarm manager has a function to remove nodes from the cluster whenever a host is removed from the environment. This function gets triggered immediately after the new host joined and the command
docker node rm $id --force was getting executed. It is still unclear why the new node that just joined the cluster was not considered part of it by the swarm-manager but after few modifications to the script we successfully managed to get a heterogeneous Swarm environment in Rancher!
In order to use our docker image you first need to do a modification in swarmkit-mon stack. This stack has the option to always pull the image before creating the container – you need to deactivate that by unticking a tickbox and upgrading the stack. To do so you will need to select the Swarm stack under “Swarm > Infrastructure”, there will be the option to upgrade the stack, you can also select the amount of managers you would like to deploy.
After this you’re good to go and you can start adding new heterogeneous hosts to your swarm cluster.
One drawback of using a Swarm environment over Cattle is that Swarm does not support privileged containers, and this is crucial if there is any sensor data that needs to be accessed in the RPi. Although it does not support privileged containers, there is a workaround that can be used to access the GPIO board with sysfs.