ROSCon 2017 – Vancouver

For third time in a row we attended ROSCon, this year held in beautiful Vancouver.
Our goals besides seeing the newest trends in the ROS and Robotics universe first hand, and finding some new robotic hardware directly from manufacturers, was to support our partners from Rapyuta Robotics (RR) in presenting and performing a demo of the first preview of their upcoming Cloud Robotics Platform.

Continue reading

Paraguayan Perspective on Cloud Applications

The Itaipu Technology Park (PTI) in Paraguay, founded in 2009, is involved with scientific and technological development which contributes positively to regional development. Several of its centres with a total number of 90 engineers and researchers put emphasis on ICT integration and the challenges connected with it. Among them are diverse plans to use cloud applications. In this context, the Service Prototyping Lab of Zurich University of Applied Sciences (SPLab) in Switzerland is conducting a two-week guest lecturing and research exchange presenting its research initiatives and outputs on the PTI premises in close proximity to Ciudad del Este.

Continue reading

PyParis’17 conference report

PyParis is a community-organised conference on all topics around the Python programming language. The expected target group are primarily practitioners and researchers in the greater capital region of France, but also international engineers and language advocates. At Zurich University of Applied Sciences, Python is taught as automation and statistics application language to more than 200 business engineering, aviation and traffic engineering undergraduates per year. It is furthermore used a lot in research, including several prototypes resulting from the Service Prototyping Lab. Therefore, it was consequential for us to attend the conference and to contribute an in-depth tutorial on one of our research topics, Function-as-a-Service, to its attendees.

Continue reading

How to Write a Cinder Driver

After too many hours of trial and error and searching for the right solution on how to properly write and integrate your own backend in cinder, here are all the steps and instructions necessary. So if you are looking for a guide on how to integrate your own cinder driver, look no further. Continue reading

Running Google Cloud Functions in OpenShift

Have you created a highly popular and frequently used JavaScript (Node.js) functions for execution in Google Cloud Functions? In this case, the economics of FaaS turn against you due to the per-invocation pricing. You might want to have more options for both testing the same function locally and for deploying it into an environment with fixed monthly pricing. This blog post explains step-by-step how to migrate functions from FaaS environments into a fixed per-month pricing container environment. The running example will be Node.js functions running in Google Cloud Functions albeit the procedure is similarly applicable to other combinations.

Continue reading

15th OpenStack Meetup

On the 21st of March we held the 15th OpenStack meetup. As ever, the talks were interesting, relevant and entertaining. It was kindly sponsored by Rackspace and held at their offices in Zürich. Much thanks goes to them and to previous sponsors!

At this meetup there were 2 talks and an interactive and impromptu panel discussion on the recent operator’s meetup in Milan.

The first talk was by Giuseppe Paterno who shared the experience in eBay on the workloads that are running there upon OpenStack.

Next up was Geoff Higginbottom from Rackspace who showed how to use Nagios and StackStorm to automate the recovery of OpenStack services. This was interesting from the lab’s perspective as much of what Geoff talked about was related to our Cloud Incident Management initiative. You can see almost the same talk that Geoff gave at the OpenStack Nordic Days.

The two presentations were followed up by the panel discussion involving those that attended  including our own Seán Murphy and was moderated by Andy Edmonds. Finally, as is now almost a tradition, we had a very nice apero!

Looking forward to the next and 16th OpenStack meetup!

Snafu – The Swiss Army Knife of Serverless Computing

The Service Prototyping Lab at Zurich University of Applied Sciences is committed to advancing the state of technology for bringing applications to the cloud, for the benefit of the society of large in general and of the local industry in particular. This obliges us to closely monitor industrial trends along with academic advances. A hot topic currently found in both is the higher-PaaS-level service class of FaaS, or Function-as-a-Service, which coincides with the marketing term Serverless Computing. We have already contributed analytical work on finding the limits and possibilities of today’s FaaS systems (preprint), and engineering work on translating legacy monolithic code into fine-grained functions (preprint). It was only a matter of time until the limits in both commercially operated FaaS services and open-source FaaS prototypes became too severe for our work. Hence, after a careful analysis of what is available, we decided to come up with an alternative FaaS host process design. The design led to an architecture, and the architecture eventually to an implementation called Snafu. This post presents Snafu and positions it as Swiss Army Knife for situations in which functions should be prototyped, tested or hosted.

Continue reading

NetSys’17 conference report

NetSys is a regular biennial conference covering networked and distributed systems. This year, NetSys’17 took place in Göttingen, Germany. We have attended and report on some of the research and technology trends, obviously with a focus on our own research directions. Apologies ahead for not giving a full account of the whole conference as our presence was limited by lecturing duties.

Continue reading

Rancher – initial experience report

In the context of the FINEXT project, we have been reviewing Rancher as a tool to support easy deployment of FIWARE components. (Our colleagues in the project have more experience with this tool – we’re still climbing the learning curve). Here are a few observations relating to Rancher.

The primary problem that Rancher solves is management of potentially disparate (sets of) IaaS resources to provide support for deploying containerized applications. Another important aspect of the Rancher vision is the application catalog – a well defined set of containerized applications that can be deployed to a container platform.

This is, of course, a very noisy area with much technology competition: the Rancher team developed their own orchestration framework – Cattle – but it was clear from some time ago that there would be many different orchestration frameworks and they intelligently decided to integrate with other platforms which were gaining traction. Specifically, they provide support for Kubernetes, Swarm and Mesos.

While playing with Rancher to understand how it works, we looked at how it supports three use cases:

  • Deployment of applications on IaaS with Cattle based container management
  • Deployment of applications on IaaS with Kubernetes container management
  • Deployment of applications on IaaS with Swarm container management

Although the Mesos case is also interesting (and we like Mesos!), we decided not to consider it as Mesos does not currently have as much momentum as the other technologies.

The basic Rancher approach

Before discussing our initial observations, it is appropriate to give some details on key concepts in Rancher.

Rancher supports so-called Environments which are defined by a specific orchestration framework (eg Cattle, Kubernetes, Swarm) and comprise of a set of Hosts. Applications can be deployed to Environments from the Application Catalog; obviously, Applications need to be defined in a manner that is compatible with the Environment’s orchestration mechanisms – for Cattle and Swarm, applications can be defined using docker-compose format; Kubernetes environments require a different pod-based format.

Hosts are typically VMs that run in cloud platforms – although it is possible to configure these manually, the intended use case is that these are created by docker-machine. docker-machine contains drivers for many different hosting providers and Rancher leverages these to enable Hosts to be provisioned on a wide range of different providers. Rancher provisioning is quite complex, but generally it comprises of deploying rancher-agent which enables Rancher to monitor and control the host and deployment of an overlay network which enables the Host to network with the other Hosts in the Environment.

The workflow then is one in which rancher-server is first deployed (typically in a VM). In rancher-server an Environment is created, Hosts are added to the environment and then Applications can be deployed from the catalog. Note that rancher-server can – typically should – manage multiple different Environments. Rancher provides good support for monitoring the state of the system: for example, it is straightforward to see all containers running on the Hosts, their logs and if they are in an error state.

Standard Cattle Management

Standard Cattle Management is the most developed Rancher capability. In this mode, Cattle – running in rancher-server – is responsible for orchestrating the application on the host. rancher-agent runs on each of the Hosts in privileged mode and hence it has the power to create and destroy containers on each Host. rancher-server communicates with rancher-agent over a websockets connection to obtain the state of the host.

The Application Catalog for this mode is well developed. Rancher comes with a default Application Catalog and supports import of applications to the catalog. Further, it supports the use of private docker registries as it is clear that many applications would not be public. In the Cattle Environment, the Application Catalog is evident (a menu item at the top of the screen). Applications comprise of three files:

  • docker-compose.yml: contains the standard information for launching and managing a multi-container service
  • rancher-compose.yml: which contains the descriptor for the application in the service catalog, what parameters it requires as well as information pertaining to deploying the application over multiple VMs, scaling, health checks etc
  • Answers.txt: contains the default values of the parameters required to launch the application

We did not experiment much with Cattle orchestration, but documentation indicates that it is a sensible orchestration framework which deploys applications in a balanced manner across the Hosts in the Environment.

Rancher with Kubernetes

Rancher also supports Environments based on Kubernetes. As such, Rancher supports rapid and easy deployment of a Kubernetes cluster across disparate hosts: the Environment is created in Rancher and Hosts are added via docker-machine.

As with the Cattle deployment, rancher-agent is deployed on all nodes in the cluster: this enables Rancher to have full visibility of each of the nodes in the Environment – what containers are running etc. It is also used in the process of deploying the Kubernetes environment.

It took us some time to understand the role of the Application Catalog in the Kubernetes context. Although Rancher has some support for a Kubernetes Application Catalog and the Catalog differs from that available for standard Cattle Environments – applications are described in terms of pods – we found that deploying these applications did not work.

The Kubernetes cluster was deployed successfully and usable. Rancher offers a web-based CLI by which applications can be deployed to the cluster (both with kubectl and helm); applications can also be deployed outside of the Kubernetes interface of course with Rancher making the Kubernetes credentials available which can be used by kubectl.

Rancher with Swarm

Rancher support for Swarm is similar to that of Kubernetes in the sense that the primary focus is on managing the Hosts in the Swarm Environment. Rancher provides support for bringing up a Swarm and enabling it to be controlled via the standard Swarm toolset.

It is worth noting that we did have some confusion working with Applications in Swarm. The Swarm deployment mode has the capability to deploy applications from a catalog, although it is not so prominent in the interface. It took us some time to realize that this was not the intended deployment mode – this mechanism uses Cattle for the application deployment rather than Swarm. This was non-obvious – the applications were docker-compose applications and, as such, we assumed that they could be deployed via Swarm. Deploying the applications appeared to work in the sense that they were visible in Rancher, but on closer inspection we found irregularities. Specifically, the application was not deployed in ‘managed’ mode, even though this was stipulated in the Application Catalog; also, docker service ls did not show the application.

The limitations noted above most probably arise because Swarm support is still experimental and will be resolved as the solution matures.

Another noteworthy point relating to Swarm usage is that Rancher provides a very useful interface to both the containers and nodes within the Swarm: this can be used to understand current state and perform troubleshooting. Unlike the Kubernetes environment, the Swarm environment has no such standard tool and Rancher provides significant value add here.

Final Comments

Rancher is a useful and interesting evolving platform. It focuses primarily on the important problem of bridging between the classical VM/IaaS world to the newer container ecosystems; another important aspect of the Rancher vision is application management. As the world of container ecosystems is evolving rapidly – with some technologies offering key parts of Rancher’s vision – it will be challenging for Rancher to span all aspects of application management from VM management to container management to application deployment and management, but the technology has obtained some momentum and solves a real problem and hence it’s likely to be around for a while.

(Thanks to Bruno and Martin for reviewing this!)

Openstack checkpointing is simplified

At ICCLab, we have recently updated the Openstack OVA onboarding tool to include an exporting functionality that can help operators migrate and checkpoint individual VMs. Furthermore, researchers can now export VMs to their local environments, even use them offline, and at any time bring them back to the cloud using the same tool.

The OpenStack OVA onboarding tool automatically transforms selected virtual machines into downloadable VMDK images. Virtual machines and their metadata are fetched from OpenStack’s Nova service, and made packed as OVA file. The tool offers a GUI integration with OpenStack’s Horizon Dashboard, but can be also deployed separately.

Continue reading