15th OpenStack Meetup

On the 21st of March we held the 15th OpenStack meetup. As ever, the talks were interesting, relevant and entertaining. It was kindly sponsored by Rackspace and held at their offices in Zürich. Much thanks goes to them and to previous sponsors!

At this meetup there were 2 talks and an interactive and impromptu panel discussion on the recent operator’s meetup in Milan.

The first talk was by Giuseppe Paterno who shared the experience in eBay on the workloads that are running there upon OpenStack.

Next up was Geoff Higginbottom from Rackspace who showed how to use Nagios and StackStorm to automate the recovery of OpenStack services. This was interesting from the lab’s perspective as much of what Geoff talked about was related to our Cloud Incident Management initiative. You can see almost the same talk that Geoff gave at the OpenStack Nordic Days.

The two presentations were followed up by the panel discussion involving those that attended  including our own Seán Murphy and was moderated by Andy Edmonds. Finally, as is now almost a tradition, we had a very nice apero!

Looking forward to the next and 16th OpenStack meetup!

ElasTest KickOff Meeting

The most limiting factor in development today is software validation, which typically requires very costly and complex testing processes. It will develop a novel test orchestration theory and toolbox enabling the creation of complex test suites as the composition of simple testing units. The ElasTest project wants to develop an elastic platform for testing complex distributed large software systems.

The ElasTest kickoff meeting took place during January in Madrid. ElasTest’s consortium comprises of 10 partners including IBM, ATOS, Technical University of Berlin and is coordinated by the University of Rey Juan Carlos.

Consortium:

  • Universidad Rey Juan Carlos (URJC)
  • Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. (Fraunhofer)
  • Technische Universitaet Berlin (TUB)
  • Consiglio Nazionale Delle Ricerche (CNR)
  • Fundación Imdea Software (IMDEA)
  • Atos Spain S.A. (ATOS)
  • Zürcher Hochschule Für Angewandte Wissenschaften (ZHAW)
  • Tikal Technologies S.L. (NAEVATEC)
  • IBM Ireland Limited (IBM IRE)
  • Production Trade And Support of Machinable Products of Software and Informatics – Relational Technology A.E. (RELATIONAL)

For more information on the ElasTest project visit our ElasTest section!

Elastest on the EU Portal

ElasTest

An elastic platform for testing complex distributed large software systems.

An elastic platform for testing complex distributed large software systems.

Summary:

The most limiting factor in development today is software validation, which typically requires very costly and complex testing processes. ElasTest pretends to improve significantly the efficiency and effectiveness of the testing process. The project aims to build an elastic platform that eases the testing phase for large and distributed software systems. This is done in order to reduce the time-to-market of the software projects and increase its quality, quality of service (QoS) as well as its quality of experience (QoE). ElasTest (Project ID: 731535) will also develop a novel test orchestration theory and toolbox enabling the creation of complex test suites as the composition of simple testing units (T-Jobs).

ElasTest just started on January of 2017 and its consortium comprises of 10 partners including IBM, ATOS, Technical University of Berlin and is coordinated by the University of Rey Juan Carlos.

In this project ZHAW is going to be working on the development of the ElasTest Platform Manager (EPM) and the ElasTest Test Orchestration and Recommendation Manager (TORM).

The EPM is the interface between the ElasTest testing components (e.g. TORM, Test Support Services, etc.) and the cloud infrastructure where ElasTest is deployed. This platform must abstract the cloud services so that Elastest becomes fully agnostic to them and provide this abstraction via Software Development Toolkits or REST APIs to the users of it. The objective of the EPM is to implement such a Platform Manager that enables ElasTest to be deployed in a target cloud infrastructure (e.g. OpenStack, CloudStack, AWS, etc.).

The TORM is the ElasTest brain and the main entry point for developers. This requires identifying, specifying and implementing a number of interfaces through which the TORM exposes its capabilities to testers. This interface include the following:

  • SuT (Software under Test) specification. Developers need to be able to specify their SuT so that the TORM can execute the tests on it.
  • Engine Hosting. The TORM enables engines to be plugged as modules.
  • Development APIs and interfaces. The TORM is the main entry point for testers, that is why it needs to expose the appropriate interfaces and tools enabling developers to consume the different capabilities exposed by the platform.

Coordinator: Universidad Rey Juan Carlos (URJC)

Consortium:

  • Universidad Rey Juan Carlos (URJC)
  • Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. (Fraunhofer)
  • Technische Universitaet Berlin (TUB)
  • Consiglio Nazionale Delle Ricerche (CNR)
  • Fundación Imdea Software (IMDEA)
  • Atos Spain S.A. (ATOS)
  • Zürcher Hochschule Für Angewandte Wissenschaften (ZHAW)
  • Tikal Technologies S.L. (NAEVATEC)
  • IBM Ireland Limited (IBM IRE)
  • Production Trade And Support of Machinable Products of Software and Informatics – Relational Technology A.E. (RELATIONAL)

SESAME, Hurtle & NetFloc

Recently, at EUCNC’16, the ZHAW SESAME team demonstrated the work of combining SESAME concepts through the use of Hurtle, our orchestration framework, and Netfloc, our SDK for datacenter network programming. The demonstration was also a joint demonstration between SESAME and the 5GPPP COHERENT project.

It was a demonstration that bridges the gap between the telco and cloud world by creating a network service based on the services and technologies coming from both projects.

Continue reading

ACeN Begins!

Recently the ICCLab, ZHAW acquired a KTI project, ACeN – Apache CloudStack for NFV. Cloudstack is one of the front running Infrastructure-as-a-service (IaaS) platforms for cloud environment. Leveraging Network-function virtualization (NFV) as the concept of replacing dedicated network hardware with a software providing the same network functions, increases network capabilities such as service availability in the cloud. This project has now commenced and the interaction between partners Citrix, Exoscale and ZHAW. Everyone is highly engaged already and from our  perspective we’re very excited to about this work.

The ACeN project will deliver services and prototypes based on the NFV standard and Apache CloudStack. A novel hybrid load-balancing service (HLBS) will be created and and key NFV demonstrators will be prototyped. It is hybrid as it combines IP address management and load balancing into one service/function. All will follow a common architectural approach, on common technology. This work will leverage and can enable access to a market worth up to $2.4 Billion by 2018.

The majority of outputs from the project will be made open source (under ASL 2.0), including the hybrid load balancing service. Much of the work in ACeN is exploiting the research work carried out in Mobile Cloud Networking and also the Hurtle orchestration framework.

From our lab’s perspective, this project demonstrates concretely our research approach of bringing foundational research and open source impact through an innovation transfer process to Swiss SMEs.

Stay tuned for more updates!

 

MCN and ICCLab Demo at EUCNC

As part of our on-going work in MobileCloud Networking the project demonstrated at this year’s EUCNC, held in a very warm (> 35*C !!!) Paris, France.

The MCN demonstration was built on top of a standard cloud infrastructure, leveraging key technologies of OpenStack and OpenShift and used (open source outputs of MCN, namely hurtle – the cloud orchestration framework of the ICCLab which is used throughout MCN to enable service delivery. Also demonstrated was the use of the ICCLab’s billing solution, Cyclops that is orchestrated using Hurtle. All of this delivers a NFV-compatible, on-demand, composed service instance.

The MobileCloud Networking (MCN) approach and architecture was demonstrated aiming to show new innovative revenue streams based on new service offerings and the optimisation of CAPEX/OPEX. Of particular note and focus, the work highlighted results of cloudifying the Radio Access Network (RAN) and delivering this capability as an on-demand service.

Supporting this focus was the composition of an end-to-end service (RAN, EPC, IMS, DNS, Monitoring & Billing) instance via the MCN dashboard. This demo service is standards compliant and features interoperable implementations of ETSI NFV, OCCI and 3GPP software.

 

Cloud Orchestration: Hurtle Released

We are proud to announce that the ICCLab has released Hurtle!

Hurtle logo Hurtle

 

Hurtle is a result of the ICCLab’s Cloud Orchestration Initiative.

What is hurtle?

With Hurtle, you automate the life-cycle management of any number of service instances in the cloud, from deployment of resources all the way to configuration and runtime management (e.g., scaling) of each instance. Our motivation is that software vendors often face questions such as “How can I easily provision and manage new instances of the service I offer for each new customer?”, this is what Hurtle aims to solve.

In short, Hurtle lets you:

offer your software as a service i.e. “hurtle it!”

In Hurtle terms, a service represents an abstract functionality that, in order to be performed, requires a set of resources, such as virtual machines or storage volumes, and an orchestrator which describes what has to be done at each step of a service lifecycle.
A “service instance” is the concrete instantiation of a service functionality with its associated set of concrete resources and service endpoints.

On top of this, Hurtle has been designed since its inception to support service composition, so that you can design complex services by (recursively!) composing simple ones.

Hurtle’s functionality revolves around the idea of services as distributed systems composed of multiple sub-applications, so the services offered are also ones that can be designed with the cloud in mind, based on the cloud-native application research of the ICCLab.

What does it mean to offer software as a service?

A bit of history first. Traditionally software has been ran locally, then was centralised and shared through intra-nets. All of this was still on company-specific infrastructure. This made hosting, provisioning and managing such software difficult and the full time job of many IT engineers and system administrators.

This quickly brought about the argument that IT in a SME or an enterprise was a cost centre that should be minimised and lead to outsourcing of such tasks to 3rd parties.

Now today with the ever growing acceptance and use of cloud computing the cost equation is again further reduced, but more interestingly, cloud computing reverses the trend of outsourcing operations to third parties if you consider the movement of devops.

In this new world organisations that create software don’t want nor need third parties to manage their software deployments. They have much of the tooling needed, developed in-house. If they don’t, yet still want to follow a devops approach they’ve quite an amount of work ahead of them.

It is in this scenario where hurtle can help!

What can hurtle do?

What will hurtle do?

  • More examples including the cloud native Zurmo implementation from ICCLab
  • Enhanced workload placement, dynamic policy-based
  • Support for docker-registry deployed containers
  • Runtime updates to service and resource topologies
  • CI and CD support
    • safe monitored dynamic service updates
  • TOSCA support
  • Support for VMware and CloudStack
  • User interface to visualise resource and services relationships
  • Additional external service endpoint protocol support

Want to know more?

Checkout: hurtle.it

Cloud Application Management

Overview

Currently today, large internet-scale services are still architected using the principles of service-orientation. The key overarching idea is that a service is not one large monolith but indeed a composite of cooperating sub-services. How these sub-services are designed and implemented are given either by the respective business function, as in the case of traditional SOA to technical function/domain-context as in the case of the microservice approach to SOA.

In the end what both approaches result in, is a set of services, each of which carrying out a specific task/function. However, in order to bring all these service units together an overarching process needs to be provided to stitch them together and manage their runtimes. In doing so present the complete service to the end-user and for the developer/provider of the service.

The basic management process of stitching these services together is known as orchestration.

Orchestration & Automation

These are two concepts that are often conflated and used as if they’re equivocal. They’re not but they are certainly related, especially when Automation refers to configuration management (CM; e.g. puppet, chef, etc.).

Nonetheless, what both certainly share is that they are oriented around the idea of software systems that expose an API. With that API, manual processes once conducted through user interfaces or command line interfaces can now be programmed and then directed by higher level supervisory software processes. 

Orchestration goes beyond automation in this regard. Automation (CM) is the process that enables the provisioning and configuration of an individual node without consideration for the dependencies that node might have on others or vice versa. This is where orchestration comes into play. Orchestration, in combination with automation, ensures the phases of: 

  1. “Deploy”: the complete fleet of resources and services are deployed according to a plan. At this stage they are not configured. 

  2. “Provision”: each resource and service is correctly provisioned and configured. This must be done such that one service or resource is not without a required operational dependency (e.g. a php application without its database). 

This process is of course a simplified one and does not include the steps of design, build and runtime management of the orchestrated components (services and/or resources). 

  • Design: where the topology and dependencies of each component is specified. The model here typically takes the form of a graph. 

  • Build: how the deployable artefacts such as VM images, python eggs, Java WAR files are created either from source or pre-existing assets. This usually has a relationship to a continuous build and integration process. 

  • Runtime: once all components of an orchestration are running the next key element is that they are managed. To manage means at the most basic level to monitor the components. Based on metrics extracted, performance indicators can be formulated using logic-based rules. These when notified where an indicator’s threshold is breached, an Orchestrator could take a remedial action ensuring reliability. 

  • Disposal: Where a service is deployed through cloud services (e.g. infrastructure; VMs) it may be required to destroy the complete orchestration to redeploy a new version or indeed part of the orchestration destroyed. 

Ultimately the goal of orchestration is to stitch together (deploy, provision) many components to deliver a functional system (e.g. replicated database system) or service (e.g. a 3-tier web application with API) that operates reliably.

Objectives

The key objective of this initiatives are: 

  • Provide a reactive architecture that covers not only the case of controlling services but also service provider specific resources. What this means is that the architecture will exhibit responsiveness, resiliency, elasticity and be message-oriented. This architecture will accommodate all aspects that answer our identified research challenges. 
  • Deliver an open-source framework that implements orchestration for services in general and more specifically cloud-based services. 
  • Provide orchestration that provides reliable and cloud-native service delivery 

There are other objectives that are more related to delivering other research challenges.

Research Challenges

  • How to best enable and support a SOA, Microservices design patterns? 
  • How to get insight and tracing within each service and across services so problems can be identified, understood? 
  • Efficient management of large-scale composed service and resource instance graphs 
  • Scaling based on ‘useful’ monitoring, resource- and service-level metrics 
    • Consider monitoring system and scaling systems e.g. monasca 
    • How to program the scaling of an orchestrator spanning multiple providers and very different services? 
  • Provision of architectural recommendations and optimisation based on orchestration logic analysis
  • How to exploit orchestration capabilities to ensure reliability? ie, “load balancer for high availability” for cloud applications. How can load balancing service be automatically injected ensuring automatic scaling? 
    • How could a service orchestration framework bring the techniques of netflix and amazon (internal services) to a wider audience? 
    • Snapshot your service, rollback to your service’s previous state 
    • Reliability of the Service Orchestrator – how to implement this? HAProxy? Pacemaker? 
  • Orchestration logic should be able to be written in many popular languages 
  • Continuous integration of orchestration code and assets
  • Provider independent orchestration execution and accomdate many resource/service providers. 
    • Hybrid cloud deployments not well considered. How can this be done? 
    • Adoption of well known standards, openid, openauth and custom providers 
  • Authentication services – how to do this over disparate providers? 
  • How to create market places to offer services. Either the service being orchestrated or that service consuming others. 
  • Integration of business services that service owners can charge clients 
  • Containers for service workloads. Where might CoreOS, Docker, Rocket, Solaris Zones fit in the picture? 
    • If windows is not a hard requirement then it makes sense from a provider’s perspective to utilise container tech. 
    • Do we really need full-blown “traditional” IaaS frameworks to offer orchestration?

Relevance to Current & Future Markets

Many companies’ products aim to provide orchestration of resources in the Cloud, such as Sixsq (Slipstream), Cloudify, ZenOSS ControlCenter, Nirmata… There are also several open source projects, especially related to OpenStack, who touch the orchestration topic: OpenStack Heat, Murano, Solum.

Our market survey established a lack of non-cross domain (different service providers), service-oriented orchestration, with many of them taking the lower-level approach of orchestrating resources directly, and very often on a single provider. One aspect that all these solutions are very different in terms of programming models, however there is a growing interest in leveraging a standards-based orchestration description, with TOSCA being the most talked about. Another identified issue is the lack of reliability of services/resources orchestrated by these products, which is a barrier to adoption this initiative aims to solve. Along with this is that many solutions either have no runtime management or has limited capabilities.

  • In a more general point of view, cloud orchestration brings the following benefits to customers:
    Orchestration reduces the overhead of configuring manually all services comprising a cloud-native application
  • Orchestration allows to get out new updates to a service implementation faster and better tested through continuous testing integration and deployment
  • Reliable orchestration ensures the linkage and composition of services remaining running all the time, even where one or more components fail. This reduces downtime experienced by clients and keeps the service providers service always available.
  • Orchestration brings reproducibility and portability in cloud services, which may run on any cloud provider which the orchestration software controls

Architecture

The key entities of the architecture and their relationships to basic entities are shown in the follow diagram. To understand the complete detailed architecture, click on the picture to get the complete view.

c-orch-arch-entity-model

Related Projects

Contact

Apache CloudStack for NFV (ACeN)

Apache CloudStack for NFV (ACeN) is a project that is funded by the Commission for Technology and Innovation.

The ACeN projects seeks to deliver services and prototypes based on the ETSI Network Function Virtualisation (NFV) standard and Apache CloudStack. A novel hybrid load-balancing service (HLBS) will be created and and key NFV demonstrators will be prototyped. All will follow a common architectural approach, on common technology with contributions to open-source communities (under ASL 2.0) by Swiss implementation partners. This work will leverage and can enable access to a market worth up to $2.4 Billion by 2018.

The HLBS will deliver, in a cloud native fashion, the combined two key functionalities of:

  1. Elastic Load Balancing (ELB)
  2. Elastic IPs (EIP)

Proof of CloudStack capabilities will be demonstrated through two NFV prototypes that are of importance to partners, namely:

  1. NFV use case (UC) one through the implementation of an on-demand tenant-based inter-data centre connectivity VPN using CloudStack.
  2. NFV UC five with an on-demand IMS service using CloudStack. IMS.

Contact

 

MobileCloud Networking Live @ Globecomm

As part of the on-going work in MobileCloud Networking the project will demonstrate outputs of the project at this year’s Globecomm industry-track demonstrations. Globecomm is being held this year in Austin, Texas.

MobileCloud Networking (MCN) approach and architecture will be demonstrated aiming to show new innovative revenue streams based on new service offerings and the optimisation of CAPEX/OPEX. MCN is based on a service-oriented architecture that delivering end-to-end, composed services using cloud computing and SDN technologies. This architecture is NFV compatible but goes beyond NFV to bring new improvements. The demonstration includes real implementations of telco equipment as software and cloud infrastructure, providing a relevant view on how the new virtualised environment will be implemented.

For taking the advantage of the technologies offered by cloud computing today’s communication networks has to be re-designed and adapted to the new paradigm both as developing a comprehensive service enablement platform as well as through the appropriate softwarization of network components. Within the Mobile Cloud Networking project this new paradigm has been developed, and early results are already available to be exploited to the community. In particular this demonstration aims at deploying a Mobile Core Network on a cloud infrastructure and show the automated, elastic and flexible mechanism that are offered by such technologies for typical networking services. This demonstration aims at showing how a mobile core network can be instantiated on demand on top of a standard cloud infrastructure, leveraging key technologies of OpenStack and OpenShift.

Screen Shot 2014-11-05 at 12.21.47

The scenario will be as following:

  1. A tenant (Enterprise End User (EEU), in MCN terminology) – may be an MVNO or an enterprise network – requests the instantiation of a mobile core network service instance via the dashboard of the MCN Service Manager – the the service front-end where tenants can come and request the automated creation of a service instance via API or user interface. In particular the deployment of such core network will be on top of a cloud hosted in Europe. At the end of the provisioning procedures, the mobile core network endpoints will be communicated to the EEU.
  2. The EEU will have the possibility to access the Web frontend of the Home Subscriber Server (HSS) and provision new subscribers. Those subscribers information will be used also for configuring the client device (in our case a laptop).
  3. The client device will send the attachment requests to the mobile core network and establish a connectivity service. Since at the moment of the demonstration the clients will be located in the USA, there will be a VPN connection to the eNodeB emulator through which the attachment request will be sent. At the end of the attachment procedure all the data traffic will be redirected to Europe. It will be possible to show that the public IPs assigned to the subscriber are part of the IP range of the European cloud testbed.
  4. The clients attached to the network will establish a call making use of the IP Multimedia Subsystem provided by the MVNO. During the call the MVNO administrator can open the Monitoring as a Service tool provided by the MCN platform and check the current situation of the services. For this two IMS clients will be installed on the demonstration device.
  5. At the end of the demonstration it will be possible to show that the MVNO can dispose the instantiated core network and release the resources which are not anymore necessary. After this operation the MVNO will receive a bill indicating the costs for running such virtualized core network.

It specifically includes:

  • An end-to-end Service Orchestrator, managing dynamically the deployment of a set of virtual networks and of a virtual telecom platform. The service is delivered from the radiohead all the way through the core network to service delivery of IMS services. The orchestration framework is developed on an open source framework available under the Apache 2.0 license and is where the ICCLab actively develops and contributes.
  • Interoperability is guaranteed throughout the stack through the adoption of telecommunication standards (3GPPP, TMForum) and cloud computing standards (OCCI).
  • A basic monitoring system for providing momentary capacity and triggers for virtual network infrastructure adaptations. This will be part of the orchestrated composition.
  • An accounting-billing system for providing cost and billing functions back to the tenant or the provisioned service instance. This will be part of the orchestrated composition.
  • A set of virtualised network functions:
  • A realistic implementation of a 3GPP IP Multimedia Subsystem (IMS) based on the open source OpenIMSCore
  • A realistic implementation of a virtual 3GPP EPC based on the Fraunhofer FOKUS OpenEPC toolkit,
  • An LTE emulation bases on the Fraunhofer FOKUS OpenEPC eNB implementation
  • Demonstration of IMS call establishment across the provisioned on-demand virtualised network functions.