Cloud Application Management

Overview

Currently today, large internet-scale services are still architected using the principles of service-orientation. The key overarching idea is that a service is not one large monolith but indeed a composite of cooperating sub-services. How these sub-services are designed and implemented are given either by the respective business function, as in the case of traditional SOA to technical function/domain-context as in the case of the microservice approach to SOA.

In the end what both approaches result in, is a set of services, each of which carrying out a specific task/function. However, in order to bring all these service units together an overarching process needs to be provided to stitch them together and manage their runtimes. In doing so present the complete service to the end-user and for the developer/provider of the service.

The basic management process of stitching these services together is known as orchestration.

Orchestration & Automation

These are two concepts that are often conflated and used as if they’re equivocal. They’re not but they are certainly related, especially when Automation refers to configuration management (CM; e.g. puppet, chef, etc.).

Nonetheless, what both certainly share is that they are oriented around the idea of software systems that expose an API. With that API, manual processes once conducted through user interfaces or command line interfaces can now be programmed and then directed by higher level supervisory software processes.

Orchestration goes beyond automation in this regard. Automation (CM) is the process that enables the provisioning and configuration of an individual node without consideration for the dependencies that node might have on others or vice versa. This is where orchestration comes into play. Orchestration, in combination with automation, ensures the phases of:

“Deploy”: the complete fleet of resources and services are deployed according to a plan. At this stage they are not configured.
“Provision”: each resource and service is correctly provisioned and configured. This must be done such that one service or resource is not without a required operational dependency (e.g. a php application without its database).

This process is of course a simplified one and does not include the steps of design, build and runtime management of the orchestrated components (services and/or resources).

Design: where the topology and dependencies of each component is specified. The model here typically takes the form of a graph.
Build: how the deployable artefacts such as VM images, python eggs, Java WAR files are created either from source or pre-existing assets. This usually has a relationship to a continuous build and integration process.
Runtime: once all components of an orchestration are running the next key element is that they are managed. To manage means at the most basic level to monitor the components. Based on metrics extracted, performance indicators can be formulated using logic-based rules. These when notified where an indicator’s threshold is breached, an Orchestrator could take a remedial action ensuring reliability.
Disposal: Where a service is deployed through cloud services (e.g. infrastructure; VMs) it may be required to destroy the complete orchestration to redeploy a new version or indeed part of the orchestration destroyed.

Ultimately the goal of orchestration is to stitch together (deploy, provision) many components to deliver a functional system (e.g. replicated database system) or service (e.g. a 3-tier web application with API) that operates reliably.

Objectives

The key objective of this initiatives are:

Provide a reactive architecture that covers not only the case of controlling services but also service provider specific resources. What this means is that the architecture will exhibit responsiveness, resiliency, elasticity and be message-oriented. This architecture will accommodate all aspects that answer our identified research challenges.
Deliver an open-source framework that implements orchestration for services in general and more specifically cloud-based services.
Provide orchestration that provides reliable and cloud-native service delivery

There are other objectives that are more related to delivering other research challenges.

Research Challenges

How to best enable and support a SOA, Microservices design patterns?
How to get insight and tracing within each service and across services so problems can be identified, understood?
Efficient management of large-scale composed service and resource instance graphs
Scaling based on ‘useful’ monitoring, resource- and service-level metrics
- Consider monitoring system and scaling systems e.g. monasca
- How to program the scaling of an orchestrator spanning multiple providers and very different services?
Provision of architectural recommendations and optimisation based on orchestration logic analysis
How to exploit orchestration capabilities to ensure reliability? ie, “load balancer for high availability” for cloud applications. How can load balancing service be automatically injected ensuring automatic scaling?
- How could a service orchestration framework bring the techniques of netflix and amazon (internal services) to a wider audience?
- Snapshot your service, rollback to your service’s previous state
- Reliability of the Service Orchestrator – how to implement this? HAProxy? Pacemaker?
Orchestration logic should be able to be written in many popular languages
Continuous integration of orchestration code and assets
Provider independent orchestration execution and accomdate many resource/service providers.
- Hybrid cloud deployments not well considered. How can this be done?
- Adoption of well known standards, openid, openauth and custom providers
Authentication services – how to do this over disparate providers?
How to create market places to offer services. Either the service being orchestrated or that service consuming others.
Integration of business services that service owners can charge clients
Containers for service workloads. Where might CoreOS, Docker, Rocket, Solaris Zones fit in the picture?
- If windows is not a hard requirement then it makes sense from a provider’s perspective to utilise container tech.
- Do we really need full-blown “traditional” IaaS frameworks to offer orchestration?

Relevance to Current & Future Markets

Many companies’ products aim to provide orchestration of resources in the Cloud, such as Sixsq (Slipstream), Cloudify, ZenOSS ControlCenter, Nirmata… There are also several open source projects, especially related to OpenStack, who touch the orchestration topic: OpenStack Heat, Murano, Solum.

Our market survey established a lack of non-cross domain (different service providers), service-oriented orchestration, with many of them taking the lower-level approach of orchestrating resources directly, and very often on a single provider. One aspect that all these solutions are very different in terms of programming models, however there is a growing interest in leveraging a standards-based orchestration description, with TOSCA being the most talked about. Another identified issue is the lack of reliability of services/resources orchestrated by these products, which is a barrier to adoption this initiative aims to solve. Along with this is that many solutions either have no runtime management or has limited capabilities.

In a more general point of view, cloud orchestration brings the following benefits to customers:
Orchestration reduces the overhead of configuring manually all services comprising a cloud-native application
Orchestration allows to get out new updates to a service implementation faster and better tested through continuous testing integration and deployment
Reliable orchestration ensures the linkage and composition of services remaining running all the time, even where one or more components fail. This reduces downtime experienced by clients and keeps the service providers service always available.
Orchestration brings reproducibility and portability in cloud services, which may run on any cloud provider which the orchestration software controls

Architecture

The key entities of the architecture and their relationships to basic entities are shown in the follow diagram. To understand the complete detailed architecture, click on the picture to get the complete view.

Service Engineering (ICCLab & SPLab)