In Quest of the “Open Cloud” (updated)

A remarkable and wonderful feature of Zurich is its vivid computer science and technology community. Hardly any week passes without an interesting event around new innovations and technologies, like the Internet of Things, novel programming languages (Go, etc), security, and of course Cloud Computing technologies (e.g. MongoDB) [footnote1]. Particularly interesting – from our / ICCLab perspective – is the ZhGeeks (@zhgeeks) community, run by one of our fellow technology and cloud evangelists Muharem. Last weeks Zhgeeks meeting was about Open Cloud and no less prominent figure than Samj was about to update us on the Open Cloud Initiative. A truly inspiring talk  (download slides).

Notwithstanding of Sam’s comprehensive and sound introduction into the world of “Cloud Openness” (from an OCI perspective) I can’t help but have to ask myself – hellya, what is this Open Cloud thing?

So let’s see what the universal oracle has to tell [footnote2]. And here we go. “Open Clouds” all over; what a surprise. So again, what is it then?

Indeed, this is an interesting question if you bet on CC, either as adopter, contributor, or simply user. As Sam summed it up, “Open Cloud is about technology openness such that the market can drive it’s evolution”, reflecting the OCI definition and principles:

Open Cloud must meet the following requirements:

  • Open Formats: All user data and metadata must be represented in Open Standard formats.
  • Open Interfaces: All functionality must be exposed by way of Open Standard interfaces.”

Having sacrificed several hours of sleep for studying the OpenStack Foundation bylaws in order to understand the implications on OpenStack’s future (you my call it “openness”) – see my earlier blog post Open- or NotSo-OpenStack? – I was wondering whether OCI’s definition also poses open governance requirements onto OSS Cloud Computing projects, like OpenStack, CloudStack, Eucalyptus and the likes, but this is seemingly not the case.

Red Hat’s Open Cloud definition goes further by not only requesting open source, standards, and interfaces, but also incorporating “technical governance” into “The Open Cloud: Red Hat’s Perspective”. It says “An Open Cloud …

Has a viable, independent community. Open source isn’t just about the code, its license, and how it can be used and extended. At least as important is the community associated with the code and how it’s governed. Realizing the collaborative potential of open source and the innovation it can deliver to everyone means having the structures and organization in place to tap it fully.”

Another popular reference is the Open Cloud Manifesto, which defines a set of open cloud principles, similar to OCI but focuses further on features of an open cloud and the defining community, in particular by embracing the fact that Cloud Computing is a community effort,  that different stakeholder groups exist, and that these ought to collaborate to avoid fragmentation.

“6. Cloud computing standards organizations, advocacy groups, and communities should work together and stay coordinated, making sure that efforts do not conflict or overlap.”
RackSpace, notably one of the founding members of OpenStack, has too an opinion; “Open Cloud Computing : History of Open Source Coding and the Open Cloud” – in this post exclusively linked to Open Source Software.
Alex Williams from TechCruch “spend some time with the technologists at CloudOpen – the Linux Foundation’s first cloud only event” and summarizes in his article “To sum it up: if VMworld is about the data center then CloudOpen is about the software” and continues “Over the past few days, I’ve tried to crystallize the conversation to some extent. Here is my take:
  • An open cloud has open APIs.
  • An open cloud has a developer community that collaborates on developing the cloud infrastructure or platform environment.
  • An open cloud has people who have deep experience in running open source projects.
  • An open cloud gives users the rights to move data as wished.
  • An open cloud is federated — you can run your apps and data across multiple cloud environments.
  • An open cloud does not require an IT administrator to provision and manage.
  • An open cloud does not require new hardware.
  • An open cloud is not a rat’s nest of licenses.
  • An open cloud is not a proprietary, new age mainframe.
  • An open cloud is not washed with marketing.
  • An open cloud can be forked.
  • An open cloud has full view into the infrastructure environment.
  • An open cloud is not hosted, legacy software
David Linthicum seems to share equal motivations which led me to this blog post. He tells in his post “InfoWorld: The ‘open cloud’ is getting awfully confusing”. The article does not go into detail but nicely summarizes the status quo:
“If you’re looking at adopting an “open cloud” technology, you have complex work ahead. Assessing their value is complicated by the fact that many of the vendors are less than a two years old and have a minimal install base that can provide insight into fit, issues, and value.

As with any significant IT endeavor, you need to do your homework, understand your requirements, and make sure to test this stuff well before you use it in your enterprise. At some point, the “open cloud” market will normalize, and when that happens, you hope your seat will still be available in the ensuing game of musical chairs.”

 from Gigao picks up this thread, citing Alex and David, for his “Prediction: More Cloud Confusion Ahead“.

Alex Williams from TechCruch cites the CloudOpen conference as trigge and here goes a summary of the discussions, 10 Insights from Linux Leaders in the Open Cloud.

Richard Kaufmann, chief technologist, HP Cloud Services, July 3, 2012, “HP Public Cloud Aims to Boost OpenStack Customer Base.”There are two important APIs out there, one is Amazon’s and the other is OpenStack. And OpenStack has Amazon compatibility. HP will continue to support those Amazon compatibility layers; We’re not trying to lead on a position about what customers should do with APIs… I (personally) believe there should be a popular cloud API for IaaS and it should not be Amazon. It could be anything else but it can’t float from above. It has to be based on popular usage.

Lew Moorman, Rackspace, July 10, 2012, “Open Cloud Key to Modern Networking.”Some people seem to think that APIs are the cloud and one thing that made the cloud so revolutionary is it’s programmatically accessible by API.  But (Amazon) S3 is a really complex distributed system. The issue with a model that says “clone Amazon” is that, unless you have the core technology underneath it, you can’t have a cloud…

OpenStack is really setting out to build an open alternative from end to end. They say we’re going to do networking, not just set out to copy Amazon. We need to really innovate and build a visionary system that can power the future of computing. Amazon, VMware and Microsoft don’t have all the answers.

Christopher Brown, CTO, Opscode, July 17, 2012, “Chef Offers a Recipe for the Open Source Cloud.”The open cloud lies both below and above the waterline of the API. At the beginning we all wanted to treat the cloud as an easier way to get the compute units that looked like the old thing we used to get buying a physical machine. But that’s not actually true. It’s not the same thing and it requires a different design underneath of a common substrate. If you look above the water line at the consumer, the way you build applications, the way they scale, etc., designing the cloud and for the cloud are different than what is now legacy.

Mark Hinkle, senior director of cloud computing community, Citrix July 24, 2012, “Citrix’s Hinkle Proposes Linux Model for an Open Source Cloud.It’s first and foremost that the orchestration platform is open source. The data you store within the cloud is open to you as the end user in a format you can manipulate easily and it’s easily transferable. The API is also open and clearly documented.

Ross Turk, vice president of community, InkTank, July 31, 2012, “An Open Source Storage Solution for the Enterprise.It can mean a cloud stack that is built on open source like OpenStack or CloudStack and that reflects the economic and community advantages behind building something that’s akin to what Amazon has done, but built on commodity hardware. It’s an open alternative to AWS. Another way to think of the open cloud doesn’t exclude AWS. It’s having cloud services with standardized APIs so applications written for one cloud can work on another cloud.

Imad Sousou, director of Intel’s Open Source Technology Center, Aug. 7, 2012, “Open Cloud Standards will Emerge With More Collaboration.”The open cloud must meet these requirements:  Use Open Formats, where all user data and metadata must be represented in Open Standard formats. Use Open Interfaces, where functionality must be exposed through Open Standard interfaces. In addition, in the open cloud, various open source technologies should be available to build the right solutions efficiently, and to drive innovation. These would include software stack and tools, such as the hypervisors or operating systems, middleware, such as databases and web servers, web content management systems, and development tools and languages. Such open source-based software solutions would reinforce interoperability of the open cloud.

Kyle MacDonald, vice president of cloud, Canonical, Aug. 14, 2012, “Canonical: Making the Open Cloud Seamless for Users.True power comes when the users can move from one cloud service to another. That’s the state of nirvana. The cool word is ‘interoperability.’ …  It will be almost required that if you’re a cloud service you publish an API that’s clear. And eventually there will be a common API or it becomes so simple the minor differences won’t be a big deal to end users. And then partners who define the services can use those same open source technologies and provide a good service.

Alan Clark, director of industry initiatives, emerging standards and open source, SUSE Aug. 21, 2012, “SUSE Aims for One-Click Enterprise Deployment on OpenStack.Enterprise IT must deliver the most efficient, scalable and flexible services possible. The open cloud provides that through the ability to have a flexible infrastructure, quick and easy deployment, service management and complete life cycle management. We’re working with partners — many are part of these open source projects – to build this together and that builds interoperability. It’s a collaboration of ideas as well as code. It accelerates bringing a solution to market that works across all the different partners.

Angel Diaz, vice president of software standards and cloud, IBM Aug. 28, 2012, “3 Projects Creating User-Driven Standards for the Open Cloud.” Our clients who use technology have a heterogeneous environment. They need to take existing systems, extend them and deal with it and they don’t want to be locked into a singe vendor solution. That is how (IBM) defines an open cloud: where end users want to have these points of interoperability.

Joe Brockmeier, open source cloud computing evangelist, Citrix Sept. 6, 2012, “Defining the Open Cloud.Some folks will argue that a cloud service or offering is open if it has open APIs and open standards.  For my money, the best definition of the open cloud came from Red Hat’s Scott Crenshaw: It enables portability across clouds; Has a pluggable, extensible, and open API; Lets you deploy to your choice of infrastructure; It’s unencumbered by patents and other IP restrictions; It’s based on open standards; It has a viable and independent community; It is open source. Having open APIs is necessary, but it’s not enough. If you depend on one vendor to provide your cloud, and you can’t pull it in-house for any reason, it’s just not open.

So are we any smarter after this, only little I fear. There are common themes, like open source, standards, interfaces, royalty-freeness from a technology ankle and freedom of choice and cross-cloud portability on the other. And perhaps the community should indeed take this one Open Cloud Manifesto principle on “open cloud community collaboration” to heart and drive consolidation; instead of contributing to one of those fundamental innovation hindrances that they try to avoid, that is fragmentation. The CloudOpen conference was one great step into this direction and the Google Hangout on OpenClouds (on Youtube) by Ben Kepes (Ben Kepes on G+) will hopefully be another way to continue this important discussion.

ANNEX

Open Cloud InitiativeOpen Cloud Principles (OCP)

Interoperability (the ability to exchange and use information) between cloud computing products and services is required for unfettered competition between vendors and unrestricted choice for users.

Users must be able to come (no barriers to entry) and go (no barriers to exit) regardless of who they are (no discrimination) and what systems they use (technological neutrality).

Supporting vendors must therefore cooperate on standards, implementing those that exist (where applicable) and collaborating via an open process to develop those that don’t, with a view to competing fairly on quality.
Open Cloud must meet the following requirements:
  • Open Formats: All user data and metadata must be represented in Open Standard formats.
  • Open Interfaces: All functionality must be exposed by way of Open Standard interfaces.

Open Standards must meet the following requirements:

  • Copyrights: The standard must be documented in all its details, published and both accessible and [re]usable free of charge.
  • Patents: Any patents possibly present on [parts of] the standard must be irrevocably made available on a royalty-free basis.
  • Trademarks: Any trademarks possibly present on identifier(s) must be used for non-discriminatory enforcement of compliance only.
  • Implementations: There must be multiple full, faithful, independent and interoperable implementations (for both client and server where applicable) and at least one such implementation must be licensed in its entirety under an Open Source Initiative (OSI) approved license or placed into the public domain.

Red Hat : An open cloud has the following characteristics:

  • Is open source. This allows adopters to control their particular implementation and doesn’t restrict them to the technology and business roadmap of a specific vendor. It lets them build and manage clouds that put them in control of their own destiny and provides them with visibility into the technology on which they’re basing their business. It provides them with the flexibility to run the workloads of their choice, including proprietary ones, in their cloud. Open source also lets them collaborate with other communities and companies to help drive innovation in the areas that are important to them.
  • Has a viable, independent community. Open source isn’t just about the code, its license, and how it can be used and extended. At least as important is the community associated with the code and how it’s governed. Realizing the collaborative potential of open source and the innovation it can deliver to everyone means having the structures and organization in place to tap it fully.
  • Is based on open standards, or protocols and formats that are moving toward standardization and that are independent of vendor and platform. Standardization in the sense of “official” cloud standards blessed by standards bodies is still in early days. That said, approaches to interoperability that aren’t under the control of individual vendors and that aren’t tied to specific platforms offer important flexibility. This allows the API specification to evolve beyond implementation constraints and creates the opportunity for communities and organizations to develop variants that meet their individual technical and commercial requirements.
  • Freedom to use IP.  Recent history has repeatedly shown that there are few guarantees that intellectual property (IP) assets will remain accessible to all from one day to the next.  To have confidence that you will continue to enjoy access to IP assets that you depend on under the terms that you depend on, permission needs to be given in ways that make that technology open and accessible to the user.  So-called “de facto standards,” which are often “standards” only insofar as they are promoted by a large vendor, often fail this test.
  • Is deployable on your choice of infrastructure. Hybrid cloud management should provide an additional layer of abstraction above virtualization, physical servers, storage, networking, and public cloud providers. This implies, or indeed requires, that cloud management be independent of virtualization and other foundational technologies. This is a fundamental reason that cloud is different from virtualization management and a fundamental enabler of hybrid clouds that span physical servers, multiple virtualization platforms, and a wide range of public cloud providers including top public clouds.
  • Is pluggable and extensible with an open API. This lets users add features, providers, and technologies from a variety of vendors or other sources. Critically, the API itself cannot be under the control of a specific vendor or tied to a specific implementation but must be under the auspices of a third-party organization that allows for contributions and extensions in an open and transparent manner. Deltacloud, an API that abstracts the differences between clouds, provides a good example. It is under the auspices of the Apache Software Foundation and is neither a Red Hat-controlled project nor tied to a particular implementation of cloud management.
  • Enables portability to other clouds. Implicit in a cloud approach that provides support for heterogeneous infrastructure is that investments made in developing for an open cloud must be portable to other such clouds. Portability takes a variety of forms including programming languages and frameworks, data, and the applications themselves. If you develop an application for one cloud, you shouldn’t need to rewrite it in a different language or use different APIs to move it somewhere else. Furthermore, a consistent runtime environment across clouds means that retesting and requalification isn’t needed every time you want to redeploy.

Open Cloud Manifesto : Open Cloud Principles

Rather, as cloud computing matures, there are several key principles that must be followed to ensure the cloud is open and delivers the choice, flexibility and agility organizations demand:
1. Cloud providers must work together to ensure that the challenges to cloud adoption (security, integration, portability, interoperability, governance/management, metering/monitoring) are addressed through open collaboration and the appropriate use of standards.
2. Cloud providers must not use their market position to lock customers into heir particular platforms and limit their choice of providers.
3. Cloud providers must use and adopt existing standards wherever appropriate. The IT industry has invested heavily in existing standards and standards organizations; there is no need to duplicate or reinvent them.
4. When new standards (or adjustments to existing standards) are needed, we must be judicious and pragmatic to avoid creating too many standards. We must ensure that standards promote innovation and do not inhibit it.
5. Any community effort around the open cloud should be driven by customer needs, not merely the technical needs of cloud providers, and should be tested or verified against real customer requirements.
6. Cloud computing standards organizations, advocacy groups, and communities should work together and stay coordinated, making sure that efforts do not conflict or overlap.
Leaders of the Open Cloud at OpenCloud

Richard Kaufmann, chief technologist, HP Cloud Services
July 3, 2012, “HP Public Cloud Aims to Boost OpenStack Customer Base.” 

There are two important APIs out there, one is Amazon’s and the other is OpenStack. And OpenStack has Amazon compatibility. HP will continue to support those Amazon compatibility layers; We’re not trying to lead on a position about what customers should do with APIs… I (personally) believe there should be a popular cloud API for IaaS and it should not be Amazon. It could be anything else but it can’t float from above. It has to be based on popular usage.

Lew Moorman, Rackspace
July 10, 2012, “Open Cloud Key to Modern Networking.”

Some people seem to think that APIs are the cloud and one thing that made the cloud so revolutionary is it’s programmatically accessible by API.  But (Amazon) S3 is a really complex distributed system. The issue with a model that says “clone Amazon” is that, unless you have the core technology underneath it, you can’t have a cloud…

OpenStack is really setting out to build an open alternative from end to end. They say we’re going to do networking, not just set out to copy Amazon. We need to really innovate and build a visionary system that can power the future of computing. Amazon, VMware and Microsoft don’t have all the answers.

Christopher Brown, CTO, Opscode
July 17, 2012, “Chef Offers a Recipe for the Open Source Cloud.” 

The open cloud lies both below and above the waterline of the API. At the beginning we all wanted to treat the cloud as an easier way to get the compute units that looked like the old thing we used to get buying a physical machine. But that’s not actually true. It’s not the same thing and it requires a different design underneath of a common substrate. If you look above the water line at the consumer, the way you build applications, the way they scale, etc., designing the cloud and for the cloud are different than what is now legacy.

Mark Hinkle, senior director of cloud computing community, Citrix
July 24, 2012, “Citrix’s Hinkle Proposes Linux Model for an Open Source Cloud.” 

It’s first and foremost that the orchestration platform is open source. The data you store within the cloud is open to you as the end user in a format you can manipulate easily and it’s easily transferable. The API is also open and clearly documented.

Ross Turk, vice president of community, InkTank
July 31, 2012, “An Open Source Storage Solution for the Enterprise.” 

It can mean a cloud stack that is built on open source like OpenStack or CloudStack and that reflects the economic and community advantages behind building something that’s akin to what Amazon has done, but built on commodity hardware. It’s an open alternative to AWS.

Another way to think of the open cloud doesn’t exclude AWS. It’s having cloud services with standardized APIs so applications written for one cloud can work on another cloud.

Imad Sousou, director of Intel’s Open Source Technology Center
Aug. 7, 2012, “Open Cloud Standards will Emerge With More Collaboration.”

The open cloud must meet these requirements:  Use Open Formats, where all user data and metadata must be represented in Open Standard formats. Use Open Interfaces, where functionality must be exposed through Open Standard interfaces.

In addition, in the open cloud, various open source technologies should be available to build the right solutions efficiently, and to drive innovation. These would include software stack and tools, such as the hypervisors or operating systems, middleware, such as databases and web servers, web content management systems, and development tools and languages. Such open source-based software solutions would reinforce interoperability of the open cloud.

Kyle MacDonald, vice president of cloud, Canonical
Aug. 14, 2012, “Canonical: Making the Open Cloud Seamless for Users.” 

True power comes when the users can move from one cloud service to another. That’s the state of nirvana. The cool word is ‘interoperability.’ …  It will be almost required that if you’re a cloud service you publish an API that’s clear. And eventually there will be a common API or it becomes so simple the minor differences won’t be a big deal to end users. And then partners who define the services can use those same open source technologies and provide a good service.

Alan Clark, director of industry initiatives, emerging standards and open source, SUSE
Aug. 21, 2012, “SUSE Aims for One-Click Enterprise Deployment on OpenStack.” 

Enterprise IT must deliver the most efficient, scalable and flexible services possible. The open cloud provides that through the ability to have a flexible infrastructure, quick and easy deployment, service management and complete life cycle management.

We’re working with partners — many are part of these open source projects – to build this together and that builds interoperability. It’s a collaboration of ideas as well as code. It accelerates bringing a solution to market that works across all the different partners.

Angel Diaz, vice president of software standards and cloud, IBM
Aug. 28, 2012, “3 Projects Creating User-Driven Standards for the Open Cloud.” 

Our clients who use technology have a heterogeneous environment. They need to take existing systems, extend them and deal with it and they don’t want to be locked into a singe vendor solution. That is how (IBM) defines an open cloud: where end users want to have these points of interoperability.

Joe Brockmeier, open source cloud computing evangelist, Citrix
Sept. 6, 2012, “Defining the Open Cloud.” 

Some folks will argue that a cloud service or offering is open if it has open APIs and open standards.  For my money, the best definition of the open cloud came from Red Hat’s Scott Crenshaw: It enables portability across clouds; Has a pluggable, extensible, and open API; Lets you deploy to your choice of infrastructure; It’s unencumbered by patents and other IP restrictions; It’s based on open standards; It has a viable and independent community; It is open source.

Having open APIs is necessary, but it’s not enough. If you depend on one vendor to provide your cloud, and you can’t pull it in-house for any reason, it’s just not open.

FOOTNOTES

[footnote1]  This is symptom and cause, concurrently, for the Zurich’s rise as the European Silicon Valley, as some already claim, based on its flourishing high-tech start-up scene that enjoys a wide array of support and attention (e.g. Zurich Start-Up PortalStartwerk PortalZurich Start-up Weekend, Zurich StartupsStartup.ch).

[footnote2] Apologies to one of my previous employers. I really didn’t mean to mention the evil.

 


Leave a Reply

Your email address will not be published. Required fields are marked *