As presented in a prior post, Singer.io is a modern, open-source ETL (Extract, Transform and Load) framework for integrating data from various sources, including online datasets and services, with a focus on being simple and light-weight. The basics of the framework were explored in our last post on the topic, so we will refer you to that if you are unfamiliar.
This post is about our process for deploying Singer to the cloud, more specifically, to the Cloud Foundry open source cloud application platform. This was done in the context of researching the maturity of data transformation tools in a cloud-native environment. We will explore the options for deploying Singer taps and targets to a cloud provider and discuss our implementation and deployment process in detail.
Looking into a possible post-cloud world, we see mentions of different computing paradigms, many of them based on decentralised structures to overcome scalability and user control limitations. Among them is blockchain-as-a-service (BCaaS or BaaS), mimicking the platform-as-a-service (PaaS) user experience for both application providers and consumers. In PaaS, providers first sign up and subscribe to the platform, then design and build their applications and deploy them to the platform where it is executing either permanently or upon incoming network requests or other event triggers. Additionally, developers may advertise their apps at technology-specific hubs such as AWS SAR or Helm Hub. Consumers then adhere to the application terms, which might require a sign-up at the provider site, before being able to invoke and make use of the application.