Tag: python

Deploying Singer.io to the Cloud

As presented in a prior post, Singer.io is a modern, open-source ETL (Extract, Transform and Load) framework for integrating data from various sources, including online datasets and services, with a focus on being simple and light-weight. The basics of the framework were explored in our last post on the topic, so we will refer you to that if you are unfamiliar.

This post is about our process for deploying Singer to the cloud, more specifically, to the Cloud Foundry open source cloud application platform. This was done in the context of researching the maturity of data transformation tools in a cloud-native environment. We will explore the options for deploying Singer taps and targets to a cloud provider and discuss our implementation and deployment process in detail.

Continue reading

Building a Singer.io tap for an open data source

Singer.io is an open-source JSON-based data shifting (ETL: extract, transform, load) framework, designed to bring simplicity when moving data between a source and a destination service on the Internet. In this post, we present the framework as entry point into the world of SaaS-level data exchange and some associated research questions.

Continue reading

Lambada update: Transforming Python code into cloud functions for multiple FaaS providers

For rapid development, deployment and testing of applications based on many cloud functions, code transformation tools are on the rise. With a process called “FaaSification”, they turn code into cloud functions by following annotations or decorators specified for developers. Termite for Java, Node2FaaS for JavaScript, and Lambada for Python are representative examples of this new class of tools, related also to other overlay tools such as PyWren. This blog post summarises the recently added software features for Lambada.

Continue reading