As presented in a prior post, Singer.io is a modern, open-source ETL (Extract, Transform and Load) framework for integrating data from various sources, including online datasets and services, with a focus on being simple and light-weight. The basics of the framework were explored in our last post on the topic, so we will refer you to that if you are unfamiliar.
This post is about our process for deploying Singer to the cloud, more specifically, to the Cloud Foundry open source cloud application platform. This was done in the context of researching the maturity of data transformation tools in a cloud-native environment. We will explore the options for deploying Singer taps and targets to a cloud provider and discuss our implementation and deployment process in detail.
Note: This post does not contain any medical advice or suggestions on how to act and react. If you are looking for that, you are looking in the wrong place.
Economy and society in Switzerland are currently highly affected by the spreading second version of the coronavirus (SARS-CoV-2) that causes the associated infectious desease (COVID-19). The World Health Organisation (WHO) has classified the virus outbreak as PHEIC on January 30 and as pandemy on March 11. In Switzerland, the state emergency level Eminent/Special Situation was reached on February 28, and further restrictions led de-facto to the subsequent level Extraordinary Situation on March 13. This blog post reports on how the outbreak evolution can be continuously visualised as a reliable service with off-the-shelf tools.
Many providers of hosted services, including cloud applications, are subject to a contradiction in handling log data. On the one hand, storing logs consumes resources and should be minimised or avoided altogether to save resource cost. On the other hand, regulatory constraints such as keeping the data for the purpose of future audits exist. A smart solution to encode the data appropriately needs to be found. The coding encompasses both compression, to keep resource use low, and encryption, to prevent leaking information to unauthorised parties, for instance when logging for the purpose of intrusion detection. On an algorithmic level, the encoded data should still be usable for computation, in particular comparison and search. In this blog post, based on the didactic log example shown in the figure below, we present algorithms and architectures to handle cloud log files in a smart way.
Back in 2018, several software developers and researchers met in Zurich at ESSCA to discuss the state of serverless applications, including upcoming technical and business/application opportunities.
Fast-forward 1½ years, it is time to have another look and present the latest frameworks, FaaSification and deployment tools, FaaS services, measurements and so forth. Hence, we invite everybody to consider submitting a talk proposal to ESSCA 2020 which takes place under the wings of the 21st International Conference on Agile Software Development on June 12 in Copenhagen, Denmark.
To keep the spirit of ESSCA as a community gathering, the possible contributions are open to include industry and business experience reports, short tool descriptions and position texts, and abstracts of lightning talks, apart from full research papers.
The Service Prototyping Lab at Zurich University of Applied Sciences is involved as co-organiser of the event, anticipating fruitful discussions about innovative application designs and technological underpinnings in cloud and post-cloud environments.
The University of St. Gallen, through its Latin-American-Swiss Center (CLS-HSG), is the Leading House for the Latin American region, granting incentives and developing joint research cooperation projects with numerous Latin American countries. One of the grant recipients is the Service Prototyping Lab at Zurich University of Applied Sciences, bringing programmability to fog-cloud continuum computing with its parters from UNICAMP in Brazil. In this blog post, a recent research slam featuring this and other chosen projects is summarised.
In Switzerland, opendata.swiss is the go-to location for any open dataset resulting from federal, cantonal or municipal sources. From a societal and economics perspective, the portal is an important asset following the “protect private data, make use of public data” mantra, and has already led to digital innovation through the availability of many third-party applications. In this research blog post, we look at some numbers associated with the portal.
Note: This information is to be understood under the condition of approval of the DIZH by the Canton Council.
The research efforts «Can virtual reality systems help us to design software as we talk?» and «Smart Cities & Regions Services Enablement» are among the first contributions to the digitalisation initiative of the canton of Zurich. They amplify the know-how on software engineering and data-intensive Internet services bundled at the Institute for Applied Information Technology for the support of commercial applications of the following decade.
Singer.io is an open-source JSON-based data shifting (ETL: extract, transform, load) framework, designed to bring simplicity when moving data between a source and a destination service on the Internet. In this post, we present the framework as entry point into the world of SaaS-level data exchange and some associated research questions.