Ever since the inception of cloud computing as a widespread phenomenon over a decade ago, the issue of API and data structure variety across providers has become a major hurdle to multi-cloud applications. Even in the most recent services such as Function-as-a-Service (FaaS) offerings to build so-called serverless applications, the issue repeats itself with each provider pushing for their own management interfaces and accepted function definitions.
Yessica Bogado and Walter Benítez from Itaipu Technology Park, currently visiting the Service Prototyping Lab at Zurich University of Applied Sciences, are among the active researchers who tackle such practical problems from the perspective of engineering software applications based on microservices. In the park’s distributed systems and parallel computing research team led by Fabio López Pires, they are witnessing first hand the increasing issues of local software and service providers to keep up with cloud trends. Appropriate tooling can therefore help to alleviate the issues. This blog post describes one such tool, the FaaS converter, which has emerged from their applied research.
FaaS is widely supposed and industry-supported to be one of the major cloud application implementation technologies which will profoundly influence the realisation of microservice-oriented applications, especially cloud-native applications which integrate economic considerations already in the design phase apart from technical characteristics. Full maturity in the sense of provider-independent, portable cloud-native applications is the next logical step in multi-cloud research and requires appropriate tools which we develop in the Service Tooling research initiative.
While tools such as snafu-import or the Serverless Framework handle multi-cloud support at the cloud function management side, the FaaS converter focuses on the syntactic definition of functions. It inspects the function declarations and ensures compatibility to FaaS providers by generating or transforming code as needed. In the current stage, the FaaS converter is a prototypical Python library with command-line client which only covers Python functions, although it is planned to extend it to other languages (primarily, due to popularity, JavaScript) in the near future. You can already download it from Git and from PyPI.
The syntactic heterogeneity of cloud functions has been documented early on. This issue covers both the parameters and the permissible function names apart from default names when not configured otherwise (if possible at all). The following figure shows the very same function in the form that is expected by the various runtime environments and providers.
The idea of portable functions has also been documented for some time. We claim to have been the first ones to describe the concept but so far required manual implementation effort to achieve the portability. With the FaaS converter, we can now automate the process and also improve tools such as the Lambada transformer by adding (potentially decorator-controlled) multi-cloud support in the near future. Also, Snafu’s current autodetection of function syntax, used to restrict the set of loaded function when run under a certain calling convention (lambda, openwhisk), could use the heuristic function signature detection in the FaaS converter library as it matures.
The converter loads source files and writes back modified files, in other words, it performs a static syntax analysis and transformation compared to a dynamic one which for some programming languages, including Python, would be an alternative approach and might be added later. From a design perspective, the conversion consists of two phases: Detection of the format of origin and conversion into the desired target formats, per function. The conversion works in two modes, a “just add a wrapper” mode and a complete function rewrite mode. Furthermore, the output can be selected to be grouped into one file or split into separate files. However, considering the growing number of providers, there is a growing amount of conflicting function names, so that already now the joint output disables some wrapper functions at the expense of others.
The conversion process itself is also not trivial. For example, AWS Lambda offers a useful context parameter which, when present in the source function, needs to be added as reimplementation in the targets so that functions which rely on for instance a countdown of the remaining execution time will continue to work. Parameters and return values need to be wrapped to adhere to the calling conventions. The format detection is heuristic due to a combination of naming conventions for both the function itself and its parameters.
The following shell script snippet is a step-by-step starter guide to using the tool with a simple example function.
$ pip3 install faas-converter $ # Run the converter on its own implementation $ ~/.local/bin/faasconverter --file ~/.local/lib/python3.5/site-packages/faas_converter/faasconverter.py »» faasconverter: track module: ... »» faasconverter: convert function check_line (string, f) »» faasconverter: converted to module: check_line_aws_portable.py ... # FaaS-Converter wrapper for aws def lambda_handler(event, context): return check_line(event['string'],event['f']) # Check the output $ ls -l *_portable.py ... # Learn more $ w3m https://github.com/walter-bd/faas-converter/blob/master/README.md
We hope that this tool is useful to software developers out there, despite its early prototype nature, and would like to hear feedback on pursuing this direction in the future. Do you have practical needs (e.g. inclusion as plugin to the Serverless framework, automated testcase transformation, …?) Do you see interesting research challenges around the topics of interoperability and standards for cloud-native applications built with microservices? Please leave a comment for any opinions or suggestions.