Service Prototyping Lab https://blog.zhaw.ch/splab/ A Blog of the ZHAW Zurich University of Applied Sciences Tue, 17 Aug 2021 07:51:13 +0000 en-GB hourly 1 https://wordpress.org/?v=6.3.3 10th IEEE International Conference on Cloud Computing in Emerging Markets https://blog.zhaw.ch/splab/2021/06/12/10th-ieee-international-conference-on-cloud-computing-in-emerging-markets/ https://blog.zhaw.ch/splab/2021/06/12/10th-ieee-international-conference-on-cloud-computing-in-emerging-markets/#respond Sat, 12 Jun 2021 13:23:38 +0000 https://blog.zhaw.ch/splab/?p=1421 The Tenth IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) continues the highly successful CCEM conference series which was originally launched in 2012. We proudly support this jubilee event. Read on for details. Cloud computing has emerged as a dominant and transformational paradigm in information technology over the last few years and is […]

Der Beitrag 10th IEEE International Conference on Cloud Computing in Emerging Markets erschien zuerst auf Service Prototyping Lab.

]]>
The Tenth IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) continues the highly successful CCEM conference series which was originally launched in 2012. We proudly support this jubilee event. Read on for details.

Cloud computing has emerged as a dominant and transformational paradigm in information technology over the last few years and is beginning to effect a multitude of industries such as government, finance, telecommunications, education, retail, energy and utilities, and transportation. Research in this field has become very active and spans a number of areas including virtualization, networking, storage, security, management of cloud services, efficient cloud architectures, massive multi-tenancy, and design of cloud applications and services. Cloud computing continues to be the way of the future and many studies indicate that more than 50% of all information technology will be in this new paradigm within the next five to ten years. This transformation has great implications for emerging markets which have the potential to leap frog mature markets in their adoption of cloud computing, combine cloud computing and mobile technologies to introduce unique services that can transform the lives of billions, drive a much larger scale of adoption and challenge existing price points, while presenting unique challenges in areas such as security and user interfaces to cloud computing.

This research driven innovation conference has established itself as a unique conference that combines top notch peer-reviewed research papers (acceptance rate less than 10%), invited talks from renowned industry and government leaders, posters and demos showcasing latest innovations, a startup showcase highlighting the vibrant and fast growing business ecosystem around cloud, tutorials on cutting edge topics, and a doctoral symposium featuring some of the deep research topics being pursued by academic leaders.

Consider submitting to CCEM until July 18 for research papers and until September 15 for other contributions – research proposals, techno business plan showcase, student project show case etc.

IEEE CCEM 2021 Website

Der Beitrag 10th IEEE International Conference on Cloud Computing in Emerging Markets erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/06/12/10th-ieee-international-conference-on-cloud-computing-in-emerging-markets/feed/ 0
CIFS 2021 – Call for papers https://blog.zhaw.ch/splab/2021/05/25/cifs-2021-call-for-papers/ https://blog.zhaw.ch/splab/2021/05/25/cifs-2021-call-for-papers/#respond Tue, 25 May 2021 09:14:25 +0000 https://blog.zhaw.ch/splab/?p=1396 We are delighted to organize the next iteration of the International Workshop on Cloud, IoT and Fog Systems (and Security) – CIFS 2021 which will be colocated with UCC 2021 conferences to be held in hybrid online mode and in Leicester, UK. CIFS 2021 website This is the call for papers: Description Sensor-based computing scenarios […]

Der Beitrag CIFS 2021 – Call for papers erschien zuerst auf Service Prototyping Lab.

]]>

We are delighted to organize the next iteration of the International Workshop on Cloud, IoT and Fog Systems (and Security) – CIFS 2021 which will be colocated with UCC 2021 conferences to be held in hybrid online mode and in Leicester, UK.

CIFS 2021 website

This is the call for papers:

Description

Sensor-based computing scenarios are on the rise. Smart cities install wide arrays of pollution, noise, temperature and traffic counting sensors. Machine equipment self-monitors to predict degradations. Monitoring by drones and satellites is becoming more affordable. There is a lack of systematic computing concepts, combining privacy and security requirements with essential performance and latency. For instance, raw video streams from dashcams must not be sent to the cloud for privacy reasons but can not always be fully processed with vehicle on-board units.

The CIFS workshop gathers researchers and innovators around the challenges of building and operating complex computing topologies and associated cloud architectures around the Internet of Things and Cyber-Physical Systems, bridging the digital world with our environment. It calls for research and experience reports, from the algorithm level to completed production systems, in order to advance the knowledge resulting from progress in overcoming the challenges. CIFS also welcomes data-centric studies, simulations, testbeds and similar helpful software solutions.

This year, CIFS particularly welcomes contributions on smart distributed systems resulting from the conscious processing and storage of information at suitable locations across the underlying infrastructures and platforms. These systems gain and exploit knowledge dynamically, for instance, to foresee and capacity bottlenecks and leverage alternative plans. This enables new applications, including liquid software with high dependability, gracefully overcoming the brittleness of many current IoT applications.

Topics of Interest

We solicit research papers (up to 6p) and technical industry reports (typically 3-6p) on the following topics:

  • IoT-fog-cloud pipelines, architectures and continuums in various domains, from industrial deployments to smart cities
  • Smartness, adaptivity, fluidity and dynamicity for processing sensed data
  • Computing paradigms: multi-cloud, fog, serverless, osmotic, continuum computing
  • Applications in digitalised health care, ecology, agriculture, urban planning and other fields
  • Secure cloud/fog computing concepts (homomorphic encryption, enclaves, hardware security modules, stealth computing, coded computing among others)
  • Long-term distributed data storage security and proof-of-ownership/provision/retrievability
  • End-to-end reliability and security concepts across IoT-fog-cloud continuums
  • Physical and digital security of IoT deployments, smart meters and other end devices
  • Privacy, scalability and economics considerations as well as trade-offs
  • Cross-border, cross-provider and cross-stack security issues
  • Human factors: simplicity and controllability of complex CPS and CPSoS
  • Computing paradigms: multi-cloud, fog, serverless, osmotic, continuum computing

Submission Information

Submitted papers should contain results or reports not already published or submitted elsewhere, in ACM format. All papers will be peer-reviewed by at least three programme committee members. The evaluation will be based on originality, relevance of the problem to the workshop topics, technical strength, quality of results, and clarity of the presentation. The publication of the workshop proceeding with all accepted papers will be by the ACM and will appear in the same or companion volume as the UCC 2021 and BDCAT 2021 conferences. At least one author of each accepted submission must register in full and attend the workshop to present and all workshop participants must pay the ACM conference or workshop registration fee.

Note: Whether you use the ACM LaTeX or Word templates, please make sure to submit in double-column format up to 6 pages including references.

Submission page

Important Dates

  • Submission August 15, 2021 → extended September 15
  • Notification October 1, 2021 → extended October 18
  • Camera-ready & registration October 31, 2021

All deadlines are indicated as AOE – Anywhere On Earth.

Technical Programme Committee

  • Joel Coffman, JHU, USA
  • Lorenzo DeCarli, WPI, USA
  • Nicola Dragoni, DTU, Denmark
  • Mozhdeh Farhadi, U-Hopper
  • Nicola J. Ferrier, ANL, USA
  • Jens Jensen, STFC, UK
  • George Kousiouris, HUA, Greece
  • Daniel Macedobatista, USP, Brazil
  • Natalia Miloslavskaya, MEPHI, Russia
  • Andrii Shalaginov, NTNU, Norway
  • Paul Pop, DTU, Denmark

Workshop Organisers

  • Panagiotis Gkikopoulos, Zurich University of Applied Sciences, Switzerland
  • Massimo Villari, University of Messina, Italy
  • Piyush Harsh, Terraview

Der Beitrag CIFS 2021 – Call for papers erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/05/25/cifs-2021-call-for-papers/feed/ 0
Structured Literature Dataset on Continuum Computing https://blog.zhaw.ch/splab/2021/05/19/structured-literature-dataset-on-continuum-computing/ https://blog.zhaw.ch/splab/2021/05/19/structured-literature-dataset-on-continuum-computing/#respond Wed, 19 May 2021 05:56:37 +0000 https://blog.zhaw.ch/splab/?p=1409 Continuum computing is seen as next evolutionary step after cloud computing, profoundly shaping the way software applications are built and delivered based around modern microservice architectures and serverless technologies. Complex software applications are decomposed, in part automatically, to yield the best possible runtime characteristics. The community around continuum computing is growing, and a growing body […]

Der Beitrag Structured Literature Dataset on Continuum Computing erschien zuerst auf Service Prototyping Lab.

]]>
Continuum computing is seen as next evolutionary step after cloud computing, profoundly shaping the way software applications are built and delivered based around modern microservice architectures and serverless technologies. Complex software applications are decomposed, in part automatically, to yield the best possible runtime characteristics. The community around continuum computing is growing, and a growing body of knowledge is correspondingly published.

Following our previous work on structured literature datasets, we would like to point out the interim release of the new dataset on continuum computing. The dataset has been curated by Julie Ann George.

http://continuum.research-output.org/

Der Beitrag Structured Literature Dataset on Continuum Computing erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/05/19/structured-literature-dataset-on-continuum-computing/feed/ 0
Explainable Deployment of Software to Computing Continuums https://blog.zhaw.ch/splab/2021/05/19/explainable-deployment-of-software-to-computing-continuums/ https://blog.zhaw.ch/splab/2021/05/19/explainable-deployment-of-software-to-computing-continuums/#respond Wed, 19 May 2021 04:58:43 +0000 https://blog.zhaw.ch/splab/?p=1401 Modern software applications need to accomodate many technical and business demands. Over the past decade, the dominant industry trend has been to decompose applications into smaller microservices and to deploy them as composition atop various cloud platforms and devices. More recently, more suitable abstractions were proposed to build and describe such software. Software becomes adaptive, […]

Der Beitrag Explainable Deployment of Software to Computing Continuums erschien zuerst auf Service Prototyping Lab.

]]>
Modern software applications need to accomodate many technical and business demands. Over the past decade, the dominant industry trend has been to decompose applications into smaller microservices and to deploy them as composition atop various cloud platforms and devices. More recently, more suitable abstractions were proposed to build and describe such software. Software becomes adaptive, liquid and osmotic within a continuum of computing resources, ranging from high-end data centres (e.g. HPC) and multiple clouds over fog/edge/middlebox systems to sensors and end user devices including machines and mobile phones.

The crucial step between authoring software applications and serving their functionality to the world is the deployment. Various approaches to deploy to continuums have been proposed, but most of them automagically arrive at a (presumably) suitable solution in terms of placing software components such as microservices across the infrastructure and platforms. We analysed the problems associated with such approaches and postulate that while fully automated approaches have their place, we need to complement them with interactive, guided and explainable deployment decisions. As application packaging improves (SAM, Helm, OAM, TOSCA, CAMEL), this new approach becomes increasingly relevant.

At the 35th International Conference on Advanced Information Networking and Applications (AINA 2021), a workshop dedicated to Multi-Clouds and Mobile Edge Computing (M²EC) took place, supported by the European H2020 project MELODIC, where we presented our approach Continuum Deployer along with an open source tool implementation. The main work on the implementation was conducted by Daniel Hass. Have a look at the presentation slides below and at the tool, and let us know of any feedback!

Der Beitrag Explainable Deployment of Software to Computing Continuums erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/05/19/explainable-deployment-of-software-to-computing-continuums/feed/ 0
Call for Contributions: 14th IEEE/ACM UCC in Leicester, UK https://blog.zhaw.ch/splab/2021/04/28/call-for-contributions-14th-ieee-acm-ucc-in-leicester-uk/ https://blog.zhaw.ch/splab/2021/04/28/call-for-contributions-14th-ieee-acm-ucc-in-leicester-uk/#respond Wed, 28 Apr 2021 15:12:42 +0000 https://blog.zhaw.ch/splab/?p=1393 The digital transformation of all areas of life is accelerated in this decade by novel cloud services, e-infrastructures, data platforms and cyber-physical system integration. This broader scope of cloud computing calls for technically sound contributions that combine scale with convenience and reliability. Society and economy depend on cloud applications delivering compute power on demand in […]

Der Beitrag Call for Contributions: 14th IEEE/ACM UCC in Leicester, UK erschien zuerst auf Service Prototyping Lab.

]]>
The digital transformation of all areas of life is accelerated in this decade by novel cloud services, e-infrastructures, data platforms and cyber-physical system integration. This broader scope of cloud computing calls for technically sound contributions that combine scale with convenience and reliability. Society and economy depend on cloud applications delivering compute power on demand in every location along data paths as a general service to the public, in analogy to conventional utilities. The engineering of such systems and applications calls for scientifically proven approaches, methods, tools and technologies.

Providing a forum to review and discuss possible solutions, UCC is the premier IEEE/ACM conference for areas related to Cloud Computing as a Utility where leading researchers and practitioners in this important and growing field gather on an annual basis.

After a period of enforced online meetings, we are looking forward to UCC 2021 being one of the first major conferences to be conducted in hybrid mode, permitting direct interactions between participants. The call for papers and other participation information is already on the website.

UCC 2021 website

Der Beitrag Call for Contributions: 14th IEEE/ACM UCC in Leicester, UK erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/04/28/call-for-contributions-14th-ieee-acm-ucc-in-leicester-uk/feed/ 0
HDocker and Docker Image Analysis Accepted for DAIS 2021 https://blog.zhaw.ch/splab/2021/04/25/hdocker-and-docker-image-analysis-accepted-for-dais-2021/ https://blog.zhaw.ch/splab/2021/04/25/hdocker-and-docker-image-analysis-accepted-for-dais-2021/#respond Sun, 25 Apr 2021 11:56:33 +0000 http://blog.zhaw.ch/splab/?p=1389 We conducted joint work with Université de Neuchâtel on improving the handling of Docker container images in the increasingly heterogeneous hardware environments. We propose to (1) finer-grained incorporate hardware dependency information in the image metadata, (2) leveraging heuristic analysis techniques to populate such information at large scale (although of course preferring properly curated metadata), and […]

Der Beitrag HDocker and Docker Image Analysis Accepted for DAIS 2021 erschien zuerst auf Service Prototyping Lab.

]]>
We conducted joint work with Université de Neuchâtel on improving the handling of Docker container images in the increasingly heterogeneous hardware environments. We propose to (1) finer-grained incorporate hardware dependency information in the image metadata, (2) leveraging heuristic analysis techniques to populate such information at large scale (although of course preferring properly curated metadata), and (3) improving the tool support around container creation from images. The work has led to new tools like hdocker and heuristic analysis rules. Furthermore, to underline the need for such a solution, we have been conducting a long-term tracking over fourteen now seventeen months of selected subsets of registered Docker container images.

This work has been accepted by the 21st International Conference on Distributed Applications and Interoperable Systems (DAIS 2021). Ahead of the event we already provide the collected data and code. Have a look!

Der Beitrag HDocker and Docker Image Analysis Accepted for DAIS 2021 erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/04/25/hdocker-and-docker-image-analysis-accepted-for-dais-2021/feed/ 0
Recent Progress in Digitally Supported Education https://blog.zhaw.ch/splab/2021/04/25/recent-progress-in-digitally-supported-education/ https://blog.zhaw.ch/splab/2021/04/25/recent-progress-in-digitally-supported-education/#respond Sun, 25 Apr 2021 11:00:00 +0000 http://blog.zhaw.ch/splab/?p=1364 Zurich University of Applied Sciences is a campus-based university with high quality presence education in bachelor, master and continuous education programmes. There are situations in which no presence is possible – most recently involuntarily invoked by the COVID situation – where smart digital solutions help with educational procedures in online or hybrid teaching modalities. Additionally, […]

Der Beitrag Recent Progress in Digitally Supported Education erschien zuerst auf Service Prototyping Lab.

]]>
Zurich University of Applied Sciences is a campus-based university with high quality presence education in bachelor, master and continuous education programmes. There are situations in which no presence is possible – most recently involuntarily invoked by the COVID situation – where smart digital solutions help with educational procedures in online or hybrid teaching modalities. Additionally, there are also situations in which such solutions might be helpful as a general complement to all teaching modalities including presence. Eventually, the goal is to ensure highly qualified alumni to boost the local economy and to fill the pipeline of next-generation researchers. A lot of the public debate has been on lectures alone. In this blog post, recent progress on digitally supported labs and examination is being summarised, demonstrating that Zurich University of Applied Sciences is also active on the solution side with novel science-based tool, service and solution designs. We are looking forward to colleagues from other institutions to discuss and advance these solutions further with us.

Digital learning games

We have recently designed TryPy, an educational game in which students answer simple questions related to the syntax and semantics of the Python programming language. As they do so, they progress through the levels. Being an online service, we will occasionally extend it with more questions, and thus students remain motivated to play it again and again to sharpen their skills. The currently implemented questions range from simple assignments (a = 10) to more complex Lambda function formulation (lambda x: x ** 2). The questions can be consecutive or connected through other means so that student answers from that point will influence answers to later questions. The service can be used anonymously so that students do not have to worry about failing.

Git repo TryPy

Cyber-physical programming labs

Two years ago we broadly introduced robotic device programming across the engineering curriculums. During the pandemic, obviously not every student can take robots home, and they also cannot visit the robots on campus. We have thus introduced a remote programming solution called EPOSS that works as bidirectional feedback loop. Students send instructions and see how the robot moves and reacts so that they can tune and debug their applications accordingly. The implementation is specific to EV3 robots but conceptually other devices can be plugged in as well. Some basic functionality of the robot can also be simulated, leading to a flexible environment in which cyber-physical scenarios can be realised.

Git repo EPOSS

Invididualisation in exams

We have read about cheating cases in written exams across universities world-wide and thought of a clever solution to reduce the cheating by inherently reducing the potential to cheat in the first place. This is a certain upfront effort to come up with appropriate tools that will pay off in the long term by filtering out unfit engineers from the job market, especially considering the increasing employment of engineers in safety-critical jobs. Students will get individualised exams, but examiners will also get individualised reference solutions that are programmatically generated from the same input in order to cut down the correction and grading effort. We have released a first prototypical implementation of a fully programmable individualisation concept called FIPE.

Git repo FIPE

Future progress

The approaches on cloud-facilitated device programming and exam individualisations were discussed at the CSEDU 2021 conference. With conventional proctoring solutions being a detriment to student privacy while at the same time allowing for circumvention and often not being more than a placebo (thanks to U Twente in NL for that cue), it appears that individualisation might have a role in future digitally conducted exams in general, both online and in presence (e.g. BYOD programming exams on campus).

The cyber-physical programming could be considered a first step towards generic digital twinning environments with applicability to industrial needs, for instance, sharing complex machine functionality, in part within a simulation or twin and in part for real, between teams across the world.

The quiz game is already deployed online. Want to give it a try? Use a TCP client (e.g. telnet, ncat, putty) and connect to host 160.85.252.148 / port 10080. Can you make it to the boss level? We are now looking forward to look at the conjunction of gamification and examination. Will future assessments be based on games? How can we best train our future workforce, the GenZ, to prepare them for the environments they will likely perform in (thanks to Oulu U for another cue)?

Der Beitrag Recent Progress in Digitally Supported Education erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/04/25/recent-progress-in-digitally-supported-education/feed/ 0
Switzerland Online: Market Share Estimation https://blog.zhaw.ch/splab/2021/02/01/switzerland-online-market-share-estimation/ https://blog.zhaw.ch/splab/2021/02/01/switzerland-online-market-share-estimation/#respond Mon, 01 Feb 2021 17:35:10 +0000 http://blog.zhaw.ch/splab/?p=1360 With the ongoing pandemic and the resulting home office / home study regulations, little is known about how people go online, especially with emphasis on local conditions. Based on recent student interactions, we give an estimation of the market shares of network providers, operating systems and web browsers. This empirical mini-study analysis is based on […]

Der Beitrag Switzerland Online: Market Share Estimation erschien zuerst auf Service Prototyping Lab.

]]>
With the ongoing pandemic and the resulting home office / home study regulations, little is known about how people go online, especially with emphasis on local conditions. Based on recent student interactions, we give an estimation of the market shares of network providers, operating systems and web browsers.

This empirical mini-study analysis is based on n=495 unique interactions, or 3182 total interactions, of around 350 users in Switzerland. All users are young engineering students, a population of high interest from a market share analysis perspective. Some of the students use multiple network connections (e.g. roaming with notebook between university VPN and home network) or even multiple devices (e.g. checking documents quickly on the mobile phone and then working on them with the notebook or PC).

The study differentiates 5 operating systems groups, 3 web browser groups and 8 network provider groups (each including one pseudo entry for other/unknown). Some of the network providers are sub-providers, reducing the total groups to 5. The other/unknown group is quite large for the network providers, due to the high number of local providers including city-level fibre. Noteworthy is the complete absence of Linux from the operating systems. The absence of Edge and other browser variants can on the other hand be attributed to the basic level of differentiation; more details might be reported in a follow-up study.

Without further ado, the figure below contains the respective market share estimations in surface-proportional visualisation, including inter-relations between networks/OS and OS/browser.

Hence, the archetypical engineering student in 2021 uses Swisscom (34%) or UPC (29%) to browse with Chrome (70%) on Windows (83%). Is such a majority setup a concern, for instance from a security perspective? Does this setup fulfil the needs of our students in remote or hybrid teaching settings? How will the numbers look in some years from now? These questions can now be discussed based on data.

Der Beitrag Switzerland Online: Market Share Estimation erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2021/02/01/switzerland-online-market-share-estimation/feed/ 0
Leveraging Docker and reproducible workflows for portable research environments https://blog.zhaw.ch/splab/2020/11/04/leveraging-docker-and-reproducible-workflows-for-portable-research-environments/ https://blog.zhaw.ch/splab/2020/11/04/leveraging-docker-and-reproducible-workflows-for-portable-research-environments/#respond Wed, 04 Nov 2020 15:29:51 +0000 http://blog.zhaw.ch/splab/?p=1351 Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a ‘best practices’ approach for research. However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are […]

Der Beitrag Leveraging Docker and reproducible workflows for portable research environments erschien zuerst auf Service Prototyping Lab.

]]>
Reproducibility is an important aspect of research. One particular concept of interest is FAIR principles. FAIR stands for Findable, Accessible, Interoperable and Reusable and defines a ‘best practices’ approach for research.

However, complying with such a concept presents its own unique challenges: Some research tools rely on complex infrastructure elements and software stacks that are hard to replicate. In cases such as these, the added workload acts as an inhibitor on ensuring reproducibility. The result is that many experiments, data pipelines and results are hard to replicate. According to our preliminary insights, based on interviews within the ZHAW, on how experiments are conducted, such complexity is indeed commonplace in our current research activities. 

It would thus be great, if the entire complex environment could be set up within a portable container, that anyone can simply download and use with no additional steps. Enter Docker. By no means ‘new’ technology, it has been around for several years and has been proven in many fields. Its well-documented advantages in software portability can also be leveraged to assist in creating portable and reproducible research software. A Dockerized research tool can run on any host that meets the hardware requirements with minimal setup, thus enabling a much simpler process of replicating the experiments and results of the original authors. There are already examples of leveraging Docker to improve reproducibility in research software, both internationally and within Switzerland. That said, eliminating the software setup is only one part of the equation.

In addition to making the software itself portable, the entire workflow of using it can be made reproducible with novel tools that enhance reproducibility. Examples include OSF and SDSC’s Renku. Renku is a tool designed entirely with the FAIR principles in mind, that augments the versioning functionality of git with the added benefits of a knowledge graph and other key features. Researchers can set up their repository much like with traditional git, with the added advantage of being able to record their entire workflow as they run it. This is stored in the repository’s knowledge graph and can be rerun on demand when new data or code changes are committed.

Both of these elements can be argued to provide an approach for increased reproducibility of research in isolation, nevertheless, we believe there are key ways in which they can complement each other to combat each other’s limitations.

Docker can make even some of the more complex tools reproducible, but researchers still need to learn how to operate the tool correctly to replicate a result.

Renku can record such a workflow and ‘replay’ it on command, assigning a much easier role to the human operator, but is restricted in the complexity of tools that it can natively manage.

We thus see the benefit in a combined approach, with the development of customized ‘Renkurized’ images that contain the necessary software stack as part of the Docker image, while also giving access to the reproducible workflow functionality and knowledge graph of Renku. Such images would also become available in a dedicated catalogue, thus making the tools easily findable and accessible to all researchers and students. Public catalogues, such as Docker Hub are also viable and indeed there are interesting tools available there already, but we argue that to guarantee long-term availability to researchers even if the public portal were to move or change, a solution dedicated to our research is still the better choice.

Questions to be answered include how feasible it is to transform complex research setups into these custom container images, and whether this solution can be sustainable, with researchers within the ZHAW willing to shift to this approach for reproducibility in the future. We expect to answer these questions with an extended study of individual use-cases within the ZHAW.

Der Beitrag Leveraging Docker and reproducible workflows for portable research environments erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2020/11/04/leveraging-docker-and-reproducible-workflows-for-portable-research-environments/feed/ 0
Processing IoT Sensor Data with Semi-Stateful Cloud Functions https://blog.zhaw.ch/splab/2020/10/27/processing-iot-sensor-data-with-semi-stateful-cloud-functions/ https://blog.zhaw.ch/splab/2020/10/27/processing-iot-sensor-data-with-semi-stateful-cloud-functions/#respond Tue, 27 Oct 2020 13:45:06 +0000 http://blog.zhaw.ch/splab/?p=1343 At Zurich University of Applied Sciences, we are currently building a test track to link applied teaching with research and innovation. Such a facility allows for covering a whole range of topics: programming, autonomous driving, robotics, cloud, serverless, continuums, sensing, open data, data science, and various computing paradigms. We expect a video to be available […]

Der Beitrag Processing IoT Sensor Data with Semi-Stateful Cloud Functions erschien zuerst auf Service Prototyping Lab.

]]>
At Zurich University of Applied Sciences, we are currently building a test track to link applied teaching with research and innovation. Such a facility allows for covering a whole range of topics: programming, autonomous driving, robotics, cloud, serverless, continuums, sensing, open data, data science, and various computing paradigms. We expect a video to be available around November that explains the facility and especially the teaching element. In this research blog post, we already report on interesting observations around the uplink between sensors and FaaS. We expect these insights to bring benefits to companies building IoT-cloud integrations.

The (simplified) architecture of the system is shown below. Various types of sensors report events to cloud functions. These functions are typically considered stateless, although in practice they are never completely (see older related post on building probabilistically stateful applications). To manage state over a longer period of time, an in-memory database is attached to the function; for security reasons, the access is controlled by a restricted virtual network. As we are still covered by GCP Research Credits for Serverless Data Integration, we are basing this work on the respective GCP offerings: GCF, VPC Connector, and the Redis Memorystore.

The ‘hit’ function will for instance increase a counter anytime a car passes the corresponding sensor. It reads the Redis credentials from environment variables. Furthermore, it reads a secret from the environment too, with a fallback to a variable definition in a secrets file. This is a pragmatic way to protect against damaging unauthorised function calls: Instead of requiring authentication at the invocation level, the function is open for public invocation but any actual processing, for example writing values to Redis, is protected. We regard low-effort starting with technologies important for prototypical research and teaching, and for this reason also disregarded GCP’s IoT Core that came with heavy double authentication (keys + JWT) requirements that are redirecting too much effort into security engineering rather than freeing the effort up for advances in technology. (Of course we assume and demand that anybody who operates such solutions in production would then take care of appropriate security.)

Once the described system was up and working, the research question emerged: Is Redis really needed? Could we not just store the counter variable as global cloud function variable, hence letting it survive requests?

There are two potential problems that may interfere with that idea. Function instances (the containers or equivalent isolation units beneath FaaS) are kept around for some time, but not forever. Specifically, an invocation will get a new instance if:

  • Parallelism occurs. In this case, a second instance is temporarily opened. Once the parallelism stops, the older instance takes precedence and the second instance is no longer used. Hence, a typical counting sequence looks like 124, 125, 1, 126, 127.
  • Recycling occurs. In this case, the original instance is permanently replaced with a new one after an idle period. This problem is potentially even more complex (a whole host VM with multiple containers may migrate or be replaced); for simplicity we are not distinguishing the reasons why a function instance goes away, but we want to know how likely it is. A typical counting sequence looks like 130, 131, 132, 1, 2, 3, 4.

Research on function instance recycling has led to interesting insights. In SLD#97, Manner et al. from University of Bamberg say that “containers on most platforms were shut down after 20 minutes of idling”. In SLD#37, Lloyd et al. from University of Washington state that all AWS containers were recycled after 45′. Numbers for GCF were not specifically reported by both, and also not in our own previous work where we observed that IBM Cloud Functions get recycled after about 10′. In another work (citation currently missing), van Eyk et al. presented a rough estimation about ~6 hours recycling time on GCF, which would be long enough to start codifying it into optimised processing logic.

With our own preliminary observations, we can confirm that. Several invocations with long 60-90′ (and even 240′) intervals in between led to no reset of the counter. This means that in controlled single tenant environments, like a function exclusively served by a single sensor or by multiple sensors when concurrency can be avoided (e.g. through an ordered queue in front of the function), using global state variables, from simple counters to complex aggregated statistics, is a viable, and highly economical, alternative to using external databases. We anticipate that advanced FaaSification frameworks allow for simply marking variables to be stateful, and then a code generator takes care of either safely using a backend store or juggling the data between global variables and occasional backend snapshots.

Der Beitrag Processing IoT Sensor Data with Semi-Stateful Cloud Functions erschien zuerst auf Service Prototyping Lab.

]]>
https://blog.zhaw.ch/splab/2020/10/27/processing-iot-sensor-data-with-semi-stateful-cloud-functions/feed/ 0