In the context of our «Smart Cities and Regions Services Enablement» efforts, space (and to some extent time) are important dimensions. First, the digital transformation has an inherent spatial component. While the research application field is pragmatically scoped to cities and regions, indeed it spans a wider spectrum from households, quarters, districts to countries and even supranational entities. The recent wave of «surface digitalisation» has primarily affected mobile citizens (pandemic apps) and workers (video conferencing in home offices) around the world. This increased the surface over the previous one that for most citizens encompassed e-banking, e-ticketing and e-tax declarations, with various degrees of voluntariness.
Many providers of hosted services, including cloud applications, are subject to a contradiction in handling log data. On the one hand, storing logs consumes resources and should be minimised or avoided altogether to save resource cost. On the other hand, regulatory constraints such as keeping the data for the purpose of future audits exist. A smart solution to encode the data appropriately needs to be found. The coding encompasses both compression, to keep resource use low, and encryption, to prevent leaking information to unauthorised parties, for instance when logging for the purpose of intrusion detection. On an algorithmic level, the encoded data should still be usable for computation, in particular comparison and search. In this blog post, based on the didactic log example shown in the figure below, we present algorithms and architectures to handle cloud log files in a smart way.