Cloud-Native Document Management

by Josef Spillner

Document management is an established software-powered business domain. As with most software applications, an ongoing trend is to move the functionality of document management systems (DMS) and related functionality (Enterprise Content Management – ECM, Content Services – CS) into well-defined services, primarily in cloud environments, resulting in cloud-native document management systems/services (CNDMS).

In the context of the research initiative on cloud-native applications and the ARKIS project within the Service Prototyping Lab, we have been looking deeper into the issues surrounding cloud-native document management and have built a prototype implementation to test-drive any ideas and new concepts. This post introduces the software and the challenges already solved with it.

Moving document management into the cloud raises important questions about rules, procedures and technologies. Some of the challenges are:

• How to deploy, run and operate the entire cloud-native application. Cloud application stacks evolve as do stack-specific languages to describe applications. The usage patterns are diverse, from batch document scans to full-text search, and the multi-tenancy and permission models are complex.
• Where to store and manage the metadata as well as the data. The obvious candidates are self-hosted (i.e. application-controlled) database services and provider-operated equivalents, each with their own shortcomings concerning strict compliance and auditing. We have previously introduced CNDBbench to get more insights and continue to do so by improving and extending the tool. Another challenge is to define a performing, secure and still affordable isolation of tenants.
• How to bill the users per use. This requires domain-specific metrics and business rules such as cost associated to document pages.
• How to establish an ecosystem around the core service. In document management, many specialised services around physical and digital documents exist and benefit from a uniformly specified service interface.

A CNDMS needs to be designed to answer these questions with satisfaction. Our design is the result of several studies and experiments in this direction.

The mentioned design has been prototypically implemented as a composite CNDMS application consisting of web frontend, processing backend and state handling parts. Each microservice is implemented as a container. All containers are orchestrated in various languages which we have covered before, for instance Vamp Blueprints and Kubernetes generated from Docker-Compose, and which we will cover soon, for instance Docker-Swarm. For simplicity reasons, the software is named after the project and versioned in incremental steps corresponding to the development iterations. Thus, ARKIS 2.5 is the latest prototype and incorporates the functionality to manage document metadata with various database server, service and multi-tenancy options. The following screenshot gives a first impression of the functionality.

Future work will focus on differentiated storage, more options in terms of implementation languages for the microservices, and additional large-scale experiments.