Introducing the Docker Compose Validator

When cloud application developers are working with docker-compose to combine multiple microservices into a single manageable entity, they can make some easy mistakes. To prevent these mistakes, they can rely on internal validation logic, which however does not catch many of the typical issues. Therefore, researchers at the Service Prototyping Lab at Zurich University of Applied Sciences wrote a dedicated quality check and assessment tool targeting developers, but also students trying to learn the technology, which has a wider range of checks. The DCValidator tool is available as a web application (see demo instance) or command-line interface. This blog post describes how to check that docker-compose files are free of issues.

One might ask what is the purpose of this tool when there is the built-in docker-compose validator? This validator unfortunately fails to validate many issues. For example, duplicate keys that result from copy & paste actions, which are incredibly common during the creation of the arguably complex YAML compose files. The built-in YAML parser of docker-compose by default ignores the duplicate keys. Imagine that you have some services in your compose file and two of them have the same name but they are different services. In this case, you will lose one of your services and this is not good news! Similar duplication issues exist for port allocations.

Before we start, let us talk about some common mistakes that developers and students make.
1- The most common things are typo mistakes.
2- Duplicate keys and tags that we mentioned before are another kind of mistakes we can make
3- And some mistakes in values of keys.

Let’s see an example:

version: '3'

services:
  app:
    build: 
      context: .
      args:
        - IMAGE_VERSION=3.7.0-alpine3.8
    image: takacsmark/flask-redis:1.0
    env_file: .env.txt
    ports:
      - 80:5000
      - 80:5000
    dns: 888.888.8888
    networks:
      - mynet
  app2:
    image: ubuntu:latest
    command: ["tail", "-f", "/dev/null"]
  redis:
    imge: redis:4.0.11-alpine
    networks:
      - mynet
    volume:
      - mydata:/data

network:
  mynet:
volumes:
  mydata:

In this case, we have duplicate ports, typo mistake, and bad DNS. These issues can be found out with the Docker Compose Validator, as explained by the video below.

Validating and Quality-Checking Docker Compose Files

Defaulttext aus wp-youtube-lyte.php

Sometimes you are working on a project and for validating your docker-compose it’s a nightmare using web application! We anticipate batch use (e.g. for CI/CD integration or developer preference for terminals) and because of that, there is a CLI!

When you run the script you may see something like this:

$ python validator-cli.py


Warning: no kafka support
Syntax: validator-cli.py [-a <org>/<basepath>] [-f <file>] [-u <url>] [-e <kafka>/<space>/<series>]
 -a: autosearch; find appropriate compose files on GitHub
 -f: filebased; load docker-compose from a docker-compose file
 -fl: filebased list; load paths or URLs as lines from a text file
 -u: urlbased; direct URL or path specification
 -e: eventing; send results to Kafka endpoint with space and series selection
 -fi: filters; you can select filters as any as you want:
    for more info flag --fih might be helpful!
Example: validator-cli.py -a elastest/deploy -e kafka.cloudlab.zhaw.ch/user-1-docker_label_consistency/nightly -fi 'Duplicate Keys,Top level property'

And when we passed our docker-compose file to the script, we got this result:

$ python validator-cli.py -u https://github.com/alidaghighi/docker-compose-test-cases/blob/master/Bad%20test%20case%202/docker-compose.yml -fi 'Duplicate Keys,Duplicate ports,Typing mistakes,DNS,Duplicate expose'
Warning: no kafka support
loading cache...
loading compose files....
checking consistency...
syntax is ok
=================== ERROR ===================
I can not find 'network' tag. Maybe you can use: 
networks

=============================================
= type: docker-compose
- service:app
Duplicate ports in service app port 80
=================== ERROR ===================
Under service: app
The DNS is not appropriate!
=============================================
- service:app2
- service:redis
=================== ERROR ===================
I can not find 'imge' tag under 'redis' service. Maybe you can use: 
image
I can not find 'volume' tag under 'redis' service. Maybe you can use: 
volumes
=============================================
services: 3
labels:
time: 0.0s
not sending message... {'agent': 'sentinel-generic-agent', 'services': 3.0, 'faulty:docker-compose-test-cases': 0.0}

Feel free to use and contribute! The code is available on the lab’s Git repository.