How to use Docker Swarm secrets to store and rotate your SSL certificates with Nginx

in DevOps , Containerization

dockerswarm nginx.png

    Docker Swarm has an excellent feature out of the box — Docker Swarm secrets. Using it you can easily keep your sensitive data like credentials, TLS certificates, etc.

    In terms of Docker Swarm services, a secret is a blob of data, such as a password, SSH private key, SSL certificate, or another piece of data that should not be transmitted over a network or stored unencrypted in a Dockerfile or in your application’s source code. You can use Docker secrets to centrally manage this data and securely transmit it to only those containers that need access to it.

    So, if we want to use it to store our certificates, at first we need a certificate. Here we have two options:

    • Use self-signed certificate
    • Buy SSL certificate

    We will use self-signed:

    ~$ mkdir certs && sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./certs/nginx.key -out ./certs/nginx.crt

    Command above generates certificate which expiration date is 1 year and place it in ./certs/ directory.

    Now we have key and crt files and we already can use it. But besides it, we should always monitor the certificate expiration date. Sure there are a few ways to do it, but it is out of scope topic. Just keep in mind that you can use alerts (Prometheus + Blackbox exporter) of certificate expiration date to trigger your script which in its turn updates the secret with the certificate.

    Now we need to create an nginx docker service with our certificate. Here is my docker-compose file with secrets section:

    version: '3.4'
    
    services:
      ingress_nginx:
        image: ingress_nginx
        build:
          context: .
        ports:
          - "80:80"
          - "443:443"
        networks:
          - network
        deploy:
          mode: global
          restart_policy:
            condition: any
            delay: 5s
          update_config:
            delay: 30s
            parallelism: 1
            failure_action: rollback
        secrets:
          - source: nginx_key
            target: /etc/nginx/nginx.key
          - source: nginx_cert
            target: /etc/nginx/nginx.crt
    
    secrets:
      nginx_key:
        file: ./certs/nginx.key
      nginx_cert:
        file: ./certs/nginx.crt
    
    networks:
      network:
        driver: overlay

    You should keep in mind that you cannot update docker secrets on the fly. It means that you should create a dummy secret every time and replace the old secret with a dummy secret. This is an example script how to update an existing secret:

    # Create dummy secret
    echo "<<< Creating dummy secret >>>" 
    docker secret create dummy_key nginx.key
    docker secret create dummy_crt nginx.crt
    
    # Delete old certificate and key from docker secret and replace them with dummy
    echo "<<< Delete old certificate and key from service and replace them with dummy >>>" 
    docker service update \
        --secret-rm ${stack}_nginx_key \
        --secret-rm ${stack}_nginx_cert \
        --secret-add source=dummy_key,target=/etc/nginx/nginx.key \
        --secret-add source=dummy_crt,target=/etc/nginx/nginx.crt \
        ${stack}_ingress_nginx
        
    echo "<<< Delete old certificate from secrets >>>"
    docker secret rm  ${stack}_nginx_key
    docker secret rm  ${stack}_nginx_cert
    
    # Deploy service with new secrets
    echo "<<< Create secret with new certificate and update service >>>"
    docker stack deploy --compose-file docker-compose.yml $stack
    
    # Delete dummy secrets
    echo "<<< Delete dummy certificate >>>"
    docker secret rm dummy_key
    docker secret rm dummy_crt

    After script execution, we have updated the certificate inside nginx container.


    Get similar stories in your inbox weekly, for free



    Share this story with your friends
    pvarfalameev
    Pavel Varfalameev, DevOps engineer @ Byndyusoft

    DevOps engineer at Byndyusoft.com

    Latest stories


    200 Million Certificates in 24 Hours

    Let's Encrypt has been providing free Certificate Authority (CA) for websites in need of them …

    Gatling VS K6

    Gatling and K6 are performance load testing tools, and they are both open source, easy …

    Red Hat Ansible Platform 1 vs 2; What’s the Difference?

    Red Hat Ansible is a platform used by enterprises to manage, unify and execute infrastructure …

    Domino Data Labs Raised $100 Million in the Latest Funding Round

    Culled from the news released by Domino Data labs on funding and the company's progress …

    New Release: The Microsoft Azure Purview Is Now Available on General Availability

    News report detailing the announcement of the release of Azure purview on GA

    Google Introduces Online Training Program to Improve Cloud Skills

    Google addresses existing cloud personnel deficiency with training programs.