Docker

• 1: Configure
• 2: Parameter
• 3: Administration
• 4: Playbook
• 5: Monitor
• 6: FAQ

    Configure docker registry, proxy, images, etc...

    Customize docker components with 8 parameters

    Manage docker images, containers, etc...

    Ansible playbooks that can be used in docker module

    Dashboards, metrics, record & alerting rules.

    Frequently Asked Questions about docker module

1 - Configure

Pigsty includes built-in Docker support, allowing you to quickly deploy containerized applications.

Quick Start

To install docker on nodes, set the docker_enabled parameter to true.

all:
  vars:

    infra:
      hosts:
        10.10.10.10: { infra_seq: 1, nodename: infra-1 }
        10.10.10.11: { infra_seq: 2, nodename: infra-2 }
      vars:
        docker_enabled: true  # Install Docker on this group

Then run the docker.yml playbook (on target hosts/groups):

./docker.yml -l infra

Docker will be installed on that infra group.

Registry

You can specify docker registry mirrors with docker_registry_mirrors:

all:
  vars:
    docker_registry_mirrors: ["https://docker.1ms.run"]

Here are some example registry mirrors:

• Alibaba Cloud: ["https://registry.cn-hangzhou.aliyuncs.com"]
• Tencent Cloud: ["https://ccr.ccs.tencentyun.com"]
• DaoCloud: ["https://docker.m.daocloud.io"]
• 1Ms: ["https://docker.1ms.run"]

You can specify multiple registry mirrors as an array, remember to quote the URL with ".

Proxy

Docker will use the proxy_env parameter if specified.

You can define it on global parameters all.vars or dedicate groups (such as infra ):

all:
  vars:
    proxy_env:
      no_proxy: "localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/16,*.pigsty,*.aliyun.com,mirrors.*"
      http_proxy: 'http://127.0.0.1:12345'
      https_proxy: 'http://127.0.0.1:12345'
      all_proxy: 'http://127.0.0.1:12345'

It will be rendered to /etc/docker/daemon.json during the docker_config task:

{
  "proxies": {
    "http-proxy": "127.0.0.1:12345",
    "https-proxy": "127.0.0.1:12345",
    "no-proxy": "localhost,127.0.0.1,10.0.0.0/8,192.168.0.0/16,*.pigsty,*.aliyun.com,mirrors.*,*.tsinghua.edu.cn"
  }
}

This is useful if direct network access is blocked due to various reasons.

Images

You can provision docker images with docker_image and docker_image_cache:

infra:
  hosts:
    10.10.10.10: { infra_seq: 1 }
  vars:
    docker_enabled: true
    docker_image:
      - redis:latest
    docker_image_cache: "/tmp/docker/*.tgz"

The images defined in docker_image will be pulled during the docker_image task, one by one. And the local docker image cache with .tgz suffix matching the docker_image_cache glob list will be loaded into docker with docker load

Accelerate

You can use accelerators on each cloud vendors:

• Aliyun ACR : https://cr.console.aliyun.com/cn-shanghai/instances/mirrors

2 - Parameter

There are 8 parameters for the Docker module:

Defaults

Docker’s default parameters are defined in roles/docker/defaults/main.yml

docker_enabled: false             # enable docker on this node?
docker_data: /var/lib/docker      # docker data directory, /var/lib/docker by default
docker_storage_driver: overlay2   # docker storage driver, can be zfs, btrfs
docker_cgroups_driver: systemd    # docker cgroup fs driver: cgroupfs,systemd
docker_registry_mirrors: []       # docker registry mirror list
docker_exporter_port: 9323        # docker metrics exporter port, 9323 by default
docker_image: []                  # docker image to be pulled after bootstrap
docker_image_cache: /tmp/docker/*.tgz # docker image cache glob pattern

docker_enabled

name: docker_enabled, type: bool, level: G/C/I

enable docker on this node? default value is false

docker_data

name: docker_data, type: path, level: C

Docker data directory, /var/lib/docker by default.

docker_storage_driver

name: docker_storage_driver, type: enum, level: C

Docker storage driver, overlay2 by default.

Please refer to: https://docs.docker.com/engine/storage/drivers/select-storage-driver/

• overlay2
• fuse-overlayfs
• brtfs
• zfs
• vfs

docker_cgroups_driver

name: docker_cgroups_driver, type: enum, level: G/C/I

docker cgroup fs driver, could be cgroupfs or systemd, default values: systemd

docker_registry_mirrors

name: docker_registry_mirrors, type: string[], level: G/C/I

docker registry mirror list, default values: [], Example:

Here are some examples of using the internal network image of each cloud vendor:

["https://docker.m.daocloud.io"]                # domestic DaoCloud image site
["https://docker.1ms.run"]                      # domestic millisecond image site
["https://mirror.ccs.tencentyun.com"]           # tencent cloud intranet image site
["https://registry.cn-hangzhou.aliyuncs.com"]   # aliyun cloud intranet image site, login required

Consider using Cloudflare Worker Docker Proxy

If the pull speed is too slow, you can also consider: docker login quay.io use other Registry.

docker_exporter_port

name: docker_exporter_port, type: port, level: G

Docker metrics exporter port, 9323 by default.

docker_image

name: docker_image, type: string[], level: G/C/I

docker image to be pulled, [] by default

The Image listed here will be pulled during docker provisioning.

docker_image_cache

name: docker_image_cache, type: path, level: G/C/I

docker image cache tarball glob list, "/tmp/docker/*.tgz" by default.

The local docker image cache with .tgz suffix match this glob list will be loaded into docker one by one:

cat *.tgz | gzip -d -c - | docker load

3 - Administration

Install

To install & enable docker on nodes, configure the docker_enabled parameter to true.

all:
  vars:

    infra:
      hosts:
        10.10.10.10: { infra_seq: 1, nodename: infra-1 }
        10.10.10.11: { infra_seq: 2, nodename: infra-2 }
      vars:
        docker_enabled: true  # Install Docker on this group

Then run the docker.yml playbook (on target hosts/groups):

./docker.yml -l infra

Docker will be installed on that infra group.

Repository

Docker repo is part of the infra repo module, and will be automatically added during repo build.

- name: docker-ce
  description: 'Docker CE'
  module: infra
  releases: [7,8,9]
  arch: [x86_64, aarch64]
  baseurl:
    default: 'https://download.docker.com/linux/centos/$releasever/$basearch/stable'
    europe:  'https://mirrors.xtom.de/docker-ce/linux/centos/$releasever/$basearch/stable'
    china:   'https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable'

- name: docker-ce
  description: 'Docker CE'
  module: infra
  releases: [11,12,20,22,24]
  arch: [x86_64, aarch64]
  baseurl:
    default: 'https://download.docker.com/linux/${distro_name} ${distro_codename} stable'
    china: 'https://mirrors.aliyun.com/docker-ce/linux/${distro_name} ${distro_codename} stable'

You can add this repo to your nodes with:

./node.yml -t node_repo -e node_repo_modules=infra -l infra

Upgrade

To upgrade Docker Daemon, using the ansible command, add docker repo, then:

ansible infra -m package -b -a 'name=docker-ce state=latest'

It will upgrade the docker-ce package to the latest version available on your configured repositories.

Remove

To remove Docker Daemon, using the ansible command to run:

ansible infra -m package -b -a 'name=docker-ce state=absent'

It will remove the docker-ce package with your os package manager.

Applications

Pigsty provides ready-to-use software templates based on Docker Compose to deploy external applications seamlessly integrated with Pigsty-managed database clusters.

4 - Playbook

The DOCKER module has only one playbook: docker.yml to install docker daemon & docker compose on target node.

docker.yml

The raw playbook: docker.yml.

Run this playbook on any host will install docker-ce and docker-compose-plugin on target node with docker_enabled: true flag.

Here are the available subtasks in the docker.yml playbook:

• docker_install: Install Docker and Docker Compose packages on the node.
• docker_admin: Add specified users to the Docker administrator user group.
• docker_config: Generate Docker daemon service configuration file.
• docker_launch: Start the Docker daemon service.
• docker_register: Register Docker daemon as a Prometheus monitoring target.
• docker_image: Attempt to load prepackaged Docker images from /tmp/docker/*.tgz if present.

The Docker module does not provide a dedicated playbook for uninstalling Docker. If you need to uninstall Docker, you can manually stop the Docker service and then uninstall it:

systemctl stop docker                        # Stop Docker daemon service
yum remove docker-ce docker-compose-plugin   # Uninstall Docker on EL systems
apt remove docker-ce docker-compose-plugin   # Uninstall Docker on Debian systems

5 - Monitor

Pigsty will add docker daemon to monitoring target if that node is docker_enabled = true

But there are no default dashboards and alerting rules for docker module, you can add your own rules to prometheus & grafana.

6 - FAQ

Who Can Run Docker Commands?

By default, Pigsty adds both the management user running the playbook on the remote host (i.e., the SSH login user) and the user defined by the node_admin_username parameter to the operating-system group docker. Any account in this group can manage Docker via the docker CLI.

Need to grant another user Docker access? Just add that OS user to the docker group:

sudo usermod -aG docker <username>

Working Through a Proxy

During installation, if the proxy_env parameter is set, Pigsty writes the specified HTTP proxy settings to /etc/docker/daemon.json.

Docker will then route all image pulls from upstream registries through this proxy.

Tip: Running the configure playbook with the -x flag automatically captures your current shell’s proxy variables and injects them into proxy_env.

Using Mirror Registries

Inside mainland China you may encounter Great Firewall restrictions. Historically, mirrors such as quay.io could be used:

docker login quay.io   # enter your credentials to log in

Update (June 2024): All previously accessible Docker mirrors in China have now been blocked. Please pull images via a proxy.

Adding Docker to Monitoring

When the Docker module is installed, you can register Docker as a Prometheus target for a specific node by running the docker_register (alias register_prometheus) task:

./docker.yml -l <your-node-selector> -t register_prometheus

Software Templates

Pigsty ships a collection of software templates that launch popular stacks with Docker Compose—ready to use out of the box.

Just make sure the Docker module is installed first.