Preparation

• 1: Hardware
• 2: Linux OS
• 3: Software
• 4: Admin
• 5: Sandbox
• 6: Vagrant
• 7: Terraform

    Nodes, specs, disks, network, VIP, domain ...

    List of supported Linux OS distributions

    Locale, Firewall, Ansible, Pigsty...

    User, Sudo, SSH, Accessibility...

    Ansible 101, basic knowledge for pigsty users

    The four-node sandbox for learning & testing

    Provision local virtual machine with vagrant

    Provision cloud servers with terraform

1 - Hardware

Node

Pigsty currently runs on nodes with Linux kernel and x86_64 / aarch64 architecture.

A “node” refers to a resource that is SSH accessible and offers a bare Linux OS environment. It could be a physical machine, a virtual machine, or an OS-like container equipped with systemd, sudo and sshd.

It requires at least 1 node to deploy pigsty, You can prepare more and setup everything in one-pass, or add them later. The minimum node spec requirement is 1C1G, it is recommended to use at least 2C2G. Higher is better, with no upper limit. Parameters will be automatically tuned based on available resource.

Spec

How many nodes do you need? It depends on your resource and your requirements.

Disk

Pigsty will use /data as the default data directory, if you have a dedicated main data disk, it is recommended to mount it there, and use /data1, /data2, /dataN for extra disk drivers.

We recommend using ext4 or xfs as the filesystem for data disks. They have the best performance for PostgreSQL.

While ext4 has more tools for data recovery, xfs is more efficient for small files. If you are running MinIO, It’s recommended to use xfs, otherwise, ext4 is recommended as the default option.

Network

Pigsty requires static network to work, you should explicitly assign a fixed IPv4 address for each node.

The IP address will be used as the node’s unique identifier, it should be the primary IP address bind to the primary network interface used for internal network communications.

Internet access is required when performing the standard (online) installation. But pigsty can be offline installed via offline package, which does not require Internet access in this case.

VIP

Pigsty supports optional L2 VIP for NODE clusters (keepalived) and PGSQL clusters (vip-manager).

To use L2 VIP features, You have to explicitly assign an L2 VIP for them. It’s not a big deal when running on your own hardware, but may become an issue when working in a public cloud environment.

Domain

Pigsty using local static domain names for the following service with WebUI. You can assign your custom domain names to these services, or use real domain names. Just change them in the infra_portal.

Domain names are optional, to use them, it is user’s responsibility to add the following records to your /etc/hosts file (local static resolution), Or add them to your DNS server / public DNS vendor.

10.10.10.10 h.pigsty g.pigsty p.pigsty a.pigsty

2 - Linux OS

Recommended

Pigsty works well on EL9, U24, D12, Recommended OS: RockyLinux 9.6, Ubuntu 24.04.2, Debian 12.11

EL

Pigsty works on RHEL / Rocky / Alma / Anolis / CentOS 8, 9, 10.

Ubuntu

Pigsty works on Ubuntu 24.04 / 22.04:

Debian

Pigsty works on Debian 12 / 13, we recommend using the latest Debian 12.11

3 - Software

Linux

Pigsty runs on Linux Operating System, it supports 14 mainstream Linux distros: Compatible OS List

We recommend using RockyLinux 9.6, Debian 12.11, or Ubuntu 24.04.5 as the default OS options.

We strongly recommend using a clean installed OS environment with en_US set as the primary language.

localedef -i en_US -f UTF-8 en_US.UTF-8
localectl set-locale LANG=en_US.UTF-8

Pigsty DOES NOT use containers, main components are packed for specific distro major version.

File System

Pigsty suggests using ext4 or xfs file systems, both of which have the best performance for PostgreSQL use cases. If you know what you are doing, you can also consider using file systems like zfs, but never run database services on network file systems like nfs.

If you need to use MinIO, it is recommended to use the xfs file system, which is the only file system recommended by MinIO. It performs better in scenarios with a large number of small files, but the tool ecosystem (e.g., data recovery) is slightly inferior to ext4.

The default recommendation for running standard PostgreSQL services is to use the ext4 file system.

Firewall

Your security policy and firewall setup should allow access to the required ports.

To access the WebUI services, you’ll have to allow the HTTP (80) / HTTPS (443) access.

To access the PostgreSQL database services, you’ll have to allow the 5432 port for postgres.

In a typical public cloud VPS setup, port 22/80/443/5432 are usually opened.

In a typical production setup, port 22/80/443 are opened to DBA/OPS from LAN / Jumpserver. And other ports are accessed from intranet. You have to make sure they are opened internally: used ports.

Ansible

Pigsty uses Ansible to initiate control from admin node to all managed nodes.

You don’t need to care about details, ansible is installed during the Bootstrap stage.

sudo apt install -y ansible python3-jmespath

sudo dnf install -y ansible python3-jmespath

sudo dnf install -y ansible python3.12-jmespath

sudo yum install -y ansible python-jmespath

brew install ansible
pip3 install jmespath

Ansible is only required on admin node, you can run ansible on macOS to use your laptop as admin node.

Pigsty

(RECOMMENDED) You can get & extract the latest stable version of pigsty source with:

curl -fsSL https://repo.pigsty.io/get | bash -s v3.7.0; cd ~/pigsty

curl -fsSL https://repo.pigsty.cc/get | bash -s v3.7.0; cd ~/pigsty   # china mirror

To install a specific version, pass the version string as the first parameter:

curl -fsSL https://repo.pigsty.io/get | bash -s v3.7.0; cd ~/pigsty

curl -fsSL https://repo.pigsty.cc/get | bash -s v3.7.0; cd ~/pigsty # china mirror

You can also use git to clone the Pigsty source repo from GitHub:

git clone https://github.com/pgsty/pigsty.git; cd ~/pigsty; git checkout v3.7.0

$ curl -fssL https://repo.pigsty.cc/get | bash -s v3.7.0
[v3.7.0] ===========================================
$ curl -fsSL https://repo.pigsty.cc/get | bash -s v3.7.0
[Docs] https://doc.pgsty.com
[Demo] https://g.pgsty.com
[Repo] https://github.com/pgsty/pigsty
[Download] ===========================================
[ OK ] version = v3.7.0 (from arg)
curl -fSL https://repo.pigsty.cc/src/pigsty-v3.7.0.tgz -o /tmp/pigsty-v3.7.0.tgz
[WARN] tarball = /tmp/pigsty-v3.7.0.tgz exists, size = 1472486, use it
[ OK ] md5sums = df64ac0c2b5aab39dd29698a640daf2e  /tmp/pigsty-v3.7.0.tgz
[Install] ===========================================
[ OK ] install = /home/vagrant/pigsty, from /tmp/pigsty-v3.7.0.tgz
[Bootstrap] ===========================================
[ OK ] ansible = ready
[ OK ] bootstrap = skip
you can run ./bootstrap to extrac offline package and install ansible
[TodoList] ===========================================
cd /home/vagrant/pigsty
./configure      # [OPTIONAL] preflight-check and config generation
./install.yml    # deployment everything described by your config

You can also download the pigsty src (pigsty-<version>.tar.gz) from GitHub Release Page manually:

wget https://repo.pigsty.io/src/pigsty-v3.7.0.tgz

wget https://pigsty.cc/pgsty/pigsty/releases/download/v3.7.0/pigsty-v3.7.0.tgz

wget https://github.com/pgsty/pigsty/releases/download/v3.7.0/pigsty-v3.7.0.tgz

If your environment does not have Internet access, consider downloading offline packages along with source tarball and upload them to your nodes.

Check Offline Installation for details.

4 - Admin

User

Pigsty requires an OS user with passwordless ssh and sudo on all managed nodes.

Naming Convention

Usually we’ll choose a name such as dba or admin for this purpose, but avoid using root or postgres:

Provide Password

The nopass requirement is optional if you can accept the password prompt for every ssh and sudo command.

./install.yml -k -K

Create Admin User

It’s user/vendor’s responsibility to create & deliver such an admin user during server provisioning stage. But if you don’t have such an admin user, or that user is restricted, you can create one with pigsty itself:

./node.yml -k -K -t node_admin -e ansible_user=[existing_admin_user]

Sudo Privileges

All the admin user should have passwordless sudo privileges on all managed nodes.

In case you want to configure an admin user with passwordless sudo privileges from scratch:

echo '%vagrant ALL=(ALL) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/vagrant

%dba ALL=(ALL) NOPASSWD: ALL

Ansible relies on sudo to execute commands with root privileges on managed nodes. So on environments where sudo is not available (like inside a slim container), you may have to install sudo first.

SSH

Your current user should have nopass ssh access to all managed nodes as corresponding admin user.

Your current user can be the admin user itself, but not required as long as you can ssh as the admin user.

SSH configuration is Linux 101, but we will cover the basics here in case you are not familiar with it:

Generate SSH Key

Generate an SSH key pair if you don’t have one

ssh-keygen -t rsa -b 2048 -N '' -f ~/.ssh/id_rsa -q

Copy SSH Key

You’ll need to distribute your generated public key remote (and local) servers, and put it into the ~/.ssh/authorized_keys file of the admin user on all nodes. The ssh-copy-id util can be used.

ssh-copy-id <ip>                        # Interactive password entry

sshpass -p <password> ssh-copy-id <ip>  # Non-interactive (use with caution)

Using Alias

When direct ssh access is not available (due to jumpserver, other port, credentials, etc…), consider:

Host meta
    HostName 10.10.10.10
    User dba                      # <--- not the same user on remote
    IdentityFile /etc/dba/id_rsa  # <--- not the ordinary key
    Port 24                       # <--- not the well-known port

nodes:
  hosts:          # if node `10.10.10.10` requires an SSH alias `meta`
    10.10.10.10: { ansible_host: meta }  # <---- access via `ssh meta`

Check Accessibility

Your should be able to access all managed nodes with nopass ssh via your current user from admin node. And the remote user (admin user) should have the privilege to run nopass sudo commands.

ssh <ip|alias> 'sudo ls'

5 - Sandbox

Pigsty has a sandbox, which is a 4-node deployment with fixed IP addresses and other identifiers.

We will use this as a standard demonstration environment for learning and testing purposes.

Description

The sandbox consists of 4 nodes with fixed IP addresses and identities:

There’s a primary singleton PostgreSQL cluster: pg-meta on the meta node, which can be used standalone, and there’s also an optional L2 VIP 10.10.10.2 and cluster DNS pg-meta bind to it.

10.10.10.10 meta pg-meta-1
10.10.10.2  pg-meta

There are three additional nodes in the sandbox, form a 3-instance PostgreSQL HA cluster pg-test. With an optional L2 VIP 10.10.10.3 and cluster DNS pg-test bind to the cluster leader.

10.10.10.11 node-1 pg-test-1
10.10.10.12 node-2 pg-test-2
10.10.10.13 node-3 pg-test-3
10.10.10.3  pg-test

There’s also a 1-node etcd cluster, and 1-node minio cluster on the meta node, too.

10.10.10.10 minio-1
10.10.10.10 etcd-1

Implementation

You can create a local sandbox with Vagrant, or a cloud sandbox with Terraform.

To utilize local vagrant templates:

make full9     # create 4-node sandbox with RockyLinux 9
make full12     # create 4-node sandbox with Debian 12
make full24     # create 4-node sandbox with Ubuntu 24.04

To utilize cloud terraform templates, use the spec/aliyun-full.tf as an example, Aliyun 4-node sandbox template for all distro & amd/arm.

make tu     # terraform up
make td     # terraform destroy
make tssh   # write ssh alias to ~/.ssh/pigsty_config

6 - Vagrant

Pigsty requires Linux environment, you can easily create local linux VMs with Vagrant.

You’ll also need a virtual machine provider, (like VirtualBox for laptops and libvirt for servers)

Get Started

You can install vagrant, virtualbox, ansible on macOS with homebrew:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install vagrant virtualbox ansible

You are all set! use the make shortcuts to create VMs:

make meta       # 1-node devbox for quick start, dev, test & playground
make full       # 4-node sandbox for HA-testing & feature demonstration
make simu       # 36-node simubox for production environment simulation
...
make meta9      # create singleton-meta node with bento/rockylinux-9 image
make full22     # create 4-node sandbox with generic/ubuntu2204 image
make simu12     # create 36-node simulation env with generic/debian12 image

Configuration

You have to define VMs in the Vagrantfile before launch. The default Vagrantfile defines a el9 (bento/rockylinux-9) 1-node virtual machine with the local virtualbox VM provider.

• vagrant/
  • spec/
    • meta.rb
    • full.rb
    • simu.rb
    • ......
  • Vagrantfile

We have a list of pre-defined VM specs available in the vagrant/spec Folder

Each spec file contains a Specs variable describe VM nodes. For example, the full.rb contains:

# full: pigsty full-featured 4-node sandbox for HA-testing & tutorial & practices

Specs = [
  { "name" => "meta"   , "ip" => "10.10.10.10" ,  "cpu" => "2" ,  "mem" => "4096" ,  "image" => "bento/rockylinux-9"  },
  { "name" => "node-1" , "ip" => "10.10.10.11" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "bento/rockylinux-9"  },
  { "name" => "node-2" , "ip" => "10.10.10.12" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "bento/rockylinux-9"  },
  { "name" => "node-3" , "ip" => "10.10.10.13" ,  "cpu" => "1" ,  "mem" => "2048" ,  "image" => "bento/rockylinux-9"  },
]

You can use specs with the config script, it will render the Vagrantfile according to spec and environment variables (resouce, image, vm provider, etc…).

cd ~/pigsty
vagrant/config [spec] [image] [scale] [provider]

vagrant/config meta                # use the 1-node spec, default el8 image
vagrant/config dual el9            # use the 2-node spec, use el9 image instead
vagrant/config trio d12 2          # use the 3-node spec, use debian12 image, double the cpu/mem resource
vagrant/config full u22 4          # use the 4-node spec, use ubuntu22 image instead, use 4x cpu/mem resource
vagrant/config simu u24 1 libvirt  # use the 36-node spec, use ubuntu24 image instead, use libvirt as provider instead of virtualbox

You can scale the resource unit with environment variable VM_SCALE, the default value is 1.

For example, VM_SCALE=2 vagrant/config meta will double the cpu / mem resources of the meta spec

Specs = [
  { "name" => "meta" , "ip" => "10.10.10.10", "cpu" => "8" , "mem" => "16384" , "image" => "bento/rockylinux-9" },
]

Shortcuts

You can create the VMs with vagrant up command after configuration.

Pigsty templates will use your ~/.ssh/id_rsa[.pub] as the default ssh key for vagrant provisioning. Make sure you have a valid ssh key pair before you start, you can generate one by: ssh-keygen -t rsa -b 2048

There are some shortcuts that wrap the vagrant commands, you can use them to manage the VMs.

make         # = make start
make new     # destroy existing vm and create new ones
make ssh     # write VM ssh config to ~/.ssh/     (required)
make dns     # write VM DNS records to /etc/hosts (optional)
make start   # launch VMs and write ssh config    (up + ssh)
make up      # launch VMs with vagrant up
make halt    # shutdown VMs (down,dw)
make clean   # destroy VMs (clean/del/destroy)
make status  # show VM status (st)
make pause   # pause VMs (suspend,pause)
make resume  # pause VMs (resume)
make nuke    # destroy all vm & volumes with virsh (if using libvirt)

Version

Pigsty currently uses the following vagrant boxes for testing:

$ vagrant box list

el8 :  bento/rockylinux-8     (libvirt, 202502.21.0, (amd64))
el9 :  bento/rockylinux-9     (libvirt, 202502.21.0, (amd64))

d11 :  generic/debian11       (libvirt, 4.3.12, (amd64))
d12 :  generic/debian12       (libvirt, 4.3.12, (amd64))

u20 :  generic/ubuntu2004     (libvirt, 4.3.12, (amd64))
u22 :  generic/ubuntu2204     (libvirt, 4.3.12, (amd64))
u24 :  bento/ubuntu-24.04     (libvirt, 20250316.0.0, (amd64))

Not all of them have arm64 arch support, so beware when using Apple Silicon MacOS.

bento/rockylinux-9 (virtualbox, 202502.21.0, (arm64))
bento/ubuntu-24.04 (virtualbox, 202502.21.0, (arm64))

You can find the supported Box Image on https://app.vagrantup.com/bento/boxes

Caveat

echo "10.0.0.0/8" | sudo tee -a /etc/vbox/networks.conf

7 - Terraform

Terraform is a popular IaC tool. You can create VMs on public cloud with one command.

Aliyun & AWS templates are used as example providers. You can take terraform.tf as an example.

Get Started

You can install terraform with homebrew on macOS

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install terraform

Then init terraform cloud provider, adjust the terraform.tf config file and apply it:

terraform init
terraform apply #-auto-approve

Print public IP Address:

terraform output | grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

AWS Setup

You have to set up aws config & credential to use AWS provider.

# ~/.aws

# ~/.aws/config
[default]
region = cn-northwest-1

# ~/.aws/credentials
[default]
aws_access_key_id = <YOUR_AWS_ACCESS_KEY>
aws_secret_access_key =  <AWS_ACCESS_SECRET>

# ~/.aws/pigsty-key
# ~/.aws/pigsty-key.pub

There’s a contrib example for AWS (Amazon Web Services), but it’s not actively maintained.

• spec/aws-cn.tf : AWS 4 node CentOS7 environment

Aliyun Setup

You can add your aliyun credentials to the environment file, such as ~/.bash_profile

export ALICLOUD_ACCESS_KEY="<your_access_key>"
export ALICLOUD_SECRET_KEY="<your_secret_key>"
export ALICLOUD_REGION="cn-beijing"

Example config files:

• spec/aliyun-meta.tf : Aliyun 1 meta node template for all distro & amd/arm (default)
• spec/aliyun-full.tf : Aliyun 4-node sandbox template for all distro & amd/arm.
• spec/aliyun-oss.tf : Aliyun 5-node building template for all distro & amd/arm.

Here are example ECS public OS images used in Aliyun:

QCloud Setup

There’s a contrib example for QCloud (Tencent Cloud), but it’s not actively maintained.

• spec/tencentcloud.tf : QCloud 4 node CentOS7 environment