{"id":16836,"date":"2021-11-16T19:52:36","date_gmt":"2021-11-16T18:52:36","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/"},"modified":"2021-11-16T19:52:36","modified_gmt":"2021-11-16T18:52:36","slug":"using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/","title":{"rendered":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage"},"content":{"rendered":"

When it comes to backup and restore of PostgreSQL we usually recommend pgBackRest<\/a> to our customers. This tool comes with many options and features and should bring everything you need. Most of the times we install a dedicated backup host for pgBackRest, which centralizes all backups of the PostgreSQL instances. Sometimes we have several of those, e.g. one for production instances, another for test and development or even a dedicated one for development. How you organize this is mostly driven by business requirements. Most of the times, these dedicated backups hosts get backed up by an enterprise backup solution for long term archiving. Another option you have is, to write the backups directly to a S3<\/a> compatible storage solution and this is the topic of this post.<\/p>\n

<\/p>\n

We could just use an AWS S3 bucket for the scope of this post, but for this you’ll need access to AWS. Being able to re-play the demos of this blog without access to external sources has always been a priority for me, so we will be using a solution that is freely available for testing. While searching for a free AWS S3 compatible storage solution I came across a MinIO<\/a>. MinIO is quite easy to setup on a single node for testing. All you need to do is this:<\/p>\n

\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] mkdir \/u02\/minio\/\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] wget https:\/\/dl.min.io\/server\/minio\/release\/linux-amd64\/minio\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] chmod +x mini\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] .\/minio server \/u02\/minio\/\n<\/pre>\n
If you want to start it automatically when the machine is starting up, you can use this systemd service file as a template:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] cat \/etc\/systemd\/system\/minio.service \n[Unit]\nDescription=MinIO\nDocumentation=https:\/\/docs.min.io\nWants=network-online.target\nAfter=network-online.target\n\n[Service]\nWorkingDirectory=\/usr\/local\n\nUser=postgres\nGroup=postgres\nProtectProc=invisible\n\nExecStart=\/home\/postgres\/minio server \/u02\/minio\/ --console-address :8888\n# User: minioadmin\n# Pwd: minioadmin\n\n# Let systemd restart this service always\nRestart=always\n\n# Specifies the maximum file descriptor number that can be opened by this process\nLimitNOFILE=65536\n\n# Specifies the maximum number of threads this process can create\nTasksMax=infinity\n\n# Disable timeout logic and wait until process is stopped\nTimeoutStopSec=infinity\nSendSIGKILL=no\n\n[Install]\nWantedBy=multi-user.target\n<\/pre>\n
Once MinIO is running, you can point your browser to http:\/\/[HOST]:8888
\n\"\"<\/a>
\n\"\"<\/a><\/p>\n
Obviously we’ll need a new bucket we can write our backups to, so let’s create a new bucket:
\n\"\"<\/a>
\n\"\"<\/a>
\nHere you will see the limitations of the single node test setup: Some features are not available and you of course should not use this kind of setup in production:
\n\"\"<\/a>
\n\"\"<\/a><\/p>\n
For accessing a bucket you need an access key and a secret, so we need to create those too:
\n\"\"<\/a>
\n\"\"<\/a>
\n\"\"<\/a><\/p>\n
Before you continue with configuring pgBackRest it is a good idea to test access to the bucket with a command line utility. The AWS Cli<\/a> is freely available, easy to install and can be used for that:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] curl \"https:\/\/awscli.amazonaws.com\/awscli-exe-linux-x86_64.zip\" -o \"awscliv2.zip\"\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] unzip awscliv2.zip \npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] sudo .\/aws\/install\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] rm -rf awscliv2.zip aws\/\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] \/usr\/local\/bin\/aws --version\naws-cli\/2.3.6 Python\/3.8.8 Linux\/5.10.0-9-amd64 exe\/x86_64.debian.11 prompt\/off\n<\/pre>\n
The AWS command line utilities by default read the credentials form “~\/.aws\/credentials”, this is where we need to add the access key and the secret we’ve created above:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] aws configure\nAWS Access Key ID [****************7890]: 1234567890\nAWS Secret Access Key [****************4321]: 0987654321\nDefault region name [eu-central-1]: eu-central-1\nDefault output format : text\n<\/pre>\n
Having that in place let’s test if we can see the bucket:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] aws configure set default.s3.signature_version s3v4\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] aws --endpoint-url http:\/\/localhost:9000 s3 ls\n2021-11-16 10:25:50 postgresql-backups\n<\/pre>\n
Looks good. Now we need to tell pgBackRest about the repository. My current configuration on the backup host looks like this:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] cat \/u01\/app\/postgres\/local\/dmk\/etc\/pgbackrest.conf \n[global]\nrepo1-path=\/u02\/backups\nrepo1-retention-full=2\nrepo1-retention-diff=1\nlog-path=\/u01\/app\/postgres\/local\/dmk\/log\nlog-level-file=detail\nspool-path=\/tmp\nstart-fast=y\narchive-async=y\nexpire-auto=y\ncompress-type=bz2\nprocess-max=4\n\n[PG14]\npg1-path=\/u02\/pgdata\/14\/PG1\/\npg1-host=192.168.100.170\npg1-user=postgres\npg1-port=5432\npg2-path=\/u02\/pgdata\/14\/PG1\/\npg2-host=192.168.100.171\npg2-user=postgres\npg2-port=5432\npg3-path=\/u02\/pgdata\/14\/PG1\/\npg3-host=192.168.100.172\npg3-user=postgres\npg3-port=5432\n<\/pre>\n
This means I have one repository pointing to a local directory. The stanza below the global configuration is a three node Patroni cluster. As pgBackRest supports multiple repositories we can just add our new repository like this:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] cat \/u01\/app\/postgres\/local\/dmk\/etc\/pgbackrest.conf\n[global]\nrepo1-path=\/u02\/backups\nrepo1-retention-full=2\nrepo1-retention-diff=1\nrepo2-s3-bucket=postgresql-backups\nrepo2-s3-endpoint=192.168.100.173\nrepo2-s3-key=1234567890\nrepo2-s3-key-secret=0987654321\nrepo2-s3-region=us-west-rack1\nrepo2-storage-port=9000\nrepo2-storage-verify-tls=n\nrepo2-s3-uri-style=path\nrepo2-type=s3\nlog-path=\/u01\/app\/postgres\/local\/dmk\/log\nlog-level-file=detail\nspool-path=\/tmp\nstart-fast=y\narchive-async=y\nexpire-auto=y\ncompress-type=bz2\nprocess-max=4\n...\n<\/pre>\n
Let’s try to create the stanza:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] pgbackrest --stanza=PG14 stanza-create --log-level-console=info \n2021-11-16 12:09:07.682 P00   INFO: stanza-create command begin 2.36: --exec-id=2255-0ef6d4c5 --log-level-console=info --log-level-file=detail --log-path=\/u01\/app\/postgres\/local\/dmk\/log --pg1-host=192.168.100.170 --pg2-host=192.168.100.171 --pg3-host=192.168.100.172 --pg1-path=\/u02\/pgdata\/14\/PG1\/ --pg2-path=\/u02\/pgdata\/14\/PG1\/ --pg3-path=\/u02\/pgdata\/14\/PG1\/ --pg1-port=5432 --pg2-port=5432 --pg3-port=5432 --pg1-user=postgres --pg2-user=postgres --pg3-user=postgres --repo1-path=\/u02\/backups --repo2-s3-bucket=postgresql-backups --repo2-s3-endpoint=127.0.0.1 --repo2-s3-key= --repo2-s3-key-secret= --repo2-s3-region=eu-central-1 --repo2-s3-uri-style=path --repo2-storage-port=9000 --no-repo2-storage-verify-tls --repo2-type=s3 --stanza=PG14\n2021-11-16 12:09:09.972 P00   INFO: stanza-create for stanza 'PG14' on repo1\n2021-11-16 12:09:09.977 P00   INFO: stanza 'PG14' already exists on repo1 and is valid\n2021-11-16 12:09:09.977 P00   INFO: stanza-create for stanza 'PG14' on repo2\nERROR: [101]: TLS error [1:336130315] wrong version number\n2021-11-16 12:11:10.430 P00   INFO: stanza-create command end: aborted with exception [101]\n<\/pre>\n
This means we need an encrypted connection, otherwise it will not work. MinIO provides a handy utility<\/a> to create a self signed certificate:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] wget https:\/\/github.com\/minio\/certgen\/releases\/download\/v0.0.2\/certgen-linux-amd64\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] chmod +x certgen-linux-amd64 \npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] .\/certgen-linux-amd64 -ca -host \"192.168.100.173\"   # replace with your IP\n2021\/11\/16 14:25:33 wrote public.crt\n2021\/11\/16 14:25:33 wrote private.key\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] mv private.key public.crt .minio\/certs\/\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] sudo systemctl restart minio\n<\/pre>\n
Once MinIO is restarted we can use https to access MinIO and creating the stanza works smoothly:<\/p>\n
\npostgres@patronipgbackrest:\/home\/postgres\/ [pg140] pgbackrest --stanza=PG14 stanza-create --log-level-console=info \n2021-11-16 14:40:17.086 P00   INFO: stanza-create command begin 2.36: --exec-id=2491-22a7ebd4 --log-level-console=info --log-level-file=detail --log-path=\/u01\/app\/postgres\/local\/dmk\/log --pg1-host=192.168.100.170 --pg2-host=192.168.100.171 --pg3-host=192.168.100.172 --pg1-path=\/u02\/pgdata\/14\/PG1\/ --pg2-path=\/u02\/pgdata\/14\/PG1\/ --pg3-path=\/u02\/pgdata\/14\/PG1\/ --pg1-port=5432 --pg2-port=5432 --pg3-port=5432 --pg1-user=postgres --pg2-user=postgres --pg3-user=postgres --repo1-path=\/u02\/backups --repo2-s3-bucket=postgresql-backups --repo2-s3-endpoint=192.168.100.173 --repo2-s3-key= --repo2-s3-key-secret= --repo2-s3-region=us-west-rack1 --repo2-s3-uri-style=path --repo2-storage-port=9000 --no-repo2-storage-verify-tls --repo2-type=s3 --stanza=PG14\n2021-11-16 14:40:19.613 P00   INFO: stanza-create for stanza 'PG14' on repo1\n2021-11-16 14:40:19.618 P00   INFO: stanza 'PG14' already exists on repo1 and is valid\n2021-11-16 14:40:19.618 P00   INFO: stanza-create for stanza 'PG14' on repo2\n2021-11-16 14:40:19.982 P00   INFO: stanza-create command end: completed successfully (2901ms)\n<\/pre>\n
If you take a look into the bucket you’ll see that the usual directories for pgBackRest have been created:
\n\"\"<\/a><\/p>\n
Before doing any backups lets perform a pgBackRest check to confirm everything is fine:<\/p>\n
\npostgres@patronipgbackrest:\/u01\/app\/postgres\/local\/dmk\/ [pg140] pgbackrest --stanza=PG14 check --log-level-console=info \n2021-11-16 16:59:34.501 P00   INFO: check command begin 2.36: --exec-id=2917-47e567f2 --log-level-console=info --log-level-file=detail --log-path=\/u01\/app\/postgres\/local\/dmk\/log --pg1-host=192.168.100.170 --pg2-host=192.168.100.171 --pg3-host=192.168.100.172 --pg1-path=\/u02\/pgdata\/14\/PG1\/ --pg2-path=\/u02\/pgdata\/14\/PG1\/ --pg3-path=\/u02\/pgdata\/14\/PG1\/ --pg1-port=5432 --pg2-port=5432 --pg3-port=5432 --pg1-user=postgres --pg2-user=postgres --pg3-user=postgres --repo1-path=\/u02\/backups --repo2-s3-bucket=postgresql-backups --repo2-s3-endpoint=192.168.100.173 --repo2-s3-key= --repo2-s3-key-secret= --repo2-s3-region=us-west-rack1 --repo2-s3-uri-style=path --repo2-storage-port=9000 --no-repo2-storage-verify-tls --repo2-type=s3 --stanza=PG14\n2021-11-16 16:59:36.842 P00   INFO: check repo1 (standby)\n2021-11-16 16:59:36.848 P00   INFO: check repo2 (standby)\n2021-11-16 16:59:36.866 P00   INFO: switch wal not performed because this is a standby\n2021-11-16 16:59:36.870 P00   INFO: check repo1 configuration (primary)\n2021-11-16 16:59:36.870 P00   INFO: check repo2 configuration (primary)\n2021-11-16 16:59:37.089 P00   INFO: check repo1 archive for WAL (primary)\n2021-11-16 16:59:38.292 P00   INFO: WAL segment 00000002000000000000000F successfully archived to '\/u02\/backups\/archive\/PG14\/14-1\/0000000200000000\/00000002000000000000000F-a63ad380ed332ebebf528c0726694cca5c2214ab.gz' on repo1\n2021-11-16 16:59:38.292 P00   INFO: check repo2 archive for WAL (primary)\n2021-11-16 16:59:38.295 P00   INFO: WAL segment 00000002000000000000000F successfully archived to '\/var\/lib\/pgbackrest\/archive\/PG14\/14-1\/0000000200000000\/00000002000000000000000F-a63ad380ed332ebebf528c0726694cca5c2214ab.gz' on repo2\n2021-11-16 16:59:38.599 P00   INFO: check command end: completed successfully (4099ms)\n<\/pre>\n
Looks good, now we can do a backup:<\/p>\n
\nostgres@patronipgbackrest:\/u01\/app\/postgres\/local\/dmk\/ [pg140] pgbackrest --stanza=PG14 backup --log-level-console=info --repo=2\n2021-11-16 20:45:57.845 P00   INFO: backup command begin 2.36: --compress-type=bz2 --exec-id=3004-6b8fb3c3 --expire-auto --log-level-console=info --log-level-file=detail --log-path=\/u01\/app\/postgres\/local\/dmk\/log --pg1-host=192.168.100.170 --pg2-host=192.168.100.171 --pg3-host=192.168.100.172 --pg1-path=\/u02\/pgdata\/14\/PG1\/ --pg2-path=\/u02\/pgdata\/14\/PG1\/ --pg3-path=\/u02\/pgdata\/14\/PG1\/ --pg1-port=5432 --pg2-port=5432 --pg3-port=5432 --pg1-user=postgres --pg2-user=postgres --pg3-user=postgres --process-max=4 --repo=2 --repo1-path=\/u02\/backups --repo1-retention-diff=1 --repo2-retention-diff=1 --repo1-retention-full=2 --repo2-retention-full=2 --repo2-s3-bucket=postgresql-backups --repo2-s3-endpoint=192.168.100.173 --repo2-s3-key= --repo2-s3-key-secret= --repo2-s3-region=us-west-rack1 --repo2-s3-uri-style=path --repo2-storage-port=9000 --no-repo2-storage-verify-tls --repo2-type=s3 --stanza=PG14 --start-fast\nWARN: no prior backup exists, incr backup has been changed to full\n2021-11-16 20:46:00.941 P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes\n2021-11-16 20:46:01.990 P00   INFO: backup start archive = 000000020000000000000011, lsn = 0\/11000028\nWARN: resumable backup 20211116-144452F of same type exists -- remove invalid files and resume\n2021-11-16 20:46:08.369 P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive\n2021-11-16 20:46:08.913 P00   INFO: backup stop archive = 000000020000000000000011, lsn = 0\/11000138\n2021-11-16 20:46:08.943 P00   INFO: check archive for segment(s) 000000020000000000000011:000000020000000000000011\n2021-11-16 20:46:09.873 P00   INFO: new backup label = 20211116-144452F\n2021-11-16 20:46:10.090 P00   INFO: full backup size = 25.3MB, file total = 957\n2021-11-16 20:46:10.090 P00   INFO: backup command end: completed successfully (12252ms)\n2021-11-16 20:46:10.091 P00   INFO: expire command begin 2.36: --exec-id=3004-6b8fb3c3 --log-level-console=info --log-level-file=detail --log-path=\/u01\/app\/postgres\/local\/dmk\/log --repo=2 --repo1-path=\/u02\/backups --repo1-retention-diff=1 --repo2-retention-diff=1 --repo1-retention-full=2 --repo2-retention-full=2 --repo2-s3-bucket=postgresql-backups --repo2-s3-endpoint=192.168.100.173 --repo2-s3-key= --repo2-s3-key-secret= --repo2-s3-region=us-west-rack1 --repo2-s3-uri-style=path --repo2-storage-port=9000 --no-repo2-storage-verify-tls --repo2-type=s3 --stanza=PG14\n2021-11-16 20:46:10.371 P00   INFO: expire command end: completed successfully (280ms)\n<\/pre>\n
All done and we have our backups in a S3 compatible storage.<\/p>\n","protected":false},"excerpt":{"rendered":"
When it comes to backup and restore of PostgreSQL we usually recommend pgBackRest to our customers. This tool comes with many options and features and should bring everything you need. Most of the times we install a dedicated backup host for pgBackRest, which centralizes all backups of the PostgreSQL instances. Sometimes we have several of […]<\/p>\n","protected":false},"author":29,"featured_media":16837,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[229],"tags":[1293,77],"type_dbi":[],"class_list":["post-16836","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-database-administration-monitoring","tag-pgbackrest","tag-postgresql"],"acf":[],"yoast_head":"\nUsing pgBackRest to backup your PostgreSQL instances to a s3 compatible storage - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage\" \/>\n<meta property=\"og:description\" content=\"When it comes to backup and restore of PostgreSQL we usually recommend pgBackRest to our customers. This tool comes with many options and features and should bring everything you need. Most of the times we install a dedicated backup host for pgBackRest, which centralizes all backups of the PostgreSQL instances. Sometimes we have several of […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2021-11-16T18:52:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1011\" \/>\n\t<meta property=\"og:image:height\" content=\"604\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Daniel Westermann\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@westermanndanie\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Westermann\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\"},\"author\":{\"name\":\"Daniel Westermann\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"headline\":\"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage\",\"datePublished\":\"2021-11-16T18:52:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\"},\"wordCount\":574,\"commentCount\":2,\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg\",\"keywords\":[\"pgbackrest\",\"PostgreSQL\"],\"articleSection\":[\"Database Administration & Monitoring\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\",\"name\":\"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg\",\"datePublished\":\"2021-11-16T18:52:36+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg\",\"contentUrl\":\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg\",\"width\":1011,\"height\":604},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66\",\"name\":\"Daniel Westermann\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g\",\"caption\":\"Daniel Westermann\"},\"description\":\"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.\",\"sameAs\":[\"https:\/\/x.com\/westermanndanie\"],\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/","og_locale":"en_US","og_type":"article","og_title":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage","og_description":"When it comes to backup and restore of PostgreSQL we usually recommend pgBackRest to our customers. This tool comes with many options and features and should bring everything you need. Most of the times we install a dedicated backup host for pgBackRest, which centralizes all backups of the PostgreSQL instances. Sometimes we have several of […]","og_url":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/","og_site_name":"dbi Blog","article_published_time":"2021-11-16T18:52:36+00:00","og_image":[{"width":1011,"height":604,"url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg","type":"image\/jpeg"}],"author":"Daniel Westermann","twitter_card":"summary_large_image","twitter_creator":"@westermanndanie","twitter_misc":{"Written by":"Daniel Westermann","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/"},"author":{"name":"Daniel Westermann","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"headline":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage","datePublished":"2021-11-16T18:52:36+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/"},"wordCount":574,"commentCount":2,"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg","keywords":["pgbackrest","PostgreSQL"],"articleSection":["Database Administration & Monitoring"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/","url":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/","name":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg","datePublished":"2021-11-16T18:52:36+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#primaryimage","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/minio1.jpg","width":1011,"height":604},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/using-pgbackrest-to-backup-your-postgresql-instances-to-a-s3-compatible-storage\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Using pgBackRest to backup your PostgreSQL instances to a s3 compatible storage"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d08e9bd996a89bd75c0286cbabf3c66","name":"Daniel Westermann","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/31350ceeecb1dd8986339a29bf040d4cd3cd087d410deccd8f55234466d6c317?s=96&d=mm&r=g","caption":"Daniel Westermann"},"description":"Daniel Westermann is Principal Consultant and Technology Leader Open Infrastructure at dbi services. He has more than 15 years of experience in management, engineering and optimization of databases and infrastructures, especially on Oracle and PostgreSQL. Since the beginning of his career, he has specialized in Oracle Technologies and is Oracle Certified Professional 12c and Oracle Certified Expert RAC\/GridInfra. Over time, Daniel has become increasingly interested in open source technologies, becoming \u201cTechnology Leader Open Infrastructure\u201d and PostgreSQL expert. \u00a0Based on community or EnterpriseDB tools, he develops and installs complex high available solutions with PostgreSQL. He is also a certified PostgreSQL Plus 9.0 Professional and a Postgres Advanced Server 9.4 Professional. He is a regular speaker at PostgreSQL conferences in Switzerland and Europe. Today Daniel is also supporting our customers on AWS services such as AWS RDS, database migrations into the cloud, EC2 and automated infrastructure management with AWS SSM (System Manager). He is a certified AWS Solutions Architect Professional. Prior to dbi services, Daniel was Management System Engineer at LC SYSTEMS-Engineering AG in Basel. Before that, he worked as Oracle Developper &\u00a0Project Manager at Delta Energy Solutions AG in Basel (today Powel AG). Daniel holds a diploma in Business Informatics (DHBW, Germany). His branch-related experience mainly covers the pharma industry, the financial sector, energy, lottery and telecommunications.","sameAs":["https:\/\/x.com\/westermanndanie"],"url":"https:\/\/www.dbi-services.com\/blog\/author\/daniel-westermann\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/16836","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/29"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=16836"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/16836\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media\/16837"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=16836"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=16836"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=16836"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=16836"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}