This blog refers to an older version of EDB’s Postgres on Kubernetes offering that is no longer available.

The last blogs in the series are already some days old: Setting up MiniShift, Deploying EDB containers in MiniShift/OpenShift, Customizing PostgreSQL parameters in EDB containers in MiniShift/OpenShift, Scaling the EDB containers in MiniShift/OpenShift, EDB Failover Manager in EDB containers in Minishift/OpenShift and EDB Failover Manager in EDB containers in Minishift/OpenShift – Failovers. What is missing is how you can backup and restore instances running in this container deployment and that is the topic of this post.

What you usually use to backup and restore EDB Postgres is BART and the container world is no exception to that. Lets see how that works.

My current deployment looks like this:
Selection_013

Two pgpool containers are serving three database containers which you can also check on the command line:

1
2
3
4
5
6
7
dwe@dwe:~$ oc get pods -o wide -L role
NAME                 READY     STATUS    RESTARTS   AGE       IP           NODE        ROLE
edb-as10-0-1-b8lvj   1/1       Running   0          3m        172.17.0.9   localhost   masterdb
edb-as10-0-1-gj76h   1/1       Running   0          1m        172.17.0.5   localhost   standbydb
edb-as10-0-1-sb5lt   1/1       Running   0          2m        172.17.0.4   localhost   standbydb
edb-pgpool-1-qzk5v   1/1       Running   0          3m        172.17.0.7   localhost   queryrouter
edb-pgpool-1-rvtl6   1/1       Running   0          3m        172.17.0.6   localhost   queryrouter

What we want to do is to backup the database instances or at least one of them. What you need to prepare before deploying the BART container is shared storage between the databases containers and the BART container. The is especially important for the restore case as the restore procedure needs to access the backup which is hosted in the BART container. Notice that this storage configuration has the “Read-Write-Many” attributes:
Selection_030

When I initially deployed the database containers I provided exactly these storage claim and volume as a parameter so I have that available in the database containers:
Selection_031

This means in any of the database containers I will be able to see the backup volume:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
dwe@dwe:~$ oc rsh edb-as10-0-1-b8lvj
sh-4.2$ ls -la /edbbackup/
total 12
drwxrwx---  3 root    root 4096 Aug  6 11:49 .
drwxr-xr-x 86 root    root 4096 Aug  8 14:03 ..
drwxrwxr-x  4 edbuser root 4096 Aug  6 11:49 edb-bart-1-89k7s
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/
total 16
drwxrwxr-x 4 edbuser root 4096 Aug  6 11:49 .
drwxrwx--- 3 root    root 4096 Aug  6 11:49 ..
drwxrwxr-x 2 edbuser root 4096 Aug  6 11:49 bart_log
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 pgbackup
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/pgbackup/
total 12
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 .dwe@dwe:~$ oc rsh edb-as10-0-1-b8lvj
sh-4.2$ ls -la /edbbackup/
total 12
drwxrwx---  3 root    root 4096 Aug  6 11:49 .
drwxr-xr-x 86 root    root 4096 Aug  8 14:03 ..
drwxrwxr-x  4 edbuser root 4096 Aug  6 11:49 edb-bart-1-89k7s
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/
total 16
drwxrwxr-x 4 edbuser root 4096 Aug  6 11:49 .
drwxrwx--- 3 root    root 4096 Aug  6 11:49 ..
drwxrwxr-x 2 edbuser root 4096 Aug  6 11:49 bart_log
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 pgbackup
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/pgbackup/
total 12
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 .
drwxrwxr-x 4 edbuser root 4096 Aug  6 11:49 ..
drwxr-xr-x 4 edbuser root 4096 Aug  6 11:52 edb
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/pgbackup/edb/
total 16
drwxr-xr-x 4 edbuser root 4096 Aug  6 11:52 .
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 ..
drwxr-xr-x 2 edbuser root 4096 Aug  6 11:52 1533556356576
drwxr-xr-x 2 edbuser root 4096 Aug  6 11:49 archived_wals
drwxrwxr-x 4 edbuser root 4096 Aug  6 11:49 ..
drwxr-xr-x 4 edbuser root 4096 Aug  6 11:52 edb
sh-4.2$ ls -la /edbbackup/edb-bart-1-89k7s/pgbackup/edb/
total 16
drwxr-xr-x 4 edbuser root 4096 Aug  6 11:52 .
drwxrwxr-x 3 edbuser root 4096 Aug  6 11:49 ..
drwxr-xr-x 2 edbuser root 4096 Aug  6 11:52 1533556356576
drwxr-xr-x 2 edbuser root 4096 Aug  6 11:49 archived_wals

The same storage configuration then needs to be provided to the BART deployment. Here is the yaml file for the BART deployment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
apiVersion: v1
kind: Template
metadata:
   name: edb-bart
   annotations:
    description: "Standard EDB Backup And Recovery Tool Deployment Config"
    tags: "database,edbas,postgres,postgresql"
    iconClass: "icon-postgresql"
labels:
   template: edb-bart
objects:
- apiVersion: v1
  kind: DeploymentConfig
  metadata:
    name: ${DATABASE_NAME}-bart
  spec:
    replicas: 1
    selector:
      name: ${DATABASE_NAME}-bart
    strategy:
      resources: {}
      rollingParams:
        intervalSeconds: 1
        maxSurge: 25%
        maxUnavailable: 25%
        timeoutSeconds: 600
        updatePeriodSeconds: 1
      type: Rolling
    template:
      metadata:
        labels:
          name: ${DATABASE_NAME}-bart
          role: backuptool
          cluster: ${DATABASE_NAME}
      spec:
        containers:
        - env:
          - name: BART_HOST_ADDRESS
            value: ${BART_HOST_ADDRESS}
          - name: DATABASE_NAME
            value: ${DATABASE_NAME}
          - name: PGHOST
            value: "${DATABASE_NAME}-service"
          - name: PGPORT
            value: ${PGPORT}
          - name: REPL_USER
            value: repl
          - name: REPL_PASSWORD
            value: postgres
          - name: BART_AUTOMATE_BACKUP
            value: ${BART_AUTOMATE_BACKUP}
          - name: BART_BACKUP_SCHEDULE
            value: "${BART_BACKUP_SCHEDULE}"
          - name: BART_NUM_BACKUPS_TO_KEEP
            value: "${BART_NUM_BACKUPS_TO_KEEP}"
          - name: BART_COMPRESS_BACKUP
            value: ${BART_COMPRESS_BACKUP}
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: ACCEPT_EULA
            value: ${ACCEPT_EULA}
          volumeMounts:
          - name: ${BACKUP_VOLUME}
            mountPath: /edbbackup
          - name: edb-bartconf
            mountPath: /bartconf
          image: localhost:5000/edb/edb-bart:2.0
          imagePullPolicy: Always
          name: edb-bart
        volumes:
        - name: ${BACKUP_VOLUME}
          persistentVolumeClaim:
            claimName: ${BACKUP_VOLUME_CLAIM}
        - name: edb-bartconf
          configMap:
            name: ${DATABASE_NAME}-bartconf-vol
    triggers:
    - type: ConfigChange
parameters:
- name: BART_HOST_ADDRESS
  displayName: BART Host Address
  description: BART host address (leave localhost for default)
  value: 'localhost'
  required: true
- name: DATABASE_NAME 
  displayName: Database Name
  description: Name of Postgres database (leave edb for default)
  value: 'edb'
  required: true
- name: PGHOST
  displayName: Database Host
  description: Database Host (leave ${DATABASE_NAME}-service for default)
  value: ${DATABASE_NAME}-service
  required: true
- name: PGPORT
  displayName: Database Port
  description: Database Port (leave 5444 for default)
  value: '5444'
  required: true
- name: BACKUP_VOLUME
  displayName: Backup Volume
  description: Backup volume name
  value: 'edb-backup-pv'
- name: BACKUP_VOLUME_CLAIM
  displayName: Backup Volume Claim
  description: Backup volume claim name
  value: 'edb-backup-pvc'
- name: BART_AUTOMATE_BACKUP
  displayName: Automate backup operation (leave 'Yes' for default)
  description: Indicates whether backups should be done automatically
  value: 'Yes'
  required: true
- name: BART_BACKUP_SCHEDULE 
  displayName: BART Backup Schedule
  description: Standard cron schedule - min (0 - 59), hour (0 - 23), day of month (1 - 31), month (1 - 12), day of week (0 - 6) (0 to 6 are Sunday to Saturday, or use names; 7 is Sunday, the same as 0)
  value: '0:0:*:*:*'  
  required: true
- name: BART_NUM_BACKUPS_TO_KEEP 
  displayName: BART Number of Backups to Keep
  description: How many backups to keep, older backups will be removed as newer backups are added
  value: '2' 
  required: true
- name: BART_COMPRESS_BACKUP
  displayName: Compress backup files (leave 'Yes' for default)
  description: Indicates whether backup files should be compressed 
  value: 'Yes'
  required: true
- name: ACCEPT_EULA
  displayName: Accept end-user license agreement (leave 'Yes' for default)
  description: Indicates whether user accepts the end-user license agreement
  value: 'Yes'
  required: true

Once that is imported we can deploy the BART container:
Selection_032
Notice that this is actually the same storage configuration as it was used to setup the database containers.
Selection_033

What I didn’t tell you is that you need to do another step before. As the BART container is supposed to backup all the instances in a project we need to pass the BART configuration file to the container via a configMap. In this setup I only have one instance so the configMap would look like this:
Selection_034

Here you would add all the instances you’d need to backup per project. Once the BART container is ready:

1
2
3
4
5
6
7
8
dwe@dwe:~$ oc get pods
NAME                 READY     STATUS    RESTARTS   AGE
edb-as10-0-1-b8lvj   1/1       Running   0          17m
edb-as10-0-1-gj76h   1/1       Running   0          14m
edb-as10-0-1-sb5lt   1/1       Running   0          16m
edb-bart-1-7cgfv     1/1       Running   0          19s
edb-pgpool-1-qzk5v   1/1       Running   0          17m
edb-pgpool-1-rvtl6   1/1       Running   0          17m

… you can connect to it and perform a manual backup:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
dwe@dwe:~$ oc rsh edb-bart-1-7cgfv
sh-4.2$ bart backup -s edb
INFO:  creating backup for server 'edb'
INFO:  backup identifier: '1533738106320'
65043/65043 kB (100%), 1/1 tablespace
 
INFO:  backup completed successfully
INFO:  backup checksum: 16fba63925ac3e77d474a36496c2a902 of base.tar
INFO: 
BACKUP DETAILS:
BACKUP STATUS: active
BACKUP IDENTIFIER: 1533738106320
BACKUP NAME: none
BACKUP PARENT: none
BACKUP LOCATION: /edbbackup/edb-bart-1-7cgfv/pgbackup/edb/1533738106320
BACKUP SIZE: 63.52 MB
BACKUP FORMAT: tar
BACKUP TIMEZONE: UTC
XLOG METHOD: fetch
BACKUP CHECKSUM(s): 1
 ChkSum                             File     
 16fba63925ac3e77d474a36496c2a902   base.tar 
 
TABLESPACE(s): 0
START WAL LOCATION: 000000010000000000000008
BACKUP METHOD: streamed
BACKUP FROM: master
START TIME: 2018-08-08 14:21:46 UTC
STOP TIME: 2018-08-08 14:21:47 UTC
TOTAL DURATION: 1 sec(s)

This backup is now available on the BART container but in addition it is accessible in the database container:

1
2
3
4
5
6
7
dwe@dwe:~$ oc rsh edb-as10-0-1-b8lvj
sh-4.2$ ls -la /edbbackup/edb-bart-1-7cgfv/pgbackup/edb/1533738106320/
total 65060
drwxr-xr-x 2 edbuser root     4096 Aug  8 14:21 .
drwxr-xr-x 4 edbuser root     4096 Aug  8 14:21 ..
-rwxr-xr-x 1 edbuser root      664 Aug  8 14:21 backupinfo
-rwxr-xr-x 1 edbuser root 66605568 Aug  8 14:21 base.tar

In case you’d need to restore that you would deploy a new database configuration specifying this backup as the “Restore file”:
Selection_035

One downside with the current versions of the containers: You can not do point in time recovery. Only restores from full backups are supported until now. This will change in the next release, though.

Have fun with the containers …

Post Views: 1,052
Thumbnail [60x60]
by
Daniel Westermann