By Mouhamadou Diaw Last day I found a curious message in the backup alert log in a dataguard environment. The message was related to an issue with the resync catalog and was something like
|
1
2
3
4
|
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============RMAN-00571: ===========================================================RMAN-03009: failure of resync command on default channel at 02/18/2016 17:48:42ORA-00001: unique constraint (RMAN.TS_P2) violated |
After few researches, it seems that this error is due to a mismatch of tempfiles between Primary and Standby. Some tempfiles were missing in the standby database.
How is it be possible as we have the parameter StandbyFileManagement set to auto on both sides
Primary database
|
1
2
3
4
5
6
7
8
9
10
|
NAME TYPE VALUE------------------------------------ ----------- ------------------------------db_unique_name string HRVA_SITE1SQL> show parameter standby_fileNAME TYPE VALUE------------------------------------ ----------- ------------------------------standby_file_management string AUTOSQL> |
Standby database
|
1
2
3
4
5
6
7
8
9
10
|
NAME TYPE VALUE------------------------------------ ----------- ------------------------------db_unique_name string HRVA_SITE2SQL> show parameter standby_fileNAME TYPE VALUE------------------------------------ ----------- ------------------------------standby_file_management string AUTOSQL> |
Let’s do some tests to better understand how oracle manages datafiles and tempfiles in dataguard environment
First we add a datafile to the primary database
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
SQL> show parameter db_uniNAME TYPE VALUE------------------------------------ ----------- --------------------db_unique_name string HRVA_SITE1SQL> alter tablespace USERS add datafile '/u01/app/oracle/oradata/HRVA/users02.dbf' size 5M;Tablespace altered.SQL> select name from v$datafile;NAME--------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/system01.dbf/u01/app/oracle/oradata/HRVA/sysaux01.dbf/u01/app/oracle/oradata/HRVA/undotbs01.dbf/u01/app/oracle/oradata/HRVA/users01.dbf/u01/app/oracle/oradata/HRVA/users02.dbf |
Now let's verify that the new datafile is present in the standby database
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
SQL> show parameter db_uniNAME TYPE VALUE------------------------------------ ----------- -----------------------db_unique_name string HRVA_SITE2SQL> select open_mode from v$database;OPEN_MODE--------------------MOUNTEDSQL> select name from v$datafile;NAME--------------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/system01.dbf/u01/app/oracle/oradata/HRVA/sysaux01.dbf/u01/app/oracle/oradata/HRVA/undotbs01.dbf/u01/app/oracle/oradata/HRVA/users01.dbf/u01/app/oracle/oradata/HRVA/users02.dbfSQL> |
This confirms that the new datafile was replicated as expected.
Now let’s add a tempfile to the primary database
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
SQL> show parameter db_uniNAME TYPE VALUE------------------------------------ ----------- --------------------------db_unique_name string HRVA_SITE1SQL> select name from v$tempfile;NAME---------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/temp01.dbfSQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/HRVA/temp02.dbf' size 10M;Tablespace altered.SQL> select name from v$tempfile;NAME--------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/temp01.dbf/u01/app/oracle/oradata/HRVA/temp02.dbfSQL> |
Connecting to the standby database, we can see that the new tempfile was not replicated (although the standby_file_management is set to auto)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
SQL> select open_mode from v$database;OPEN_MODE--------------------MOUNTEDSQL> show parameter db_uniNAME TYPE VALUE------------------------------------ ----------- ------------------------------db_unique_name string HRVA_SITE2SQL> select name from v$tempfile;NAME--------------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/temp01.dbfSQL> |
In fact when you add tempfiles in the Primary database, new tempfiles are not added automatically in the Physical Standby database
because no redo is generated. (Oracle documentation Doc ID 834174.1)
We have to manually create the added tempfile in the standby. For this we have to open the standby database in Read Only mode.
1- First we disable the redo apply on the standby database to not activate ACTIVE DATAGUARD option if ever we do not have the license
|
1
2
3
4
5
6
7
8
9
10
|
DGMGRL> EDIT DATABASE 'HRVA_SITE2' SET STATE='APPLY-OFF';Succeeded.DGMGRL> show database verbose 'HRVA_SITE2';Database - HRVA_SITE2 Role: PHYSICAL STANDBY Intended State: APPLY-OFF Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 46 seconds (computed 1 second ago) Apply Rate: (unknown) Real Time Query: OFF |
2- We open the standby database in read only mode
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
SQL> alter database open read only;Database altered.********* dbi services Ltd. *********STATUS : OPENDB_UNIQUE_NAME : HRVA_SITE2OPEN_MODE : READ ONLYLOG_MODE : ARCHIVELOGDATABASE_ROLE : PHYSICAL STANDBYFLASHBACK_ON : YESFORCE_LOGGING : YESVERSION : 11.2.0.4.0*************************************oracle@rac12e:/home/oracle/ [HRVA] |
3- We manually add the missing tempfile in the standby database
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
SQL> select name from v$tempfile;NAME---------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/temp01.dbfSQL> alter tablespace temp add tempfile '/u01/app/oracle/oradata/HRVA/temp02.dbf' size 10M;Tablespace altered.SQL> select name from v$tempfile;NAME---------------------------------------------------------------------------/u01/app/oracle/oradata/HRVA/temp01.dbf/u01/app/oracle/oradata/HRVA/temp02.dbf |
4- We shut down the standby database and mount the database
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
SQL> shutdown immediate;Database closed.Database dismounted.ORACLE instance shut down.SQL> startup mount;ORACLE instance started.Total System Global Area 417546240 bytesFixed Size 2253824 bytesVariable Size 331353088 bytesDatabase Buffers 79691776 bytesRedo Buffers 4247552 bytesDatabase mounted.SQL> |
5- We enable the redo apply
|
1
2
3
4
5
6
7
8
9
10
11
12
|
DGMGRL> EDIT DATABASE 'HRVA_SITE2' SET STATE='APPLY-ON';Succeeded.DGMGRL> show database verbose 'HRVA_SITE2';Database - HRVA_SITE2 Role: PHYSICAL STANDBY Intended State: APPLY-ON Transport Lag: 0 seconds (computed 0 seconds ago) Apply Lag: 0 seconds (computed 0 seconds ago) Apply Rate: 0 Byte/s Real Time Query: OFF Instance(s): HRVA |
Conclusion
In a dataguard environment whenever we add tempfiles or temporary tablespaces, we have to create manually the corresponding tempfiles in the standby database.