If you are using RMAN to back up your database, you are already doing the right thing. However, RMAN does not take care of everything. e.g. it is not backing up the following things, to mention just a few.<\/p>\n
There are for sure many more, but in this article I would like to take a closer look at the Data Guard broker configuraiton files. The DataGuard Broker Configuration Files are quite important files. The contain entries that describe the state and the properties of the DataGuard configuration like the sites and databases that are part of the configuration, the roles and properties of each of the databases, and the state of each of the elements of the configuration. The broker configuration files are very small, in my case they have only 12K. Don’t expect them to grow very big. They usually stay at this size.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 09:27 dr1DBIT121_SITE1.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 09:34 dr2DBIT121_SITE1.dat<\/pre>\nYou can have a maximum of two different copies which can be configured via the DG_BROKER_CONFIG_FILEn (where n = 1, 2) parameter. Maybe, maximum is not the correct word in this context, because if you don’t specify the broker file location, Oracle still creates one in the default directory which is OS dependent. On Linux they end up in $ORACLE_HOME\/dbs\/.<\/p>\n
However, not like you might expect, the broker configuration files are not a 1:1 mirror like e.g. redo log members or controlfiles. They are individual copies and they maintain the last known good state of the configuration. You can check it quite easily yourself by editing e.g. the FastStartFailoverThreshold and check the time stamps of the broker files afterwards.<\/p>\n
DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold='40';\nProperty \"faststartfailoverthreshold\" updated\n\n-- Primary\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -l dr*\n-rw-r----- 1 oracle oinstall 12288 Feb 28 09:27 dr1DBIT121_SITE1.dat\n-rw-r----- 1 oracle oinstall 12288 Feb 28 09:34 dr2DBIT121_SITE1.dat\n\n-- Standby\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -l dr*\n-rw-r----- 1 oracle oinstall 12288 Feb 28 09:34 dr1DBIT121_SITE2.dat\n-rw-r----- 1 oracle oinstall 12288 Feb 28 09:27 dr2DBIT121_SITE2.dat<\/pre>\nAs you can see here, the broker configuration files have two different time stamps. One with the last know good state at 09:27 and one with new state and 09:34. When the broker is started for the first time, it is expected that you see only one configuration file. But don’t panic, the other will be created by the next updates done by the broker.<\/p>\n
The broker configuration file is a binary file, however the readable contents of that file can be extracted via the strings command. It gives an idea of some contents of the broker file. e.g. it shows you that the broker config file is brought to you by the fine folks from NEDC, whoever they are. \ud83d\ude42 With 12cR1 is looks like the following.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] file dr2DBIT121_SITE1.dat\ndr2DBIT121_SITE1.dat: data\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] strings dr2DBIT121_SITE1.dat\n}|{z\nDBIT121\nDBIT121_SITE1\nDBIT121_SITE2\nDBIT121_SITE2\nBrought to you by the fine folks at NEDC.\nTRUETRUEdbidg03ALLDBIT121FALSECONTINUE\nDBIT121_SITE1DBIT121_SITE1FALSEFALSEFALSESYNCoptionalDISABLEON1,1948637,931101276,0*AUTO0,0,00,0,0AUTODBIT121_SITE2\ndbidg01DBIT121DBIT121(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dbidg01)(PORT=1521))(CONNECT_DATA=(SERVICE_NA\nME=DBIT121_SITE1_DGMGRL)(INSTANCE_NAME=DBIT121)(SERVER=DEDICATED)))USE_DB_RECOVERY_FILE_DEST%t_%s_%r.dbf4\nDBIT121_SITE2DBIT121_SITE2FALSEFALSEFALSESYNCoptionalDISABLEON1,1948637,931101276,0*DBIT121AUTO0,0,00,0,0AUTODBIT121_SITE1\ndbidg02DBIT121DBIT121(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=dbidg02)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBIT121_SITE2_DGMGRL)(INSTANCE_NAME=DBIT121)(SERVER=DEDICATED)))USE_DB_RECOVERY_FILE_DEST%t_%s_%r.dbf<\/pre>\nThings are changing with Oracle 12cR2. The broker config files are still binary files, however the readable content via the strings command is very good now. You can get a lot of very useful information out of it. And even with Oracle 12cR2, it looks like that it is still brought to you by the same fine folks from NEDC. \ud83d\ude42 Looks like that the Oracle developers have a sort of humor. \ud83d\ude42<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT122\/pfile\/ [DBIT122] file dr2DBIT122_SITE1.dat\ndr2DBIT122_SITE1.dat: data\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT122\/pfile\/ [DBIT122] strings dr2DBIT122_SITE1.dat\n}|{z\nDBIT122\nDBIT122_SITE1\nBrought to you by the fine folks at NEDC.\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<DRC Version=\"12.2.0.1.0\" Name=\"DBIT122\" CurrentPath=\"True\">\n <PlannedState>OFFLINE<\/PlannedState>\n <Status>\n <Severity>Success<\/Severity>\n <Error>0<\/Error>\n <\/Status>\n <DefaultState>ONLINE<\/DefaultState>\n <IntendedState>ONLINE<\/IntendedState>\n <MIV PropertyID=\"1\">0<\/MIV>\n <PRIMARY_SITE_ID PropertyID=\"26\">513<\/PRIMARY_SITE_ID>\n <DRC_UNIQUE_ID PropertyID=\"4\">152596437<\/DRC_UNIQUE_ID>\n <DRC_UNIQUE_ID_SEQUENCE PropertyID=\"5\">18<\/DRC_UNIQUE_ID_SEQUENCE>\n <EXT_COND PropertyID=\"29\">7<\/EXT_COND>\n <OVERALL_PROTECTION_MODE PropertyID=\"30\">2<\/OVERALL_PROTECTION_MODE>\n <FastStartFailoverMode PropertyID=\"32\">0<\/FastStartFailoverMode>\n <FSFO_MIV PropertyID=\"33\">11<\/FSFO_MIV>\n <FastStartFailoverOBID1 PropertyID=\"119\">470173189<\/FastStartFailoverOBID1>\n <FastStartFailoverOBID2 PropertyID=\"120\">470173183<\/FastStartFailoverOBID2>\n <FastStartFailoverOBID3 PropertyID=\"121\">470173184<\/FastStartFailoverOBID3>\n <ObserverVersion1 PropertyID=\"133\">0<\/ObserverVersion1>\n <Configuration_Name PropertyID=\"31\">DBIT122<\/Configuration_Name>\n <ObserverName1 PropertyID=\"129\">dbidg03<\/ObserverName1>\n <ConfigurationWideServiceName PropertyID=\"132\">DBIT122_CFG<\/ConfigurationWideServiceName>\n <RoleChangeHistory PropertyID=\"106\">\n <RoleChangeRecord>\n <Event>PhysicalFailover<\/Event>\n <OldPrimary>DBIT122_SITE1<\/OldPrimary>\n <NewPrimary>DBIT122_SITE2<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>931098450<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalFailover<\/Event>\n <OldPrimary>DBIT122_SITE2<\/OldPrimary>\n <NewPrimary>DBIT122_SITE1<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>931098812<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalFailover<\/Event>\n <OldPrimary>DBIT122_SITE1<\/OldPrimary>\n <NewPrimary>DBIT122_SITE2<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932306689<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE2<\/OldPrimary>\n <NewPrimary>DBIT122_SITE1<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932307856<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE1<\/OldPrimary>\n <NewPrimary>DBIT122_SITE2<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932377455<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE2<\/OldPrimary>\n <NewPrimary>DBIT122_SITE1<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932381717<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE1<\/OldPrimary>\n <NewPrimary>DBIT122_SITE2<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932382294<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE2<\/OldPrimary>\n <NewPrimary>DBIT122_SITE1<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>932383387<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE1<\/OldPrimary>\n <NewPrimary>DBIT122_SITE2<\/NewPrimary>\n <Status>0<\/Status>\n <Timestamp>934017954<\/Timestamp>\n <\/RoleChangeRecord>\n <RoleChangeRecord>\n <Event>PhysicalSwitchover<\/Event>\n <OldPrimary>DBIT122_SITE2<\/OldPrimary>\n <NewPrimary>DBIT122_SITE1<\/New\nPrimary>\n <Status>0<\/Status>\n <Timestamp>934018548<\/Timestamp>\n <\/RoleChangeRecord>\n <\/RoleChangeHistory>\n <Member MemberID=\"1\" Name=\"DBIT122_SITE1\" CurrentPath=\"True\" Enabled=\"True\" MultiInstanced=\"True\">\n <PlannedState>STANDBY<\/PlannedState>\n <StandbyRole>PhysicalStandby<\/StandbyRole>\n <Status>\n <Severity>Success<\/Severity>\n <Error>0<\/Error>\n <\/Status>\n <DefaultState>PRIMARY<\/DefaultState>\n <IntendedState>PRIMARY<\/IntendedState>\n <ResourceType>Database<\/ResourceType>\n <CurrentState>PRIMARY<\/CurrentState>\n <Role>\n <ConditionState>PRIMARY<\/ConditionState>\n <DefaultState>READ-WRITE-XPTON<\/DefaultState>\n <IntendedState>READ-WRITE-XPTON<\/IntendedState>\n <\/Role>\n <Role>\n <ConditionState>STANDBY<\/ConditionState>\n <DefaultState>PHYSICAL-APPLY-ON<\/DefaultState>\n <IntendedState>OFFLINE<\/IntendedState>\n <\/Role>\n <DB_Unique_Name PropertyID=\"23\">DBIT122_SITE1<\/DB_Unique_Name>\n <DGConnectIdentifier PropertyID=\"6\">DBIT122_SITE1<\/DGConnectIdentifier>\n <DbDomain PropertyID=\"37\"\/>\n <ClusterDatabase PropertyID=\"42\">FALSE<\/ClusterDatabase>\n <DbChangeCritical PropertyID=\"8\">FALSE<\/DbChangeCritical>\n <DbIsCritical PropertyID=\"9\">FALSE<\/DbIsCritical>\n <LogXptMode PropertyID=\"40\">SYNC<\/LogXptMode>\n <IncarnationTable PropertyID=\"57\">6,2568637,932306696,5*5,2514031,931098817,4#4,2513489,931098453,2#2,1396169,929894741,1#1,1,924281211,0#<\/IncarnationTable>\n <SRLStatus PropertyID=\"58\">0<\/SRLStatus>\n <ActualApplyInstance PropertyID=\"7\"\/>\n <StandbyFileManagement PropertyID=\"72\">AUTO<\/StandbyFileManagement>\n <ArchiveLagTarget PropertyID=\"73\">0<\/ArchiveLagTarget>\n <LogArchiveMaxProcesses PropertyID=\"74\">4<\/LogArchiveMaxProcesses>\n <LogArchiveMinSucceedDest PropertyID=\"75\">1<\/LogArchiveMinSucceedDest>\n <DataGuardSyncLatency PropertyID=\"138\">0<\/DataGuardSyncLatency>\n <DbFileNameConvert PropertyID=\"76\"\/>\n <LogFileNameConvert PropertyID=\"77\"\/>\n <FastStartFailoverTarget PropertyID=\"38\">DBIT122_SITE2<\/FastStartFailoverTarget>\n <ReinstateContextArray PropertyID=\"39\"\/>\n <Instance InstanceID=\"1\" Name=\"DBIT122\" CurrentPath=\"True\" Enabled=\"True\" MultiInstanced=\"True\" DefaultWriteOnce=\"True\">\n <PlannedState>OFFLINE<\/PlannedState>\n <HostName PropertyID=\"2\" Default=\"True\">dbidg01<\/HostName>\n <SidName PropertyID=\"3\">DBIT122<\/SidName>\n <InstanceName PropertyID=\"36\">DBIT122<\/InstanceName>\n <StaticConnectIdentifier PropertyID=\"25\">(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dbidg01)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBIT122_SITE1_DGMGRL)(UR=A)(INSTANCE_NAME=DBIT122)(SERVER=DEDICATED)))<\/StaticConnectIdentifier>\n <StandbyArchiveLocation PropertyID=\"96\" Default=\"True\">USE_DB_RECOVERY_FILE_DEST<\/StandbyArchiveLocation>\n <LogArchiveTrace PropertyID=\"98\">0<\/LogArchiveTrace>\n <LogArchiveFormat PropertyID=\"99\">%t_%s_%r.dbf<\/LogArchiveFormat>\n <\/Instance>\n <\/Member>\n <Member MemberID=\"2\" Name=\"DBIT122_SITE2\" CurrentPath=\"True\" Enabled=\"True\" MultiInstanced=\"True\">\n <PlannedState>STANDBY<\/PlannedState>\n <StandbyRole>PhysicalStandby<\/StandbyRole>\n <Status>\n <Severity>Success<\/Severity>\n <Error>0<\/Error>\n <\/Status>\n <DefaultState>STANDBY<\/DefaultState>\n <IntendedState>STANDBY<\/IntendedState>\n <ResourceType>Database<\/ResourceType>\n <CurrentState>STANDBY<\/CurrentState>\n <Role>\n <ConditionState>PRIMARY<\/ConditionState>\n <DefaultState>READ-WRITE-XPTON<\/DefaultState>\n <IntendedState>OFFLINE<\/IntendedState>\n <\/Role>\n <Role>\n <ConditionState>STANDBY<\/ConditionState>\n <DefaultState>PHYSICAL-APPLY-ON<\/DefaultState>\n <IntendedState>PHYSICAL-APPLY-ON<\/IntendedState>\n <\/Role>\n <DB_Unique_Name PropertyID=\"23\">DBIT122_SITE2<\/DB_Unique_Name>\n <DGConnectIdentifier PropertyID=\"6\">DBIT122_SITE2<\/DGConnectIdentifier>\n <DbDomain PropertyID=\"37\"\/>\n <ClusterDatabase PropertyID=\"42\">FALSE<\/ClusterDatabase>\n <DbChangeCritical PropertyID=\"8\">FALSE<\/DbChangeCritical>\n <DbIsCritical PropertyID=\"9\">FALSE<\/DbIsC\nritical>\n <LogXptMode PropertyID=\"40\">SYNC<\/LogXptMode>\n <IncarnationTable PropertyID=\"57\">8,2568637,932306696,4*4,2514031,931098817,3#3,2513489,931098453,2#2,1396169,929894741,1#1,1,924281211,0#<\/IncarnationTable>\n <SRLStatus PropertyID=\"58\">0<\/SRLStatus>\n <ActualApplyInstance PropertyID=\"7\">DBIT122<\/ActualApplyInstance>\n <StandbyFileManagement PropertyID=\"72\">AUTO<\/StandbyFileManagement>\n <ArchiveLagTarget PropertyID=\"73\">0<\/ArchiveLagTarget>\n <LogArchiveMaxProcesses PropertyID=\"74\">4<\/LogArchiveMaxProcesses>\n <LogArchiveMinSucceedDest PropertyID=\"75\">1<\/LogArchiveMinSucceedDest>\n <DataGuardSyncLatency PropertyID=\"138\">0<\/DataGuardSyncLatency>\n <DbFileNameConvert PropertyID=\"76\"\/>\n <LogFileNameConvert PropertyID=\"77\"\/>\n <FastStartFailoverTarget PropertyID=\"38\">DBIT122_SITE1<\/FastStartFailoverTarget>\n <ReinstateContextArray PropertyID=\"39\"\/>\n <Instance InstanceID=\"1\" Name=\"DBIT122\" CurrentPath=\"True\" Enabled=\"True\" MultiInstanced=\"True\" DefaultWriteOnce=\"True\">\n <PlannedState>OFFLINE<\/PlannedState>\n <HostName PropertyID=\"2\" Default=\"True\">dbidg02<\/HostName>\n <SidName PropertyID=\"3\">DBIT122<\/SidName>\n <InstanceName PropertyID=\"36\">DBIT122<\/InstanceName>\n <StaticConnectIdentifier PropertyID=\"25\">(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dbidg02)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DBIT122_SITE2_DGMGRL)(UR=A)(INSTANCE_NAME=DBIT122)(SERVER=DEDICATED)))<\/StaticConnectIdentifier>\n <StandbyArchiveLocation PropertyID=\"96\" Default=\"True\">USE_DB_RECOVERY_FILE_DEST<\/StandbyArchiveLocation>\n <LogArchiveTrace PropertyID=\"98\">0<\/LogArchiveTrace>\n <LogArchiveFormat PropertyID=\"99\">%t_%s_%r.dbf<\/LogArchiveFormat>\n <\/Instance>\n <\/Member>\n<\/DRC><\/pre>\nIn 12cR2, Oracle put more information into the broker files and that’s why they are getting bigger. With a standard config they have now 16k instead of 12k like they had before with 12cR1. Ok .. it looks like I am drifting away. Let’s get back to the original question, how do I recover the broker configuration files, in case they get lost? Like always … it depends. You could lose the old copy on the standby, you could lose the new copy on the standby, or you could lose both copies on the standby, and you could lose them while the standby is up and running or while it is shutdown. The same applies to the primary. It might get even more complicated in case you have a far sync database in between, or more standby’s.<\/p>\n
Not making it too complex, in the end, we might end up with 6 different recovery scenarios.<\/p>\n
1.) We lose the old copy of the broker config file on either the primary or the standby
\n2.) We lose both copies of the broker config file on the standby while it is up and running
\n3.) We lose both copies of the broker config file on the standby while it is shut down
\n4.) We lose both copies of the broker config file on the primary while it is up and running
\n5.) We lose both copies of the broker config file on the primary while it is shut down
\n6.) We lose both copies of the broker contfig files on the primary and the standby (quite unlikely to happen)<\/p>\nMy demos are built on a simple primary\/standby setup with 12cR1. The primary is on host dbidg01 and the standby is currently on host dbidg02.<\/p>\n
DGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 2 seconds ago)\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:03 dr1DBIT121_SITE1.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:04 dr2DBIT121_SITE1.dat\n\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:04 dr1DBIT121_SITE2.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:03 dr2DBIT121_SITE2.dat<\/pre>\nScenario 1.) We lose the old copy of the broker config file on either the primary or the standby<\/h3>\n
There is absolutely not need to panic in this case. I am deleting the old copies on the primary and standby at the same time.<\/p>\n
-- primary\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr1DBIT121_SITE1.dat\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]\n\n-- standby\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr2DBIT121_SITE2.dat\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]<\/pre>\nThe broker does not care at all.<\/p>\n
DGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 57 seconds ago)<\/pre>\nThe second copy will be created automatically as soon as I am changing a property. e.g. I am setting the FastStartFailoverThreshold to the same value as it had before.<\/p>\n
DGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold='40';\nProperty \"faststartfailoverthreshold\" updated<\/pre>\nThe broker log has no information about the new created file and also the alert.log does not say anything, but automagically we end up with two broker config files after the update was done. So .. nothing to do here. Oracle handles this itself.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/diag\/rdbms\/dbit121_site1\/DBIT121\/trace\/ [DBIT121] tail -20f drcDBIT121.log\n...\n02\/28\/2017 11:10:25\nEDIT CONFIGURATION SET PROPERTY faststartfailoverthreshold = 40\nFSFO threshold value set to 40 seconds\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -l dr*\n-rw-r----- 1 oracle oinstall 12288 Feb 28 11:10 dr1DBIT121_SITE1.dat\n-rw-r----- 1 oracle oinstall 12288 Feb 28 11:04 dr2DBIT121_SITE1.dat\n\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:04 dr1DBIT121_SITE2.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:10 dr2DBIT121_SITE2.dat<\/pre>\n<\/p>\n
2.) We lose both copies of the broker config file on the standby while it is up and running<\/h3>\n
But what happens if I lose both copies on the standby while it is up and running.<\/p>\n
oracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*.dat\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\nls: cannot access dr*: No such file or directory<\/pre>\nI am logging in into the standby database and check the configuration. The broker does not care at all if the files are there or not. It seems like nothing has happend.<\/p>\n
oracle@dbidg02:\/home\/oracle\/ [DBIT121] dgmgrl -debug -xml\nDGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production\n\nCopyright (c) 2000, 2013, Oracle. All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL> connect sys\/manager@DBIT121_SITE2\n[W000 02\/28 11:17:00.64] Connecting to database using DBIT121_SITE2.\n[W000 02\/28 11:17:01.72] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 11:17:01.72] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 8 seconds ago)<\/pre>\nThe broker is reading the information from memory, maybe because of performance reasons. The current state is also reflected in the x$drc.<\/p>\n
SQL> desc x$drc\n Name Null? Type\n ----------------------------------------- -------- ----------------------------\n ADDR RAW(8)\n INDX NUMBER\n INST_ID NUMBER\n CON_ID NUMBER\n OBJECT_ID NUMBER\n ATTRIBUTE VARCHAR2(30)\n VALUE VARCHAR2(512)\n PARENT_ID VARCHAR2(15)\n STATUS VARCHAR2(30)\n MESSAGE VARCHAR2(256)\n ERRNUM NUMBER\n VALUE_RAW RAW(512)\n ERRTIME NUMBER<\/pre>\nSo … how do we get the broker files back? \u00a0By simply editing any property. You can choose any property you want.<\/p>\n
oracle@dbidg02:\/home\/oracle\/ [DBIT121] dgmgrl -debug -xml\nDGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production\n\nCopyright (c) 2000, 2013, Oracle. All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL> connect sys\/manager@DBIT121_SITE2\n[W000 02\/28 11:27:58.23] Connecting to database using DBIT121_SITE2.\n[W000 02\/28 11:27:59.33] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 11:27:59.33] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold='40';\n<DO_CONFIGURE_DRC version=\"12.1\"><EDIT_DRC><PROPERTY name=\"faststartfailoverthreshold\" value=\"40\" context=\"0\"\/><\/EDIT_DRC><\/DO_CONFIGURE_DRC>\n<RESULT ><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><\/RESULT>\n<RESULT ><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><\/RESULT>\nProperty \"faststartfailoverthreshold\" updated<\/pre>\nAs soon as you have edited your configuration, a new broker file appears on the standby.<\/p>\n
oracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:28 dr1DBIT121_SITE2.dat<\/pre>\nIf you run the same command again, the second one appears as well.<\/p>\n
oracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:28 dr1DBIT121_SITE2.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:30 dr2DBIT121_SITE2.dat<\/pre>\n<\/p>\n
3.) We lose both copies of the broker config file on the standby while it is shut down<\/h3>\n
What happens if we lose both copies while the standby is shut down. Does the standby come up correctly? Is there something in the drc.log or alert.log?<\/p>\n
SQL> shutdown immediate\nDatabase closed.\nDatabase dismounted.\nORACLE instance shut down.\n\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*.dat\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]\n\nSQL> startup mount\nORACLE instance started.\n\nTotal System Global Area 1325400064 bytes\nFixed Size 2924112 bytes\nVariable Size 352321968 bytes\nDatabase Buffers 956301312 bytes\nRedo Buffers 13852672 bytes\nDatabase mounted.<\/pre>\nThe standby comes up and no entries in the alert.log, however the drc.log shows that the broker files are missing.<\/p>\n
02\/28\/2017 11:35:26\n>> Starting Data Guard Broker bootstrap <<\nBroker Configuration File Locations:\n dg_broker_config_file1 = \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\"\n dg_broker_config_file2 = \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE2.dat\"\n2017-02-28 11:35:26.313 DMON: Attach state object\n2017-02-28 11:35:26.314 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\", retrying\n2017-02-28 11:35:27.334 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\"\n2017-02-28 11:35:27.334 ORA-27037: unable to obtain file status\n2017-02-28 11:35:27.335 Linux-x86_64 Error: 2: No such file or directory\n2017-02-28 11:35:27.335 Additional information: 3\n2017-02-28 11:35:27.335 DMON: Error opening \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\", error = ORA-16572\n2017-02-28 11:35:27.335 DMON: Establishing \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE2.dat\" as the more current file\n2017-02-28 11:35:27.335 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE2.dat\", retrying\n2017-02-28 11:35:28.355 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE2.dat\"\n...\n2017-02-28 11:35:42.893 Configuration does not exist, Data Guard broker ready\n2017-02-28 11:35:42.894 7fffffff 0 DMON: Entered rfm_release_chief_lock() for CTL_BOOTSTRAP\n\n\nDGMGRL> connect sys\/manager@DBIT121_SITE2\n[W000 02\/28 11:38:50.58] Connecting to database using DBIT121_SITE2.\n[W000 02\/28 11:38:51.67] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 11:38:51.68] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> show configuration;\nORA-16532: Oracle Data Guard broker configuration does not exist\n\nConfiguration details cannot be determined by DGMGRL\n\n\nSQL> select attribute, value from x$drc;\nselect attribute, value from x$drc\n *\nERROR at line 1:\nORA-16532: Oracle Data Guard broker configuration does not exist<\/pre>\nIf we logon to the primary, we can see that the standby DBIT121_SITE2 was disabled.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] dgmgrl -debug -xml\nDGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production\n\nCopyright (c) 2000, 2013, Oracle. All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL> connect sys\/manager@DBIT121_SITE1\n[W000 02\/28 11:45:40.56] Connecting to database using DBIT121_SITE1.\n[W000 02\/28 11:45:41.68] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 11:45:41.68] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n Warning: ORA-16629: database reports a different protection level from the protection mode\n\n DBIT121_SITE2 - Physical standby database (disabled)\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nWARNING (status updated 7 seconds ago)<\/pre>\nHowever, after we enable the standby, we can see in the drc.log that the standby receives the metadata from the primary and creates the broker config file.<\/p>\n
DGMGRL> enable database 'DBIT121_SITE2';\n<DO_CONTROL version=\"12.1\"><DO_COMMAND type=\"Enable\" object_id=\"33554432\"\/><\/DO_CONTROL>\n<RESULT ><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><\/RESULT>\nEnabled.\nDGMGRL>\n\n\ndrc.log\n\n2017-02-28 11:46:03.352 DRCX: Start receiving metadata file: \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\"\n2017-02-28 11:46:03.356 DRCX: Receiving block #1 (containing Seq.MIV = 2.20), 2 blocks\n2017-02-28 11:46:03.358 DRCX: End receiving metadata file: opcode CTL_ENABLE (1.1.631286030)\n2017-02-28 11:46:03.360 DMON: Entered rfm_get_chief_lock() for CTL_ENABLE, reason 1\n2017-02-28 11:46:03.360 02001000 631286030 DMON: start task execution: for metadata resynchronization\n2017-02-28 11:46:03.360 02001000 631286030 DMON: status from posting standby instances for RESYNCH = ORA-00000\n2017-02-28 11:46:03.360 INSV: Received message for inter-instance publication\n2017-02-28 11:46:03.361 req ID 1.1.631286030, opcode CTL_ENABLE, phase RESYNCH, flags 8005\n2017-02-28 11:46:03.361 02001000 631286030 DMON: Metadata available (1.1.631286030), loading from \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE2.dat\"\n2017-02-28 11:46:03.361 02001000 631286030 Opcode = CTL_ENABLE, Chief Instance I_ID = 1\n2017-02-28 11:46:03.364 DMON Registering service DBIT121_SITE2_DGB with listener(s)\n2017-02-28 11:46:03.364 DMON: Executing SQL [ALTER SYSTEM REGISTER]\n2017-02-28 11:46:03.365 SQL [ALTER SYSTEM REGISTER] Executed successfully\n02\/28\/2017 11:46:06\nCreating process RSM0\n\n\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 11:46 dr1DBIT121_SITE2.dat<\/pre>\nAgain, if you do any property change again, the second broker file will be created.<\/p>\n
4.) We lose both copies of the broker config file on the primary while it is up and running<\/h3>\n
There is not much difference with the scenario we have seen with the standby. The broker just reads from memory and as soon as any update to the config is done, the broker file is created again. There will be no entries in the drc.log or the alert.log. The broker file is just silently recreated.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\nls: cannot access dr*: No such file or directory\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] dgmgrl -debug -xml\nDGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production\n\nCopyright (c) 2000, 2013, Oracle. All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL> connect sys\/manager\n[W000 02\/28 12:28:45.07] Connecting to database using .\n[W000 02\/28 12:28:45.10] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 12:28:45.10] Oracle database version is '12.1.0.2.0'\nConnected as SYSDG.\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 9 seconds ago)\n\n\nDGMGRL> EDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold='40';\n<DO_CONFIGURE_DRC version=\"12.1\"><EDIT_DRC><PROPERTY name=\"faststartfailoverthreshold\" value=\"40\" context=\"0\"\/><\/EDIT_DRC><\/DO_CONFIGURE_DRC>\n<RESULT ><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><\/RESULT>\n<RESULT ><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><MESSAGE ><SUCCESS ><\/SUCCESS><\/MESSAGE><\/RESULT>\nProperty \"faststartfailoverthreshold\" updated\n\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 12:30 dr2DBIT121_SITE1.dat<\/pre>\n<\/p>\n
5.) We lose both copies of the broker config file on the primary while it is shut down<\/h3>\n
Let’s do the whole thing again while the primary DB is shutdown.<\/p>\n
SQL> shutdown immediate\nDatabase closed.\nDatabase dismounted.\nORACLE instance shut down.\nSQL>\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -lh dr*\nls: cannot access dr*: No such file or directory<\/pre>\nIn the drc.log log file you can immediately see that there is something wrong.<\/p>\n
>> Starting Data Guard Broker bootstrap <<\nBroker Configuration File Locations:\n dg_broker_config_file1 = \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE1.dat\"\n dg_broker_config_file2 = \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE1.dat\"\n2017-02-28 12:34:45.866 DMON: Attach state object\n2017-02-28 12:34:45.866 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE1.dat\", retrying\n2017-02-28 12:34:46.876 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE1.dat\"\n2017-02-28 12:34:46.880 ORA-27037: unable to obtain file status\n2017-02-28 12:34:46.881 Linux-x86_64 Error: 2: No such file or directory\n2017-02-28 12:34:46.881 Additional information: 3\n2017-02-28 12:34:46.881 DMON: Error opening \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE1.dat\", error = ORA-16572\n2017-02-28 12:34:46.881 DMON: Establishing \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE1.dat\" as the more current file\n2017-02-28 12:34:46.882 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE1.dat\", retrying\n2017-02-28 12:34:47.899 DMON: cannot open configuration file \"\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr2DBIT121_SITE1.dat\"\n...\n2017-02-28 12:35:02.058 7fffffff 0 DMON: Entered rfm_release_chief_lock() for CTL_BOOTSTRAP\n2017-02-28 12:35:02.424 Fore: Continuing with primary evaluation, rfmsp.drc_status_rfmp = ORA-16532\n2017-02-28 12:35:03.507 Fore: Initiating post-open tasks\n2017-02-28 12:35:09.192 DMON: Initiating post-open tasks\n2017-02-28 12:35:22.242 00000000 1934847279 DMON: GET_FSFO will be retired\n2017-02-28 12:35:22.242 00000000 1934847279 severity = ORA-16501, status = ORA-16532\n2017-02-28 12:35:22.242 00000000 1934847279 DMON: GET_FSFO operation completed\n2017-02-28 12:35:52.250 00000000 1934847280 DMON: GET_FSFO will be retired\n2017-02-28 12:35:52.250 00000000 1934847280 severity = ORA-16501, status = ORA-16532\n2017-02-28 12:35:52.251 00000000 1934847280 DMON: GET_FSFO operation completed<\/pre>\nIf you take a look at the alert.log, everything is fine.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] dgmgrl -debug -xml\nDGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production\n\nCopyright (c) 2000, 2013, Oracle. All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL> connect sys\/manager@DBIT121_SITE1\n[W000 02\/28 12:39:01.18] Connecting to database using DBIT121_SITE1.\n[W000 02\/28 12:39:02.28] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 12:39:02.28] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> show configuration;\nORA-16532: Oracle Data Guard broker configuration does not exist\n\nConfiguration details cannot be determined by DGMGRL<\/pre>\nBut if connected to the standby, you will see a clear error message: ORA-16532: Oracle Data Guard broker configuration does not exist.<\/p>\n
DGMGRL> connect sys\/manager@DBIT121_SITE2\n[W000 02\/28 12:39:18.81] Connecting to database using DBIT121_SITE2.\n[W000 02\/28 12:39:19.90] Checking broker version [BEGIN :version := dbms_drs.dg_broker_info('VERSION'); END;].\n[W000 02\/28 12:39:19.90] Oracle database version is '12.1.0.2.0'\nConnected as SYSDBA.\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n Error: ORA-16532: Oracle Data Guard broker configuration does not exist\n\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nERROR (status updated 0 seconds ago)<\/pre>\nTaking a close look at the error message, it does not give you a hint how to correct the issue.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] oerr ora 16532\n16532, 00000, \"Oracle Data Guard broker configuration does not exist\"\n\/\/ *Cause: A broker operation was requested that required a broker\n\/\/ configuration to already be created.\n\/\/ *Action: Create a Data Guard broker configuration prior to performing\n\/\/ other broker operations. If only one instance of a RAC\n\/\/ database is reporting this error, ensure that the\n\/\/ DG_BROKER_CONFIG_FILE[1|2] initialization parameters are\n\/\/ set to file locations that are shared by all instances of\n\/\/ the RAC database.<\/pre>\nLet’s try the same trick, as we have done on the standby by simply enabling the database again.<\/p>\n
DGMGRL> enable database 'DBIT121_SITE1';\n<DO_CONTROL version=\"12.1\"><DO_COMMAND type=\"Enable\" object_id=\"16777216\"\/><\/DO_CONTROL>\n<RESULT ><MESSAGE ><FAILURE error_num=\"16532\" error_prefix=\"ORA\"><ERROR_TEXT >ORA-16532: Oracle Data Guard broker configuration does not exist\n<\/ERROR_TEXT><\/FAILURE><\/MESSAGE><MESSAGE ><FAILURE error_num=\"16625\" error_prefix=\"ORA\" error_tag1=\"DBIT121_SITE1\"><ERROR_TEXT >ORA-16625: cannot reach database "DBIT121_SITE1"\n<\/ERROR_TEXT><\/FAILURE><\/MESSAGE><\/RESULT>\n<RESULT ><MESSAGE ><FAILURE error_num=\"16532\" error_prefix=\"ORA\"><ERROR_TEXT >ORA-16532: Oracle Data Guard broker configuration does not exist\n<\/ERROR_TEXT><\/FAILURE><\/MESSAGE><MESSAGE ><FAILURE error_num=\"16625\" error_prefix=\"ORA\" error_tag1=\"DBIT121_SITE1\"><ERROR_TEXT >ORA-16625: cannot reach database "DBIT121_SITE1"\n<\/ERROR_TEXT><\/FAILURE><\/MESSAGE><\/RESULT>\nFailed.<\/pre>\nHmmmm … does not look good. To recovery from that situation, we need to figure out the latest broker file version on the standby, which is dr2DBIT121_SITE2.dat is my case, and copy it over to the primary. Before doing that, we need to stop the broker on the primary.<\/p>\n
SQL> alter system set dg_broker_start=false;\n\nSystem altered.<\/pre>\nNow we can copy the latest version over.<\/p>\n
-- standby\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] ls -rlth dr*\n-rw-r----- 1 oracle oinstall 12K Feb 28 12:30 dr1DBIT121_SITE2.dat\n-rw-r----- 1 oracle oinstall 12K Feb 28 12:33 dr2DBIT121_SITE2.dat\n\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] scp -p dr2DBIT121_SITE2.dat oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/dr1DBIT121_SITE1.dat\ndr2DBIT121_SITE2.dat 100% 12KB 12.0KB\/s 00:00<\/pre>\nAnd optionally create the second broker file on the primary.<\/p>\n
-- primary\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] cp -p dr1DBIT121_SITE1.dat dr2DBIT121_SITE1.dat\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]<\/pre>\nLast but not least, we need to enable the broker again.<\/p>\n
SQL> alter system set dg_broker_start=true;\n\nSystem altered.\n\nDGMGRL> connect sys\/manager@DBIT121_SITE1\nConnected as SYSDBA.\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 7 seconds ago)<\/pre>\n6.) We lose both copies of the broker config files on the primary and the standby (quite unlikely to happen)<\/h3>\n
This scenario is quite unlikely to happen, but if it happens it is good to have your Data Guard configuration as a script in place.<\/p>\n
-- shutdown primary\nSQL> shutdown immediate\nDatabase closed.\nDatabase dismounted.\nORACLE instance shut down.\n\n-- shutdown standby\nSQL> shutdown immediate\nORA-01109: database not open\nDatabase dismounted.\nORACLE instance shut down.<\/pre>\nRemove the broker files on the primary and the standby<\/p>\n
-- primary\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]\n\n-- standby\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121] rm dr*\noracle@dbidg02:\/u01\/app\/oracle\/admin\/DBIT121\/pfile\/ [DBIT121]<\/pre>\nNow, after we have lost everything, meaning all broker config files, the only chance that we have is to create it from scratch. It is quite cool, if you have the configuration is place, because in a blink of an eye you recreate the whole configuration.<\/p>\n
oracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/create\/ [DBIT121] cat broker.cfg\nCONNECT SYS\/manager@DBIT121_SITE1\nREMOVE CONFIGURATION;\nsql \"alter system set log_archive_dest_2=''\";\nCREATE CONFIGURATION 'DBIT121' AS\nPRIMARY DATABASE IS 'DBIT121_SITE1'\nCONNECT IDENTIFIER IS 'DBIT121_SITE1';\nADD DATABASE 'DBIT121_SITE2' AS\nCONNECT IDENTIFIER IS 'DBIT121_SITE2';\nEDIT DATABASE 'DBIT121_SITE1' SET PROPERTY StandbyFileManagement='AUTO';\nEDIT DATABASE 'DBIT121_SITE2' SET PROPERTY StandbyFileManagement='AUTO';\nEDIT DATABASE 'DBIT121_SITE1' SET PROPERTY LogXptMode='SYNC';\nEDIT DATABASE 'DBIT121_SITE2' SET PROPERTY LogXptMode='SYNC';\nENABLE CONFIGURATION;\nEDIT CONFIGURATION SET PROTECTION MODE AS MaxAvailability;\nEDIT CONFIGURATION SET PROPERTY FastStartFailoverThreshold='40';\nEDIT DATABASE 'DBIT121_SITE1' SET PROPERTY FastStartFailoverTarget='DBIT121_SITE2';\nEDIT DATABASE 'DBIT121_SITE2' SET PROPERTY FastStartFailoverTarget='DBIT121_SITE1';\nsql \"alter system archive log current\";\nsql \"alter system register\";\nSHOW CONFIGURATION;\n\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/create\/ [DBIT121] dgmgrl -debug -xml < broker.cfg > broker.log 2>&1\noracle@dbidg01:\/u01\/app\/oracle\/admin\/DBIT121\/create\/ [DBIT121]\n\nDGMGRL> show configuration;\n\nConfiguration - DBIT121\n\n Protection Mode: MaxAvailability\n Members:\n DBIT121_SITE1 - Primary database\n DBIT121_SITE2 - Physical standby database\n\nFast-Start Failover: DISABLED\n\nConfiguration Status:\nSUCCESS (status updated 2 seconds ago)<\/pre>\nConclusion<\/h3>\n
Loosing broker configuration files is not the end of the world. As you have seen in the blog, it makes quite a huge difference if you loose the broker file while the DB is up and running or it is shut down. In case you haven’t lost all of them you can recover them in most of the situations. However, the alert.log is not a good candidate to check if the broker files are missing. So better scan the alert.log and the drc.log with your preferred monitoring tool to get the whole picture. Besides that, it is good practice to adjust your broker.cfg file after you have done some changes because it makes it very easy to rebuild the whole broker config from scratch.<\/p>\n","protected":false},"excerpt":{"rendered":"
By William Sescu If you are using RMAN to back up your database, you are already doing the right thing. However, RMAN does not take care of everything. e.g. it is not backing up the following things, to mention just a few. Oracle Home Grid Infrastructure Home Data Guard broker files Password File SQL*Net file […]<\/p>\n","protected":false},"author":27,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[229,198],"tags":[221,209],"type_dbi":[],"class_list":["post-9833","post","type-post","status-publish","format-standard","hentry","category-database-administration-monitoring","category-database-management","tag-data-guard","tag-oracle-12c"],"acf":[],"yoast_head":"\n
Oracle 12c - How to Recover lost DataGuard Broker Configuration Files - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Oracle 12c - How to Recover lost DataGuard Broker Configuration Files\" \/>\n<meta property=\"og:description\" content=\"By William Sescu If you are using RMAN to back up your database, you are already doing the right thing. However, RMAN does not take care of everything. e.g. it is not backing up the following things, to mention just a few. Oracle Home Grid Infrastructure Home Data Guard broker files Password File SQL*Net file […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2017-02-28T12:53:50+00:00\" \/>\n<meta name=\"author\" content=\"Oracle Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Oracle Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"31 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\"},\"author\":{\"name\":\"Oracle Team\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee\"},\"headline\":\"Oracle 12c – How to Recover lost DataGuard Broker Configuration Files\",\"datePublished\":\"2017-02-28T12:53:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\"},\"wordCount\":1727,\"commentCount\":2,\"keywords\":[\"Data Guard\",\"Oracle 12c\"],\"articleSection\":[\"Database Administration & Monitoring\",\"Database management\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\",\"name\":\"Oracle 12c - How to Recover lost DataGuard Broker Configuration Files - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"datePublished\":\"2017-02-28T12:53:50+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Oracle 12c – How to Recover lost DataGuard Broker Configuration Files\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee\",\"name\":\"Oracle Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g\",\"caption\":\"Oracle Team\"},\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/oracle-team\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Oracle 12c - How to Recover lost DataGuard Broker Configuration Files - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/","og_locale":"en_US","og_type":"article","og_title":"Oracle 12c - How to Recover lost DataGuard Broker Configuration Files","og_description":"By William Sescu If you are using RMAN to back up your database, you are already doing the right thing. However, RMAN does not take care of everything. e.g. it is not backing up the following things, to mention just a few. Oracle Home Grid Infrastructure Home Data Guard broker files Password File SQL*Net file […]","og_url":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/","og_site_name":"dbi Blog","article_published_time":"2017-02-28T12:53:50+00:00","author":"Oracle Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Oracle Team","Est. reading time":"31 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/"},"author":{"name":"Oracle Team","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee"},"headline":"Oracle 12c – How to Recover lost DataGuard Broker Configuration Files","datePublished":"2017-02-28T12:53:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/"},"wordCount":1727,"commentCount":2,"keywords":["Data Guard","Oracle 12c"],"articleSection":["Database Administration & Monitoring","Database management"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/","url":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/","name":"Oracle 12c - How to Recover lost DataGuard Broker Configuration Files - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2017-02-28T12:53:50+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/oracle-12c-how-to-recover-lost-dataguard-broker-configuration-files\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Oracle 12c – How to Recover lost DataGuard Broker Configuration Files"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/66ab87129f2d357f09971bc7936a77ee","name":"Oracle Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/f711f7cd2c9b09bf2627133755b569fb5be0694810cfd33033bdd095fedba86d?s=96&d=mm&r=g","caption":"Oracle Team"},"url":"https:\/\/www.dbi-services.com\/blog\/author\/oracle-team\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/9833","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/27"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=9833"}],"version-history":[{"count":0,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/9833\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=9833"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=9833"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=9833"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=9833"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}