{"id":32178,"date":"2024-03-31T08:50:26","date_gmt":"2024-03-31T06:50:26","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/?p=32178"},"modified":"2024-03-31T08:50:30","modified_gmt":"2024-03-31T06:50:30","slug":"zdm-physical-online-migration-failing-during-zdm_switchover_src-phase","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/","title":{"rendered":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase"},"content":{"rendered":"\n<p>When using ZDM for a Physical Online Migration, even in the last current version, 21.4.5.0.0, switchover is failing. In this blog, I will share the reason and the solution to solve the problem before moving forward with the migration.<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-problem-description\">Problem description<\/h2>\n\n\n\n<p>We are using last version of ZDM :<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,4]\">\n[zdmuser@zdmhost migration]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli -build\nversion: 21.0.0.0.0\nfull version: 21.4.0.0.0\npatch version: 21.4.5.0.0\nlabel date: 221207.30\nZDM kit build date: Mar 21 2024 22:07:12 UTC\nCPAT build version: 23.12.0\n<\/pre>\n<\/br>\n\n\n\n<p>ZDM Response file parameter has been setup to use broker during migration: <code>ZDM_USE_DG_BROKER=TRUE<\/code><\/p>\n\n\n\n<p>This problem might not happened if not using the broker to run the switchover operation. We have decided to use the broker to be able to easily check and monitor the synchronisation between our on-premise database and the ExaCC one.<\/p>\n\n\n\n<p>The on-premise database is here called ONPR, and the final PDB, ZDM needs to create and migrate to, will be ONPRZ_APP_001T. We configured ZDM to include non-cdb to cdb migration.<\/p>\n\n\n\n<p>We have run ZDM Migration and paused it just after Data Guard is configured.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,9,34]\">\n[zdmuser@zdmhost migration]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli query job -jobid 75\nzdmhost.balgroupit.com: Audit ID: 1164\nJob ID: 75\nUser: zdmuser\nClient: zdmhost\nJob Type: \"MIGRATE\"\nScheduled job command: \"zdmcli migrate database -sourcesid ONPR -rsp \/home\/zdmuser\/migration\/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -srcarg3 sudo_location:\/usr\/bin\/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -tgtarg3 sudo_location:\/usr\/bin\/sudo -tdekeystorepasswd -tgttdekeystorepasswd -pauseafter ZDM_CONFIGURE_DG_SRC\"\nScheduled job execution start time: 2024-03-22T15:29:56+01. Equivalent local time: 2024-03-22 15:29:56\nCurrent status: PAUSED\nCurrent Phase: \"ZDM_CONFIGURE_DG_SRC\"\nResult file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.log\"\nMetrics file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.json\"\nJob execution start time: 2024-03-22 15:30:13\nJob execution end time: 2024-03-22 16:32:20\nJob execution elapsed time: 19 minutes 44 seconds\nZDM_GET_SRC_INFO ................ COMPLETED\nZDM_GET_TGT_INFO ................ COMPLETED\nZDM_PRECHECKS_SRC ............... COMPLETED\nZDM_PRECHECKS_TGT ............... COMPLETED\nZDM_SETUP_SRC ................... COMPLETED\nZDM_SETUP_TGT ................... COMPLETED\nZDM_PREUSERACTIONS .............. COMPLETED\nZDM_PREUSERACTIONS_TGT .......... COMPLETED\nZDM_VALIDATE_SRC ................ COMPLETED\nZDM_VALIDATE_TGT ................ COMPLETED\nZDM_DISCOVER_SRC ................ COMPLETED\nZDM_COPYFILES ................... COMPLETED\nZDM_PREPARE_TGT ................. COMPLETED\nZDM_SETUP_TDE_TGT ............... COMPLETED\nZDM_RESTORE_TGT ................. COMPLETED\nZDM_RECOVER_TGT ................. COMPLETED\nZDM_FINALIZE_TGT ................ COMPLETED\nZDM_CONFIGURE_DG_SRC ............ COMPLETED\nZDM_SWITCHOVER_SRC .............. PENDING\nZDM_SWITCHOVER_TGT .............. PENDING\nZDM_POST_DATABASE_OPEN_TGT ...... PENDING\nZDM_NONCDBTOPDB_PRECHECK ........ PENDING\nZDM_NONCDBTOPDB_CONVERSION ...... PENDING\nZDM_POST_MIGRATE_TGT ............ PENDING\nTIMEZONE_UPGRADE_PREPARE_TGT .... PENDING\nTIMEZONE_UPGRADE_TGT ............ PENDING\nZDM_POSTUSERACTIONS ............. PENDING\nZDM_POSTUSERACTIONS_TGT ......... PENDING\nZDM_CLEANUP_SRC ................. PENDING\nZDM_CLEANUP_TGT ................. PENDING\n\nPause After Phase: \"ZDM_CONFIGURE_DG_SRC\"\n[zdmuser@zdmhost migration]$\n<\/pre>\n<\/br>\n\n\n\n<p>This will give us the opportunity to have done all the preparation without any downtime and we can wait for the migration maintenance window.<\/p>\n\n\n\n<p>The Standby on the ExaCC is synchronised with the primary on-premise database. There is no lag.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,9,13,21,22]\">\noracle@vmonpr:\/home\/oracle\/ [ONPR] dgmgrl\nDGMGRL for Linux: Release 19.0.0.0.0 - Production on Fri Mar 22 16:40:23 2024\nVersion 19.22.0.0.0\n\nCopyright (c) 1982, 2019, Oracle and\/or its affiliates.  All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\n\nDGMGRL&gt; connect \/\nConnected to \"ONPR\"\nConnected as SYSDG.\n\nDGMGRL&gt; show configuration lag\n\nConfiguration - ZDM_onpr\n\n  Protection Mode: MaxPerformance\n  Members:\n  onpr           - Primary database\n    onprz_app_001t - Physical standby database\n                     Transport Lag:      0 seconds (computed 1 second ago)\n                     Apply Lag:          0 seconds (computed 1 second ago)\n\nFast-Start Failover:  Disabled\n\nConfiguration Status:\nSUCCESS   (status updated 22 seconds ago)\n<\/pre>\n<\/br>\n\n\n\n<p>We resume ZDM migration job now, adding a pause just after switchover has been completed.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1]\">\n[zdmuser@zdmhost migration]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli resume job -jobid 75 -pauseafter ZDM_SWITCHOVER_TGT\nzdmhost.balgroupit.com: Audit ID: 1165\n<\/pre>\n<\/br>\n\n\n\n<p>As we can see the job failed during the ZDM_SWITCHOVER_SRC phase.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,9,33]\">\n[zdmuser@zdmhost migration]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli query job -jobid 75\nzdmhost.balgroupit.com: Audit ID: 1166\nJob ID: 75\nUser: zdmuser\nClient: zdmhost\nJob Type: \"MIGRATE\"\nScheduled job command: \"zdmcli migrate database -sourcesid ONPR -rsp \/home\/zdmuser\/migration\/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -srcarg3 sudo_location:\/usr\/bin\/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -tgtarg3 sudo_location:\/usr\/bin\/sudo -tdekeystorepasswd -tgttdekeystorepasswd -pauseafter ZDM_CONFIGURE_DG_SRC\"\nScheduled job execution start time: 2024-03-22T15:29:56+01. Equivalent local time: 2024-03-22 15:29:56\nCurrent status: FAILED\nResult file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.log\"\nMetrics file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.json\"\nJob execution start time: 2024-03-22 15:30:13\nJob execution end time: 2024-03-22 16:43:34\nJob execution elapsed time: 22 minutes 4 seconds\nZDM_GET_SRC_INFO ................ COMPLETED\nZDM_GET_TGT_INFO ................ COMPLETED\nZDM_PRECHECKS_SRC ............... COMPLETED\nZDM_PRECHECKS_TGT ............... COMPLETED\nZDM_SETUP_SRC ................... COMPLETED\nZDM_SETUP_TGT ................... COMPLETED\nZDM_PREUSERACTIONS .............. COMPLETED\nZDM_PREUSERACTIONS_TGT .......... COMPLETED\nZDM_VALIDATE_SRC ................ COMPLETED\nZDM_VALIDATE_TGT ................ COMPLETED\nZDM_DISCOVER_SRC ................ COMPLETED\nZDM_COPYFILES ................... COMPLETED\nZDM_PREPARE_TGT ................. COMPLETED\nZDM_SETUP_TDE_TGT ............... COMPLETED\nZDM_RESTORE_TGT ................. COMPLETED\nZDM_RECOVER_TGT ................. COMPLETED\nZDM_FINALIZE_TGT ................ COMPLETED\nZDM_CONFIGURE_DG_SRC ............ COMPLETED\nZDM_SWITCHOVER_SRC .............. FAILED\nZDM_SWITCHOVER_TGT .............. PENDING\nZDM_POST_DATABASE_OPEN_TGT ...... PENDING\nZDM_NONCDBTOPDB_PRECHECK ........ PENDING\nZDM_NONCDBTOPDB_CONVERSION ...... PENDING\nZDM_POST_MIGRATE_TGT ............ PENDING\nTIMEZONE_UPGRADE_PREPARE_TGT .... PENDING\nTIMEZONE_UPGRADE_TGT ............ PENDING\nZDM_POSTUSERACTIONS ............. PENDING\nZDM_POSTUSERACTIONS_TGT ......... PENDING\nZDM_CLEANUP_SRC ................. PENDING\nZDM_CLEANUP_TGT ................. PENDING\n\nPause After Phase: \"ZDM_SWITCHOVER_TGT\"\n<\/pre>\n<\/br>\n\n\n\n<p>ZDM log will display following information, showing that the switchover is failing on starting the old primary on-premise database.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>####################################################################\nPRGZ-3605 : Oracle Data Guard Broker switchover to database \"ONPRZ_APP_001T\" on database \"ONPR\" failed.\nONPRZ_APP_001T\nDGMGRL for Linux: Release 19.0.0.0.0 - Production on Fri Mar 22 15:43:19 2024\nVersion 19.22.0.0.0\n\nCopyright (c) 1982, 2019, Oracle and\/or its affiliates.  All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nConnected to \"ONPR\"\nConnected as SYSDG.\nDGMGRL&gt; Performing switchover NOW, please wait...\nOperation requires a connection to database \"onprz_app_001t\"\nConnecting ...\nConnected to \"ONPRZ_APP_001T\"\nConnected as SYSDBA.\nNew primary database \"onprz_app_001t\" is opening...\nOperation requires start up of instance \"ONPR\" on database \"onpr\"\nStarting instance \"ONPR\"...\nORA-01017: invalid username\/password; logon denied\n\n\nPlease complete the following steps to finish switchover:\n        start up and mount instance \"ONPR\" of database \"onpr\"\n\nDGMGRL&gt;<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-root-cause-analyses\">Root Cause analyses<\/h2>\n\n\n\n<p>We can see that the temporary database, created on the ExaCC by ZDM, has got the primary role and is opened READ WRITE.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,4,6,18,22]\">\noracle@ExaCC-cl01n1:\/u02\/app\/oracle\/zdm\/zdm_ONPR_RZ2_70\/zdm\/log\/ [ONPR1 (CDB$ROOT)] ps -ef | grep [p]mon | grep -i onprz\noracle    51392      1  0 14:23 ?        00:00:00 ora_pmon_ONPRZ_APP_001T1\n\noracle@ExaCC-cl01n1:\/u02\/app\/oracle\/zdm\/zdm_ONPR_RZ2_70\/zdm\/log\/ [ONPR1 (CDB$ROOT)] export ORACLE_SID=ONPRZ_APP_001T1\n\noracle@ExaCC-cl01n1:\/u02\/app\/oracle\/zdm\/zdm_ONPR_RZ2_72\/zdm\/log\/ [ONPRZ_APP_001T1 (CDB$ROOT)] sqh\n\nSQL*Plus: Release 19.0.0.0.0 - Production on Fri Mar 22 16:45:32 2024\nVersion 19.22.0.0.0\n\nCopyright (c) 1982, 2023, Oracle.  All rights reserved.\n\n\nConnected to:\nOracle Database 19c EE Extreme Perf Release 19.0.0.0.0 - Production\nVersion 19.22.0.0.0\n\nSQL&gt; select open_mode, database_role from v$database;\n\nOPEN_MODE            DATABASE_ROLE\n-------------------- ----------------\nREAD WRITE           PRIMARY\n<\/pre>\n<\/br>\n\n\n\n<p>But the on-premise database is stopped when it should be started in MOUNT status and have standby role.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1]\">\noracle@vmonpr:\/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/ [ONPR] ONPR\n********* dbi services Ltd. *********\nSTATUS          : STOPPED\n*************************************\n<\/pre>\n<\/br>\n\n\n\n<p>This is due to the fact that ZDM is incorrectly setting the connection, using a local SYS authentication (dgmgrl \/) rather than going through the listener, which is mandatory for a switchover operation. This can be seen in the ZDM logs.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,3,4]\">\noracle@vmonpr:\/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/ [ONPR] grep dgmgrl zdm_switchover_src_24209.log\n[jobid-75][2024-03-22T15:43:19Z][mZDM_Queries.pm:9597]:[DEBUG] Will be running following dgmgrl statements as user: oracle:\n                  \/u00\/app\/oracle\/product\/19.22.0.0.240116.EE\/bin\/dgmgrl \/\n[jobid-75][2024-03-22T15:43:19Z][mZDM_Utils.pm:3450]:[DEBUG] run_as_user2InMem: Running \/u00\/app\/oracle\/product\/19.22.0.0.240116.EE\/bin\/dgmgrl \/\n[jobid-75][2024-03-22T15:43:34Z][mZDM_Utils.pm:3473]:[DEBUG] Remove \/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/zdm_dgmgrl_out_c7pQRGXf\n[jobid-75][2024-03-22T15:43:34Z][mZDM_Utils.pm:3482]:[DEBUG] \/u00\/app\/oracle\/product\/19.22.0.0.240116.EE\/bin\/dgmgrl \/ successfully executed\n[jobid-75][2024-03-22T15:43:34Z][mZDM_Queries.pm:9544]:[DEBUG] Successfully executed dgmgrl script 'switchover to 'onprz_app_001t';'\noracle@vmonpr:\/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/ [ONPR]\n<\/pre>\n<\/br>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-solution\">Solution<\/h2>\n\n\n\n<p>We first start the on-premise database.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,10]\">\noracle@vmonpr:\/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/ [ONPR] sqh\n\nSQL*Plus: Release 19.0.0.0.0 - Production on Fri Mar 22 16:45:51 2024\nVersion 19.22.0.0.0\n\nCopyright (c) 1982, 2023, Oracle.  All rights reserved.\n\nConnected to an idle instance.\n\nSQL&gt; startup mount\nORACLE instance started.\n\nTotal System Global Area 2147479664 bytes\nFixed Size                  8941680 bytes\nVariable Size            1224736768 bytes\nDatabase Buffers          905969664 bytes\nRedo Buffers                7831552 bytes\nDatabase mounted.\n<\/pre>\n<\/br>\n\n\n\n<p>We check Data Guard configuration to ensure the on-premise standby is synchronised with the primary database. We should not have any gap.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,8,12,20,21]\">\noracle@vmonpr:\/u00\/app\/oracle\/zdm\/zdm_ONPR_75\/zdm\/log\/ [ONPR] dgmgrl\nDGMGRL for Linux: Release 19.0.0.0.0 - Production on Fri Mar 22 16:47:29 2024\nVersion 19.22.0.0.0\n\nCopyright (c) 1982, 2019, Oracle and\/or its affiliates.  All rights reserved.\n\nWelcome to DGMGRL, type \"help\" for information.\nDGMGRL&gt; connect \/\nConnected to \"ONPR\"\nConnected as SYSDG.\n\nDGMGRL&gt; show configuration lag\n\nConfiguration - ZDM_onpr\n\n  Protection Mode: MaxPerformance\n  Members:\n  onprz_app_001t - Primary database\n    onpr           - Physical standby database\n                     Transport Lag:      0 seconds (computed 1 second ago)\n                     Apply Lag:          0 seconds (computed 1 second ago)\n\nFast-Start Failover:  Disabled\n\nConfiguration Status:\nSUCCESS   (status updated 16 seconds ago)\n\nDGMGRL&gt;\n<\/pre>\n<\/br>\n\n\n\n<p>Now, we can not just resume ZDM job, because it will retry the last failed phase, which is ZDM_SWITCHOVER_SRC, trying to do the switchover again. And it will fail as on-premise database is not primary any more. The role has been already switched.<\/p>\n\n\n\n<p>This is why, we do not have any other choice than updating the ZDM metadata XML file to change the status of this phase to SUCCESS, knowing we have manually resolved and completed it.<\/p>\n\n\n\n<p>The XML metadata file can be found in the directory <code>GHcheckpoints\/&lt;SOURCE_HOST&gt;+&lt;ORACLE_SID&gt;+&lt;TARGET_HOST&gt;<\/code>. The XML file name is <code>&lt;SOURCE_HOST&gt;+&lt;ORACLE_SID&gt;+&lt;TARGET_HOST&gt;.xml<\/code>.<\/p>\n\n\n\n<p>We need to update the file as following.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,4,6,8]\">\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ pwd\n\/u01\/app\/oracle\/chkbase\/GHcheckpoints\/vmonpr+ONPR+ExaCC-cl01n1\n\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ cp -p vmonpr+ONPR+ExaCC-cl01n1.xml vmonpr+ONPR+ExaCC-cl01n1.xml.20240322_prob_broker\n\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ vi vmonpr+ONPR+ExaCC-cl01n1.xml\n\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ diff vmonpr+ONPR+ExaCC-cl01n1.xml vmonpr+ONPR+ExaCC-cl01n1.xml.20240322_prob_broker\n106c106\n&lt;    &lt;CHECKPOINT LEVEL=\"MAJOR\" NAME=\"ZDM_SWITCHOVER_SRC\" DESC=\"ZDM_SWITCHOVER_SRC\" STATE=\"SUCCESS\"\/&gt;\n---\n&gt;    &lt;CHECKPOINT LEVEL=\"MAJOR\" NAME=\"ZDM_SWITCHOVER_SRC\" DESC=\"ZDM_SWITCHOVER_SRC\" STATE=\"START\"\/&gt;\n<\/pre>\n<\/br>\n\n\n\n<p>We can now resume the job as before the problem.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1]\">\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli resume job -jobid 75 -pauseafter ZDM_SWITCHOVER_TGT\nzdmhost.balgroupit.com: Audit ID: 1167\n<\/pre>\n<\/br>\n\n\n\n<p>And see that it has been completed successfully now, and waiting for the next resume. All new phases have been completed successfully, and the job status is set to PAUSED.<\/p>\n\n\n\n<pre class=\"brush: sql; gutter: true; first-line: 1; highlight: [1,9,35]\">\n[zdmuser@zdmhost vmonpr+ONPR+ExaCC-cl01n1]$ \/u01\/app\/oracle\/product\/zdm\/bin\/zdmcli query job -jobid 75\nzdmhost.balgroupit.com: Audit ID: 1168\nJob ID: 75\nUser: zdmuser\nClient: zdmhost\nJob Type: \"MIGRATE\"\nScheduled job command: \"zdmcli migrate database -sourcesid ONPR -rsp \/home\/zdmuser\/migration\/zdm_ONPR_physical_online.rsp -sourcenode vmonpr -srcauth zdmauth -srcarg1 user:oracle -srcarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -srcarg3 sudo_location:\/usr\/bin\/sudo -targetnode ExaCC-cl01n1 -tgtauth zdmauth -tgtarg1 user:opc -tgtarg2 identity_file:\/home\/zdmuser\/.ssh\/id_rsa -tgtarg3 sudo_location:\/usr\/bin\/sudo -tdekeystorepasswd -tgttdekeystorepasswd -pauseafter ZDM_CONFIGURE_DG_SRC\"\nScheduled job execution start time: 2024-03-22T15:29:56+01. Equivalent local time: 2024-03-22 15:29:56\nCurrent status: PAUSED\nCurrent Phase: \"ZDM_SWITCHOVER_TGT\"\nResult file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.log\"\nMetrics file path: \"\/u01\/app\/oracle\/chkbase\/scheduled\/job-75-2024-03-22-15:30:13.json\"\nJob execution start time: 2024-03-22 15:30:13\nJob execution end time: 2024-03-22 16:53:21\nJob execution elapsed time: 24 minutes 42 seconds\nZDM_GET_SRC_INFO ................ COMPLETED\nZDM_GET_TGT_INFO ................ COMPLETED\nZDM_PRECHECKS_SRC ............... COMPLETED\nZDM_PRECHECKS_TGT ............... COMPLETED\nZDM_SETUP_SRC ................... COMPLETED\nZDM_SETUP_TGT ................... COMPLETED\nZDM_PREUSERACTIONS .............. COMPLETED\nZDM_PREUSERACTIONS_TGT .......... COMPLETED\nZDM_VALIDATE_SRC ................ COMPLETED\nZDM_VALIDATE_TGT ................ COMPLETED\nZDM_DISCOVER_SRC ................ COMPLETED\nZDM_COPYFILES ................... COMPLETED\nZDM_PREPARE_TGT ................. COMPLETED\nZDM_SETUP_TDE_TGT ............... COMPLETED\nZDM_RESTORE_TGT ................. COMPLETED\nZDM_RECOVER_TGT ................. COMPLETED\nZDM_FINALIZE_TGT ................ COMPLETED\nZDM_CONFIGURE_DG_SRC ............ COMPLETED\nZDM_SWITCHOVER_SRC .............. COMPLETED\nZDM_SWITCHOVER_TGT .............. COMPLETED\nZDM_POST_DATABASE_OPEN_TGT ...... PENDING\nZDM_NONCDBTOPDB_PRECHECK ........ PENDING\nZDM_NONCDBTOPDB_CONVERSION ...... PENDING\nZDM_POST_MIGRATE_TGT ............ PENDING\nTIMEZONE_UPGRADE_PREPARE_TGT .... PENDING\nTIMEZONE_UPGRADE_TGT ............ PENDING\nZDM_POSTUSERACTIONS ............. PENDING\nZDM_POSTUSERACTIONS_TGT ......... PENDING\nZDM_CLEANUP_SRC ................. PENDING\nZDM_CLEANUP_TGT ................. PENDING\n\nPause After Phase: \"ZDM_SWITCHOVER_TGT\"\n<\/pre>\n<\/br>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"h-to-wrap-up\">To wrap up<\/h2>\n\n\n\n<p>ZDM has got the flexibility to complete a phase manually and change ZDM metadata in order to move forward. This solution needs to be executed carefully. If you are not full confident, I would recommend you to open a SR. Of course, we can not use such method just to resolve a part of a phase. We will need to complete the phase manually until the end, executing manually all operations. Easy for a phase that is just running a switchover. It might be more complex for another phase.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When using ZDM for a Physical Online Migration, even in the last current version, 21.4.5.0.0, switchover is failing. In this blog, I will share the reason and the solution to solve the problem before moving forward with the migration.<\/p>\n","protected":false},"author":48,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[59],"tags":[280,2598,2596,2562,96,1574,1575],"type_dbi":[],"class_list":["post-32178","post","type-post","status-publish","format-standard","hentry","category-oracle","tag-database","tag-exacc-3","tag-exadata-2","tag-migration-2","tag-oracle","tag-zdm","tag-zero-downtime-migration"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.2) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase\" \/>\n<meta property=\"og:description\" content=\"When using ZDM for a Physical Online Migration, even in the last current version, 21.4.5.0.0, switchover is failing. In this blog, I will share the reason and the solution to solve the problem before moving forward with the migration.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-31T06:50:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-31T06:50:30+00:00\" \/>\n<meta name=\"author\" content=\"Marc Wagner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marc Wagner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\"},\"author\":{\"name\":\"Marc Wagner\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628\"},\"headline\":\"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase\",\"datePublished\":\"2024-03-31T06:50:26+00:00\",\"dateModified\":\"2024-03-31T06:50:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\"},\"wordCount\":589,\"commentCount\":4,\"keywords\":[\"database\",\"exacc\",\"exadata\",\"migration\",\"Oracle\",\"zdm\",\"zero downtime migration\"],\"articleSection\":[\"Oracle\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\",\"name\":\"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\"},\"datePublished\":\"2024-03-31T06:50:26+00:00\",\"dateModified\":\"2024-03-31T06:50:30+00:00\",\"author\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.dbi-services.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#website\",\"url\":\"https:\/\/www.dbi-services.com\/blog\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628\",\"name\":\"Marc Wagner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g\",\"caption\":\"Marc Wagner\"},\"url\":\"https:\/\/www.dbi-services.com\/blog\/author\/marc-wagner\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/","og_locale":"en_US","og_type":"article","og_title":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase","og_description":"When using ZDM for a Physical Online Migration, even in the last current version, 21.4.5.0.0, switchover is failing. In this blog, I will share the reason and the solution to solve the problem before moving forward with the migration.","og_url":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/","og_site_name":"dbi Blog","article_published_time":"2024-03-31T06:50:26+00:00","article_modified_time":"2024-03-31T06:50:30+00:00","author":"Marc Wagner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Marc Wagner","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/"},"author":{"name":"Marc Wagner","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628"},"headline":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase","datePublished":"2024-03-31T06:50:26+00:00","dateModified":"2024-03-31T06:50:30+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/"},"wordCount":589,"commentCount":4,"keywords":["database","exacc","exadata","migration","Oracle","zdm","zero downtime migration"],"articleSection":["Oracle"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/","url":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/","name":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"datePublished":"2024-03-31T06:50:26+00:00","dateModified":"2024-03-31T06:50:30+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/zdm-physical-online-migration-failing-during-zdm_switchover_src-phase\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"ZDM Physical Online Migration failing during ZDM_SWITCHOVER_SRC phase"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/225d9884b8467ead9a872823acb14628","name":"Marc Wagner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a873cc6e7fbdbbcbdbcaf5dbded14ad9a77b2ec2c3e03b4d724ed33d35d5f328?s=96&d=mm&r=g","caption":"Marc Wagner"},"url":"https:\/\/www.dbi-services.com\/blog\/author\/marc-wagner\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/32178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/48"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=32178"}],"version-history":[{"count":18,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/32178\/revisions"}],"predecessor-version":[{"id":32196,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/32178\/revisions\/32196"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=32178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=32178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=32178"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=32178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}