I recently had to move a database from a 19.23 to a 19.26 dbhome on an ExaCC. This will include the database to be patched. The command dbaascli database move failed during the datapatch steps with following error:

DBD::Oracle::st execute failed: ORA-01691: unable to extend lob segment SYS.SYS_LOB0000023009C00008$$ by 1024 in tablespace SYSTEM

With this blog I would like to share my troubleshooting with you and how I resolved the problem.

Moving database to new dbhome

In order to move/patch the database I used dbaascli database move command.

[root@exacc-cl01n1 ~]# dbaascli database move --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_3 --dbname CDBTEST
DBAAS CLI version 25.1.1.0.0
Executing command database move --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_3 --dbname CDBTEST
Job id: 17f6fef5-2143-4bb7-8007-a27cf5232099
Session log: /var/opt/oracle/log/CDBTEST/database/move/dbaastools_2025-05-05_02-47-51-PM_237874.log
Loading PILOT...
Session ID of the current execution is: 15920
Log file location: /var/opt/oracle/log/CDBTEST/database/move/pilot_2025-05-05_02-47-55-PM_238225
-----------------
Running initialization job
Completed initialization job
-----------------
Running validate_user_input job
Completed validate_user_input job
-----------------
Running validate_database job
[WARNING] [DBAAS-70643] Following pluggable databases '{CDBTEST=[PDB_001T]}' do not have services configured.
   ACTION: Make sure to configure the services of pluggable databases so that pluggable databases are started after the database bounce.
Completed validate_database job
-----------------
Running validate_creg_file_existence job
Completed validate_creg_file_existence job
-----------------
Running validate_source_home job
Completed validate_source_home job
-----------------
Running validate_major_version job
Completed validate_major_version job
-----------------
Running validate_oracle_home_type job
Completed validate_oracle_home_type job
-----------------
Running check_target_source_home_not_same job
Completed check_target_source_home_not_same job
-----------------
Running validate_home_existence job
Completed validate_home_existence job
-----------------
Running validate_home_consistency job
Completed validate_home_consistency job
-----------------
Running validate_home_options job
Completed validate_home_options job
-----------------
Running validate_disk_space job
Completed validate_disk_space job
-----------------
Acquiring write lock: cdbtest
Acquiring read lock: _u02_app_oracle_product_19.0.0.0_dbhome_1
Acquiring read lock: _u02_app_oracle_product_19.0.0.0_dbhome_3
Running pre_patch_lock_manager job
Completed pre_patch_lock_manager job
-----------------
Running enable_wallet_root job
Completed enable_wallet_root job
-----------------
Running copy_config_files job
Completed copy_config_files job
-----------------
Running stop_database_instance-exacc-cl01n1 job
Completed stop_database_instance-exacc-cl01n1 job
-----------------
Running update_database_resource-exacc-cl01n1 job
Completed update_database_resource-exacc-cl01n1 job
-----------------
Running start_database_instance-exacc-cl01n1 job
Completed start_database_instance-exacc-cl01n1 job
-----------------
Running exacs_post_patch_node_updation job
Completed exacs_post_patch_node_updation job
-----------------
Running update_dba_directories job
Completed update_dba_directories job
-----------------
Running datapatch_and_recompile_invalid_objects job
Datapatch execution on database 'CDBTEST' is in progress
Execution of datapatch_and_recompile_invalid_objects failed
[FATAL] [DBAAS-60022] Command '/u02/app/oracle/product/19.0.0.0/dbhome_3/OPatch/datapatch -verbose' has failed on nodes [exacc-cl01n1].
*MORE DETAILS*
Result of node:exacc-cl01n1
[Interim patch 28318139 (ORA-31003 ERROR WHEN IMPORTING FULL DATABASE IN PARALLEL):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 06-MAR-22 10.57.36.095921 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 28555193 (DBMS_METADATA.GET_DDL CAPTURE INCORRECT STORAGE OPTIONS OF THE XML COLUMN ON GTT):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 06-MAR-22 10.57.37.111786 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 30944402 (SELECT FROM MASTER TABLE RUNS SLOW DURING TABLE_DATA EXPORT WHEN THERE ARE MANY SUBPARTITIONS):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 19-AUG-23 09.58.11.269998 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 32067171 (OJVM RELEASE UPDATE: 19.10.0.0.210119 (32067171)):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 06-MAR-22 10.57.35.074017 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 33192694 (OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 19-AUG-23 09.58.11.262033 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 33522539 (MERGE ON DATABASE RU 19.13.0.0.0 OF 33280027):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 19-AUG-23 09.58.11.271219 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 34786990 (OJVM RELEASE UPDATE: 19.18.0.0.230117 (34786990)):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 17-AUG-24 08.59.17.242116 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 34972375 (DATAPUMP BUNDLE PATCH 19.18.0.0.0):,   Binary registry: Not installed,   PDB PDB_001T: Rolled back successfully on 17-AUG-24 08.59.19.038902 AM,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 36199232 (OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)):,   Binary registry: Not installed,   PDB PDB_001T: Applied successfully on 17-AUG-24 08.59.17.362926 AM,   PDB CDB$ROOT: Applied successfully on 18-APR-24 10.08.56.234640 AM,   PDB PDB$SEED: Applied successfully on 18-APR-24 10.19.33.039940 AM, Interim patch 36420641 (DATAPUMP BUNDLE PATCH 19.23.0.0.0):,   Binary registry: Not installed,   PDB PDB_001T: Applied successfully on 17-AUG-24 09.00.21.159959 AM,   PDB CDB$ROOT: Applied successfully on 02-OCT-24 11.47.34.300679 AM,   PDB PDB$SEED: Applied successfully on 02-OCT-24 11.50.34.101031 AM, Interim patch 37102264 (OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)):,   Binary registry: Installed,   PDB PDB_001T: Not installed,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Interim patch 37470729 (DATAPUMP BUNDLE PATCH 19.26.0.0.0):,   Binary registry: Installed,   PDB PDB_001T: Not installed,   PDB CDB$ROOT: Not installed,   PDB PDB$SEED: Not installed, Current state of release update SQL patches:,   Binary registry:,     19.26.0.0.0 Release_Update 250118124854: Installed,   PDB PDB_001T:,     Applied 19.23.0.0.0 Release_Update 240406004238 successfully on 17-AUG-24 08.59.50.930598 AM,   PDB CDB$ROOT:,     Applied 19.23.0.0.0 Release_Update 240406004238 successfully on 18-APR-24 10.19.15.621942 AM,   PDB PDB$SEED:,     Applied 19.23.0.0.0 Release_Update 240406004238 successfully on 18-APR-24 10.28.06.019862 AM, Adding patches to installation queue and performing prereq checks...done, Installation queue:,   For the following PDBs: CDB$ROOT PDB$SEED PDB_001T,     The following interim patches will be rolled back:,       36199232 (OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)),       26749785 (PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK),       27605010 (DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999),       36420641 (DATAPUMP BUNDLE PATCH 19.23.0.0.0),     Patch 37260974 (Database Release Update : 19.26.0.0.250121 (37260974)):,       Apply from 19.23.0.0.0 Release_Update 240406004238 to 19.26.0.0.0 Release_Update 250118124854,     The following interim patches will be applied:,       37102264 (OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)),       26749785 (PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK),       27605010 (DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999),       37470729 (DATAPUMP BUNDLE PATCH 19.26.0.0.0), DBD::Oracle::st execute failed: ORA-01691: unable to extend lob segment SYS.SYS_LOB0000023009C00008$$ by 1024 in tablespace SYSTEM, ORA-06512: at line 2 (DBD ERROR: OCIStmtExecute) [for Statement "BEGIN,            INSERT INTO sys.dba_registry_sqlpatch_ru_info,              (patch_id,,               patch_uid,,               patch_descriptor,,               ru_version,,               ru_build_description,,               ru_build_timestamp,,               patch_directory),            VALUES,              (:patch_id,,               :patch_uid,,               :patch_descriptor,,               :ru_version,,               :ru_build_description,,               TO_TIMESTAMP(:ru_build_timestamp, 'YYMMDDHH24MISS'),,               :patch_directory);,            COMMIT;,          END;" with ParamValues: :patch_descriptor=OCIXMLTypePtr=SCALAR(0x4b4dcb0), :patch_directory='PK.........y4Z�&(.�...........37260974_rollback.sql�.]s.ֺ.��+t��ng�.o����v.�?2�����F..�$ . [���[email protected]���dQ.�Tqc��û��z���rv�8.�������.���..����(M�ϳ��´.]%�,u
                                                                                                    �z��.�8���.��.s�f�����,J�Wa.�a6�y�.zp�������,..�?�l���t���..�/fW�(j�}pp�8^��..~w�N.�?�..�.��.(Z���e��.������b���.�������K6E�+�?��A.ln._�.=.�qp����������.������ǫ.qﻋ��.���.�.β �\/x��.M�Q�_�q�ý7.-N�Tory�|���y�&E..y��i��..)�N.�.7��.׌��������oqxz|�|st�X...������G'��.N�_�.?..*�.�..../����.��?|�pWn.�*��;;;.}��.�.�+����..fJ�`., 7+��z�>.?
                                                                                                                                                        �Y�.�.��...�.,...N��9��.��.Y�rp����O��8_���O��%�.E�F�.��t�|��.��.�Q|�L�����y.��[�'�|.���.�.y�ë.�:xy�..pS>��.|U.��r�/j�?=:�\...������.�������ry�����QY�J.�.}N��.�.S%������j�^j.=7T�Z�e.W�z.�?..�>��?�w�.{w9{������/Nj�������f])�.�����.O�*�
��϶!����?�.��&)���I�tzv�x��., �LU.�IV�..7.R7.y��&h{*�3.�mJm}n(, _�...��2�Lݰ�r*ç..�S7ղ@�V��.1>���'�2�.��., ...�9.m)s.�S.ȴ�T��K.�.$!'�..;�Z�.#T�.��(�f.��.�i.pBK��h�fX*ߩ...��i�, �.*d�.�e..6.Oy!*�.:�.MQ4s���.�...�SW��|��...F�OC�,e�..�"/��:5�...', :patch_id='37260974', :patch_uid='26040769', :ru_build_description="Release_Update", :ru_build_timestamp="250118124854", :ru_version="19.26.0.0.0"] at /u02/app/oracle/product/19.0.0.0/dbhome_3/sqlpatch/sqlpatch.pm line 5337., Please refer to MOS Note 1609718.1 and/or the invocation log, /u02/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_258023_2025_05_05_14_50_06/sqlpatch_invocation.log, for information on how to resolve the above errors., SQL Patching tool complete on Mon May  5 14:50:40 2025]
Exit code of the operation:1
Releasing lock: cdbtest
Releasing lock: _u02_app_oracle_product_19.0.0.0_dbhome_1
Releasing lock: _u02_app_oracle_product_19.0.0.0_dbhome_3
*** Executing jobs which need to be run always... ***
-----------------
Running post_patch_lock_manager job
Completed post_patch_lock_manager job
******** PLUGIN EXECUTION FAILED ********
To resume this failed session, run the following command:
dbaascli database move --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_3 --dbname CDBTEST --sessionID 15920 --resume

As we can see the command failed with following error:

[FATAL] [DBAAS-60022] Command '/u02/app/oracle/product/19.0.0.0/dbhome_3/OPatch/datapatch -verbose' has failed on nodes [exacc-cl01n1].
7470729 (DATAPUMP BUNDLE PATCH 19.26.0.0.0), DBD::Oracle::st execute failed: ORA-01691: unable to extend lob segment SYS.SYS_LOB0000023009C00008$$ by 1024 in tablespace SYSTEM

Troubleshooting

So let’s troubleshoot!

The database has been moved to the new dbhome, and the dbaascli command failed on datapatch level.

oracle@exacc-cl01n1:/u02/app/oracle/local/dmk/etc/ [rdbms1900] CDBTEST1
2025-05-05_14-56-12::DMK_ENV.pm::read_oratab            ::INFO ==> changed ORACLE_HOME found in oratab: +ASM1 - old=/u01/app/19.0.0.0/grid, new=/u02/app/23.0.0.0/gridhome_1

 **********************************
 INSTANCE_NAME   : CDBTEST1
 DB_NAME         : CDBTEST
 DB_UNIQUE_NAME  : CDBTEST_CHZ2
 STATUS          : OPEN READ WRITE
 LOG_MODE        : ARCHIVELOG
 USERS/SESSIONS  : 2/7
 DATABASE_ROLE   : PRIMARY
 FLASHBACK_ON    : YES
 FORCE_LOGGING   : YES
 VERSION         : 19.26.0.0.0
 CDB_ENABLED     : YES
 PDBs            : PDB_001T  PDB$SEED
 **********************************

 PDB color: pdbname=mount, pdbname=open read-write, pdbname=open read-only
 Statustime: 2025-05-05 14:56:12

oracle@exacc-cl01n1:/u02/app/oracle/local/dmk/etc/ [CDBTEST1 (CDB$ROOT)] echo $ORACLE_HOME
/u02/app/oracle/product/19.0.0.0/dbhome_3

And the PDB is opened read write.

oracle@exacc-cl01n1:/u02/app/oracle/local/dmk/etc/ [CDBTEST1 (CDB$ROOT)] sqh

SQL*Plus: Release 19.0.0.0.0 - Production on Mon May 5 14:56:31 2025
Version 19.26.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c EE Extreme Perf Release 19.0.0.0.0 - Production
Version 19.26.0.0.0

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 PDB_001T                       READ WRITE NO
SQL>

Let’s try to run datapatch manually, expecting it will fail with same errors. But it might help me to extract more easily datapatch result.

oracle@exacc-cl01n1:/u02/app/oracle/product/19.0.0.0/dbhome_3/ [CDBTEST1 (CDB$ROOT)] cd OPatch/

oracle@exacc-cl01n1:/u02/app/oracle/product/19.0.0.0/dbhome_3/OPatch/ [CDBTEST1 (CDB$ROOT)] ./datapatch -verbose
SQL Patching tool version 19.26.0.0.0 Production on Mon May  5 14:57:00 2025
Copyright (c) 2012, 2025, Oracle.  All rights reserved.

Log file for this invocation: /u02/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_309940_2025_05_05_14_57_00/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
Interim patch 26749785 (PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK):
  Binary registry: Not installed
  PDB PDB_001T: Rolled back successfully on 17-AUG-24 08.59.18.040319 AM
  PDB CDB$ROOT: Not installed
  PDB PDB$SEED: Not installed
Interim patch 26749785 (PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK):
  Binary registry: Not installed
  PDB PDB_001T: Applied successfully on 17-AUG-24 08.59.56.646477 AM
  PDB CDB$ROOT: Applied successfully on 02-OCT-24 11.46.11.005940 AM
  PDB PDB$SEED: Applied successfully on 02-OCT-24 11.47.47.794119 AM
...
...
...
DBD::Oracle::st execute failed: ORA-01691: unable to extend lob segment SYS.SYS_LOB0000023009C00008$$ by 1024 in tablespace SYSTEM
ORA-06512: at line 2 (DBD ERROR: OCIStmtExecute) [for Statement "BEGIN
           INSERT INTO sys.dba_registry_sqlpatch_ru_info
             (patch_id,
              patch_uid,
              patch_descriptor,
              ru_version,
              ru_build_description,
              ru_build_timestamp,
              patch_directory)
           VALUES
             (:patch_id,
              :patch_uid,
              :patch_descriptor,
              :ru_version,
              :ru_build_description,
              TO_TIMESTAMP(:ru_build_timestamp, 'YYMMDDHH24MISS'),
              :patch_directory);
           COMMIT;
         END;" with ParamValues: :patch_descriptor=OCIXMLTypePtr=SCALAR(0x4611c98), :patch_directory='PK.........y4Z▒&(.▒...........37260974_rollback.sql▒.]s.ֺ.▒▒+t▒▒ng▒.o▒▒▒▒v.▒͌?2▒▒▒▒▒F..▒$ . [▒▒▒[email protected]▒▒▒dQ.▒Tqc▒▒û▒▒z▒▒▒rv▒8.▒▒▒▒▒▒▒.▒▒▒..▒▒▒▒(M▒ϳ▒▒´.]%▒,u
                                                                                                                                                                                                                                                              ▒z▒▒.▒8▒▒▒.▒▒.s▒f▒▒▒▒▒,J▒Wa.▒a6▒y▒.zp▒▒▒▒▒▒▒,..▒?▒l▒▒▒t▒▒▒..▒/fW▒(j▒}pp▒8^▒▒..~w▒N.▒?▒..▒.▒▒.(Z▒▒▒e▒▒.▒▒▒▒▒▒b▒▒▒.▒▒▒▒▒▒▒K6E▒+▒?▒▒A.ln._▒.=.▒ܸqp▒▒▒▒▒▒▒▒▒▒.▒▒▒▒▒▒ǫ.qﻋ▒▒.▒▒▒.▒.β ▒\/x▒.M▒Q▒_▒q▒ý7.-N▒Tory▒|▒▒▒y▒&E..y▒▒i▒▒..)▒N.▒.7▒▒.׌▒▒▒▒▒▒▒▒oqxz|▒|st▒X...▒▒▒▒▒▒G'▒▒.N▒_▒.?..*▒.▒..../▒▒▒▒.▒▒?|▒pWn.▒*▒▒;;;.}▒▒.▒.▒+▒▒▒▒..fJ▒`.
7+▒▒z▒>.?
         ▒Y▒.▒.▒▒...▒.,...N▒▒9▒.▒▒.Y▒rp▒▒ⷷ▒▒O▒8_▒▒▒O▒▒%▒.E▒F▒.▒▒t▒|▒▒.▒▒.▒Q|▒L▒▒▒▒y.▒▒[▒'▒|.▒▒▒.▒.y▒ë.▒:xy▒..pS>▒▒.|U.▒▒r▒/j▒?=:▒\...▒▒▒▒▒▒.▒▒▒▒▒▒▒ry▒▒▒▒▒QY▒J.▒.}N▒▒.▒.S%▒▒▒▒▒▒j▒^j.=7T▒Z▒e.W▒z.▒?..▒>▒▒?▒w▒.{w9{▒▒▒▒▒▒/Nj▒▒▒▒▒▒▒f])▒.▒▒▒▒.O▒*▒
▒▒϶!▒▒▒▒?▒.▒▒&)▒▒▒I▒tzv▒x▒▒.
▒LU.▒IV▒..7.R7.y▒▒&h{*▒3.▒mJm}n(
_▒...▒▒2▒Lݰ▒r*ç..▒S7ղ@▒V▒▒.1>▒▒▒'▒2▒.▒▒.
...▒9.m)s.▒S.ȴ▒T▒▒K.▒.$!'▒..;▒Z▒.#T▒.▒▒(▒f.▒▒.▒i.pBK▒▒h▒fX*ߩ...▒▒i▒
▒.*d▒.▒e..6.Oy!*▒.:▒.MQ4s▒▒▒.▒...▒SW▒▒|▒▒...F▒OC▒,e▒..▒"/▒▒:5▒...', :patch_id="37260974", :patch_uid="26040769", :ru_build_description="Release_Update", :ru_build_timestamp="250118124854", :ru_version="19.26.0.0.0"] at /u02/app/oracle/product/19.0.0.0/dbhome_3/sqlpatch/sqlpatch.pm line 5337.


Please refer to MOS Note 1609718.1 and/or the invocation log
/u02/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_309940_2025_05_05_14_57_00/sqlpatch_invocation.log
for information on how to resolve the above errors.

SQL Patching tool complete on Mon May  5 14:57:34 2025

Let’s check SYSTEM tablespace usage in the CDB$ROOT.

oracle@exacc-cl01n1:~/ [CDBTEST1 (CDB$ROOT)] sqh

SQL*Plus: Release 19.0.0.0.0 - Production on Mon May 5 15:02:00 2025
Version 19.26.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c EE Extreme Perf Release 19.0.0.0.0 - Production
Version 19.26.0.0.0

SQL> @qdbstbssize.sql

PL/SQL procedure successfully completed.


                             Nb      Extent Segment    Alloc.      Space        Max. Percent Block
Name                      files Type Mgmnt  Mgmnt    Size (GB)  Free (GB)  Size (GB)  used % size  Log Encrypt Compress
------------------------- ----- ---- ------ ------- ---------- ---------- ---------- ------- ----- --- ------- --------
SYSAUX                        1 DATA LM-SYS AUTO          5.95       4.47   32768.00     .00 8 KB  YES YES     NO
SYSTEM                        1 DATA LM-SYS MANUAL        1.95        .62   32768.00     .00 8 KB  YES YES     NO
TEMP                          1 TEMP LM-UNI MANUAL        1.00       2.31     512.00    -.26 8 KB  NO  YES     NO
UNDOTBS1                      1 UNDO LM-SYS MANUAL        1.95       1.89     512.00     .01 8 KB  YES YES     NO
USERS                         1 DATA LM-SYS AUTO          1.00        .93   32768.00     .00 8 KB  YES YES     NO
                          -----                     ---------- ---------- ----------
TOTAL                         5                          11.85      10.22   99328.00

SQL>

We can see all is ok. Let’s check the same in the PDB.

SQL> alter session set container=PDB_001T;

Session altered.

SQL> @qdbstbssize.sql
...
...
...
SYSAUX                        1 DATA         LM-SYS AUTO         82.62      74.52   32768.00     .02 8 KB  YES YES     NO
SYSTEM                        1 DATA         LM-SYS MANUAL       24.71        .10   32768.00     .08 8 KB  YES YES     NO
TEMP                          1 TEMP         LM-UNI MANUAL      114.26     342.75   32768.00    -.70 8 KB  NO  YES     NO
UNDO                          1 UNDO         LM-SYS MANUAL      684.57     673.58   32768.00     .03 8 KB  YES YES     NO
USERS                         1 DATA         LM-SYS AUTO         20.00      19.93      20.00     .33 8 KB  YES YES     NO
                          -----                             ---------- ---------- ----------
TOTAL                      2732                                7101.04    1385.79 ##########

2732 rows selected.

SQL>

It’s all good as well. And the pdb$seed?

SQL> alter session set container=PDB$SEED;

Session altered.

SQL> @qdbstbssize.sql

PL/SQL procedure successfully completed.


                             Nb      Extent Segment    Alloc.      Space        Max. Percent Block
Name                      files Type Mgmnt  Mgmnt    Size (GB)  Free (GB)  Size (GB)  used % size  Log Encrypt Compress
------------------------- ----- ---- ------ ------- ---------- ---------- ---------- ------- ----- --- ------- --------
SYSAUX                        1 DATA LM-SYS AUTO           .59        .08   32768.00     .00 8 KB  YES NO      NO
SYSTEM                        1 DATA LM-SYS MANUAL         .59        .00   32768.00     .00 8 KB  YES NO      NO
TEMP                          1 TEMP LM-UNI MANUAL         .18        .00   32768.00     .00 8 KB  NO  NO      NO
UNDOTBS1                      1 UNDO LM-SYS MANUAL         .59        .33     512.00     .05 8 KB  YES NO      NO
                          -----                     ---------- ---------- ----------
TOTAL                         4                           1.95        .41   98816.00

SQL>

All is ok as well.

Let’s check where this SYS_LOB belongs to. cdb$root? PDB? pdb$seed?

SQL> alter session set container=PDB$SEED;

Session altered.

SQL> select tablespace_name, table_name from dba_lobs where segment_name='SYS_LOB0000023009C00008$$' and owner='SYS';

TABLESPACE_NAME                TABLE_NAME
------------------------------ --------------------------------------------------------------------------------------------------------------------------------
SYSTEM                         REGISTRY$SQLPATCH_RU_INFO

SQL> alter session set container=cdb$root;

Session altered.

SQL> select tablespace_name, table_name from dba_lobs where segment_name='SYS_LOB0000023009C00008$$' and owner='SYS';

no rows selected

SQL> alter session set container=PDB_001T;

Session altered.

SQL> select tablespace_name, table_name from dba_lobs where segment_name='SYS_LOB0000023009C00008$$' and owner='SYS';

no rows selected

SQL>

So the LOB belongs to the pdb$seed.

From the alert log I can see:

PDB$SEED(2):Pluggable database PDB$SEED opening in read write
PDB$SEED(2):Autotune of undo retention is turned on.
PDB$SEED(2):This instance was first to open pluggable database PDB$SEED (container=2)
PDB$SEED(2):queued attach DA request 0xb2325ed8 for pdb 2, ospid 3457
2025-05-05T15:11:01.061406+02:00
Domain Action Reconfiguration started (domid 2, new da inc 19, cluster inc 2)
Instance 1 is attaching to domain 2
 Global Resource Directory partially frozen for domain action
Domain Action Reconfiguration complete (total time 0.0 secs)
2025-05-05T15:11:01.067485+02:00
PDB$SEED(2):Endian type of dictionary set to little
PDB$SEED(2):Undo initialization recovery: Parallel FPTR failed: start:1139322373 end:1139322379 diff:6 ms (0.0 seconds)
PDB$SEED(2):Undo initialization recovery: err:0 start: 1139322373 end: 1139322392 diff: 19 ms (0.0 seconds)
PDB$SEED(2):[3457] Successfully onlined Undo Tablespace 2.
PDB$SEED(2):Undo initialization online undo segments: err:0 start: 1139322392 end: 1139322638 diff: 246 ms (0.2 seconds)
PDB$SEED(2):Undo initialization finished serial:0 start:1139322373 end:1139322643 diff:270 ms (0.3 seconds)
PDB$SEED(2):Database Characterset for PDB$SEED is AL32UTF8
PDB$SEED(2):SUPLOG: Set PDB SUPLOG SGA at PDB OPEN, old 0x0, new 0x0 (no suplog)
PDB$SEED(2):Opening pdb with no Resource Manager plan active
2025-05-05T15:11:01.806446+02:00
PDB$SEED(2):joxcsys_required_dirobj_exists: directory object exists with required path /u02/app/oracle/product/19.0.0.0/dbhome_3/javavm/admin/, pid 3457 cid 2
Pluggable database PDB$SEED opened read write
2025-05-05T15:11:03.081311+02:00
QPI: opatch file present, opatch
QPI: qopiprep.bat file present
QPI: Cleaning and refreshing metadata..
2025-05-05T15:11:09.187099+02:00
QPI: Cleaning and refreshing metadata..
2025-05-05T15:11:23.765174+02:00
PDB$SEED(2):ORA-1691: unable to extend lobsegment SYS.SYS_LOB0000023009C00008$$ by 1024 in tablespace SYSTEM [PDB$SEED] (ospid 3221)
2025-05-05T15:11:24.482146+02:00
PDB$SEED(2):Pluggable database PDB$SEED closing

So we can see that once the pdb$seed is opened in read write mode, datapatch is failing due to the fact it is not possible to extend the SYS.SYS_LOB0000023009C00008$$ lob segment.

Let’s look into the ASM to see if there is enough disk space.

ASMCMD> lsdg
State    Type  Rebal  Sector  Logical_Sector  Block       AU   Total_MB    Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  HIGH  N         512             512   4096  4194304   49072128      56708                0           18902              0             Y  DATAC4/
MOUNTED  HIGH  N         512             512   4096  4194304   16356864   13063776                0         4354592              0             N  RECOC4/
MOUNTED  HIGH  N         512             512   4096  4194304  163568640  163563936                0        54521312              0             N  SPRC4/
ASMCMD>

There is not a lot of usable space for the DATA Disk Group, certainly not enough, but still some space. Let’s try to open the pdb$seed in read write and to create a table.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO

SQL> shutdown immediate
Pluggable Database closed.

SQL> alter pluggable database open read write;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ WRITE NO

SQL> create table xxx (n number) tablespace system;

Table created.

SQL> insert into xxx values (10);

1 row created.

SQL> commit;

Commit complete.

SQL> drop table xxx purge;

Table dropped.

This is possible. Let’s try to resize system data file.

SQL> alter database datafile 2 resize 2048M;
alter database datafile 2 resize 2048M
*
ERROR at line 1:
ORA-01237: cannot extend datafile 2
ORA-01110: data file 2: '+DATAC4/CDBTEST_CHZ2/165C22D4CFFCA759E0638A534664B46C/DATAFILE/system.615.1181302705'
ORA-17505: ksfdrsz:1 Failed to resize file to size 262144 blocks
ORA-15041: diskgroup "DATAC4" space exhausted

This is not possible and it is confirmed DATA Disk Group does not have enough available space.

I first will reopen the pdb$seed in read only.

SQL> shutdown immediate
Pluggable Database closed.

SQL> alter pluggable database open read only;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
SQL>

Following DocId would explain the problem. One or more disk having 0 free MB:

Datapatch failed with ORA-01691: unable to extend lob segment on MGMTDB (Doc ID 2352895.1)

Let’s check, connecting to the +ASM instance.

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
+ASM1

SQL> select group_number, name from v$asm_diskgroup;

GROUP_NUMBER NAME
------------ ------------------------------
           1 DATAC4
           2 RECOC4
           3 SPRC4

SQL> select DISK_NUMBER, FREE_MB from v$asm_disk where group_number=1 and FREE_MB=0;

DISK_NUMBER    FREE_MB
----------- ----------
         48          0

SQL> select count(*) from v$asm_disk where group_number=1 and FREE_MB  0;

  COUNT(*)
----------
        95

SQL>

Yes, this is exactly the problem! Disk Number 48 has got 0 free MB.

Resolution

Let’s resolve the problem.

Through the OCI console, I added 1 TB to the ASM. The usable space of the disk groups then became as following:

ASMCMD> lsdg
State    Type  Rebal  Sector  Logical_Sector  Block       AU   Total_MB    Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  HIGH  Y         512             512   4096  4194304   50959872    1932164                0          644054              0             Y  DATAC4/
MOUNTED  HIGH  N         512             512   4096  4194304   16356864   13058556                0         4352852              0             N  RECOC4/
MOUNTED  HIGH  N         512             512   4096  4194304  163568640  163563936                0        54521312              0             N  SPRC4/
ASMCMD>

I checked to ensure there is no more disks with 0 free MB.

[grid@exacc-cl01n1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems on Mon May 5 16:28:11 2025
Version 23.7.0.25.01

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 23ai Enterprise Edition Release 23.0.0.0.0 - for Oracle Cloud and Engineered Systems
Version 23.7.0.25.01

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
+ASM1

SQL> select DISK_NUMBER, FREE_MB from v$asm_disk where group_number=1 and FREE_MB=0;

no rows selected

SQL> select count(*) from v$asm_disk where group_number=1 and FREE_MB  0;

  COUNT(*)
----------
        96

SQL>

I resume the dbaascli move operation, which became successful.

[root@exacc-cl01n1 ~]# dbaascli database move --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_3 --dbname CDBTEST --sessionID 15920 --resume
DBAAS CLI version 25.1.1.0.0
Executing command database move --oracleHome /u02/app/oracle/product/19.0.0.0/dbhome_3 --dbname CDBTEST --sessionID 15920 --resume
Job id: 7e59d18a-ca1b-4808-a927-c3ce9131b968
Session log: /var/opt/oracle/log/CDBTEST/database/move/dbaastools_2025-05-05_05-52-50-PM_350983.log
Loading PILOT...
Session ID of the current execution is: 15932
Log file location: /var/opt/oracle/log/CDBTEST/database/move/pilot_2025-05-05_05-52-54-PM_351354
-----------------
Running initialization job
Completed initialization job
-----------------
Acquiring write lock: cdbtest
Acquiring read lock: _u02_app_oracle_product_19.0.0.0_dbhome_1
Acquiring read lock: _u02_app_oracle_product_19.0.0.0_dbhome_3
Running datapatch_and_recompile_invalid_objects job
Datapatch execution on database 'CDBTEST' is in progress
Datapatch execution on database 'CDBTEST' is complete
Recompilation of invalid objects on database 'CDBTEST' is in progress
Recompilation of invalid objects on database 'CDBTEST' is complete
Completed datapatch_and_recompile_invalid_objects job
-----------------
Running update_pdb_status job
Completed update_pdb_status job
-----------------
Running post_patch_lock_manager job
Completed post_patch_lock_manager job
Releasing lock: cdbtest
Releasing lock: _u02_app_oracle_product_19.0.0.0_dbhome_1
Releasing lock: _u02_app_oracle_product_19.0.0.0_dbhome_3
-----------------
Running invoke_backup_asst job
Completed invoke_backup_asst job
-----------------
Running post_move_validation job
Completed post_move_validation job
-----------------
Running generate_dbsystem_details job
Acquiring native write lock: global_dbsystem_details_generation
Releasing native lock: global_dbsystem_details_generation
Completed generate_dbsystem_details job

dbaascli execution completed
You have new mail in /var/spool/mail/root

And I checked database patch version for the cdb$root and the pdb.

oracle@exacc-cl01n1:~/ [rdbms1900] CDBTEST1
2025-05-05_18-01-21::DMK_ENV.pm::read_oratab            ::INFO ==> changed ORACLE_HOME found in oratab: +ASM1 - old=/u01/app/19.0.0.0/grid, new=/u02/app/23.0.0.0/gridhome_1

 **********************************
 INSTANCE_NAME   : CDBTEST1
 DB_NAME         : CDBTEST
 DB_UNIQUE_NAME  : CDBTEST_CHZ2
 STATUS          : OPEN READ WRITE
 LOG_MODE        : ARCHIVELOG
 USERS/SESSIONS  : 2/7
 DATABASE_ROLE   : PRIMARY
 FLASHBACK_ON    : YES
 FORCE_LOGGING   : YES
 VERSION         : 19.26.0.0.0
 CDB_ENABLED     : YES
 PDBs            : PDB_001T  PDB$SEED
 **********************************

 PDB color: pdbname=mount, pdbname=open read-write, pdbname=open read-only
 Statustime: 2025-05-05 18:01:22

oracle@exacc-cl01n1:~/ [CDBTEST1 (CDB$ROOT)] sqh

SQL*Plus: Release 19.0.0.0.0 - Production on Mon May 5 18:01:25 2025
Version 19.26.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c EE Extreme Perf Release 19.0.0.0.0 - Production
Version 19.26.0.0.0

SQL> set lines 300
SQL> set tab off
SQL> set pages 500

SQL> alter session set nls_date_format='DD/MM/YYYY HH24:MI:SS';

Session altered.

SQL> col action_time for a30
SQL> select action_time, patch_id, source_version, target_version, status, description from dba_registry_sqlpatch;

ACTION_TIME                      PATCH_ID SOURCE_VERSION  TARGET_VERSION  STATUS                    DESCRIPTION
------------------------------ ---------- --------------- --------------- ------------------------- ----------------------------------------------------------------------------------------------------
18-APR-24 10.08.56.234640 AM     36199232 19.1.0.0.0      19.23.0.0.0     SUCCESS                   OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)
18-APR-24 10.19.15.621942 AM     36233263 19.1.0.0.0      19.23.0.0.0     SUCCESS                   Database Release Update : 19.23.0.0.240416 (36233263)
02-OCT-24 11.46.11.005940 AM     26749785 19.23.0.0.0     19.23.0.0.0     SUCCESS                   PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
02-OCT-24 11.46.13.343508 AM     27605010 19.23.0.0.0     19.23.0.0.0     SUCCESS                   DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
02-OCT-24 11.47.34.300679 AM     36420641 19.23.0.0.0     19.23.0.0.0     SUCCESS                   DATAPUMP BUNDLE PATCH 19.23.0.0.0
05-MAY-25 05.27.25.226657 PM     36199232 19.23.0.0.0     19.26.0.0.0     SUCCESS                   OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)
05-MAY-25 05.27.26.115643 PM     37102264 19.23.0.0.0     19.26.0.0.0     SUCCESS                   OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)
05-MAY-25 05.27.26.697687 PM     26749785 19.23.0.0.0     19.26.0.0.0     SUCCESS                   PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
05-MAY-25 05.27.28.734483 PM     27605010 19.23.0.0.0     19.26.0.0.0     SUCCESS                   DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
05-MAY-25 05.27.28.839820 PM     36420641 19.23.0.0.0     19.26.0.0.0     SUCCESS                   DATAPUMP BUNDLE PATCH 19.23.0.0.0
05-MAY-25 05.28.35.890001 PM     37260974 19.23.0.0.0     19.26.0.0.0     SUCCESS                   Database Release Update : 19.26.0.0.250121 (37260974)
05-MAY-25 05.28.44.283893 PM     26749785 19.23.0.0.0     19.26.0.0.0     SUCCESS                   PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
05-MAY-25 05.28.46.272241 PM     27605010 19.23.0.0.0     19.26.0.0.0     SUCCESS                   DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
05-MAY-25 05.29.22.101438 PM     37470729 19.23.0.0.0     19.26.0.0.0     SUCCESS                   DATAPUMP BUNDLE PATCH 19.26.0.0.0

14 rows selected.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 PDB_001T                 READ WRITE NO

SQL> alter session set container=PDB_001T;

Session altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         4 PDB_001T                 READ WRITE NO

SQL> select action_time, patch_id, source_version, target_version, status, description from dba_registry_sqlpatch;

ACTION_TIME                      PATCH_ID SOURCE_VERSI TARGET_VERSI STATUS               DESCRIPTION
------------------------------ ---------- ------------ ------------ -------------------- ----------------------------------------------------------------------------------------------------
19-JUN-21 09.56.16.648813 AM     32067171 19.1.0.0.0   19.1.0.0.0   SUCCESS              OJVM RELEASE UPDATE: 19.10.0.0.210119 (32067171)
19-JUN-21 09.56.16.641919 AM     32218454 19.1.0.0.0   19.10.0.0.0  SUCCESS              Database Release Update : 19.10.0.0.210119 (32218454)
19-JUN-21 09.56.17.529234 AM     28318139 19.1.0.0.0   19.1.0.0.0   SUCCESS              ORA-31003 ERROR WHEN IMPORTING FULL DATABASE IN PARALLEL
19-JUN-21 09.56.18.407598 AM     28555193 19.1.0.0.0   19.1.0.0.0   WITH ERRORS          DBMS_METADATA.GET_DDL CAPTURE INCORRECT STORAGE OPTIONS OF THE XML COLUMN ON GTT
19-JUN-21 10.13.13.165620 AM     28555193 19.10.0.0.0  19.10.0.0.0  SUCCESS              DBMS_METADATA.GET_DDL CAPTURE INCORRECT STORAGE OPTIONS OF THE XML COLUMN ON GTT
06-MAR-22 10.57.35.074017 AM     32067171 19.13.0.0.0  19.13.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.10.0.0.210119 (32067171)
06-MAR-22 10.57.39.351946 AM     33192694 19.10.0.0.0  19.10.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)
06-MAR-22 10.57.36.095921 AM     28318139 19.13.0.0.0  19.13.0.0.0  SUCCESS              ORA-31003 ERROR WHEN IMPORTING FULL DATABASE IN PARALLEL
06-MAR-22 10.57.37.111786 AM     28555193 19.13.0.0.0  19.13.0.0.0  SUCCESS              DBMS_METADATA.GET_DDL CAPTURE INCORRECT STORAGE OPTIONS OF THE XML COLUMN ON GTT
06-MAR-22 10.57.39.348199 AM     33192793 19.10.0.0.0  19.13.0.0.0  SUCCESS              Database Release Update : 19.13.0.0.211019 (33192793)
06-MAR-22 10.57.40.368792 AM     30944402 19.10.0.0.0  19.10.0.0.0  SUCCESS              SELECT FROM MASTER TABLE RUNS SLOW DURING TABLE_DATA EXPORT WHEN THERE ARE MANY SUBPARTITIONS
06-MAR-22 10.57.41.384551 AM     33522539 19.10.0.0.0  19.10.0.0.0  SUCCESS              MERGE ON DATABASE RU 19.13.0.0.0 OF 33280027
19-AUG-23 09.58.11.262033 AM     33192694 19.18.0.0.0  19.18.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)
19-AUG-23 09.58.13.172768 AM     34786990 19.13.0.0.0  19.13.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.18.0.0.230117 (34786990)
19-AUG-23 09.58.11.269998 AM     30944402 19.18.0.0.0  19.18.0.0.0  SUCCESS              SELECT FROM MASTER TABLE RUNS SLOW DURING TABLE_DATA EXPORT WHEN THERE ARE MANY SUBPARTITIONS
19-AUG-23 09.58.11.271219 AM     33522539 19.18.0.0.0  19.18.0.0.0  SUCCESS              MERGE ON DATABASE RU 19.13.0.0.0 OF 33280027
19-AUG-23 09.58.13.169832 AM     34765931 19.13.0.0.0  19.18.0.0.0  SUCCESS              DATABASE RELEASE UPDATE : 19.18.0.0.230117 (REL-JAN230131) (34765931)
19-AUG-23 09.58.13.340768 AM     26749785 19.13.0.0.0  19.13.0.0.0  SUCCESS              PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
19-AUG-23 09.58.13.346063 AM     27605010 19.13.0.0.0  19.13.0.0.0  SUCCESS              DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
19-AUG-23 09.58.14.253669 AM     34972375 19.13.0.0.0  19.13.0.0.0  SUCCESS              DATAPUMP BUNDLE PATCH 19.18.0.0.0
17-AUG-24 08.59.17.242116 AM     34786990 19.18.0.0.0  19.23.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.18.0.0.230117 (34786990)
17-AUG-24 08.59.17.362926 AM     36199232 19.18.0.0.0  19.23.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)
17-AUG-24 08.59.18.040319 AM     26749785 19.18.0.0.0  19.23.0.0.0  SUCCESS              PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
17-AUG-24 08.59.18.971587 AM     27605010 19.18.0.0.0  19.23.0.0.0  SUCCESS              DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
17-AUG-24 08.59.19.038902 AM     34972375 19.18.0.0.0  19.23.0.0.0  SUCCESS              DATAPUMP BUNDLE PATCH 19.18.0.0.0
17-AUG-24 08.59.50.930598 AM     36233263 19.18.0.0.0  19.23.0.0.0  SUCCESS              Database Release Update : 19.23.0.0.240416 (36233263)
17-AUG-24 08.59.56.646477 AM     26749785 19.18.0.0.0  19.23.0.0.0  SUCCESS              PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
17-AUG-24 08.59.57.504574 AM     27605010 19.18.0.0.0  19.23.0.0.0  SUCCESS              DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
17-AUG-24 09.00.21.159959 AM     36420641 19.18.0.0.0  19.23.0.0.0  SUCCESS              DATAPUMP BUNDLE PATCH 19.23.0.0.0
05-MAY-25 05.31.01.733241 PM     36199232 19.23.0.0.0  19.26.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.23.0.0.240416 (36199232)
05-MAY-25 05.31.03.413696 PM     37102264 19.23.0.0.0  19.26.0.0.0  SUCCESS              OJVM RELEASE UPDATE: 19.26.0.0.250121 (37102264)
05-MAY-25 05.31.04.821299 PM     26749785 19.23.0.0.0  19.26.0.0.0  SUCCESS              PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
05-MAY-25 05.31.06.205197 PM     27605010 19.23.0.0.0  19.26.0.0.0  SUCCESS              DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
05-MAY-25 05.31.06.379357 PM     36420641 19.23.0.0.0  19.26.0.0.0  SUCCESS              DATAPUMP BUNDLE PATCH 19.23.0.0.0
05-MAY-25 05.32.25.947558 PM     37260974 19.23.0.0.0  19.26.0.0.0  SUCCESS              Database Release Update : 19.26.0.0.250121 (37260974)
05-MAY-25 05.32.32.616612 PM     26749785 19.23.0.0.0  19.26.0.0.0  SUCCESS              PERF_DIAG  NEED TO HAVE MORE CONTROL IN DICTIONARY FOR AUTO_STATS_ADVISOR_TASK
05-MAY-25 05.32.33.746247 PM     27605010 19.23.0.0.0  19.26.0.0.0  SUCCESS              DST UPGRADE SLOW (BEGIN_UPGRADE AND UPGRADE_DATABASE) EVEN WITH PATCH 19704999
05-MAY-25 05.35.48.324333 PM     37470729 19.23.0.0.0  19.26.0.0.0  SUCCESS              DATAPUMP BUNDLE PATCH 19.26.0.0.0

38 rows selected.

And finally I checked PDB violations.

oracle@exacc-cl01n1:/u02/app/oracle/product/19.0.0.0/dbhome_3/OPatch/ [CDBTEST1 (CDB$ROOT)] sqh

SQL*Plus: Release 19.0.0.0.0 - Production on Mon May 5 18:10:56 2025
Version 19.26.0.0.0

Copyright (c) 1982, 2024, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c EE Extreme Perf Release 19.0.0.0.0 - Production
Version 19.26.0.0.0

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         4 PDB_001T                 READ WRITE NO

SQL> alter session set container=PDB_001T;

Session altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         4 PDB_001T                 READ WRITE NO

SQL> col name for a15
SQL> col message for a120
SQL> col cause for a50
SQL> set lines 300
SQL> col cause for a20
SQL> col type for a10

SQL> select name,cause,type,message,status from PDB_PLUG_IN_VIOLATIONS where status  'RESOLVED';

NAME            CAUSE                TYPE       MESSAGE                                                                                                                  STATUS
--------------- -------------------- ---------- ------------------------------------------------------------------------------------------------------------------------ ---------------------------
PDB_001T  OPTION               WARNING    Database option APS mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                              PENDING
PDB_001T  OPTION               WARNING    Database option CONTEXT mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                          PENDING
PDB_001T  OPTION               WARNING    Database option DV mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                               PENDING
PDB_001T  OPTION               WARNING    Database option OLS mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                              PENDING
PDB_001T  OPTION               WARNING    Database option ORDIM mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                            PENDING
PDB_001T  OPTION               WARNING    Database option OWM mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                              PENDING
PDB_001T  OPTION               WARNING    Database option SDO mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                              PENDING
PDB_001T  OPTION               WARNING    Database option XOQ mismatch: PDB installed version NULL. CDB installed version 19.0.0.0.0.                              PENDING

8 rows selected.

SQL>

Yes, that’s it, all is ok now.

To wrap up…

The database move on another higher version dbhome on the ExaCC failed on the datapatch step, due to the fact one of the Disk had 0 free MB. Adding space to the ASM resolved the issue.

Post Views: 62
Thumbnail [60x60]
by
Marc Wagner