Archives des oda - dbi Blog https://www.dbi-services.com/blog/tag/oda-2/ Thu, 18 Jun 2026 13:09:40 +0000 en-US hourly 1 https://www.dbi-services.com/blog/wp-content/uploads/sites/2/2025/05/cropped-favicon_512x512px-min-32x32.png Archives des oda - dbi Blog https://www.dbi-services.com/blog/tag/oda-2/ 32 32 Temporary tablespace and ODA 19.31. https://www.dbi-services.com/blog/temporary-tablespace-and-oda-19-31/ https://www.dbi-services.com/blog/temporary-tablespace-and-oda-19-31/#respond Thu, 11 Jun 2026 06:55:15 +0000 https://www.dbi-services.com/blog/?p=45038 I recently patched Oracle Database Appliances (ODAs) to 19.31. For DBs with more than 1 tempfile in the temporary tablespace I got an error during the “odacli update-dbhome” : The issue is that Oracle tries to “set autoextend on” on the temporary files in 19.31., but runs a syntactically wrong command. E.g. in the alert.log […]

L’article Temporary tablespace and ODA 19.31. est apparu en premier sur dbi Blog.

]]> I recently patched Oracle Database Appliances (ODAs) to 19.31. For DBs with more than 1 tempfile in the temporary tablespace I got an error during the “odacli update-dbhome” :

# odacli describe-job -i 2ff3fd5d-f611-4ae6-aae0-63df7c08ec71
 
Job details
----------------------------------------------------------------
                     ID:  2ff3fd5d-f611-4ae6-aae0-63df7c08ec71
            Description:  DB Home Patching to 19.31.0.0.0: Home ID is 1c101fd3-2ca8-55f6-9415-22ec5abc53da
                 Status:  Failure (To view Error Correlation report, run "odacli describe-job -i 2ff3fd5d-f611-4ae6-aae0-63df7c08ec71 --ecr" command)
                Created:  June 09, 2026 15:07:23 CEST
                Message:  DCS-10001:Internal error encountered: Unable to autoextend tablespace TEMP.

The issue is that Oracle tries to “set autoextend on” on the temporary files in 19.31., but runs a syntactically wrong command. E.g. in the alert.log I could see

2026-06-09T15:54:14.898088+02:00
alter database tempfile '/u02/app/oracle/oradata/mydbt_dc2/MYDBT_DC2/datafile/o1_mf_temp_m2d4kz32_.tmp /u02/app/oracle/oradata/mydbt_dc2/MYDBT_DC2/datafile/o1_mf_temp_m2d4l3wf_.tmp' autoextend on next 100m maxsize 20g
ORA-1516 signalled during:  alter database tempfile '/u02/app/oracle/oradata/mydbt_dc2/MYDBT_DC2/datafile/o1_mf_temp_m2d4kz32_.tmp /u02/app/oracle/oradata/mydbt_dc2/MYDBT_DC2/datafile/o1_mf_temp_m2d4l3wf_.tmp' autoextend on next 100m maxsize 20g ...

I.e. Oracle just listed all tempfiles separated by blank in one string. This is of course wrong and produces the error.

The workaround was to 

 – create a temporary tablespace temp2

 – make temp2 the default temporary tablespace

 – drop temporary tablespace temp

 – create the temporary tablespace temp with just 1 tempfile

 – make temp the default temporary tablespace

 – drop temporary tablespace temp2

Then run the “odacli update-dbhome” again and it will be successful. Afterwards add your temporary files to tablespace temp again. 

Please consider that it is not obvious in the produced errors of odacli in what DB the error happened (odacli update-dbhome patches all DBs running in the specified ORACLE_HOME). To find out what DB is affected it’s a good idea to scan the alert-logs for the error:

$ find /u01/app/odaorabase/oracle/diag/rdbms/*/*/trace -name "alert*.log" -exec grep -il "ORA-1516" {}\;

To proactively avoid the issue when patching to 19.31. on the ODA, I’d recommend to check your databases in advance if they have more than 1 tempfile in the temporary tablespace and implement the workaround mentioned above.

select count(*) from dba_temp_files where tablespace_name='TEMP';

We opened a SR with Oracle to get the issue fixed for the next release.

Update 18.06.2026:

If you do use a Standby DB (Data Guard) and use the workaround mentioned above, then please consider that you have to add the tempfiles at the standby site as well after recreating TEMP. See here, here and here.

L’article Temporary tablespace and ODA 19.31. est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/temporary-tablespace-and-oda-19-31/feed/ 0 Not enough space for using ODABR? Reduce /u01 and /opt size on your ODA https://www.dbi-services.com/blog/not-enough-space-for-using-odabr-reduce-u01-and-opt-size-on-your-oda/ https://www.dbi-services.com/blog/not-enough-space-for-using-odabr-reduce-u01-and-opt-size-on-your-oda/#respond Fri, 24 Apr 2026 06:42:11 +0000 https://www.dbi-services.com/blog/?p=44065 Introduction The patching of an Oracle Database Appliance needs to be secured. This is because it won’t limit to patching the databases: there is plenty of other components embedded with the patch, the goal being to keep everything updated. A possible rollback may be needed. For sure, there is no possible rollback for firmwares, BIOS […]

L’article Not enough space for using ODABR? Reduce /u01 and /opt size on your ODA est apparu en premier sur dbi Blog.

]]> Introduction

The patching of an Oracle Database Appliance needs to be secured. This is because it won’t limit to patching the databases: there is plenty of other components embedded with the patch, the goal being to keep everything updated. A possible rollback may be needed.

For sure, there is no possible rollback for firmwares, BIOS and ILOM (it’s unlikely you would need to go back to previous version), but you may need to rollback when it comes to patching the OS and the GI stack. Unlike DB homes, OS and GI stack are located on the ODA local disks, and can be protected during patching with LVM snapshots. This is the purpose of ODABR: making LVM snapshots of /, /u01 and /opt before applying the patch. If something goes wrong, you can revert to the previous stable state.

ODABR will need enough space on local disks for creating the snapshots, and sometimes available free space is not enough. How to deal with that?

When taking the snapshots?

Before using ODABR, I would recommend doing a cleanup of the following filesystems: /, /u01 and /opt. You will need 20+% of free space in these filesystems, otherwise the patch prechecks won’t give you the green light. ODABR is based on Copy-On-Write technology, meaning that old versions of the changed blocks will be kept for going back in time if needed. Don’t unzip the patch files after taking the snapshots if the patch files reside on the local disks! Once you’re ready to patch, the minute before registering the patch and applying the DCS components update, you can use ODABR to take the snapshots.

When releasing the snapshots?

Once you successfully applied the system patch and GI patch, and once you verified that everything runs fine with the new GI binaries, you can safely use ODABR to delete the snapshots. If you need to apply multiple patches, make new snapshots before each jump, you will never need to revert to the oldest version once you are in a stable intermediate version.

Are the ODABR snapshots useful for DB homes?

Absolutely not. DB homes are located under /u01/app/odaorahome, a dedicated ACFS filesystem on the DATA diskgroup. You don’t need snapshot features on this very filesystem because patching a DB home will create a new home and move the database into this new DB home, the old one being kept as long as you want to. You can delete older DB homes months after patching was done. I usually delete old ones as part of the patching job once everything is fine, but I always backup the old DB home with a tar czf in case of. If you need to revert a database to an old DB home, you can do it manually and use a RMAN backup to restore the database, or eventually rollback the datapatch if needed. This is rarely used.

Location of the snapshots

ODABR relies on LVM snapshots, not something really specific to ODA. Snapshots are stored in the remaining space of the Volume Group. Most ODAs were sold with 2x 500GB local disks (SSDs) protected with a software RAID and allocated to a single Volume Group VolGroupSys. When doing the ODA setup, only a part of the storage is allocated to the system volumes as Logical Volumes (LogVolSwap, LogVolOpt, LogVolU01, LogVolRoot), meaning that a comfortable amount of space is available for ODABR:

pvs
  PV           VG          Fmt  Attr PSize   PFree
  /dev/md126p3 VolGroupSys lvm2 a--  446.09g 285.09g

/opt/odabr/odabr backup -snap
...
SUCCESS: 2026-04-22 17:10:33: LVM snapshots backup done successfully

pvs
  PV           VG          Fmt  Attr PSize   PFree
  /dev/md126p3 VolGroupSys lvm2 a--  446.09g 150.09g

ODABR additionally backups ASM metadata, because if you revert to a previous set of LVM snapshots with an older GI release, your ASM metadata would also need to be in the corresponding version.

Default snapshot sizes and real needs

Default snapshots size is the size of the Logical Volumes, here is an example for a 30GB /, a 55GB /opt and a 50GB /u01:

/opt/odabr/odabr infosnap

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
 odabr - ODA node Backup Restore - Version: 2.0.2-06
 Copyright 2013, 2025, Oracle and/or its affiliates.
 --------------------------------------------------------
 RACPack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

LVM snap name         status                COW Size              Data%
-------------         ----------            ----------            ------
root_snap             active                30.00 GiB             0.01%
opt_snap              active                55.00 GiB             0.01%
u01_snap              active                50.00 GiB             0.01%

This is quite confortable, there won’t be 100% blocks changed during the patching, for sure. If you don’t have these 135GB of free space, you can specify lower values for the snapshots, but make sure to limit modifications on the files residing on these 3 Logical Volumes during the patch:

/opt/odabr/odabr backup -snap -osize 18 -rsize 10 -usize 25

These settings were enough for my latest patches.

The ODA X9-2 series exception

As far as I remember, ODAs always had a confortable system disk sizes. 500GB is large enough for OS, odacli and GI stack plus a couple of files of your own. Since version 19.10, DB homes and diagnostic destination are now located within ACFS volumes, meaning that it doesn’t take a single MB from the local disks:

df -h | grep -e Filesystem -e u01
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolU01    49G   27G   20G  58% /u01
/dev/asm/odabase_n0-225              30G  2.5G   28G   9% /u01/app/odaorabase0
/dev/asm/orahome_sh-225              80G   44G   37G  54% /u01/app/odaorahome

As 19.10 was already available when X9-2 was released, Oracle decided to decrease the system disks from 500GB to 250GB, meaning that you didn’t have the margin you had with the previous generations. The problem was solved with X10 series and onwards, system disks are now back to 500GB as they were on X8-2 series.

What I mean is that you’ll be most probably concerned about filesystem contention for ODABR if you have X9-2 series.

How can you deal with insufficient disk space when using ODABR?

Imagine you have this kind of configuration:

pvs
  PV           VG          Fmt  Attr PSize   PFree
  /dev/md126p3 VolGroupSys lvm2 a--  222.56g 2.56g

On this ODA, /opt and /u01 have been extended for some reasons, and there is almost no space left on the Volume Group.

If you try to use ODABR to take snapshots, you will get this error:

Available LVM size X is less than required snapshot size Y

2.56GB is way too small for using ODABR: you will need to reduce the size of /u01, /opt or both. First, a cleanup is needed to free up some space. Once done, you’ll be able to reduce the size of /u01 and/or /opt. But these filesystems will need to be unmounted. And it cannot be done without stopping/starting some processes. Actually, pretty much all the processes running on your ODA.

Stopping the processes and reducing the filesystems

Let’s reduce the /u01 and /opt Logical Volumes’ size in this example:

su - root

df -h /u01 /opt

Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolU01    61G   23G   36G  39% /u01
/dev/mapper/VolGroupSys-LogVolOpt    70G   48G   22G  68% /opt

export ORACLE_HOME=/u01/app/19.30.0.0/grid
$ORACLE_HOME/bin/crsctl stop crs

lvreduce -L 50G /dev/VolGroupSys/LogVolU01 -r
Do you want to unmount "/u01" ? [Y|n] y
umount: /u01: target is busy.
fsadm: Cannot proceed with mounted filesystem "/u01".
  /usr/sbin/fsadm failed: 1
  Filesystem resize failed.

lsof /u01/
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
java    23727 root   26r   REG  252,3 16303574 1208968 /u01/app/19.30.0.0/grid/jlib/srvm.jar

kill -9 23727 ; umount /u01
umount: /u01: target is busy.

kill -9 23727 ; umount /u01
-bash: kill: (23727) - No such process

lvreduce -L 50G /dev/VolGroupSys/LogVolU01 -r
fsck from util-linux 2.32.1
/dev/mapper/VolGroupSys-LogVolU01: 76718/4063232 files (7.3% non-contiguous), 6220430/16252928 blocks
resize2fs 1.46.2 (28-Feb-2021)
Resizing the filesystem on /dev/mapper/VolGroupSys-LogVolU01 to 13107200 (4k) blocks.
The filesystem on /dev/mapper/VolGroupSys-LogVolU01 is now 13107200 (4k) blocks long.

  Size of logical volume VolGroupSys/LogVolU01 changed from 62.00 GiB (1984 extents) to 50.00 GiB (1600 extents).
  Logical volume VolGroupSys/LogVolU01 successfully resized.


systemctl stop initdcsagent
systemctl stop initdcscontroller
systemctl stop initdcsadmin
systemctl stop oda-mysql
systemctl stop oracle-ODA_DCS-ODA_DCS0
systemctl stop oracle-tfa
systemctl stop oracle-ohasd


lsof /opt
COMMAND     PID USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
bash       6697 root  cwd    DIR  252,2     4096 1703937 /opt/dbi
OSWatcher  7799 grid  cwd    DIR  252,2     4096 2235031 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb
OSWatcher  7799 grid    1w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
OSWatcher  7799 grid    2w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
OSWatcher  7799 grid  255r   REG  252,2    65286 2235047 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb/OSWatcher.sh
OSWatcher  9903 grid  cwd    DIR  252,2     4096 2235031 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb
OSWatcher  9903 grid    1w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
OSWatcher  9903 grid    2w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
OSWatcher  9903 grid  255r   REG  252,2     8035 2235046 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb/OSWatcherFM.sh
sleep     56757 grid  cwd    DIR  252,2     4096 2235031 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb
sleep     56757 grid    1w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
sleep     56757 grid    2w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
sleep     56835 grid  cwd    DIR  252,2     4096 2235031 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/oswbb
sleep     56835 grid    1w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)
sleep     56835 grid    2w   REG  252,2     4703 2228894 /opt/oracle/dcs/oracle.ahf/data/repository/suptools/dbioda01/oswbb/grid/run_1773762195.log (deleted)

kill -9 7799 9903


lvreduce -L 55G /dev/VolGroupSys/LogVolOpt -r
Do you want to unmount "/opt" ? [Y|n] y
fsck from util-linux 2.32.1
/dev/mapper/VolGroupSys-LogVolOpt: Inode 655911 extent tree (at level 1) could be narrower.  IGNORED.
/dev/mapper/VolGroupSys-LogVolOpt: 57911/4587520 files (2.4% non-contiguous), 12785629/18350080 blocks
resize2fs 1.46.2 (28-Feb-2021)
Resizing the filesystem on /dev/mapper/VolGroupSys-LogVolOpt to 14417920 (4k) blocks.
The filesystem on /dev/mapper/VolGroupSys-LogVolOpt is now 14417920 (4k) blocks long.

  Size of logical volume VolGroupSys/LogVolOpt changed from 70.00 GiB (2240 extents) to 55.00 GiB (1760 extents).
  Logical volume VolGroupSys/LogVolOpt successfully resized.

As I don’t want to manually restart the various processes, and as I usually do a sanity reboot before patching, let’s reboot the server. Everything will then be properly started after.

reboot
...

df -h /u01 /opt
Filesystem                         Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolU01   49G   23G   25G  49% /u01
/dev/mapper/VolGroupSys-LogVolOpt   54G   48G  4.0G  93% /opt

Further recommendations

You don’t really need a lot of space on the system disks. Put your patch files and other specific files on a NFS volume shared across your ODAs, and keep the system disks to their default’s size to keep a confortable margin for ODABR, especially on ODA X9-2 series. Eventually, allow a few more GB to /u01 and/or /opt but keep at least 70GB free for ODABR snapshots. Anticipate the patching and test ODABR several days before, it does not cost anything, and snapshots can be removed immediately without any downtime. Worst case is having to patch now and discovering that there is no space left on disks for using ODABR.

Not being able to secure your patch with ODABR is the second NOGO for patching, the first one being a faulty component reported by odaadmcli show server.

Conclusion

If you don’t have enough free space for ODABR snapshots, postpone the patching. ODABR is a mandatory safety net. Reducing /u01 and /opt filesystems’s size is possible. It will require stopping everything, but as you will patch your ODA, you’ve already allocated a sufficient downtime.

L’article Not enough space for using ODABR? Reduce /u01 and /opt size on your ODA est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/not-enough-space-for-using-odabr-reduce-u01-and-opt-size-on-your-oda/feed/ 0 Loosing java remote console full control during ODA Reimage https://www.dbi-services.com/blog/loosing-java-remote-console-full-control-during-oda-reimage/ https://www.dbi-services.com/blog/loosing-java-remote-console-full-control-during-oda-reimage/#respond Wed, 22 Apr 2026 08:18:39 +0000 https://www.dbi-services.com/blog/?p=43970 Recently I had to patch an ODA at one of our customer from 19.20 to 19.26, going through 19.24. As you are aware of, knowing we go from Oracle Linux 7 (19.20) to Oracle Linux 8 (19.24), we need to patch the ODA using Data Preserving Reprovisioning. This includes a reimage of the nodes… Data […]

L’article Loosing java remote console full control during ODA Reimage est apparu en premier sur dbi Blog.

]]> Recently I had to patch an ODA at one of our customer from 19.20 to 19.26, going through 19.24. As you are aware of, knowing we go from Oracle Linux 7 (19.20) to Oracle Linux 8 (19.24), we need to patch the ODA using Data Preserving Reprovisioning. This includes a reimage of the nodes…

Data Preserving Reprovisioning will allow you reprovisioning an already deployed Oracle Database Appliance system keeping the storage and the databases on the appliance, so without any modifications on the information stored in the ASM (storage). The information of the source system will be saved in a server data archive files. The appliance will then be reimaged with Oracle Database Appliance release iso image and the saved metadata will be used to directly reprovision the system and bring back all the resources such as databases, DB systems, Oracle ASR, and others.

The reimage can only be done using the java remote console, available with the ILOM. Remember that to attach the iso on the java console and use it during the reimage, you need to add it and connect it as described in the next picture.

It is important to see the following red message to ensure that the connection with the iso is made and that it can be used.

The problem is that after a time, during the reimage process, the java remote console lost full control on the node, having as consequence a lost of the iso connection and a failure in the reimage process.

This could be seen in the console with the message (View Only) displayed in the bar and the Red Cross on the top of the Computer (screen)/mouse image.

Here:

When it should be:

Here the whole picture showing the connection problem.

And we can see the message : “Sorry, keyboard and mouse connection has been lost. \nIf storage was being shared it has been lost also”.

When a correct connection would be the following one, having (Full Control) message displayed in the bar, with no lost of keyboard/mouse and storage (iso) connection.

After several tries it was always the same problem. After a few minutes, lost of the connection…

I then decided to check the ILOM sessions, connecting to the ILOM through ssh connection.

login as: root
Keyboard-interactive authentication prompts from server:
| Password:
End of keyboard-interactive prompts from server

Oracle(R) Integrated Lights Out Manager

Version 5.1.1.23 r151958

Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved.

Warning: HTTPS certificate is set to factory default.

Hostname: ODA01-ilo

-> show /SP/sessions

 /SP/sessions
    Targets:
        55342
        55346 (current)

    Properties:

    Commands:
        cd
        delete
        show

I found surprising to see so high sessions id. I decided to reset the ILOM. This would of course not restart the node.

-> reset /SP
Are you sure you want to reset /SP (y/n)? y
Performing reset on /SP

And connected again to the ILOM through ssh to checked the sessions:

login as: root
Keyboard-interactive authentication prompts from server:
| Password:
End of keyboard-interactive prompts from server

Oracle(R) Integrated Lights Out Manager

Version 5.1.1.23 r151958

Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved.

Warning: HTTPS certificate is set to factory default.

Hostname: ODA01-ilo

-> show /SP/sessions

 /SP/sessions
    Targets:
        1 (current)

    Properties:

    Commands:
        cd
        delete
        show

The sessions id were back to one, and I did not have any lost connection issue any more. I could successfully reimage the ODA to 19.24 version.

L’article Loosing java remote console full control during ODA Reimage est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/loosing-java-remote-console-full-control-during-oda-reimage/feed/ 0 MySQL Metadata Store service not able to start on an ODA https://www.dbi-services.com/blog/mysql-metadata-store-service-not-able-to-start-on-an-oda/ https://www.dbi-services.com/blog/mysql-metadata-store-service-not-able-to-start-on-an-oda/#respond Tue, 31 Mar 2026 20:32:30 +0000 https://www.dbi-services.com/blog/?p=43699 I was recently doing some consulting on some ODA at one of our customers, and I faced an issue with the DCS Agent and the MySQL Metadata Store. In this blog, I will show you the problem and how I could resolved it. Problem description On an Oracle Database Appliance, the DCS agent is the […]

L’article MySQL Metadata Store service not able to start on an ODA est apparu en premier sur dbi Blog.

]]> I was recently doing some consulting on some ODA at one of our customers, and I faced an issue with the DCS Agent and the MySQL Metadata Store. In this blog, I will show you the problem and how I could resolved it.

Problem description

On an Oracle Database Appliance, the DCS agent is the software providing the full automation to manage the appliance. It will have no impact on the well working of the databases but is mandatory to manage the appliance (patching, create dbhome, create database and any appliance resources). The DCS agent will perform various operations on the appliance, such as provisioning, database lifecycle management, backup and recovery, and storage management. The DCS Agent will use a MySQL database to store the metadata. In order for the DCS Agent to work, the MySQL database needs to be up and running.

On the node 0 of an ODA X9-2-HA, where I was preparing the next database patching operation, I wanted to run some cleanup.

[root@ODA-HA_node0 ~]# odacli cleanup-patchrepo -comp DB,GI -v 19.20.0.0.0
DCS-10009:Failed to create new service job report.

For which I could get a DCS failure.

I checked the job and realised that an ODA cleaning job where still in running status for about 3 months…

[root@ODA-HA_node0 ~]# odacli list-jobs

ID                                       Description                                                                 Created                             Status
---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------------
80f2bfbc-a02e-47bb-b57c-754d3d6d3d3a     Server Patching                                                             2025-11-12 08:00:14 CET             Success
2e36d676-f175-402d-a7e4-735b4eb965de     Patch pre-checks for [STORAGE]                                              2025-11-12 08:58:19 CET             Success
...
c4611880-fdc8-44fd-90a5-c2dd27d12f7c     Auto purge job data                                                         2025-12-09 03:59:48 CET             Success
ed09f270-5eff-4141-9b7b-51a2e5d30e32     Auto purge job data                                                         2025-12-10 03:59:50 CET             Success
aecd8ca5-44fb-4e2c-af2b-0e42e954381a     Auto purge job data                                                         2025-12-11 03:59:52 CET             Running

[root@ODA-HA_node0 ~]#

Troubleshooting

I restarted the DCS agent.

[root@ODA-HA_node0 ~]# systemctl stop initdcsagent

[root@ODA-HA_node0 ~]# systemctl start initdcsagent

[root@ODA-HA_node0 ~]# systemctl status initdcsagent
● initdcsagent.service - Oracle dcs-agent startup
   Loaded: loaded (/etc/systemd/system/initdcsagent.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2026-03-18 14:35:41 CET; 2s ago
 Main PID: 92038 (sh)
    Tasks: 96 (limit: 79998)
   Memory: 331.7M
   CGroup: /system.slice/initdcsagent.service
           ├─92038 /bin/sh -c . /opt/oracle/dcs/bin/setupJreAgent.sh;LD_LIBRARY_PATH=/opt/oracle/rhp/lib:$JRE_HOME/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib $JAVA -Xms128m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC -XX:ParallelGCThreads=4 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:/opt/oracle/dcs/log/gc-dcs-agent-%t-%p.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFi>
           └─92039 /opt/oracle/dcs/java/1.8.0_441/bin/java -Xms128m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC -XX:ParallelGCThreads=4 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:/opt/oracle/dcs/log/gc-dcs-agent-%t-%p.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Doracle.security.jps.config=/opt/oracle/dcs/agent/jps-config.xml -Dio.nett>

Mar 18 14:35:41 ODA-HA_node0 systemd[1]: Started Oracle dcs-agent startup.
[root@ODA-HA_node0 ~]# odacli list-jobs | tail -n3

Which did not help. In the DCS Agent log I could find some errors with HAMI. Oracle HAMI is the Oracle High Availability Metadata Infrastructure service providing distributed services required by DCS including locking and synchronizing configuration details in the cluster.

[root@ODA-HA_node0 log]# view dcs-agent-infra.log
...
...
...
2026-03-18 14:59:37,776 DEBUG [DcsInfraProvisioningThread] [] c.o.d.i.h.c.CommandExecutor: Going to execute command '/opt/oracle/dcs/hami/bin/hamictl.sh ssh-equiv --ensemble ODA_DCS --hosts ODA-HA_node0-priv,ODA-HA_node1-priv --json'
2026-03-18 14:59:38,014 DEBUG [DcsInfraProvisioningThread] [] c.o.d.i.h.c.CommandExecutor: Got result from execution of '/opt/oracle/dcs/hami/bin/hamictl.sh ssh-equiv --ensemble ODA_DCS --hosts ODA-HA_node0-priv,ODA-HA_node1-priv --json':
 - RC:     1
 - OUTPUT: /opt/oracle/dcs/oda-python3/lib/python3.12/getpass.py:91: GetPassWarning: Can not control echo on the terminal.
Warning: Password input may be echoed.
hamictl_oda_dcs's password: *********
{"operation": "ssh-equiv", "result": "FAILED", "message": "Invalid password"}

2026-03-18 14:59:38,015 ERROR [DcsInfraProvisioningThread] [] c.o.d.a.i.DcsInfraProvisioningThread: Could not provision DCS infrastructure
com.oracle.dcs.infra.hams.ctl.HamiCtl$HamiCtlException: Invalid password

I decided to restart MySQL service.

[root@ODA-HA_node0 ~]# systemctl stop oda-mysql

[root@ODA-HA_node0 ~]# systemctl start oda-mysql
Job for oda-mysql.service failed because the control process exited with error code.
See "systemctl status oda-mysql.service" and "journalctl -xe" for details.

Which failed…

Checking the logs:

[root@ODA-HA_node0 ~]# journalctl -xe
...
-- Unit oda-mysql.service has begun starting up.
Mar 31 09:23:10 ODA-HA_node0 systemd[90052]: oda-mysql.service: Failed to execute command: Permission denied
Mar 31 09:23:10 ODA-HA_node0 systemd[90052]: oda-mysql.service: Failed at step EXEC spawning /opt/oracle/dcs/mysql/bin/mysqld: Permission denied

I could see that it was complaining about permissions.

I checked the permissions on the MySQL binary:

[root@ODA-HA_node0 ~]# ls -ltrh /opt/oracle/dcs/mysql/bin/mysqld
-rwxr-x--- 1 odamysql odamysql 338M Mar 26 16:55 /opt/oracle/dcs/mysql/bin/mysqld
[root@ODA-HA_node0 ~]#

And that one was ok. The linux user starting MySQL, odamysql, was having the right permissions.

I then checked all permissions including the directories:

[root@ODA-HA_node0 log]# ls -ld /opt /opt/oracle /opt/oracle/dcs /opt/oracle/dcs/mysql /opt/oracle/dcs/mysql/bin
drwxrwx---  11 root     oinstall 4096 Nov 21 13:58 /opt
drwxr-xr-x.  9 root     root     4096 Nov 14 16:31 /opt/oracle
drwxr-xr-x. 30 root     root     4096 Nov 14 12:40 /opt/oracle/dcs
drwxr-xr-x. 13 root     root     4096 Nov 14 09:10 /opt/oracle/dcs/mysql
drwxr-x---.  2 odamysql odamysql 4096 Mar 26 16:55 /opt/oracle/dcs/mysql/bin
[root@ODA-HA_node0 log]#

And I could see that there were wrong on the /opt directory… The odamysql directory is not able to enter and read in the /opt directory, as it does not belong to the group and there is no permissions on the other users…

I checked on a valid ODA and as expected the permissions should be:

drwxr-xr-x. 11 root     root     4096 Mar 18 10:02 /opt

So this is mainly due to an human mistake…

Resolution

I first checked and could confirm the issue only affect the /opt directory itself.

[root@ODA-HA_node0 opt]# pwd
/opt

[root@ODA-HA_node0 opt]# ls -ltrh
total 48K
drwxr-xr-x.  3 root root     4.0K Sep 17  2024 lsi
drwxr-xr-x   2 root root     4.0K Jan 15  2025 oracle.ahf
drwx------.  2 root root      16K Nov 11 15:42 lost+found
drwxr-xr-x.  4 root root     4.0K Nov 11 16:32 MegaRAID
drwxr-xr-x   3 root root     4.0K Nov 12 10:20 ORCLfmap
drwxr-xr-x   5 root root     4.0K Nov 12 13:36 odabr
drwxr-xr-x.  9 root root     4.0K Nov 14 16:31 oracle
drwxrwx---  16 root oinstall 4.0K Feb 16 11:19 commvault
drwx------  11 root root     4.0K Mar 30 02:08 splunkforwarder
[root@ODA-HA_node0 opt]#

I change the permissions for other users.

[root@ODA-HA_node0 /]# chmod o+rx /opt

[root@ODA-HA_node0 /]# ls -ld /opt /opt/oracle /opt/oracle/dcs /opt/oracle/dcs/mysql /opt/oracle/dcs/mysql/bin
drwxrwxr-x  11 root     oinstall 4096 Nov 21 13:58 /opt
drwxr-xr-x.  9 root     root     4096 Nov 14 16:31 /opt/oracle
drwxr-xr-x. 30 root     root     4096 Nov 14 12:40 /opt/oracle/dcs
drwxr-xr-x. 13 root     root     4096 Nov 14 09:10 /opt/oracle/dcs/mysql
drwxr-x---.  2 odamysql odamysql 4096 Mar 26 16:55 /opt/oracle/dcs/mysql/bin
[root@ODA-HA_node0 /]#

And I could successfully start the MySQL service.

[root@ODA-HA_node0 /]# systemctl start oda-mysql

[root@ODA-HA_node0 /]# systemctl status oda-mysql
● oda-mysql.service - MySQL Server for ODA
   Loaded: loaded (/etc/systemd/system/oda-mysql.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2026-03-31 09:55:48 CEST; 6s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
 Main PID: 55837 (mysqld)
   Status: "Server is operational"
    Tasks: 33 (limit: 79998)
   Memory: 423.5M
   CGroup: /system.slice/oda-mysql.service
           └─55837 /opt/oracle/dcs/mysql/bin/mysqld --defaults-file=/opt/oracle/dcs/mysql/etc/mysqldb.cnf --skip-mysqlx

Mar 31 09:55:48 ODA-HA_node0 systemd[1]: Starting MySQL Server for ODA...
Mar 31 09:55:48 ODA-HA_node0 systemd[1]: Started MySQL Server for ODA.
[root@ODA-HA_node0 /]#

And I could successfully start the DCS Agent.

[root@ODA-HA_node0 /]# systemctl status initdcsagent
● initdcsagent.service - Oracle dcs-agent startup
   Loaded: loaded (/etc/systemd/system/initdcsagent.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Wed 2026-03-18 15:22:48 CET; 1 weeks 5 days ago
 Main PID: 84546 (code=killed, signal=TERM)

Mar 19 16:18:04 ODA-HA_node0 systemd[1]: Dependency failed for Oracle dcs-agent startup.
Mar 19 16:18:04 ODA-HA_node0 systemd[1]: initdcsagent.service: Job initdcsagent.service/start failed with result 'dependency'.
Mar 24 13:38:38 ODA-HA_node0 systemd[1]: Dependency failed for Oracle dcs-agent startup.
Mar 24 13:38:38 ODA-HA_node0 systemd[1]: initdcsagent.service: Job initdcsagent.service/start failed with result 'dependency'.
Mar 25 09:46:35 ODA-HA_node0 systemd[1]: Dependency failed for Oracle dcs-agent startup.
Mar 25 09:46:35 ODA-HA_node0 systemd[1]: initdcsagent.service: Job initdcsagent.service/start failed with result 'dependency'.
Mar 26 11:06:50 ODA-HA_node0 systemd[1]: Dependency failed for Oracle dcs-agent startup.
Mar 26 11:06:50 ODA-HA_node0 systemd[1]: initdcsagent.service: Job initdcsagent.service/start failed with result 'dependency'.
Mar 30 16:07:15 ODA-HA_node0 systemd[1]: Dependency failed for Oracle dcs-agent startup.
Mar 30 16:07:15 ODA-HA_node0 systemd[1]: initdcsagent.service: Job initdcsagent.service/start failed with result 'dependency'.

[root@ODA-HA_node0 /]# systemctl start initdcsagent

[root@ODA-HA_node0 /]# systemctl status initdcsagent
● initdcsagent.service - Oracle dcs-agent startup
   Loaded: loaded (/etc/systemd/system/initdcsagent.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2026-03-31 09:57:34 CEST; 1s ago
 Main PID: 59253 (sh)
    Tasks: 96 (limit: 79998)
   Memory: 270.0M
   CGroup: /system.slice/initdcsagent.service
           ├─59253 /bin/sh -c . /opt/oracle/dcs/bin/setupJreAgent.sh;LD_LIBRARY_PATH=/opt/oracle/rhp/lib:$JRE_HOME/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib $JAVA -Xms128m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC -XX:ParallelGCThreads=4 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:/opt/oracle/dcs/log/gc-dcs-agent-%t-%p.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFi>
           └─59254 /opt/oracle/dcs/java/1.8.0_441/bin/java -Xms128m -Xmx256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=256m -XX:+DisableExplicitGC -XX:ParallelGCThreads=4 -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xloggc:/opt/oracle/dcs/log/gc-dcs-agent-%t-%p.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M -Doracle.security.jps.config=/opt/oracle/dcs/agent/jps-config.xml -Dio.nett>

Mar 31 09:57:34 ODA-HA_node0 systemd[1]: Started Oracle dcs-agent startup.
[root@ODA-HA_node0 /]#

And I could successfully run the cleanup:

[root@ODA-HA_node0 ~]# odacli cleanup-patchrepo -comp DB,GI -v 19.20.0.0.0
{
  "jobId" : "08b395be-ae5a-4d6f-8e5d-1c0fa232f672",
  "status" : "Created",
  "message" : "",
  "reports" : [ ],
  "createTimestamp" : "March 31, 2026 11:21:58 CEST",
  "resourceList" : [ ],
  "description" : "Cleanup patchrepos",
  "updatedTime" : "March 31, 2026 11:21:58 CEST",
  "jobType" : null,
  "cpsMetadata" : null
}

[root@ODA-HA_node0 ~]# odacli describe-job -i 08b395be-ae5a-4d6f-8e5d-1c0fa232f672

Job details
----------------------------------------------------------------
                     ID:  08b395be-ae5a-4d6f-8e5d-1c0fa232f672
            Description:  Cleanup patchrepos
                 Status:  Success
                Created:  March 31, 2026 11:21:58 CEST
                Message:

Task Name                                Node Name                 Start Time                               End Time                                 Status
---------------------------------------- ------------------------- ---------------------------------------- ---------------------------------------- ----------------
Cleanup Repository                       ODA-HA_node1                 March 31, 2026 11:21:58 CEST             March 31, 2026 11:21:59 CEST             Success
Cleanup old ASR rpm                      ODA-HA_node0                 March 31, 2026 11:21:59 CEST             March 31, 2026 11:21:59 CEST             Success

[root@ODA-HA_node0 ~]#

Finally, I have also changed the group for /opt accordingly to root group.

[root@ODA-HA_node0 /]# chgrp root /opt

[root@ODA-HA_node0 /]# ls -ld /opt /opt/oracle /opt/oracle/dcs /opt/oracle/dcs/mysql /opt/oracle/dcs/mysql/bin
drwxrwxr-x  11 root     root     4096 Nov 21 13:58 /opt
drwxr-xr-x.  9 root     root     4096 Nov 14 16:31 /opt/oracle
drwxr-xr-x. 30 root     root     4096 Nov 14 12:40 /opt/oracle/dcs
drwxr-xr-x. 13 root     root     4096 Nov 14 09:10 /opt/oracle/dcs/mysql
drwxr-x---.  2 odamysql odamysql 4096 Mar 26 16:55 /opt/oracle/dcs/mysql/bin
[root@ODA-HA_node0 /]#

To wrap up…

DCS Agent and MySQL database will have no effect on the well working of your databases, but if they are not working you are not able to manage your appliance any more. Here, wrong permissions (for sure a human mistake) had sad effects on appliance software. Solving the permissions issue resolved my problem.

L’article MySQL Metadata Store service not able to start on an ODA est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/mysql-metadata-store-service-not-able-to-start-on-an-oda/feed/ 0 Starting ODA 19.29 – Old grid version cleanup https://www.dbi-services.com/blog/starting-oda-19-29-old-grid-version-cleanup/ https://www.dbi-services.com/blog/starting-oda-19-29-old-grid-version-cleanup/#respond Mon, 30 Mar 2026 17:05:27 +0000 https://www.dbi-services.com/blog/?p=43683 We know that during the ODA patching, a new grid version will be installed. To avoid the /u01 file system to fill up, we need to cleanup the old version. In this blog, I will show you how you can do it and also few changes coming with version 19.29, as you might need a […]

L’article Starting ODA 19.29 – Old grid version cleanup est apparu en premier sur dbi Blog.

]]> We know that during the ODA patching, a new grid version will be installed. To avoid the /u01 file system to fill up, we need to cleanup the old version. In this blog, I will show you how you can do it and also few changes coming with version 19.29, as you might need a reboot before.

Remember that until 19.28, the patch of the server was done with odacli update-server command, and this will patch the whole server (OS, ILOM, BIOS, …) and the grid in the same step.

Starting 19.29 you will run the server and the grid patching separately. odacli update-servercomponents will patch the server (OS, ILOM, HMP, firmware for local disk and controller, …) and odacli update-gihome will patch and install the new grid version. A reboot of the node will only be performed after the servercomponents update due to the ILOM and BIOS update.

Once the patching process was successfully done, we can cleanup old grid version. In my example, I patched the ODA from version 19.28 to 19.30.

Check usage

First of all we check usage of the grid files. The new grid version is 19.30, so I expect to have file using 19.30 grid.

[root@ODA01 ~]# lsof | grep "/u01/app/19.30.0.0/" | wc -l
29545

Before any cleanup, I need to ensure the old 19.28 grid infrastructure is not used any more.

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | wc -l
3295

Which is definitely note the case.

I check which process are still using old grid infra.

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | grep -i oakd | wc -l
3233

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | grep -iv oakd | wc -l
62

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | grep -iv oakd
java      86061                            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2390 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2394 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2440 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2545 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2546 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2547 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  2548 LegacyRun            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061  4274 ForkJoinP            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 16159 process              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86062 TFAServer            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86063 ParGC\x20            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86064 ParGC\x20            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86065 ParGC\x20            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86066 VM\x20Thr            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86068 Reference            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86069 Finalizer            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86071 Signal               root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86072 Service              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86073 C2\x20Com            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86074 C1\x20Com            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86075 Sweeper              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86078 VM\x20Per            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86079 Common-Cl            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86145 process              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86252 LogFlushe            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86253 DataCorru            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86296 INCompres            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86297 Cleaner-1            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86298 Checkpoin            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86396 pool-3-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86397 pool-3-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86398 pool-3-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86551 TFAIndexe            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 86983 TFAMonito            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87763 ActionHan            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87776 ClusterEv            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87777 Repositor            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87782 PeriodicC            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87786 InstanceM            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87788 Thread-13            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87790 DiskUsage            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87794 process              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87797 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87799 PurgeMana            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87800 TFAMaster            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87804 LegacyTel            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87805 TFAPlugin            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87806 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87807 TFAInvent            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87808 TFADbUtlS            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87809 TFADbUtlP            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 87810 RetryUplo            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 88715 Tailing              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 88718 Tailing              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 88719 Tailing              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 88720 Tailing              root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 91944 pool-6-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 98253 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 98254 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 98331 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar
java      86061 98388 pool-2-th            root   26r      REG             252,10     16284857    1713748 /u01/app/19.28.0.0/grid/jlib/srvm.jar

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | grep -i oakd | tail -n3
oakd      65702 71035 oakd                 root  118r      REG             252,10       174592    1717769 /u01/app/19.28.0.0/grid/rdbms/mesg/diaus.msb
oakd      65702 71035 oakd                 root  119r      REG             252,10       174592    1717769 /u01/app/19.28.0.0/grid/rdbms/mesg/diaus.msb
oakd      65702 71035 oakd                 root  120r      REG             252,10       174592    1717769 /u01/app/19.28.0.0/grid/rdbms/mesg/diaus.msb
[root@ODA01 ~]#

So OAK and TFA are still running on old grid version and using old grid jar library.

This comes from the fact that after the server components patches, the server is rebooted, the oak, tfa and other process will start with the existing grid version, 19.28. The grid patch will not reboot the server and not restart those processes, so they will still run with old grid infra.

This is why before any cleanup and starting 19.29 only, we need to reboot the ODA.

Reboot the DOA:

[root@ODA01 ~]# systemctl reboot
[root@ODA01 ~]#

After reboot we can see that only new grid version is used:

[root@ODA01 ~]# uptime
 16:16:17 up 3 min,  1 user,  load average: 2.06, 1.58, 0.65

[root@ODA01 ~]# lsof | grep "/u01/app/19.28.0.0/" | wc -l
0

[root@ODA01 ~]# lsof | grep "/u01/app/19.30.0.0" | wc -l
45798
[root@ODA01 ~]#

Cleanup old grid

Current space for /u1 is 6.2 GB:

[root@ODA01 ~]# df -h / /opt /u01
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolRoot   30G  5.1G   23G  19% /
/dev/mapper/VolGroupSys-LogVolOpt    48G   26G   20G  57% /opt
/dev/mapper/VolGroupSys-LogVolU01    49G   41G  6.2G  87% /u01
[root@ODA01 ~]#

Remove old version of grid:

[root@ODA01 ~]# cd /u01/app

[root@ODA01 app]# mv 19.28.0.0 BAK_19.28.0.0

[root@ODA01 app]# lsof | grep "/u01/app/19.28.0.0" | wc -l
0

[root@ODA01 app]# lsof | grep "/u01/app/BAK_19.28.0.0" | wc -l
0

[root@ODA01 app]# rm -rf BAK_19.28.0.0
[root@ODA01 app]#

Check current space:

[root@ODA01 app]# df -h / /u01 /opt /boot
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolRoot   30G  5.1G   23G  19% /
/dev/mapper/VolGroupSys-LogVolU01    49G   27G   20G  58% /u01
/dev/mapper/VolGroupSys-LogVolOpt    48G   26G   20G  57% /opt
/dev/md126p2                        488M  218M  235M  49% /boot
[root@ODA01 app]#

We have taken back about 14 GB.

Check oracle inventory

Checking oracle inventory:

[root@ODA01 app]# cd /u01/app/oraInventory/ContentsXML/

[root@ODA01 ContentsXML]# cat inventory.xml

You will see both grid home:

<HOME NAME="OraGrid192800" LOC="/u01/app/19.28.0.0/grid" TYPE="O" IDX="13"/>
<HOME NAME="OraGrid193000" LOC="/u01/app/19.30.0.0/grid" TYPE="O" IDX="15" CRS="true"/>

The current running one is the one having CRS option to true.

Detach old grid home from the inventory

We can now detach old grid from the inventory:

[grid@ODA01 ~]$ echo $ORACLE_SID
+ASM1

[grid@ODA01 ~]$ echo $ORACLE_HOME
/u01/app/19.30.0.0/grid

[grid@ODA01 ~]$ cd $ORACLE_HOME/oui/bin

[grid@ODA01 bin]$ ./runInstaller -detachHome -silent -local ORACLE_HOME=/u01/app/19.28.0.0/grid
Starting Oracle Universal Installer...

Checking swap space: must be greater than 500 MB.   Actual 24575 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
You can find the log of this install session at:
 /u01/app/oraInventory/logs/DetachHome2026-03-29_04-34-43PM.log
'DetachHome' was successful.

Check inventory

Checking the inventory again:

[grid@ODA01 bin]$ cd /u01/app/oraInventory/ContentsXML/

[grid@ODA01 ContentsXML]$ cat inventory.xml

We will see old grid with option REMOVE=”T”

<HOME NAME="OraGrid193000" LOC="/u01/app/19.30.0.0/grid" TYPE="O" IDX="15" CRS="true"/>
<HOME NAME="OraGrid192800" LOC="/u01/app/19.28.0.0/grid" TYPE="O" IDX="13" REMOVED="T"/>

L’article Starting ODA 19.29 – Old grid version cleanup est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/starting-oda-19-29-old-grid-version-cleanup/feed/ 0 ODA 19.30 prepatch report failing on OS patches https://www.dbi-services.com/blog/oda-19-30-prepatch-report-failing-on-os-patches/ https://www.dbi-services.com/blog/oda-19-30-prepatch-report-failing-on-os-patches/#respond Mon, 30 Mar 2026 15:03:50 +0000 https://www.dbi-services.com/blog/?p=43662 I had to recently patch several ODAs from 19.28 to 19.30, and one ODA patching failed running the prepatch report which is mandatory and helpful before going to patch the ODA. For each step (server, grid or storage) a prepatch report is needed. I had to troubleshoot and I could solve the problem, and I […]

L’article ODA 19.30 prepatch report failing on OS patches est apparu en premier sur dbi Blog.

]]> I had to recently patch several ODAs from 19.28 to 19.30, and one ODA patching failed running the prepatch report which is mandatory and helpful before going to patch the ODA. For each step (server, grid or storage) a prepatch report is needed. I had to troubleshoot and I could solve the problem, and I thought it would be more than interesting to share it, expecting it might be helpful.

Problem description

After dcs-admin update, dcs components update, repository update with grid and rdbms clones, I have run a prepatch report, and decided to run it already for all components : server, grid and storage.

This is why we will see in the output following part : OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI.

Running the prepatch report:

[root@ODA01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -sc -gi -st -v 19.30.0.0.0

Job details
----------------------------------------------------------------
                     ID:  6681af21-907a-4ffc-8add-d40af11441ae
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  Created
                Created:  March 29, 2026 10:35:01 CEST
                Message:  Use 'odacli describe-prepatchreport -i 6681af21-907a-4ffc-8add-d40af11441ae' to check details of results

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------

[root@ODA01 ~]#

Describing the report, I could see that it failed on the OS rpm verification:

[root@ODA01 ~]# odacli describe-prepatchreport -i 6681af21-907a-4ffc-8add-d40af11441ae

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  6681af21-907a-4ffc-8add-d40af11441ae
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  FAILED
                Created:  March 29, 2026 10:35:01 AM CEST
                 Result:  One or more pre-checks failed for [OS]

Node Name
---------------
ODA01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Verify All OS patches           Failed    DCS-10252 - dnf dry-run execution
                                          failed: check
                                          /opt/oracle/dcs/log/jobfiles/
                                          dnfdryrunout_2026-03-29_10-35-
                                          15.0673_536.log file for more details
Validate command execution      Success   Validated command execution

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Validate command execution      Success   Validated command execution

__STORAGE__
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Patch location validation       Success   Verified patch location
Patch tag validation            Success   Verified patch tag
Storage patch tag validation    Success   Verified storage patch location
Verify ASM disks status         Success   ASM disks are online
Validate command execution      Success   Validated command execution

__RHPGI__
Validate available space        Success   Validated free space under /u01
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate ASM is online          Success   ASM is online
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate DB start dependencies  Success   DBs START dependency check passed
Validate DB stop dependencies   Success   DBs STOP dependency check passed
Validate space for clones       Success   Clones volume is already created
volume
Validate command execution      Success   Validated command execution

[root@ODA01 ~]#

I checked the appropriate log, and could easily guess that the process complained about missing rhn-client-tools:

[root@ODA01 ~]# view /opt/oracle/dcs/log/jobfiles/dnfdryrunout_2026-03-29_10-35-15.0673_536.log
2026-03-29 10:35:29,977 [DNFHelper] INFO    : File path /tmp/dcsfiles/dnfoperation_2026-03-29_10-35-15.0676_536.json exists
2026-03-29 10:35:29,977 [DNFHelper] INFO    : File path /tmp/dcsfiles/dnfoperation_2026-03-29_10-35-15.0676_536.json is a regular file
2026-03-29 10:35:29,979 [dnf      ] DEBUG   : User-Agent: constructed: 'libdnf (Oracle Linux Server 8.10; server; Linux.x86_64)'
2026-03-29 10:35:29,991 [dnf      ] DEBUG   : User-Agent: constructed: 'libdnf (Oracle Linux Server 8.10; server; Linux.x86_64)'
2026-03-29 10:35:30,139 [dnf      ] DEBUG   : repo: using cache for: PrecheckOSPatchRepo
2026-03-29 10:35:30,147 [dnf      ] DEBUG   : PrecheckOSPatchRepo: using metadata from Sun 29 Mar 2026 10:35:12 AM CEST.
2026-03-29 10:35:30,148 [dnf      ] INFO    : Last metadata expiration check: 0:00:18 ago on Sun 29 Mar 2026 10:35:12 AM CEST.
2026-03-29 10:35:31,907 [DNFHelper] DEBUG   : === DNF Plugin Status ===
2026-03-29 10:35:31,907 [DNFHelper] DEBUG   : DNF plugins are enabled
2026-03-29 10:35:31,910 [DNFHelper] DEBUG   : === Debug Info ===
2026-03-29 10:35:31,910 [DNFHelper] DEBUG   : Operation RPM Metadata: {"install": ["hmpipmitool.x86_64", "rhn-setup.x86_64", "uptrack-updates-5.15.0-316.196.4.1.el8uek.x86_64.noarch", "ksplice-offline.x86_64", "ksplice.x86_64"], "exclude": ["linux-firmware-core.noarch"], "remove": ["uptrack-updates-5.15.0-310.184.5.2.el8uek.x86_64.noarch"]}
2026-03-29 10:35:31,910 [DNFHelper] DEBUG   : Installed packages: ['MegaCli-8.07.07-1.noarch', 'NetworkManager-1:1.40.16-19.0.1.el8_10.x86_64', 'NetworkManager-config-server-1:1.40.16-19.0.1.el8_10.noarch', 'NetworkManager-libnm-1:1.40.16-19.0.1.el8_10.x86_64', 'NetworkManager-team-1:1.40.16-19.0.1.el8_10.x86_64', 'NetworkManager-tui-1:1.40.16-19.0.1.el8_10.x86_64', 'VRTSnbcfg-10.2.0.1-1686143553.x86_64', 'VRTSnbclibs-10.2.0.1-1686143611.x86_64', 'VRTSnbclt-10.2.0.1-1686143729.x86_64', 'VRTSnbpck-10.2.0.1-1686143786.x86_64', 'VRTSpbx-1.17.62.5-5.x86_64', 'VRTSpddea-18.0.1.0-0023.x86_64', 'abattis-cantarell-fonts-0.0.25-6.el8.noarch', 'acl-2.2.53-3.el8.x86_64', 'adcli-0.9.2-1.el8.x86_64', 'adwaita-cursor-theme-3.28.0-3.el8.noarch', 'adwaita-icon-theme-3.28.0-3.el8.noarch', 'aide-0.16-14.el8_5.1.x86_64', 'alsa-lib-1.2.10-2.el8.x86_64', 'annobin-11.13-2.0.6.el8.x86_64', 'at-3.1.20-12.el8.x86_64', 'at-spi2-atk-2.26.2-1.el8.x86_64', 'at-spi2-core-2.28.0-1.el8.x86_64', 'atk-2.28.1-1.el8.x86_64', 'attr-2.4.48-3.el8.x86_64', 'audit-3.1.2-1.0.1.el8.x86_64', 'audit-libs-3.1.2-1.0.1.el8.x
...
...
...
2026-03-29 10:35:32,505 [DNFHelper] ERROR   : Dependency solving failed
2026-03-29 10:35:32,505 [DNFHelper] ERROR   : Error occured:
Traceback (most recent call last):
  File "/opt/oracle/oak/pkgrepos/scripts/19.30.0.0.0/dnfhelper.py", line 315, in main
    metadata_file=metadata_file
  File "/opt/oracle/oak/pkgrepos/scripts/19.30.0.0.0/dnfhelper.py", line 226, in run_combined_dnf_txn
    base.resolve()
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 906, in resolve
    raise exc
dnf.exceptions.DepsolveError:
 Problem: conflicting requests
  - nothing provides rhn-client-tools = 2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe needed by rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64 from PrecheckOSPatchRepo

Analysing problem and root cause

I first checked if the package is really missing:

[root@ODA01 ~]# rpm -qa | grep -i rhn-client-tools
[root@ODA01 ~]#

And it was definitively missing, when on other ODA it was existing:

[root@ODA02 ~]# rpm -qa | grep -i rhn-client-tools
python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
[root@ODA02 ~]#

I checked in the history of the package management if one transaction could have affected this.

[root@ODA01 ~]# dnf history | grep -i rhn-client-tools
     2 | remove -y rhn-client-tools                                    | 2024-03-12 17:27 | Removed        |   16  <
[root@ODA01 ~]#

And it was the case! Someone manually removed the package 2 years ago. And also we can see that 16 dependencies rpm have been removed as well. What a mess!

I take the opportunity here to remind that an ODA IS AN APPLIANCE, and not a normal linux server. Any yum or dnf changes will have huge impact on the ODA patching, knowing that each version is coming with its bundle patch version, and the ODA will lock the repository. If a package is needed (for example, veeam agent plugin) it is always better to prioritise an installation with a tar file which will be totally transparent for the ODA patching. Otherwise it is better to install each rpm individually, avoiding if possible yum or dnf. The worst of course is removing a package with several dependencies. dnf remove including dependencies is of course something that I would never recommend to run on an ODA.

The second question we can raise, is why now? We have been running several version from 2024 to 2026 and it seems only the 19.30 version is complaining about it.

Let’s list all the dependant packages that have been removed during this dnf transaction…

[root@ODA01 ~]# dnf history info 2
Transaction ID : 2
Begin time     : Tue 12 Mar 2024 05:27:35 PM CET
Begin rpmdb    : 1162:21b3b42bf2d4360e1ace842a41737674018ff3da
End time       : Tue 12 Mar 2024 05:27:36 PM CET (1 seconds)
End rpmdb      : 1146:a140a149ea28fdac865a8abdbe954771f896a078
User           : root 
Return-Code    : Success
Releasever     : 8
Command Line   : remove -y rhn-client-tools
Comment        :
Packages Altered:
    Removed dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch         @@System
    Removed python3-dmidecode-3.12.3-2.el8.x86_64                                          @@System
    Removed python3-dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch @@System
    Removed python3-dnf-plugin-ulninfo-0.3-2.module+el8.4.0+20142+f119a2fa.noarch          @@System
    Removed python3-hwdata-2.3.6-3.el8.noarch                                              @@System
    Removed python3-librepo-1.14.2-4.el8.x86_64                                            @@System
    Removed python3-netifaces-0.10.6-4.el8.x86_64                                          @@System
    Removed python3-newt-0.52.20-11.el8.x86_64                                             @@System
    Removed python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64           @@System
    Removed python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64    @@System
    Removed python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64           @@System
    Removed rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                   @@System
    Removed rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64            @@System
    Removed rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                   @@System
    Removed rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64                             @@System
    Removed usermode-1.113-2.el8.x86_64                                                    @@System
[root@ODA01 yum.repos.d]#

Of course, I do not need to mention that such action has not been performed on the other ODA from the customer field, and this is why the patching did not raise this issue on the other ODAs.

[root@ODA02 ~]# dnf history | grep -i rhn-client-tools
[root@ODA02 ~]#

Resolution

The ODA had no internet access. That was a pity as I could have easily installed the needed packages that were removed from the official oracle linux distribution package manager, using dnf. So the only way of resolution for me was to download and install each package manually. In any case it was not worth to try otherwise the patching activity could have been stopped here. Stuff work as I had to download and install about 16 RPMs.

At the beginning, I tried to find the needed rpm from the Oracle linux distribution (yum.oracle.com) directly, but I need to say that it was not that so easy to find all the rpm. Also the maintenance windows was running on, and I had to find quickly a solution. I then get used of pkgs.org project which is a kind of search engine linking to the appropriate distribution mirror to download the package. We can see the home page of each project. It was much helpful because you can easily find the package, and in the download part it refers you to the exact rpm package you need from the official documentation. I found it so helpful knowing the time constraints.

For example for rhn-setup package, we can easily find it from pkgs.org:

https://oraclelinux.pkgs.org/8/ol8-appstream-x86_64/rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm.html

In the download part you will find the exact distribution link for the source and the binary, as well as the mirror, as displayed in next table:

TypeURL
Binary Packagehttps://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/getPackage/rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
Source Packagehttps://yum.oracle.com/repo/OracleLinux/OL8/appstream/x86_64/getPackageSource/rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.src.rpm
Mirroryum.oracle.com

So the package comes from the official distribution, and you can confirm for each of your downloads.

I could install the first package which are needed for rhn-client-tools:

[root@ODA01 19.30]# rpm -ivh python3-newt-0.52.20-11.el8.x86_64.rpm
warning: python3-newt-0.52.20-11.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:python3-newt-0.52.20-11.el8      ################################# [100%]

[root@ODA01 19.30]# rpm -ivh python3-netifaces-0.10.6-4.el8.x86_64.rpm
warning: python3-netifaces-0.10.6-4.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:python3-netifaces-0.10.6-4.el8   ################################# [100%]

[root@ODA01 19.30]# rpm -ivh python3-hwdata-2.3.6-3.el8.noarch.rpm
warning: python3-hwdata-2.3.6-3.el8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:python3-hwdata-2.3.6-3.el8       ################################# [100%]

[root@ODA01 19.30]# rpm -ivh python3-dmidecode-3.12.2-15.el8.x86_64.rpm
warning: python3-dmidecode-3.12.2-15.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:python3-dmidecode-3.12.2-15.el8  ################################# [100%]

I then tried to install rhn-client-tools and python3-rhn-client-tools:

[root@ODA01 19.30]# rpm -ivh python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
warning: python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
error: Failed dependencies:
        rhn-client-tools = 2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe is needed by python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64

[root@ODA01 19.30]# rpm -ivh rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
warning: rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
error: Failed dependencies:
        python3-rhn-client-tools = 2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe is needed by rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
[root@ODA01 19.30]#

But it was like each one needs the other, so it was not possible to install it with rpm binary. I then used dnf to install them.

[root@ODA01 19.30]# mkdir local_repo_issue_rhn
[root@ODA01 19.30]# mv rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm ./local_repo_issue_rhn/
[root@ODA01 19.30]# mv python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm ./local_repo_issue_rhn/
[root@ODA01 19.30]# cd local_repo_issue_rhn/
[root@ODA01 local_repo_issue_rhn]# dnf install *.rpm
BKB_Common                                                                                                                                                                                                                            95 kB/s | 1.5 kB     00:00
Dependencies resolved.
=====================================================================================================================================================================================================================================================================
 Package                                                           Architecture                                    Version                                                                               Repository                                             Size
=====================================================================================================================================================================================================================================================================
Installing:
 python3-rhn-client-tools                                          x86_64                                          2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                           @commandline                                          110 k
 rhn-client-tools                                                  x86_64                                          2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                           @commandline                                          394 k

Transaction Summary
=====================================================================================================================================================================================================================================================================
Install  2 Packages

Total size: 503 k
Installed size: 2.4 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                                             1/1
  Installing       : rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                         1/2
  Running scriptlet: rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                         1/2
  Installing       : python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                 2/2
  Running scriptlet: python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                 2/2
  Verifying        : python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                 1/2
  Verifying        : rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                         2/2

Installed:
  python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                           rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64

Complete!
[root@ODA01 local_repo_issue_rhn]#

Working, I had now the rhn-client-tools installed:

[root@ODA01 local_repo_issue_rhn]# rpm -qa | grep -i rhn-client
rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
[root@ODA01 local_repo_issue_rhn]#

I tried a new prepatch report:

[root@ODA01 local_repo_issue_rhn]# cd
[root@ODA01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -sc -gi -st -v 19.30.0.0.0

Job details
----------------------------------------------------------------
                     ID:  0af9bc46-2bfe-42a1-a6dc-bb89b9c0abb8
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  Created
                Created:  March 29, 2026 12:57:10 CEST
                Message:  Use 'odacli describe-prepatchreport -i 0af9bc46-2bfe-42a1-a6dc-bb89b9c0abb8' to check details of results

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------

[root@ODA01 ~]#

Which failed again on the same OS patches verification step:

[root@ODA01 ~]# odacli describe-prepatchreport -i 0af9bc46-2bfe-42a1-a6dc-bb89b9c0abb8

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  0af9bc46-2bfe-42a1-a6dc-bb89b9c0abb8
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  FAILED
                Created:  March 29, 2026 12:57:10 PM CEST
                 Result:  One or more pre-checks failed for [OS]

Node Name
---------------
ODA01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Verify All OS patches           Failed    DCS-10252 - dnf dry-run execution
                                          failed: check
                                          /opt/oracle/dcs/log/jobfiles/
                                          dnfdryrunout_2026-03-29_12-57-
                                          23.0066_1614.log file for more details
...
...
...

And the log showed other missing packages that was part of the dependencies removed by the manual dnf remove command.

[root@ODA01 ~]# view /opt/oracle/dcs/log/jobfiles/dnfdryrunout_2026-03-29_12-57-23.0066_1614.log
...
...
...
dnf.exceptions.DepsolveError:
 Problem: package rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64 from PrecheckOSPatchRepo requires rhnsd, but none of the providers can be installed
  - package rhnsd-5.0.35-3.0.2.module+el8.10.0+90373+b70ceaf0.x86_64 from PrecheckOSPatchRepo requires rhn-check >= 0.0.8, but none of the providers can be installed
  - conflicting requests
  - nothing provides dnf-plugin-spacewalk >= 2.4.0 needed by rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64 from PrecheckOSPatchRepo

So I installed all other dependencies rpm.

I installed usermode rpm:

[root@ODA01 19.30]# rpm -ivh usermode-1.113-2.el8.x86_64.rpm
warning: usermode-1.113-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ad986da3: NOKEY
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing..a
   1:usermode-1.113-2.el8             ################################# [100%]
[root@ODA01 19.30]#

And finally all others with dnf:

[root@ODA01 local_repo_issue_rhn]# ls -ltrh
total 568K
-rw-r--r--. 1 root root 33K Mar 29 13:24 rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
-rw-r--r--. 1 root root 26K Mar 29 13:31 rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
-rw-r--r--. 1 root root 72K Mar 29 13:31 python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm
-rw-r--r--. 1 root root 52K Mar 29 13:41 rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64.rpm
-rw-r--r--. 1 root root 31K Mar 29 13:53 python3-dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch.rpm
-rw-r--r--. 1 root root 54K Mar 29 14:08 python3-librepo-1.14.2-4.el8.x86_64.rpm
-rw-r--r--. 1 root root 24K Mar 29 14:08 dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch.rpm
-rw-r--r--. 1 root root 12K Mar 29 14:31 python3-dnf-plugin-ulninfo-0.3-3.module+el8.10.0+90380+96a02ce9.noarch.rpm
-rw-r--r--. 1 root root 93K Mar 29 14:31 librepo-1.14.2-4.el8.x86_64.rpm
-rw-r--r--. 1 root root 40K Mar 29 14:43 python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64.rpm

[root@ODA01 local_repo_issue_rhn]# dnf install *.rpm
BKB_Common                                                                                                                                                                                                                            37 kB/s | 1.5 kB     00:00
Dependencies resolved.
=====================================================================================================================================================================================================================================================================
 Package                                                              Architecture                                   Version                                                                              Repository                                            Size
=====================================================================================================================================================================================================================================================================
Installing:
 dnf-plugin-spacewalk                                                 noarch                                         2.8.5-11.0.3.module+el8.3.0+20070+f5719e00                                           @commandline                                          24 k
 python3-dnf-plugin-spacewalk                                         noarch                                         2.8.5-11.0.3.module+el8.3.0+20070+f5719e00                                           @commandline                                          31 k
 python3-dnf-plugin-ulninfo                                           noarch                                         0.3-3.module+el8.10.0+90380+96a02ce9                                                 @commandline                                          11 k
 python3-librepo                                                      x86_64                                         1.14.2-4.el8                                                                         @commandline                                          54 k
 python3-rhn-check                                                    x86_64                                         2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                          @commandline                                          39 k
 python3-rhn-setup                                                    x86_64                                         2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                          @commandline                                          71 k
 rhn-check                                                            x86_64                                         2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                          @commandline                                          26 k
 rhn-setup                                                            x86_64                                         2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe                                          @commandline                                          32 k
 rhnsd                                                                x86_64                                         5.0.35-3.0.1.module+el8+5192+3173336a                                                @commandline                                          51 k
Downgrading:
 librepo                                                              x86_64                                         1.14.2-4.el8                                                                         @commandline                                          93 k

Transaction Summary
=====================================================================================================================================================================================================================================================================
Install    9 Packages
Downgrade  1 Package

Total size: 432 k
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                                                             1/1
  Downgrading      : librepo-1.14.2-4.el8.x86_64                                                                                                                                                                                                                1/11
  Installing       : python3-librepo-1.14.2-4.el8.x86_64                                                                                                                                                                                                        2/11
  Running scriptlet: dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                                     3/11
  Installing       : dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                                     3/11
  Running scriptlet: dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                                     3/11
  Installing       : python3-dnf-plugin-ulninfo-0.3-3.module+el8.10.0+90380+96a02ce9.noarch                                                                                                                                                                     4/11
  Installing       : python3-dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                             5/11
  Installing       : python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                       6/11
  Installing       : rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                               7/11
  Installing       : rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64                                                                                                                                                                                         8/11
  Running scriptlet: rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64                                                                                                                                                                                         8/11
  Installing       : python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                       9/11
  Installing       : rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                              10/11
  Cleanup          : librepo-1.14.2-5.el8.x86_64                                                                                                                                                                                                               11/11
  Running scriptlet: librepo-1.14.2-5.el8.x86_64                                                                                                                                                                                                               11/11
  Verifying        : librepo-1.14.2-4.el8.x86_64                                                                                                                                                                                                                1/11
  Verifying        : librepo-1.14.2-5.el8.x86_64                                                                                                                                                                                                                2/11
  Verifying        : dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                                     3/11
  Verifying        : python3-dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch                                                                                                                                                             4/11
  Verifying        : python3-dnf-plugin-ulninfo-0.3-3.module+el8.10.0+90380+96a02ce9.noarch                                                                                                                                                                     5/11
  Verifying        : python3-librepo-1.14.2-4.el8.x86_64                                                                                                                                                                                                        6/11
  Verifying        : python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                       7/11
  Verifying        : python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                       8/11
  Verifying        : rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                               9/11
  Verifying        : rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64                                                                                                                                                                                        10/11
  Verifying        : rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                                                                                                                                                              11/11

Downgraded:
  librepo-1.14.2-4.el8.x86_64
Installed:
  dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch              python3-dnf-plugin-spacewalk-2.8.5-11.0.3.module+el8.3.0+20070+f5719e00.noarch              python3-dnf-plugin-ulninfo-0.3-3.module+el8.10.0+90380+96a02ce9.noarch
  python3-librepo-1.14.2-4.el8.x86_64                                                 python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                        python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
  rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                        rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64                                rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64

Complete!
[root@ODA01 local_repo_issue_rhn]#

I checked the packages were there:

[root@ODA01 local_repo_issue_rhn]# rpm -qa | grep -i rhn
python3-rhnlib-2.8.6-8.0.2.module+el8.7.0+21027+f0093b7a.noarch
rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
python3-rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
python3-rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
rhnsd-5.0.35-3.0.1.module+el8+5192+3173336a.x86_64
python3-rhn-client-tools-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
rhn-check-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
rhn-setup-2.8.16-13.0.6.module+el8.7.0+21032+057d0dfe.x86_64
rhnlib-2.8.6-8.0.2.module+el8.7.0+21027+f0093b7a.noarch
[root@ODA01 local_repo_issue_rhn]#

And rerun a precheck report:

[root@ODA01 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -sc -gi -st -v 19.30.0.0.0

Job details
----------------------------------------------------------------
                     ID:  3d5f5d96-f566-40c7-ad84-38c436fa3bba
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  Created
                Created:  March 29, 2026 14:46:06 CEST
                Message:  Use 'odacli describe-prepatchreport -i 3d5f5d96-f566-40c7-ad84-38c436fa3bba' to check details of results

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------

[root@ODA01 ~]#

Which was that time successful:

[root@ODA01 ~]# odacli describe-prepatchreport -i 3d5f5d96-f566-40c7-ad84-38c436fa3bba

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  3d5f5d96-f566-40c7-ad84-38c436fa3bba
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER, STORAGE, RHPGI, GI] to 19.30.0.0.0
                 Status:  SUCCESS
                Created:  March 29, 2026 2:46:06 PM CEST
                 Result:  All pre-checks succeeded

Node Name
---------------
ODA01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Verify All OS patches           Success   No dependencies found for RPMs being
                                          removed, updated and installed. Check
                                          /opt/oracle/dcs/log/jobfiles/
                                          dnfdryrunout_2026-03-29_14-46-
                                          18.0970_2312.log file for more details
Validate command execution      Success   Validated command execution

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Validate command execution      Success   Validated command execution

__STORAGE__
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Patch location validation       Success   Verified patch location
Patch tag validation            Success   Verified patch tag
Storage patch tag validation    Success   Verified storage patch location
Verify ASM disks status         Success   ASM disks are online
Validate command execution      Success   Validated command execution

__RHPGI__
Validate available space        Success   Validated free space under /u01
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate ASM is online          Success   ASM is online
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate DB start dependencies  Success   DBs START dependency check passed
Validate DB stop dependencies   Success   DBs STOP dependency check passed
Validate space for clones       Success   Clones volume is already created
volume
Validate command execution      Success   Validated command execution

[root@ODA01 ~]#

I could then successfully patch the server components part:

[root@ODA01 ~]# /opt/oracle/dcs/bin/odacli update-servercomponents -v 19.30.0.0.0
{
  "jobId" : "fd6ed1d5-1982-46da-8c8c-dfa862f6427a",
  "status" : "Created",
  "message" : "A server update job has been initiated. If a reboot is required, it will be performed as part of the update process. The final status will be provided once the update operation has been completed on all nodes.",
  "reports" : [ ],
  "createTimestamp" : "March 29, 2026 15:00:09 CEST",
  "resourceList" : [ ],
  "description" : "Server Patching to 19.30.0.0.0",
  "updatedTime" : "March 29, 2026 15:00:09 CEST",
  "jobType" : null,
  "cpsMetadata" : null,
  "ecJobId" : null
}
[root@ODA01 ~]#


[root@ODA01 ~]# odacli describe-job -i fd6ed1d5-1982-46da-8c8c-dfa862f6427a

Job details
----------------------------------------------------------------
                     ID:  fd6ed1d5-1982-46da-8c8c-dfa862f6427a
            Description:  Server Patching to 19.30.0.0.0
                 Status:  Success
                Created:  March 29, 2026 15:00:09 CEST
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Modify BM udev rules                     March 29, 2026 15:00:14 CEST             March 29, 2026 15:00:26 CEST             Success
Validate ILOM server reachable           March 29, 2026 15:00:14 CEST             March 29, 2026 15:00:14 CEST             Success
Validating GI user metadata              March 29, 2026 15:00:14 CEST             March 29, 2026 15:00:14 CEST             Success
Stop oakd                                March 29, 2026 15:00:26 CEST             March 29, 2026 15:00:30 CEST             Success
Creating local repository                March 29, 2026 15:00:30 CEST             March 29, 2026 15:00:31 CEST             Success
OSPatchBaseRepo
Updating versionlock plugin              March 29, 2026 15:00:31 CEST             March 29, 2026 15:00:34 CEST             Success
Applying OS Patches                      March 29, 2026 15:00:34 CEST             March 29, 2026 15:07:16 CEST             Success
Applying HMP Patches                     March 29, 2026 15:07:16 CEST             March 29, 2026 15:07:19 CEST             Success
Creating local repository HMPPatchRepo   March 29, 2026 15:07:16 CEST             March 29, 2026 15:07:16 CEST             Success
Patch location validation                March 29, 2026 15:07:19 CEST             March 29, 2026 15:07:19 CEST             Success
Setting SELinux mode                     March 29, 2026 15:07:19 CEST             March 29, 2026 15:07:19 CEST             Success
Oda-hw-mgmt upgrade                      March 29, 2026 15:07:20 CEST             March 29, 2026 15:07:53 CEST             Success
Installing SQLcl software                March 29, 2026 15:07:53 CEST             March 29, 2026 15:07:57 CEST             Success
OSS Patching                             March 29, 2026 15:07:53 CEST             March 29, 2026 15:07:53 CEST             Success
Applying Firmware local Disk Patches     March 29, 2026 15:07:57 CEST             March 29, 2026 15:07:59 CEST             Success
Applying Firmware local Controller Patch March 29, 2026 15:07:59 CEST             March 29, 2026 15:11:23 CEST             Success
Applying Firmware shared Controller      March 29, 2026 15:11:23 CEST             March 29, 2026 15:11:42 CEST             Success
Patch
Checking Ilom patch Version              March 29, 2026 15:11:43 CEST             March 29, 2026 15:11:43 CEST             Success
Disabling IPMI v2                        March 29, 2026 15:11:43 CEST             March 29, 2026 15:11:44 CEST             Success
Patch location validation                March 29, 2026 15:11:43 CEST             March 29, 2026 15:11:43 CEST             Success
Save password in Wallet                  March 29, 2026 15:11:43 CEST             March 29, 2026 15:11:43 CEST             Success
Apply Ilom patch                         March 29, 2026 15:11:44 CEST             March 29, 2026 15:22:19 CEST             Success
Copying Flash Bios to Temp location      March 29, 2026 15:22:19 CEST             March 29, 2026 15:22:19 CEST             Success
Start oakd                               March 29, 2026 15:22:20 CEST             March 29, 2026 15:22:36 CEST             Success
Add SYSNAME in Env                       March 29, 2026 15:22:37 CEST             March 29, 2026 15:22:37 CEST             Success
Cleanup JRE Home                         March 29, 2026 15:22:37 CEST             March 29, 2026 15:22:37 CEST             Success
Starting the clusterware                 March 29, 2026 15:22:37 CEST             March 29, 2026 15:23:29 CEST             Success
Update lvm.conf file                     March 29, 2026 15:23:30 CEST             March 29, 2026 15:23:30 CEST             Success
Generating and saving BOM                March 29, 2026 15:23:31 CEST             March 29, 2026 15:25:37 CEST             Success
Update System full patch version         March 29, 2026 15:23:31 CEST             March 29, 2026 15:23:31 CEST             Success
Update System rebootless patch version   March 29, 2026 15:23:31 CEST             March 29, 2026 15:23:31 CEST             Success
PreRebootNode Actions                    March 29, 2026 15:25:37 CEST             March 29, 2026 15:26:16 CEST             Success
Reboot Node                              March 29, 2026 15:26:16 CEST             March 29, 2026 15:39:48 CEST             Success

[root@ODA01 ~]#

To wrap up…

ODA is an appliance. Any bad behaviour with the rpm and dependencies will bring life more difficult to the consultant coming to patch your ODA. So thanks for sparing a thought for the consultant who will have to patch your ODA before you start playing with yum or dnf on your ODA. 😉

L’article ODA 19.30 prepatch report failing on OS patches est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/oda-19-30-prepatch-report-failing-on-os-patches/feed/ 0 How to patch your ODA to 19.30? https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-30/ https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-30/#respond Fri, 06 Mar 2026 15:10:54 +0000 https://www.dbi-services.com/blog/?p=43377 Introduction Patch 19.30 is now available for Oracle Database Appliance series. Let’s find out what’s new and how to apply this patch. What’s new? The real new feature is the possibility to rollback a server component patch done by an odacli update-servercomponents. Until now, in the rare cases you would need to do a rollback, […]

L’article How to patch your ODA to 19.30? est apparu en premier sur dbi Blog.

]]> Introduction

Patch 19.30 is now available for Oracle Database Appliance series. Let’s find out what’s new and how to apply this patch.

What’s new?

The real new feature is the possibility to rollback a server component patch done by an odacli update-servercomponents. Until now, in the rare cases you would need to do a rollback, you could only rely on ODABR. And guess what? This new rollback feature makes use of ODABR. So keep doing manual ODABR snapshots prior attempting to patch your ODA.

Regarding the odacli update-dcscomponents, it now has a dedicated job engine you can query with a new command odacli describe-admin-job to see the progress. This is useful because this job now lasts longer. It’s always good to know the involved steps and their status of such a process.

This version is 19.30, meaning that bare metal GI stack is still using 19c binaries. 23/26ai databases are still limited to DB Systems, meaning that bare metal databases are limited to 19c.

As you can guess, this patch is mainly a bundle patch for security and bug fixes. A bigger update is expected later this year.

Which ODA is compatible with this 19.30 release?

The latest ODAs X11-HA, X11-L and X11-S are supported, as well as X10, X9-2 and X8-2 series. X7-2 series and older ones are not supported anymore. If you own one from these older generations, you will not be able to patch it anymore. If you’re using X8-2 ODAs, available from late 2019 to mid 2022, the last patch is planned for August 2027.

I still recommend keeping your ODA 7 years, not less, not more. This blog post is still relevant today: https://www.dbi-services.com/blog/why-you-should-consider-keeping-your-oda-more-than-5-years/.

Is this patch a cumulative one?

The rule is now well established: you can apply a patch on top of the four previous ones. 19.30 can then be applied on top of 19.29, 19.28, 19.27 and 19.26. Patching once a year will prevent having to apply 2 or more patches, meaning a longer downtime.

What’s new since 19.29 is the additional monthly system patch. This is a special patch for system only, for those who cannot wait for the global patch to be released. First one was 19.29.0.1.0. 19.30 can be applied on top of it.

In my lab at dbi services, I will use an ODA X8-2M running 19.29 with one DB home, one database and one DB System. The DB System is already running a 26ai database. The given patching procedure should be the same if you come from 19.26 or later.

Is there also a patch for my databases?

Only databases version 19c are supported on bare metal. Patch for 19c is obviously 19.30. For a 26ai database running inside a DB System, you will patch from 23.26.0.0 to 23.26.1.0. This is how the new version numbering works now.

Download the patch and clone files

These files are mandatory:

38776074 => the patch itself
30403673 => the GI clone needed for deploying newer 19c GI version
30403662 => the DB clone for deploying newer version of 19c

These files are optional:

30403643 => ISO file for reimaging, not needed for patching
36524660 => System image for deploying a new 26ai DB System
36524627 => the GI clone needed for deploying/patching to newer 26ai GI version
36524642 => the DB clone for deploying/patching to newer 26ai version
32451228 => The newer system image for 19c DB Systems
38776071 => The patch for 26ai DB Systems

Be sure to choose the very latest 19.30 when downloading some files, patch number is the same for older versions of GI clones, DB clones and ISO files.

Prepare the patching

Before starting, please check these prerequisites:

  • filesystems /, /opt, /u01 and /root have at least 20% of available free space
  • unzip the downloaded patch files
  • additional manually installed rpms must be removed
  • revert profile scripts to default’s one (for grid and oracle users)
  • make sure you’ve planned a sufficient downtime (4+ hours depending on the number of databases and DB Systems)
  • do a sanity reboot before patching to kill zombie processes
  • use ODABR to make snapshots of the important filesystems prior patching: this tool is now included in the software distribution
df -h / /u01 /opt/
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolRoot   30G   13G   16G  46% /
/dev/mapper/VolGroupSys-LogVolU01    59G   23G   34G  40% /u01
/dev/mapper/VolGroupSys-LogVolOpt    69G   34G   33G  52% /opt

cd /opt/dbi
for a in `ls p*1930*.zip` ; do unzip -o $a ; rm -f $a ; done

reboot
...

pvs
  PV           VG          Fmt  Attr PSize   PFree
  /dev/md126p3 VolGroupSys lvm2 a--  446.09g 260.09g

/opt/odabr/odabr backup --snap
INFO: 2026-02-26 11:29:54: Please check the logfile '/opt/odabr/out/log/odabr_24079.log' for more details


│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
 odabr - ODA node Backup Restore - Version: 2.0.2-05
 Copyright 2013, 2025, Oracle and/or its affiliates.
 --------------------------------------------------------
 RACPack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2026-02-26 11:29:54: Checking superuser
INFO: 2026-02-26 11:29:54: Checking Bare Metal

INFO: 2026-02-26 11:29:54: Removing existing LVM snapshots
WARNING: 2026-02-26 11:29:54: LVM snapshot for 'opt' does not exist
WARNING: 2026-02-26 11:29:54: LVM snapshot for 'u01' does not exist
WARNING: 2026-02-26 11:29:54: LVM snapshot for 'root' does not exist

INFO: 2026-02-26 11:29:54: Checking current OS version...
INFO: 2026-02-26 11:29:54: Checking LVM restore backgroud process
INFO: 2026-02-26 11:29:54: Checking LVM size

INFO: 2026-02-26 11:29:54: Boot device backup
INFO: 2026-02-26 11:29:54: Getting EFI device
INFO: 2026-02-26 11:29:54: ...step1 - unmounting EFI
INFO: 2026-02-26 11:29:54: ...step2 - making efi device backup
SUCCESS: 2026-02-26 11:29:57: ...EFI device backup saved as '/opt/odabr/out/hbi/efi.img'
INFO: 2026-02-26 11:29:57: ...step3 - checking EFI device backup
INFO: 2026-02-26 11:29:57: Getting boot device
INFO: 2026-02-26 11:29:57: ...step1 - making boot device backup using tar
SUCCESS: 2026-02-26 11:30:03: ...boot content saved as '/opt/odabr/out/hbi/boot.tar.gz'
INFO: 2026-02-26 11:30:03: ...step2 - unmounting boot
INFO: 2026-02-26 11:30:04: ...step3 - making boot device backup using dd
SUCCESS: 2026-02-26 11:30:09: ...boot device backup saved as '/opt/odabr/out/hbi/boot.img'
INFO: 2026-02-26 11:30:09: ...step4 - mounting boot
INFO: 2026-02-26 11:30:09: ...step5 - mounting EFI
INFO: 2026-02-26 11:30:10: ...step6 - checking boot device backup

INFO: 2026-02-26 11:30:10: Making OCR physical backup
INFO: 2026-02-26 11:30:11: ...ocr backup saved as '/opt/odabr/out/hbi/ocrbackup_24079.bck'
SUCCESS: 2026-02-26 11:30:11: OCR physical backup created successfully
INFO: 2026-02-26 11:30:11: OCR export backup
INFO: 2026-02-26 11:30:12: ...ocr export saved as '/opt/odabr/out/hbi/ocrexport_24079.bck'
SUCCESS: 2026-02-26 11:30:12: OCR export backup created successfully
INFO: 2026-02-26 11:30:12: Saving clusterware patch level as '/opt/odabr/out/hbi/clusterware_patch_level.info'
SUCCESS: 2026-02-26 11:30:12: Clusterware patch level saved successfully

INFO: 2026-02-26 11:30:12: Making LVM snapshot backup
SUCCESS: 2026-02-26 11:30:13: ...snapshot backup for 'opt' created successfully
SUCCESS: 2026-02-26 11:30:14: ...snapshot backup for 'u01' created successfully
SUCCESS: 2026-02-26 11:30:14: ...snapshot backup for 'root' created successfully
SUCCESS: 2026-02-26 11:30:14: LVM snapshots backup done successfully

Version precheck

Start to check the current version of the various components:

odacli describe-component
System Version
--------------
19.29.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.29.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.29.0.0.0          up-to-date

GI                                       19.29.0.0.251021     up-to-date

DB {
     OraDB19000_home9                    19.29.0.0.251021     up-to-date
     [CPROD19]
}

DCSCONTROLLER                            19.29.0.0.0          up-to-date

DCSCLI                                   19.29.0.0.0          up-to-date

DCSAGENT                                 19.29.0.0.0          up-to-date

DCSADMIN                                 19.29.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.22.r165351     up-to-date

BIOS                                     52160100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

List the DB homes, databases, DB Systems and VMs:

odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb     OraDB19000_home9     19.29.0.0.251021     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9     CONFIGURED


odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
976a80f2-4653-469f-8cd4-ddc1a21aff51     CPROD19    SI       19.29.0.0.251021     true    OLTP     EE       odb8     ASM      CONFIGURED   57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb


odacli list-dbsystems
Name                  Shape       GI version          DB info                         Status                  Created                   Updated
--------------------  ----------  ------------------  ------------------------------  ----------------------  ------------------------  ------------------------
dbs-04-tst            dbs2        23.26.0.0.0         23.26(CONFIGURED=1)             CONFIGURED              2026-01-12 10:14:46 CET   2026-01-12 10:45:45 CET

odacli list-vms
No data found for resource VM.

Update the DCS components

Updating DCS components is the first step, after registering the patch file:

odacli update-repository -f /opt/dbi/oda-sm-19.30.0.0.0-260210-server.zip
sleep 30 ; odacli describe-job -i e3ba068f-01db-45c3-949d-b79f43c8d6b7
Job details
----------------------------------------------------------------
                     ID:  e3ba068f-01db-45c3-949d-b79f43c8d6b7
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 11:30:48 CET
                Message:  /opt/dbi/oda-sm-19.30.0.0.0-260210-server.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 11:30:49 CET           February 26, 2026 11:31:10 CET           Success


odacli describe-component
System Version
--------------
19.29.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.29.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.29.0.0.0          19.30.0.0.0

GI                                       19.29.0.0.251021     19.30.0.0.260120

DB {
     OraDB19000_home9                    19.29.0.0.251021     19.30.0.0.260120
     [CPROD19]
}

DCSCONTROLLER                            19.29.0.0.0          19.30.0.0.0

DCSCLI                                   19.29.0.0.0          19.30.0.0.0

DCSAGENT                                 19.29.0.0.0          19.30.0.0.0

DCSADMIN                                 19.29.0.0.0          19.30.0.0.0

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.22.r165351     5.1.5.29.r167438

BIOS                                     52160100             52170100

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

Let’s update the DCS components to 19.30:

odacli update-dcsadmin -v 19.30.0.0.0

sleep 90 ; odacli describe-job -i "c3c278ad-89e6-4c7d-b9fd-27833a187e43"
Job details
----------------------------------------------------------------
                     ID:  c3c278ad-89e6-4c7d-b9fd-27833a187e43
            Description:  DcsAdmin patching to 19.30.0.0.0
                 Status:  Success
                Created:  February 26, 2026 11:32:11 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Dcs-admin upgrade                        February 26, 2026 11:32:11 CET           February 26, 2026 11:32:21 CET           Success
Ping DCS Admin                           February 26, 2026 11:32:21 CET           February 26, 2026 11:33:29 CET           Success


sleep 30 ;  odacli update-dcscomponents -v 19.30.0.0.0
sleep 300 ; odacli describe-admin-job -i 2aeda3f3-df4d-4f7c-a0ce-b57eeab0448b
Job details
----------------------------------------------------------------
                     ID:  2aeda3f3-df4d-4f7c-a0ce-b57eeab0448b
            Description:  Update-dcscomponents to 19.30.0.0.0
                 Status:  Success
                Created:  February 26, 2026 11:34:51 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Pre-checks for update DCS components     February 26, 2026 11:34:57 CET           February 26, 2026 11:35:05 CET           Success
Update DCS components                    February 26, 2026 11:35:05 CET           February 26, 2026 11:35:05 CET           Success
Stop DCS-Agent                           February 26, 2026 11:35:05 CET           February 26, 2026 11:35:05 CET           Success
Update MySql                             February 26, 2026 11:35:05 CET           February 26, 2026 11:35:42 CET           Success
Apply metadata schema changes            February 26, 2026 11:35:42 CET           February 26, 2026 11:35:42 CET           Success
Modify MySQL Metadata                    February 26, 2026 11:35:42 CET           February 26, 2026 11:35:43 CET           Success
Update DCS-Agent                         February 26, 2026 11:35:43 CET           February 26, 2026 11:35:57 CET           Success
Update DCS-Cli                           February 26, 2026 11:35:57 CET           February 26, 2026 11:35:59 CET           Success
Update DCS-Controller                    February 26, 2026 11:35:59 CET           February 26, 2026 11:36:22 CET           Success
Update AHF RPM                           February 26, 2026 11:36:22 CET           February 26, 2026 11:38:41 CET           Success
Reset Keystore password                  February 26, 2026 11:38:41 CET           February 26, 2026 11:39:02 CET           Success
Update HAMI                              February 26, 2026 11:39:02 CET           February 26, 2026 11:39:54 CET           Success
Remove old library files                 February 26, 2026 11:39:54 CET           February 26, 2026 11:39:54 CET           Success
Post DCS update actions                  February 26, 2026 11:39:54 CET           February 26, 2026 11:39:54 CET           Success

System patching

Let’s do the prepatching of the system:

odacli create-prepatchreport -sc -v 19.30.0.0.0

sleep 180 ; odacli describe-prepatchreport -i c06cd4d0-d30c-4063-a3c1-3b86db6625b0

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  c06cd4d0-d30c-4063-a3c1-3b86db6625b0
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER] to 19.30.0.0.0
                 Status:  SUCCESS
                Created:  February 26, 2026 11:40:17 AM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Verify All OS patches           Success   No dependencies found for RPMs being
                                          removed, updated and installed. Check
                                          /opt/oracle/dcs/log/jobfiles/
                                          dnfdryrunout_2026-02-26_11-40-
                                          34.0688_236.log file for more details
Validate command execution      Success   Validated command execution

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Validate DB System AFD state    Success   All DB Systems are on required
                                          versions
Validate command execution      Success   Validated command execution

OK let’s apply the system patch:

odacli update-servercomponents -v 19.30.0.0.0
...

The server will reboot at the end of the patching (it took 40 minutes on my X8-2M). Let’s then check the job:

odacli describe-job -i "8d5b4b63-3a92-46e4-b466-9fd46cdf8b3a"
Job details
----------------------------------------------------------------
                     ID:  8d5b4b63-3a92-46e4-b466-9fd46cdf8b3a
            Description:  Server Patching to 19.30.0.0.0
                 Status:  Success
                Created:  February 26, 2026 11:42:47 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Validating GI user metadata              February 26, 2026 11:42:53 CET           February 26, 2026 11:42:54 CET           Success
Modify BM udev rules                     February 26, 2026 11:42:54 CET           February 26, 2026 11:43:05 CET           Success
Validate ILOM server reachable           February 26, 2026 11:42:54 CET           February 26, 2026 11:42:54 CET           Success
Stop oakd                                February 26, 2026 11:43:05 CET           February 26, 2026 11:43:08 CET           Success
Creating local repository                February 26, 2026 11:43:09 CET           February 26, 2026 11:43:11 CET           Success
OSPatchBaseRepo
Updating versionlock plugin              February 26, 2026 11:43:11 CET           February 26, 2026 11:43:14 CET           Success
Applying OS Patches                      February 26, 2026 11:43:14 CET           February 26, 2026 11:50:26 CET           Success
Applying HMP Patches                     February 26, 2026 11:50:27 CET           February 26, 2026 11:50:30 CET           Success
Creating local repository HMPPatchRepo   February 26, 2026 11:50:27 CET           February 26, 2026 11:50:27 CET           Success
Oda-hw-mgmt upgrade                      February 26, 2026 11:50:31 CET           February 26, 2026 11:51:02 CET           Success
Patch location validation                February 26, 2026 11:50:31 CET           February 26, 2026 11:50:31 CET           Success
Setting SELinux mode                     February 26, 2026 11:50:31 CET           February 26, 2026 11:50:31 CET           Success
Installing SQLcl software                February 26, 2026 11:51:02 CET           February 26, 2026 11:51:06 CET           Success
OSS Patching                             February 26, 2026 11:51:02 CET           February 26, 2026 11:51:02 CET           Success
Applying Firmware local Disk Patches     February 26, 2026 11:51:06 CET           February 26, 2026 11:51:10 CET           Success
Applying Firmware local Controller Patch February 26, 2026 11:51:10 CET           February 26, 2026 11:51:14 CET           Success
Applying Firmware shared Controller      February 26, 2026 11:51:15 CET           February 26, 2026 11:51:19 CET           Success
Patch
Checking Ilom patch Version              February 26, 2026 11:51:19 CET           February 26, 2026 11:51:19 CET           Success
Patch location validation                February 26, 2026 11:51:19 CET           February 26, 2026 11:51:19 CET           Success
Disabling IPMI v2                        February 26, 2026 11:51:20 CET           February 26, 2026 11:51:21 CET           Success
Save password in Wallet                  February 26, 2026 11:51:20 CET           February 26, 2026 11:51:20 CET           Success
Apply Ilom patch                         February 26, 2026 11:51:21 CET           February 26, 2026 12:02:07 CET           Success
Copying Flash Bios to Temp location      February 26, 2026 12:02:07 CET           February 26, 2026 12:02:07 CET           Success
Start oakd                               February 26, 2026 12:02:08 CET           February 26, 2026 12:02:24 CET           Success
Add SYSNAME in Env                       February 26, 2026 12:02:25 CET           February 26, 2026 12:02:25 CET           Success
Cleanup JRE Home                         February 26, 2026 12:02:25 CET           February 26, 2026 12:02:25 CET           Success
Starting the clusterware                 February 26, 2026 12:02:25 CET           February 26, 2026 12:03:59 CET           Success
Update lvm.conf file                     February 26, 2026 12:04:00 CET           February 26, 2026 12:04:00 CET           Success
Generating and saving BOM                February 26, 2026 12:04:01 CET           February 26, 2026 12:04:34 CET           Success
Update System full patch version         February 26, 2026 12:04:01 CET           February 26, 2026 12:04:01 CET           Success
Update System rebootless patch version   February 26, 2026 12:04:01 CET           February 26, 2026 12:04:01 CET           Success
PreRebootNode Actions                    February 26, 2026 12:04:34 CET           February 26, 2026 12:07:10 CET           Success
Reboot Node                              February 26, 2026 12:07:10 CET           February 26, 2026 12:18:17 CET           Success

GI patching

Let’s register the patch file, and do the precheck for GI:

odacli update-repository -f /opt/dbi/odacli-dcs-19.30.0.0.0-260210-GI-19.30.0.0.zip
sleep 70 ; odacli describe-job -i "5699a201-8f50-499c-98a8-18b2e79ca356"

Job details
----------------------------------------------------------------
                     ID:  5699a201-8f50-499c-98a8-18b2e79ca356
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 12:32:04 CET
                Message:  /opt/dbi/odacli-dcs-19.30.0.0.0-260210-GI-19.30.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 12:32:05 CET           February 26, 2026 12:33:14 CET           Success

odacli create-prepatchreport -gi -v 19.30.0.0.0

sleep 180 ; odacli describe-prepatchreport -i ac157412-cfe2-4b2e-ae04-e7082cd4014f

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  ac157412-cfe2-4b2e-ae04-e7082cd4014f
            Description:  Patch pre-checks for [RHPGI, GI] to 19.30.0.0.0
                 Status:  SUCCESS
                Created:  February 26, 2026 12:34:00 PM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__RHPGI__
Validate available space        Success   Validated free space under /u01
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate ASM is online          Success   ASM is online
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate DB start dependencies  Success   DBs START dependency check passed
Validate DB stop dependencies   Success   DBs STOP dependency check passed
Validate space for clones       Success   Clones volume is already created
volume
Validate command execution      Success   Validated command execution

Let’s apply the GI update now:

odacli update-gihome -v 19.30.0.0.0
sleep 500 ;  odacli describe-job -i "857bb637-82ec-4de9-a820-5ab9b895e9f8"

Job details
----------------------------------------------------------------
                     ID:  857bb637-82ec-4de9-a820-5ab9b895e9f8
            Description:  Patch GI with RHP to 19.30.0.0.0
                 Status:  Success
                Created:  February 26, 2026 12:38:29 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Patch GI with RHP to 19.30.0.0.0         February 26, 2026 12:38:44 CET           February 26, 2026 12:46:00 CET           Success
Registering image                        February 26, 2026 12:38:45 CET           February 26, 2026 12:38:45 CET           Success
Registering working copy                 February 26, 2026 12:38:45 CET           February 26, 2026 12:38:46 CET           Success
Starting the clusterware                 February 26, 2026 12:38:45 CET           February 26, 2026 12:38:45 CET           Success
Creating GI home directories             February 26, 2026 12:38:46 CET           February 26, 2026 12:38:46 CET           Success
Extract GI clone                         February 26, 2026 12:38:46 CET           February 26, 2026 12:38:46 CET           Success
Provisioning Software Only GI with RHP   February 26, 2026 12:38:46 CET           February 26, 2026 12:38:46 CET           Success
Registering image                        February 26, 2026 12:38:46 CET           February 26, 2026 12:38:46 CET           Success
Patch GI with RHP                        February 26, 2026 12:39:20 CET           February 26, 2026 12:45:25 CET           Success
Set CRS ping target                      February 26, 2026 12:45:25 CET           February 26, 2026 12:45:25 CET           Success
Updating .bashrc                         February 26, 2026 12:45:25 CET           February 26, 2026 12:45:26 CET           Success
Updating GI home metadata                February 26, 2026 12:45:26 CET           February 26, 2026 12:45:26 CET           Success
Updating GI home version                 February 26, 2026 12:45:26 CET           February 26, 2026 12:45:31 CET           Success
Updating All DBHome version              February 26, 2026 12:45:31 CET           February 26, 2026 12:45:36 CET           Success
Starting the clusterware                 February 26, 2026 12:45:56 CET           February 26, 2026 12:45:56 CET           Success
Validate ACFS resources are running      February 26, 2026 12:45:56 CET           February 26, 2026 12:45:57 CET           Success
Validate GI availability                 February 26, 2026 12:45:56 CET           February 26, 2026 12:45:56 CET           Success
Validate DB System VMs states            February 26, 2026 12:45:57 CET           February 26, 2026 12:45:58 CET           Success
Patch CPU Pools distribution             February 26, 2026 12:45:58 CET           February 26, 2026 12:45:58 CET           Success
Patch DB System domain config            February 26, 2026 12:45:58 CET           February 26, 2026 12:45:58 CET           Success
Patch KVM CRS type                       February 26, 2026 12:45:58 CET           February 26, 2026 12:45:58 CET           Success
Patch VM vDisks CRS dependencies         February 26, 2026 12:45:58 CET           February 26, 2026 12:45:58 CET           Success
Save custom VNetworks to storage         February 26, 2026 12:45:58 CET           February 26, 2026 12:45:59 CET           Success
Add network filters to DB Systems        February 26, 2026 12:45:59 CET           February 26, 2026 12:46:00 CET           Success
Create network filters                   February 26, 2026 12:45:59 CET           February 26, 2026 12:45:59 CET           Success
Patch DB Systems custom scale metadata   February 26, 2026 12:46:00 CET           February 26, 2026 12:46:00 CET           Success
Patch DB Systems vDisks CRS dependencies February 26, 2026 12:46:00 CET           February 26, 2026 12:46:00 CET           Success

No reboot is needed for this patch.

Check the versions

odacli describe-component
System Version
--------------
19.30.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home9                    19.29.0.0.251021     19.30.0.0.260120
     [CPROD19]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167438     up-to-date

BIOS                                     52170100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

Patching the storage

Patching the storage is only needed if describe-component tells you that you’re not up-to-date. On my X8-2M, it wasn’t needed. If your ODA need the storage patch, it’s easy:

odacli update-storage -v 19.30.0.0.0
odacli describe-job -i ...

The server will reboot once done.

Patching the DB homes

It’s now time to patch the DB home and the database on my ODA. Let’s first unzip and register the patch file in the repository:

odacli update-repository -f /opt/dbi/odacli-dcs-19.30.0.0.0-260210-DB-19.30.0.0.zip 
sleep 60; odacli describe-job -i 5690c811-9030-427e-82f1-caeeba236329

Job details
----------------------------------------------------------------
                     ID:  5690c811-9030-427e-82f1-caeeba236329
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 12:54:57 CET
                Message:  /opt/dbi/odacli-dcs-19.30.0.0.0-260210-DB-19.30.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 12:54:57 CET           February 26, 2026 12:55:50 CET           Success



odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb     OraDB19000_home9     19.29.0.0.251021     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9     CONFIGURED

Let’s check if the patch can be applied:

odacli create-prepatchreport -d -i 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb -v 19.30.0.0.0

sleep 600; odacli describe-prepatchreport -i 3522c79c-7444-44d7-9422-9d1daab161d2

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  3522c79c-7444-44d7-9422-9d1daab161d2
            Description:  Patch pre-checks for [DB, RHPDB, ORACHKDB] to 19.30.0.0.0: DbHome is OraDB19000_home9
                 Status:  FAILED
                Created:  February 26, 2026 12:56:59 PM CET
                 Result:  One or more pre-checks failed for [ORACHK, DB]

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate data corruption in     Failed    DCS-10315 - Patch described in My
patching                                  Oracle Support Note KB867473 must be
                                          applied.
Validate DB Home ID             Success   Validated DB Home ID:
                                          57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate there is usable        Success   Successfully validated Oracle Base
space under oracle base                   usable space
Validate glogin.sql file        Success   Successfully verified glogin.sql
                                          won't break patching
Validate dbHomesOnACFS          Success   User has configured disk group for
configured                                Database homes on ACFS
Validate Oracle base            Success   Successfully validated Oracle Base
Is DB clone available           Success   Successfully validated clone file
                                          exists
Validate command execution      Success   Validated command execution

__RHPDB__
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Failed    DCS-10702 - ORAchk validation failed:
                                          .
Validate command execution      Success   Validated command execution
Verify the Fast Recovery Area   Failed    AHF-2929: FRA space management
(FRA) has reclaimable space               problem file types are present
                                          without an RMAN backup completion
                                          within the last 7 days

I need to fix 2 problems. The first one is a bug that appeared in 19.29, let’s download and unzip the patch. Be careful because this patch is available for multiple versions: you will need the one for the version you’re currently using (19.29 in my case).

su - oracle
unzip -d /home/oracle /opt/dbi/p38854064_1929000DBRU_Linux-x86-64.zip
Archive:  /opt/dbi/p38854064_1929000DBRU_Linux-x86-64.zip
   creating: /home/oracle/38854064/
   creating: /home/oracle/38854064/files/
   creating: /home/oracle/38854064/files/lib/
   creating: /home/oracle/38854064/files/lib/libserver19.a/
  inflating: /home/oracle/38854064/files/lib/libserver19.a/kjfc.o
  inflating: /home/oracle/38854064/README.txt
   creating: /home/oracle/38854064/etc/
   creating: /home/oracle/38854064/etc/config/
  inflating: /home/oracle/38854064/etc/config/inventory.xml
  inflating: /home/oracle/38854064/etc/config/actions.xml
  inflating: /home/oracle/PatchSearch.xml

Let’s stop the database and apply this patch:

. oraenv <<< CPROD19
srvctl stop database -db CPROD19_S1

cd 38854064
$ORACLE_HOME/OPatch/opatch apply
Oracle Interim Patch Installer version 12.2.0.1.47
Copyright (c) 2026, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9/oraInst.loc
OPatch version    : 12.2.0.1.47
OUI version       : 12.2.0.7.0
Log file location : /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9/cfgtoollogs/opatch/opatch2026-02-26_14-29-22PM_1.log

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   38854064

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9')


Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '38854064' to OH '/u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9'

Patching component oracle.rdbms, 19.0.0.0.0...
Patch 38854064 successfully applied.
Log file location: /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9/cfgtoollogs/opatch/opatch2026-02-26_14-29-22PM_1.log

OPatch succeeded.

srvctl start database -db CPROD19_S1

Second problem is because my database doesn’t have a proper backup strategy, let’s then remove the useless archivelogs:

rman target /
delete force noprompt archivelog all;
exit;
exit

Now let’s retry the precheck:

odacli create-prepatchreport -d -i 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb -v 19.30.0.0.0

odacli describe-prepatchreport -i b932011d-cfcf-402d-901e-5c7eac888f1f

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  b932011d-cfcf-402d-901e-5c7eac888f1f
            Description:  Patch pre-checks for [DB, RHPDB, ORACHKDB] to 19.30.0.0.0: DbHome is OraDB19000_home9
                 Status:  FAILED
                Created:  February 26, 2026 3:01:14 PM CET
                 Result:  One or more pre-checks failed for [ORACHK, DB]

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate data corruption in     Failed    DCS-10315 - Patch described in My
patching                                  Oracle Support Note KB867473 must be
                                          applied.
Validate DB Home ID             Success   Validated DB Home ID:
                                          57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
Validate patching tag           Success   Validated patching tag: 19.30.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate there is usable        Success   Successfully validated Oracle Base
space under oracle base                   usable space
Validate glogin.sql file        Success   Successfully verified glogin.sql
                                          won't break patching
Validate dbHomesOnACFS          Success   User has configured disk group for
configured                                Database homes on ACFS
Validate Oracle base            Success   Successfully validated Oracle Base
Is DB clone available           Success   Successfully validated clone file
                                          exists
Validate command execution      Success   Validated command execution

__RHPDB__
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Failed    DCS-10702 - ORAchk validation failed:
                                          .
Validate command execution      Success   Validated command execution
Verify the Fast Recovery Area   Failed    AHF-2929: FRA space management
(FRA) has reclaimable space               problem file types are present
                                          without an RMAN backup completion
                                          within the last 7 days

This view doesn’t look updated, and the ODA documentation tells us that updating the DB home will need to be forced, let’s do that:

odacli update-dbhome -i 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb -v 19.30.0.0.0 --force

sleep 600;  odacli describe-job -i "bd511055-7a35-45b4-b9f2-3a003c7ecb31"
Job details
----------------------------------------------------------------
                     ID:  bd511055-7a35-45b4-b9f2-3a003c7ecb31
            Description:  DB Home Patching to 19.30.0.0.0: Home ID is 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
                 Status:  Success
                Created:  February 26, 2026 15:08:46 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Creating wallet for DB Client            February 26, 2026 15:09:33 CET           February 26, 2026 15:09:33 CET           Success
Patch databases by RHP - [CPROD19]       February 26, 2026 15:09:33 CET           February 26, 2026 15:16:44 CET           Success
Updating database metadata               February 26, 2026 15:16:44 CET           February 26, 2026 15:16:44 CET           Success
Upgrade pwfile to 12.2                   February 26, 2026 15:16:44 CET           February 26, 2026 15:16:47 CET           Success
Set log_archive_dest for Database        February 26, 2026 15:16:47 CET           February 26, 2026 15:16:50 CET           Success
Populate PDB metadata                    February 26, 2026 15:16:51 CET           February 26, 2026 15:16:52 CET           Success
Generating and saving BOM                February 26, 2026 15:16:52 CET           February 26, 2026 15:17:33 CET           Success
TDE parameter update                     February 26, 2026 15:18:06 CET           February 26, 2026 15:18:06 CET           Success

Everything is now OK.

Let’s check the DB homes and databases:

odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
d3b5fa9c-ad85-46c3-b11a-cd264978b653     OraDB19000_home10    19.30.0.0.260120     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_10    CONFIGURED
57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb     OraDB19000_home9     19.29.0.0.251021     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9     CONFIGURED

odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
976a80f2-4653-469f-8cd4-ddc1a21aff51     CPROD19    SI       19.30.0.0.260120     true    OLTP     EE       odb8     ASM      CONFIGURED   d3b5fa9c-ad85-46c3-b11a-cd264978b653

Let’s now remove the old DB home. Note that DB homes are not protected by ODABR, I would recommend doing a backup before removing an old DB home:

tar czf /backup/`hostname -s`_dbhome_9.tgz /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9

odacli delete-dbhome -i 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
sleep 40 ; odacli describe-job -i 4589a4d7-6986-4e16-818c-78d585f44443
Job details
----------------------------------------------------------------
                     ID:  4589a4d7-6986-4e16-818c-78d585f44443
            Description:  Database Home OraDB19000_home9 Deletion with ID 57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
                 Status:  Success
                Created:  February 26, 2026 15:24:44 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Setting up SSH equivalence               February 26, 2026 15:24:44 CET           February 26, 2026 15:24:44 CET           Success
Setting up SSH equivalence               February 26, 2026 15:24:44 CET           February 26, 2026 15:24:44 CET           Success
Validate DB Home                         February 26, 2026 15:24:44 CET           February 26, 2026 15:24:44 CET           Success
57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb
for deletion
Deleting DB Home by RHP                  February 26, 2026 15:24:45 CET           February 26, 2026 15:25:21 CET           Success

Cleanse the old patches

Let’s remove the previous patch from the repository:

odacli cleanup-patchrepo -comp all -v 19.29.0.0.0

odacli describe-job -i "a9e29414-8f12-4b55-a6d4-9ad82e9a4c74"
Job details
----------------------------------------------------------------
                     ID:  a9e29414-8f12-4b55-a6d4-9ad82e9a4c74
            Description:  Cleanup patchrepos
                 Status:  Success
                Created:  February 26, 2026 15:29:46 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Cleanup Repository                       February 26, 2026 15:29:46 CET           February 26, 2026 15:29:47 CET           Success
Cleanup old ASR rpm                      February 26, 2026 15:29:47 CET           February 26, 2026 15:29:47 CET           Success

Old GI binaries are still using space in /u01, it’s better to remove them manually:

du -hs /u01/app/19.*
14G	/u01/app/19.29.0.0
14G	/u01/app/19.30.0.0

rm -rf /u01/app/19.29.0.0

I would recommend doing a reboot to check if everything run fine. But let’s first check the components:

odacli describe-component
System Version
--------------
19.30.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home10                   19.30.0.0.260120     up-to-date
     [CPROD19]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167438     up-to-date

BIOS                                     52170100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

reboot
...

ps -ef | grep pmon
grid      8292     1  0 15:37 ?        00:00:00 asm_pmon_+ASM1
grid     11539     1  0 15:37 ?        00:00:00 apx_pmon_+APX1
oracle   20494     1  0 15:38 ?        00:00:00 ora_pmon_CPROD19
root     23559 23363  0 15:39 pts/1    00:00:00 grep --color=auto pmon

Everything is fine.

Post-patching tasks

Dont’ forget these post-patching tasks:

  • remove the ODABR snapshots
  • add your additional RPMs
  • put back your profile scripts for grid and oracle users
  • check if monitoring still works
/opt/odabr/odabr infosnap

│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
 odabr - ODA node Backup Restore - Version: 2.0.2-06
 Copyright 2013, 2025, Oracle and/or its affiliates.
 --------------------------------------------------------
 RACPack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│


LVM snap name         status                COW Size              Data%
-------------         ----------            ----------            ------
root_snap             active                30.00 GiB             6.08%
opt_snap              active                70.00 GiB             11.41%
u01_snap              active                60.00 GiB             25.30%

/opt/odabr/odabr delsnap

INFO: 2026-02-26 15:39:47: Please check the logfile '/opt/odabr/out/log/odabr_23962.log' for more details


│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
 odabr - ODA node Backup Restore - Version: 2.0.2-06
 Copyright 2013, 2025, Oracle and/or its affiliates.
 --------------------------------------------------------
 RACPack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│

INFO: 2026-02-26 15:39:47: Removing LVM snapshots
INFO: 2026-02-26 15:39:47: ...removing LVM snapshot for 'opt'
SUCCESS: 2026-02-26 15:39:48: ...snapshot for 'opt' removed successfully
INFO: 2026-02-26 15:39:48: ...removing LVM snapshot for 'u01'
SUCCESS: 2026-02-26 15:39:48: ...snapshot for 'u01' removed successfully
INFO: 2026-02-26 15:39:48: ...removing LVM snapshot for 'root'
SUCCESS: 2026-02-26 15:39:48: ...snapshot for 'root' removed successfully
SUCCESS: 2026-02-26 15:39:48: LVM snapshots removed successfully

Patching the DB System

If you use DB Systems on your ODA, meaning that some of your databases are running in dedicated VMs, you will need to apply the patch inside each DB System. If you’re using 26ai, you first need to register the new clones in the repository before connecting to your DB System:

odacli update-repository -f /opt/dbi/odacli-dcs-23.26.1.0.0-260211-GI-23.26.1.0.zip
sleep 30 ; odacli describe-job -i 8612ef6a-7df4-419d-8d05-176e11126f48

Job details
----------------------------------------------------------------
                     ID:  8612ef6a-7df4-419d-8d05-176e11126f48
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 15:44:00 CET
                Message:  /opt/dbi/odacli-dcs-23.26.1.0.0-260211-GI-23.26.1.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 15:44:00 CET           February 26, 2026 15:44:13 CET           Success


odacli update-repository -f /opt/dbi/odacli-dcs-23.26.1.0.0-260211-DB-23.26.1.0.zip
sleep 30 ; odacli describe-job -i 9dd624f2-9048-4897-b63b-400b955c803c

Job details
----------------------------------------------------------------
                     ID:  9dd624f2-9048-4897-b63b-400b955c803c
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 15:45:12 CET
                Message:  /opt/dbi/odacli-dcs-23.26.1.0.0-260211-DB-23.26.1.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 15:45:12 CET           February 26, 2026 15:45:40 CET           Success

odacli update-repository -f /opt/dbi/oda-sm-23.26.1.0.0-260211-server.zip
sleep 20 ; odacli describe-job -i 6a24023f-de2e-4481-b9c4-d8511d54be48

Job details
----------------------------------------------------------------
                     ID:  6a24023f-de2e-4481-b9c4-d8511d54be48
            Description:  Repository Update
                 Status:  Success
                Created:  February 26, 2026 15:59:07 CET
                Message:  /opt/dbi/oda-sm-23.26.1.0.0-260211-server.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             February 26, 2026 15:59:08 CET           February 26, 2026 15:59:30 CET           Success

odacli list-availablepatches
-------------------- ------------------------- ------------------------- ------------------------------
ODA Release Version  Supported DB Versions     Available DB Versions     Supported Platforms
-------------------- ------------------------- ------------------------- ------------------------------
19.30.0.0.0          23.26.1.0.0               23.26.1.0.0               DB System
                     21.8.0.0.221018           Clone not available       DB System
                     19.30.0.0.260120          19.30.0.0.260120          DB System, Bare Metal

Applying the patch is done the same way you’ve done it on bare metal, but here you need to specify the 23.26.1.0.0 version:

ssh dbs-04-tst

odacli update-dcsadmin -v 23.26.1.0.0

sleep 60 ; odacli describe-job -i 4b83ab57-ccb1-4f9f-8c70-572b45ada49b

Job details
----------------------------------------------------------------
                     ID:  4b83ab57-ccb1-4f9f-8c70-572b45ada49b
            Description:  DcsAdmin patching to 23.26.1.0.0
                 Status:  Success
                Created:  March 05, 2026 10:07:44 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Dcs-admin upgrade                        March 05, 2026 10:07:45 CET              March 05, 2026 10:07:59 CET              Success
Ping DCS Admin                           March 05, 2026 10:07:59 CET              March 05, 2026 10:09:07 CET              Success

sleep 30 ;  odacli update-dcscomponents -v 23.26.1.0.0
{
  "jobId" : "cb674b3e-d6eb-4351-be39-0f19b8c56f9d",
  "status" : "Success",
  "message" : "Update-dcscomponents is successful on all the node(s): DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata schema update is done. Script '/opt/oracle/dcs/log/jobfiles/cb674b3e-d6eb-4351-be39-0f19b8c56f9d/apply_metadata_change.sh' ran successfully. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. ahf RPM upgrade is successful.  Successfully reset the Keystore password. HAMI RPM is already updated.  Removed old Libs Successfully ran setupAgentAuth.sh ",
  "reports" : null,
  "createTimestamp" : "March 05, 2026 10:10:13 AM CET",
  "description" : "Update-dcscomponents job completed and is not part of Agent job list",
  "updatedTime" : "March 05, 2026 10:18:38 AM CET",
  "jobType" : null,
  "externalRequestId" : null,
  "action" : null
}

odacli describe-admin-job -i cb674b3e-d6eb-4351-be39-0f19b8c56f9d
odacli: 'describe-admin-job' is not an odacli command.
usage: odacli	[-h/--help]
		<category> [-h/--help]
		<operation> [-h/--help]
		<command> [-h/--help]
		<command> [<args>]

Note that there is no describe-admin-job feature on DB Systems.

odacli create-prepatchreport -sc -v 23.26.1.0.0

sleep 20 ; odacli describe-prepatchreport -i 1b104d06-bc0c-45b8-ab25-b5b6a102a857

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  1b104d06-bc0c-45b8-ab25-b5b6a102a857
            Description:  Patch pre-checks for [OS, ORACHKSERVER, SERVER] to 23.26.1.0.0
                 Status:  SUCCESS
                Created:  March 05, 2026 10:59:34 CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbs-04-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 23.26.1.0.0.
Is patch location available     Success   Patch location is available.
Verify All OS patches           Success   No dependencies found for RPMs being
                                          removed, updated and installed. Check
                                          /opt/oracle/dcs/log/jobfiles/
                                          dnfdryrunout_2026-03-05_10-59-
                                          50.0718_832.log file for more details
Validate there is usable        Success   Successfully validated
space under repo volume                   /opt/oracle/dcs/repo usable space
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Enable support for Multi-DB     Success   No need to convert the DB System
Validate DB System AFD state    Success   AFD is not configured
Validate there is usable        Success   Successfully validated
space under repo volume                   /opt/oracle/dcs/repo usable space
Validate command execution      Success   Validated command execution


odacli update-servercomponents -v 23.26.1.0.0

The DB System will reboot.

odacli describe-job -i 2b4da73a-7f64-48e0-af76-a1d687a0169f

Job details
----------------------------------------------------------------
                     ID:  2b4da73a-7f64-48e0-af76-a1d687a0169f
            Description:  Server Patching to 23.26.1.0.0
                 Status:  Success
                Created:  March 05, 2026 11:04:19 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Deactivate Unit[dnf-makecache.timer]     March 05, 2026 11:04:21 CET              March 05, 2026 11:04:21 CET              Success
Validating GI user metadata              March 05, 2026 11:04:21 CET              March 05, 2026 11:04:21 CET              Success
Deactivate Unit[kdump.service]           March 05, 2026 11:04:22 CET              March 05, 2026 11:04:23 CET              Success
Modify DBVM udev rules                   March 05, 2026 11:04:23 CET              March 05, 2026 11:04:34 CET              Success
Creating local repository                March 05, 2026 11:04:34 CET              March 05, 2026 11:04:37 CET              Success
OSPatchBaseRepo
Updating versionlock plugin              March 05, 2026 11:04:37 CET              March 05, 2026 11:04:41 CET              Success
Applying OS Patches                      March 05, 2026 11:04:41 CET              March 05, 2026 11:07:33 CET              Success
Creating local repository HMPPatchRepo   March 05, 2026 11:07:34 CET              March 05, 2026 11:07:34 CET              Success
Applying HMP Patches                     March 05, 2026 11:07:35 CET              March 05, 2026 11:07:38 CET              Success
Patch location validation                March 05, 2026 11:07:39 CET              March 05, 2026 11:07:39 CET              Success
Setting SELinux mode                     March 05, 2026 11:07:39 CET              March 05, 2026 11:07:39 CET              Success
Oda-hw-mgmt upgrade                      March 05, 2026 11:07:40 CET              March 05, 2026 11:08:08 CET              Success
Installing SQLcl software                March 05, 2026 11:08:08 CET              March 05, 2026 11:08:13 CET              Success
Cleanup JRE Home                         March 05, 2026 11:08:14 CET              March 05, 2026 11:08:14 CET              Success
Generating and saving BOM                March 05, 2026 11:08:17 CET              March 05, 2026 11:08:24 CET              Success
Update System full patch version         March 05, 2026 11:08:17 CET              March 05, 2026 11:08:17 CET              Success
Update System rebootless patch version   March 05, 2026 11:08:17 CET              March 05, 2026 11:08:17 CET              Success
PreRebootNode Actions                    March 05, 2026 11:08:24 CET              March 05, 2026 11:08:25 CET              Success
Reboot Node                              March 05, 2026 11:08:25 CET              March 05, 2026 11:09:59 CET              Success

odacli create-prepatchreport -gi -v 23.26.1.0.0

sleep 240 ; odacli describe-prepatchreport -i dd5d216b-d1bc-44cf-bcf8-381da0729469

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  dd5d216b-d1bc-44cf-bcf8-381da0729469
            Description:  Patch pre-checks for [RHPGI, GI] to 23.26.1.0.0
                 Status:  SUCCESS
                Created:  March 05, 2026 11:13:21 CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbs-04-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__RHPGI__
Validate available space        Success   Validated free space under /u01
Evaluate GI patching            Success   Successfully validated GI patching
Validate there is usable        Success   Successfully validated
space under repo volume                   /opt/oracle/dcs/repo usable space
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Validate there is usable        Success   Successfully validated
space under repo volume                   /opt/oracle/dcs/repo usable space
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 23.26.1.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate BM versions            Success   Validated BM server components
                                          versions
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate command execution      Success   Validated command execution


odacli update-gihome -v 23.26.1.0.0

sleep 600 ; odacli describe-job -i c93f84fc-5cb2-41bb-9f23-f7ce22b9f5de

Job details
----------------------------------------------------------------
                     ID:  c93f84fc-5cb2-41bb-9f23-f7ce22b9f5de
            Description:  Patch GI with RHP to 23.26.1.0.0
                 Status:  Success
                Created:  March 05, 2026 11:22:47 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Patch GI with RHP to 23.26.1.0.0         March 05, 2026 11:22:59 CET              March 05, 2026 11:27:54 CET              Success
Starting the clusterware                 March 05, 2026 11:22:59 CET              March 05, 2026 11:22:59 CET              Success
Creating GI home directories             March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Extract GI clone                         March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Provisioning Software Only GI with RHP   March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Registering image                        March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Registering image                        March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Registering working copy                 March 05, 2026 11:23:01 CET              March 05, 2026 11:23:01 CET              Success
Patch GI with RHP                        March 05, 2026 11:23:47 CET              March 05, 2026 11:26:58 CET              Success
Set CRS ping target                      March 05, 2026 11:26:58 CET              March 05, 2026 11:26:59 CET              Success
Updating .bashrc                         March 05, 2026 11:26:59 CET              March 05, 2026 11:26:59 CET              Success
Updating GI home metadata                March 05, 2026 11:26:59 CET              March 05, 2026 11:27:00 CET              Success
Updating GI home version                 March 05, 2026 11:27:00 CET              March 05, 2026 11:27:04 CET              Success
Updating All DBHome version              March 05, 2026 11:27:04 CET              March 05, 2026 11:27:08 CET              Success
Patch DB System on BM                    March 05, 2026 11:27:48 CET              March 05, 2026 11:27:54 CET              Success
Starting the clusterware                 March 05, 2026 11:27:48 CET              March 05, 2026 11:27:48 CET              Success


odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
9116603b-3b5e-4e92-aa63-baad8ae1d6a8     OraDB23000_home1     23.26.0.0.0          EE         /u01/app/oracle/product/23.0.0.0/dbhome_1                CONFIGURED

odacli create-prepatchreport -d -i 9116603b-3b5e-4e92-aa63-baad8ae1d6a8 -v 23.26.1.0.0

sleep 600 ;  odacli describe-prepatchreport -i bb16e390-3dcb-4ea0-b8c5-0c22f38ba271

odacli describe-prepatchreport -i bb16e390-3dcb-4ea0-b8c5-0c22f38ba271

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  bb16e390-3dcb-4ea0-b8c5-0c22f38ba271
            Description:  Patch pre-checks for [DB, RHPDB, ORACHKDB] to 23.26.1.0.0: DbHome is OraDB23000_home1
                 Status:  FAILED
                Created:  March 05, 2026 11:59:29 CET
                 Result:  One or more pre-checks failed for [ORACHK]

Node Name
---------------
dbs-04-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID             Success   Validated DB Home ID:
                                          9116603b-3b5e-4e92-aa63-baad8ae1d6a8
Validate patching tag           Success   Validated patching tag: 23.26.1.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate there is usable        Success   Successfully validated Oracle Base
space under oracle base                   usable space
Validate glogin.sql file        Success   Successfully verified glogin.sql
                                          won't break patching
Is DB clone available           Success   Successfully validated clone file
                                          exists
Validate command execution      Success   Validated command execution

__RHPDB__
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Failed    DCS-10702 - ORAchk validation failed:
                                          .
Validate command execution      Success   Validated command execution
Verify the Fast Recovery Area   Failed    AHF-2929: FRA space management
(FRA) has reclaimable space               problem file types are present
                                          without an RMAN backup completion
                                          within the last 7 days

The failure is similar to the one I had when patching the bare metal DB home, but I can ignore this and update the DB home with the force option:

odacli update-dbhome -i 9116603b-3b5e-4e92-aa63-baad8ae1d6a8 -v 23.26.1.0.0 -f

sleep 1200 ; odacli describe-job -i 4fc89556-2f7c-4e5b-a12f-55e32d7e748a

Job details
----------------------------------------------------------------
                     ID:  4fc89556-2f7c-4e5b-a12f-55e32d7e748a
            Description:  DB Home Patching to 23.26.1.0.0: Home ID is 9116603b-3b5e-4e92-aa63-baad8ae1d6a8
                 Status:  Success
                Created:  March 05, 2026 13:36:42 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Creating wallet for DB Client            March 05, 2026 13:37:21 CET              March 05, 2026 13:37:21 CET              Success
Patch databases by RHP - [CTEST26]       March 05, 2026 13:37:21 CET              March 05, 2026 13:54:26 CET              Success
Updating database metadata               March 05, 2026 13:54:26 CET              March 05, 2026 13:54:27 CET              Success
Upgrade pwfile to 12.2                   March 05, 2026 13:54:27 CET              March 05, 2026 13:54:32 CET              Success
Set log_archive_dest for Database        March 05, 2026 13:54:32 CET              March 05, 2026 13:54:37 CET              Success
Populate PDB metadata                    March 05, 2026 13:54:38 CET              March 05, 2026 13:54:39 CET              Success
Generating and saving BOM                March 05, 2026 13:54:39 CET              March 05, 2026 13:55:08 CET              Success
TDE parameter update                     March 05, 2026 13:55:44 CET              March 05, 2026 13:55:44 CET              Success

odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
276bf458-db09-4c9a-9cd9-a821e5274fb0     CTEST26    SI       23.26.1.0.0          true    OLTP     EE       odb2     ASM      CONFIGURED   9c51039d-ccba-4508-b879-a81b8c18d46a

odacli delete-dbhome -i 9116603b-3b5e-4e92-aa63-baad8ae1d6a8
sleep 100 ; odacli describe-job -i 0994b96e-e174-4776-8699-f179c1d89af0

Job details
----------------------------------------------------------------
                     ID:  0994b96e-e174-4776-8699-f179c1d89af0
            Description:  Database Home OraDB23000_home1 Deletion with ID 9116603b-3b5e-4e92-aa63-baad8ae1d6a8
                 Status:  Success
                Created:  March 05, 2026 13:58:36 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Setting up SSH equivalence               March 05, 2026 13:58:36 CET              March 05, 2026 13:58:36 CET              Success
Setting up SSH equivalence               March 05, 2026 13:58:36 CET              March 05, 2026 13:58:37 CET              Success
Validate DB Home                         March 05, 2026 13:58:36 CET              March 05, 2026 13:58:36 CET              Success
9116603b-3b5e-4e92-aa63-baad8ae1d6a8
for deletion
Deleting DB Home by RHP                  March 05, 2026 13:58:38 CET              March 05, 2026 13:59:15 CET              Success

odacli describe-component
System Version
--------------
23.26.1.0.0

System Node Name
----------------
dbs-04-tst

Local System Version
--------------------
23.26.1.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      23.26.1.0.0          up-to-date

GI                                       23.26.1.0.0          up-to-date

DB {
     OraDB23000_home2                    23.26.1.0.0          up-to-date
     [CTEST26]
}

DCSCONTROLLER                            23.26.1.0.0          up-to-date

DCSCLI                                   23.26.1.0.0          up-to-date

DCSAGENT                                 23.26.1.0.0          up-to-date

DCSADMIN                                 23.26.1.0.0          up-to-date

OS                                       8.10                 up-to-date

Finally, let’s remove obsolete GI binaries:

du -hs /u01/app/23.26.*
3.9G	/u01/app/23.26.0.0
3.6G	/u01/app/23.26.1.0

rm -rf /u01/app/23.26.0.0/

Don’t forget to apply this procedure to the other DB Systems.

Conclusion

Applying this patch is OK, as soon as everything is clean and under control. When patching, only use the force option when you’re sure that you know what you’re doing. As always, patching an ODA with DB Systems can take quite a big amount of time, mainly depending on the number of DB Systems.

L’article How to patch your ODA to 19.30? est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-30/feed/ 0 Migrate your database to 26ai on ODA https://www.dbi-services.com/blog/migrate-your-database-to-26ai-on-oda/ https://www.dbi-services.com/blog/migrate-your-database-to-26ai-on-oda/#respond Mon, 12 Jan 2026 14:36:59 +0000 https://www.dbi-services.com/blog/?p=42419 Introduction Oracle 26ai is not yet available on Bare Metal ODA, but it will probably be in a few weeks. For now, 26ai is available as a DB System on Oracle Database Appliance, meaning that it’s already possible to test this new version. Let’s try to migrate a 19c Bare Metal PDB to a 26ai […]

L’article Migrate your database to 26ai on ODA est apparu en premier sur dbi Blog.

]]> Introduction

Oracle 26ai is not yet available on Bare Metal ODA, but it will probably be in a few weeks. For now, 26ai is available as a DB System on Oracle Database Appliance, meaning that it’s already possible to test this new version. Let’s try to migrate a 19c Bare Metal PDB to a 26ai DB System using 3 different methods.

Prerequisites

26ai is only available with the minimum release 19.29, meaning on ODA X8-2 and newer models. You will need to patch your ODA to 19.29 first if you want to test 26ai. 26ai is only compatible with multitenant architecture, so another prerequisite is to migrate your non-container databases to PDBs. If you’re still using non-CDB architecture, and among these 3 methods, only the first one will work for you.

The recommended patching method as told by Oracle is using AutoUpgrade: please refer to this blog post if you want more details.

On my side, I will explore these 3 other possibilities:

  • Datapump Export/Import
  • PDB copy through a DB link
  • PDB unplug/plug

Download the needed files for 26ai DB System

These files are needed for creating a 26ai DB System:

36524660 => System image for 26ai DB Systems
36524627 => the GI clone needed for deploying newer 26ai GI version
36524642 => the DB clone for deploying new version 26ai

Be sure to choose the very latest 19.29 patch when downloading these files, download link from MOS will first propose older versions.

My 19c source database

My source database is a 19c container database with one PDB running on an ODA X8-2M. I want to migrate this PDB to 26ai:

su - oracle
 . oraenv <<< CPROD19
ORACLE_SID = [CPROD19] ? The Oracle base remains unchanged with value /u01/app/odaorabase/oracle
sqlplus / as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Jan 8 09:53:43 2026
Version 19.29.0.0.0

Copyright (c) 1982, 2025, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.29.0.0.0

show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PRODDB01                       READ WRITE NO

exit

Provision a new 26ai DB System

It’s not yet possible to deploy a 26ai container database on Bare Metal, so let’s create a DB System for testing this migration. First unzip and register the VM template, GI 26ai and DB 26ai:

cd /opt/dbi

unzip -o p36524660_1929000_Linux-x86-64.zip
unzip -o p36524627_1929000_Linux-x86-64.zip
unzip -o p36524642_1929000_Linux-x86-64.zip

odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-ODAVM-19.29.0.0.zip
odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-GI-23.26.0.0.zip
odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-DB-23.26.0.0.zip

sleep 30 ; odacli list-jobs | tail -n 4
d4700315-db8c-4522-af55-0fddd262bfe4     Repository Update                                                           2025-12-08 15:20:14 CET             Success
ba7d452f-e03d-46d0-a607-fd7c758cd1b1     Repository Update                                                           2025-12-08 15:20:59 CET             Success
bfc102d1-985b-4792-8054-03709aa8d949     Repository Update                                                           2025-12-08 15:21:20 CET             Success

odacli describe-dbsystem-image | grep 23.26
DBVM                  23.26.0.0.0           23.26.0.0.0
GI                    23.26.0.0.0           23.26.0.0.0
DB                    23.26.0.0.0           23.26.0.0.0

A CPU pool is needed for DB Systems, if you don’t have one. A 2-core CPU pool is enough:

odacli create-cpupool -c 2 -n cpupool2c -dbs

Now let’s create a json file, then create a new DB System based on this file:

cat /opt/dbi/create_dbs-04-tst-cdb.json
{
    "system": {
        "name": "dbs-04-tst",
        "shape": "dbs2",
        "systemPassword": "***********",
        "timeZone": "Europe/Zurich",
        "diskGroup": "DATA",
        "cpuPoolName": "cpupool2c",
        "enableRoleSeparation": true,
        "customRoleSeparation": {
            "groups": [
                {
                    "name": "oinstall",
                    "id": 1001,
                    "role": "oinstall"
                },
                {
                    "name": "dbaoper",
                    "id": 1002,
                    "role": "dbaoper"
                },
                {
                    "name": "dba",
                    "id": 1003,
                    "role": "dba"
                },
                {
                    "name": "asmadmin",
                    "id": 1004,
                    "role": "asmadmin"
                },
                {
                    "name": "asmoper",
                    "id": 1005,
                    "role": "asmoper"
                },
                {
                    "name": "asmdba",
                    "id": 1006,
                    "role": "asmdba"
                }
            ],
            "users": [
                {
                    "name": "grid",
                    "id": 1001,
                    "role": "gridUser"
                },
                {
                    "name": "oracle",
                    "id": 1000,
                    "role": "oracleUser"
                }
            ]
        }
    },
    "database": {
        "name": "CTEST26",
        "uniqueName": "CTEST26_S1",
        "domainName": "dbi-lab.ch",
        "adminPassword": "**********",
        "version": "23.26.0.0.0",
        "edition": "EE",
        "type": "SI",
        "dbClass": "OLTP",
        "shape": "odb2",
        "role": "PRIMARY",
        "enableDbConsole": false,
        "enableFlashStorage": false,
        "redundancy": null,
        "characterSet": {
            "characterSet": "AL32UTF8",
            "nlsCharacterset": "AL16UTF16",
            "dbTerritory": "AMERICA",
            "dbLanguage": "AMERICAN"
        },
        "rmanBackupPassword": null,
        "enableTDE": false,
        "isCdb": true,
        "pdbName": "pdb1",
        "pdbAdminUser": "pdbadmin",
        "tdePassword": null
    },
    "network": {
        "domainName": "dbi-lab.ch",
        "ntpServers": [
            "85.195.224.28"
        ],
        "dnsServers": [
            "10.16.0.250"
        ],
        "nodes": [
            {
                "name": "dbs-04-tst",
                "ipAddress": "10.16.0.247",
                "netmask": "255.255.255.0",
                "gateway": "10.16.0.1",
                "number": 0
            }
        ],
        "publicVNetwork": "pubnet"
    },
    "grid": {
        "language": "en"
    }
}

odacli create-dbsystem -p /opt/dbi/create_dbs-04-tst-cdb.json
Enter password for system "dbs-04-tst":
Retype password for system "dbs-04-tst":
Enter administrator password for DB "CTEST26":
Retype administrator password for DB "CTEST26":
...

35 minutes later, my new DB System is ready to use:

odacli describe-job -i c79ecbbf-dd0c-4b75-ba10-249c78b25f33

Job details
----------------------------------------------------------------
                     ID:  c79ecbbf-dd0c-4b75-ba10-249c78b25f33
            Description:  DB System dbs-04-tst creation
                 Status:  Success
                Created:  January 08, 2026 10:23:39 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Create DB System metadata                January 08, 2026 10:23:39 CET            January 08, 2026 10:23:41 CET            Success
Persist new DB System                    January 08, 2026 10:23:41 CET            January 08, 2026 10:23:41 CET            Success
Remove unused network interfaces         January 08, 2026 10:23:41 CET            January 08, 2026 10:23:41 CET            Success
Validate DB System creation              January 08, 2026 10:23:41 CET            January 08, 2026 10:23:46 CET            Success
Create the infra quota group             January 08, 2026 10:23:46 CET            January 08, 2026 10:23:50 CET            Success
Setup DB System environment              January 08, 2026 10:23:46 CET            January 08, 2026 10:23:46 CET            Success
Create DB System ASM volume              January 08, 2026 10:23:50 CET            January 08, 2026 10:24:01 CET            Success
Create DB System ACFS filesystem         January 08, 2026 10:24:01 CET            January 08, 2026 10:24:13 CET            Success
Create DB System ACFS mount point        January 08, 2026 10:24:01 CET            January 08, 2026 10:24:01 CET            Success
Create DB System VM ACFS snapshots	 January 08, 2026 10:24:13 CET            January 08, 2026 10:24:28 CET            Success
Setup ASM client cluster config          January 08, 2026 10:24:28 CET            January 08, 2026 10:24:30 CET            Success
Create temporary SSH key pair            January 08, 2026 10:24:30 CET            January 08, 2026 10:24:31 CET            Success
Create BM certificates in ACFS location  January 08, 2026 10:24:31 CET            January 08, 2026 10:24:32 CET            Success
Create DB System cloud-init config	 January 08, 2026 10:24:32 CET            January 08, 2026 10:24:32 CET            Success
Provision DB System VM(s)                January 08, 2026 10:24:32 CET            January 08, 2026 10:24:35 CET            Success
Create u01 vDisk for DB System           January 08, 2026 10:24:35 CET            January 08, 2026 10:24:36 CET            Success
Add DB System to Clusterware             January 08, 2026 10:24:36 CET            January 08, 2026 10:24:37 CET            Success
Attach disks to DB System                January 08, 2026 10:24:36 CET            January 08, 2026 10:24:36 CET            Success
Attach u01 vDisk to DB System            January 08, 2026 10:24:36 CET            January 08, 2026 10:24:36 CET            Success
Add extra PCI slots                      January 08, 2026 10:24:37 CET            January 08, 2026 10:24:37 CET            Success
Create Network Filters                   January 08, 2026 10:24:37 CET            January 08, 2026 10:24:37 CET            Success
Export clones repository                 January 08, 2026 10:24:37 CET            January 08, 2026 10:24:37 CET            Success
Start DB System                          January 08, 2026 10:24:37 CET            January 08, 2026 10:24:40 CET            Success
Wait DB System VM first boot             January 08, 2026 10:24:40 CET            January 08, 2026 10:26:49 CET            Success
Setup Mutual TLS (mTLS)                  January 08, 2026 10:26:49 CET            January 08, 2026 10:26:51 CET            Success
Cleanup temporary SSH key pair           January 08, 2026 10:26:51 CET            January 08, 2026 10:26:52 CET            Success
Copy ASM client cluster config           January 08, 2026 10:26:51 CET            January 08, 2026 10:26:51 CET            Success
Delete Mutual TLS (mTLS) Artifacts	 January 08, 2026 10:26:51 CET            January 08, 2026 10:26:51 CET            Success
Wait DB System VM DCS Agent readiness    January 08, 2026 10:26:52 CET            January 08, 2026 10:27:12 CET            Success
Add Network Filters                      January 08, 2026 10:27:12 CET            January 08, 2026 10:27:14 CET            Success
Install DB System                        January 08, 2026 10:27:14 CET            January 08, 2026 10:53:57 CET            Success
Wait DB System VM DCS Agent readiness    January 08, 2026 10:53:57 CET            January 08, 2026 10:55:58 CET            Success
Set DB System as configured              January 08, 2026 10:55:58 CET            January 08, 2026 10:55:58 CET            Success

odacli list-dbsystems

Name                  Shape       GI version          DB info                         Status                  Created                   Updated
--------------------  ----------  ------------------  ------------------------------  ----------------------  ------------------------  ------------------------
dbs-04-tst            dbs2        23.26.0.0.0         23.26(CONFIGURED=1)             CONFIGURED              2026-01-08 10:23:39 CET   2026-01-08 10:55:58 CET

Migration using datapump

Let’s consider a classic migration using datapump. This tool has been the official one for migration for nearly 2 decades now.

Let’s first create a new PDB inside the container database of the new DB System:

ssh root@dbs-04-tst
su - oracle
. oraenv <<< CTEST26
sqlplus / as sysdba
show pdbs
    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 PDB1 			  READ WRITE NO

alter pluggable database PDB1 close immediate;
Pluggable database altered.

drop pluggable database PDB1 including datafiles;
Pluggable database dropped.

create pluggable database MIG26DTPUMP admin user pdbadmin identified by "*********";
Pluggable database created.

alter pluggable database MIG26DTPUMP open;
Pluggable database altered.

alter pluggable database MIG26DTPUMP save state;
Pluggable database altered.

show pdbs
    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 4 MIG26DTPUMP			  READ WRITE NO

alter session set container=MIG26DTPUMP;

create tablespace users datafile size 100M autoextend on maxsize 10G;
Tablespace created.

exit

On source database, non-CDB or PDB it doesn’t matter, export data using datapump:

su - oracle
. oraenv <<< CPROD19
export ORACLE_PDB_SID=PRODDB01
sqlplus / as sysdba
create or replace directory mig as '/backup/dumps/';
col current_scn for 999999999999
select current_scn from v$database;
    CURRENT_SCN
---------------
        6292616
exit
expdp system directory=MIG full=y dumpfile=PRODDB01_20260112.dmp logfile=exp_PRODDB01_20260112.log flashback_scn=6292616

Export: Release 19.0.0.0.0 - Production on Mon Jan 12 10:36:52 2026
Version 19.29.0.0.0

Copyright (c) 1982, 2019, Oracle and/or its affiliates.  All rights reserved.
Password:

Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Starting "SYSTEM"."SYS_EXPORT_FULL_01":  system/******** directory=MIG full=y dumpfile=PRODDB01_20260112.dmp logfile=exp_PRODDB01_20260112.log flashback_scn=6292616
Processing object type DATABASE_EXPORT/EARLY_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/TABLE_DATA
Processing object type DATABASE_EXPORT/NORMAL_OPTIONS/VIEWS_AS_TABLES/TABLE_DATA
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/INDEX/STATISTICS/INDEX_STATISTICS
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/STATISTICS/TABLE_STATISTICS
Processing object type DATABASE_EXPORT/STATISTICS/MARKER
Processing object type DATABASE_EXPORT/PRE_SYSTEM_IMPCALLOUT/MARKER
Processing object type DATABASE_EXPORT/PRE_INSTANCE_IMPCALLOUT/MARKER
Processing object type DATABASE_EXPORT/TABLESPACE
...

. . exported "WMSYS"."WM$METADATA_MAP"                   11.16 KB       0 rows
. . exported "MDSYS"."RDF_PARAM$"                        6.515 KB       3 rows
. . exported "SYS"."AUDTAB$TBS$FOR_EXPORT"               5.960 KB       2 rows
. . exported "SYS"."DBA_SENSITIVE_DATA"                  8.070 KB       0 rows
. . exported "SYS"."DBA_TSDP_POLICY_PROTECTION"          7.656 KB       0 rows
. . exported "SYS"."NACL$_ACE_EXP"                       9.812 KB       0 rows
. . exported "SYS"."NACL$_HOST_EXP"                      6.921 KB       1 rows
. . exported "SYS"."NACL$_WALLET_EXP"                    5.929 KB       0 rows
. . exported "DBI"."CONSULTANTS"                         5.937 KB       1 rows
Master table "SYSTEM"."SYS_EXPORT_FULL_01" successfully loaded/unloaded
******************************************************************************
Dump file set for SYSTEM.SYS_EXPORT_FULL_01 is:
  /backup/dumps/PRODDB01_20260112.dmp
Job "SYSTEM"."SYS_EXPORT_FULL_01" successfully completed at Mon Jan 12 10:38:52 2026 elapsed 0 00:01:54


ls -lrth /backup/dumps/*20260112*
-rw-r-----. 1 oracle asmadmin  34M Jan 12 10:38 /backup/dumps/PRODDB01_20260112.dmp
-rw-r--r--. 1 oracle asmadmin 9.8K Jan 12 10:38 /backup/dumps/exp_PRODDB01_20260112.log

On target database, let’s do the import:

ssh root@dbs-04-tst
su - oracle
. oraenv <<< CTEST26
export ORACLE_PDB_SID=MIG26DTPUMP
sqlplus / as sysdba
create or replace directory mig as '/backup/dumps/';
exit

vi /backup/dumps/imp_PRODDB01.par
directory=MIG
schemas=dbi
dumpfile=PRODDB01_20260112.dmp
logfile=imp_PRODDB01_to_MIG26DTPUMP_20260112.log
exclude=statistics

impdp system parfile=/backup/dumps/imp_PRODDB01.par
Import: Release 23.26.0.0.0 - for Oracle Cloud and Engineered Systems on Mon Jan 12 10:51:39 2026
Version 23.26.0.0.0

Copyright (c) 1982, 2025, Oracle and/or its affiliates.  All rights reserved.
Password:

Connected to: Oracle AI Database 26ai Enterprise Edition Release 23.26.0.0.0 - for Oracle Cloud and Engineered Systems
Master table "SYSTEM"."SYS_IMPORT_SCHEMA_01" successfully loaded/unloaded
import done in AL32UTF8 character set and AL16UTF16 NCHAR character set
export done in WE8MSWIN1252 character set and AL16UTF16 NCHAR character set
Warning: possible data loss in character set conversions
Starting "SYSTEM"."SYS_IMPORT_SCHEMA_01":  system/******** parfile=/backup/dumps/imp_PRODDB01.par
Processing object type DATABASE_EXPORT/SCHEMA/USER
Processing object type DATABASE_EXPORT/SCHEMA/GRANT/SYSTEM_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/ROLE_GRANT
Processing object type DATABASE_EXPORT/SCHEMA/DEFAULT_ROLE
Processing object type DATABASE_EXPORT/SCHEMA/PROCACT_SCHEMA/LOGREP
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE
Processing object type DATABASE_EXPORT/SCHEMA/TABLE/TABLE_DATA
. . imported "DBI"."CONSULTANTS"                           5.9 KB       1 rows
Job "SYSTEM"."SYS_IMPORT_SCHEMA_01" successfully completed at Mon Jan 12 10:51:54 2026 elapsed 0 00:00:08

Let’s now compute statistics for my schema:

sqlplus / as sysdba
exec dbms_stats.gather_schema_stats(ownname => 'DBI');
PL/SQL procedure successfully completed.
exit;

Let’s do a quick check on data:

sqlplus / as sysdba
select start_date from dbi.consultants where name like 'DUB%';

START_DAT
---------
04-SEP-17

exit

That’s fine.

Pros & cons

  • + Same procedure as previous migrations
  • + Reorganize data and indexes
  • + Keep the source database untouched
  • – Rather slow when database is big

If the source database is a PDB, it can be copied through a DB link, this is a very convenient way of migrating a database.

First, it’s weird but the PDB administrator of the source DB must have the CREATE PLUGGABLE DATABASE privilege:

su - oracle
. oraenv <<< CPROD19
sqlplus / as sysdba
alter session set container=PRODDB01;
grant create pluggable database to pdbadmin;
exit

On the DB System, let’s create a new TNS entry for the 19c PDB:

ssh dbs-04-tst
su - oracle
. oraenv <<< CTEST26
vi $ORACLE_HOME/network/admin/tnsnames.ora

...
PRODDB01 =
  (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = dbioda01.dbi-lab.ch)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = proddb01.dbi-lab.ch)
    )
  )

On the target CDB, create a new database link to the source database:

CREATE DATABASE LINK PRODDB01 CONNECT TO PDBADMIN IDENTIFIED BY "*********"  USING 'PRODDB01';

select count(*) from dual@PRODDB01;
  COUNT(*)
----------
         1

Now let’s copy the source PDB under a new name on the 26ai container:

create pluggable database MIG26DBLINK from PRODDB01@PRODDB01;
Pluggable database created.

alter pluggable database MIG26DBLINK open;

Opening this PDB will take some minutes because it automatically triggers the upgrade from 19c to 26ai:

tail -n 15 --f /u01/app/oracle/diag/rdbms/ctest26_s1/CTEST26/trace/alert_CTEST26.log 
...
MIG26DBLINK(3):Starting Upgrade on PDB Open
2026-01-12T10:57:49.606651+01:00
MIG26DBLINK(3):alter pluggable database application APP$CDB$CATALOG begin install '19.0.0.0.0' on error capture
MIG26DBLINK(3):Completed: alter pluggable database application APP$CDB$CATALOG begin install '19.0.0.0.0' on error capture
MIG26DBLINK(3):alter pluggable database application APP$CDB$CATALOG end install '19.0.0.0.0'
MIG26DBLINK(3):Completed: alter pluggable database application APP$CDB$CATALOG end install '19.0.0.0.0'
MIG26DBLINK(3):alter pluggable database application APP$CDB$CATALOG begin upgrade
  '19.0.0.0.0' to '23.0.0.0.0.partial' on error capture
MIG26DBLINK(3):Completed: alter pluggable database application APP$CDB$CATALOG begin upgrade
  '19.0.0.0.0' to '23.0.0.0.0.partial' on error capture
...
MIG26DBLINK(3):SERVER COMPONENT id=CATJAVA: status=UPGRADED, version=23.26.0.0.0, timestamp=2026-01-12 11:09:54 Container=MIG26DBLINK Id=3
2026-01-12T11:09:58.200636+01:00
MIG26DBLINK(3):Resize operation completed for file# 19, fname +DATA/CTEST26_S1/482EE16D86417106E063F700240A6555/DATAFILE/undotbs1.294.1222340161, old size 450560K, new size 455680K
2026-01-12T11:09:59.496448+01:00
MIG26DBLINK(3):Resize operation completed for file# 19, fname +DATA/CTEST26_S1/482EE16D86417106E063F700240A6555/DATAFILE/undotbs1.294.1222340161, old size 455680K, new size 460800K
2026-01-12T11:10:26.701609+01:00
MIG26DBLINK(3):oracle :
MIG26DBLINK(3):SERVER COMPONENT id=XDB: status=UPGRADED, version=23.26.0.0.0, timestamp=2026-01-12 11:10:26 Container=MIG26DBLINK Id=3
2026-01-12T11:10:36.378304+01:00
MIG26DBLINK(3):Resize operation completed for file# 19, fname +DATA/CTEST26_S1/482EE16D86417106E063F700240A6555/DATAFILE/undotbs1.294.1222340161, old size 460800K, new size 465920K
2026-01-12T11:10:45.077998+01:00
MIG26DBLINK(3):Resize operation completed for file# 19, fname +DATA/CTEST26_S1/482EE16D86417106E063F700240A6555/DATAFILE/undotbs1.294.1222340161, old size 465920K, new size 471040K
...
MIG26DBLINK(3):***************************************************************
MIG26DBLINK(3):WARNING: Pluggable Database MIG26DBLINK with pdb id - 3 is
MIG26DBLINK(3):         altered with errors or warnings. Please look into
MIG26DBLINK(3):         PDB_PLUG_IN_VIOLATIONS view for more details.
MIG26DBLINK(3):***************************************************************
MIG26DBLINK(3):--ATTENTION--
MIG26DBLINK(3):Errors reported while opening PDB (ContainerId: 3) and have been recorded in pdb_alert$ table.
MIG26DBLINK(3) Error Violation: SQL Patch, Cause: '23.26.0.0.0 Release_Update 2509270520' is installed in the CDB but no release updates are installed in the PDB, Action: Call datapatch to install in the PDB or the CDB
2026-01-12T11:13:31.632629+01:00
MIG26DBLINK(3):Opening pdb with no Resource Manager plan active
Violations: Type: 1, Count: 1
Violations: Type: 2, Count: 2
Completed: Pluggable database MIG26DBLINK opened read write
Completed: alter pluggable database MIG26DBLINK open

After a couple of minutes, the PDB is opened with errors:

Warning: PDB altered with errors.
exit

Let’s apply datapatch on this very PDB:

$ORACLE_HOME/OPatch/datapatch -pdbs MIG26DBLINK
SQL Patching tool version 23.26.0.0.0 Lim on Mon Jan 12 11:14:55 2026
Copyright (c) 2012, 2025, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/product/23.0.0.0/dbhome_1/cfgtoollogs/sqlpatch/sqlpatch_sid_CTEST26_ts_2026_01_12_11_14_55_pid_5074/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
  No interim patches found

Current state of release update SQL patches:
  Binary registry:
    23.26.0.0.0 Release_Update 250927052030: Installed
  PDB MIG26DBLINK:
    Applied 23.26.0.0.0 Release_Update 250927052030 successfully

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: MIG26DBLINK
    No interim patches need to be rolled back
    No release update patches need to be installed
    No interim patches need to be applied

Bypass install queue:
  For the following PDBs: MIG26DBLINK
    No interim rollbacks will bypass install
    Patch 38404116 (Database Release Update : 23.26.0.0.0 (38404116) Gold Image): will bypass install
      Apply from 23.26.0.0.0 Release_Update 250927052030 to 23.26.0.0.0 Release_Update 250927052030
    No interim applys will bypass install


Installation queue after removing bypass entries...
Installation queue:
  For the following PDBs: MIG26DBLINK
    No interim patches need to be rolled back
    No release update patches need to be installed
    No interim patches need to be applied


Processing bypass install queue:
  Patch 38404116 apply (pdb MIG26DBLINK): SUCCESS (bypass_install)

SQL Patching tool complete on Mon Jan 12 11:15:03 2026

Everything is fine, now let’s close and open this PDB in normal mode:

sqlplus / as sysdba
show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE YES
	 4 MIG26DTPUMP			  READ WRITE NO

alter pluggable database MIG26DBLINK close immediate;
alter pluggable database MIG26DBLINK open;
alter pluggable database MIG26DBLINK save state;

show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE NO
	 4 MIG26DTPUMP			  READ WRITE NO

Let’s do a quick data check:

sqlplus / as sysdba
alter session set container=MIG26DBLINK;
select start_date from dbi.consultants where name like 'DUB%';
START_DAT
---------
04-SEP-17

exit

Database is ready to use.

Pros & cons

  • + Maybe the easiest way of migrating a PDB
  • + Keep the source database untouched
  • + Rather fast
  • + No downtime of the source database
  • – Need to apply datapatch

Migration using unplug/plug features

If the source database is a PDB, it can be unplugged and plugged back to another container. This is how multitenant architecture was presented when it first came to 12cR1.

Let’s unplug the source PDB:

su - oracle
. oraenv <<< CPROD19
sqlplus / as sysdba
alter pluggable database PRODDB01 close immediate;
Pluggable database altered.

alter pluggable database PRODDB01 unplug into '/backup/dumps/PRODDB01.xml';
Pluggable database altered.

exit

Now let’s plug this PDB to the new 26ai container:

ssh root@dbs-04-tst
su - oracle
. oraenv <<< CTEST26

sqlplus / as sysdba

show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE NO
	 4 MIG26DTPUMP			  READ WRITE NO

create pluggable database MIG26REPLUG using '/backup/dumps/PRODDB01.xml' COPY;
create pluggable database MIG26REPLUG using '/backup/dumps/PRODDB01.xml' COPY
*
ERROR at line 1:
ORA-19505: failed to identify file
"+DATA/CPROD19_S1/450AA2873245B6F5E063F100240AC35A/DATAFILE/system.302.122233608
1"
ORA-17503: ksfdopn:2 Failed to open file
+DATA/CPROD19_S1/450AA2873245B6F5E063F100240AC35A/DATAFILE/system.302.1222336081
ORA-15260: permission denied on ASM disk group
Help: https://docs.oracle.com/error-help/db/ora-19505/

Files from the source database are not readable by this DB System. Let’s change the permission on these files from the Bare Metal ASM instance:

exit
exit
exit
su - grid

asmcmd ls -lt +DATA/CPROD19_S1/450AA2873245B6F5E063F100240AC35A/DATAFILE/* --permission
Type      Redund  Striped  Time             Sys  User                                     Group  Permission  Name
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-------  USERS.297.1222336081
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-------  UNDOTBS1.306.1222336081
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-------  SYSTEM.302.1222336081
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-------  SYSAUX.305.1222336081

asmcmd ls -lt +DATA/CTEST26_S1/DATAFILE/* --permission
Type      Redund  Striped  Time             Sys  User                                     Group  Permission  Name
DATAFILE  MIRROR  COARSE   JAN 12 10:00:00  Y    oracle@b94b488f82b1cf5cbf58f54162aafac0          rw-------  USERS.299.1222339507
DATAFILE  MIRROR  COARSE   JAN 12 10:00:00  Y    oracle@b94b488f82b1cf5cbf58f54162aafac0          rw-------  UNDOTBS1.693.1222338993
DATAFILE  MIRROR  COARSE   JAN 12 10:00:00  Y    oracle@b94b488f82b1cf5cbf58f54162aafac0          rw-------  SYSTEM.906.1222338951
DATAFILE  MIRROR  COARSE   JAN 12 10:00:00  Y    oracle@b94b488f82b1cf5cbf58f54162aafac0          rw-------  SYSAUX.832.1222338987

asmcmd chmod a+r +DATA/CPROD19_S1/450AA2873245B6F5E063F100240AC35A/DATAFILE/*
asmcmd ls -lt +DATA/CPROD19_S1/450AA2873245B6F5E063F100240AC35A/DATAFILE/* --permission
Type      Redund  Striped  Time             Sys  User                                     Group  Permission  Name
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-r--r--  USERS.297.1222342153
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-r--r--  UNDOTBS1.301.1222341995
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-r--r--  SYSTEM.305.1222341995
DATAFILE  MIRROR  COARSE   JAN 12 11:00:00  Y    oracle@e7d9d36f6b157fdabf89026bcf48d757          rw-r--r--  SYSAUX.302.1222341995

exit

ssh root@dbs-04-tst
su - oracle
. oraenv <<< CTEST26

sqlplus / as sysdba

create pluggable database MIG26REPLUG using '/backup/dumps/PRODDB01.xml' COPY;
Pluggable database created.

show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE NO
	 4 MIG26DTPUMP			  READ WRITE NO
	 6 MIG26REPLUG			  MOUNTED

alter pluggable database MIG26REPLUG open;

It will also take a couple of minutes for this upgrade to 26ai. You will see corresponding traces in the alert.log file of the 26ai container:

tail -f /u01/app/oracle/diag/rdbms/ctest26_s1/CTEST26/trace/alert_CTEST26.log
...
2026-01-12T11:34:21.075364+01:00
MIG26REPLUG(6):Starting Upgrade on PDB Open
...
MIG26REPLUG(6):***************************************************************
MIG26REPLUG(6):WARNING: Pluggable Database MIG26REPLUG with pdb id - 6 is
MIG26REPLUG(6):         altered with errors or warnings. Please look into
MIG26REPLUG(6):         PDB_PLUG_IN_VIOLATIONS view for more details.
MIG26REPLUG(6):***************************************************************
MIG26REPLUG(6):--ATTENTION--
MIG26REPLUG(6):Errors reported while opening PDB (ContainerId: 6) and have been recorded in pdb_alert$ table.
MIG26REPLUG(6) Error Violation: SQL Patch, Cause: '23.26.0.0.0 Release_Update 2509270520' is installed in the CDB but no release updates are installed in the PDB, Action: Call datapatch to install in the PDB or the CDB
2026-01-12T11:49:56.836062+01:00
MIG26REPLUG(6):Opening pdb with no Resource Manager plan active
Violations: Type: 1, Count: 1
Violations: Type: 2, Count: 1
Completed: Pluggable database MIG26REPLUG opened read write
Completed: alter pluggable database MIG26REPLUG open

Let’s have a look at my PDBs and apply the datapatch on MIG26REPLUG:

show pdbs
    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE NO
	 4 MIG26DTPUMP			  READ WRITE NO
	 6 MIG26REPLUG			  READ WRITE YES

exit

$ORACLE_HOME/OPatch/datapatch -pdbs MIG26REPLUG

SQL Patching tool version 23.26.0.0.0 Lim on Mon Jan 12 11:51:42 2026
Copyright (c) 2012, 2025, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/product/23.0.0.0/dbhome_1/cfgtoollogs/sqlpatch/sqlpatch_sid_CTEST26_ts_2026_01_12_11_51_42_pid_24026/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
       that are in an open state, no patches will be applied to closed PDBs.
       Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
       (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of interim SQL patches:
  No interim patches found

Current state of release update SQL patches:
  Binary registry:
    23.26.0.0.0 Release_Update 250927052030: Installed
  PDB MIG26REPLUG:
    Applied 23.26.0.0.0 Release_Update 250927052030 successfully

Adding patches to installation queue and performing prereq checks...done
Installation queue:
  For the following PDBs: MIG26REPLUG
    No interim patches need to be rolled back
    No release update patches need to be installed
    No interim patches need to be applied

Bypass install queue:
  For the following PDBs: MIG26REPLUG
    No interim rollbacks will bypass install
    Patch 38404116 (Database Release Update : 23.26.0.0.0 (38404116) Gold Image): will bypass install
      Apply from 23.26.0.0.0 Release_Update 250927052030 to 23.26.0.0.0 Release_Update 250927052030
    No interim applys will bypass install


Installation queue after removing bypass entries...
Installation queue:
  For the following PDBs: MIG26REPLUG
    No interim patches need to be rolled back
    No release update patches need to be installed
    No interim patches need to be applied


Processing bypass install queue:
  Patch 38404116 apply (pdb MIG26REPLUG): SUCCESS (bypass_install)

SQL Patching tool complete on Mon Jan 12 11:51:50 2026

Now let’s restart the PDB in normal mode:

sqlplus / as sysdba

alter pluggable database MIG26REPLUG close immediate;
alter pluggable database MIG26REPLUG open;
alter pluggable database MIG26REPLUG save state;

show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 MIG26DBLINK			  READ WRITE NO
	 4 MIG26DTPUMP			  READ WRITE NO
	 6 MIG26REPLUG			  READ WRITE NO

This PDB is now running on 26ai.

Let’s check my consultants table:

alter session set container=MIG26REPLUG;
select start_date from dbi.consultants where name like 'DUB%';
START_DAT
---------
04-SEP-17

exit

Pros & cons

  • + Easy migration
  • + Preserve source files with the copy option
  • + Rather fast
  • – Need a downtime of the source PDB
  • – Reuse of source database is only possible by unplugging and plugging back the PDB with the xml file

Conclusion

There are multiple ways of migrating to 26ai. There is quite a lot of new features coming with 26ai. You will use them or not, but trying to migrate to this version is something to start thinking about in 2026.

L’article Migrate your database to 26ai on ODA est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/migrate-your-database-to-26ai-on-oda/feed/ 0 How to patch your ODA to 19.29? https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-29/ https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-29/#respond Mon, 08 Dec 2025 16:32:04 +0000 https://www.dbi-services.com/blog/?p=41797 How to patch your Oracle Database Appliance to 19.29

L’article How to patch your ODA to 19.29? est apparu en premier sur dbi Blog.

]]> Introduction

Patch 19.29 is now available for Oracle Database Appliance series. Let’s find out what’s new and how to apply this patch.

What’s new?

The most important new component is probably database 26ai (as a DB System only). But don’t get fooled, 26ai is only the new name of 23ai (23.26). It doesn’t matter, and this is nice to see this latest version coming to the ODA. 26ai is also coming to all on-premise systems.

The other change that comes with this version is the way of applying the system patch. Several years ago, you should have applied the server patch and the GI patch separately. It was then grouped within the same update-server command we’re using since years. Now the split makes a come back and update-server is replaced by update-servercomponents for system stuff, and update-gihome for GI. Also note that updade-dcsagent vanished in 19.27.

Improvements have also been made on the security side, with SE Linux now being enabled. Regarding DB Systems, CPU and memory allocation is now more flexible, this is great. Note that a DB System is no more limited to 1 container database since 19.23, making virtualization on ODA more appealing than before.

My overall feeling about this patch is “maturity and stability”. It’s all that we need for this kind of platform.

Which ODA is compatible with this 19.29 release?

The latest ODAs X11-HA, X11-L and X11-S are supported, as well as X10, X9-2 and X8-2 series. X7-2 series and older ones are not supported anymore. If you own one from these older generations, you should have a renewal plan for the coming months. I still recommend keeping your ODA 7 years, not less, not more. This blog post is still relevant today: https://www.dbi-services.com/blog/why-you-should-consider-keeping-your-oda-more-than-5-years/.

Is this patch a cumulative one?

The rule is now well established: you can apply a patch on top of the four previous ones. 19.29 can then be applied on top of 19.28, 19.27, 19.26 and 19.25. It’s why it makes sense patching once a year: this is the perfect balance between moderate security needs and ease of patching.

In my lab, I will use an ODA X8-2M running 19.28 with one DB home, one database and one DB System. This procedure should apply the same way on your ODA a soon as you’re using 19.25 or later.

Is there also a patch for my databases?

Only databases version 19c are supported for bare metal. You should be able to patch a 23ai database running as a DB System to 26ai, but you’d probably better deploying a new DB System and unplug/plug your PDBs into the brand new DB System.

Download the patch and clone files

These files are mandatory:

  • 38427251 => the patch itself
  • 30403673 => the GI clone needed for deploying newer 19c GI version
  • 30403662 => the DB clone for deploying newer version of 19c

These files are optional:

  • 30403643 => ISO file for reimaging, not needed for patching
  • 36524660 => System image for 26ai DB Systems
  • 36524627 => the GI clone needed for deploying new 26ai GI version
  • 36524642 => the DB clone for deploying new 26ai DB version
  • 32451228 => The newer system image for 19c DB Systems

Be sure to choose the very latest 19.29 when downloading some files, download link from MOS will first propose older versions for GI clones, DB clones and ISO files.

Prepare the patching

Before starting, please check these prerequisites:

  • filesystems /, /opt, /u01 and /root have at least 20% of available free space
  • additional manually installed rpms must be removed
  • revert profile scripts to default’s one (for grid and oracle users)
  • make sure you’ve planned a sufficient downtime (4+ hours depending on the number of databases and DB Systems)
  • do a sanity reboot before patching to kill zombie processes
  • use ODABR to make snapshots of the important filesystems prior patching: this tool is now included in the software distribution

Version precheck

Start to check current versions of the various components:

odacli describe-component
System Version
--------------
19.28.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.28.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.28.0.0.0          up-to-date

GI                                       19.28.0.0.250715     up-to-date

DB {
     OraDB19000_home7                    19.28.0.0.250715     up-to-date
     [CPROD19]
}

DCSCONTROLLER                            19.28.0.0.0          up-to-date

DCSCLI                                   19.28.0.0.0          up-to-date

DCSAGENT                                 19.28.0.0.0          up-to-date

DCSADMIN                                 19.28.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.4.25.r160118     up-to-date

BIOS                                     52140100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

List the DB homes, databases, DB Systems and VMs:

odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
e120c4c9-91b9-47b4-a234-b8ada12fce69     OraDB19000_home7     19.28.0.0.250715     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_7     CONFIGURED



odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID               
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
976a80f2-4653-469f-8cd4-ddc1a21aff51     CPROD19    SI       19.28.0.0.250715     true    OLTP     EE       odb8     ASM      CONFIGURED   e120c4c9-91b9-47b4-a234-b8ada12fce69


odacli list-dbsystems
Name                  Shape       GI version          DB info                         Status                  Created                   Updated
--------------------  ----------  ------------------  ------------------------------  ----------------------  ------------------------  ------------------------
dbs-03-tst            dbs2        19.28.0.0.250715    19.28(CONFIGURED=1)             CONFIGURED              2025-12-03 15:05:31 CET   2025-12-03 15:47:19 CET

odacli list-vms
No data found for resource VM.

Update the DCS components

Updating DCS components is the first step, after registering the patch file:

cd /opt/dbi
unzip p38427251_1929000_Linux-x86-64.zip

odacli update-repository -f /opt/dbi/oda-sm-19.29.0.0.0-251117-server.zip
sleep 30 ; odacli describe-job -i "91189839-e855-48ea-a6b1-7d9695da52a5"
Job details
----------------------------------------------------------------
                     ID:  7e69a05f-61fe-4b13-af5d-d78cfb7f11a9
            Description:  Repository Update
                 Status:  Success
                Created:  December 03, 2025 16:09:45 CET
                Message:  /opt/dbi/oda-sm-19.29.0.0.0-251117-server.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             December 03, 2025 16:09:49 CET           December 03, 2025 16:10:00 CET           Success


odacli describe-component
System Version
--------------
19.28.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.28.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.28.0.0.0          19.29.0.0.0

GI                                       19.28.0.0.250715     19.29.0.0.251021

DB {
     OraDB19000_home7                    19.28.0.0.250715     19.29.0.0.251021
     [CPROD19]
}

DCSCONTROLLER                            19.28.0.0.0          19.29.0.0.0

DCSCLI                                   19.28.0.0.0          19.29.0.0.0

DCSAGENT                                 19.28.0.0.0          19.29.0.0.0

DCSADMIN                                 19.28.0.0.0          19.29.0.0.0

OS                                       8.10                 up-to-date

ILOM                                     5.1.4.25.r160118     5.1.5.22.r165351

BIOS                                     52140100             52160100

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

Let’s update the DCS components to 19.29:

odacli update-dcsadmin -v 19.29.0.0.0

sleep 60 ; odacli describe-job -i "f2d216d5-f60d-46d6-a967-900c6e137421"
Job details
----------------------------------------------------------------
                     ID:  f2d216d5-f60d-46d6-a967-900c6e137421
            Description:  DcsAdmin patching to 19.29.0.0.0
                 Status:  Success
                Created:  December 03, 2025 16:12:26 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Dcs-admin upgrade                        December 03, 2025 16:12:26 CET           December 03, 2025 16:12:36 CET           Success
Ping DCS Admin                           December 03, 2025 16:12:36 CET           December 03, 2025 16:13:43 CET           Success



sleep 30 ;  odacli update-dcscomponents -v 19.29.0.0.0
{
  "jobId" : "5cd855d4-7b35-44dc-84f9-4625f84d461b",
  "status" : "Success",
  "message" : "Update-dcscomponents is successful on all the node(s): DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata schema update is done. Script '/opt/oracle/dcs/log/jobfiles/5cd855d4-7b35-44dc-84f9-4625f84d461b/apply_metadata_change.sh' ran successfully. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. ahf RPM upgrade is successful.  Successfully reset the Keystore password. HAMI RPM and DCS ensemble update was successful.  Skipped removing old Libs. Successfully ran setupAgentAuth.sh ",
  "reports" : null,
  "createTimestamp" : "December 03, 2025 16:14:32 PM CET",
  "description" : "Update-dcscomponents job completed and is not part of Agent job list",
  "updatedTime" : "December 03, 2025 16:19:11 PM CET",
  "jobType" : null,
  "externalRequestId" : null,
  "action" : null
}

System patching

Let’s do the prepatching of the system with the new -sc option:

odacli create-prepatchreport -sc -v 19.29.0.0.0

sleep 180 ; odacli describe-prepatchreport -i 317b0f75-fed7-480b-9dba-af7c635fabea

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  317b0f75-fed7-480b-9dba-af7c635fabea
            Description:  Patch pre-checks for [OS, ILOM, ORACHKSERVER, SERVER] to 19.29.0.0.0
                 Status:  SUCCESS
                Created:  December 3, 2025 4:20:13 PM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is patch location available     Success   Patch location is available.
Verify OS patch                 Success   No dependencies found for RPMs being
                                          removed, updated and installed. Check
                                          /opt/oracle/dcs/log/jobfiles/
                                          yumdryrunout_2025-12-03_16-20-
                                          29.0193.1_251.log file for more
                                          details
Validate command execution      Success   Validated command execution

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Validate DB System AFD state    Success   All DB Systems are on required
                                          versions
Validate command execution      Success   Validated command execution

OK let’s do the system patch:

odacli update-servercomponents -v 19.29.0.0.0
...

The server will reboot at the end of the patching. Let’s then check the job:

odacli describe-job -i "7f52ba58-f0d5-4055-864b-caae4209ce29"

Job details
----------------------------------------------------------------
                     ID:  7f52ba58-f0d5-4055-864b-caae4209ce29
            Description:  Server Patching to 19.29.0.0.0
                 Status:  Success
                Created:  December 03, 2025 16:26:08 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Deactivate Unit[dnf-makecache.timer]     December 03, 2025 16:26:14 CET           December 03, 2025 16:26:15 CET           Success
Validate ILOM server reachable           December 03, 2025 16:26:14 CET           December 03, 2025 16:26:14 CET           Success
Validating GI user metadata              December 03, 2025 16:26:14 CET           December 03, 2025 16:26:14 CET           Success
Deactivate Unit[kdump.service]           December 03, 2025 16:26:15 CET           December 03, 2025 16:26:15 CET           Success
Modify BM udev rules                     December 03, 2025 16:26:15 CET           December 03, 2025 16:26:35 CET           Success
Stop oakd                                December 03, 2025 16:26:35 CET           December 03, 2025 16:26:39 CET           Success
Creating repositories using yum          December 03, 2025 16:26:39 CET           December 03, 2025 16:26:41 CET           Success
Updating YumPluginVersionLock rpm        December 03, 2025 16:26:41 CET           December 03, 2025 16:26:44 CET           Success
Applying OS Patches                      December 03, 2025 16:26:44 CET           December 03, 2025 16:32:20 CET           Success
Applying HMP Patches                     December 03, 2025 16:32:20 CET           December 03, 2025 16:32:23 CET           Success
Creating repositories using yum          December 03, 2025 16:32:20 CET           December 03, 2025 16:32:20 CET           Success
Oda-hw-mgmt upgrade                      December 03, 2025 16:32:23 CET           December 03, 2025 16:32:52 CET           Success
Patch location validation                December 03, 2025 16:32:23 CET           December 03, 2025 16:32:23 CET           Success
Setting SELinux mode                     December 03, 2025 16:32:23 CET           December 03, 2025 16:32:23 CET           Success
Applying Firmware local Disk Patches     December 03, 2025 16:32:53 CET           December 03, 2025 16:32:57 CET           Success
OSS Patching                             December 03, 2025 16:32:53 CET           December 03, 2025 16:32:53 CET           Success
Applying Firmware local Controller Patch December 03, 2025 16:32:57 CET           December 03, 2025 16:33:01 CET           Success
Checking Ilom patch Version              December 03, 2025 16:33:01 CET           December 03, 2025 16:33:01 CET           Success
Patch location validation                December 03, 2025 16:33:01 CET           December 03, 2025 16:33:01 CET           Success
Save password in Wallet                  December 03, 2025 16:33:01 CET           December 03, 2025 16:33:02 CET           Success
Apply Ilom patch                         December 03, 2025 16:33:02 CET           December 03, 2025 16:43:49 CET           Success
Disabling IPMI v2                        December 03, 2025 16:33:02 CET           December 03, 2025 16:33:02 CET           Success
Copying Flash Bios to Temp location      December 03, 2025 16:43:49 CET           December 03, 2025 16:43:49 CET           Success
Start oakd                               December 03, 2025 16:43:49 CET           December 03, 2025 16:44:06 CET           Success
Add SYSNAME in Env                       December 03, 2025 16:44:06 CET           December 03, 2025 16:44:06 CET           Success
Cleanup JRE Home                         December 03, 2025 16:44:06 CET           December 03, 2025 16:44:06 CET           Success
Starting the clusterware                 December 03, 2025 16:44:06 CET           December 03, 2025 16:45:42 CET           Success
Generating and saving BOM                December 03, 2025 16:45:42 CET           December 03, 2025 16:46:12 CET           Success
Update System version                    December 03, 2025 16:45:42 CET           December 03, 2025 16:45:42 CET           Success
Update lvm.conf file                     December 03, 2025 16:45:42 CET           December 03, 2025 16:45:42 CET           Success
PreRebootNode Actions                    December 03, 2025 16:46:12 CET           December 03, 2025 16:47:17 CET           Success
Reboot Node                              December 03, 2025 16:47:17 CET           December 03, 2025 16:47:17 CET           Success

GI patching

Let’s unzip and register the patch file, and do the precheck for GI:

cd /opt/dbi
unzip -o p30403673_1929000_Linux-x86-64.zip

odacli update-repository -f /opt/dbi/odacli-dcs-19.29.0.0.0-251117-GI-19.29.0.0.zip
sleep 30 ; odacli describe-job -i "2e15156c-451b-470f-a523-03c4d024b726"

Job details
----------------------------------------------------------------
                     ID:  2e15156c-451b-470f-a523-03c4d024b726
            Description:  Repository Update
                 Status:  Success
                Created:  December 03, 2025 17:29:43 CET
                Message:  /opt/dbi/odacli-dcs-19.29.0.0.0-251117-GI-19.29.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             December 03, 2025 17:29:45 CET           December 03, 2025 17:30:32 CET           Success


odacli create-prepatchreport -gi -v 19.29.0.0.0

sleep 180 ; odacli describe-prepatchreport -i be3c7eb0-7bd8-4295-aa7f-ecd9e104d66f

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  be3c7eb0-7bd8-4295-aa7f-ecd9e104d66f
            Description:  Patch pre-checks for [RHPGI, GI] to 19.29.0.0.0
                 Status:  SUCCESS
                Created:  December 3, 2025 5:30:45 PM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__RHPGI__
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Validate available space        Success   Validated free space under /u01
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate ASM in online          Success   ASM is online
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate minimum agent version  Success   GI patching enabled in current
                                          DCSAGENT version
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate DB start dependencies  Success   DBs START dependency check passed
Validate DB stop dependencies   Success   DBs STOP dependency check passed
Validate space for clones       Success   Clones volume is already created
volume
Validate command execution      Success   Validated command execution

Let’s apply the GI update now:

odacli update-gihome -v 19.29.0.0.0
sleep 400 ;  odacli describe-job -i "5eaaa4ad-996b-4024-858d-a0f0082705d5"

Job details
----------------------------------------------------------------
                     ID:  5eaaa4ad-996b-4024-858d-a0f0082705d5
            Description:  Patch GI with RHP to 19.29.0.0.0
                 Status:  Success
                Created:  December 03, 2025 17:37:30 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Starting the clusterware                 December 03, 2025 17:37:44 CET           December 03, 2025 17:37:44 CET           Success
Creating GI home directories             December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Extract GI clone                         December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Provisioning Software Only GI with RHP   December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Registering image                        December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Registering image                        December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Registering working copy                 December 03, 2025 17:37:45 CET           December 03, 2025 17:37:45 CET           Success
Patch GI with RHP                        December 03, 2025 17:38:21 CET           December 03, 2025 17:43:07 CET           Success
Set CRS ping target                      December 03, 2025 17:43:07 CET           December 03, 2025 17:43:07 CET           Success
Updating .bashrc                         December 03, 2025 17:43:07 CET           December 03, 2025 17:43:07 CET           Success
Updating GI home metadata                December 03, 2025 17:43:07 CET           December 03, 2025 17:43:07 CET           Success
Updating GI home version                 December 03, 2025 17:43:07 CET           December 03, 2025 17:43:12 CET           Success
Updating All DBHome version              December 03, 2025 17:43:12 CET           December 03, 2025 17:43:17 CET           Success
Starting the clusterware                 December 03, 2025 17:43:38 CET           December 03, 2025 17:43:39 CET           Success
Validate ACFS resources are running      December 03, 2025 17:43:39 CET           December 03, 2025 17:43:39 CET           Success
Validate DB System VMs states            December 03, 2025 17:43:39 CET           December 03, 2025 17:43:40 CET           Success
Validate GI availability                 December 03, 2025 17:43:39 CET           December 03, 2025 17:43:39 CET           Success
Patch CPU Pools distribution             December 03, 2025 17:43:40 CET           December 03, 2025 17:43:40 CET           Success
Patch DB System domain config            December 03, 2025 17:43:40 CET           December 03, 2025 17:43:40 CET           Success
Patch KVM CRS type                       December 03, 2025 17:43:40 CET           December 03, 2025 17:43:40 CET           Success
Patch VM vDisks CRS dependencies         December 03, 2025 17:43:40 CET           December 03, 2025 17:43:40 CET           Success
Save custom VNetworks to storage         December 03, 2025 17:43:40 CET           December 03, 2025 17:43:41 CET           Success
Add network filters to DB Systems        December 03, 2025 17:43:41 CET           December 03, 2025 17:43:41 CET           Success
Create network filters                   December 03, 2025 17:43:41 CET           December 03, 2025 17:43:41 CET           Success
Patch DB Systems vDisks CRS dependencies December 03, 2025 17:43:41 CET           December 03, 2025 17:43:42 CET           Success
Patch DB Systems custom scale metadata   December 03, 2025 17:43:42 CET           December 03, 2025 17:43:42 CET           Success

No reboot is needed for this patch.

Check the versions

odacli describe-component
System Version
--------------
19.29.0.0.0

System Node Name
----------------
dbioda01

Local System Version
--------------------
19.29.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.29.0.0.0          up-to-date

GI                                       19.29.0.0.251021     up-to-date

DB {
     OraDB19000_home7                    19.28.0.0.250715     19.29.0.0.251021
     [CPROD19]
}

DCSCONTROLLER                            19.29.0.0.0          up-to-date

DCSCLI                                   19.29.0.0.0          up-to-date

DCSAGENT                                 19.29.0.0.0          up-to-date

DCSADMIN                                 19.29.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.22.r165351     up-to-date

BIOS                                     52160100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c4]                                8000D9AB             up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1]                            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c2d0, c2d1]                        XC311132             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

Patching the storage

Patching the storage is only needed if describe-component tells you that you’re not up-to-date. On my X8-2M, it wasn’t needed. If your ODA needs the storage patch, it’s easy:

odacli update-storage -v 19.29.0.0.0
odacli describe-job -i ...

The server will reboot when done.

Patching the DB homes

It’s now time to patch the DB home and the database on my ODA. Let’s first unzip and register the patch file in the repository:

unzip -o p30403662_1929000_Linux-x86-64.zip
odacli update-repository -f /opt/dbi/odacli-dcs-19.29.0.0.0-251117-DB-19.29.0.0.zip 
sleep 30; odacli describe-job -i "480c3911-d673-47fd-b6c5-f65b2cc4a1b9"

Job details
----------------------------------------------------------------
                     ID:  480c3911-d673-47fd-b6c5-f65b2cc4a1b9
            Description:  Repository Update
                 Status:  Success
                Created:  December 03, 2025 17:51:13 CET
                Message:  /opt/dbi/odacli-dcs-19.29.0.0.0-251117-DB-19.29.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Unzip bundle                             December 03, 2025 17:51:13 CET           December 03, 2025 17:51:49 CET           Success



odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
e120c4c9-91b9-47b4-a234-b8ada12fce69     OraDB19000_home7     19.28.0.0.250715     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_7     CONFIGURED

Let’s check if the patch can be applied, then patch this DB home:

odacli create-prepatchreport -d -i e120c4c9-91b9-47b4-a234-b8ada12fce69 -v 19.29.0.0.0

sleep 600; odacli describe-prepatchreport -i fed289b2-848a-460f-9ba7-ef87c2a08dca

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  fed289b2-848a-460f-9ba7-ef87c2a08dca
            Description:  Patch pre-checks for [DB, RHPDB, ORACHKDB] to 19.29.0.0.0: DbHome is OraDB19000_home7
                 Status:  SUCCESS
                Created:  December 3, 2025 5:53:51 PM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbioda01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID             Success   Validated DB Home ID:
                                          e120c4c9-91b9-47b4-a234-b8ada12fce69
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate there is usable        Success   Successfully validated Oracle Base
space under oracle base                   usable space
Validate glogin.sql file        Success   Successfully verified glogin.sql
                                          won't break patching
Validate dbHomesOnACFS          Success   User has configured disk group for
configured                                Database homes on ACFS
Validate Oracle base            Success   Successfully validated Oracle Base
Is DB clone available           Success   Successfully validated clone file
                                          exists
Validate command execution      Success   Validated command execution

__RHPDB__
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

odacli update-dbhome -i e120c4c9-91b9-47b4-a234-b8ada12fce69 -v 19.29.0.0.0

sleep 600;  odacli describe-job -i "b2676a55-96de-4101-8686-98c6a88b8477"
Job details
----------------------------------------------------------------
                     ID:  b2676a55-96de-4101-8686-98c6a88b8477
            Description:  DB Home Patching to 19.29.0.0.0: Home ID is e120c4c9-91b9-47b4-a234-b8ada12fce69
                 Status:  Success
                Created:  December 03, 2025 18:05:45 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Creating wallet for DB Client            December 03, 2025 18:06:34 CET           December 03, 2025 18:06:34 CET           Success
Patch databases by RHP - [CPROD19]       December 03, 2025 18:06:34 CET           December 03, 2025 18:13:06 CET           Success
Updating database metadata               December 03, 2025 18:13:06 CET           December 03, 2025 18:13:06 CET           Success
Upgrade pwfile to 12.2                   December 03, 2025 18:13:06 CET           December 03, 2025 18:13:09 CET           Success
Set log_archive_dest for Database        December 03, 2025 18:13:09 CET           December 03, 2025 18:13:12 CET           Success
Populate PDB metadata                    December 03, 2025 18:13:13 CET           December 03, 2025 18:13:14 CET           Success
Generating and saving BOM                December 03, 2025 18:13:15 CET           December 03, 2025 18:13:54 CET           Success
TDE parameter update                     December 03, 2025 18:14:24 CET           December 03, 2025 18:14:24 CET           Success

Let’s check if everything is fine:

odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
e120c4c9-91b9-47b4-a234-b8ada12fce69     OraDB19000_home7     19.28.0.0.250715     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_7     CONFIGURED
57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb     OraDB19000_home9     19.29.0.0.251021     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_9     CONFIGURED

odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID               
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
976a80f2-4653-469f-8cd4-ddc1a21aff51     CPROD19    SI       19.29.0.0.251021     true    OLTP     EE       odb8     ASM      CONFIGURED   57c0dd7f-dcf4-4a38-9e79-4bf8c78e81bb

Let’s now remove the old DB home:

odacli delete-dbhome -i e120c4c9-91b9-47b4-a234-b8ada12fce69
...

Cleanse the old patches

Don’t forget to remove the previous patch from the repository:

odacli cleanup-patchrepo -comp all -v 19.28.0.0.0

odacli describe-job -i "76ba3e95-bb71-4ebe-b7b2-f3cac07d89dd"
Job details
----------------------------------------------------------------
                     ID:  76ba3e95-bb71-4ebe-b7b2-f3cac07d89dd
            Description:  Cleanup patchrepos
                 Status:  Success
                Created:  December 03, 2025 18:19:22 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Cleanup Repository                       December 03, 2025 18:19:22 CET           December 03, 2025 18:19:22 CET           Success
Cleanup old ASR rpm                      December 03, 2025 18:19:22 CET           December 03, 2025 18:19:22 CET           Success

Old GI binaries are still there, it’s better removing them manually:

du -hs /u01/app/19.2*
14G     /u01/app/19.28.0.0
14G     /u01/app/19.29.0.0

rm -rf /u01/app/19.28.0.0

Post-patching tasks

You will need to put back your specific configuration:

  • add your additional RPMs
  • put back your profile scripts for grid and oracle users
  • check if monitoring still works

Patching the DB System

If you use DB Systems on your ODA, meaning that some of your databases are running in dedicated VMs, you will need to apply the patch inside each DB System. As the repository is shared, patch files are already available for DB Systems. Applying the patch is similar to what you did on bare metal:

ssh dbs-03-tst

odacli update-dcsadmin -v 19.29.0.0.0

sleep 60 ; odacli describe-job -i 43df9afb-adc1-479c-8987-c8d24f056c02

Job details
----------------------------------------------------------------
                     ID:  43df9afb-adc1-479c-8987-c8d24f056c02
            Description:  DcsAdmin patching to 19.29.0.0.0
                 Status:  Success
                Created:  December 08, 2025 10:15:13 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Dcs-admin upgrade                        December 08, 2025 10:15:14 CET           December 08, 2025 10:15:25 CET           Success
Ping DCS Admin                           December 08, 2025 10:15:25 CET           December 08, 2025 10:16:33 CET           Success

sleep 30 ;  odacli update-dcscomponents -v 19.29.0.0.0
{
  "jobId" : "91fd5f67-5ba8-4636-9046-0fe1921a659e",
  "status" : "Success",
  "message" : "Update-dcscomponents is successful on all the node(s): DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata schema update is done. Script '/opt/oracle/dcs/log/jobfiles/91fd5f67-5ba8-4636-9046-0fe1921a659e/apply_metadata_change.sh' ran successfully. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. ahf RPM upgrade is successful.  Successfully reset the Keystore password. HAMI RPM and DCS ensemble update was successful.  Skipped removing old Libs. Successfully ran setupAgentAuth.sh ",
  "reports" : null,
  "createTimestamp" : "December 08, 2025 10:17:38 AM CET",
  "description" : "Update-dcscomponents job completed and is not part of Agent job list",
  "updatedTime" : "December 08, 2025 10:23:37 AM CET",
  "jobType" : null,
  "externalRequestId" : null,
  "action" : null
}

odacli create-prepatchreport -sc -v 19.29.0.0.0

sleep 20 ; odacli describe-prepatchreport -i f2f90339-16b8-49a5-be8c-408dd0e9f28b
ps -ef | grep pmon

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  f2f90339-16b8-49a5-be8c-408dd0e9f28b
            Description:  Patch pre-checks for [OS, ORACHKSERVER, SERVER] to 19.29.0.0.0
                 Status:  SUCCESS
                Created:  December 8, 2025 10:25:17 AM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbs-03-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is patch location available     Success   Patch location is available.
Verify OS patch                 Success   No dependencies found for RPMs being
                                          removed, updated and installed. Check
                                          /opt/oracle/dcs/log/jobfiles/
                                          yumdryrunout_2025-12-08_10-25-
                                          34.0670.1_222.log file for more
                                          details
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate all KVM ACFS           Success   All KVM ACFS resources are running
resources are running
Validate DB System VM states    Success   All DB System VMs states are expected
Enable support for Multi-DB     Success   No need to convert the DB System
Validate DB System AFD state    Success   AFD is not configured
Validate command execution      Success   Validated command execution

odacli update-servercomponents -v 19.29.0.0.0

The DB System will reboot.

odacli describe-job -i 5a23ae5b-43ed-4c39-ba79-21cd8a125b79

Job details
----------------------------------------------------------------
                     ID:  5a23ae5b-43ed-4c39-ba79-21cd8a125b79
            Description:  Server Patching to 19.29.0.0.0
                 Status:  Success
                Created:  December 08, 2025 10:30:04 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Validating GI user metadata              December 08, 2025 10:30:13 CET           December 08, 2025 10:30:13 CET           Success
Deactivate Unit[dnf-makecache.timer]     December 08, 2025 10:30:14 CET           December 08, 2025 10:30:14 CET           Success
Deactivate Unit[kdump.service]           December 08, 2025 10:30:14 CET           December 08, 2025 10:30:15 CET           Success
Modify DBVM udev rules                   December 08, 2025 10:30:15 CET           December 08, 2025 10:30:36 CET           Success
Creating repositories using yum          December 08, 2025 10:30:36 CET           December 08, 2025 10:30:39 CET           Success
Updating YumPluginVersionLock rpm        December 08, 2025 10:30:39 CET           December 08, 2025 10:30:42 CET           Success
Applying OS Patches                      December 08, 2025 10:30:42 CET           December 08, 2025 10:34:11 CET           Success
Creating repositories using yum          December 08, 2025 10:34:11 CET           December 08, 2025 10:34:12 CET           Success
Applying HMP Patches                     December 08, 2025 10:34:12 CET           December 08, 2025 10:34:15 CET           Success
Setting SELinux mode                     December 08, 2025 10:34:15 CET           December 08, 2025 10:34:15 CET           Success
Oda-hw-mgmt upgrade                      December 08, 2025 10:34:16 CET           December 08, 2025 10:34:44 CET           Success
Patch location validation                December 08, 2025 10:34:16 CET           December 08, 2025 10:34:16 CET           Success
Cleanup JRE Home                         December 08, 2025 10:34:45 CET           December 08, 2025 10:34:45 CET           Success
Generating and saving BOM                December 08, 2025 10:34:56 CET           December 08, 2025 10:35:09 CET           Success
Update System version                    December 08, 2025 10:34:56 CET           December 08, 2025 10:34:56 CET           Success
PreRebootNode Actions                    December 08, 2025 10:35:09 CET           December 08, 2025 10:35:09 CET           Success
Reboot Node                              December 08, 2025 10:35:09 CET           December 08, 2025 10:35:09 CET           Success

odacli create-prepatchreport -gi -v 19.29.0.0.0

sleep 240 ; odacli describe-prepatchreport -i 56c7b4b1-3787-42af-b4b0-0fa6715a91f7

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  56c7b4b1-3787-42af-b4b0-0fa6715a91f7
            Description:  Patch pre-checks for [RHPGI, GI] to 19.29.0.0.0
                 Status:  SUCCESS
                Created:  December 8, 2025 10:37:05 AM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbs-03-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__RHPGI__
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Validate available space        Success   Validated free space under /u01
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate BM versions            Success   Validated BM server components
                                          versions
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate minimum agent version  Success   GI patching enabled in current
                                          DCSAGENT version
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate command execution      Success   Validated command execution

odacli update-gihome -v 19.29.0.0.0

sleep 600 ; odacli describe-job -i 571205f2-bdf2-43a2-944a-ec2765148446

odacli describe-job -i 571205f2-bdf2-43a2-944a-ec2765148446

Job details
----------------------------------------------------------------
                     ID:  571205f2-bdf2-43a2-944a-ec2765148446
            Description:  Patch GI with RHP to 19.29.0.0.0
                 Status:  Success
                Created:  December 08, 2025 10:43:47 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Starting the clusterware                 December 08, 2025 10:44:07 CET           December 08, 2025 10:44:07 CET           Success
Registering image                        December 08, 2025 10:44:08 CET           December 08, 2025 10:44:08 CET           Success
Registering working copy                 December 08, 2025 10:44:08 CET           December 08, 2025 10:44:09 CET           Success
Creating GI home directories             December 08, 2025 10:44:09 CET           December 08, 2025 10:44:09 CET           Success
Extract GI clone                         December 08, 2025 10:44:09 CET           December 08, 2025 10:44:09 CET           Success
Provisioning Software Only GI with RHP   December 08, 2025 10:44:09 CET           December 08, 2025 10:44:09 CET           Success
Registering image                        December 08, 2025 10:44:09 CET           December 08, 2025 10:44:09 CET           Success
Patch GI with RHP                        December 08, 2025 10:44:49 CET           December 08, 2025 10:49:16 CET           Success
Set CRS ping target                      December 08, 2025 10:49:16 CET           December 08, 2025 10:49:16 CET           Success
Updating .bashrc                         December 08, 2025 10:49:16 CET           December 08, 2025 10:49:16 CET           Success
Updating GI home metadata                December 08, 2025 10:49:16 CET           December 08, 2025 10:49:17 CET           Success
Updating GI home version                 December 08, 2025 10:49:17 CET           December 08, 2025 10:49:24 CET           Success
Updating All DBHome version              December 08, 2025 10:49:24 CET           December 08, 2025 10:49:30 CET           Success
Patch DB System on BM                    December 08, 2025 10:50:05 CET           December 08, 2025 10:50:11 CET           Success
Starting the clusterware                 December 08, 2025 10:50:05 CET           December 08, 2025 10:50:05 CET           Success


odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
46268d88-e958-4c16-b45b-c32d5e0203fb     OraDB19000_home1     19.28.0.0.250715     EE         /u01/app/oracle/product/19.0.0.0/dbhome_1                CONFIGURED

odacli create-prepatchreport -d -i 46268d88-e958-4c16-b45b-c32d5e0203fb -v 19.29.0.0.0

sleep 600 ;  odacli describe-prepatchreport -i eb00906a-0ecc-4a9d-968a-272d4c3719f4

Prepatch Report
------------------------------------------------------------------------
                 Job ID:  eb00906a-0ecc-4a9d-968a-272d4c3719f4
            Description:  Patch pre-checks for [DB, RHPDB, ORACHKDB] to 19.29.0.0.0: DbHome is OraDB19000_home1
                 Status:  SUCCESS
                Created:  December 8, 2025 11:27:25 AM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
dbs-03-tst

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID             Success   Validated DB Home ID:
                                          46268d88-e958-4c16-b45b-c32d5e0203fb
Validate patching tag           Success   Validated patching tag: 19.29.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate there is usable        Success   Successfully validated Oracle Base
space under oracle base                   usable space
Validate glogin.sql file        Success   Successfully verified glogin.sql
                                          won't break patching
Is DB clone available           Success   Successfully validated clone file
                                          exists
Validate command execution      Success   Validated command execution

__RHPDB__
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution


odacli update-dbhome -i 46268d88-e958-4c16-b45b-c32d5e0203fb -v 19.29.0.0.0

sleep 600 ; odacli describe-job -i aac90798-a6a4-4740-bfa6-77bcb80cba7c

Job details
----------------------------------------------------------------
                     ID:  aac90798-a6a4-4740-bfa6-77bcb80cba7c
            Description:  DB Home Patching to 19.29.0.0.0: Home ID is 46268d88-e958-4c16-b45b-c32d5e0203fb
                 Status:  Success
                Created:  December 08, 2025 11:36:57 CET
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- ----------------
Creating wallet for DB Client            December 08, 2025 11:37:43 CET           December 08, 2025 11:37:43 CET           Success
Patch databases by RHP - [CTEST19]       December 08, 2025 11:37:43 CET           December 08, 2025 11:45:11 CET           Success
Updating database metadata               December 08, 2025 11:45:11 CET           December 08, 2025 11:45:12 CET           Success
Upgrade pwfile to 12.2                   December 08, 2025 11:45:12 CET           December 08, 2025 11:45:17 CET           Success
Set log_archive_dest for Database        December 08, 2025 11:45:17 CET           December 08, 2025 11:45:21 CET           Success
Populate PDB metadata                    December 08, 2025 11:45:22 CET           December 08, 2025 11:45:24 CET           Success
Generating and saving BOM                December 08, 2025 11:45:24 CET           December 08, 2025 11:46:10 CET           Success
TDE parameter update                     December 08, 2025 11:46:39 CET           December 08, 2025 11:46:39 CET           Success

odacli list-databases
ID                                       DB Name    DB Type  DB Version           CDB     Class    Edition  Shape    Storage  Status       DB Home ID
---------------------------------------- ---------- -------- -------------------- ------- -------- -------- -------- -------- ------------ ----------------------------------------
54e88627-a3cf-4696-956b-6262bbd51cf0     CTEST19    SI       19.29.0.0.251021     true    OLTP     EE       odb2     ASM      CONFIGURED   85b6e4eb-5db4-4165-bfb3-e3da52dc4777

odacli delete-dbhome -i 46268d88-e958-4c16-b45b-c32d5e0203fb
...

odacli describe-component
System Version
--------------
19.29.0.0.0

System Node Name
----------------
dbs-03-tst

Local System Version
--------------------
19.29.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.29.0.0.0          up-to-date

GI                                       19.29.0.0.251021     up-to-date

DB {
     OraDB19000_home2                    19.29.0.0.251021     up-to-date
     [CTEST19]
}

DCSCONTROLLER                            19.29.0.0.0          up-to-date

DCSCLI                                   19.29.0.0.0          up-to-date

DCSAGENT                                 19.29.0.0.0          up-to-date

DCSADMIN                                 19.29.0.0.0          up-to-date

OS                                       8.10                 up-to-date

Don’t forget to apply this procedure on each of your DB Systems.

Provision a new 26ai DB System

This is an optional step, only if you’d like to try 26ai database. First unzip and register the VM template, GI 26ai and DB 26ai:

unzip -o p36524660_1929000_Linux-x86-64.zip
unzip -o p36524627_1929000_Linux-x86-64.zip
unzip -o p36524642_1929000_Linux-x86-64.zip

odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-ODAVM-19.29.0.0.zip
odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-GI-23.26.0.0.zip
odacli update-repository -f /opt/dbi/odacli-dcs-23.26.0.0.0-251116-DB-23.26.0.0.zip

sleep 30 ; odacli list-jobs | tail -n 4
d4700315-db8c-4522-af55-0fddd262bfe4     Repository Update                                                           2025-12-08 15:20:14 CET             Success
ba7d452f-e03d-46d0-a607-fd7c758cd1b1     Repository Update                                                           2025-12-08 15:20:59 CET             Success
bfc102d1-985b-4792-8054-03709aa8d949     Repository Update                                                           2025-12-08 15:21:20 CET             Success

odacli describe-dbsystem-image | grep 23.26
DBVM                  23.26.0.0.0           23.26.0.0.0
GI                    23.26.0.0.0           23.26.0.0.0
DB                    23.26.0.0.0           23.26.0.0.0

Now let’s create a json file based on the one I used to create my 19.28 DB System and adjust some parameters. Then create the DB System:

cat create_dbs-03-tst-cdb.json | sed 's/dbs-03-tst/dbs-04-tst/g' | sed 's/10.16.0.146/10.16.0.147/g' | sed 's/CTEST19/CTEST26/g' | sed 's/19.28.0.0.250715/23.26.0.0.0/g' > create_dbs-04-tst-cdb.json

odacli create-dbsystem -p /opt/dbi/create_dbs-04-tst-cdb.json

odacli describe-job -i c0c8b0a0-5033-46b5-81a1-f326f6caa761
...

35 minutes later, my new DB System is ready to use:

Name                  Shape	  GI version          DB info                         Status                  Created                   Updated
--------------------  ----------  ------------------  ------------------------------  ----------------------  ------------------------  ------------------------
dbs-03-tst            dbs2        19.29.0.0.251021    19.29(CONFIGURED=1)             CONFIGURED              2025-12-03 15:05:31 CET   2025-12-08 10:50:06 CET
dbs-04-tst            dbs2        23.26.0.0.0         23.26(CONFIGURED=1)             CONFIGURED              2025-12-08 15:32:48 CET   2025-12-08 16:09:10 CET

Conclusion

Applying this patch is rather easy. Remember these key points when using an ODA:

  • keep it clean
  • keep it under control
  • keep it updated

L’article How to patch your ODA to 19.29? est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/how-to-patch-your-oda-to-19-29/feed/ 0 Reimage your ODA from scratch https://www.dbi-services.com/blog/reimage-your-oda-from-scratch/ https://www.dbi-services.com/blog/reimage-your-oda-from-scratch/#respond Tue, 09 Sep 2025 17:45:04 +0000 https://www.dbi-services.com/blog/?p=40229 Reimage your ODA from scratch: is it a good alternative to patching?

L’article Reimage your ODA from scratch est apparu en premier sur dbi Blog.

]]> Introduction

Oracle Database Appliance is still very popular today. Provisioning is fast: from unboxing to a first running database is done in one day. And everything is standardized.

Patching is not that straight forward. First, you can only apply the latest patch on top of the 4 previous ones, and if you come from an older release, you may have 2, 3 or more patches to apply to reach the current version. It could take hours, even days.

The question one could ask is if it wouldn’t be more efficient to do a full ODA reimaging instead of patching. Let’s try to find out.

Environment and upgrade plan

This is a “real” environment composed of 4x ODA X8-2M: 2x ODAs for production, 2x ODAs for test/development. These servers have been provisioned in 2020 with the first 19c production available on ODA: 19.6. They were updated twice in the past years to reach 19.20 in 2024. These 2 upgrades were not that easy for some reasons, and systems are not that clean in my opinion.

According to MOS note 2757884.1, these ODAs are supported until August 2027, meaning that the last patch for these servers will be provided at this time. My client wants to keep them for 2 more years at least. So it’s definitely worth it to apply the patch.

I could have applied patch 19.24 then patch 19.27 on these ODAs, but as the first jump requires a Data Preserving Reprovisioning (meaning an OS reimaging) and as my ODAs are not that clean, I discussed with my client and we chose to reimage the servers from scratch.

Note that the production ODAs we will focus on are running 15 Enterprise Edition databases, and that a Data Guard configuration exists for each of them.

The plan is to switch all primaries to one ODA, reimage the other one, wait 2 weeks to see if everything is fine, do the switchover to the freshly reimaged ODA and reimage the other one the same way. As I don’t want to patch my databases now, I will deploy a 19.20 DB home and migrate to the latest version later.

Note that when I started this reimaging, current patch was 19.27. In the meantime, 19.28 becomes available.

Before starting: check ODA’s health

Let’s first check if the ODA I planned to reimage is healthy. You should normally monitor the hardware alerts on your ODAs, and eventually configure Automatic Service Request (ASR) to address hardware issues as soon as possible.

odaadmcli show server | grep Problem
	Open Problems            : 0
	Open Problems Report     : System is healthy

No problem on this ODA. I wouldn’t do anything on an ODA that is not healthy: fix up hardware troubles before patching or reimaging an ODA.

Check if standby databases are synced and disable archive transport

Apart from DBTEST which is primary (I always have a standalone test database on my ODAs), every database must be a Physical Standby Database. Expected downtime for such a reimaging is several hours or days depending on the number of databases. So, checking if sync is OK is a good practice. I also disable transport to standby databases and step down Data Guard to Maximum Performance:

vi check_role.sql
conn / as sysdba
set pages 0
select name||' is here '|| database_role from v$database;
exit;

for a in `ps -ef | grep pmon  | grep -v grep | grep -v ASM | grep -v APX | cut -c 58- | sort`; do  . oraenv <<< $a 1> /dev/null ; sqlplus -s /nolog @check_role; done
BPAPRD is here PHYSICAL STANDBY

DBTEST is here PRIMARY

BMC3PRD is here PHYSICAL STANDBY

MLOGAR is here PHYSICAL STANDBY

NIMSHMP is here PHYSICAL STANDBY

NOVOTEC is here PHYSICAL STANDBY

NTRAMPRD is here PHYSICAL STANDBY

NWTASKP is here PHYSICAL STANDBY

OPERA is here PHYSICAL STANDBY

PBYPRD is here PHYSICAL STANDBY

PRAXPRD is here PHYSICAL STANDBY

RCAT is here PHYSICAL STANDBY

SMARTPRD is here PHYSICAL STANDBY

SMKPRD is here PHYSICAL STANDBY

SPORT1 is here PHYSICAL STANDBY

SPORT2 is here PHYSICAL STANDBY


for a in `ps -ef | grep pmon  | grep -v grep | grep -v ASM | grep -v APX | grep -v DBTEST | cut -c 58- | sort`; do echo $a ; . oraenv <<< $a 1> /dev/null ; dgmgrl / "show configuration lag" | grep Lag; done
BPAPRD
             Transport Lag:	 0 seconds (computed 1 second ago)
             Apply Lag:          0 seconds (computed 1 second ago)
BMC3PRD
                  Transport Lag:      0 seconds (computed 0 seconds ago)
                  Apply Lag:          0 seconds (computed 0 seconds ago)
MLOGAR
                   Transport Lag:      0 seconds (computed 1 second ago)
                   Apply Lag:          0 seconds (computed 1 second ago)
NIMSHMP
                  Transport Lag:      0 seconds (computed 0 seconds ago)
                  Apply Lag:          0 seconds (computed 0 seconds ago)
NOVOTEC
                  Transport Lag:      0 seconds (computed 0 seconds ago)
                  Apply Lag:          0 seconds (computed 0 seconds ago)
NTRAMPRD
                    Transport Lag:	0 seconds (computed 0 seconds ago)
                    Apply Lag:          0 seconds (computed 0 seconds ago)
NWTASKP
                  Transport Lag:      0 seconds (computed 0 seconds ago)
                  Apply Lag:          0 seconds (computed 0 seconds ago)
OPERA
             Transport Lag:	 0 seconds (computed 0 seconds ago)
             Apply Lag:          0 seconds (computed 0 seconds ago)
PBYPRD
                  Transport Lag:      0 seconds (computed 0 seconds ago)
                  Apply Lag:          0 seconds (computed 0 seconds ago)
PRAXPRD
             Transport Lag:	 0 seconds (computed 1 second ago)
             Apply Lag:          0 seconds (computed 1 second ago)
RCAT
              Transport Lag:	  0 seconds (computed 1 second ago)
              Apply Lag:          0 seconds (computed 1 second ago)
 
SMARTPRD
             Transport Lag:	 0 seconds (computed 0 seconds ago)
             Apply Lag:          0 seconds (computed 0 seconds ago)
SMKPRD
             Transport Lag:	 0 seconds (computed 1 second ago)
             Apply Lag:          0 seconds (computed 1 second ago)
SPORT1
             Transport Lag:	 0 seconds (computed 0 seconds ago)
             Apply Lag:          0 seconds (computed 0 seconds ago)
SPORT2
             Transport Lag:	 0 seconds (computed 0 seconds ago)
             Apply Lag:          0 seconds (computed 0 seconds ago)

for a in `ps -ef | grep pmon  | grep -v grep | grep -v ASM | grep -v APX | cut -c 58- | sort`; do echo $a ; . oraenv <<< $a 1> /dev/null ; dgmgrl / "edit configuration set protection mode as MaxPerformance;" | grep Succeeded; dgmgrl / "EDIT DATABASE ${a}_IN SET STATE='TRANSPORT-OFF';" | grep Succeed ; done
BPAPRD
Succeeded.
Succeeded.
DBTEST
BMC3PRD
Succeeded.
Succeeded.
MLOGAR
Succeeded.
Succeeded
NIMSHMP
Succeeded.
Succeeded.
NOVOTEC
Succeeded.
Succeeded.
NTRAMPRD
Succeeded.
Succeeded
NWTASKP
Succeeded.
Succeeded.
OPERA
Succeeded.
Succeeded.
PBYPRD
Succeeded.
Succeeded.
PRAXPRD
Succeeded.
Succeeded.
RCAT
Succeeded.
Succeeded.
SMARTPRD
Succeeded.
Succeeded.
SMKPRD
Succeeded.
Succeeded.
SPORT1
Succeeded.
Succeeded.
SPORT2
Succeeded.
Succeeded.

Backup important files

As you will erase everything on this ODA, backup everything that needs to be backed up on a remote filesystem (/backup here). Usually, I use this script:

cat /backup/Patch19.27/dbi_prepatch_backup.sh
# Backup important files before patching
export BKPPATH=/backup/Patch19.27/backup_ODA_`hostname`_`date +"%Y%m%d_%H%M"`
echo "Backing up to " $BKPPATH
mkdir -p $BKPPATH
odacli list-databases > $BKPPATH/list-databases.txt
ps -ef | grep pmon | grep -v ASM | grep -v APX | grep -v grep | cut -c 58- | sort > $BKPPATH/running-instances.txt
odacli list-dbhomes > $BKPPATH/list-dbhomes.txt
odacli list-dbsystems > $BKPPATH/list-dbsystems.txt
odacli list-vms > $BKPPATH/list-vms.txt
crontab -u oracle -l  > $BKPPATH/crontab-oracle.txt
crontab -u grid -l  > $BKPPATH/crontab-grid.txt
crontab -l  > $BKPPATH/crontab-root.txt

cat /etc/fstab >  $BKPPATH/fstab.txt
cat /etc/oratab >  $BKPPATH/oratab.txt
cat /etc/sysconfig/network >  $BKPPATH/etc-sysconfig-network.txt
cat /etc/hosts  >  $BKPPATH/hosts
cat /etc/resolv.conf  >  $BKPPATH/resolv.conf
cat /etc/sysctl.conf  >  $BKPPATH/

cp /etc/krb5.conf  $BKPPATH/
cp /etc/krb5.keytab  $BKPPATH/
mkdir $BKPPATH/network-scripts
cp  /etc/sysconfig/network-scripts/ifcfg*  $BKPPATH/network-scripts/
odacli describe-system > $BKPPATH/describe-system.txt
odacli  describe-component >  $BKPPATH/describe-component.txt
HISTFILE=~/.bash_history
set -o history
history > $BKPPATH/history-root.txt
cp /home/oracle/.bash_history $BKPPATH/history-oracle.txt
df -h >  $BKPPATH/filesystems-status.txt

for a in `odacli list-dbhomes -j | grep dbHomeLocation | awk -F '"' '{print $4}' | sort` ; do mkdir -p $BKPPATH/$a/network/admin/ ; cp $a/network/admin/tnsnames.ora $BKPPATH/$a/network/admin/; cp $a/network/admin/sqlnet.ora $BKPPATH/$a/network/admin/; done
for a in `odacli list-dbhomes -j | grep dbHomeLocation | awk -F '"' '{print $4}' | sort` ; do mkdir -p $BKPPATH/$a/owm/ ; cp -r $a/owm/* $BKPPATH/$a/owm/; done
cp `ps -ef | grep -v grep | grep LISTENER | awk -F ' ' '{print $8}' | awk -F 'bin' '{print $1}'`network/admin/listener.ora $BKPPATH/gridhome-listener.ora
cp `ps -ef | grep -v grep | grep LISTENER | awk -F ' ' '{print $8}' | awk -F 'bin' '{print $1}'`/network/admin/sqlnet.ora $BKPPATH/gridhome-sqlnet.ora

tar czf $BKPPATH/u01-app-oracle-admin.tgz /u01/app/oracle/admin/
tar czf $BKPPATH/u01-app-odaorabase-oracle-admin.tgz /u01/app/odaorabase/oracle/admin/
tar czf $BKPPATH/u01-app-oracle-local.tgz /u01/app/oracle/local/
tar czf $BKPPATH/home.tgz /home/
cp /etc/passwd $BKPPATH/
cp /etc/group $BKPPATH/

echo "End"
echo "Backup files size:"
du -hs  $BKPPATH
echo "Backup files content:"
ls -lrt  $BKPPATH

sh /backup/Patch19.27/dbi_prepatch_backup.sh
...

This backup script is probably not extensive: take some time to make a suitable one for your needs.

Let’s also take a backup of the DB homes, in my case I only have one:

tar czf /backup/Patch19.27/odaprdcr_dbhome.tgz /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_4

ls -lrth /backup/Patch19.27/odaprdcr_dbhome.tgz

-rw-r--r-- 1 root root 5.9G Sep  3 10:50 /backup/Patch19.27/odaprdcr_dbhome.tgz

Backup standby spfiles and controlfiles

As this ODA is only running standby databases, I don’t need to back them up. I can use the backup from primary assuming a reliable backup strategy has been deployed on production’s databases. 2 files differ between primary and standby databases: spfile and controlfile. So let’s backup these 2 files on all databases, it just needs a couple of minutes for all:

su - oracle

mkdir /backup/Patch19.27/ctl_spf

cat /backup/Patch19.27/bck_ctl_spf.rcv

backup current controlfile format '/backup/Patch19.27/ctl_spf/ctl_%d_%T.bck';
backup spfile format '/backup/Patch19.27/ctl_spf/spf_%d_%T.bck';
exit;

for a in `ps -ef | grep pmon  | grep -v grep | grep -v ASM | grep -v APX | grep -v DBTEST | cut -c 58- | sort`; do echo $a ; . oraenv <<< $a 1> /dev/null ; rman target / cmdfile="/backup/Patch19.27/bck_ctl_spf.rcv" | grep Succeeded; done

ls -lrt /backup/Patch19.27/ctl_spf

total 33592
-rw-r----- 1 oracle asmadmin 46104576 Sep  3 10:33 ctl_BPAPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:33 spf_BPAPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 21233664 Sep  3 10:33 ctl_BMC3PRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:33 spf_BMC3PRD_20250903.bck
-rw-r----- 1 oracle asmadmin 27656192 Sep  3 10:33 ctl_MLOGAR_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:34 spf_MLOGAR_20250903.bck
-rw-r----- 1 oracle asmadmin 30965760 Sep  3 10:34 ctl_NIMSHMP_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:34 spf_NIMSHMP_20250903.bck
-rw-r----- 1 oracle asmadmin 27394048 Sep  3 10:34 ctl_NOVOTEC_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:34 spf_NOVOTEC_20250903.bck
-rw-r----- 1 oracle asmadmin 77955072 Sep  3 10:34 ctl_NTRAMPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:34 spf_NTRAMPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 29294592 Sep  3 10:34 ctl_NWTASKP_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:34 spf_NWTASKP_20250903.bck
-rw-r----- 1 oracle asmadmin 66387968 Sep  3 10:35 ctl_OPERA_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:35 spf_OPERA_20250903.bck
-rw-r----- 1 oracle asmadmin 20905984 Sep  3 10:35 ctl_PBYPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:35 spf_PBYPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 28868608 Sep  3 10:35 ctl_PRAXPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:35 spf_PRAXPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 14352384 Sep  3 10:35 ctl_RCAT_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:35 spf_RCAT_20250903.bck
-rw-r----- 1 oracle asmadmin 35028992 Sep  3 10:35 ctl_SMARTPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:35 spf_SMARTPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 21594112 Sep  3 10:35 ctl_SMKPRD_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:36 spf_SMKPRD_20250903.bck
-rw-r----- 1 oracle asmadmin 36798464 Sep  3 10:36 ctl_SPORT1_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:36 spf_SPORT1_20250903.bck
-rw-r----- 1 oracle asmadmin 20840448 Sep  3 10:36 ctl_SPORT2_20250903.bck
-rw-r----- 1 oracle asmadmin   114688 Sep  3 10:36 spf_SPORT2_20250903.bck

The restore will be easier with these files.

Prepare the json file for ODA deployment

Before doing the cleanup, prepare the json file you will use for deploying again your ODA.

You can use the one used for initial deployment, or eventually create a new one.

You will need these things for the json:

  • which edition you use: Standard Edition or Enterprise Edition
  • hostname, IP, netmask and gateway of your server
  • hostname, IP, netmask and gateway of your ILOM
  • user and group ids if you don’t use the default ones
  • ratio between DATA and RECO for disk partitioning (you may change from what was configured before)
  • DNS and NTP IPs
  • default’s network interface (usually btbond1)
  • DBTEST parameters if you create a test database during provisioning (I would recommend doing so)

You will find most of these parameters in the backup you’ve done with the script.

Here is my json file:

{
    "instance": {
        "instanceBaseName": "odaprdcr",
        "dbEdition": "EE",
        "objectStoreCredentials": null,
        "name": "odaprdcr",
        "systemPassword": "*********",
        "timeZone": "Europe/Zurich",
        "domainName": "dbi-services.ch",
        "ntpServers" : ["10.8.122.132","10.8.122.133"],
        "dnsServers" : ["10.8.122.132","10.8.122.233"],
        "isMultiUserAccessEnabled": "false",
        "isRoleSeparated": true,
        "osUserGroup": {
            "users": [
                {
                    "userName": "oracle",
                    "userRole": "oracleUser",
                    "userId": 1000
                },
                {
                    "userName": "grid",
                    "userRole": "gridUser",
                    "userId": 1001
                }
            ],
            "groups": [
                {
                    "groupName": "oinstall",
                    "groupRole": "oinstall",
                    "groupId": 1001
                },
                {
                    "groupName": "dbaoper",
                    "groupRole": "dbaoper",
                    "groupId": 1002
                },
                {
                    "groupName": "dba",
                    "groupRole": "dba",
                    "groupId": 1000
                },
                {
                    "groupName": "asmadmin",
                    "groupRole": "asmadmin",
                    "groupId": 1004
                },
                {
                    "groupName": "asmoper",
                    "groupRole": "asmoper",
                    "groupId": 1005
                },
                {
                    "groupName": "asmdba",
                    "groupRole": "asmdba",
                    "groupId": 1006
                }
            ]
        }
    },
    "nodes": [
        {
            "nodeNumber": "0",
            "nodeName": "odaprdcr",
            "network": [
                {
                    "ipAddress" : "10.192.100.103",
                    "subNetMask" : "255.255.255.0",
                    "gateway" : "10.192.100.254",
                    "nicName": "btbond1",
                    "networkType": [
                        "Public"
                    ],
                    "isDefaultNetwork": true
                }
            ],
            "ilom": {
                "ilomName": "odaprdcr-ilom",
                "ipAddress":"10.192.200.103",
                "subNetMask":"255.255.255.0",
                "gateway":"10.192.200.254"
            }
        }
    ],
    "grid": {
        "vip": [],
        "diskGroup": [
            {
                "diskGroupName": "DATA",
                "diskPercentage": 85,
                "redundancy": "NORMAL"
            },
            {
                "diskGroupName": "RECO",
                "diskPercentage": 15,
                "redundancy": "NORMAL"
            }
        ],
        "language": "en",
        "scan": null
    },
    "database": {
        "dbName": "DBTEST",
        "dbCharacterSet": {
            "characterSet": "AL32UTF8",
            "nlsCharacterset": "AL16UTF16",
            "dbTerritory": "AMERICA",
            "dbLanguage": "AMERICAN"
        },
        "dbRedundancy": null,
        "dbRole": "PRIMARY",
        "adminPassword": "*********",
        "dbEdition": "EE",
        "databaseUniqueName": "DBTEST",
        "dbClass": "OLTP",
        "dbVersion": "19.27.0.0.250415",
        "dbHomeId": null,
        "instanceOnly": false,
        "isCdb": true,
        "pdBName": "PDB1",
        "dbShape": "odb1",
        "pdbAdminuserName": "pdbadmin",
        "enableTDE": false,
        "dbType": "SI",
        "dbStorage": "ASM",
        "dbConsoleEnable": false,
        "dbOnFlashStorage": false,
        "backupConfigId": null,
        "rmanBkupPassword": null,
        "tdePassword": null
    },
    "acr": {
        "acrEnable": false
    },
    "sysOraHomeStorage": {
        "diskgroup": "DATA",
        "volumeSize": 80
    }
}

Cleanup the data disks

Cleanup of the data disk headers is mandatory before reimaging, otherwise provisioning will fail. Make sure you’re on the correct ODA, you can’t undo this action:

/opt/oracle/oak/onecmd/cleanup.pl
INFO: Log file is /opt/oracle/oak/log/odaprdcr/cleanup/cleanup_2025-09-03_10-53-04.log
INFO: Log file is /opt/oracle/oak/log/odaprdcr/cleanup/dcsemu_diag_precleanup_2025-09-03_10-53-04.log

INFO: *******************************************************************
INFO: ** Starting process to cleanup provisioned host odaprdcr         **
INFO: *******************************************************************
INFO: Default mode being used to cleanup a provisioned system.
INFO: It will change all ASM disk status from MEMBER to FORMER
Do you want to continue (yes/no) : yes
INFO: nodes will be rebooted
Do you want to continue (yes/no) : yes
INFO: /u01/app/19.20.0.0/oracle/bin/crsctl.bin

INFO: *************************************
INFO: ** Checking for GI bits presence
INFO: *************************************
INFO: GI bits /u01/app/19.20.0.0/oracle found on system under /u01/app directory...

INFO: ** Disabling AFD filtering
SUCCESS: AFD filtering disabled on all devices
INFO: ** Saving disk partition info
INFO: *************************************
INFO: ** Executing Stop Clusterware commands
INFO: *************************************
INFO: *************************************
INFO: ** Remove Oracle Stack
…

INFO: Rebooting the system via <reboot>...
INFO: Executing <reboot>

INFO: Cleanup was successful
INFO: Log file is /opt/oracle/oak/log/odaprdcr/cleanup/cleanup_2025-09-03_10-53-04.log

WARNING: After system reboot, please re-run "odacli update-repository" for GI/DB clones,
WARNING: before running "odacli create-appliance".

Reimage with the latest OS

ODA patch file 30403643 includes an ISO image you will use to reimage your ODA. From the ILOM, start the remote console, connect this ISO file, define next boot device as CDROM and do a power cycle of the server. Reimaging an ODA takes about 50 minutes and doesn’t need any input. Keep the remote console open for the next operation.

Configure firstnet

Once your ODA runs on a brand new OS, you will need to do the initial network setup through the remote console. So log into the console with root/welcome1, type configure-firstnet and provide IP address, netmask and gateway of your server. LACP mode will depend on your switch configuration, if you didn’t use it before, keep it disabled. On an ODA deployed in 2020, LACP mode was not available. You normally won’t use DHCP, neither VLAN as VLAN is usually tagged on the switch ports where your ODA is connected to.

configure-firstnet
...

Once configured, you can reach your server through the network and copy the needed files for the next steps.

Patch the microcodes

Reimaging your ODA doesn’t mean your microcodes are updated, it’s only the software part that is new. Therefore, first task after reimaging is patching these microcodes.

Let’s copy and register the patch file on the ODA and apply the updates (patch 37817290 for 19.27):

cd /opt/dbi
unzip p37817290_1927000_Linux-x86-64.zip
odacli update-repository -f /opt/dbi/oda-sm-19.27.0.0.0-250601-server.zip

odacli describe-job -i "1e2dd483-2237-47b9-9c60-5395fa720fd2"
Job details
----------------------------------------------------------------
                     ID:  1e2dd483-2237-47b9-9c60-5395fa720fd2
            Description:  Repository Update
                 Status:  Success
                Created:  September 03, 2025 09:53:24 UTC
                Message:  /opt/dbi/oda-sm-19.27.0.0.0-250601-server.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Unzip bundle                             September 03, 2025 09:53:39 UTC          September 03, 2025 09:53:54 UTC          Success

odacli describe-component
System Version
--------------
19.27.0.0.0

System Node Name
----------------
oak

Local System Version
--------------------
19.27.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.27.0.0.0          up-to-date

DCSCONTROLLER                            19.27.0.0.0          up-to-date

DCSCLI                                   19.27.0.0.0          up-to-date

DCSAGENT                                 19.27.0.0.0          up-to-date

DCSADMIN                                 19.27.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.1.23.r151750     5.1.4.25.r160118

BIOS                                     52100600             52140100

LOCAL CONTROLLER FIRMWARE {
     [c8, c9]                            214.2.271.9          up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1, c2, c3, c4, c5]            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c6d0, c6d1]                        XC311102             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

odacli create-prepatchreport -s -v 19.27.0.0.0
 
sleep 120 ; odacli describe-prepatchreport -i 8010302b-feca-4f23-92bc-6878aa586713
Prepatch Report
------------------------------------------------------------------------
                 Job ID:  8010302b-feca-4f23-92bc-6878aa586713
            Description:  Patch pre-checks for [OS, ILOM, SERVER] to 19.27.0.0.0
                 Status:  SUCCESS
                Created:  September 3, 2025 9:54:56 AM UTC
                 Result:  All pre-checks succeeded

Node Name
---------------
oak

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.27.0.0.0.
Is patch location available     Success   Patch location is available.
Verify OS patch                 Success   There are no packages available for
                                          an update
Validate command execution      Success   Skipped command execution verfication
                                          - Instance is not provisioned

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.27.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Skipped command execution verfication
                                          - Instance is not provisioned

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate command execution      Success   Skipped command execution verfication
                                          - Instance is not provisioned

odacli update-server -v 19.27.0.0.0

sleep 900 ; odacli describe-job -i 3a4ce5a7-6305-483d-a7c3-e6328d8a831e
Job details
----------------------------------------------------------------
                     ID:  3a4ce5a7-6305-483d-a7c3-e6328d8a831e
            Description:  Server Patching to 19.27.0.0.0
                 Status:  Success
                Created:  September 03, 2025 09:57:34 UTC
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Validate ILOM server reachable           September 03, 2025 09:57:36 UTC          September 03, 2025 09:57:37 UTC          Success
Stop DCS Admin                           September 03, 2025 09:57:38 UTC          September 03, 2025 09:57:39 UTC          Success
Generate mTLS certificates               September 03, 2025 09:57:39 UTC          September 03, 2025 09:57:40 UTC          Success
Exporting Public Keys                    September 03, 2025 09:57:40 UTC          September 03, 2025 09:57:41 UTC          Success
Creating Trust Store                     September 03, 2025 09:57:41 UTC          September 03, 2025 09:57:43 UTC          Success
Restart DCS Admin                        September 03, 2025 09:57:43 UTC          September 03, 2025 09:58:03 UTC          Success
Update config files                      September 03, 2025 09:57:43 UTC          September 03, 2025 09:57:43 UTC          Success
Deactivate Unit[dnf-makecache.timer]     September 03, 2025 09:58:04 UTC          September 03, 2025 09:58:04 UTC          Success
Deactivate Unit[kdump.service]           September 03, 2025 09:58:04 UTC          September 03, 2025 09:58:05 UTC          Success
Modify BM udev rules                     September 03, 2025 09:58:05 UTC          September 03, 2025 09:58:25 UTC          Success
Creating repositories using yum          September 03, 2025 09:58:25 UTC          September 03, 2025 09:58:27 UTC          Success
Updating YumPluginVersionLock rpm        September 03, 2025 09:58:27 UTC          September 03, 2025 09:58:29 UTC          Success
Applying OS Patches                      September 03, 2025 09:58:29 UTC          September 03, 2025 09:59:03 UTC          Success
Creating repositories using yum          September 03, 2025 09:59:03 UTC          September 03, 2025 09:59:03 UTC          Success
Applying HMP Patches                     September 03, 2025 09:59:04 UTC          September 03, 2025 09:59:06 UTC          Success
Applying Firmware local Disk Patches     September 03, 2025 09:59:07 UTC          September 03, 2025 09:59:10 UTC          Success
Oda-hw-mgmt upgrade                      September 03, 2025 09:59:07 UTC          September 03, 2025 09:59:07 UTC          Success
Patch location validation                September 03, 2025 09:59:07 UTC          September 03, 2025 09:59:07 UTC          Success
Applying Firmware local Controller Patch September 03, 2025 09:59:10 UTC          September 03, 2025 09:59:15 UTC          Success
Checking Ilom patch Version              September 03, 2025 09:59:15 UTC          September 03, 2025 09:59:15 UTC          Success
Patch location validation                September 03, 2025 09:59:15 UTC          September 03, 2025 09:59:15 UTC          Success
Save password in Wallet                  September 03, 2025 09:59:15 UTC          September 03, 2025 09:59:16 UTC          Success
Apply Ilom patch                         September 03, 2025 09:59:16 UTC          September 03, 2025 10:09:24 UTC          Success
Disabling IPMI v2                        September 03, 2025 09:59:16 UTC          September 03, 2025 09:59:16 UTC          Success
Cleanup JRE Home                         September 03, 2025 10:09:24 UTC          September 03, 2025 10:09:24 UTC          Success
Copying Flash Bios to Temp location      September 03, 2025 10:09:24 UTC          September 03, 2025 10:09:24 UTC          Success
Generating and saving BOM                September 03, 2025 10:09:24 UTC          September 03, 2025 10:09:27 UTC          Success
Update System version                    September 03, 2025 10:09:24 UTC          September 03, 2025 10:09:24 UTC          Success
PreRebootNode Actions                    September 03, 2025 10:09:27 UTC          September 03, 2025 10:09:27 UTC          Success
Reboot Node                              September 03, 2025 10:09:27 UTC          September 03, 2025 10:09:27 UTC          Success

The system will reboot once patched, let’s check the versions:

odacli describe-component
System Version
--------------
19.27.0.0.0

System Node Name
----------------
oak

Local System Version
--------------------
19.27.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.27.0.0.0          up-to-date

DCSCONTROLLER                            19.27.0.0.0          up-to-date

DCSCLI                                   19.27.0.0.0          up-to-date

DCSAGENT                                 19.27.0.0.0          up-to-date

DCSADMIN                                 19.27.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.4.25.r160118     up-to-date

BIOS                                     52140100             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c8, c9]                            214.2.271.9          up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c0, c1, c2, c3, c4, c5]            VDV1RL06             up-to-date
}

LOCAL DISK FIRMWARE {
     [c6d0, c6d1]                        XC311102             up-to-date
}

HMP                                      2.4.10.1.600         up-to-date

No need to apply the storage patch here, but if needed, apply it with:

odacli update-storage -v 19.27.0.0.0

Provision the appliance

Now it’s time to put back Oracle binaries on this system. First register the GI and DB clones, then do the create-appliance with the json file you prepared earlier:

cd /opt/dbi
unzip p30403662_1927000_Linux-x86-64.zip 
unzip p30403673_1927000_Linux-x86-64.zip

odacli update-repository -f /opt/dbi/odacli-dcs-19.27.0.0.0-250527-GI-19.27.0.0.zip
sleep 90 ; odacli describe-job -i c8f3559f-7f93-4df1-ba99-7511111f27ce

Job details
----------------------------------------------------------------
                     ID:  c8f3559f-7f93-4df1-ba99-7511111f27ce
            Description:  Repository Update
                 Status:  Success
                Created:  September 03, 2025 10:19:04 UTC
                Message:  /opt/dbi/odacli-dcs-19.27.0.0.0-250527-GI-19.27.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Unzip bundle                             September 03, 2025 10:19:04 UTC          September 03, 2025 10:20:08 UTC          Success

odacli update-repository -f /opt/dbi/odacli-dcs-19.27.0.0.0-250527-DB-19.27.0.0.zip
sleep 90 ; odacli describe-job -i 96187596-16f6-413c-8905-9d85c6d3e106

Job details
----------------------------------------------------------------
                     ID:  96187596-16f6-413c-8905-9d85c6d3e106
            Description:  Repository Update
                 Status:  Success
                Created:  September 03, 2025 10:20:50 UTC
                Message:  /opt/dbi/odacli-dcs-19.27.0.0.0-250527-DB-19.27.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Unzip bundle                             September 03, 2025 10:20:50 UTC          September 03, 2025 10:21:46 UTC          Success
 

odacli create-appliance -r /opt/dbi/deploy_odaprdcr.txt

Enter an initial password for Web Console account (oda-admin): **********
Confirm the password for Web Console account (oda-admin): **********
User 'oda-admin' created successfully...

odacli describe-job -i 22fec8d6-d9ae-4576-8f18-d8e1a0b00370
Job details
----------------------------------------------------------------
                     ID:  22fec8d6-d9ae-4576-8f18-d8e1a0b00370
            Description:  Provisioning service creation
                 Status:  Success
                Created:  September 03, 2025 12:45:37 CEST
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Validate absence of Interconnect         September 03, 2025 12:45:40 CEST         September 03, 2025 12:45:40 CEST         Success
network configuration file
Restart network interface pubnet         September 03, 2025 12:45:43 CEST         September 03, 2025 12:45:49 CEST         Success
Setting up Network                       September 03, 2025 12:45:43 CEST         September 03, 2025 12:45:43 CEST         Success
Setting up Vlan                          September 03, 2025 12:45:50 CEST         September 03, 2025 12:45:51 CEST         Success
Restart network interface priv0.100      September 03, 2025 12:45:51 CEST         September 03, 2025 12:45:52 CEST         Success
Restart network interface privasm        September 03, 2025 12:45:52 CEST         September 03, 2025 12:45:53 CEST         Success
Restart network interface privasm        September 03, 2025 12:45:53 CEST         September 03, 2025 12:45:59 CEST         Success
Setting up Network                       September 03, 2025 12:45:53 CEST         September 03, 2025 12:45:53 CEST         Success
Network update                           September 03, 2025 12:45:59 CEST         September 03, 2025 12:46:13 CEST         Success
Restart network interface btbond1        September 03, 2025 12:45:59 CEST         September 03, 2025 12:46:03 CEST         Success
Setting up Network                       September 03, 2025 12:45:59 CEST         September 03, 2025 12:45:59 CEST         Success
Updating network                         September 03, 2025 12:45:59 CEST         September 03, 2025 12:46:13 CEST         Success
Restart network interface btbond1        September 03, 2025 12:46:03 CEST         September 03, 2025 12:46:07 CEST         Success
Restart network interface pubnet         September 03, 2025 12:46:07 CEST         September 03, 2025 12:46:13 CEST         Success
OS usergroup 'asmadmin' creation         September 03, 2025 12:46:13 CEST         September 03, 2025 12:46:13 CEST         Success
OS usergroup 'asmdba' creation           September 03, 2025 12:46:13 CEST         September 03, 2025 12:46:13 CEST         Success
OS usergroup 'asmoper' creation          September 03, 2025 12:46:13 CEST         September 03, 2025 12:46:13 CEST         Success
OS usergroup 'dba' creation              September 03, 2025 12:46:13 CEST         September 03, 2025 12:46:14 CEST         Success
Validate availability of pubnet          September 03, 2025 12:46:13 CEST         September 03, 2025 12:46:13 CEST         Success
OS user 'grid' creation                  September 03, 2025 12:46:14 CEST         September 03, 2025 12:46:15 CEST         Success
OS usergroup 'dbaoper' creation          September 03, 2025 12:46:14 CEST         September 03, 2025 12:46:14 CEST         Success
OS usergroup 'oinstall' creation         September 03, 2025 12:46:14 CEST         September 03, 2025 12:46:14 CEST         Success
OS user 'oracle' creation                September 03, 2025 12:46:15 CEST         September 03, 2025 12:46:16 CEST         Success
Add SYSNAME in Env                       September 03, 2025 12:46:16 CEST         September 03, 2025 12:46:16 CEST         Success
Backup Config name validation            September 03, 2025 12:46:16 CEST         September 03, 2025 12:46:16 CEST         Success
Backup config metadata persist           September 03, 2025 12:46:16 CEST         September 03, 2025 12:46:16 CEST         Success
Default backup policy creation           September 03, 2025 12:46:16 CEST         September 03, 2025 12:46:16 CEST         Success
Grant permission to RHP files            September 03, 2025 12:46:16 CEST         September 03, 2025 12:46:16 CEST         Success
Install oracle-ahf                       September 03, 2025 12:46:16 CEST         September 03, 2025 12:47:23 CEST         Success
Stop DCS Admin                           September 03, 2025 12:48:25 CEST         September 03, 2025 12:48:26 CEST         Success
Generate mTLS certificates               September 03, 2025 12:48:26 CEST         September 03, 2025 12:48:27 CEST         Success
 
Exporting Public Keys                    September 03, 2025 12:48:27 CEST         September 03, 2025 12:48:28 CEST         Success
Creating Trust Store                     September 03, 2025 12:48:29 CEST         September 03, 2025 12:48:31 CEST         Success
Restart DCS Admin                        September 03, 2025 12:48:31 CEST         September 03, 2025 12:48:52 CEST         Success
Update config files                      September 03, 2025 12:48:31 CEST         September 03, 2025 12:48:31 CEST         Success
Creating GI home directories             September 03, 2025 12:48:52 CEST         September 03, 2025 12:48:52 CEST         Success
Extract GI clone                         September 03, 2025 12:48:52 CEST         September 03, 2025 12:50:11 CEST         Success
Grid home creation                       September 03, 2025 12:48:52 CEST         September 03, 2025 12:50:13 CEST         Success
Storage discovery                        September 03, 2025 12:50:13 CEST         September 03, 2025 12:51:33 CEST         Success
Creating wallet for Root User            September 03, 2025 12:51:33 CEST         September 03, 2025 12:51:36 CEST         Success
Creating wallet for ASM Client           September 03, 2025 12:51:36 CEST         September 03, 2025 12:51:39 CEST         Success
Grid stack creation                      September 03, 2025 12:51:39 CEST         September 03, 2025 13:00:38 CEST         Success
Provisioning GI with RHP                 September 03, 2025 12:51:39 CEST         September 03, 2025 12:57:27 CEST         Success
Set CRS ping target                      September 03, 2025 12:57:28 CEST         September 03, 2025 12:57:29 CEST         Success
Updating GI home version                 September 03, 2025 12:57:29 CEST         September 03, 2025 12:57:34 CEST         Success
Restarting Clusterware                   September 03, 2025 12:57:35 CEST         September 03, 2025 13:00:38 CEST         Success
Post cluster OAKD configuration          September 03, 2025 13:00:38 CEST         September 03, 2025 13:01:33 CEST         Success
Disk group 'RECO' creation               September 03, 2025 13:01:41 CEST         September 03, 2025 13:01:52 CEST         Success
Setting ACL for disk groups              September 03, 2025 13:01:52 CEST         September 03, 2025 13:01:55 CEST         Success
Modify DB file attributes                September 03, 2025 13:01:55 CEST         September 03, 2025 13:02:03 CEST         Success
Register Scan and Vips to Public Network September 03, 2025 13:02:03 CEST         September 03, 2025 13:02:05 CEST         Success
Configure export clones resource         September 03, 2025 13:03:24 CEST         September 03, 2025 13:03:24 CEST         Success
Volume 'commonstore' creation            September 03, 2025 13:03:24 CEST         September 03, 2025 13:03:41 CEST         Success
ACFS File system 'DATA' creation         September 03, 2025 13:03:41 CEST         September 03, 2025 13:04:03 CEST         Success
Creating ACFS database home              September 03, 2025 13:04:04 CEST         September 03, 2025 13:05:22 CEST         Success
Database home creation                   September 03, 2025 13:04:04 CEST         September 03, 2025 13:07:55 CEST         Success
Validating dbHome available space        September 03, 2025 13:05:23 CEST         September 03, 2025 13:05:23 CEST         Success
Create required directories              September 03, 2025 13:05:24 CEST         September 03, 2025 13:05:24 CEST         Success
Creating DbHome Directory                September 03, 2025 13:05:24 CEST         September 03, 2025 13:05:24 CEST         Success
Extract DB clone                         September 03, 2025 13:05:24 CEST         September 03, 2025 13:06:18 CEST         Success
ProvDbHome by using RHP                  September 03, 2025 13:06:18 CEST         September 03, 2025 13:07:33 CEST         Success
Enable DB options                        September 03, 2025 13:07:33 CEST         September 03, 2025 13:07:47 CEST         Success
Creating wallet for DB Client            September 03, 2025 13:07:51 CEST         September 03, 2025 13:07:55 CEST         Success
Database Creation by RHP                 September 03, 2025 13:07:56 CEST         September 03, 2025 13:20:28 CEST         Success
Provisioning service creation            September 03, 2025 13:07:56 CEST         September 03, 2025 13:23:47 CEST         Success
Change permission for xdb wallet files   September 03, 2025 13:20:28 CEST         September 03, 2025 13:20:29 CEST         Success
Add Startup Trigger to Open all PDBS     September 03, 2025 13:20:29 CEST         September 03, 2025 13:20:30 CEST         Success
Place SnapshotCtrlFile in sharedLoc      September 03, 2025 13:20:30 CEST         September 03, 2025 13:20:33 CEST         Success
SqlPatch upgrade                         September 03, 2025 13:22:08 CEST         September 03, 2025 13:22:30 CEST         Success
Running dbms_stats init_package          September 03, 2025 13:22:30 CEST         September 03, 2025 13:22:31 CEST         Success
Set log_archive_dest for Database        September 03, 2025 13:22:31 CEST         September 03, 2025 13:22:34 CEST         Success
Updating the Database version            September 03, 2025 13:22:34 CEST         September 03, 2025 13:22:38 CEST         Success
Users tablespace creation                September 03, 2025 13:23:47 CEST         September 03, 2025 13:23:50 CEST         Success
Clear all listeners from Database        September 03, 2025 13:23:50 CEST         September 03, 2025 13:23:50 CEST         Success
Clear local and remote listeners from    September 03, 2025 13:23:52 CEST         September 03, 2025 13:23:53 CEST         Success
Database
Copy Pwfile to Shared Storage            September 03, 2025 13:23:53 CEST         September 03, 2025 13:23:59 CEST         Success
Persist new agent state entry            September 03, 2025 13:24:02 CEST         September 03, 2025 13:24:02 CEST         Success
Persist new agent state entry            September 03, 2025 13:24:02 CEST         September 03, 2025 13:24:02 CEST         Success
Provisioning service creation            September 03, 2025 13:24:02 CEST         September 03, 2025 13:24:02 CEST         Success
Restart DCS Agent                        September 03, 2025 13:24:02 CEST         September 03, 2025 13:24:03 CEST         Success

Creating the appliance usually takes 40 to 50 minutes.

Check core reduction

Reimaging an ODA will keep the core reduction, but make sure it’s true with these commands:

odacli list-cpucores
Node  Cores  Modified                           Job Status
----- ------ ---------------------------------- ---------------
0     4      September 3, 2025 9:46:11 AM CEST  CONFIGURED

lscpu | grep Core
Core(s) per socket:  2

Here I have 4 enabled cores, meaning 2 processor licenses.

Setup older DB home

As I will bring back my Data Guard setup with 19.20 databases, I will need a 19.20 DB home. Let’s register the clone file and create the DB home. An increase of the diagnostic dest is probably needed, 10GB is way too small for me:

odacli list-dbhome-storages
=============================================================================================================
ID                                     Node Description          Disk Group Volume      Size(GB)   Status
-------------------------------------- ---- -------------------- ---------- ----------- ---------- ----------
bef3ae8a-96d3-44d4-b179-1e5dbdd2a939   0    ORACLE_HOME          DATA       orahome_sh  80         CONFIGURED
e972cc17-ff92-4203-b23f-55fab02a7cc0   0    ORACLE_BASE          DATA       odabase_n0  10         CONFIGURED
=============================================================================================================

odacli modify-dbhome-storage -i e972cc17-ff92-4203-b23f-55fab02a7cc0 -s 50

sleep 10 ; odacli list-dbhome-storages
=============================================================================================================
ID                                     Node Description          Disk Group Volume      Size(GB)   Status
-------------------------------------- ---- -------------------- ---------- ----------- ---------- ----------
bef3ae8a-96d3-44d4-b179-1e5dbdd2a939   0    ORACLE_HOME          DATA       orahome_sh  80         CONFIGURED
e972cc17-ff92-4203-b23f-55fab02a7cc0   0    ORACLE_BASE          DATA       odabase_n0  50         CONFIGURED
=============================================================================================================


cd /opt/dbi
unzip p30403662_1920000_Linux-x86-64.zip 
odacli update-repository -f /opt/dbi/odacli-dcs-19.20.0.0.0-230720-DB-19.20.0.0.zip

sleep 30 ; odacli describe-job -i 3ee5db4d-d350-47b4-8e24-18949d244ab7
Job details
----------------------------------------------------------------
                     ID:  3ee5db4d-d350-47b4-8e24-18949d244ab7
            Description:  Repository Update
                 Status:  Success
                Created:  September 03, 2025 13:47:23 CEST
                Message:  /opt/dbi/odacli-dcs-19.20.0.0.0-230720-DB-19.20.0.0.zip

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Unzip bundle                             September 03, 2025 13:47:23 CEST         September 03, 2025 13:47:55 CEST         Success

odacli create-dbhome -v 19.20.0.0.230718

sleep 240 ; odacli describe-job -i 3123544b-60c5-418a-82b5-c110dd68aafa
Job details
----------------------------------------------------------------
                     ID:  3123544b-60c5-418a-82b5-c110dd68aafa
            Description:  Database Home OraDB19000_home2 creation with version :19.20.0.0.230718
                 Status:  Success
                Created:  September 03, 2025 13:53:21 CEST
                Message:  Create Database Home

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Setting up SSH equivalence               September 03, 2025 13:53:32 CEST         September 03, 2025 13:53:34 CEST         Success
Setting up SSH equivalence               September 03, 2025 13:53:34 CEST         September 03, 2025 13:53:35 CEST         Success
Creating ACFS database home              September 03, 2025 13:53:35 CEST         September 03, 2025 13:53:35 CEST         Success
Validating dbHome available space        September 03, 2025 13:53:35 CEST         September 03, 2025 13:53:35 CEST         Success
Create required directories              September 03, 2025 13:53:36 CEST         September 03, 2025 13:53:36 CEST         Success
Creating DbHome Directory                September 03, 2025 13:53:36 CEST         September 03, 2025 13:53:36 CEST         Success
Extract DB clone                         September 03, 2025 13:53:36 CEST         September 03, 2025 13:54:23 CEST         Success
ProvDbHome by using RHP                  September 03, 2025 13:54:23 CEST         September 03, 2025 13:55:39 CEST         Success
Enable DB options                        September 03, 2025 13:55:39 CEST         September 03, 2025 13:55:54 CEST         Success
Creating wallet for DB Client            September 03, 2025 13:55:57 CEST         September 03, 2025 13:55:58 CEST         Success

odacli list-dbhomes
ID                                       Name                 DB Version           DB Edition Home Location                                            Status
---------------------------------------- -------------------- -------------------- ---------- -------------------------------------------------------- ----------
19ee71be-07ae-4373-9bb6-31db8195e36c     OraDB19000_home1     19.27.0.0.250415     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_1     CONFIGURED
cd2dc372-3f95-40f1-917c-ee0793c26664     OraDB19000_home2     19.20.0.0.230718     EE         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2     CONFIGURED

Create the databases

Your ODA registry needs to know the databases, so I will create them before doing the restore. I don’t use the -io (instance only) of odacli for database creation as srvctl entries will be missing. So let’s create a new standalone database and then remove its files:

odacli create-database -cs AL32UTF8 -no-c -u BMC3PRD_CR -dh 'cd2dc372-3f95-40f1-917c-ee0793c26664' -n BMC3PRD -s odb1s -l AMERICAN -dt AMERICA -no-co -r asm

sleep 660 ; odacli describe-job -i 40b80359-0607-4330-ae6e-055d284a780e
Job details
----------------------------------------------------------------
                     ID:  40b80359-0607-4330-ae6e-055d284a780e
            Description:  Database service creation with DB name: BMC3PRD
                 Status:  Success
                Created:  September 03, 2025 15:11:48 CEST
                Message:

Task Name                                Start Time                               End Time                                 Status
---------------------------------------- ---------------------------------------- ---------------------------------------- -------
Database Creation by RHP                 September 03, 2025 15:11:50 CEST         September 03, 2025 15:19:24 CEST         Success
Database Service creation                September 03, 2025 15:11:50 CEST         September 03, 2025 15:22:19 CEST         Success
Setting up SSH equivalence               September 03, 2025 15:11:50 CEST         September 03, 2025 15:11:50 CEST         Success
Setting up SSH equivalence               September 03, 2025 15:11:50 CEST         September 03, 2025 15:11:50 CEST         Success
Change permission for xdb wallet files   September 03, 2025 15:19:24 CEST         September 03, 2025 15:19:25 CEST         Success
Place SnapshotCtrlFile in sharedLoc      September 03, 2025 15:19:25 CEST         September 03, 2025 15:19:28 CEST         Success
SqlPatch upgrade                         September 03, 2025 15:20:43 CEST         September 03, 2025 15:20:50 CEST         Success
Running dbms_stats init_package          September 03, 2025 15:20:50 CEST         September 03, 2025 15:20:51 CEST         Success
Set log_archive_dest for Database        September 03, 2025 15:20:51 CEST         September 03, 2025 15:20:54 CEST         Success
Updating the Database version            September 03, 2025 15:20:54 CEST         September 03, 2025 15:20:58 CEST         Success
Create Users tablespace                  September 03, 2025 15:22:19 CEST         September 03, 2025 15:22:20 CEST         Success
Clear all listeners from Database        September 03, 2025 15:22:20 CEST         September 03, 2025 15:22:21 CEST         Success
Copy Pwfile to Shared Storage            September 03, 2025 15:22:23 CEST         September 03, 2025 15:22:27 CEST         Success

su - oracle
. oraenv <<< BMC3PRD
sqlplus / as sysdba
set pages 0
set feedback off
spool /tmp/rmfiles.sh
select 'asmcmd rm -fr '||file_name from dba_data_files;
select 'asmcmd rm -fr '||file_name from dba_temp_files;
select 'asmcmd rm -fr '||member from v$logfile;
select 'asmcmd rm -fr '||name from v$controlfile;
select 'asmcmd rm -fr '||value from v$parameter where name='spfile';
spool off
shutdown abort;
exit
exit
su - grid
sh /tmp/rmfiles.sh
exit

Do the same for the other databases. Don’t try to create your databases with multiple odacli jobs running at the same time: this is something ODA cannot do in parallel, creation will be serialized and jobs will be waiting for the current one to complete.

Restore the databases

I’m used to move the spfile from ASM to the filesystem, so it’s the exact same path as the other server. In the same time, I change the role of the database as odacli created it as a primary:

su - oracle
. oraenv <<< BMC3PRD
srvctl modify database -db BMC3PRD_CR -spfile /u01/app/oracle/admin/BMC3PRD/pfile/spfileBMC3PRD.ora -startoption mount -role PHYSICAL_STANDBY

Now let’s restore the spfile and the controlfile. RMAN is able to start an instance without any spfile for recovering the spfile, it’s very convenient:

rman target / 

startup nomount;
restore spfile from '/backup/Patch19.27/ctl_spf/spf_BMC3PRD_20250903.bck';
Starting restore at 04-SEP-25
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=433 device type=DISK
channel ORA_DISK_1: restoring spfile from AUTOBACKUP /backup/Patch19.27/ctl_spf/spf_BMC3PRD_20250903.bck
channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
Finished restore at 04-SEP-25
 
startup force nomount;

restore controlfile from '/backup/Patch19.27/ctl_spf/ctl_BMC3PRD_20250903.bck';

Starting restore at 04-SEP-25
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=26 device type=DISK
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:05
output file name=+RECO/BMC3PRD_CR/CONTROLFILE/current.486.1210937913
Finished restore at 04-SEP-25

Now I can start restoring the database. Choose the parallel degree carefully: as your ODA has nothing else to do, I would recommend using the maximum channels you could for big databases, for example between 4 and 8 channels for an ODA with 4 enabled cores:

startup force mount;
run {
allocate channel c1 device type disk;
allocate channel c2 device type disk;
allocate channel c3 device type disk;
allocate channel c4 device type disk;
restore database ;
}


…
channel c4: restoring datafile 00004 to +DATA/BMC3PRD/DATAFILE/undotbs1.291.1042201603
channel c4: restoring datafile 00005 to +DATA/BMC3PRD/DATAFILE/users.294.1042202001
channel c4: reading from backup piece /backup/BMC3PRD/backupset/20250901_000002_inc0_BMC3PRD_45931446_s36914_p1.bck
channel c3: piece handle=/backup/BMC3PRD/backupset/20250901_000002_inc0_BMC3PRD_45931446_s36908_p2.bck tag=INC0_20250901_000002
channel c3: restored backup piece 2
channel c3: restore complete, elapsed time: 00:00:38
channel c1: piece handle=/backup/BMC3PRD/backupset/20250901_000002_inc0_BMC3PRD_45931446_s36911_p2.bck tag=INC0_20250901_000002
channel c1: restored backup piece 2
channel c1: restore complete, elapsed time: 00:00:42
channel c4: piece handle=/backup/BMC3PRD/backupset/20250901_000002_inc0_BMC3PRD_45931446_s36914_p1.bck tag=INC0_20250901_000002
channel c4: restored backup piece 1
channel c4: restore complete, elapsed time: 00:00:35
channel c2: piece handle=/backup/BMC3PRD/backupset/20250901_000002_inc0_BMC3PRD_45931446_s36911_p1.bck tag=INC0_20250901_000002
channel c2: restored backup piece 1
channel c2: restore complete, elapsed time: 00:00:54
Finished restore at 04-SEP-25
released channel: c1
released channel: c2
released channel: c3
released channel: c4

exit

Add TNS entries for Data Guard and a static entry for the listener

Your database is now restored, you’ll need to configure communication with the primary. Let’s add the following TNS entries:

vi $ORACLE_HOME/network/admin/tnsnames.ora
…

BMC3PRD_CR =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = odaprdcr)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = BMC3PRD_CR)
    )
  )

BMC3PRD_IN =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = odaprdin)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = BMC3PRD_IN)
    )
  )

You will also need the static entry for your database in the listener.ora file:

su - grid
vi $ORACLE_HOME/network/admin/listener.ora
SID_LIST_LISTENER=
(SID_LIST=
  (SID_DESC=
    (GLOBAL_DBNAME = BMC3PRD_CR)
    (ORACLE_HOME = /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2)
    (SID_NAME = BMC3PRD)
  )
)
exit

Copy the password file and modify its location

Pick up the password file from primary server and relocate the file to the filesystem with srvctl:

scp oracle@odaprdin:/u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2/dbs/orapwBMC3PRD /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2/dbs/

srvctl modify database -db BMC3PRD_CR -pwfile
srvctl stop database -db BMC3PRD_CR ; sleep 10 ; srvctl start database -db BMC3PRD_CR -o mount

You can use a symbolic link in the dbs subfolder if you want to put this password file alongside your spfile.

Recover the database

Restored datafiles are probably several hours old, so let’s do a recover on the standby from the primary service before configuring Data Guard:

rman target sys
recover database from service 'BMC3PRD_IN';
…
channel ORA_DISK_1: using network backup set from service BMC3PRD_IN
destination for restore of datafile 00007: +DATA/BMC3PRD_CR/DATAFILE/dm_mc3prd_docbase.477.1210938063
channel ORA_DISK_1: restore complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: using network backup set from service BMC3PRD_IN
destination for restore of datafile 00008: +DATA/BMC3PRD_CR/DATAFILE/dm_mc3prd_docbase.478.1210938063
channel ORA_DISK_1: restore complete, elapsed time: 00:00:08
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: using network backup set from service BMC3PRD_IN
destination for restore of datafile 00009: +DATA/BMC3PRD_CR/DATAFILE/dm_mc3prd_docbase.479.1210938067
channel ORA_DISK_1: restore complete, elapsed time: 00:00:03
channel ORA_DISK_1: starting incremental datafile backup set restore
channel ORA_DISK_1: using network backup set from service BMC3PRD_IN
destination for restore of datafile 00010: +DATA/BMC3PRD_CR/DATAFILE/dm_mc3prd_docbase.480.1210938165
channel ORA_DISK_1: restore complete, elapsed time: 00:00:07

starting media recovery

media recovery complete, elapsed time: 00:00:00
Finished recover at 04-SEP-25

exit

If archivelog files are missing or if recovery is not possible, it doesn’t mean that it’s a problem. Data Guard should be able to solve a small or bigger archive gap.

Rebuild Data Guard configuration

Instead of adding back my standby database to the existing Data Guard configuration, I will recreate this configuration. It’s a matter of seconds:

dgmgrl sys
*****
Remove configuration;
create configuration DTG as primary database is 'BMC3PRD_IN' connect identifier is 'BMC3PRD_IN';
add database 'BMC3PRD_CR' as connect identifier is 'BMC3PRD_CR';
enable configuration;
edit database 'BMC3PRD_CR' set property LogXptMode='SYNC';
edit database 'BMC3PRD_IN' set property LogXptMode='SYNC';
edit database 'BMC3PRD_CR' set property StandbyFileManagement='AUTO';
edit database 'BMC3PRD_IN' set property StandbyFileManagement='AUTO';
EDIT DATABASE 'BMC3PRD_CR' SET PROPERTY 'ArchiveLagTarget'=1200; 
EDIT DATABASE 'BMC3PRD_IN' SET PROPERTY 'ArchiveLagTarget'=1200; 
edit configuration set protection mode as maxavailability;

Depending on the lag between your primary and this standby, it can takes several minutes to bring back the sync of both databases:

show configuration lag;
Configuration - dtg

  Protection Mode: MaxAvailability
  Members:
  BMC3PRD_IN - Primary database
    BMC3PRD_CR - Physical standby database
                  Transport Lag:      0 seconds (computed 1 second ago)
                  Apply Lag:          0 seconds (computed 1 second ago)

Fast-Start Failover:  Disabled

Configuration Status:
SUCCESS   (status updated 6 seconds ago)

This database is OK, let’s continue with the next ones.

Additional settings

If you made additional settings on your ODA, you’ll have to configure them again. It could be:

  • monitoring
  • additional network interfaces
  • additional services
  • mount points and shares
  • firewall rules
  • kerberos authentication
  • compatibility modes in sqlnet.ora file
  • …

Conclusion

With several multi-TB databases to restore, it took me less than 3 days per ODA for this operation. And these ODAs are now clean and ready for their second life.

Reimaging an ODA from scratch is a good solution as soon as a Data Guard setup allows the databases to run on another ODA for several days. And with a good preparation, the job is not that huge. And there is much less troubleshooting compared to patching. This is something to consider.

L’article Reimage your ODA from scratch est apparu en premier sur dbi Blog.

]]> https://www.dbi-services.com/blog/reimage-your-oda-from-scratch/feed/ 0