Introduction

Patch 19.18 is now available on Oracle Database Appliance. It’s time to test it.

What’s new?

This version brings latest PSUs to database and grid homes, with their bug fixes as usual. It also brings latest 21c and 19c DB Systems. If you plan using 21c, remember that it’s an innovation release only. 23c is the new long-term release, but it’s only available as a beta version and not yet for ODA.

No breaking new features for this version, but some nice additions:

  • support for LACP: some of us were complaining about the bonding mode only supporting active/failover, meaning that only 1 of the 2 interfaces was carrying the load. This is no more true as LACP is finally supported (active/active). This is definitely a good thing! It can be configured for new deployments as well as on already deployed servers with the -o option of the configure-firstnet command line. Be careful because LACP means a specific configuration on your switch ports, I would only setup this mode for new deployments
  • enhancement for update-registry: this tool is useful to do a kind of resync your ODA environment with the ODA registry that may be needed in some specific situations

Which ODA is compatible with this 19.18?

The ODAs X9-2S/L/HA are supported, as well as X8, X7 and X6 series. X5-2HA is still on the compatibility list, but I would recommand renewing this rather old appliance this year. You can expect a big increase of performance moving to a newer SSD-based ODA.

Is this patch a cumulative one?

This 19.18 can be applied on top of 19.14 or later. I just applied this new patch on a X8-2S running on 19.7 using 19.10 and 19.14 as intermediate versions.

Is there also a patch for my databases?

Only databases version 19c are supported, and this is OK because this is the only one you should use now. You may use Data Preserving Reprovisioning if you come from a very old version (<=18.8) and want to keep your 11g or 18c for example. I wrote a blog post on this feature several months ago.

If you absolutely need an older release, it should be possible to register an old DB clone belonging to an older release without any problem. Several days ago, I deployed a 12.2 DB home from 19.14 on top of a brand new X9-2S running 19.18. But I would recommend using 19c only with this patch version. Using old databases is not supported, may not work properly with odacli, and you should be aware that 11g, 12cR1/R2 and 18c are obsolete now.

Download the patch and clone files

Download the patch and the corresponding clones to be able to apply the complete patch.

34956008 => the patch itself
30403673 => the GI clone needed for deploying newer GI version (mandatory)
30403662 => the DB clone for deploying new version of 19c

You don’t need the ISO file for patching, but I would recommend to download it (patch 30403643).

Be sure to choose the very latest 19.18 when downloading the clones, download link will first propose older versions.

Prepare the patching

Before running prepatch, please check these prerequisites:

  • filesystems have 20% available free space (does not concern acfs volumes)
  • additional rpms manually installed should be removed
  • revert profile scripts to default’s one (for grid and oracle users)
  • make sure you planned a generous downtime, 4 hours being the bare minimum for patching and troubleshooting. 1 day is never too much. If you need to apply intermediate patches, you will need 4 hours for each jump
  • you may need to remove old kernel version if you already patched multiple times your ODA, I described this in the next chapter

You should use odabr to make snapshots of the important filesystems prior patching. It’s fast and doesn’t cost anything. I would also recommend doing a backup of most important files: tnsnames.ora and listener.ora, database list, network configuration files, aso.

Clean-up old kernels

If you have already patched multiple times your ODA, you may need to clean-up old kernels. Basically, each time an OS patch is applied, a new kernel is installed and it may fill up your /boot volume:

df -h /boot 
Filesystem      Size  Used Avail Use% Mounted on
/dev/md126p2    477M  359M   93M  80% /boot


uname -a 
Linux chlaora01 4.14.35-2047.510.5.4.el7uek.x86_64 #2 SMP Thu Jan 20 09:10:35 PST 2022 x86_64 x86_64 x86_64 GNU/Linux


rpm -qa | grep kernel-uek-4
kernel-uek-4.14.35-2047.510.5.4.el7uek.x86_64
kernel-uek-4.14.35-2025.404.1.2.el7uek.x86_64
kernel-uek-4.14.35-1902.301.1.el7uek.x86_64

In this example, current kernel is 4.14.35-2047. The 1902 and 2025 kernel headers are not needed anymore. Let’s manually remove the packages:

rpm -e kernel-uek-4.14.35-1902.301.1.el7uek.x86_64
rpm -e kernel-uek-4.14.35-2025.404.1.2.el7uek.x86_64

df -h /boot
Filesystem      Size  Used Avail Use% Mounted on
/dev/md126p2    477M  153M  300M  34% /boot

You can now start your patching.

Version precheck

Start to check current version on all components:

odacli describe-component | grep -v ^$
System Version
---------------
19.14.0.0.0
System node Name
---------------
chlaora01
Local System Version
---------------
19.14.0.0.0
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK
                                          19.14.0.0.0           up-to-date
GI
                                          19.14.0.0.220118      up-to-date
DB {
[ OraDB19000_home1 ]
                                          19.10.0.0.210119      19.14.0.0.220118
[ OraDB19000_home2 ]
                                          19.14.0.0.220118      up-to-date
}
DCSCONTROLLER
                                          19.14.0.0.0           up-to-date
DCSCLI
                                          19.14.0.0.0           up-to-date
DCSAGENT
                                          19.14.0.0.0           up-to-date
DCSADMIN
                                          19.14.0.0.0           up-to-date
OS
                                          7.9                   up-to-date
ILOM
                                          5.0.2.24.r141466      up-to-date
BIOS
                                          52050300              up-to-date
SHARED CONTROLLER FIRMWARE
                                          VDV1RL04              up-to-date
LOCAL DISK FIRMWARE
                                          1120                  1132
SHARED DISK FIRMWARE
                                          1120                  1132
ASR
                                          21.2.2                up-to-date
HMP
                                          2.4.8.0.600           up-to-date

Once the patch will be registered in the ODA repository, the “Available Version” column will be updated with versions provided within the patch.

Prepararing the patch and updating the DCS tools

Copy the patch files on your ODA in a temp directory. On ODA X9-2 system disks are now smaller, so don’t hesitate to put the file on a nfs share or in the FRA acfs volume if you have acfs configured for your databases. Then unzip the files:

cd /backup/ODA_patches
unzip p34956008_1918000_Linux-x86-64.zip
unzip p30403673_1918000_Linux-x86-64.zip
unzip p30403662_1918000_Linux-x86-64.zip


odacli update-repository -f /backup/ODA_patches/odacli-dcs-19.18.0.0.0-230205-GI-19.18.0.0.zip,/backup/ODA_patches/oda-sm-19.18.0.0.0-230205-server.zip

odacli describe-job -i "8b54784c-b51c-4924-baed-8e356482f714"

Job details
----------------------------------------------------------------
                     ID:  8b54784c-b51c-4924-baed-8e356482f714
            Description:  Repository Update
                 Status:  Success
                Created:  March 14, 2023 2:32:45 PM CET
                Message:  /backup/ODA_patches/odacli-dcs-19.18.0.0.0-230205-GI-19.18.0.0.zip,/backup/ODA_patches/oda-sm-19.18.0.0.0-230205-server.zip

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle                             March 14, 2023 2:32:45 PM CET       March 14, 2023 2:33:46 PM CET       Success



odacli describe-component | grep -v ^$
System Version
---------------
19.14.0.0.0
System node Name
---------------
chlaora01
Local System Version
---------------
19.14.0.0.0
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK
                                          19.14.0.0.0           19.18.0.0.0
GI
                                          19.14.0.0.220118      19.18.0.0.230117
DB {
[ OraDB19000_home1 ]
                                          19.10.0.0.210119      19.18.0.0.230117
[ OraDB19000_home2 ]
                                          19.14.0.0.220118      19.18.0.0.230117
}
DCSCONTROLLER
                                          19.14.0.0.0           19.18.0.0.0
DCSCLI
                                          19.14.0.0.0           19.18.0.0.0
DCSAGENT
                                          19.14.0.0.0           19.18.0.0.0
DCSADMIN
                                          19.14.0.0.0           19.18.0.0.0
OS
                                          7.9                   up-to-date
ILOM
                                          5.0.2.24.r141466      5.1.0.23.r146986
BIOS
                                          52050300              52080100
SHARED CONTROLLER FIRMWARE
                                          VDV1RL04              VDV1RL05
LOCAL DISK FIRMWARE
                                          1120                  not-available
SHARED DISK FIRMWARE
                                          1120                  not-available
ASR
                                          21.2.2                22.2.0
HMP
                                          2.4.8.0.600           2.4.8.9.604

Patching from 19.14 is the maximum gap for this 19.18 patch. Let’s start with the DCS components update:

odacli update-dcsadmin -v 19.18.0.0.0
odacli describe-job -i "34225798-f9dd-488d-9f80-051785adadb1"

Job details
----------------------------------------------------------------
                     ID:  34225798-f9dd-488d-9f80-051785adadb1
            Description:  DcsAdmin patching
                 Status:  Success
                Created:  March 14, 2023 2:38:04 PM CET
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Patch location validation                March 14, 2023 2:38:05 PM CET       March 14, 2023 2:38:05 PM CET       Success
dcs-admin upgrade                        March 14, 2023 2:38:05 PM CET       March 14, 2023 2:38:11 PM CET       Success

odacli update-dcscomponents -v 19.18.0.0.0
This job is interactive
{
  "jobId" : "047df467-1cef-4862-92e6-bcc627fb4ff3",
  "status" : "Success",
  "message" : "Update-dcscomponents is successful on all the node(s):DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata migration is done before. Metadata schema update is done. dcsagent RPM upgrade is successful.  dcscli RPM upgrade is successful.  dcscontroller RPM upgrade is successful.  Successfully ran setupAgentAuth.sh zookeeper RPM upgrade is successful.  ",
  "reports" : null,
  "createTimestamp" : "March 14, 2023 14:39:02 PM CET",
  "description" : "Update-dcscomponents job completed and is not part of Agent job list",
  "updatedTime" : "March 14, 2023 14:40:38 PM CET"
}

odacli update-dcsagent -v 19.18.0.0.0

sleep 180 ; odacli describe-job -i "89abf479-e673-41f2-98a2-d56dc1847653"


Job details
----------------------------------------------------------------
                     ID:  89abf479-e673-41f2-98a2-d56dc1847653
            Description:  DcsAgent patching
                 Status:  Success
                Created:  March 14, 2023 2:40:58 PM CET
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Stop DCS Admin                           March 14, 2023 2:41:00 PM CET       March 14, 2023 2:41:00 PM CET       Success
Generate mTLS certificates               March 14, 2023 2:41:00 PM CET       March 14, 2023 2:41:01 PM CET       Success
Exporting Public Keys                    March 14, 2023 2:41:01 PM CET       March 14, 2023 2:41:02 PM CET       Success
Creating Trust Store                     March 14, 2023 2:41:02 PM CET       March 14, 2023 2:41:04 PM CET       Success
Update config files                      March 14, 2023 2:41:04 PM CET       March 14, 2023 2:41:05 PM CET       Success
Restart DCS Admin                        March 14, 2023 2:41:05 PM CET       March 14, 2023 2:41:25 PM CET       Success
Dcs-agent upgrade  to version 19.18.0.0.0 March 14, 2023 2:41:25 PM CET       March 14, 2023 2:44:03 PM CET       Success
Update System version                    March 14, 2023 2:44:03 PM CET       March 14, 2023 2:44:03 PM CET       Success

System prepatching report

Let’s do the prepatching test:

odacli create-prepatchreport -s -v 19.18.0.0.0

odacli describe-prepatchreport -i ebdec9e7-655b-4879-b932-18acd3c1783d

Patch pre-check report
------------------------------------------------------------------------
                 Job ID:  ebdec9e7-655b-4879-b932-18acd3c1783d
            Description:  Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER, SERVER]
                 Status:  SUCCESS
                Created:  March 14, 2023 2:44:58 PM CET
                 Result:  All pre-checks succeeded

Node Name
---------------
chlaora01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.18.0.0.0.
Is patch location available     Success   Patch location is available.
Verify OS patch                 Success   Verified OS patch
Validate command execution      Success   Validated command execution

__ILOM__
Validate ILOM server reachable  Success   Successfully connected with ILOM
                                          server using public IP and USB
                                          interconnect
Validate supported versions     Success   Validated minimum supported versions.
Validate patching tag           Success   Validated patching tag: 19.18.0.0.0.
Is patch location available     Success   Patch location is available.
Checking Ilom patch Version     Success   Successfully verified the versions
Patch location validation       Success   Successfully validated location
Validate command execution      Success   Validated command execution

__GI__
Validate GI metadata            Success   Successfully validated GI metadata
Validate supported GI versions  Success   Successfully validated minimum version
Validate available space        Success   Validated free space under /u01
Is clusterware running          Success   Clusterware is running
Validate patching tag           Success   Validated patching tag: 19.18.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate ASM in online          Success   ASM is online
Validate kernel log level       Success   Successfully validated the OS log
                                          level
Validate minimum agent version  Success   GI patching enabled in current
                                          DCSAGENT version
Validate Central Inventory      Success   oraInventory validation passed
Validate patching locks         Success   Validated patching locks
Validate clones location exist  Success   Validated clones location
Validate DB start dependencies  Success   DBs START dependency check passed
Validate DB stop dependencies   Success   DBs STOP dependency check passed
Validate space for clones       Success   Clones volume is already created
volume
Evaluate GI patching            Success   Successfully validated GI patching
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Success   Successfully ran Orachk
Validate command execution      Success   Validated command execution

__SERVER__
Validate local patching         Success   Successfully validated server local
                                          patching
Validate command execution      Success   Validated command execution

Everything is OK to start patching.

Patching server and GI

Let’s start the update-server:

odacli update-server -v 19.18.0.0.0
odacli describe-job -i e99e6059-c616-4abc-83f3-0b45bc435c87


Job details
----------------------------------------------------------------
                     ID:  e99e6059-c616-4abc-83f3-0b45bc435c87
            Description:  Server Patching
                 Status:  Success
                Created:  March 14, 2023 2:53:42 PM CET
                Message:  Successfully patched GI with RHP

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validating GI user metadata              March 14, 2023 2:53:58 PM CET       March 14, 2023 2:53:58 PM CET       Success
Validate ILOM server reachable           March 14, 2023 2:53:58 PM CET       March 14, 2023 2:53:59 PM CET       Success
Validate DCS Admin mTLS setup            March 14, 2023 2:53:59 PM CET       March 14, 2023 2:53:59 PM CET       Success
Configure export clones resource         March 14, 2023 2:54:00 PM CET       March 14, 2023 2:54:00 PM CET       Success
Creating repositories using yum          March 14, 2023 2:54:00 PM CET       March 14, 2023 2:54:04 PM CET       Success
Updating YumPluginVersionLock rpm        March 14, 2023 2:54:04 PM CET       March 14, 2023 2:54:04 PM CET       Success
Applying OS Patches                      March 14, 2023 2:54:04 PM CET       March 14, 2023 3:03:06 PM CET       Success
Creating repositories using yum          March 14, 2023 3:03:06 PM CET       March 14, 2023 3:03:06 PM CET       Success
Applying HMP Patches                     March 14, 2023 3:03:06 PM CET       March 14, 2023 3:03:22 PM CET       Success
Patch location validation                March 14, 2023 3:03:22 PM CET       March 14, 2023 3:03:22 PM CET       Success
Oda-hw-mgmt upgrade                      March 14, 2023 3:03:23 PM CET       March 14, 2023 3:03:53 PM CET       Success
OSS Patching                             March 14, 2023 3:03:54 PM CET       March 14, 2023 3:03:54 PM CET       Success
Applying Firmware Disk Patches           March 14, 2023 3:03:54 PM CET       March 14, 2023 3:03:59 PM CET       Success
Applying Firmware Controller Patches     March 14, 2023 3:03:59 PM CET       March 14, 2023 3:10:22 PM CET       Success
Checking Ilom patch Version              March 14, 2023 3:10:22 PM CET       March 14, 2023 3:10:22 PM CET       Success
Patch location validation                March 14, 2023 3:10:22 PM CET       March 14, 2023 3:10:22 PM CET       Success
Save password in Wallet                  March 14, 2023 3:10:23 PM CET       March 14, 2023 3:10:23 PM CET       Success
Disabling IPMI v2                        March 14, 2023 3:10:23 PM CET       March 14, 2023 3:10:23 PM CET       Success
Apply Ilom patch                         March 14, 2023 3:10:23 PM CET       March 14, 2023 3:14:06 PM CET       Success
Copying Flash Bios to Temp location      March 14, 2023 3:14:06 PM CET       March 14, 2023 3:14:06 PM CET       Success
Patch location validation                March 14, 2023 3:14:06 PM CET       March 14, 2023 3:14:06 PM CET       Success
ASR Manager RPM update                   March 14, 2023 3:14:06 PM CET       March 14, 2023 3:19:02 PM CET       Success
Modify JavaExec Path                     March 14, 2023 3:19:02 PM CET       March 14, 2023 3:19:02 PM CET       Success
Remove AsrConfBackup File                March 14, 2023 3:19:02 PM CET       March 14, 2023 3:19:02 PM CET       Success
Starting the clusterware                 March 14, 2023 3:19:02 PM CET       March 14, 2023 3:20:39 PM CET       Success
Registering image                        March 14, 2023 3:20:39 PM CET       March 14, 2023 3:20:40 PM CET       Success
Registering working copy                 March 14, 2023 3:20:40 PM CET       March 14, 2023 3:20:40 PM CET       Success
Registering image                        March 14, 2023 3:20:40 PM CET       March 14, 2023 3:20:40 PM CET       Success
Creating GI home directories             March 14, 2023 3:20:40 PM CET       March 14, 2023 3:20:40 PM CET       Success
Extract GI clone                         March 14, 2023 3:20:40 PM CET       March 14, 2023 3:20:40 PM CET       Success
Provisioning Software Only GI with RHP   March 14, 2023 3:20:40 PM CET       March 14, 2023 3:20:40 PM CET       Success
Patch GI with RHP                        March 14, 2023 3:20:40 PM CET       March 14, 2023 3:26:32 PM CET       Success
Updating GIHome version                  March 14, 2023 3:26:32 PM CET       March 14, 2023 3:26:36 PM CET       Success
Validate GI availability                 March 14, 2023 3:26:40 PM CET       March 14, 2023 3:26:40 PM CET       Success
Patch KVM CRS type                       March 14, 2023 3:26:40 PM CET       March 14, 2023 3:26:40 PM CET       Success
Patch VM vDisks CRS dependencies         March 14, 2023 3:26:40 PM CET       March 14, 2023 3:26:40 PM CET       Success
Patch DB System domain config            March 14, 2023 3:26:40 PM CET       March 14, 2023 3:26:41 PM CET       Success
Update System version                    March 14, 2023 3:26:41 PM CET       March 14, 2023 3:26:41 PM CET       Success
Cleanup JRE Home                         March 14, 2023 3:26:41 PM CET       March 14, 2023 3:26:41 PM CET       Success
Add SYSNAME in Env                       March 14, 2023 3:26:41 PM CET       March 14, 2023 3:26:41 PM CET       Success
Starting the clusterware                 March 14, 2023 3:26:41 PM CET       March 14, 2023 3:26:41 PM CET       Success
Setting ACL for disk groups              March 14, 2023 3:26:41 PM CET       March 14, 2023 3:26:44 PM CET       Success
Enable LKCE                              March 14, 2023 3:28:14 PM CET       March 14, 2023 3:30:13 PM CET       Success
Update lvm.conf file                     March 14, 2023 3:30:14 PM CET       March 14, 2023 3:30:14 PM CET       Success
Update previous workarounds              March 14, 2023 3:30:14 PM CET       March 14, 2023 3:30:14 PM CET       Success
Generating and saving BOM                March 14, 2023 3:30:14 PM CET       March 14, 2023 3:31:13 PM CET       Success
PreRebootNode Actions                    March 14, 2023 3:31:13 PM CET       March 14, 2023 3:32:00 PM CET       Success
Reboot Ilom                              March 14, 2023 3:32:00 PM CET       March 14, 2023 3:32:00 PM CET       Success

Server reboots 5 minutes after the patch ends. On this X8-2S server patching lasted 40 minutes.

odacli describe-component | grep -v ^$
System Version
---------------
19.18.0.0.0
System node Name
---------------
chlaora01
Local System Version
---------------
19.18.0.0.0
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK
                                          19.18.0.0.0           up-to-date
GI
                                          19.18.0.0.230117      up-to-date
DB {
[OraDB19000_home1]
                                          19.10.0.0.210119      19.18.0.0.230117
[OraDB19000_home2 [DBTEST,BFCTEST,
TLITEST,TLIDEV1,DCMTEST,TLIPRD,BFCPRD]]   19.14.0.0.220118      19.18.0.0.230117
}
DCSCONTROLLER
                                          19.18.0.0.0           up-to-date
DCSCLI
                                          19.18.0.0.0           up-to-date
DCSAGENT
                                          19.18.0.0.0           up-to-date
DCSADMIN
                                          19.18.0.0.0           up-to-date
OS
                                          7.9                   up-to-date
ILOM
                                          5.0.2.24.r141466      5.1.0.23.r146986
BIOS
                                          52050300              52080100
LOCAL CONTROLLER FIRMWARE {
[c3]
                                          80000681              up-to-date
[c4]
                                          8000C470              up-to-date
[c5]
                                          214.2.271.9           up-to-date
}
SHARED CONTROLLER FIRMWARE
                                          VDV1RL04              VDV1RL05
LOCAL DISK FIRMWARE
                                          XC311120              XC311132
SHARED DISK FIRMWARE
                                          XC311120              XC311132
ASR
                                          22.2.0                up-to-date
HMP
                                          2.4.8.9.604           up-to-date

ILOM and BIOS should have been patched here, but odacli didn’t manage to update these components. This is not a problem and you could do this manually as I described here.

Patches for BIOS and ILOM are located here:

ls -lrt /opt/oracle/oak/pkgrepos/ilom/x8-2l/5.1.0.23.r146986/
total 39428
-rwxrwx--- 1 root root 40322121 Feb  5 16:36 ILOM-5_1_0_23_r146986-ORACLE_SERVER_X8-2L-rom.pkg
-rwxrwx--- 1 root root      465 Feb  5 16:36 componentmetadata.xml

Patching the storage

Patching the storage is only needed if the storage components are not up-to-date. On my X8-2S I had to patch:

odacli update-storage -v 19.18.0.0.0
odacli describe-job -i "e9bd7b60-306b-4634-b59d-ea1c8f6f3b4f"

Job details
----------------------------------------------------------------
                     ID:  e9bd7b60-306b-4634-b59d-ea1c8f6f3b4f
            Description:  Storage Firmware Patching
                 Status:  Success
                Created:  March 14, 2023 3:40:47 PM CET
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Applying Firmware Disk Patches           March 14, 2023 3:40:52 PM CET       March 14, 2023 3:40:58 PM CET       Success
Applying Firmware Controller Patches     March 14, 2023 3:40:58 PM CET       March 14, 2023 3:44:39 PM CET       Success
Generating and saving BOM                March 14, 2023 3:44:39 PM CET       March 14, 2023 3:44:51 PM CET       Success
PreRebootNode Actions                    March 14, 2023 3:44:51 PM CET       March 14, 2023 3:44:51 PM CET       Success
Reboot Ilom                              March 14, 2023 3:44:51 PM CET       March 14, 2023 3:44:51 PM CET       Success


odacli describe-component | grep -v ^$

System Version
---------------
19.18.0.0.0
System node Name
---------------
chlaora01
Local System Version
---------------
19.18.0.0.0
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK
                                          19.18.0.0.0           up-to-date
GI
                                          19.18.0.0.230117      up-to-date
DB {
[OraDB19000_home1]
                                          19.10.0.0.210119      19.18.0.0.230117
[OraDB19000_home2 [DBTEST,BFCTEST,
TLITEST,TLIDEV1,DCMTEST,TLIPRD,BFCPRD]]   19.14.0.0.220118      19.18.0.0.230117
}
DCSCONTROLLER
                                          19.18.0.0.0           up-to-date
DCSCLI
                                          19.18.0.0.0           up-to-date
DCSAGENT
                                          19.18.0.0.0           up-to-date
DCSADMIN
                                          19.18.0.0.0           up-to-date
OS
                                          7.9                   up-to-date
ILOM
                                          5.0.2.24.r141466      5.1.0.23.r146986
BIOS
                                          52050300              52080100
LOCAL CONTROLLER FIRMWARE {
[c3]
                                          80000681              up-to-date
[c4]
                                          8000C470              up-to-date
[c5]
                                          214.2.271.9           up-to-date
}
SHARED CONTROLLER FIRMWARE
                                          VDV1RL05              up-to-date
LOCAL DISK FIRMWARE
                                          XC311120              XC311132
SHARED DISK FIRMWARE
                                          XC311120              XC311132
ASR
                                          22.2.0                up-to-date
HMP
                                          2.4.8.9.604           up-to-date

Disks are not up-to-date, but it may be related to ILOM and BIOS patches. I won’t do anything for this now.

Patching the DB homes

Time for patching the DB homes depends on the number of DB homes and number of databases. In this example, I will apply the patch on my 19.14 DB home. The 19.10 DB home is an old one coming from the previous patch.

odacli update-repository -f /backup/ODA_patches/odacli-dcs-19.18.0.0.0-230205-DB-19.18.0.0.zip

odacli describe-job -i "682e6a7e-590d-4993-b1ec-d21ff862281e"

Job details
----------------------------------------------------------------
                     ID:  682e6a7e-590d-4993-b1ec-d21ff862281e
            Description:  Repository Update
                 Status:  Success
                Created:  March 14, 2023 3:51:54 PM CET
                Message:  /backup/ODA_patches/odacli-dcs-19.18.0.0.0-230205-DB-19.18.0.0.zip

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle                             March 14, 2023 3:51:54 PM CET       March 14, 2023 4:43:21 PM CET       Success

odacli list-dbhomes
ID                                       Name                 DB Version                               Home Location                                 Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
68db772c-4131-4981-83b9-4d5ea6bd43e3     OraDB19000_home1     19.10.0.0.210119                         /u01/app/oracle/product/19.0.0.0/dbhome_1     CONFIGURED
9e3a0ba4-de37-4087-94b5-156c260930c5     OraDB19000_home2     19.14.0.0.220118                         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2 CONFIGURED



odacli create-prepatchreport -d -i 9e3a0ba4-de37-4087-94b5-156c260930c5 -v 19.18.0.0.0
odacli describe-prepatchreport -i 73d9dafe-360b-438b-bd22-3a587f51bb51

Patch pre-check report
------------------------------------------------------------------------
                 Job ID:  73d9dafe-360b-438b-bd22-3a587f51bb51
            Description:  Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home2
                 Status:  FAILED
                Created:  March 14, 2023 4:45:19 PM CET
                 Result:  One or more pre-checks failed for [ORACHK]

Node Name
---------------
chlaora01

Pre-Check                      Status   Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID             Success   Validated DB Home ID:
                                          9e3a0ba4-de37-4087-94b5-156c260930c5
Validate patching tag           Success   Validated patching tag: 19.18.0.0.0.
Is system provisioned           Success   Verified system is provisioned
Validate minimum agent version  Success   Validated minimum agent version
Is GI upgraded                  Success   Validated GI is upgraded
Validate available space for    Success   Validated free space required under
db                                        /u01
Validate dbHomesOnACFS          Success   User has configured disk group for
configured                                Database homes on ACFS
Validate Oracle base            Success   Successfully validated Oracle Base
Is DB clone available           Success   Successfully validated clone file
                                          exists
Evaluate DBHome patching with   Success   Successfully validated updating
RHP                                       dbhome with RHP.  and local patching
                                          is possible
Validate command execution      Success   Validated command execution

__ORACHK__
Running orachk                  Failed    ORAchk validation failed: .
Validate command execution      Success   Validated command execution
Check for parameter             Failed    AHF-: Database parameter GLOBAL_NAMES
global_names                              is not set to recommended value
Check for parameter             Failed    AHF-: Database parameter GLOBAL_NAMES
global_names                              is not set to recommended value
Check for parameter             Failed    AHF-: Database parameter GLOBAL_NAMES
global_names                              is not set to recommended value
Check for parameter             Failed    AHF-: Database parameter GLOBAL_NAMES
global_names                              is not set to recommended value
Check for parameter recyclebin  Failed    AHF-: RECYCLEBIN on PRIMARY should be
                                          set to the recommended value
Verify the Fast Recovery Area   Failed    AHF-: FRA space management problem
(FRA) has reclaimable space               file types are present without an
                                          RMAN backup completion within the
                                          last 7 days

I don’t care about Orachk recommendations on my databases because it has been set like that. I will apply the patch on this DB home with the force option:

odacli update-dbhome -i 9e3a0ba4-de37-4087-94b5-156c260930c5 -v 19.18.0.0.0 -f

odacli describe-job -i "833dc970-baf7-4d52-a510-49e70ef3df72"


Job details
----------------------------------------------------------------
                     ID:  833dc970-baf7-4d52-a510-49e70ef3df72
            Description:  DB Home Patching: Home Id is 9e3a0ba4-de37-4087-94b5-156c260930c5
                 Status:  Success
                Created:  March 14, 2023 5:14:01 PM CET
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Creating wallet for DB Client            March 14, 2023 5:14:50 PM CET       March 14, 2023 5:14:50 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:14:50 PM CET       March 14, 2023 5:19:25 PM CET       Success
Updating database metadata               March 14, 2023 5:19:25 PM CET       March 14, 2023 5:19:25 PM CET       Success
Set log_archive_dest for Database        March 14, 2023 5:19:25 PM CET       March 14, 2023 5:19:28 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:19:28 PM CET       March 14, 2023 5:23:44 PM CET       Success
Updating database metadata               March 14, 2023 5:23:44 PM CET       March 14, 2023 5:23:44 PM CET       Success
Set log_archive_dest for Database        March 14, 2023 5:23:44 PM CET       March 14, 2023 5:23:46 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:23:46 PM CET       March 14, 2023 5:24:39 PM CET       Success
Updating database metadata               March 14, 2023 5:24:39 PM CET       March 14, 2023 5:24:39 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:24:39 PM CET       March 14, 2023 5:25:32 PM CET       Success
Updating database metadata               March 14, 2023 5:25:32 PM CET       March 14, 2023 5:25:32 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:25:32 PM CET       March 14, 2023 5:29:47 PM CET       Success
Updating database metadata               March 14, 2023 5:29:47 PM CET       March 14, 2023 5:29:47 PM CET       Success
Set log_archive_dest for Database        March 14, 2023 5:29:47 PM CET       March 14, 2023 5:29:50 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:29:50 PM CET       March 14, 2023 5:33:57 PM CET       Success
Updating database metadata               March 14, 2023 5:33:57 PM CET       March 14, 2023 5:33:57 PM CET       Success
Set log_archive_dest for Database        March 14, 2023 5:33:57 PM CET       March 14, 2023 5:33:59 PM CET       Success
Patch databases by RHP                   March 14, 2023 5:33:59 PM CET       March 14, 2023 5:38:07 PM CET       Success
Updating database metadata               March 14, 2023 5:38:07 PM CET       March 14, 2023 5:38:07 PM CET       Success
Set log_archive_dest for Database        March 14, 2023 5:38:07 PM CET       March 14, 2023 5:38:10 PM CET       Success
Update System version                    March 14, 2023 5:38:10 PM CET       March 14, 2023 5:38:10 PM CET       Success
Generating and saving BOM                March 14, 2023 5:38:10 PM CET       March 14, 2023 5:39:20 PM CET       Success
TDE parameter update                     March 14, 2023 5:40:06 PM CET       March 14, 2023 5:40:06 PM CET       Success

odacli list-dbhomes

ID                                       Name                 DB Version                               Home Location                                 Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
68db772c-4131-4981-83b9-4d5ea6bd43e3     OraDB19000_home1     19.10.0.0.210119                         /u01/app/oracle/product/19.0.0.0/dbhome_1     CONFIGURED
9e3a0ba4-de37-4087-94b5-156c260930c5     OraDB19000_home2     19.14.0.0.220118                         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_2 CONFIGURED
d701329e-4d58-4cbd-a081-57d473fbf06f     OraDB19000_home3     19.18.0.0.230117                         /u01/app/odaorahome/oracle/product/19.0.0.0/dbhome_3 CONFIGURED

odacli list-databases

ID                                       DB Name    DB Type  DB Version           CDB        Class    Shape    Storage    Status        DbHomeID
---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ----------------------------------------
605e9186-8aa1-47d7-8c0a-705d52583078     DBTEST     SI       19.18.0.0.230117     false      OLTP     odb1     ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
0a9e8668-f466-4df2-b4ec-214d784c514d     BFRTEST    SI       19.18.0.0.230117     false      OLTP     odb1s    ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
95258c52-fd37-4da2-a175-498e9e8da7b2     TLRTEST    SI       19.18.0.0.230117     false      OLTP     odb1s    ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
ec87f711-0e87-496e-8035-0fb96d3df6b2     TLRDEV     SI       19.18.0.0.230117     false      OLTP     odb1s    ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
cc11f047-a9bb-41cd-a91c-c879365e1464     DCRTEST    SI       19.18.0.0.230117     false      OLTP     odb1s    ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
26c9180b-a9f5-4fbd-98cf-4b7ba4124270     TLRPRD     SI       19.18.0.0.230117     false      OLTP     odb1     ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f
c110dc36-aa4b-4d48-8402-a2630dc5d22f     BFRPRD     SI       19.18.0.0.230117     false      OLTP     odb1     ASM        CONFIGURED   d701329e-4d58-4cbd-a081-57d473fbf06f

Final checks

Let’s get the final versions:

odacli describe-component | grep -v ^$
System Version
---------------
19.18.0.0.0
System node Name
---------------
chlaora01
Local System Version
---------------
19.18.0.0.0
Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK
                                          19.18.0.0.0           up-to-date
GI
                                          19.18.0.0.230117      up-to-date
DB {
[OraDB19000_home1]
                                          19.10.0.0.210119      19.18.0.0.230117
[OraDB19000_home2]
                                          19.14.0.0.220118      19.18.0.0.230117
[OraDB19000_home3 [DBTEST,BFCTEST,
TLITEST,TLIDEV1,DCMTEST,TLIPRD,BFCPRD]]   19.18.0.0.230117      up-to-date
}
DCSCONTROLLER
                                          19.18.0.0.0           up-to-date
DCSCLI
                                          19.18.0.0.0           up-to-date
DCSAGENT
                                          19.18.0.0.0           up-to-date
DCSADMIN
                                          19.18.0.0.0           up-to-date
OS
                                          7.9                   up-to-date
ILOM
                                          5.0.2.24.r141466      5.1.0.23.r146986
BIOS
                                          52050300              52080100
LOCAL CONTROLLER FIRMWARE {
[c3]
                                          80000681              up-to-date
[c4]
                                          8000C470              up-to-date
[c5]
                                          214.2.271.9           up-to-date
}
SHARED CONTROLLER FIRMWARE
                                          VDV1RL05              up-to-date
LOCAL DISK FIRMWARE
                                          XC311120              XC311132
SHARED DISK FIRMWARE
                                          XC311120              XC311132
ASR
                                          22.2.0                up-to-date
HMP
                                          2.4.8.9.604           up-to-date

This is not exactly what I was expecting, but as the software part is OK, it’s not that bad.

Cleanse the old patches

The old patches will never be used again, so don’t forget to remove them from the repository if your ODA has already been patched:

odacli cleanup-patchrepo -cl -v 19.14.0.0.0

odacli describe-job -i b42190e0-1d36-4639-8c62-d8a7c652c6f9


Job details
----------------------------------------------------------------
                     ID:  b42190e0-1d36-4639-8c62-d8a7c652c6f9
            Description:  Cleanup patchrepos
                 Status:  Success
                Created:  March 14, 2023 5:42:59 PM CET
                Message:

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Cleanup Repository                       March 14, 2023 5:42:59 PM CET       March 14, 2023 5:42:59 PM CET       Success
Cleanup JRE Home                         March 14, 2023 5:42:59 PM CET       March 14, 2023 5:42:59 PM CET       Success
Cleanup old ASR rpm                      March 14, 2023 5:42:59 PM CET       March 14, 2023 5:42:59 PM CET       Success

Post-patching tasks

Once everything is OK, don’t forget to:

  • add your additional rpms manually if needed
  • put back your profile scripts for grid and oracle users
  • remove the snapshots done with odabr
  • remove the old DB homes if they are not needed anymore

Patching DB Systems

If you use DB Systems on your ODA, meaning that some of your databases are running in dedicated VMs, you will need to apply the patch inside each DB System. I haven’t tried on my X8-2S because it only runs bare metal databases, but you will need to list your DB Systems, connect to each one, and apply the server and database patches:

odacli list-dbsystems
ssh ...
odacli update-server -v 19.18.0.0.0
odacli update-dbhome -i ... -v 19.18.0.0.0

Patching an ODA with DB Systems will need more time than a bare metal configuration.

Conclusion

This release is easy to apply coming from 19.14 and also probably from newer versions. Future releases may include a major OS upgrade and 23c sooner or later.