By Mouhamadou Diaw
In this blog, I describe the patching of an ODA X8 from 19.9 to 19.11. We use following patches
p32790643_1911000_Linux-x86-64.zip : Oracle Database Appliance Server Patch for ODACLI/DCS Stack
p30403673_1911000_Linux-x86-64.zip : Oracle Database Appliance GI Clone for ODACLI/DCS stack
p30403662_1911000_Linux-x86-64.zip : Oracle Database Appliance RDBMS Clone 19.11.0.0.210420 for ODACLI/DCS stack
Patching the Oracle Database Appliance Server
Before starting the process we first take a backup of the system with odabr ( ODA (Oracle Database Appliance): ODABR a System Backup/Restore Utility (Doc ID 2466177.1))
After dowloading the latest version, we do install odabr
[root@oda-prod software]# rpm -ivh odabr-2.0.1-70.noarch.rpm
warning: odabr-2.0.1-70.noarch.rpm: Header V4 RSA/SHA1 Signature, key ID 939112d6: NOKEY
Preparing... ################################# [100%]
Updating / installing...
1:odabr-2.0.1-70 ################################# [100%]
odabr-2.0.1.70 has been installed on /opt/odabr succesfully!
[root@oda-prod software]#
We can verify that there is actually no snapshots
[root@oda-prod software]# /opt/odabr/odabr infosnap
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
odabr - ODA node Backup Restore - Version: 2.0.1-70
Copyright 2013, 2021, Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton <[email protected]>
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
LVM snap name Status COW Size Data%
------------- ---------- ---------- ------
WARNING: 2022-08-30 09:07:58: LVM snapshot for 'root' does not exist
WARNING: 2022-08-30 09:07:58: LVM snapshot for 'opt' does not exist
WARNING: 2022-08-30 09:07:58: LVM snapshot for 'u01' does not exist
Let’s do a backup with odabr
[root@oda-prod software]# /opt/odabr/odabr backup -snap -rsize 15 -usize 90 -osize 60
INFO: 2022-08-30 09:08:51: Please check the logfile '/opt/odabr/out/log/odabr_35684.log' for more details
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
odabr - ODA node Backup Restore - Version: 2.0.1-70
Copyright 2013, 2021, Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton <[email protected]>
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
INFO: 2022-08-30 09:08:51: Checking superuser
INFO: 2022-08-30 09:08:51: Checking Bare Metal
INFO: 2022-08-30 09:08:51: Removing existing LVM snapshots
WARNING: 2022-08-30 09:08:51: LVM snapshot for 'opt' does not exist
WARNING: 2022-08-30 09:08:51: LVM snapshot for 'u01' does not exist
WARNING: 2022-08-30 09:08:51: LVM snapshot for 'root' does not exist
INFO: 2022-08-30 09:08:51: Checking current OS version
INFO: 2022-08-30 09:08:51: Checking LVM restore backgroud process
INFO: 2022-08-30 09:08:51: Checking LVM size
INFO: 2022-08-30 09:08:51: Boot device backup
INFO: 2022-08-30 09:08:51: Getting EFI device
INFO: 2022-08-30 09:08:51: ...step1 - unmounting EFI
INFO: 2022-08-30 09:08:51: ...step2 - making efi device backup
SUCCESS: 2022-08-30 09:08:52: ...EFI device backup saved as '/opt/odabr/out/hbi/efi.img'
INFO: 2022-08-30 09:08:52: ...step3 - checking EFI device backup
INFO: 2022-08-30 09:08:52: Getting boot device
INFO: 2022-08-30 09:08:52: ...step1 - making boot device backup using tar
SUCCESS: 2022-08-30 09:08:57: ...boot content saved as '/opt/odabr/out/hbi/boot.tar.gz'
INFO: 2022-08-30 09:08:57: ...step2 - unmounting boot
INFO: 2022-08-30 09:08:57: ...step3 - making boot device backup using dd
SUCCESS: 2022-08-30 09:08:59: ...boot device backup saved as '/opt/odabr/out/hbi/boot.img'
INFO: 2022-08-30 09:08:59: ...step4 - mounting boot
INFO: 2022-08-30 09:08:59: ...step5 - mounting EFI
INFO: 2022-08-30 09:08:59: ...step6 - checking boot device backup
INFO: 2022-08-30 09:08:59: Making OCR physical backup
INFO: 2022-08-30 09:09:02: ...ocr backup saved as '/opt/odabr/out/hbi/ocrbackup_35684.bck'
SUCCESS: 2022-08-30 09:09:02: OCR physical backup created successfully
INFO: 2022-08-30 09:09:02: OCR export backup
INFO: 2022-08-30 09:09:03: ...ocr export saved as '/opt/odabr/out/hbi/ocrexport_35684.bck'
SUCCESS: 2022-08-30 09:09:03: OCR export backup created successfully
INFO: 2022-08-30 09:09:03: Making LVM snapshot backup
SUCCESS: 2022-08-30 09:09:04: ...snapshot backup for 'opt' created successfully
SUCCESS: 2022-08-30 09:09:05: ...snapshot backup for 'u01' created successfully
SUCCESS: 2022-08-30 09:09:05: ...snapshot backup for 'root' created successfully
SUCCESS: 2022-08-30 09:09:05: LVM snapshots backup done successfully
We should now have snapshots
[root@oda-prod software]# /opt/odabr/odabr infosnap
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
odabr - ODA node Backup Restore - Version: 2.0.1-70
Copyright 2013, 2021, Oracle and/or its affiliates.
--------------------------------------------------------
Author: Ruggero Citton <[email protected]>
RAC Pack, Cloud Innovation and Solution Engineering Team
│▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒│
LVM snap name Status COW Size Data%
------------- ---------- ---------- ------
root_snap active 15.00 GiB 0.01%
opt_snap active 60.00 GiB 0.01%
u01_snap active 90.00 GiB 0.01%
[root@oda-prod software]#
Now we can start the patching . First let’s unpack the patch p32790643_1911000_Linux-x86-64.zip
[root@oda-prod software]# unzip p32790643_1911000_Linux-x86-64.zip
Archive: p32790643_1911000_Linux-x86-64.zip
extracting: oda-sm-19.11.0.0.0-210530-server.zip
inflating: README.txt
[root@x0042db02 local_dumps]#
Then update the repository with the patch
[root@oda-prod software]# odacli describe-job -i "832fbb4d-2186-4b85-9978-7f642e04f3a9"
Job details
----------------------------------------------------------------
ID: 832fbb4d-2186-4b85-9978-7f642e04f3a9
Description: Repository Update
Status: Success
Created: August 30, 2022 8:56:54 AM CEST
Message: /software/oda-sm-19.11.0.0.0-210530-server.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle August 30, 2022 8:56:54 AM CEST August 30, 2022 8:57:18 AM CEST Success
[root@oda-prod software]#
Validate that the job is successful
[root@oda-prod software]# odacli describe-job -i "832fbb4d-2186-4b85-9978-7f642e04f3a9"
Job details
----------------------------------------------------------------
ID: 832fbb4d-2186-4b85-9978-7f642e04f3a9
Description: Repository Update
Status: Success
Created: August 30, 2022 8:56:54 AM CEST
Message: /software/oda-sm-19.11.0.0.0-210530-server.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle August 30, 2022 8:56:54 AM CEST August 30, 2022 8:57:18 AM CEST Success
[root@oda-prod software]#
Then we update the DCS_ADMIN
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-dcsadmin -v 19.11.0.0.0
{
"jobId" : "fb14a1d0-52ce-43c3-871d-c93fdaff75b9",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "August 30, 2022 09:11:57 AM CEST",
"resourceList" : [ ],
"description" : "DcsAdmin patching",
"updatedTime" : "August 30, 2022 09:11:57 AM CEST"
}
The job should return SUCCESS
[root@oda-prod software]# odacli describe-job -i "fb14a1d0-52ce-43c3-871d-c93fdaff75b9"
Job details
----------------------------------------------------------------
ID: fb14a1d0-52ce-43c3-871d-c93fdaff75b9
Description: DcsAdmin patching
Status: Success
Created: August 30, 2022 9:11:57 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Patch location validation August 30, 2022 9:11:57 AM CEST August 30, 2022 9:11:57 AM CEST Success
dcs-admin upgrade August 30, 2022 9:11:57 AM CEST August 30, 2022 9:12:03 AM CEST Success
[root@oda-prod software]#
Let’s update the DCS components
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-dcscomponents -v 19.11.0.0. 0
{
"jobId" : "a274b632-92a8-4f8f-b05d-b3c5d7c67dae",
"status" : "Success",
"message" : "Update-dcscomponents is successful on all the node(s):DCS-Agent shutdown is successful. MySQL installation is successful. Metadata migration is successful. Metadata schema update is done. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. Successfully ran setupAgentAuth.sh zookeeper RPM upgrade is successful. DCS-Agent restart is successful. ",
"reports" : null,
"createTimestamp" : "August 30, 2022 09:13:37 AM CEST",
"description" : "Update-dcscomponents job completed and is not part of Agent job list",
"updatedTime" : "August 30, 2022 09:15:41 AM CEST"
}
[root@oda-prod software]#
The describe-job for this job doesn’t return anything. It’s normal, don’t worry
[root@oda-prod software]# odacli describe-job -i "a274b632-92a8-4f8f-b05d-b3c5d7c67dae"
DCS-10000:Resource Job with ID a274b632-92a8-4f8f-b05d-b3c5d7c67dae is not found.
We can see the status of our components at this point
[root@oda-prod software]# odacli describe-component
System Version
---------------
19.11.0.0.0
System node Name
---------------
oda-prod
Local System Version
---------------
19.11.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.9.0.0.0 19.11.0.0.0
GI 19.9.0.0.201020 19.11.0.0.210420
DB {
[ OraDB19000_home1 ] 19.9.0.0.201020 19.11.0.0.210420
[ OraDB11204_home1 ] 11.2.0.4.201020 not-available
}
DCSAGENT 19.11.0.0.0 up-to-date
OS 7.8 7.9
ILOM 5.0.1.21.r136383 5.0.1.21.a.r138015
BIOS 52030400 up-to-date
FIRMWARECONTROLLER VDV1RL04 up-to-date
FIRMWAREDISK 1102 up-to-date
ASR 20.3.0 up-to-date
HMP 2.4.7.0.1 2.4.8.0.600
[root@oda-prod software]#
We can now update the DCS agent
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-dcsagent -v 19.11.0.0.0
{
"jobId" : "0ea4fc54-4797-47a0-966d-de1f3d14bda4",
"status" : "Created",
"message" : "Dcs agent will be restarted after the update. Please wait for 2-3 mins before executing the other commands",
"reports" : [ ],
"createTimestamp" : "August 30, 2022 09:21:11 AM CEST",
"resourceList" : [ ],
"description" : "DcsAgent patching",
"updatedTime" : "August 30, 2022 09:21:11 AM CEST"
}
[root@oda-prod software]#
The job status should return SUCCESS
[root@oda-prod software]# odacli describe-job -i "0ea4fc54-4797-47a0-966d-de1f3d14bda4"
Job details
----------------------------------------------------------------
ID: 0ea4fc54-4797-47a0-966d-de1f3d14bda4
Description: DcsAgent patching
Status: Success
Created: August 30, 2022 9:21:11 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
dcs-agent upgrade to version 19.11.0.0.0 August 30, 2022 9:21:12 AM CEST August 30, 2022 9:23:32 AM CEST Success
Update System version August 30, 2022 9:23:32 AM CEST August 30, 2022 9:23:32 AM CEST Success
[root@oda-prod software]#
Now it’s time to update the repository with grid patch
We unpack the corresponding patch
[root@oda-prod software]# unzip p30403673_1911000_Linux-x86-64.zip
Archive: p30403673_1911000_Linux-x86-64.zip
extracting: odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip
replace README.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: y
inflating: README.txt
[root@oda-prod software]# ls -l odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip
-rwxrwxrwx 1 root root 5550326480 May 31 2021 odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip
[root@oda-prod software]#
We update the repository with the patch
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-repository -f /software/odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip
{
"jobId" : "17adc146-c10f-418a-b08b-f7d4c9390cbc",
"status" : "Created",
"message" : "/software/odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip",
"reports" : [ ],
"createTimestamp" : "August 30, 2022 09:27:17 AM CEST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "August 30, 2022 09:27:17 AM CEST"
}
[root@oda-prod software]#
Be sure that the job was fine
[root@oda-prod software]# odacli describe-job -i "17adc146-c10f-418a-b08b-f7d4c9390cbc"
Job details
----------------------------------------------------------------
ID: 17adc146-c10f-418a-b08b-f7d4c9390cbc
Description: Repository Update
Status: Success
Created: August 30, 2022 9:27:17 AM CEST
Message: /software/odacli-dcs-19.11.0.0.0-210530-GI-19.11.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle August 30, 2022 9:27:20 AM CEST August 30, 2022 9:27:58 AM CEST Success
registering image August 30, 2022 9:27:58 AM CEST August 30, 2022 9:27:58 AM CEST Success
[root@oda-prod software]#
Before applying the patch, we can run the prepatch checks
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli create-prepatchreport -s -v 19.11.0.0.0
Job details
----------------------------------------------------------------
ID: 09ff843d-1b4c-4a90-91ec-61004684903e
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER]
Status: Created
Created: August 30, 2022 9:30:58 AM CEST
Message: Use 'odacli describe-prepatchreport -i 09ff843d-1b4c-4a90-91ec-61004684903e' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
[root@oda-prod software]#
The results can be found below
[root@oda-prod software]# odacli describe-prepatchreport -i 09ff843d-1b4c-4a90-91ec-61004684903e
Patch pre-check report
------------------------------------------------------------------------
Job ID: 09ff843d-1b4c-4a90-91ec-61004684903e
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER]
Status: SUCCESS
Created: August 30, 2022 9:30:58 AM CEST
Result: All pre-checks succeeded
Node Name
---------------
oda-prod
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.11.0.0.0.
Is patch location available Success Patch location is available.
Verify OS patch Success Verified OS patch
Validate command execution Success Validated command execution
__ILOM__
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.11.0.0.0.
Is patch location available Success Patch location is available.
Checking Ilom patch Version Success Successfully verified the versions
Patch location validation Success Successfully validated location
Validate command execution Success Validated command execution
__GI__
Validate supported GI versions Success Validated minimum supported versions.
Validate available space Success Validated free space under /u01
Is clusterware running Success Clusterware is running
Validate patching tag Success Validated patching tag: 19.11.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate ASM in online Success ASM is online
Validate minimum agent version Success GI patching enabled in current
DCSAGENT version
Validate clones location exist Success Validated clones location
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
[root@oda-prod software]#
As everything is fine, we can apply the patch. We just ensure that we have at least 20G of free space for / /opt and /u01
[root@oda-prod software]# df -h / /opt /u01
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolRoot 30G 6.5G 22G 24% /
/dev/mapper/VolGroupSys-LogVolOpt 99G 33G 61G 36% /opt
/dev/mapper/VolGroupSys-LogVolU01 99G 65G 29G 70% /u01
[root@oda-prod software]#
And we apply the patch for the server
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-server -v 19.11.0.0.0
{
"jobId" : "5d3a7493-4426-4272-8ffc-59fad66a1eab",
"status" : "Created",
"message" : "Success of server update will trigger reboot of the node after 4-5 minutes. Please wait until the node reboots.",
"reports" : [ ],
"createTimestamp" : "August 30, 2022 09:37:06 AM CEST",
"resourceList" : [ ],
"description" : "Server Patching",
"updatedTime" : "August 30, 2022 09:37:06 AM CEST"
}
[root@oda-prod software]#
A few moments later we verify that the patch was fine
[root@oda-prod log]# odacli describe-job -i 5d3a7493-4426-4272-8ffc-59fad66a1eab
Job details
----------------------------------------------------------------
ID: 5d3a7493-4426-4272-8ffc-59fad66a1eab
Description: Server Patching
Status: Success
Created: August 30, 2022 9:37:06 AM CEST
Message: Successfully patched GI with RHP
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Patch location validation August 30, 2022 9:37:14 AM CEST August 30, 2022 9:37:14 AM CEST Success
dcs-controller upgrade August 30, 2022 9:37:17 AM CEST August 30, 2022 9:37:17 AM CEST Success
Creating repositories using yum August 30, 2022 9:37:17 AM CEST August 30, 2022 9:37:19 AM CEST Success
Updating YumPluginVersionLock rpm August 30, 2022 9:37:19 AM CEST August 30, 2022 9:37:19 AM CEST Success
Applying OS Patches August 30, 2022 9:37:19 AM CEST August 30, 2022 9:46:49 AM CEST Success
Creating repositories using yum August 30, 2022 9:46:49 AM CEST August 30, 2022 9:46:49 AM CEST Success
Applying HMP Patches August 30, 2022 9:46:49 AM CEST August 30, 2022 9:47:05 AM CEST Success
Patch location validation August 30, 2022 9:47:05 AM CEST August 30, 2022 9:47:05 AM CEST Success
oda-hw-mgmt upgrade August 30, 2022 9:47:05 AM CEST August 30, 2022 9:47:33 AM CEST Success
OSS Patching August 30, 2022 9:47:33 AM CEST August 30, 2022 9:47:33 AM CEST Success
Applying Firmware Disk Patches August 30, 2022 9:47:34 AM CEST August 30, 2022 9:47:40 AM CEST Success
Applying Firmware Controller Patches August 30, 2022 9:47:40 AM CEST August 30, 2022 9:47:44 AM CEST Success
Checking Ilom patch Version August 30, 2022 9:47:44 AM CEST August 30, 2022 9:47:45 AM CEST Success
Patch location validation August 30, 2022 9:47:45 AM CEST August 30, 2022 9:47:45 AM CEST Success
Save password in Wallet August 30, 2022 9:47:45 AM CEST August 30, 2022 9:47:45 AM CEST Success
Apply Ilom patch August 30, 2022 9:47:45 AM CEST August 30, 2022 9:56:18 AM CEST Success
Copying Flash Bios to Temp location August 30, 2022 9:56:18 AM CEST August 30, 2022 9:56:18 AM CEST Success
Patch location validation August 30, 2022 9:56:18 AM CEST August 30, 2022 9:56:18 AM CEST Success
ASR Manager RPM update August 30, 2022 9:56:19 AM CEST August 30, 2022 9:56:19 AM CEST Success
Modify JavaExec Path August 30, 2022 9:56:19 AM CEST August 30, 2022 9:56:19 AM CEST Success
Remove AsrConfBackup File August 30, 2022 9:56:19 AM CEST August 30, 2022 9:56:19 AM CEST Success
Starting the clusterware August 30, 2022 9:56:19 AM CEST August 30, 2022 9:58:23 AM CEST Success
create ASM priv network August 30, 2022 9:58:23 AM CEST August 30, 2022 9:58:27 AM CEST Success
configure ASM priv network August 30, 2022 9:58:27 AM CEST August 30, 2022 9:58:29 AM CEST Success
Starting the clusterware August 30, 2022 9:58:29 AM CEST August 30, 2022 9:58:29 AM CEST Success
registering image August 30, 2022 9:58:29 AM CEST August 30, 2022 9:58:29 AM CEST Success
registering working copy August 30, 2022 9:58:29 AM CEST August 30, 2022 9:58:30 AM CEST Success
registering image August 30, 2022 9:58:30 AM CEST August 30, 2022 9:58:30 AM CEST Success
Creating GI home directories August 30, 2022 9:58:30 AM CEST August 30, 2022 9:58:30 AM CEST Success
Extract Gi clone August 30, 2022 9:58:30 AM CEST August 30, 2022 9:59:55 AM CEST Success
Provisioning Software Only GI with RHP August 30, 2022 9:59:55 AM CEST August 30, 2022 10:00:56 AM CEST Success
Patch GI with RHP August 30, 2022 10:00:56 AM CEST August 30, 2022 10:09:13 AM CEST Success
Updating GiHome version August 30, 2022 10:09:13 AM CEST August 30, 2022 10:09:15 AM CEST Success
Starting the clusterware August 30, 2022 10:09:19 AM CEST August 30, 2022 10:09:19 AM CEST Success
remove network public interface August 30, 2022 10:09:19 AM CEST August 30, 2022 10:09:22 AM CEST Success
create bridge network August 30, 2022 10:09:22 AM CEST August 30, 2022 10:09:27 AM CEST Success
modify network public interface August 30, 2022 10:09:27 AM CEST August 30, 2022 10:09:28 AM CEST Success
Update System version August 30, 2022 10:09:28 AM CEST August 30, 2022 10:09:28 AM CEST Success
Cleanup JRE Home August 30, 2022 10:09:28 AM CEST August 30, 2022 10:09:28 AM CEST Success
Add SYSNAME in Env August 30, 2022 10:09:28 AM CEST August 30, 2022 10:09:28 AM CEST Success
Setting ACL for disk groups August 30, 2022 10:09:28 AM CEST August 30, 2022 10:09:31 AM CEST Success
preRebootNode Actions August 30, 2022 10:10:39 AM CEST August 30, 2022 10:11:30 AM CEST Success
Reboot Ilom August 30, 2022 10:11:30 AM CEST August 30, 2022 10:11:30 AM CEST Success
[root@oda-prod log]#
Below the component version. We can see that all components are up-to-date except the Oracle databases 19c. We will path the database later
[root@oda-prod ~]# odacli describe-component
System Version
---------------
19.11.0.0.0
System node Name
---------------
oda-prod
Local System Version
---------------
19.11.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.11.0.0.0 up-to-date
GI 19.11.0.0.210420 up-to-date
DB {
[ OraDB19000_home1 ] 19.9.0.0.201020 19.11.0.0.210420
[ OraDB11204_home1 ] 11.2.0.4.201020 not-available
}
DCSAGENT 19.11.0.0.0 up-to-date
OS 7.9 up-to-date
ILOM 5.0.1.21.a.r138015 up-to-date
BIOS 52030400 up-to-date
FIRMWARECONTROLLER VDV1RL04 up-to-date
FIRMWAREDISK 1102 up-to-date
ASR 20.3.0 up-to-date
HMP 2.4.8.0.600 up-to-date
[root@oda-prod ~]#
To patch the storage, we run the following command
[root@oda-prod ~]# /opt/oracle/dcs/bin/odacli update-storage -v 19.11.0.0.0
{
"jobId" : "eea8b12b-76d1-4695-9b29-f0102a8a67da",
"status" : "Created",
"message" : "Success of Storage Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "August 30, 2022 10:16:38 AM CEST",
"resourceList" : [ ],
"description" : "Storage Firmware Patching",
"updatedTime" : "August 30, 2022 10:16:38 AM CEST"
}
[root@oda-prod ~]#
The Job returns SUCCESS
[root@oda-prod ~]# odacli describe-job -i "eea8b12b-76d1-4695-9b29-f0102a8a67da"
Job details
----------------------------------------------------------------
ID: eea8b12b-76d1-4695-9b29-f0102a8a67da
Description: Storage Firmware Patching
Status: Success
Created: August 30, 2022 10:16:38 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Applying Firmware Disk Patches August 30, 2022 10:16:45 AM CEST August 30, 2022 10:16:49 AM CEST Success
preRebootNode Actions August 30, 2022 10:16:50 AM CEST August 30, 2022 10:16:50 AM CEST Success
Reboot Ilom August 30, 2022 10:16:50 AM CEST August 30, 2022 10:16:50 AM CEST Success
[root@oda-prod ~]#
As the patch went fine, we can remove the odabr snapshots now
[root@oda-prod software]# /opt/odabr/odabr delsnap
INFO: 2022-08-30 10:22:10: Please check the logfile '/opt/odabr/out/log/odabr_64634.log' for more details
INFO: 2022-08-30 10:22:10: Removing LVM snapshots
INFO: 2022-08-30 10:22:10: ...removing LVM snapshot for 'opt'
SUCCESS: 2022-08-30 10:22:11: ...snapshot for 'opt' removed successfully
INFO: 2022-08-30 10:22:11: ...removing LVM snapshot for 'u01'
SUCCESS: 2022-08-30 10:22:11: ...snapshot for 'u01' removed successfully
INFO: 2022-08-30 10:22:11: ...removing LVM snapshot for 'root'
SUCCESS: 2022-08-30 10:22:11: ...snapshot for 'root' removed successfully
SUCCESS: 2022-08-30 10:22:11: Remove LVM snapshots done successfully
[root@oda-prod software]#
Patching existing databases
In my case only 19c databases are updated. Oracle 11g is not supported for the update.
The first step is to unpack the corresponding file
[root@oda-prod software]# unzip p30403662_1911000_Linux-x86-64.zip
Archive: p30403662_1911000_Linux-x86-64.zip
extracting: odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip
inflating: README.txt
[root@oda-prod software]#
[root@oda-prod software]# ls -l odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip
-rwxrwxrwx 1 root root 4953108233 May 31 2021 odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip
[root@oda-prod software]#
We load the DB 19c clone on the repository
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli update-repository -f /software/odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip
{
"jobId" : "87445cac-e643-474c-87ec-93b7811caa17",
"status" : "Created",
"message" : "/software/odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip",
"reports" : [ ],
"createTimestamp" : "August 30, 2022 10:47:07 AM CEST",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "August 30, 2022 10:47:07 AM CEST"
}
[root@oda-prod software]#
We validate the status of the job
[root@oda-prod software]# odacli describe-job -i "87445cac-e643-474c-87ec-93b7811caa17"
Job details
----------------------------------------------------------------
ID: 87445cac-e643-474c-87ec-93b7811caa17
Description: Repository Update
Status: Success
Created: August 30, 2022 10:47:07 AM CEST
Message: /software/odacli-dcs-19.11.0.0.0-210530-DB-19.11.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Unzip bundle August 30, 2022 10:47:09 AM CEST August 30, 2022 10:47:44 AM CEST Success
registering image August 30, 2022 10:47:44 AM CEST August 30, 2022 10:47:45 AM CEST Success
[root@oda-prod software]#
In the documentation we can fine following lines
Starting with Oracle Database Appliance release 19.11, Oracle Database homes are not created on the local disk. Oracle Database homes are created on an Oracle ACFS-managed file system. For a database user oracle, the new database homes are created under /u01/app/odaorahome/oracle/.
Before patching the database home, configure the database home storage with the odacli configure-dbhome-storage command:
Then we create the storage for the Dbhome
[root@oda-prod software]# odacli configure-dbhome-storage -dg DATA
{
"jobId" : "c82c5d33-fe67-4092-86d8-5746db2ac447",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "August 30, 2022 10:50:07 AM CEST",
"resourceList" : [ ],
"description" : "Configure database home storage",
"updatedTime" : "August 30, 2022 10:50:07 AM CEST"
}
You have new mail in /var/spool/mail/root
[root@oda-prod software]#
The status was fine
[root@oda-prod software]# odacli describe-job -i "c82c5d33-fe67-4092-86d8-5746db2ac447"
Job details
----------------------------------------------------------------
ID: c82c5d33-fe67-4092-86d8-5746db2ac447
Description: Configure database home storage
Status: Success
Created: August 30, 2022 10:50:07 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Configuring database home storage August 30, 2022 10:50:07 AM CEST August 30, 2022 10:50:07 AM CEST Success
[root@oda-prod software]#
run the prepatch
[root@oda-prod software]# odacli list-dbhomes
ID Name DB Version Home Location Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
2a82f095-5952-42fa-b22c-a81ca6da5f30 OraDB19000_home1 19.9.0.0.201020 /u01/app/oracle/product/19.0.0.0/dbhome_1 CONFIGURED
100c93b2-0f50-43e6-a04f-bb4a78c41aeb OraDB11204_home1 11.2.0.4.201020 /u01/app/oracle/product/11.2.0.4/dbhome_1 CONFIGURED
[root@oda-prod software]#
Before patching the HOMES, we do a precheck.
[root@oda-prod software]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 2a82f095-5952-42fa-b22c-a81ca6da5f30 -v 19.11.0.0.0
Job details
----------------------------------------------------------------
ID: 8ac2998e-ce0e-47e5-b9c3-6e5fd0c49139
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home1
Status: Created
Created: August 30, 2022 10:52:56 AM CEST
Message: Use 'odacli describe-prepatchreport -i 8ac2998e-ce0e-47e5-b9c3-6e5fd0c49139' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
[root@oda-prod software]#
Below the results
[root@oda-prod software]# odacli describe-prepatchreport -i 8ac2998e-ce0e-47e5-b9c3-6e5fd0c49139
Patch pre-check report
------------------------------------------------------------------------
Job ID: 8ac2998e-ce0e-47e5-b9c3-6e5fd0c49139
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home1
Status: FAILED
Created: August 30, 2022 10:52:56 AM CEST
Result: One or more pre-checks failed for [ORACHK]
Node Name
---------------
oda-prod
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID Success Validated DB Home ID:
2a82f095-5952-42fa-b22c-a81ca6da5f30
Validate patching tag Success Validated patching tag: 19.11.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate minimum agent version Success Validated minimum agent version
Is GI upgraded Success Validated GI is upgraded
Validate available space for Success Validated free space required under
db /u01
Is DB clone registered Success Successfully validated clone
registered
Is DB clone available Success Successfully validated clone file
exists
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Failed Orachk validation failed: .
Validate command execution Success Validated command execution
Verify the Alternate Archive Failed AHF-4940: One or more log archive
Destination is Configured to destination and alternate log archive
Prevent Database Hangs destination settings are not as
recommended
Verify the Alternate Archive Failed AHF-4940: One or more log archive
Destination is Configured to destination and alternate log archive
Prevent Database Hangs destination settings are not as
recommended
Verify the Alternate Archive Failed AHF-4940: One or more log archive
Destination is Configured to destination and alternate log archive
Prevent Database Hangs destination settings are not as
recommended
Verify the Alternate Archive Failed AHF-4940: One or more log archive
Destination is Configured to destination and alternate log archive
Prevent Database Hangs destination settings are not as
recommended
Check for parameter Failed AHF-3744: Database parameter
global_names GLOBAL_NAMES is not set to
recommended value
Check for parameter Failed AHF-3744: Database parameter
global_names GLOBAL_NAMES is not set to
recommended value
Check for parameter Failed AHF-3744: Database parameter
global_names GLOBAL_NAMES is not set to
recommended value
Verify the Fast Recovery Area Failed AHF-2929: FRA space management
(FRA) has reclaimable space problem file types are present
without an RMAN backup completion
within the last 7 days
[root@oda-prod software]#
We got some oracheck errors. After a look on Oracle documentation of the known Issues we found following lines
AHF error in prepatch report for the update-dbhome command
When you patch server to Oracle Database Appliance release 19.13, the odacli update-dbhome command may fail.
The following error message is displayed in the pre-patch report:
Verify the Alternate Archive Failed AHF-4940: One or more log archive
Destination is Configured to destination and alternate log archive
Prevent Database Hangs destination settings are not as recommended
Hardware Models
All Oracle Database Appliance hardware models
Workaround
Follow these steps:
Run the odacli update-dbhome command with the -f option.
/opt/oracle/dcs/bin/odacli update-dbhome –dbhomeid 7c67c5b4-f585-4ba9-865f-c719c63c0a6e -v 19.13.0.0.0 -f
This issue is tracked with Oracle bug 33144170.
For the error relative to the global_name, we change the value to true and will set it back to false after the patch
And then we run the patch with the option -f
[root@oda-prod archivelog]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 2a82f095-5952-42fa-b22c-a81ca6da5f30 -v 19.11.0.0.0 -f
{
"jobId" : "4b499623-9ca9-4089-b18b-5c13af1d9067",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "August 30, 2022 11:15:58 AM CEST",
"resourceList" : [ ],
"description" : "DB Home Patching: Home Id is 2a82f095-5952-42fa-b22c-a81ca6da5f30",
"updatedTime" : "August 30, 2022 11:15:58 AM CEST"
}
[root@oda-prod archivelog]#
A few moments after the patch was fine
[root@oda-prod ~]# odacli describe-job -i "4b499623-9ca9-4089-b18b-5c13af1d9067"
Job details
----------------------------------------------------------------
ID: 4b499623-9ca9-4089-b18b-5c13af1d9067
Description: DB Home Patching: Home Id is 2a82f095-5952-42fa-b22c-a81ca6da5f30
Status: Success
Created: August 30, 2022 11:15:58 AM CEST
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Creating ACFS database home August 30, 2022 11:16:11 AM CEST August 30, 2022 11:17:01 AM CEST Success
Validating dbHome available space August 30, 2022 11:17:01 AM CEST August 30, 2022 11:17:01 AM CEST Success
Creating DbHome Directory August 30, 2022 11:17:01 AM CEST August 30, 2022 11:17:01 AM CEST Success
Create required directories August 30, 2022 11:17:01 AM CEST August 30, 2022 11:17:01 AM CEST Success
Extract DB clones August 30, 2022 11:17:01 AM CEST August 30, 2022 11:18:09 AM CEST Success
ProvDbHome by using RHP August 30, 2022 11:18:09 AM CEST August 30, 2022 11:19:11 AM CEST Success
Enable DB options August 30, 2022 11:19:11 AM CEST August 30, 2022 11:19:26 AM CEST Success
Creating wallet for DB Client August 30, 2022 11:19:29 AM CEST August 30, 2022 11:19:29 AM CEST Success
Creating wallet for DB Client August 30, 2022 11:19:53 AM CEST August 30, 2022 11:19:53 AM CEST Success
Patch databases by RHP August 30, 2022 11:19:53 AM CEST August 30, 2022 11:24:11 AM CEST Success
updating database metadata August 30, 2022 11:24:54 AM CEST August 30, 2022 11:24:54 AM CEST Success
Patch databases by RHP August 30, 2022 11:24:54 AM CEST August 30, 2022 11:29:01 AM CEST Success
updating database metadata August 30, 2022 11:29:41 AM CEST August 30, 2022 11:29:41 AM CEST Success
Update System version August 30, 2022 11:29:41 AM CEST August 30, 2022 11:29:41 AM CEST Success
Updating metadata for database August 30, 2022 11:30:09 AM CEST August 30, 2022 11:30:09 AM CEST Success
Updating metadata for database August 30, 2022 11:30:09 AM CEST August 30, 2022 11:30:09 AM CEST Success
[root@oda-prod ~]#
Connecting to the 19c database we can verify that the RU is now 19.11
SQL> select description from dba_registry_sqlpatch;
DESCRIPTION
--------------------------------------------------------------------------------
OJVM RELEASE UPDATE: 19.9.0.0.201020 (31668882)
Database Release Update : 19.9.0.0.201020 (31771877)
OJVM RELEASE UPDATE: 19.9.0.0.201020 (31668882)
OJVM RELEASE UPDATE: 19.11.0.0.210420 (32399816)
Database Release Update : 19.11.0.0.210420 (32545013)
SQL>
Conclusion
Hope this blog help
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/12/oracle-square.png)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/DWE_web-min-scaled.jpg)