By Mouhamadou Diaw
In this blog I am describing an ODA HA (serveroda1 and serveroda2) patch from 19.10 to 19.17. The process is the same that patching a non-HA ODA. Few tasks may need to be done on both servers, otherwise all steps are done from one node. If there are some tasks needed on both nodes, we will specify.
My colleague Jerome already do a nice blog for non-HA ODA.
In the documentation we can find possible versions that can upgraded to version 19.17.
As we are actually on 19.10, we decide to do following path
19.10 ==>19.14 ==>19.17
Required patches
Download following patches and transfer them to one node of the ODA HA
For 19.14
patch 33702951 Oracle Database Appliance Server Patch for the ODACLI/DCS stack
patch 30403673 Oracle Grid Infrastructure clone files
patch 30403662 Oracle Database clone files
For 19.17
patch 34753059 Oracle Database Appliance Server Patch
patch 30403673 Oracle Grid Infrastructure clone files
patch 30403662 Oracle Database clone files
Below our source configuration
[root@serveroda1 bin]# odacli describe-component
System Version
---------------
19.10.0.0.0
System node Name
---------------
serveroda1
Local System Version
---------------
19.10.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.10.0.0.0 up-to-date
GI 19.10.0.0.210119 up-to-date
DB 19.10.0.0.210119 up-to-date
DCSAGENT 19.10.0.0.0 up-to-date
OS 7.9 up-to-date
ILOM 5.0.1.21.a.r138015 up-to-date
BIOS 41080800 up-to-date
FIRMWARECONTROLLER 16.00.08.00 up-to-date
FIRMWAREEXPANDER 0310 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] 0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, A170 up-to-date
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ]
}
HMP 2.4.7.0.1 up-to-date
System node Name
---------------
serveroda2
Local System Version
---------------
19.10.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.10.0.0.0 up-to-date
GI 19.10.0.0.210119 up-to-date
DB 19.10.0.0.210119 up-to-date
DCSAGENT 19.10.0.0.0 up-to-date
OS 7.9 up-to-date
ILOM 5.0.1.21.a.r138015 up-to-date
BIOS 41080800 up-to-date
FIRMWARECONTROLLER 16.00.08.00 up-to-date
FIRMWAREEXPANDER 0310 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] 0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, A170 up-to-date
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ]
}
HMP 2.4.7.0.1 up-to-date
1- Patch from 19.10 to 19.14
Update the repository with the server patch
Unpack the zip files and use following command to update the repository
[root@serveroda1 bin]# odacli update-repository -f /u03/app/oracle/soft/19.14/oda-sm-19.14.0.0.0-220215-server.zip
{
"jobId" : "f5308fcf-4bf9-42dd-8742-8ade96105968",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.14/oda-sm-19.14.0.0.0-220215-server.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 08:48:14 AM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 08:48:14 AM CET"
}
Validate that the job was fine
[root@serveroda1 bin]# odacli describe-job -i "f5308fcf-4bf9-42dd-8742-8ade96105968"At this moment we can again check the status of the components
[root@serveroda1 bin]# odacli describe-component
System Version
---------------
19.10.0.0.0
System node Name
---------------
serveroda1
Local System Version
---------------
19.10.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.10.0.0.0 19.14.0.0.0
GI 19.10.0.0.210119 19.14.0.0.220118
DB 19.10.0.0.210119 19.14.0.0.220118
DCSAGENT 19.10.0.0.0 19.14.0.0.0
OS 7.9 up-to-date
ILOM 5.0.1.21.a.r138015 5.0.2.24.r141466
BIOS 41080800 41100500
FIRMWARECONTROLLER 16.00.08.00 up-to-date
FIRMWAREEXPANDER 0310 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] 0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, A170 up-to-date
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ]
}
HMP 2.4.7.0.1 2.4.8.0.600
System node Name
---------------
serveroda2
Local System Version
---------------
19.10.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK 19.10.0.0.0 19.14.0.0.0
GI 19.10.0.0.210119 19.14.0.0.220118
DB 19.10.0.0.210119 19.14.0.0.220118
DCSAGENT 19.10.0.0.0 19.14.0.0.0
OS 7.9 up-to-date
ILOM 5.0.1.21.a.r138015 5.0.2.24.r141466
BIOS 41080800 41100500
FIRMWARECONTROLLER 16.00.08.00 up-to-date
FIRMWAREEXPANDER 0310 up-to-date
FIRMWAREDISK {
[ c0d0,c0d1 ] 0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6, A170 up-to-date
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ]
}
HMP 2.4.7.0.1 2.4.8.0.600
Update the dcsadmin
[root@serveroda1 bin]# odacli update-dcsadmin -v 19.14.0.0.0
{
"jobId" : "e36e0cba-f853-4c84-adeb-67660236a858",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "January 03, 2023 08:51:10 AM CET",
"resourceList" : [ ],
"description" : "DcsAdmin patching",
"updatedTime" : "January 03, 2023 08:51:10 AM CET"
}
Validate the job status
root@serveroda1 bin]# odacli describe-job -i "e36e0cba-f853-4c84-adeb-67660236a858"
Job details
----------------------------------------------------------------
ID: e36e0cba-f853-4c84-adeb-67660236a858
Description: DcsAdmin patching
Status: Success
Created: January 3, 2023 8:51:10 AM CET
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Patch location validation January 3, 2023 8:51:11 AM CET January 3, 2023 8:51:11 AM CET Success
Patch location validation January 3, 2023 8:51:11 AM CET January 3, 2023 8:51:11 AM CET Success
dcs-admin upgrade January 3, 2023 8:51:11 AM CET January 3, 2023 8:51:17 AM CET Success
dcs-admin upgrade January 3, 2023 8:51:17 AM CET January 3, 2023 8:51:24 AM CET Success
Update the dcscomponents
[root@serveroda1 bin]# odacli update-dcscomponents -v 19.14.0.0.0
{
"jobId" : "b057eaa3-eaaf-49a7-b662-6c53eb95b3c0",
"status" : "Success",
"message" : "Update-dcscomponents is successful on all the node(s):DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata migration is done before. Metadata schema update is done. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. Successfully ran setupAgentAuth.sh zookeeper RPM upgrade is successful. ",
"reports" : null,
"createTimestamp" : "January 03, 2023 08:52:31 AM CET",
"description" : "Update-dcscomponents job completed and is not part of Agent job list",
"updatedTime" : "January 03, 2023 08:56:11 AM CET"
}
It’s normal that the describe-job for this job does not return any value. You just should have SUCCESS on the status for the command above
[root@serveroda1 bin]# odacli describe-job -i "b057eaa3-eaaf-49a7-b662-6c53eb95b3c0"
DCS-10000:Resource Job with ID b057eaa3-eaaf-49a7-b662-6c53eb95b3c0 is not found.
Update the dcsagent
[root@serveroda1 bin]# odacli update-dcsagent -v 19.14.0.0.0
{
"jobId" : "e1511288-b202-42c5-bbe8-5c13f4b1cfa9",
"status" : "Created",
"message" : "Dcs agent will be restarted after the update. Please wait for 2-3 mins before executing the other commands",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 08:57:54 AM CET",
"resourceList" : [ ],
"description" : "DcsAgent patching",
"updatedTime" : "January 03, 2023 08:57:54 AM CET"
}
The following command should return SUCCESS
[root@serveroda1 bin]# odacli describe-job -i "e1511288-b202-42c5-bbe8-5c13f4b1cfa9"Update the repository with the 19.14 GI and DB clones
GI clone
[root@serveroda1 bin]# odacli update-repository -f /u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-GI-19.14.0.0.zip
{
"jobId" : "b05f15ed-2513-47d3-864f-fae03394c0a5",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-GI-19.14.0.0.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 09:02:54 AM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 09:02:54 AM CET"
}
Validate the status of the job
[root@serveroda1 bin]# odacli describe-job -i "b05f15ed-2513-47d3-864f-fae03394c0a5"
Job details
----------------------------------------------------------------
ID: b05f15ed-2513-47d3-864f-fae03394c0a5
Description: Repository Update
Status: Success
Created: January 3, 2023 9:02:54 AM CET
Message: /u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-GI-19.14.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace January 3, 2023 9:03:06 AM CET January 3, 2023 9:03:06 AM CET Success
Setting up ssh equivalance January 3, 2023 9:03:06 AM CET January 3, 2023 9:03:06 AM CET Success
Copy BundleFile January 3, 2023 9:03:07 AM CET January 3, 2023 9:03:47 AM CET Success
Validating CopiedFile January 3, 2023 9:03:47 AM CET January 3, 2023 9:03:58 AM CET Success
Unzip bundle January 3, 2023 9:03:58 AM CET January 3, 2023 9:05:35 AM CET Success
Unzip bundle January 3, 2023 9:05:35 AM CET January 3, 2023 9:07:04 AM CET Success
Removing ssh keys January 3, 2023 9:07:04 AM CET January 3, 2023 9:07:05 AM CET Success
DB clone
[root@serveroda1 bin]# odacli update-repository -f /u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-DB-19.14.0.0.zip
{
"jobId" : "20b39c38-bf4f-4815-9223-fffa8ee06b4f",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-DB-19.14.0.0.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 09:08:42 AM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 09:08:42 AM CET"
}
Validate the job
[root@serveroda1 bin]# odacli describe-job -i "20b39c38-bf4f-4815-9223-fffa8ee06b4f"
Job details
----------------------------------------------------------------
ID: 20b39c38-bf4f-4815-9223-fffa8ee06b4f
Description: Repository Update
Status: Success
Created: January 3, 2023 9:08:42 AM CET
Message: /u03/app/oracle/soft/19.14/odacli-dcs-19.14.0.0.0-220215-DB-19.14.0.0.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace January 3, 2023 9:08:51 AM CET January 3, 2023 9:08:51 AM CET Success
Setting up ssh equivalance January 3, 2023 9:08:51 AM CET January 3, 2023 9:08:51 AM CET Success
Copy BundleFile January 3, 2023 9:08:52 AM CET January 3, 2023 9:09:21 AM CET Success
Validating CopiedFile January 3, 2023 9:09:22 AM CET January 3, 2023 9:09:31 AM CET Success
Unzip bundle January 3, 2023 9:09:31 AM CET January 3, 2023 9:10:37 AM CET Success
Unzip bundle January 3, 2023 9:10:37 AM CET January 3, 2023 9:11:43 AM CET Success
Removing ssh keys January 3, 2023 9:11:43 AM CET January 3, 2023 9:11:44 AM CET Success
Generate a prepatch report
[root@serveroda1 bin]# odacli create-prepatchreport -s -v 19.14.0.0.0
Job details
----------------------------------------------------------------
ID: d05aa323-7ce8-4393-a646-9e6052fc2e12
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER]
Status: Created
Created: January 3, 2023 9:20:33 AM CET
Message: Use 'odacli describe-prepatchreport -i d05aa323-7ce8-4393-a646-9e6052fc2e12' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check If there is any error. Any failed step should be corrected
[root@serveroda1 bin]# odacli describe-prepatchreport -i d05aa323-7ce8-4393-a646-9e6052fc2e12
Patch pre-check report
------------------------------------------------------------------------
Job ID: d05aa323-7ce8-4393-a646-9e6052fc2e12
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER]
Status: SUCCESS
Created: January 3, 2023 9:20:33 AM CET
Result: All pre-checks succeeded
Node Name
---------------
serveroda1
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is patch location available Success Patch location is available.
Verify OS patch Success Verified OS patch
Validate command execution Success Validated command execution
__ILOM__
Validate ILOM server reachable Success Successfully connected with ILOM
server using public IP and USB
interconnect
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is patch location available Success Patch location is available.
Checking Ilom patch Version Success Successfully verified the versions
Patch location validation Success Successfully validated location
Validate command execution Success Validated command execution
__GI__
Validate GI metadata Success Successfully validated GI metadata
Validate supported GI versions Success Validated minimum supported versions.
Validate available space Success Validated free space under /u01
Is clusterware running Success Clusterware is running
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate ASM in online Success ASM is online
Validate kernel log level Success Successfully validated the OS log
level
Validate minimum agent version Success GI patching enabled in current
DCSAGENT version
Validate Central Inventory Success oraInventory validation passed
Validate patching locks Success Validated patching locks
Validate clones location exist Success Validated clones location
Validate DB start dependencies Success DBs START dependency check passed
Validate DB stop dependencies Success DBs STOP dependency check passed
Evaluate GI patching Success Successfully validated GI patching
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
Node Name
---------------
serveroda2
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__OS__
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is patch location available Success Patch location is available.
Verify OS patch Success Verified OS patch
Validate command execution Success Validated command execution
__ILOM__
Validate ILOM server reachable Success Successfully connected with ILOM
server using public IP and USB
interconnect
Validate supported versions Success Validated minimum supported versions.
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is patch location available Success Patch location is available.
Checking Ilom patch Version Success Successfully verified the versions
Patch location validation Success Successfully validated location
Validate command execution Success Validated command execution
__GI__
Validate GI metadata Success Successfully validated GI metadata
Validate supported GI versions Success Validated minimum supported versions.
Validate available space Success Validated free space under /u01
Is clusterware running Success Clusterware is running
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate ASM in online Success ASM is online
Validate kernel log level Success Successfully validated the OS log
level
Validate minimum agent version Success GI patching enabled in current
DCSAGENT version
Validate Central Inventory Success oraInventory validation passed
Validate patching locks Success Validated patching locks
Validate clones location exist Success Validated clones location
Validate DB start dependencies Success DBs START dependency check passed
Validate DB stop dependencies Success DBs STOP dependency check passed
Evaluate GI patching Success Successfully validated GI patching
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
Update the server now
As there is no error, we can now launch the server patching
[root@serveroda1 bin]# /opt/oracle/dcs/bin/odacli update-server -v 19.14.0.0.0
{
"jobId" : "aa4ffe45-eae2-4eca-8cd0-f4c0b9ae70f8",
"status" : "Created",
"message" : "Success of server update will trigger reboot of the node after 4-5 minutes. Please wait until the node reboots.",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 09:38:09 AM CET",
"resourceList" : [ ],
"description" : "Server Patching",
"updatedTime" : "January 03, 2023 09:38:09 AM CET"
}
And the end the result should be SUCCESS
[root@serveroda1 bin]# odacli describe-job -i "aa4ffe45-eae2-4eca-8cd0-f4c0b9ae70f8"
Job details
----------------------------------------------------------------
ID: aa4ffe45-eae2-4eca-8cd0-f4c0b9ae70f8
Description: Server Patching
Status: Success
Created: January 3, 2023 9:38:09 AM CET
Message: Successfully patched GI with RHP
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validating GI user metadata January 3, 2023 9:38:30 AM CET January 3, 2023 9:38:30 AM CET Success
Validating GI user metadata January 3, 2023 9:38:30 AM CET January 3, 2023 9:38:30 AM CET Success
Validate ILOM server reachable January 3, 2023 9:38:30 AM CET January 3, 2023 9:38:30 AM CET Success
Validate ILOM server reachable January 3, 2023 9:38:30 AM CET January 3, 2023 9:38:30 AM CET Success
Configure export clones resource January 3, 2023 9:40:48 AM CET January 3, 2023 9:40:50 AM CET Success
Creating repositories using yum January 3, 2023 9:40:56 AM CET January 3, 2023 9:41:02 AM CET Success
Updating YumPluginVersionLock rpm January 3, 2023 9:41:02 AM CET January 3, 2023 9:41:02 AM CET Success
Applying OS Patches January 3, 2023 9:41:02 AM CET January 3, 2023 9:46:37 AM CET Success
Creating repositories using yum January 3, 2023 9:46:54 AM CET January 3, 2023 9:46:54 AM CET Success
Applying HMP Patches January 3, 2023 9:46:54 AM CET January 3, 2023 9:47:10 AM CET Success
Patch location validation January 3, 2023 9:47:12 AM CET January 3, 2023 9:47:12 AM CET Success
Patch location validation January 3, 2023 9:47:12 AM CET January 3, 2023 9:47:12 AM CET Success
oda-hw-mgmt upgrade January 3, 2023 9:47:12 AM CET January 3, 2023 9:47:45 AM CET Success
oda-hw-mgmt upgrade January 3, 2023 9:47:45 AM CET January 3, 2023 9:48:16 AM CET Success
OSS Patching January 3, 2023 9:48:16 AM CET January 3, 2023 9:48:17 AM CET Success
Applying Firmware Disk Patches January 3, 2023 9:48:21 AM CET January 3, 2023 9:48:24 AM CET Success
Applying Firmware Controller Patches January 3, 2023 9:48:28 AM CET January 3, 2023 9:48:32 AM CET Success
Checking Ilom patch Version January 3, 2023 9:48:32 AM CET January 3, 2023 9:48:32 AM CET Success
Checking Ilom patch Version January 3, 2023 9:48:32 AM CET January 3, 2023 9:48:32 AM CET Success
Patch location validation January 3, 2023 9:48:33 AM CET January 3, 2023 9:48:33 AM CET Success
Patch location validation January 3, 2023 9:48:33 AM CET January 3, 2023 9:48:33 AM CET Success
Save password in Wallet January 3, 2023 9:48:33 AM CET January 3, 2023 9:48:34 AM CET Success
Disabling IPMI v2 January 3, 2023 9:48:36 AM CET January 3, 2023 9:48:37 AM CET Success
Apply Ilom patch January 3, 2023 9:48:37 AM CET January 3, 2023 9:57:18 AM CET Success
Apply Ilom patch January 3, 2023 9:57:18 AM CET January 3, 2023 10:05:31 AM CET Success
Copying Flash Bios to Temp location January 3, 2023 10:05:31 AM CET January 3, 2023 10:05:31 AM CET Success
Copying Flash Bios to Temp location January 3, 2023 10:05:31 AM CET January 3, 2023 10:05:31 AM CET Success
Adding USER SSH_EQUIVALENCE January 3, 2023 10:05:32 AM CET January 3, 2023 10:05:33 AM CET Success
Adding USER SSH_EQUIVALENCE January 3, 2023 10:05:33 AM CET January 3, 2023 10:05:34 AM CET Success
Adding USER SSH_EQUIVALENCE January 3, 2023 10:05:34 AM CET January 3, 2023 10:05:35 AM CET Success
Starting the clusterware January 3, 2023 10:07:56 AM CET January 3, 2023 10:10:10 AM CET Success
registering image January 3, 2023 10:10:12 AM CET January 3, 2023 10:10:13 AM CET Success
registering working copy January 3, 2023 10:10:14 AM CET January 3, 2023 10:10:14 AM CET Success
registering image January 3, 2023 10:10:14 AM CET January 3, 2023 10:10:16 AM CET Success
Creating GI home directories January 3, 2023 10:10:17 AM CET January 3, 2023 10:10:20 AM CET Success
Extract GI clone January 3, 2023 10:10:21 AM CET January 3, 2023 10:10:21 AM CET Success
Extract GI clone January 3, 2023 10:10:21 AM CET January 3, 2023 10:10:21 AM CET Success
Provisioning Software Only GI with RHP January 3, 2023 10:10:22 AM CET January 3, 2023 10:10:22 AM CET Success
Patch GI with RHP January 3, 2023 10:10:22 AM CET January 3, 2023 10:26:57 AM CET Success
Updating GIHome version January 3, 2023 10:26:59 AM CET January 3, 2023 10:27:06 AM CET Success
Updating GIHome version January 3, 2023 10:26:59 AM CET January 3, 2023 10:27:03 AM CET Success
Validate GI availability January 3, 2023 10:27:14 AM CET January 3, 2023 10:27:14 AM CET Success
Patch KVM CRS type January 3, 2023 10:27:14 AM CET January 3, 2023 10:27:14 AM CET Success
Update System version January 3, 2023 10:27:14 AM CET January 3, 2023 10:27:14 AM CET Success
Update System version January 3, 2023 10:27:14 AM CET January 3, 2023 10:27:14 AM CET Success
Cleanup JRE Home January 3, 2023 10:27:14 AM CET January 3, 2023 10:27:15 AM CET Success
Add SYSNAME in Env January 3, 2023 10:27:15 AM CET January 3, 2023 10:27:15 AM CET Success
Starting the clusterware January 3, 2023 10:27:15 AM CET January 3, 2023 10:27:15 AM CET Success
Setting ACL for disk groups January 3, 2023 10:27:15 AM CET January 3, 2023 10:27:22 AM CET Success
Update previous workarounds January 3, 2023 10:29:12 AM CET January 3, 2023 10:29:12 AM CET Success
Update previous workarounds January 3, 2023 10:29:12 AM CET January 3, 2023 10:29:12 AM CET Success
preRebootNode Actions January 3, 2023 10:29:12 AM CET January 3, 2023 10:30:16 AM CET Success
preRebootNode Actions January 3, 2023 10:30:16 AM CET January 3, 2023 10:31:09 AM CET Success
Reboot Ilom January 3, 2023 10:31:10 AM CET January 3, 2023 10:31:10 AM CET Success
Reboot Ilom January 3, 2023 10:31:10 AM CET January 3, 2023 10:31:10 AM CET Success
Validate the new components
[root@serveroda1 bin]# odacli describe-component
System Version
---------------
19.14.0.0.0
System node Name
---------------
serveroda1
Local System Version
---------------
19.14.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK
19.14.0.0.0 up-to-date
GI
19.14.0.0.220118 up-to-date
DB
19.10.0.0.210119 19.14.0.0.220118
DCSCONTROLLER
19.14.0.0.0 up-to-date
DCSCLI
19.14.0.0.0 up-to-date
DCSAGENT
19.14.0.0.0 up-to-date
DCSADMIN
19.14.0.0.0 up-to-date
OS
7.9 up-to-date
ILOM
5.0.2.24.r141466 up-to-date
BIOS
41100500 up-to-date
SHARED CONTROLLER FIRMWARE
16.00.08.00 up-to-date
FIRMWAREEXPANDER
0310 up-to-date
LOCAL DISK FIRMWARE
0112 up-to-date
SHARED DISK FIRMWARE {
[ c0d0,c0d1 ]
0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6,
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ] A170 up-to-date
}
HMP
2.4.8.0.600 up-to-date
System node Name
---------------
serveroda2
Local System Version
---------------
19.14.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK
19.14.0.0.0 up-to-date
GI
19.14.0.0.220118 up-to-date
DB
19.10.0.0.210119 19.14.0.0.220118
DCSCONTROLLER
19.14.0.0.0 up-to-date
DCSCLI
19.14.0.0.0 up-to-date
DCSAGENT
19.14.0.0.0 up-to-date
DCSADMIN
19.14.0.0.0 up-to-date
OS
7.9 up-to-date
ILOM
5.0.2.24.r141466 up-to-date
BIOS
41100500 up-to-date
SHARED CONTROLLER FIRMWARE
16.00.08.00 up-to-date
FIRMWAREEXPANDER
0310 up-to-date
LOCAL DISK FIRMWARE
0112 up-to-date
SHARED DISK FIRMWARE {
[ c0d0,c0d1 ]
0112 up-to-date
[ c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6,
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8 ] A170 up-to-date
}
HMP
2.4.8.0.600 up-to-date
Update the storage components
Run a prepatch report for storage
root@serveroda1 bin]# /opt/oracle/dcs/bin/odacli create-prepatchreport -st -v 19.14.0.0.0
Job details
----------------------------------------------------------------
ID: 3197445b-00a6-44a7-92c4-eded9dd0ed8a
Description: Patch pre-checks for [STORAGE]
Status: Created
Created: January 3, 2023 10:51:44 AM CET
Message: Use 'odacli describe-prepatchreport -i 3197445b-00a6-44a7-92c4-eded9dd0ed8a' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- -----------------------------------
Validate that there is no error
[root@serveroda1 bin]# odacli describe-prepatchreport -i 3197445b-00a6-44a7-92c4-eded9dd0ed8a
Patch pre-check report
------------------------------------------------------------------------
Job ID: 3197445b-00a6-44a7-92c4-eded9dd0ed8a
Description: Patch pre-checks for [STORAGE]
Status: SUCCESS
Created: January 3, 2023 10:51:44 AM CET
Result: All pre-checks succeeded
Node Name
---------------
serveroda1
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__STORAGE__
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Patch location validation Success Verified patch location
Patch tag validation Success Verified patch tag
Storage patch tag validation Success Verified storage patch location
Verify ASM disks status Success ASM disks are online
Validate rolling patch Success Rolling mode patching allowed as
there is no expander and controller
upgrade.
Validate command execution Success Validated command execution
Node Name
---------------
serveroda2
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__STORAGE__
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Patch location validation Success Verified patch location
Patch tag validation Success Verified patch tag
Storage patch tag validation Success Verified storage patch location
Verify ASM disks status Success ASM disks are online
Validate rolling patch Success Rolling mode patching allowed as
there is no expander and controller
upgrade.
Validate command execution Success Validated command execution
Now update the storage
[root@serveroda1 bin]# odacli update-storage -v 19.14.0.0.0 --rolling
{
"jobId" : "860f95b3-f1dd-402a-897d-ebf0ca9b4584",
"status" : "Created",
"message" : "Success of Storage Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 10:57:20 AM CET",
"resourceList" : [ ],
"description" : "Storage Firmware Patching",
"updatedTime" : "January 03, 2023 10:57:20 AM CET"
Validate the job status
[root@serveroda1 bin]# odacli describe-job -i 860f95b3-f1dd-402a-897d-ebf0ca9b4584Configure dbhome storage
Starting with Oracle Database Appliance release 19.11, Oracle Database homes are not created on the local disk. Oracle Database homes are created on an Oracle ACFS-managed file system. As we are patching from 19.10, we have to configure a dbhome storage. In fact we decide on which disk group future dbhomes will reside
[root@serveroda1 bin]# odacli configure-dbhome-storage -dg DATA
{
"jobId" : "5e33c0e5-64c0-404e-a3b9-5ca061830b86",
"status" : "Created",
"message" : "DBhome storage configuration is stored in the metadata and will take effect on next database home creation.",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 11:03:00 AM CET",
"resourceList" : [ ],
"description" : "Configure database home storage",
"updatedTime" : "January 03, 2023 11:03:00 AM CET"
}
Validate the job
[root@serveroda1 bin]# odacli describe-job -i "5e33c0e5-64c0-404e-a3b9-5ca061830b86"
Job details
----------------------------------------------------------------
ID: 5e33c0e5-64c0-404e-a3b9-5ca061830b86
Description: Configure database home storage
Status: Success
Created: January 3, 2023 11:03:00 AM CET
Message: DBhome storage configuration is stored in the metadata and will take effect on next dbhome creation.
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Configuring database home storage January 3, 2023 11:03:00 AM CET January 3, 2023 11:03:00 AM CET SuccessUpdate dbhome
In our case we decide to only update the oracle 19c homes
[root@serveroda2 ~]# odacli list-dbhomes
ID Name DB Version Home Location Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
968b790a-9193-4450-884b-04b1e53fdf2d OraDB19000_home1 19.10.0.0.210119 /u01/app/oracle/product/19.0.0.0/dbhome_1 CONFIGURED
21e5db93-d0e8-4e47-bcb1-5da8876e8d24 OraDB19000_home2 19.10.0.0.210119 /u01/app/oracle/product/19.0.0.0/dbhome_2 CONFIGURED
Run a prepatch report for the dbhome to be updated
[root@serveroda2 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 968b790a-9193-4450-884b-04b1e53fdf2d -v 19.14.0.0.0
Job details
----------------------------------------------------------------
ID: ab290e9f-2647-4302-9d1b-e0887d29e254
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home1
Status: Created
Created: January 3, 2023 11:17:10 AM CET
Message: Use 'odacli describe-prepatchreport -i ab290e9f-2647-4302-9d1b-e0887d29e254' to check details of results
Check that the report is fine
[root@serveroda1 bin]# odacli describe-prepatchreport -i ab290e9f-2647-4302-9d1b-e0887d29e254
Patch pre-check report
------------------------------------------------------------------------
Job ID: ab290e9f-2647-4302-9d1b-e0887d29e254
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home1
Status: SUCCESS
Created: January 3, 2023 11:17:10 AM CET
Result: All pre-checks succeeded
Node Name
---------------
serveroda1
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID Success Validated DB Home ID:
968b790a-9193-4450-884b-04b1e53fdf2d
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate minimum agent version Success Validated minimum agent version
Is GI upgraded Success Validated GI is upgraded
Validate available space for Success Space for /u01/app/odaorahome will be
db created during patching
Validate dbHomesOnACFS Success User has configured diskgroup for
configured Database homes on ACFS
Validate Oracle base Success Successfully validated Oracle Base
Is DB clone available Success Successfully validated clone file
exists
Evaluate DBHome patching with Success Successfully validated updating
RHP dbhome with RHP
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
Node Name
---------------
serveroda2
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID Success Validated DB Home ID:
968b790a-9193-4450-884b-04b1e53fdf2d
Validate patching tag Success Validated patching tag: 19.14.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate minimum agent version Success Validated minimum agent version
Is GI upgraded Success Validated GI is upgraded
Validate available space for Success Space for /u01/app/odaorahome will be
db created during patching
Validate dbHomesOnACFS Success User has configured diskgroup for
configured Database homes on ACFS
Validate Oracle base Success Successfully validated Oracle Base
Is DB clone available Success Successfully validated clone file
exists
Evaluate DBHome patching with Success Successfully validated updating
RHP dbhome with RHP
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
And then update the dbhome
[root@serveroda1 bin]# /opt/oracle/dcs/bin/odacli update-dbhome --dbhomeid 968b790a-9193-4450-884b-04b1e53fdf2d -v 19.14.0.0.0
{
"jobId" : "eb8fa19d-60a1-4f45-88e1-c95b4a059417",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "January 03, 2023 11:40:23 AM CET",
"resourceList" : [ ],
"description" : "DB Home Patching: Home Id is 968b790a-9193-4450-884b-04b1e53fdf2d",
"updatedTime" : "January 03, 2023 11:40:24 AM CET"
}
The job should return SUCCESS
[root@serveroda1 bin]# odacli describe-job -i "eb8fa19d-60a1-4f45-88e1-c95b4a059417"
Job details
----------------------------------------------------------------
ID: eb8fa19d-60a1-4f45-88e1-c95b4a059417
Description: DB Home Patching: Home Id is 968b790a-9193-4450-884b-04b1e53fdf2d
Status: Success
Created: January 3, 2023 11:40:23 AM CET
Message:
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Adding USER SSH_EQUIVALENCE January 3, 2023 11:40:37 AM CET January 3, 2023 11:40:38 AM CET Success
Adding USER SSH_EQUIVALENCE January 3, 2023 11:40:38 AM CET January 3, 2023 11:40:39 AM CET Success
Adding USER SSH_EQUIVALENCE January 3, 2023 11:40:39 AM CET January 3, 2023 11:40:40 AM CET Success
Creating wallet for DB Client January 3, 2023 11:41:20 AM CET January 3, 2023 11:41:20 AM CET Success
Patch databases by RHP January 3, 2023 11:41:20 AM CET January 3, 2023 11:43:16 AM CET Success
updating database metadata January 3, 2023 11:44:10 AM CET January 3, 2023 11:44:10 AM CET Success
Set log_archive_dest for Database January 3, 2023 11:44:12 AM CET January 3, 2023 11:44:13 AM CET Success
Patch databases by RHP January 3, 2023 11:44:13 AM CET January 3, 2023 11:51:35 AM CET Success
updating database metadata January 3, 2023 11:52:44 AM CET January 3, 2023 11:52:44 AM CET Success
Set log_archive_dest for Database January 3, 2023 11:52:45 AM CET January 3, 2023 11:52:47 AM CET Success
Patch databases by RHP January 3, 2023 11:52:47 AM CET January 3, 2023 11:59:52 AM CET Success
updating database metadata January 3, 2023 12:01:06 PM CET January 3, 2023 12:01:06 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:01:08 PM CET January 3, 2023 12:01:11 PM CET Success
Patch databases by RHP January 3, 2023 12:01:11 PM CET January 3, 2023 12:02:49 PM CET Success
updating database metadata January 3, 2023 12:03:43 PM CET January 3, 2023 12:03:43 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:03:46 PM CET January 3, 2023 12:03:47 PM CET Success
Patch databases by RHP January 3, 2023 12:03:47 PM CET January 3, 2023 12:10:54 PM CET Success
updating database metadata January 3, 2023 12:12:01 PM CET January 3, 2023 12:12:01 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:12:02 PM CET January 3, 2023 12:12:05 PM CET Success
Patch databases by RHP January 3, 2023 12:12:05 PM CET January 3, 2023 12:13:45 PM CET Success
updating database metadata January 3, 2023 12:14:38 PM CET January 3, 2023 12:14:38 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:14:41 PM CET January 3, 2023 12:14:42 PM CET Success
Patch databases by RHP January 3, 2023 12:14:42 PM CET January 3, 2023 12:21:49 PM CET Success
updating database metadata January 3, 2023 12:22:57 PM CET January 3, 2023 12:22:57 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:22:59 PM CET January 3, 2023 12:23:02 PM CET Success
Patch databases by RHP January 3, 2023 12:23:02 PM CET January 3, 2023 12:30:09 PM CET Success
updating database metadata January 3, 2023 12:31:15 PM CET January 3, 2023 12:31:15 PM CET Success
Set log_archive_dest for Database January 3, 2023 12:31:16 PM CET January 3, 2023 12:31:19 PM CET Success
Update System version January 3, 2023 12:31:19 PM CET January 3, 2023 12:31:19 PM CET Success
Update System version January 3, 2023 12:31:19 PM CET January 3, 2023 12:31:19 PM CET Success
TDE parameter update January 3, 2023 12:32:03 PM CET January 3, 2023 12:32:03 PM CET Success
For each database of the updated dbhome run the following command
SQL> alter system set "_enable_numa_support"=true scope=spfile;2- Patch from 19.10 to 19.14
The steps are the same. Some ouputs below are truncated
Update the repository with the server patch and validate that the job is fine
[root@serveroda1 bin]# /opt/oracle/dcs/bin/odacli update-repository -f /u03/app/oracle/soft/19.17/oda-sm-19.17.0.0.0-221126.1-server.zip
{
"jobId" : "45dedaa4-0287-4e4b-843c-0b2cd57b48b7",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.17/oda-sm-19.17.0.0.0-221126.1-server.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 12:47:13 PM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 12:47:13 PM CET"
}
[root@serveroda1 bin]# odacli describe-job -i "45dedaa4-0287-4e4b-843c-0b2cd57b48b7"
Job details
----------------------------------------------------------------
ID: 45dedaa4-0287-4e4b-843c-0b2cd57b48b7
Description: Repository Update
Status: Success
Created: January 3, 2023 12:47:13 PM CET
Message: /u03/app/oracle/soft/19.17/oda-sm-19.17.0.0.0-221126.1-server.zip
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Check AvailableSpace January 3, 2023 12:47:21 PM CET January 3, 2023 12:47:21 PM CET Success
...
...
January 3, 2023 12:48:12 PM CET January 3, 2023 12:48:42 PM CET Success
Removing ssh keys January 3, 2023 12:48:44 PM CET January 3, 2023 12:48:45 PM CET Success
Update the DscAdmin
[root@serveroda1 bin]# odacli update-dcsadmin -v 19.17.0.0.0
{
"jobId" : "32c630b5-d796-42e0-94a2-9900c8ce2886",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "January 03, 2023 12:52:14 PM CET",
"resourceList" : [ ],
"description" : "DcsAdmin patching",
"updatedTime" : "January 03, 2023 12:52:14 PM CET"
}
[root@serveroda1 bin]# odacli describe-job -i "32c630b5-d796-42e0-94a2-9900c8ce2886"
Job details
----------------------------------------------------------------
ID: 32c630b5-d796-42e0-94a2-9900c8ce2886
Description: DcsAdmin patching
Status: Success
Created: January 3, 2023 12:52:14 PM CET
Message:
...
...Update the DCS components
[root@serveroda1 bin]# odacli update-dcscomponents -v 19.17.0.0.0
{
"jobId" : "aed5c87c-d974-4942-baea-80b524436c45",
"status" : "Success",
"message" : "Update-dcscomponents is successful on all the node(s):DCS-Agent shutdown is successful. MySQL upgrade is successful. Metadata migration is done before. Metadata schema update is done. dcsagent RPM upgrade is successful. dcscli RPM upgrade is successful. dcscontroller RPM upgrade is successful. Successfully ran setupAgentAuth.sh zookeeper RPM upgrade is successful. ",
"reports" : null,
"createTimestamp" : "January 03, 2023 12:53:19 PM CET",
"description" : "Update-dcscomponents job completed and is not part of Agent job list",
"updatedTime" : "January 03, 2023 12:56:49 PM CET"
}
Update the DCS agent
[root@serveroda1 bin]# odacli update-dcsagent -v 19.17.0.0.0
{
"jobId" : "c401c1c6-cb7f-4ac8-8b91-31b91ddf429c",
"status" : "Created",
"message" : "Dcs agent will be restarted after the update. Please wait for 2-3 mins before executing the other commands",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 12:58:04 PM CET",
"resourceList" : [ ],
"description" : "DcsAgent patching",
"updatedTime" : "January 03, 2023 12:58:04 PM CET",
"jobType" : null
}
[root@serveroda1 bin]# odacli describe-job -i "c401c1c6-cb7f-4ac8-8b91-31b91ddf429c"
Job details
----------------------------------------------------------------
ID: c401c1c6-cb7f-4ac8-8b91-31b91ddf429c
Description: DcsAgent patching
Status: Success
Created: January 3, 2023 12:58:04 PM CET
Message:
...
...Update the repository with the GI and DB clones
GI
[root@serveroda1 bin]# odacli update-repository -f /u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-GI-19.17.0.0.zip
{
"jobId" : "a6c09e2e-f3ee-45a7-9b88-e0dcd85df560",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-GI-19.17.0.0.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 13:04:16 PM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 13:04:16 PM CET",
"jobType" : null
}
[root@serveroda1 bin]# odacli describe-job -i "a6c09e2e-f3ee-45a7-9b88-e0dcd85df560"
Job details
----------------------------------------------------------------
ID: a6c09e2e-f3ee-45a7-9b88-e0dcd85df560
Description: Repository Update
Status: Success
Created: January 3, 2023 1:04:16 PM CET
Message: /u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-GI-19.17.0.0.zip
...
...DB
[root@serveroda1 bin]# odacli update-repository -f /u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-DB-19.17.0.0.zip
{
"jobId" : "6d660d44-78c5-4f52-98cd-c7e25b734c65",
"status" : "Created",
"message" : "/u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-DB-19.17.0.0.zip",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 13:07:00 PM CET",
"resourceList" : [ ],
"description" : "Repository Update",
"updatedTime" : "January 03, 2023 13:07:00 PM CET",
"jobType" : null
}
[root@serveroda1 bin]# odacli describe-job -i "6d660d44-78c5-4f52-98cd-c7e25b734c65"
Job details
----------------------------------------------------------------
ID: 6d660d44-78c5-4f52-98cd-c7e25b734c65
Description: Repository Update
Status: Success
Created: January 3, 2023 1:07:00 PM CET
Message: /u03/app/oracle/soft/19.17/odacli-dcs-19.17.0.0.0-221029-DB-19.17.0.0.zip
...
...Run prepatch report for the server
[root@serveroda1 bin]# odacli create-prepatchreport -s -v 19.17.0.0.0
Job details
----------------------------------------------------------------
ID: 03a5816c-fa51-42cf-9445-7ee429445c48
Description: Patch pre-checks for [OS, ILOM, GI, ORACHKSERVER, SERVER]
Status: Created
Created: January 3, 2023 1:10:25 PM CET
Message: Use 'odacli describe-prepatchreport -i 03a5816c-fa51-42cf-9445-7ee429445c48' to check details of results
Check the results for any error and correct it
[root@serveroda1 bin]# 'odacli describe-prepatchreport -i 03a5816c-fa51-42cf-9445-7ee429445c48
Update the server
[root@serveroda1 bin]# odacli update-server -v 19.17.0.0.0
{
"jobId" : "57d58ccd-b570-419f-b0b5-d25d58f5adec",
"status" : "Created",
"message" : "Success of server update will trigger reboot of the node after 4-5 minutes. Please wait until the node reboots.",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 13:43:29 PM CET",
"resourceList" : [ ],
"description" : "Server Patching",
"updatedTime" : "January 03, 2023 13:43:29 PM CET",
Validate the results
[root@serveroda1 bin]# odacli describe-job -i "57d58ccd-b570-419f-b0b5-d25d58f5adec"
Job details
----------------------------------------------------------------
ID: 57d58ccd-b570-419f-b0b5-d25d58f5adec
Description: Server Patching
Status: Success
Created: January 3, 2023 1:43:29 PM CET
Message: Successfully patched GI with RHP
Task Name Node Name Start Time End Time Status
---------------------------------------- ------------------------- ----------------------------------- ----------------------------------- ----------
Validating GI user metadata serveroda1 January 3, 2023 1:43:52 PM CET January 3, 2023 1:43:52 PM CET Success
Validating GI user metadata serveroda2 January 3, 2023 1:43:52 PM CET January 3, 2023 1:43:52 PM CET Success
Validate ILOM server reachable serveroda2 January 3, 2023 1:43:52 PM CET January 3, 2023 1:43:52 PM CET Success
Validate ILOM server reachable serveroda1 January 3, 2023 1:43:52 PM CET January 3, 2023 1:43:53 PM CET Success
Validate DCS Admin mTLS setup serveroda2 January 3, 2023 1:43:53 PM CET January 3, 2023 1:43:53 PM CET Success
Validate DCS Admin mTLS setup serveroda1 January 3, 2023 1:43:53 PM CET January 3, 2023 1:43:53 PM CET Success
Configure export clones resource serveroda1 January 3, 2023 1:43:55 PM CET January 3, 2023 1:43:55 PM CET Success
Creating repositories using yum serveroda2 January 3, 2023 1:43:59 PM CET January 3, 2023 1:44:03 PM CET Success
Updating YumPluginVersionLock rpm serveroda2 January 3, 2023 1:44:03 PM CET January 3, 2023 1:44:03 PM CET Success
Applying OS Patches serveroda2 January 3, 2023 1:44:03 PM CET January 3, 2023 1:55:38 PM CET Success
Creating repositories using yum serveroda2 January 3, 2023 1:55:39 PM CET January 3, 2023 1:55:39 PM CET Success
Applying HMP Patches serveroda2 January 3, 2023 1:55:39 PM CET January 3, 2023 1:55:58 PM CET Success
Patch location validation serveroda2 January 3, 2023 1:55:59 PM CET January 3, 2023 1:55:59 PM CET Success
Patch location validation serveroda1 January 3, 2023 1:55:59 PM CET January 3, 2023 1:55:59 PM CET Success
Oda-hw-mgmt upgrade serveroda1 January 3, 2023 1:55:59 PM CET January 3, 2023 1:56:33 PM CET Success
Oda-hw-mgmt upgrade serveroda2 January 3, 2023 1:56:33 PM CET January 3, 2023 1:57:06 PM CET Success
OSS Patching serveroda2 January 3, 2023 1:57:06 PM CET January 3, 2023 1:57:07 PM CET Success
Applying Firmware Disk Patches serveroda2 January 3, 2023 1:57:09 PM CET January 3, 2023 1:57:12 PM CET Success
Applying Firmware Controller Patches serveroda2 January 3, 2023 1:58:52 PM CET January 3, 2023 2:00:33 PM CET Success
Checking Ilom patch Version serveroda1 January 3, 2023 2:00:33 PM CET January 3, 2023 2:00:33 PM CET Success
Checking Ilom patch Version serveroda2 January 3, 2023 2:00:33 PM CET January 3, 2023 2:00:34 PM CET Success
Patch location validation serveroda1 January 3, 2023 2:00:34 PM CET January 3, 2023 2:00:34 PM CET Success
Patch location validation serveroda2 January 3, 2023 2:00:34 PM CET January 3, 2023 2:00:34 PM CET Success
Save password in Wallet serveroda2 January 3, 2023 2:00:34 PM CET January 3, 2023 2:00:35 PM CET Success
Disabling IPMI v2 serveroda2 January 3, 2023 2:00:36 PM CET January 3, 2023 2:00:36 PM CET Success
Apply Ilom patch serveroda1 January 3, 2023 2:00:36 PM CET January 3, 2023 2:09:54 PM CET Success
Apply Ilom patch serveroda2 January 3, 2023 2:09:55 PM CET January 3, 2023 2:18:47 PM CET Success
Copying Flash Bios to Temp location serveroda1 January 3, 2023 2:18:47 PM CET January 3, 2023 2:18:47 PM CET Success
Copying Flash Bios to Temp location serveroda2 January 3, 2023 2:18:47 PM CET January 3, 2023 2:18:47 PM CET Success
Starting the clusterware serveroda2 January 3, 2023 2:20:44 PM CET January 3, 2023 2:22:28 PM CET Success
Registering image serveroda2 January 3, 2023 2:22:28 PM CET January 3, 2023 2:22:28 PM CET Success
Registering working copy serveroda1 January 3, 2023 2:22:28 PM CET January 3, 2023 2:22:28 PM CET Success
Registering image serveroda2 January 3, 2023 2:22:28 PM CET January 3, 2023 2:22:29 PM CET Success
Creating GI home directories serveroda2 January 3, 2023 2:22:29 PM CET January 3, 2023 2:22:29 PM CET Success
Extract GI clone serveroda1 January 3, 2023 2:22:29 PM CET January 3, 2023 2:22:29 PM CET Success
Extract GI clone serveroda2 January 3, 2023 2:22:29 PM CET January 3, 2023 2:22:29 PM CET Success
Provisioning Software Only GI with RHP serveroda1 January 3, 2023 2:22:29 PM CET January 3, 2023 2:22:29 PM CET Success
Patch GI with RHP serveroda1 January 3, 2023 2:22:29 PM CET January 3, 2023 2:36:51 PM CET Success
Updating GIHome version serveroda1 January 3, 2023 2:36:52 PM CET January 3, 2023 2:36:55 PM CET Success
Updating GIHome version serveroda2 January 3, 2023 2:36:52 PM CET January 3, 2023 2:36:55 PM CET Success
Validate GI availability serveroda1 January 3, 2023 2:37:04 PM CET January 3, 2023 2:37:05 PM CET Success
Patch KVM CRS type serveroda1 January 3, 2023 2:37:05 PM CET January 3, 2023 2:37:05 PM CET Success
Patch VM vDisks CRS dependencies serveroda1 January 3, 2023 2:37:05 PM CET January 3, 2023 2:37:05 PM CET Success
Patch DB System domain config serveroda1 January 3, 2023 2:37:05 PM CET January 3, 2023 2:37:05 PM CET Success
Update System version serveroda1 January 3, 2023 2:37:05 PM CET January 3, 2023 2:37:05 PM CET Success
Update System version serveroda2 January 3, 2023 2:37:05 PM CET January 3, 2023 2:37:05 PM CET Success
Cleanup JRE Home serveroda1 January 3, 2023 2:37:06 PM CET January 3, 2023 2:37:06 PM CET Success
Add SYSNAME in Env serveroda2 January 3, 2023 2:37:06 PM CET January 3, 2023 2:37:06 PM CET Success
Starting the clusterware serveroda2 January 3, 2023 2:37:06 PM CET January 3, 2023 2:37:06 PM CET Success
Setting ACL for disk groups serveroda1 January 3, 2023 2:37:06 PM CET January 3, 2023 2:37:13 PM CET Success
Enable LKCE serveroda2 January 3, 2023 2:39:09 PM CET January 3, 2023 2:41:48 PM CET Success
Enable LKCE serveroda1 January 3, 2023 2:39:09 PM CET January 3, 2023 2:41:47 PM CET Success
Update previous workarounds serveroda1 January 3, 2023 2:41:49 PM CET January 3, 2023 2:41:49 PM CET Success
Update previous workarounds serveroda2 January 3, 2023 2:41:49 PM CET January 3, 2023 2:41:49 PM CET Success
Generating and saving BOM serveroda2 January 3, 2023 2:41:49 PM CET January 3, 2023 2:43:17 PM CET Success
Generating and saving BOM serveroda1 January 3, 2023 2:41:49 PM CET January 3, 2023 2:43:22 PM CET Success
PreRebootNode Actions serveroda1 January 3, 2023 2:43:22 PM CET January 3, 2023 2:44:18 PM CET Success
PreRebootNode Actions serveroda2 January 3, 2023 2:44:18 PM CET January 3, 2023 2:45:11 PM CET Success
Reboot Ilom serveroda1 January 3, 2023 2:45:11 PM CET January 3, 2023 2:45:11 PM CET Success
Reboot Ilom serveroda2 January 3, 2023 2:45:11 PM CET January 3, 2023 2:45:11 PM CET Success
Validate the components after the patch
[root@serveroda1 ~]# odacli describe-component
System Version
---------------
19.17.0.0.0
System node Name
---------------
serveroda1
Local System Version
---------------
19.17.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK
19.17.0.0.0 up-to-date
GI
19.17.0.0.221018 up-to-date
DB {
[OraDB19000_home1]
19.10.0.0.210119 19.17.0.0.221018
[OraDB19000_home2]
19.10.0.0.210119 19.17.0.0.221018
[OraDB19000_home3 [OMESTCT1,OMEMECD1,
OMECSCD1,OMECSCP1,OMESTCD1,OMEMECP1,
OMESTCP1,OMESTCT2]] 19.14.0.0.220118 19.17.0.0.221018
}
DCSCONTROLLER
19.17.0.0.0 up-to-date
DCSCLI
19.17.0.0.0 up-to-date
DCSAGENT
19.17.0.0.0 up-to-date
DCSADMIN
19.17.0.0.0 up-to-date
OS
7.9 up-to-date
ILOM
5.1.0.23.r146986 up-to-date
BIOS
41120100 up-to-date
LOCAL CONTROLLER FIRMWARE {
[c3,c4]
214.2.271.9 up-to-date
[c5]
80000690 up-to-date
}
SHARED CONTROLLER FIRMWARE
16.00.08.00 up-to-date
FIRMWAREEXPANDER
0310 up-to-date
LOCAL DISK FIRMWARE
N2010112 up-to-date
SHARED DISK FIRMWARE {
[c0d0,c0d1]
N2010112 up-to-date
[c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6,
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8] A170 up-to-date
}
HMP
2.4.8.9.603 up-to-date
System node Name
---------------
serveroda2
Local System Version
---------------
19.17.0.0.0
Component Installed Version Available Version
---------------------------------------- -------------------- --------------------
OAK
19.17.0.0.0 up-to-date
GI
19.17.0.0.221018 up-to-date
DB {
[OraDB19000_home1]
19.10.0.0.210119 19.17.0.0.221018
[OraDB19000_home2]
19.10.0.0.210119 19.17.0.0.221018
[OraDB19000_home3 [OMESTCT1,OMEMECD1,
OMECSCD1,OMECSCP1,OMESTCD1,OMEMECP1,
OMESTCP1,OMESTCT2]] 19.14.0.0.220118 19.17.0.0.221018
}
DCSCONTROLLER
19.17.0.0.0 up-to-date
DCSCLI
19.17.0.0.0 up-to-date
DCSAGENT
19.17.0.0.0 up-to-date
DCSADMIN
19.17.0.0.0 up-to-date
OS
7.9 up-to-date
ILOM
5.1.0.23.r146986 up-to-date
BIOS
41120100 up-to-date
LOCAL CONTROLLER FIRMWARE {
[c3]
80000690 up-to-date
[c4,c5]
214.2.271.9 up-to-date
}
SHARED CONTROLLER FIRMWARE
16.00.08.00 up-to-date
FIRMWAREEXPANDER
0310 up-to-date
LOCAL DISK FIRMWARE
N2010112 up-to-date
SHARED DISK FIRMWARE {
[c0d0,c0d1]
N2010112 up-to-date
[c1d0,c1d1,c1d2,c1d3,c1d4,c1d5,c1d6,
c1d7,c1d8,c2d0,c2d1,c2d2,c2d3,c2d4,c2d5,
c2d6,c2d7,c2d8] A170 up-to-date
}
HMP
2.4.8.9.603 up-to-date
Update the storage components
Run a prepatch report for storage
[root@serveroda1 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport -st -v 19.17.0.0.0
Job details
----------------------------------------------------------------
ID: e50389ae-8944-44ff-b45a-c454cb33e050
Description: Patch pre-checks for [STORAGE]
Status: Created
Created: January 3, 2023 2:57:54 PM CET
Message: Use 'odacli describe-prepatchreport -i e50389ae-8944-44ff-b45a-c454cb33e050' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
[root@serveroda1 ~]# odacli describe-prepatchreport -i e50389ae-8944-44ff-b45a-c454cb33e050
Patch pre-check report
------------------------------------------------------------------------
Job ID: e50389ae-8944-44ff-b45a-c454cb33e050
Description: Patch pre-checks for [STORAGE]
Status: SUCCESS
Created: January 3, 2023 2:57:54 PM CET
Result: All pre-checks succeeded
...
...Patch the storage components
[root@serveroda1 ~]# /opt/oracle/dcs/bin/odacli update-storage -v 19.17.0.0.0 --rolling
{
"jobId" : "d02fa340-c82c-482e-9196-aac0d18d2dfc",
"status" : "Created",
"message" : "Success of Storage Update may trigger reboot of node after 4-5 minutes. Please wait till node restart",
"reports" : [ ],
"createTimestamp" : "January 03, 2023 15:02:10 PM CET",
"resourceList" : [ ],
"description" : "Storage Firmware Patching",
"updatedTime" : "January 03, 2023 15:02:10 PM CET",
"jobType" : null
}
[root@serveroda1 ~]# odacli describe-job -i "d02fa340-c82c-482e-9196-aac0d18d2dfc"
Job details
----------------------------------------------------------------
ID: d02fa340-c82c-482e-9196-aac0d18d2dfc
Description: Storage Firmware Patching
Status: Success
Created: January 3, 2023 3:02:10 PM CET
Message:
Task Name Node Name Start Time End Time Status
---------------------------------------- ------------------------- ----------------------------------- ----------------------------------- ----------
Applying Firmware Disk Patches serveroda1 January 3, 2023 3:02:22 PM CET January 3, 2023 3:02:30 PM CET Success
Generating and saving BOM serveroda1 January 3, 2023 3:02:30 PM CET January 3, 2023 3:04:15 PM CET Success
Generating and saving BOM serveroda2 January 3, 2023 3:02:30 PM CET January 3, 2023 3:04:10 PM CET Success
Updating databases
For 19.17, we plan to update only some specific databases. In fact in 19.17 there is a command update-database
We first run the prepatch report for the corresponding dbhome
[root@serveroda1 ~]# /opt/oracle/dcs/bin/odacli create-prepatchreport --dbhome --dbhomeid 3d5f2d22-16db-4724-87e5-4965d6428e2e -v 19.17.0.0.0
Job details
----------------------------------------------------------------
ID: cd643e18-1ed7-40db-81a9-cf03d8c1734e
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home3
Status: Created
Created: January 3, 2023 3:07:49 PM CET
Message: Use 'odacli describe-prepatchreport -i cd643e18-1ed7-40db-81a9-cf03d8c1734e' to check details of results
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
We then validate that the report is fine
[root@serveroda1 ~]# odacli describe-prepatchreport -i cd643e18-1ed7-40db-81a9-cf03d8c1734e
Patch pre-check report
------------------------------------------------------------------------
Job ID: cd643e18-1ed7-40db-81a9-cf03d8c1734e
Description: Patch pre-checks for [DB, ORACHKDB]: DbHome is OraDB19000_home3
Status: SUCCESS
Created: January 3, 2023 3:07:49 PM CET
Result: All pre-checks succeeded
Node Name
---------------
serveroda1
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID Success Validated DB Home ID:
3d5f2d22-16db-4724-87e5-4965d6428e2e
Validate patching tag Success Validated patching tag: 19.17.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate minimum agent version Success Validated minimum agent version
Is GI upgraded Success Validated GI is upgraded
Validate available space for Success Validated free space required under
db /u01
Validate dbHomesOnACFS Success User has configured disk group for
configured Database homes on ACFS
Validate Oracle base Success Successfully validated Oracle Base
Is DB clone available Success Successfully validated clone file
exists
Evaluate DBHome patching with Success Successfully validated updating
RHP dbhome with RHP. and local patching
is possible
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
Node Name
---------------
serveroda2
Pre-Check Status Comments
------------------------------ -------- --------------------------------------
__DB__
Validate DB Home ID Success Validated DB Home ID:
3d5f2d22-16db-4724-87e5-4965d6428e2e
Validate patching tag Success Validated patching tag: 19.17.0.0.0.
Is system provisioned Success Verified system is provisioned
Validate minimum agent version Success Validated minimum agent version
Is GI upgraded Success Validated GI is upgraded
Validate available space for Success Validated free space required under
db /u01
Validate dbHomesOnACFS Success User has configured disk group for
configured Database homes on ACFS
Validate Oracle base Success Successfully validated Oracle Base
Is DB clone available Success Successfully validated clone file
exists
Evaluate DBHome patching with Success Successfully validated updating
RHP dbhome with RHP. and local patching
is possible
Validate command execution Success Validated command execution
__ORACHK__
Running orachk Success Successfully ran Orachk
Validate command execution Success Validated command execution
And we run the update-database command
[root@serveroda1 ~]# /opt/oracle/dcs/bin/odacli update-database -i a2b3ce19-1ce4-4394-b436-d0e4817d8111 -to ed8970e5-baf5-4b12-8209-809cbacbcbc7
{
"jobId" : "97828641-18ee-4624-a150-c66b42dfae5f",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "January 03, 2023 15:28:40 PM CET",
"resourceList" : [ ],
"description" : "DB Patching: database ID is a2b3ce19-1ce4-4394-b436-d0e4817d8111",
"updatedTime" : "January 03, 2023 15:28:40 PM CET",
"jobType" : ""
}
A few minutes after the job fails
[root@serveroda1 ~]# odacli describe-job -i "97828641-18ee-4624-a150-c66b42dfae5f"
Job details
----------------------------------------------------------------
ID: 97828641-18ee-4624-a150-c66b42dfae5f
Description: DB Patching: database ID is a2b3ce19-1ce4-4394-b436-d0e4817d8111
Status: Failure
Created: January 3, 2023 3:28:40 PM CET
Message: DCS-10001:Internal error encountered: null.
Task Name Node Name Start Time End Time Status
---------------------------------------- ------------------------- ----------------------------------- ----------------------------------- ----------
DB Patching serveroda1 January 3, 2023 3:28:43 PM CET January 3, 2023 3:31:50 PM CET Failure
DB Patching serveroda1 January 3, 2023 3:28:43 PM CET January 3, 2023 3:31:50 PM CET Failure
task:TaskSequential_7670 serveroda1 January 3, 2023 3:28:50 PM CET January 3, 2023 3:30:55 PM CET Failure
Creating wallet for DB Client serveroda1 January 3, 2023 3:29:29 PM CET January 3, 2023 3:29:30 PM CET Success
Patch databases by RHP serveroda1 January 3, 2023 3:29:30 PM CET January 3, 2023 3:30:55 PM CET Failure
We check the various but did not find the root cause. An SR is opened with Oracle
Conclusion
To resume patching an ODA HA is almost the same as patching a non-HA ODA server. We recommend to carefully read the documentation before starting the patch
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/12/oracle-square.png)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2024/01/HME_web.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/11/CBL_web-min-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/HER_web-min-scaled.jpg)
Hi Alex
Can we go from 19.14 to 19.17? Yes
Can we go from 19.10 to 19.14? yes
So we can go with 19.10 ==> 19.14 ==> 19.17
Regards