Introduction
If you use Enterprise Edition on your Oracle Database Appliance environment, and if you don’t have unlimited licences, you probably lowered the CPU cores available on each server to dispatch your licences accross your environment. This operation is normally done after deploying your ODAs, and this number of cores could eventually be increased if you need more and if you buy additional licences later. For most of us, the CPU core configuration will stay in its initial setup. But patching your ODA can enable back all the cores on your server, without any warning. Your configuration will then no more comply with your licences, and you may not even notice.
Use case
I recently patched several ODAs X8-2M from 19.11 to 19.14. After solving issues described in the pre-patch report, I was able to apply the system patch without any problem.
odacli describe-job -i 81c88506-5e05-4050-a244-b36d0c343327
Job details
----------------------------------------------------------------
ID: 81c88506-5e05-4050-a244-b36d0c343327
Description: Server Patching
Status: Success
Created: August 17, 2022 9:13:32 AM CEST
Message: Successfully patched GI with RHP
Task Name Start Time End Time Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validating GI user metadata August 17, 2022 9:13:42 AM CEST August 17, 2022 9:13:42 AM CEST Success
Validate ILOM server reachable August 17, 2022 9:13:42 AM CEST August 17, 2022 9:13:42 AM CEST Success
Configure export clones resource August 17, 2022 9:15:31 AM CEST August 17, 2022 9:15:32 AM CEST Success
Creating repositories using yum August 17, 2022 9:15:32 AM CEST August 17, 2022 9:15:36 AM CEST Success
Updating YumPluginVersionLock rpm August 17, 2022 9:15:36 AM CEST August 17, 2022 9:15:36 AM CEST Success
Applying OS Patches August 17, 2022 9:15:36 AM CEST August 17, 2022 9:24:57 AM CEST Success
Creating repositories using yum August 17, 2022 9:24:57 AM CEST August 17, 2022 9:24:57 AM CEST Success
Applying HMP Patches August 17, 2022 9:24:57 AM CEST August 17, 2022 9:25:14 AM CEST Success
Patch location validation August 17, 2022 9:25:14 AM CEST August 17, 2022 9:25:14 AM CEST Success
oda-hw-mgmt upgrade August 17, 2022 9:25:14 AM CEST August 17, 2022 9:25:45 AM CEST Success
OSS Patching August 17, 2022 9:25:45 AM CEST August 17, 2022 9:25:46 AM CEST Success
Applying Firmware Disk Patches August 17, 2022 9:25:46 AM CEST August 17, 2022 9:25:48 AM CEST Success
Applying Firmware Controller Patches August 17, 2022 9:25:48 AM CEST August 17, 2022 9:25:50 AM CEST Success
Checking Ilom patch Version August 17, 2022 9:25:50 AM CEST August 17, 2022 9:25:50 AM CEST Success
Patch location validation August 17, 2022 9:25:50 AM CEST August 17, 2022 9:25:50 AM CEST Success
Save password in Wallet August 17, 2022 9:25:50 AM CEST August 17, 2022 9:25:51 AM CEST Success
Disabling IPMI v2 August 17, 2022 9:25:51 AM CEST August 17, 2022 9:25:52 AM CEST Success
Apply Ilom patch August 17, 2022 9:25:52 AM CEST August 17, 2022 9:34:52 AM CEST Success
Copying Flash Bios to Temp location August 17, 2022 9:34:52 AM CEST August 17, 2022 9:34:52 AM CEST Success
Patch location validation August 17, 2022 9:34:52 AM CEST August 17, 2022 9:34:52 AM CEST Success
ASR Manager RPM update August 17, 2022 9:34:52 AM CEST August 17, 2022 9:49:30 AM CEST Success
Modify JavaExec Path August 17, 2022 9:49:30 AM CEST August 17, 2022 9:49:30 AM CEST Success
Remove AsrConfBackup File August 17, 2022 9:49:30 AM CEST August 17, 2022 9:49:30 AM CEST Success
Starting the clusterware August 17, 2022 9:49:31 AM CEST August 17, 2022 9:53:15 AM CEST Success
registering image August 17, 2022 9:53:15 AM CEST August 17, 2022 9:53:16 AM CEST Success
registering working copy August 17, 2022 9:53:16 AM CEST August 17, 2022 9:53:16 AM CEST Success
registering image August 17, 2022 9:53:16 AM CEST August 17, 2022 9:53:16 AM CEST Success
Creating GI home directories August 17, 2022 9:53:16 AM CEST August 17, 2022 9:53:16 AM CEST Success
Extract GI clone August 17, 2022 9:53:16 AM CEST August 17, 2022 9:53:16 AM CEST Success
Provisioning Software Only GI with RHP August 17, 2022 9:53:16 AM CEST August 17, 2022 9:53:16 AM CEST Success
Patch GI with RHP August 17, 2022 9:53:16 AM CEST August 17, 2022 10:00:49 AM CEST Success
Updating GIHome version August 17, 2022 10:00:49 AM CEST August 17, 2022 10:00:52 AM CEST Success
Validate GI availability August 17, 2022 10:01:04 AM CEST August 17, 2022 10:01:04 AM CEST Success
Patch KVM CRS type August 17, 2022 10:01:04 AM CEST August 17, 2022 10:01:05 AM CEST Success
Update System version August 17, 2022 10:01:05 AM CEST August 17, 2022 10:01:05 AM CEST Success
Cleanup JRE Home August 17, 2022 10:01:05 AM CEST August 17, 2022 10:01:05 AM CEST Success
Add SYSNAME in Env August 17, 2022 10:01:05 AM CEST August 17, 2022 10:01:05 AM CEST Success
Starting the clusterware August 17, 2022 10:01:05 AM CEST August 17, 2022 10:01:05 AM CEST Success
Setting ACL for disk groups August 17, 2022 10:01:05 AM CEST August 17, 2022 10:01:10 AM CEST Success
Update lvm.conf file August 17, 2022 10:02:40 AM CEST August 17, 2022 10:02:40 AM CEST Success
Update previous workarounds August 17, 2022 10:02:40 AM CEST August 17, 2022 10:02:40 AM CEST Success
preRebootNode Actions August 17, 2022 10:02:40 AM CEST August 17, 2022 10:16:45 AM CEST Success
Reboot Ilom August 17, 2022 10:16:45 AM CEST August 17, 2022 10:16:45 AM CEST SuccessHow to detect the problem?
After patching the server and after the reboot, all cores are enabled as if I never applied the core reduction:
odacli describe-cpucore
Node Cores Modified Job Status
----- ------ ---------------------------------- ---------------
0 8 January 19, 2021 1:58:17 PM CET CONFIGURED
lscpu | grep CPU
CPU op-mode(s): 32-bit, 64-bit
CPU(s): 64
On-line CPU(s) list: 0-63
CPU family: 6
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU MHz: 2800.135
CPU max MHz: 3900.0000
CPU min MHz: 1000.0000
NUMA node0 CPU(s): 0-15,32-47
NUMA node1 CPU(s): 16-31,48-63
Threads from 0 to 63 are enabled, meaning 32 cores. This is the total number of cores of my two Xeon Gold 5218. But I only need 8 cores and paid for 8 cores, meaning 4 Enterprise Edition licences. My ODA is no more compliant with my licences after this patch.
How to fix the problem?
I will try to configure again 8 cores on my ODA using odacli:
odacli update-cpucore -c 8
Modifying the enabled number of CPU requires a reboot of all nodes in the ODA system. Are you sure you want to proceed with this operation? (Y/N): Y
DCS-10044:Object cpu cores already exists with same value 8.It doesn’t work because 8 is my actual core configuration according to odacli.
I will decrease the number of cores, and then increase it again:
odacli update-cpucore -c 6 -f
Modifying the enabled number of CPU requires a reboot of all nodes in the ODA system. Are you sure you want to proceed with this operation? (Y/N): Y
{
"jobId" : "c0e0befd-119b-4c96-9b88-cfa5b21e37b5",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "August 17, 2022 11:34:18 AM CEST",
"resourceList" : [ ],
"description" : "CPU cores service update",
"updatedTime" : "August 17, 2022 11:34:18 AM CEST"
}
WARNING: Reducing the number of enabled cores may set offline any of the CPUs used by a CPU Pool. Thus, please review the status of all the CPU Pools after this operation using the 'describe-cpupool' command. In case of any CPU from a CPU Pool was offline please run 'remap-cpupools -f' to update the list of CPUs and use only online CPUs.
WARNING: All the nodes in this ODA system will reboot after this operation has completed.
odacli update-cpucore -c 8
Modifying the enabled number of CPU requires a reboot of all nodes in the ODA system. Are you sure you want to proceed with this operation? (Y/N): Y
{
"jobId" : "574a6b7f-ef91-4fd0-9202-1f977fad151b",
"status" : "Created",
"message" : null,
"reports" : [ ],
"createTimestamp" : "August 17, 2022 11:47:23 AM CEST",
"resourceList" : [ ],
"description" : "CPU cores service update",
"updatedTime" : "August 17, 2022 11:47:23 AM CEST"
}
WARNING: Reducing the number of enabled cores may set offline any of the CPUs used by a CPU Pool. Thus, please review the status of all the CPU Pools after this operation using the 'describe-cpupool' command. In case of any CPU from a CPU Pool was offline please run 'remap-cpupools -f' to update the list of CPUs and use only online CPUs.
WARNING: All the nodes in this ODA system will reboot after this operation has completed.
odacli describe-cpucore
Node Cores Modified Job Status
----- ------ ---------------------------------- ---------------
0 8 August 17, 2022 11:49:12 AM CEST CONFIGURED
lscpu | grep CPU
CPU op-mode(s): 32-bit, 64-bit
CPU(s): 16
On-line CPU(s) list: 0-15
CPU family: 6
Model name: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU MHz: 3699.938
CPU max MHz: 3900.0000
CPU min MHz: 1000.0000
NUMA node0 CPU(s): 0-3,8-11
NUMA node1 CPU(s): 4-7,12-15
odacli describe-system | grep Core
CPU Core Count: 8My cores setup is back to normal.
Limits
This will not work if you only have 2 cores configured on your ODA. You will then need to increase to 4 cores, and decrease to 2 cores. This can be a problem as this temporary increase can be detected and it’s not allowed. I would recommend opening a Service Request on My Oracle Support. You’re not supposed to move CPU cores’ configuration if it’s not related to licences upgrade.
Conclusion
I don’t know if this problem is specific to this environment or if it’s more widespread. But you should add another post-patch check when applying a patch on your ODA: check with lscpu if the number of enabled cores matches your odacli configuration.
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/JDU_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/ALN_web-min-scaled.jpg)