dbi Blog

Introduction

Oracle Database Appliance now fully supports virtualization with KVM, and this could be interesting for some of us. If you’re using Enterprise Edition, you probably decreased the number of cores to comply with your licenses. Now you can increase your core configuration with more cores, some being dedicated to your databases, and some for user-managed VMs. Let’s find out how you can migrate from bare-metal core allocation to CPU pools and discover the possibilities of this feature.

Updated the 1st of July, 2022: it looks like Bare-Metal CPU pools ARE NOT considered as hard-partitioning with Enterprise Edition. It means that despite using them for your Bare-Metal databases, all the enabled cores on your ODA need to be licensed. Those used for Bare-Metal CPU pools but also those remaining on the system, even if they are dedicated to VMs, and never used for your databases. The only supported mode for hard-partitioning is databases in DB Systems only, or core reduction according to your license on your ODA if you need Bare-Metal databases.

3 kinds of CPU pool

CPU pools are of 3 types on ODA:

Bare-Metal: these are CPU pools you can associate with Bare-Metal databases (must be licensed)
DB Systems: these are CPU pools you can associate with your DB Systems (must be licensed)
user-managed VMs: these are CPU pools for your multi-purpose VMs that don’t require any Oracle Database license

You can define multiple CPU pools of each kind, or none on your ODA.

This CPU pool configuration comes on top of your CPU core configuration. Previously, the only way to match your license with your ODA was to decrease the number of cores available on your system with odacli update-cpucore. Now it is much more flexible.

Configuration without using CPU pools

Without using CPU pools, on Enterprise Edition, core reduction is needed to match your licenses. For example, for 2 EE licenses on my ODA X8-2M, I need to lower my cores available to 4 after deployment:

odacli describe-cpucore


Node  Cores  Modified                           Job Status
----- ------ ---------------------------------- ---------------
0     32     February 17, 2022 1:55:24 PM CET   CONFIGURED

lscpu | grep MHz
CPU MHz:               2799.817
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000

odacli update-cpucore -c 4
Modifying the enabled number of CPU requires a reboot of all nodes in the ODA system. Are you sure you want to proceed with this operation? (Y/N): Y

After the reboot, check new core configuration and CPU speed:

odacli describe-cpucore


Node  Cores  Modified                           Job Status
----- ------ ---------------------------------- ---------------
0     4      February 18, 2022 4:05:44 PM CET   CONFIGURED

lscpu | grep MHz
CPU MHz:               3899.943
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000

As I already told you in another blog post, reducing the cores is giving more speed thanks to Turbo Boost. With only a few cores enabled, you can expect full speed from your Xeons.

Increase core configuration

On my ODA configured with 4 cores, I will first increase the number of cores on the server. I want to keep only 4 cores for databases according to my license.

odacli update-cpucore -c 8
Modifying the enabled number of CPU requires a reboot of all nodes in the ODA system. Are you sure you want to proceed with this operation? (Y/N): Y

Let’s check configuration and CPU speed after the reboot:

odacli describe-cpucore


Node  Cores  Modified                           Job Status
----- ------ ---------------------------------- ---------------
0     8      February 18, 2022 4:25:11 PM CET   CONFIGURED

lscpu | grep MHz
CPU MHz:               3685.555
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000

As expected, my cores are running a little bit slower. Note that you’re not supposed to decrease the core numbers. You are only allowed to increase them after initial configuration. And according to your license, this sounds obvious.

Configuring and playing with the CPU pools

Now let’s create a CPU pool for bare metal databases:

odacli create-cpupool -n cpupool4BM -c 4 -bm
odacli describe-job -i a818a597-a60a-4824-a8e3-85c53d3087e6 | grep Status:
                 Status:  Success

And let’s associate this CPU pool to all my bare metal databases:

odacli list-databases

ID                                       DB Name    DB Type  DB Version           CDB        Class    Shape    Storage    Status        DbHomeID
---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ----------------------------------------
42656d90-e18e-4325-a69a-13d3a6b58710     DBITST     SI       19.13.0.0.211019     true       OLTP     odb1     ASM        CONFIGURED   5e7a146a-c18d-427b-a206-e532ec9907cf
a2193c80-d004-4295-847e-ec8277952bd7     DBTAG38    SI       19.13.0.0.211019     false      OLTP     odb1     ASM        CONFIGURED   5e7a146a-c18d-427b-a206-e532ec9907cf


odacli modify-database -i 42656d90-e18e-4325-a69a-13d3a6b58710 -cp cpupool4BM
DB will be restarted as part of CPU Pool operation. Do you want to continue [y/n]:y


odacli describe-job -i ef4c6aff-c0e2-4493-9500-c3069559157e | grep Status:
                 Status:  Success

odacli modify-database -i a2193c80-d004-4295-847e-ec8277952bd7 -cp cpupool4BM
DB will be restarted as part of CPU Pool operation. Do you want to continue [y/n]:y

odacli describe-job -i c5e73921-619f-48dc-92c4-80be97128a23 | grep Status:
                 Status:  Success

Let’s create a CPU pool for user-managed VMs:

odacli create-cpupool -n cpupool4VM -c 4 -vm

odacli list-cpupools
Name                  Type                Configured on              Cores  Associated resources            Created                  Updated
--------------------  ------------------  -------------------------  -----  ------------------------------  -----------------------  -----------------------
cpupool4VM            VM                  dbi-oda-x8                 4      NONE                            2022-02-18 16:49:53 CET  2022-02-18 16:49:53 CET
cpupool4BM            BM                  dbi-oda-x8                 4      DBITST, DBTAG38                 2022-02-18 16:39:39 CET  2022-02-18 16:46:32 CET

Let’s create several VMs in this CPU pool:

odacli create-vmstorage -n VMstore -s 100G
odacli create-vm -n srvapp01 -m 8G -src /opt/dbi/AlmaLinux-8.4-x86_64-dvd.iso -vc 2 -cp cpupool4VM -vn pubnet -vms VMstore -s 16G -g "vnc,listen=10.36.0.241"
odacli create-vm -n srvapp02 -m 8G -src /opt/dbi/AlmaLinux-8.4-x86_64-dvd.iso -vc 2 -cp cpupool4VM -vn pubnet -vms VMstore -s 16G -g "vnc,listen=10.36.0.241"
odacli create-vm -n srvapp03 -m 8G -src /opt/dbi/AlmaLinux-8.4-x86_64-dvd.iso -vc 2 -cp cpupool4VM -vn pubnet -vms VMstore -s 16G -g "vnc,listen=10.36.0.241"

odacli list-vms
Name                  VM Storage            Node             Current State    Target State     Created                  Updated
--------------------  --------------------  ---------------  ---------------  ---------------  -----------------------  -----------------------
srvapp02              VMstore               dbi-oda-x8       ONLINE           ONLINE           2022-02-18 16:54:15 CET  2022-02-18 16:54:15 CET
srvapp01              VMstore               dbi-oda-x8       ONLINE           ONLINE           2022-02-18 16:53:45 CET  2022-02-18 16:53:45 CET
srvapp03              VMstore               dbi-oda-x8       ONLINE           ONLINE           2022-02-18 16:54:32 CET  2022-02-18 16:54:32 CET

No problem to configure 6 vCPUs on this 4-core CPU pool.

Normally I shouldn’t have enough cores to create a pool for DB Systems, let’s try:

odacli create-cpupool -n cpupool4DBS -c 4 -dbs

odacli describe-job  -i c0a27a41-469b-4145-bd1d-2bd770aeb5a7

Job details
----------------------------------------------------------------
                     ID:  c0a27a41-469b-4145-bd1d-2bd770aeb5a7
            Description:  CPU Pool cpupool4DBS creation
                 Status:  Failure
                Created:  February 18, 2022 4:55:45 PM CET
                Message:  DCS-10001:Internal error encountered: Not enough physical CPUs available for CPU Pool 'cpupool4DBS' on node 'dbi-oda-x8'.

Task Name                                Start Time                          End Time                            Status
---------------------------------------- ----------------------------------- ----------------------------------- ----------
Validate CPU Pool doesn't exist          February 18, 2022 4:55:45 PM CET    February 18, 2022 4:55:45 PM CET    Success
Validate NUMA state                      February 18, 2022 4:55:45 PM CET    February 18, 2022 4:55:45 PM CET    Success
Create metadata                          February 18, 2022 4:55:45 PM CET    February 18, 2022 4:55:45 PM CET    InternalError

Correct. This is not possible.

As I don’t want to add more licenses, let’s decrease cores on bare metal CPU pool. I will share my 2 licenses between BM and DBS:

odacli modify-cpupool -n cpupool4BM -c 2
WARNING: Must restart the following Oracle Databases associated with CPU Pool 'cpupool4BM' for the update to take effect: DBITST,DBTAG38

odacli describe-job -i 259d7fe1-afba-450f-beca-5410a5993160 | grep Status:
                 Status:  Success


odacli create-cpupool -n cpupool4DBS -c 2 -dbs

odacli describe-job -i f595cae9-fc00-4055-89e2-218d12307b06 | grep Status:
                 Status:  Success

I can now create some DB Systems using this CPU pool:

grep pool /opt/dbi/create_dbsystem_srvdb4*.json
/opt/dbi/create_dbsystem_srvdb47.json:        "cpuPoolName": "cpupool4DBS",
/opt/dbi/create_dbsystem_srvdb48.json:        "cpuPoolName": "cpupool4DBS",
/opt/dbi/create_dbsystem_srvdb49.json:        "cpuPoolName": "cpupool4DBS",

odacli create-dbsystem -p /opt/dbi/create_dbsystem_srvdb47.json
odacli create-dbsystem -p /opt/dbi/create_dbsystem_srvdb48.json
odacli create-dbsystem -p /opt/dbi/create_dbsystem_srvdb49.json

odacli list-dbsystems

Name                  Shape	  Cores  Memory      GI version          DB version          Status           Created                  Updated
--------------------  ----------  -----  ----------  ------------------  ------------------  ---------------  -----------------------  -----------------------
srvdb47               odb2        2	 16.00 GB    19.13.0.0.211019    19.13.0.0.211019    CONFIGURED       2022-02-22 17:40:42 CET  2022-02-22 18:13:44 CET
srvdb49               odb2        2	 16.00 GB    19.13.0.0.211019    19.13.0.0.211019    CONFIGURED       2022-02-22 16:27:54 CET  2022-02-22 16:58:43 CET
srvdb48               odb2        2	 16.00 GB    19.13.0.0.211019    19.13.0.0.211019    CONFIGURED       2022-02-22 15:53:25 CET  2022-02-22 16:23:22 CET

odacli list-cpupools

Name                  Type                Configured on              Cores  Associated resources            Created                  Updated
--------------------  ------------------  -------------------------  -----  ------------------------------  -----------------------  -----------------------
cpupool4VM            VM                  dbi-oda-x8                 4      srvapp02, srvapp01, srvapp03    2022-02-18 16:49:53 CET  2022-02-18 16:49:53 CET
cpupool4BM            BM                  dbi-oda-x8                 2      DBITST, DBTAG38                 2022-02-18 16:39:39 CET  2022-02-18 16:58:21 CET
cpupool4DBS           DB_SYSTEM_SHARED    dbi-oda-x8                 2      srvdb47, srvdb49, srvdb48       2022-02-18 16:59:34 CET  2022-02-18 16:59:34 CET

As you can see, there is no problem configuring multiple DB Systems on a 2-core CPU pool, as there is no problem configuring 10 bare metal databases on a 2-core CPU pool.

Another interesting thing is that you can define multiple CPU pools of each kind. For example, if you want to dedicate cores to specific bare metal databases, you just need to create a new bare metal pool (here I also need to decrease another pool to free up 2 cores – and it will cost me 1 more EE license):

odacli modify-cpupool -n cpupool4VM -c 2
odacli create-cpupool -n cpupool4BMPROD  -c 2 -bm
odacli modify-database -i a2193c80-d004-4295-847e-ec8277952bd7 -cp cpupool4BMPROD

odacli list-cpupools

Name                  Type                Configured on              Cores  Associated resources            Created                  Updated
--------------------  ------------------  -------------------------  -----  ------------------------------  -----------------------  -----------------------
cpupool4VM            VM                  dbi-oda-x8                 2      srvapp02, srvapp01, srvapp03    2022-02-18 16:49:53 CET  2022-02-22 18:38:39 CET
cpupool4BMPROD        BM                  dbi-oda-x8                 2      DBTAG38                         2022-02-22 18:39:20 CET  2022-02-22 18:41:07 CET
cpupool4BM            BM                  dbi-oda-x8                 2      DBITST                          2022-02-18 16:39:39 CET  2022-02-22 18:41:06 CET
cpupool4DBS           DB_SYSTEM_SHARED    dbi-oda-x8                 2      srvdb47, srvdb49, srvdb48       2022-02-18 16:59:34 CET  2022-02-18 16:59:34 CET

Conclusion

CPU pools are a brilliant addition to ODA. And it works like a charm. As this is still compatible with core reduction, you also benefit from high CPU core speed and load isolation. Bravo!

Post Views: 2,185

ODA: Migrating from bare metal to CPU pools

Introduction

3 kinds of CPU pool

Configuration without using CPU pools

Increase core configuration

Configuring and playing with the CPU pools

Conclusion

Jérôme Dubar

Leave a Reply:

Related blog articles

Introduction

3 kinds of CPU pool

Configuration without using CPU pools

Increase core configuration

Configuring and playing with the CPU pools

Conclusion

Jérôme Dubar

Leave a Reply:

Related blog articles

Add authentication in a Feathers.js REST API

A first look at openSUSE Leap Micro 6.0

Another file system for Linux: bcachefs (3) – Mirroring/Replicas

Another file system for Linux: bcachefs (2) – multi device file systems