Configure Oracle DB for using Large Pages on AIX

When it comes to deploying a new Oracle infrastructure, 90% of it is implemented on Linux. This makes sense, because Oracle runs very stable and performant on Linux and it is also Oracle’s own development platform. But since variety makes life interesting, I recently had the privilege to deploy a new Oracle environment on AIX for a customer.
AIX is not linux and some things work a bit different, but the installation of an Oracle database runs more or less smoothly.
Since according to Oracle best practices large pages should be used for the SGA, so we prepared 200 large pages with a size of 16M for the database. We could verify this with svmon utility

svmon

PageSize   PoolSize       inuse        pgsp         pin     virtual
s    4 KB         -      812827       29012      228798      252287
m   64 KB         -       61455         202       14189       61743
L   16 MB       200           0           0         200           0
S   16 GB         -           0           0           0           0

So we can see, that the 200 Pages are availabe but not in use, because at this point in time i don’t setup a database on the server. The next step was to create a Oracle 19c database with dbca in silent mode with all the necessary settings. In Oracle 19c the default setting for the init parameter USE_LARGE_PAGES=TRUE. That means if oracle find some large pages on the server it will use it and if not it will allocate the memory for the sga with normal pages.
So after the database was created i checked the alert.log of the database and see the following output

  PAGESIZE  AVAILABLE_PAGES  EXPECTED_PAGES  ALLOCATED_PAGES  ERROR(s)
2022-08-11T12:57:12.094226+02:00
        4K       Configured               0               0        NONE
2022-08-11T12:57:12.096517+02:00
       64K       Configured             118             118        NONE  --> PGA
2022-08-11T12:57:12.099008+02:00
       16M       Configured             192             192        NONE  --> SGA
2022-08-11T12:57:12.101851+02:00
       16G       Configured               0               0        NONE
2022-08-11T12:57:12.104649+02:00

So far so good. In the alert.log for the database i can see, that oracle has allocated 192 Large Pages for the SGA. To be really sure that everything is fine we check svmon again

svmon

PageSize   PoolSize       inuse        pgsp         pin     virtual
s    4 KB         -      812827       29012      228798      252287
m   64 KB         -       61455         202       14189       61743
L   16 MB       200           0           0         200           0
S   16 GB         -           0           0           0           0

Mhm… wait a moment. Oracle tell us, that it has allocated 192 16M Pages but the AIX tells us that all 200 Pages are still not in use. So who is right? Let’s do another check. We can change the Parameter USE_LARGE_PAGES from TRUE to ONLY. That means if oracle is not able to allocate large pages during startup the instance will fail. So we change the Parameter and reboot the database. If it comes up large pages were used and oracle is right and if we get an error during startup we know oracle can’t allocate some Large Pages and the svmon was right. (That was my oppinion, which was wrong 🙁 )

SQL> alter system set use_large_pages=only scope=spfile;
SQL> startup force;

The database came up without issues, and the alert.log tells me that 192 Pages were allocated. So it must be good, because i set use_large_pages=only, the database can only start with large pages. Right? No unfortunately not. After some resarch on oracle support i finally found out, that this parameter is useless on AIX. The tricky part ist, that oracle tells you, that it has allocated the pages but that is not true. To allocate the 16M Pages for the database you have to set the following init.ora parameter

SQL> alter system set lock_sga=true;
SQL> startup force

After setting the lock_sga parameter to true and restart the database you can see, that now also svmon shows that the pages are in use

svmon
 
PageSize   PoolSize       inuse        pgsp         pin     virtual
s    4 KB         -      812827       29012      228798      252287
m   64 KB         -       61455         202       14189       61743
L   16 MB       200         193           0         200         193
S   16 GB         -           0           0           0           0