Do you prefer on-premises or public clouds for your database infrastructure? Or is your company in transit between both? Who really knows what’s the best solution? Is it a mix? With security and sovereignty being hot topics today, decisions are even more difficult to take.

For its database, Oracle has multiple offers. On-premises for sure (mostly Oracle Database Appliance and Exadata), public cloud for sure (Oracle Cloud Infrastructure or Oracle Database Service in the other main cloud providers). There is also an intermediate solution since several years now: Exadata Cloud@Customer. This is basically an Exadata delivered in your datacenter and paid as a subscription, with hardware managed by Oracle.

The problem of this solution is that Exadata platform is too expensive for most of us. Something less ambitious was required for answering most of Oracle database needs. This is why a new offering has been recently presented by Oracle: Oracle Base Database Cloud@Customer. Here are my initial thoughts about this new offering.

Key points of this new offer

Key points are quite similar to Exadata Cloud@Customer, with some differences:

• Oracle provides the pieces of hardware: you don’t buy them
• Oracle manages hardware and the underlying software (firmware, BIOS, hypervisor OS)
• OCI console (web based) is used to provision your servers (VM clusters) and your databases
• It can run Enterprise Edition (EE) databases like Exadata Cloud@Customer, but also Standard Edition 2 (SE2) databases
• You pay a fixed monthly price for hardware plus average CPU usage
• Unlike Exadata Cloud@Customer, you may also create application VMs with CPUs not used for databases
• CPU cost for application VMs is lower than CPU cost for databases
• You can Bring Your Own Oracle Database Licenses (BYOL) or pay them by usage (billed with the subscription)
• You can configure High-Availability with Real Application Cluster
• You can automate the lifecycle of your environment using APIs (ansible, DevOps, …)
• Root access to the database VMs is allowed, meaning that you can setup additional components
• You can configure a Disaster Recovery with a Data Guard configuration starting from 2x Oracle Base Database Cloud@Customer

Technical specs

Now, let’s have a deeper look at the technical specifications.

This solution is composed of 3 components:

• 2 identical servers with each 2x 32-core AMD Epyc processors and 768GB RAM (2x 2U)
• 1 storage shelf (SAS) with 6, 12, 18 or 24 SSD disks of 7.68TB (1x 4U)

The servers runs on Oracle Linux and KVM.

On top of this system, you will provision VMs within these limits:

• 120 usable cores (60 usable cores on each node)
• 1320GB of RAM (660GB on each node)
• 11.6, 23.4, 35.3 or 47.2TB of usable storage depending on your configuration

A 2-core VM will be initially configured by Oracle for OCI remote access on each server. This VM is called the Control Plane Proxy Service (CPS). A physical 2-port Ethernet interface on each node is associated to this VM for physical isolation.

Embedded licenses model

You can keep your existing licenses for running your database workload on this platform (BYOL), or you can choose to go for embedded licenses.

If you choose embedded licenses, you will have 4 possibilities when you create an Oracle database VM:

• Standard Edition: basically the same features as SE2 + Transparent Database Encryption
• Enterprise Edition: classic EE features + Real Application Testing + Diagnostic and Tuning Pack
• Enterprise Edition High Performance : same as previous one + Partitioning + Advanced Compression + Database Vault + Multitenant
• Enterprise Edition Extreme Performance : same as previous one + Active Data Guard + RAC + In Memory

High Performance and Extreme Performance don’t really bring more performance, but they pack more options. Note than you cannot add a single option. For example, if you need more than 3 PDBs on your container database, you must go for Enterprise Edition High Performance for the Multitenant option.

With the embedded licenses, you can optimize the cost of your licenses:

• Mix various editions on your Base Database Cloud@Customer, according to your needs
• License cost nothing when the VM is down
• You can increase the number of CPUs for some VMs during specific timeframes in the year

Would it be a better choice to keep your existing licenses? I would answer with another question: if you don’t own the servers, why would you keep your license ownership?

Is it an ODA Cloud@Customer?

Oracle Database Appliance (ODA) is a cheaper solution compared to Exadata. But until now, it didn’t exist as a Cloud@Customer solution. Is Base Database Cloud@Customer a kind of ODA Cloud@Customer?

Yes, and no.

Yes because technically, this solution is based on an ODA X11-HA with slightly more RAM (768GB on each node) with a single storage shelf (SSD-only – high performance version). This shelf is populated with 6, 12, 18 or 24 SSDs according to your needs. It will mostly cover the same features as the HA ODA, creating DB Systems with 19c and/or 26ai databases, running SE2 or EE databases, creating application VMs…

No because Oracle didn’t choose ODA Cloud@Customer naming for obvious reasons. First, odacli commands don’t exist for managing your environment. Since 2016, odacli is the central CLI used for managing mostly bare-metal ODAs. Second reason is that you won’t be able to configure bare metal databases on this new solution: you can’t connect to the OS of the nodes. These nodes are under Oracle responsibility. Last reason for not calling it an ODA: it’s a service based on classic servers. ODA is also based on classic Oracle servers, with no dedicated technologies for databases.

What is the benefit compared to ODA?

ODA became expensive, especially this year. Base price is 37’655$ for the small one, according to the latest price list from June. RAM and disk price hikes made some of our customers not able to afford an ODA infrastructure anymore. Renewing a database infrastructure without buying servers may help.

Another trend is less complexity. Some experienced Oracle DBAs are gently approaching retirement, some others have much more database technologies to handle and they cannot be Oracle experts anymore. Less complexity regarding Oracle database is welcomed, and managing database lifecycle through OCI console is rather easy.

If buying servers doesn’t make any sense for your company, and if you don’t have enough skills to deploy, patch and manage an ODA, this new solution should be considered.

How it compares to Exadata Cloud@Customer?

What makes Exadata so different than any other solution is the smart storage. Instead of having direct-attached disks or a SAN with its own intelligence, Exadata has several dedicated servers with disks, flash cards and database intelligence. These servers are called the storage cells. These cells will process some SQL statements and do the filtering before sending the blocks to the compute nodes. This is what’s called cell offloading. An old legend tells that you don’t need indexes for a database running on Exadata… For sure, with some optimizations you can probably expect the highest performance on the market.

Base Exadata is much more complex than ODA as it includes 2 computes nodes, 3 storage cells and 2 RoCE fabrics (basically 40Gbps switches for high-speed connection between the compute nodes and the cells). And you can scale up for more compute nodes (maximum of 8) and more storage cells (maximum of 12): you cannot do that on an ODA.

You can replace Exadata with Exadata Cloud@Customer and ODA with Base Database Cloud@Customer in these previous sentences.

Some features won’t be available on Base Database Cloud@Customer like Autonomous database.

On the other hand, Exadata Cloud@Customer cannot run SE2 databases and application VMs.

A colleague of mine told me that unlike Exadata Cloud@Customer, Base Database Cloud@Customer won’t come with a complete rack: setup will be done in an existing rack.

What about the cost?

Pricing is publicly available here: https://www.oracle.com/database/base-database-service/pricing/

Like an Exadata Cloud@Customer, you won’t buy this machine. But you will pay $8,000 once for the initial setup.

This system has a base configuration with 6 disks, and you can buy (lease) up to 3 storage expansions of 6 disks. Each storage expansion costs $1,900.00 each month.

Here is a summary of monthly hardware cost:

You’ll need to add usage cost, based on ECPU per hour. On this platform, ECPU equals to 1/4 of a core. You can Bring Your Own License, or pay the license by usage:

Example: if you need a Base Database Cloud@Customer with 12 disks and an average of 32 ECPUs with embedded EE licenses, the overall cost for one year is:
10,021 * 12 + 32 * 0.1075 * 24 * 365 = $150’386 without the initial setup cost.

Note that you should consider 2 machines because you probably need a Disaster Recovery setup. I would recommend 2 identical machines.

Advantages and limits of this solution

These are the advantages I see:

• you don’t buy the servers, they belong to Oracle and you just pay for using them: this is better for a lot of companies
• you will benefit from TDE, Transparent Database Encryption: TDE is normally an option on top of Enterprise Edition only
• you won’t manage the servers yourself: you will only manage the VMs and the databases more easily
• opting for embedded licenses brings flexibility: increase the CPU when you need to, stop your test and development databases when they are not in use…

On the other hand, these are the limits I see:

• the cost is still too expensive for those running a small number of databases on small ODAs like X11-S and X11-L
• the limit of 32 ECPUs for Standard Edition VM (8 cores): this limit does not exist on ODA X11 series
• the storage, 44TB being not that big
• database version: don’t expect to deploy database versions older than 1-year old
• choosing embedded licenses means stopping paying Oracle support for your perpetual licenses: you’ll need to buy them again later if you want to go back to on-premises or IaaS solutions

Is this the best solution for your needs?

For now, this solution is new and no one at dbi services gets his hands on it. It could come in the coming weeks or months.

To find out if this solution could fit your needs, you must have a deep knowledge of your current infrastructure and an overview on your future needs. I strongly advise starting with a complete infrastructure review. Correct sizing for whatever solution you may consider has to be done with accuracy. Especially for Cloud or Hybrid Cloud solutions that requires surgical precision!

If you cannot do this kind of work yourself, this is something we’re used to do at dbi services since many years.

Conclusion

This new offering brings new possibilities for those who prefer the subscription model (OPEX) without going to the public cloud. But it has some limits. Main limit is that it’s not affordable enough for companies used to lower cost solutions like ODA X11-S and ODA X11-L. But this is definitely an in-between: you want a modern platform without the cost of Exadata Cloud@Customer? That might be Base Database Cloud@Customer. My thought is that performance could be quite similar to Exadata Cloud@Customer for most databases.

L’article Oracle Base Database Cloud@Customer est apparu en premier sur dbi Blog.

Open source is not free! The licence is, but the hours your team spends making it work are not.

At dbi services we love open source. We run PostgreSQL, Kafka and Kubernetes for many customers and we contribute where we can. But over the last years I see the same story again and again: a company picks an open source tool because there is no licence to pay, and then spends months building, tuning and debugging something that was never designed for their use case.

On the budget slide it looks great. No CAPEX. But nobody puts the OPEX on that slide, and that is where the real bill is!

A typical case: Debezium between two database engines

The requirement sounds simple. Replicate data from Oracle to PostgreSQL, or from SQL Server to something else, with low latency.

Someone finds Debezium. It is a great project for change data capture into Kafka. So the team starts, and quickly the architecture looks like this:

• 1. Debezium with a source connector reading the redo logs or the transaction log
• 2. A Kafka cluster, plus Kafka Connect
• 3. A schema registry, because the data types have to survive the trip
• 4. A sink connector, often with custom code, to write into the target engine
• Monitoring for all of the above

For a use case that is “copy these tables from A to B”, that is five new components to install, secure, patch and understand. And then the real fun starts: data type mapping between engines, DDL changes on the source, LOB columns, initial loads of big tables, connector restarts that replay or lose events. Every one of these is solvable. Every one of these costs days.

Where the money really goes

The cost is not in one big invoice. It is spread over many small things that nobody adds up.

• Build time. Weeks or months of senior engineers wiring components together instead of delivering business value.
• Troubleshooting. When replication lags at 2am, who knows Kafka Connect internals well enough to fix it? Usually one person.
• Knowledge silo. That one person becomes the system. When they go on holiday or leave the company, you have a production pipeline nobody understands.
• Upgrades. Five components means five release cycles, five sets of breaking changes, five compatibility matrices to check.
• Support. Community forums are great, but they do not have an SLA. Commercial support for the stack exists, and then it is not free anymore either.
• Hidden cost. Every hour spent on plumbing is an hour not spent on the project the replication was supposed to enable..

Let’s do the maths over three years

Over three years, the “free” option can easily cost more than the tool with a licence. Here is a simple example, with an internal cost of CHF 1’000 per engineer day. The numbers are illustrative, but they are close to what we see in real projects.

And this table does not even include the risk of the key person leaving!

This is not an anti open source post

The problem is not open source. The problem is choosing a tool for the wrong reason.

Debezium is the right choice when you already run Kafka, you have people who know it, and you need events flowing to many consumers. It is also the right choice when the goal is not just syncing two databases, but turning raw data changes into meaningful business events (order placed, inventory updated, customer status changed, invoice issued) that multiple downstream systems can react to independently. In that context it is excellent. It is the wrong choice when the only need is to keep two databases in sync and nobody in the team has ever operated Kafka.

The same applies to many other tools. The question is never “open source or not”. The question is “what will this cost us to run, every day, for the next three years?”

L’article The real OPEX cost of the wrong open source choice est apparu en premier sur dbi Blog.

Why global temporary table DDL is excluded

The data of a global temporary table is private to each session and is not replicated. The extract therefore skips the DDL that creates and drops it, and the table does not exist on the target. However, TRUNCATE operations are captured by GoldenGate.

Setup

• GoldenGate 23.26.4.0.0 Microservices Architecture, deployment ogg_test_01.
• Source: CDB01/PDB1, Oracle 19c. Target: CDB02/PDB2, Oracle 19c.
• A GTTTEST schema exists in both PDBs.
• Integrated extract EGTT connects to PDB1 directly (PDB-level capture), with DDL INCLUDE ALL.
• Integrated replicat RGTT applies to PDB2, with DDL INCLUDE ALL.

The parameter file of the extract:

EXTRACT EGTT
EXTTRAIL gt
USERIDALIAS source_pdb1 DOMAIN OracleGoldenGate
DDL INCLUDE ALL
DDLOPTIONS REPORT
TABLE GTTTEST.*;

The parameter file of the replicat:

REPLICAT RGTT
USERIDALIAS oggadmin_pdb2 DOMAIN OracleGoldenGate
DDL INCLUDE ALL
DDLOPTIONS REPORT
MAP GTTTEST.*, TARGET GTTTEST.*;

The extract is registered with REGISTER EXTRACT EGTT DATABASE, and the replicat uses the checkpoint table OGGADMIN.CHECKPOINT.

Workload on the source

The three statements run as GTTTEST on PDB1:

SQL> CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS;

Table created.

SQL> TRUNCATE TABLE temp_test;

Table truncated.

SQL> DROP TABLE temp_test PURGE;

Table dropped.

What the extract does

The extract report shows the three DDL statements:

• CREATE and DROP are excluded.
• TRUNCATE is included.

2026-10-24 07:26:05  INFO    OGG-01487  DDL found, operation [CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS (size 74)], ... ].
2026-10-24 07:26:05  INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
2026-10-24 07:26:05  INFO    OGG-01487  DDL found, operation [TRUNCATE TABLE temp_test (size 24)], ... ].
2026-10-24 07:26:05  INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
2026-10-24 07:26:05  INFO    OGG-01487  DDL found, operation [DROP TABLE temp_test PURGE (size 26)], ... ].
2026-10-24 07:26:05  INFO    OGG-10473  DDL on global temporary table excluded, optype [DROP], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".

Only the TRUNCATE reaches the trail.

What the replicat does

The replicat maps the TRUNCATE to GTTTEST.TEMP_TEST and executes it on PDB2. The table does not exist there, and the replicat abends:

2026-10-24 07:26:08  INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
2026-10-24 07:26:08  INFO    OGG-00489  DDL is of mapped scope, after mapping new operation [TRUNCATE TABLE "GTTTEST"."TEMP_TEST" (size 36)].
2026-10-24 07:26:08  INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
2026-10-24 07:26:08  INFO    OGG-00484  Executing DDL operation.
2026-10-24 07:26:08  ERROR   OGG-00519  Fatal error executing DDL replication: error [Error code [942], ORA-00942: table or view does not exist
2026-10-24 07:26:08  ERROR   OGG-30538  Fatal error executing DDL statement [TRUNCATE TABLE "GTTTEST"."TEMP_TEST"].
2026-10-24 07:26:08  ERROR   OGG-00664  Error 942 on OCI call [ORA] - ORA-00942: table or view does not exist
2026-10-24 07:26:13  ERROR   OGG-01668  PROCESS ABENDING.

The status in the adminclient:

OGG (http://localhost:7810 ogg_test_01) 2> INFO ALL

Program     Status      Group       Type                   Lag at Chkpt       Time Since Chkpt
EXTRACT     RUNNING     EGTT        INTEGRATED                  00:00:03           00:00:04
REPLICAT    ABENDED     RGTT        INTEGRATED                  00:00:04           00:00:13

DDLOPTIONS for global temporary tables

The DDLOPTIONS documentation describes two options for global temporary tables. By default, GTT DDL is not visible to the extract, which does not match the TRUNCATE captured above.

• CAPTUREGLOBALTEMPTABLE (extract): makes GTT DDL visible to the extract. It is filtered like any other DDL and, if it passes, written to the trail.
• NOAPPLYGLOBALTEMPTABLE (replicat): prevents the replicat from applying GTT DDL. Without it, the replicat applies all GTT DDL from the trail.

Every case uses the same workload. It has the three GTT statements, then a regular table with one row going through CREATE, TRUNCATE and DROP. Each case starts from a new extract, replicat and trail.

NOAPPLYGLOBALTEMPTABLE on the replicat

The parameter file of the replicat:

REPLICAT RGTT
USERIDALIAS oggadmin_pdb2 DOMAIN OracleGoldenGate
DDL INCLUDE ALL
DDLOPTIONS REPORT, NOAPPLYGLOBALTEMPTABLE
MAP GTTTEST.*, TARGET GTTTEST.*;

The replicat still abends on the TRUNCATE, with the same errors as without the option:

INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
ERROR   OGG-00519  Fatal error executing DDL replication: error [Error code [942], ORA-00942: table or view does not exist
ERROR   OGG-01668  PROCESS ABENDING.

If the GTT already exists on the target, the replicat does not abend, because the TRUNCATE is applied to that table.

CAPTUREGLOBALTEMPTABLE on the extract

With DDLOPTIONS REPORT, CAPTUREGLOBALTEMPTABLE in the extract, the CREATE and DROP are no longer excluded:

OGG-00487  DDL operation included [INCLUDE ALL], optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
OGG-00487  DDL operation included [INCLUDE ALL], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
OGG-00487  DDL operation included [INCLUDE ALL], optype [DROP], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".

The replicat creates the table on the target, truncates it and drops it. It keeps running:

INFO    OGG-00482  DDL found, operation [CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS (size 74)].
INFO    OGG-00489  DDL is of mapped scope, after mapping new operation [CREATE GLOBAL TEMPORARY TABLE "GTTTEST"."TEMP_TEST" (c number) ON COMMIT PRESERVE ROWS (size 86)].
INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
INFO    OGG-00482  DDL found, operation [DROP TABLE temp_test PURGE (size 26)].

Both options together

With CAPTUREGLOBALTEMPTABLE on the extract and NOAPPLYGLOBALTEMPTABLE on the replicat, the replicat excludes the CREATE and still executes the TRUNCATE:

INFO    OGG-00482  DDL found, operation [CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS (size 74)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
ERROR   OGG-00519  Fatal error executing DDL replication: error [Error code [942], ORA-00942: table or view does not exist
ERROR   OGG-01668  PROCESS ABENDING.

The replicat abends on the TRUNCATE before it reaches the DROP. To check the DROP, the same two options run with a workload that has no TRUNCATE on the global temporary table. The replicat excludes both the CREATE and the DROP and keeps running:

INFO    OGG-00482  DDL found, operation [CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS (size 74)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00482  DDL found, operation [DROP TABLE temp_test PURGE (size 26)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [DROP], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".

So NOAPPLYGLOBALTEMPTABLE skips the CREATE and DROP of a global temporary table, but not its TRUNCATE.

Which DDL statements are affected

The TRUNCATE is not the only statement that behaves differently from CREATE and DROP. This workload runs on the source with the default extract (no CAPTUREGLOBALTEMPTABLE):

CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS;
ALTER TABLE temp_test ADD (d number);
COMMENT ON TABLE temp_test IS 'gtt';
CREATE INDEX temp_test_i ON temp_test(c);
TRUNCATE TABLE temp_test;
ALTER TABLE temp_test RENAME TO temp_test2;
DROP TABLE temp_test2 PURGE;

The extract excludes five of the seven statements and includes COMMENT and TRUNCATE:

INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-10473  DDL on global temporary table excluded, optype [ALTER], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [COMMENT], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [INDEX], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-10473  DDL on global temporary table excluded, optype [ALTER], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST2".
INFO    OGG-10473  DDL on global temporary table excluded, optype [DROP], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST2".

With CAPTUREGLOBALTEMPTABLE on the extract and NOAPPLYGLOBALTEMPTABLE on the replicat, the extract captures every statement and the replicat makes the same decision. It excludes CREATE and ALTER (OGG-10473), then applies the COMMENT and abends:

INFO    OGG-00482  DDL found, operation [ALTER TABLE temp_test ADD (d number) (size 36)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [ALTER], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00482  DDL found, operation [COMMENT ON TABLE temp_test IS 'gtt' (size 35)].
INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [COMMENT], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
ERROR   OGG-00519  Fatal error executing DDL replication: error [Error code [942], ORA-00942: table or view does not exist
ERROR   OGG-30538  Fatal error executing DDL statement [COMMENT ON TABLE "GTTTEST"."TEMP_TEST" IS 'gtt'].
ERROR   OGG-01668  PROCESS ABENDING.

Each statement runs separately to see what the replicat does with it:

Scope and truncate options do not change it

A difference in DDL scope could explain why CREATE is excluded and TRUNCATE is not. However, with DDL INCLUDE MAPPED on the replicat, the TRUNCATE is still included and the replicat abends:

INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
INFO    OGG-00487  DDL operation included [INCLUDE MAPPED], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
ERROR   OGG-00519  Fatal error executing DDL replication: error [Error code [942], ORA-00942: table or view does not exist
ERROR   OGG-30538  Fatal error executing DDL statement [TRUNCATE TABLE "GTTTEST"."TEMP_TEST"].
ERROR   OGG-01668  PROCESS ABENDING.

The same happens with GETTRUNCATES and with IGNORETRUNCATES in the replicat, and with a global temporary table created with ON COMMIT DELETE ROWS. Those parameters control the truncate operation of the data, not the TRUNCATE DDL statement.

Answer from Oracle Support

I opened a service request with Oracle. The Product Team answered that TRUNCATE TABLE is a DDL operation and has to be excluded with an EXCLUDE clause of the DDL parameter. DDL cannot be filtered by temporary versus permanent tables, so the exclusion is done by table name. If that is too much work, the alternative is DDLERROR 942 DISCARD. I then pointed out that CREATE and DROP are DDL operations too and are excluded. The answer was that this is how it is designed for temporary tables.

Keeping the replicat running

Exclude the TRUNCATE by table name

The DDL parameter of the replicat excludes the TRUNCATE of the table:

REPLICAT RGTT
USERIDALIAS oggadmin_pdb2 DOMAIN OracleGoldenGate
DDL INCLUDE ALL, EXCLUDE OBJNAME GTTTEST.TEMP_TEST OPTYPE TRUNCATE
DDLOPTIONS REPORT, NOAPPLYGLOBALTEMPTABLE
MAP GTTTEST.*, TARGET GTTTEST.*;

The replicat finds the TRUNCATE and skips it with an informational message, OGG-00488. It keeps running:

INFO    OGG-00482  DDL found, operation [CREATE GLOBAL TEMPORARY TABLE temp_test (c number) ON COMMIT PRESERVE ROWS (size 74)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [CREATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
INFO    OGG-00489  DDL is of mapped scope, after mapping new operation [TRUNCATE TABLE "GTTTEST"."TEMP_TEST" (size 36)].
INFO    OGG-00488  DDL operation excluded [EXCLUDE OBJNAME GTTTEST.TEMP_TEST OPTYPE TRUNCATE], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00482  DDL found, operation [DROP TABLE temp_test PURGE (size 26)].
INFO    OGG-10473  DDL on global temporary table excluded, optype [DROP], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".

The exclusion works per table name, so it has to list every global temporary table. COMMENT ON TABLE and COMMENT ON COLUMN abend the same way, so the exclusion must cover them too.

Discard the ORA-00942 error

DDLERROR 942 DISCARD needs no table list. The replicat applies the TRUNCATE, receives ORA-00942 from the target and discards the error:

REPLICAT RGTT
USERIDALIAS oggadmin_pdb2 DOMAIN OracleGoldenGate
DISCARDFILE RGTT.dsc, APPEND
DDL INCLUDE ALL
DDLOPTIONS REPORT
DDLERROR 942 DISCARD
MAP GTTTEST.*, TARGET GTTTEST.*;

The report shows OGG-00494 instead of the abend:

INFO    OGG-00482  DDL found, operation [TRUNCATE TABLE temp_test (size 24)].
INFO    OGG-00487  DDL operation included [INCLUDE ALL], optype [TRUNCATE], objtype [TABLE], objowner "GTTTEST", objname "TEMP_TEST".
INFO    OGG-00494  DDL error discarded: error code [942], filter [include all (default)], error text [Current time: 2026-10-24 07:55:57
 Error text [Error code [942], ORA-00942: table or view does not exist

OGG (http://localhost:7810 ogg_test_01) 2> INFO ALL

Program     Status      Group       Type                   Lag at Chkpt       Time Since Chkpt
EXTRACT     RUNNING     EGTT        INTEGRATED                  00:00:03           00:00:07
REPLICAT    RUNNING     RGTT        INTEGRATED                  00:00:00           00:00:06

The discard file RGTT.dsc has the statement:

Current time: 2026-10-24 07:55:57
 Error text [Error code [942], ORA-00942: table or view does not exist
Help: https://docs.oracle.com/error-help/db/ora-00942/
], operation [TRUNCATE TABLE "GTTTEST"."TEMP_TEST" (size 36)]
Operation failed at seqno 0 rba 1,333.

The parameter applies to every DDL statement that fails with ORA-00942, not only to temporary tables. A statement that fails because a regular table is missing on the target is discarded too, so check the discard file regularly.

Summary

• With DDL INCLUDE ALL, the extract excludes CREATE, ALTER, CREATE INDEX and DROP of a global temporary table (OGG-10473) but includes TRUNCATE and COMMENT (OGG-00487).
• The TRUNCATE is written to the trail and applied by the replicat on a target that has no such table.
• The replicat abends with ORA-00942 (OGG-00519, OGG-30538, OGG-01668).
• DDL INCLUDE MAPPED, GETTRUNCATES and IGNORETRUNCATES do not change the abend.
• NOAPPLYGLOBALTEMPTABLE on the replicat does not prevent it, because the TRUNCATE is still executed.
• CAPTUREGLOBALTEMPTABLE on the extract makes the whole CREATE, TRUNCATE and DROP sequence replicate without error.
• Oracle support confirmed that this is the designed behavior for temporary tables.
• DDL EXCLUDE OBJNAME <table> OPTYPE TRUNCATE on the replicat skips the TRUNCATE for the listed tables.
• DDLERROR 942 DISCARD on the replicat keeps it running for all tables and writes the failed statement to the discard file.

L’article GoldenGate captures TRUNCATE on a global temporary table and the replicat abends est apparu en premier sur dbi Blog.

It is time now to test the MCP server integrated with OEM 24ai Release 1 Update 12. In this blog, I will show you how I configured a client to connect and interact with OEM MCP server in order to get information from the OMR, and perform first short tests.

Requirements

I need to be at least in OEM 24ai Release 1 Update 12.

[oracle@oem24ai ~]$ $OMS_HOME/bin/emctl getversion oms
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
       Oracle Enterprise Manager OMS 24ai Release 1

[oracle@oem24ai ~]$ export ORACLE_HOME=$OMS_HOME

[oracle@oem24ai ~]$ $ORACLE_HOME/OPatch/opatch lspatches
39984317;EM Stack Patch Bundle 24.1.0.0.0(ID:260906.1632) (Patch 39984254)
39979941;OPSS Bundle Patch 12.2.1.4.260904
39930279;ADF BUNDLE PATCH 12.2.1.4.260825
39919567;WLS PATCH SET UPDATE 12.2.1.4.260823
39901894;WebCenter Core Bundle Patch 12.2.1.4.260818
39896168;OWSM BUNDLE PATCH 12.2.1.4.260818
39861406;PERL 5.42.2 patch for EM 24.1.0.0.0
39798204;RDA release 26.3-2026915 for OFM 12.2.1.4 SPB
39611663;OHS (NATIVE) DB19C BUNDLE PATCH 12.2.1.4.260624
39574504;FMW Thirdparty Bundle Patch 12.2.1.4.260617
39567777;OSS 19C BUNDLE PATCH 12.2.1.4.260616
39487427;JDBC19.31 BUNDLE PATCH 12.2.1.4.260603
39425385;DB CLIENT 19.31 FOR FMW BUNDLE PATCH 12.2.1.4.260721 (39425385)
38832179;One-off
38098858;One-off
1221432;Coherence Cumulative Patch 12.2.1.4.32
39808046;Oracle Enterprise Manager for Cloud Framework 24ai Release 1 Plug-in Update 13 (24.1.1.13) for Oracle Management Service
39808030;Oracle Enterprise Manager for Systems Infrastructure 24ai Release 1 Plug-in Update 13 (24.1.1.13) for Oracle Management Service
39808017;Oracle Enterprise Manager for Exadata 24ai Release 1 Plug-in Update 13 (24.1.1.13) for Oracle Management Service
39807976;Oracle Enterprise Manager for Oracle Database 24ai Release 1 Plug-in Update 13 (24.1.1.13) for Oracle Management Service
39807963;Oracle Enterprise Manager for Fusion Middleware 24ai Release 1 Plug-in Update 13 (24.1.1.13) for Oracle Management Service
39807951;Oracle Enterprise Manager 24ai Release 1 Platform Update 13 (24.1.0.13) for Oracle Management Service
36789759;FMW PLATFORM BUNDLE PATCH 12.2.1.4.240812
34065178;One-off
30152128;One-off
34809489;One-off
35965629;ADR FOR WEBLOGIC SERVER 12.2.1.4.0 CPU JAN 2024

OPatch succeeded.
[oracle@oem24ai ~]$

This is ok as I patched my OEM before performing further tests, and I’m now running Update 13.

Connectivity tests without any client

TCP connectivity

The address to reach the MCP server from my lab is the following:

https://oem24ai.jew.lab:7803/em/api/mcp

Before going further in configuring any client, I made a local connection test to the MCP server from my Mac using curl command. Option v for verbose and k to disable TLS certificate verification.

The command confirmed the connection is ok (DNS resolution, TCP connectivity). HTTP Error 401 (Unauthorized) is ok as we did not provide any user and password, so OEM rejected the unauthorized request.

maw@dbi-servicess-MacBook-Pro-2 ~ % curl -vk \
> https://oem24ai.jew.lab:7803/em/api/mcp
* Host oem24ai.jew.lab:7803 was resolved.
* IPv6: (none)
* IPv4: 10.0.10.250
*   Trying 10.0.10.250:7803...
* Connected to oem24ai.jew.lab (10.0.10.250) port 7803
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / [blank] / UNDEF
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=oem24ai.iaas.jewlab.oraclevcn.com
*  start date: Oct 13 13:55:47 2026 GMT
*  expire date: Oct 11 13:55:47 2036 GMT
*  issuer: O=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com; OU=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com; L=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com; ST=CA; C=US; CN=oem24ai.iaas.jewlab.oraclevcn.com
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
* using HTTP/1.x
> GET /em/api/mcp HTTP/1.1
> Host: oem24ai.jew.lab:7803
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Date: Mon, 26 Oct 2026 13:07:26 GMT
< Date: Mon, 26 Oct 2026 13:07:26 GMT
< X-ORACLE-DMS-ECID: d5e13a8c-fc9e-4654-8dd2-ecd46f90471c-000004af
< X-ORACLE-DMS-RID: 0
< X-ORCL-EM-APIGW-ERR: 0
< X-ORCL-EM-APIGW-GUID: e525b50c-f555-1c15-b8ca-a13fe481ea20
< X-ORCL-EMOA: true
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
<
* Connection #0 to host oem24ai.jew.lab left intact
maw@dbi-servicess-MacBook-Pro-2 ~ %

MCP connection

Let’s test MCP connection with user and password using curl command. We will use following options:

• -s for silent execution
• -S to show errors
• -k to skip TLS certificate verification
• -u to provide user/password
• -H to provide the header
• -X to give the HTTP method
• -d to display the data to be sent in the HTTP request body in json format

The MCP method specified in the JSON request body is initialize. We use this method to establish an initial connection with the MCP server and negotiate protocol capabilities.

First securely read the password into a shell variable.

[oracle@oem24ai ~]$ read -rs -p "OEM password: " OEM_PASS
OEM password: [oracle@oem24ai ~]$

Then run the curl command to test the MCP connection.

[oracle@oem24ai ~]$ curl -sS -k -u "sysman:${OEM_PASS}" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -X POST "https://oem24ai.jew.lab:7803/em/api/mcp" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2025-06-18",
      "capabilities": {},
      "clientInfo": {
        "name": "oem-mcp-test",
        "version": "1.0"
      }
    }
  }'
{"id":1,"jsonrpc":"2.0","result":{"protocolVersion":"2025-06-18","serverInfo":{"version":"1.0.0","name":"em-mcp-server","title":"Enterprise Manager MCP Server"},"capabilities":{"prompts":{"listChanged":false},"resources":{"listChanged":false,"subscribe":false},"tools":{"listChanged":false}},"instructions":"This is an MCP server to interface with Enterprise Manager (EM) Unless otherwise specified, assume that databases and database targets refers to the following target types: oracle_database, oracle_pdb, rac_database, oracle_cloud_atp, oracle_cloud_adw."}}[oracle@oem24ai ~]$

As we can see in the result, the connection to the MCP server was successful.

Query MCP tool lists

Let’s retrieve the lists of available tools exposed by the MCP server. This time, we will use the MCP method tools/list.

[oracle@oem24ai ~]$ curl -sS -k -u "sysman:$OEM_PASS" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "MCP-Protocol-Version: 2025-06-18" \
  -X POST "https://oem24ai.jew.lab:7803/em/api/mcp" \
  -d '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/list",
    "params": {}
  }'
{"id":2,"jsonrpc":"2.0","result":{"tools":[{"name":"ExecuteSql","title":"Execute SQL","description":"Execute SQL through the Free Query API. By default the query runs against the Enterprise Manager repository; include targetConnection to run against a target database. Use bind parameters instead of direct text substitution when applicable. The response contains query results, response time, and available result metadata.","inputSchema":{"properties":{"isSqlMonitorEnabled":{"description":"optional flag to enable SQL monitoring. Defaults to false","type":"boolean"},"excludeNullValues":{"description":"optional flag to omit null-valued properties from repository query results. Defaults to false","type":"boolean"},"resourcePlan":{"description":"optional resource plan action information for query tracking. Target database queries require action because it cannot be inferred from a query name","type":"object","properties":{"action":{"type":"string","description":"optional repository query action and required target database query action"}}},"query":{"description":"the sql query to execute","type":"string"},"bindType":{"description":"optional bind parameter mode. Defaults to INDEX. Use NAME when parameters are bound by alias","type":"string","enum":["INDEX","NAME"]},"transforms":{"description":"optional response transforms currently used for localized target type columns","type":"array","items":{"type":"object","properties":{"type":{"type":"string","description":"transform type"},"columnName":{"type":"string","description":"source column name"},"localizedColumnName":{"type":"string","description":"localized output column name"}}}},"limit":{"description":"optional positive integer row limit. Va
...

As we can see in the result we see the list of all available tools exposed by the MCP server, the first one been ExecuteSql.

This confirm our connection and use of the MCP server is ok.

Configure the MCP client

As MCP client I will use my Visual Studio Code installed on my Mac. To use it with the MCP server, I will install and use the Codex extension from OpenAI. I will sign in with my ChatGPT account.

OpenAI Codex is an AI-powered coding agent developed by OpenAI. It helps developers write, understand, debug, test, and modify code, often working directly with an existing software project.

Configure Codex extension

I will configure Codex with sysman password converted into a Base64-encoded string. For this I will use the base64 linux command:

maw@dbi-servicess-MacBook-Pro-2 ~ % printf '%s' 'sysman:XXXXXXX' | base64 | tr -d '\n'
c3lzbXXXXXXXXXXXXXXyMDI2%                                                                                                                                            maw@dbi-servicess-MacBook-Pro-2 ~ %

And I will configure Codex extension, see next 2 pictures. I provided OEM-MCP-native as MCP server name.

This configuration will be stored in the config.toml file from the .codex directory.

The connection was not working. Asking for the list of tools exposed by the MCP server was just provided an output that no tool was existing. I could find the following in the codex output terminal:

026-10-26 15:03:53.265 [warning] Failed to discover app tools authStatus=bearerToken error="MCP startup failed: handshaking with MCP server failed: Send message error Transport [codex_rmcp_client::tool_list_changes::ToolListChanges<codex_rmcp_client::event_notification_transport::EventNotificationTransport<rmcp::transport::worker::WorkerTransport<rmcp::transport::streamable_http_client::StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>>] error: Client error: HTTP request failed: http/request failed: error sending request for url (https://oem24ai.jew.lab:7803/em/api/mcp), when send initialize request: Send message error Transport [codex_rmcp_client::tool_list_changes::ToolListChanges<codex_rmcp_client::event_notification_transport::EventNotificationTransport<rmcp::transport::worker::WorkerTransport<rmcp::transport::streamable_http_client::StreamableHttpClientWorker<codex_rmcp_client::http_client_adapter::StreamableHttpClientAdapter>>>>] error: Client error: HTTP request failed: http/request failed: error sending request for url (https://oem24ai.jew.lab:7803/em/api/mcp), when send initialize request" hostId=local pluginId=null server=OEM-MCP-native

I made some further check with curl command omitting -k option:

maw@dbi-servicess-MacBook-Pro-2 .codex % curl -v \
  https://oem24ai.jew.lab:7803/em/api/mcp
* Host oem24ai.jew.lab:7803 was resolved.
* IPv6: (none)
* IPv4: 10.0.10.250
*   Trying 10.0.10.250:7803...
* Connected to oem24ai.jew.lab (10.0.10.250) port 7803
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: self signed certificate in certificate chain
* Closing connection
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
maw@dbi-servicess-MacBook-Pro-2 .codex %

And I could confirm that the problem comes from missing TLS certificate.

I ran following command to see what certificate was sent by the MCP server and understand why certificate verification failed.

[oracle@oem24ai ~]$ openssl s_client \
  -connect oem24ai.jew.lab:7803 \
  -servername oem24ai.jew.lab \
  -showcerts &1 |
grep -E 'subject=|issuer=|verify error|Verify return code'
verify error:num=19:self-signed certificate in certificate chain
subject=CN=oem24ai.iaas.jewlab.oraclevcn.com
issuer=O=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com, OU=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com, L=EnterpriseManager on oem24ai.iaas.jewlab.oraclevcn.com, ST=CA, C=US, CN=oem24ai.iaas.jewlab.oraclevcn.com
    Verify return code: 19 (self-signed certificate in certificate chain)

The return code: 19 confirm that the certificate could not be considered as trusted.

The second command will display the DNS name available in the certificate, here we do not have any:

[oracle@oem24ai ~]$ openssl s_client \
  -connect oem24ai.jew.lab:7803 \
  -servername oem24ai.jew.lab 2>/dev/null |
openssl x509 -noout -ext subjectAltName
No extensions in certificate

Also we can see that the domain name registered in the MCP Server certificate is not the same that the one we use to connect: . jew.lab versus jewlab.oraclevcn.com. We will have to provide both names in the new certificate we will create.

Configure a new trusted certificate and implement it into OEM

To help with this following oracle documentation is useful:

https://docs.oracle.com/en/enterprise-manager/cloud-control/enterprise-manager-cloud-control/24.1/emadm/emctl-security-commands.html

https://docs.oracle.com/en/enterprise-manager/cloud-control/enterprise-manager-cloud-control/24.1/emsec/configuring-secure-communication.html

See parts: “Configuring Third Party Certificates”, then “Configuring a Third Party Certificate for HTTPS Console Users”.

Generate the private Certificate Authority (CA)

Let’s create the private key to be used to sign certificates (CA private key):

maw@dbi-servicess-MacBook-Pro-2 oem-pki % pwd
/Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl genrsa -out oem-root-ca.key 4096
Generating RSA private key, 4096 bit long modulus
.......++++
.............................................................................................................................................................................++++
e is 65537 (0x10001)

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 8
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I generated the CA certificate:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % pwd
/Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 8
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl req -new -x509 -sha256 \
  -days 3650 \
  -key oem-root-ca.key \
  -out oem-root-ca.crt \
  -subj "/CN=OEM Lab Root CA" \
  -addext "basicConstraints=critical,CA:TRUE" \
  -addext "keyUsage=critical,keyCertSign,cRLSign"

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 16
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I checked the certificate:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem-root-ca.crt \
  -noout -subject -issuer -dates
subject= /CN=OEM Lab Root CA
issuer= /CN=OEM Lab Root CA
notBefore=Oct 29 10:08:18 2026 GMT
notAfter=Oct 26 10:08:18 2036 GMT

I checked the basic constraints extension:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem-root-ca.crt -noout -text | \
  grep -A 3 "Basic Constraints"
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

All good, the certificate is marked as a Certificate Authority.

I verified the Root CA is self-signed:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl verify -CAfile oem-root-ca.crt oem-root-ca.crt
oem-root-ca.crt: OK
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

Generate the OMS server certificate with both DNS names

I generated the OMS private key and CSR:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % pwd
/Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 16
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl req -new -newkey rsa:3072 -nodes \
  -keyout oem24ai.key \
  -out oem24ai.csr \
  -subj "/CN=oem24ai.iaas.jewlab.oraclevcn.com" \
  -addext "subjectAltName=DNS:oem24ai.iaas.jewlab.oraclevcn.com,DNS:oem24ai.jew.lab"
Generating a 3072 bit RSA private key
.....................++++
........................++++
writing new private key to 'oem24ai.key'
-----

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 32
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   2.4K Oct 29 11:16 oem24ai.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.3K Oct 29 11:16 oem24ai.csr
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

We have now oem24ai.key which is OMS server private key, oem24ai.csr which is certificate signing request.

I created the certificate extensions (we need to ensure the signed certificate contains the correct SAN entries):

maw@dbi-servicess-MacBook-Pro-2 oem-pki % cat > oem24ai.ext <<'EOF'
basicConstraints=critical,CA:FALSE
keyUsage=critical,digitalSignature,keyEncipherment
extendedKeyUsage=serverAuth
subjectAltName=DNS:oem24ai.iaas.jewlab.oraclevcn.com,DNS:oem24ai.jew.lab
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid,issuer
EOF

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 40
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   2.4K Oct 29 11:16 oem24ai.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.3K Oct 29 11:16 oem24ai.csr
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   249B Oct 29 11:18 oem24ai.ext
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I signed the certificate with my private CA:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -req \
  -in oem24ai.csr \
  -CA oem-root-ca.crt \
  -CAkey oem-root-ca.key \
  -CAcreateserial \
  -out oem24ai.crt \
  -days 825 \
  -sha256 \
  -extfile oem24ai.ext
Signature ok
subject=/CN=oem24ai.iaas.jewlab.oraclevcn.com
Getting CA Private Key

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 56
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   2.4K Oct 29 11:16 oem24ai.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.3K Oct 29 11:16 oem24ai.csr
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   249B Oct 29 11:18 oem24ai.ext
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users    17B Oct 29 11:19 oem-root-ca.srl
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.8K Oct 29 11:19 oem24ai.crt
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

oem24ai.crt is now created and signed by my private CA.

oem-root-ca.key must never be copied and must stay locally on my Mac.

I checked the certificate subject, issuer, validity and SAN:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem24ai.crt -noout -subject -issuer -dates
subject= /CN=oem24ai.iaas.jewlab.oraclevcn.com
issuer= /CN=OEM Lab Root CA
notBefore=Oct 29 10:19:27 2026 GMT
notAfter=Jan 31 10:19:27 2029 GMT
maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem24ai.crt -noout -text | \
grep -A 1 "Subject Alternative Name"
            X509v3 Subject Alternative Name:
                DNS:oem24ai.iaas.jewlab.oraclevcn.com, DNS:oem24ai.jew.lab
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I can see that both my DNS names are now existing.

I verified the certificate chain:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl verify -CAfile oem-root-ca.crt oem24ai.crt
oem24ai.crt: OK
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I also verified if the certificate is suitable for HTTPS:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem24ai.crt -noout -text | \
grep -A 2 -E "Basic Constraints:|Key Usage:"
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication
            X509v3 Subject Alternative Name:
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

Create the Oracle Wallet for OEM 24ai

I have created all needed certificates for my OEM HTTPS console access. Following are available.

I transferred the needed certificate files to the OMS server:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % scp -i /Users/maw/Documents/Dokument/pem_ssh_key/OEM-lab/OEM-lab.key oem-root-ca.crt oem24ai.crt oem24ai.key oracle@oem24ai.jew.lab:/home/oracle/
oem-root-ca.crt                                                                                                                                                                                                                                                           100% 1744   121.2KB/s   00:00
oem24ai.crt                                                                                                                                                                                                                                                               100% 1822   127.0KB/s   00:00
oem24ai.key                                                                                                                                                                                                                                                               100% 2488   156.6KB/s   00:00
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

I created the needed directory for certificate and moved the files:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] pwd
/u01/app/oracle/oms24ai

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] mkdir -p oem-ssl/wallet

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] chmod -R 700 oem-ssl

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] mv ~/oem-root-ca.crt ~/oem24ai.crt ~/oem24ai.key oem-ssl/

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh oem-ssl
total 12K
-rw-r--r--. 1 oracle oinstall 1.8K Oct 29 10:33 oem-root-ca.crt
-rw-r--r--. 1 oracle oinstall 1.8K Oct 29 10:33 oem24ai.crt
-rw-r--r--. 1 oracle oinstall 2.5K Oct 29 10:33 oem24ai.key
drwx------. 2 oracle oinstall    6 Oct 29 10:39 wallet

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] chmod 600 oem-ssl/oem24ai.key
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I created the Oracle Wallet:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] export WALLET_DIR=/u01/app/oracle/oms24ai/oem-ssl/wallet

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet create \
  -wallet "$WALLET_DIR" \
  -auto_login
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

Enter password:
Enter password again:
Operation is successfully completed.
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I checked the created wallet files:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh "$WALLET_DIR"
total 8.0K
-rw-------. 1 oracle oinstall   0 Oct 29 10:46 ewallet.p12.lck
-rw-------. 1 oracle oinstall 235 Oct 29 10:46 ewallet.p12
-rw-------. 1 oracle oinstall   0 Oct 29 10:46 cwallet.sso.lck
-rw-------. 1 oracle oinstall 280 Oct 29 10:46 cwallet.sso
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I checked the private key format, should be private key:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] head -n 1 /u01/app/oracle/oms24ai/oem-ssl/oem24ai.key
-----BEGIN PRIVATE KEY-----
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

The private key format should be compatible with storing it into Oracle Wallet.

We can check that certificate and private keys match:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] openssl x509 -in /u01/app/oracle/oms24ai/oem-ssl/oem24ai.crt \
  -pubkey -noout | openssl dgst -sha256
SHA2-256(stdin)= 5a80b1977405XXXXf5b0cc39ab1ea

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] openssl pkey -in /u01/app/oracle/oms24ai/oem-ssl/oem24ai.key \
  -pubout | openssl dgst -sha256
SHA2-256(stdin)= 5a80b1977405XXXXf5b0cc39ab1ea
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

Both SHA-256 digests match.

Now I will import the signed OMS certificate and private key into the wallet. I will first setup variable for the directories and check wallet files:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] echo $WALLET_DIR
/u01/app/oracle/oms24ai/oem-ssl/wallet

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] export CERT_DIR=/u01/app/oracle/oms24ai/oem-ssl

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] echo $CERT_DIR
/u01/app/oracle/oms24ai/oem-ssl

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -l "$WALLET_DIR"/ewallet.p12 "$WALLET_DIR"/cwallet.sso
-rw-------. 1 oracle oinstall 280 Oct 29 10:46 /u01/app/oracle/oms24ai/oem-ssl/wallet/cwallet.sso
-rw-------. 1 oracle oinstall 235 Oct 29 10:46 /u01/app/oracle/oms24ai/oem-ssl/wallet/ewallet.p12
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I will secure wallet password into a variable:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] read -rs -p "Wallet password: " WALLET_PWD
Wallet password: oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

And import the key into the wallet:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet import_private_key \
  -wallet "$WALLET_DIR" \
  -pwd "$WALLET_PWD" \
  -alias oem24ai \
  -pvtkeyfile "$CERT_DIR/oem24ai.key" \
  -cert "$CERT_DIR/oem24ai.crt" \
  -cacert "$CERT_DIR/oem-root-ca.crt"
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

Enter private key password:
PKI-07014: Unable to import private key. Header not present in private key

It seems the format of the key is not compatible. openSSL created a PKCS#8 PEM format key, let’s convert it to RSA PEM format:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] cp $CERT_DIR/oem24ai.key $CERT_DIR/oem24ai.key.bak.20261029_1202

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ssh-keygen -p -m PEM -f $CERT_DIR/oem24ai.key
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] head -n 1 "/u01/app/oracle/oms24ai/oem-ssl/oem24ai.key"
-----BEGIN RSA PRIVATE KEY-----
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

Now I have a RSA key format.

Let’s retry the import of the key into the wallet:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet import_private_key \
  -wallet "$WALLET_DIR" \
  -pwd "$WALLET_PWD" \
  -alias oem24ai \
  -pvtkeyfile "$CERT_DIR/oem24ai.key" \
  -cert "$CERT_DIR/oem24ai.crt" \
  -cacert "$CERT_DIR/oem-root-ca.crt"
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

Enter private key password:
PKI-07014: Unable to import private key. Header not present in private key
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

Still not ok. Let’s try converting it into a new p12 format key.

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] openssl pkcs12 -export \
  -inkey "$CERT_DIR/oem24ai.key" \
  -in "$CERT_DIR/oem24ai.crt" \
  -certfile "$CERT_DIR/oem-root-ca.crt" \
  -name oem24ai \
  -out "$CERT_DIR/oem24ai.p12"
Enter Export Password:
Verifying - Enter Export Password:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh $CERT_DIR
total 24K
-rw-r--r--. 1 oracle oinstall 1.8K Oct 29 10:33 oem-root-ca.crt
-rw-r--r--. 1 oracle oinstall 1.8K Oct 29 10:33 oem24ai.crt
drwx------. 2 oracle oinstall   90 Oct 29 10:46 wallet
-rw-------. 1 oracle oinstall 2.5K Oct 29 11:02 oem24ai.key.bak.20261029_1202
-rw-------. 1 oracle oinstall 2.5K Oct 29 11:02 oem24ai.key
-rw-------. 1 oracle oinstall 4.9K Oct 29 11:12 oem24ai.p12
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I secured the new p12 key password and try the import again.

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] read -rs -p "PKCS12 password: " PKCS12_PWD
PKCS12 password: oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet import_pkcs12 \
  -wallet "$WALLET_DIR" \
  -pwd "$WALLET_PWD" \
  -pkcs12file "$CERT_DIR/oem24ai.p12" \
  -pkcs12pwd "$PKCS12_PWD"
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

orapki command import_pkcs12 executed successfully.
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

The import was successful now!

I checked the wallet content:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet display \
  -wallet "$WALLET_DIR" \
  -complete
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject:        CN=oem24ai.iaas.jewlab.oraclevcn.com
Issuer:         CN=OEM Lab Root CA
Serial Number:  00:B3:5D:86:6A:43:33:F1:1F
Key Length      3072
MD5 digest:     EC:14:E4:12:22:16:34:FC:28:1D:59:B8:F9:EA:F0:B2
SHA digest:     31:EE:0B:44:44:2C:05:A7:CC:B8:3B:70:C6:F9:E8:83:0E:55:F2:7D

Trusted Certificates:
Subject:        CN=OEM Lab Root CA
Issuer:         CN=OEM Lab Root CA
Serial Number:  00:D7:B3:06:4D:B3:41:CF:9F
Key Length      4096
MD5 digest:     C5:16:F1:A9:AA:AE:9A:FD:1A:A6:6E:72:9C:0B:0A:95
SHA digest:     AA:E4:F4:49:EF:6C:17:4D:9E:30:42:E9:34:A6:80:5D:AB:F3:7D:02

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

Configure OEM 24ai to use the new wallet for HTTPS on port 7803

I first checked that the Wallet is autologin and checked wallet content (should have the OEM server certificate):

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -l "$WALLET_DIR"/ewallet.p12 "$WALLET_DIR"/cwallet.sso
-rw-------. 1 oracle oinstall 4931 Oct 29 11:14 /u01/app/oracle/oms24ai/oem-ssl/wallet/cwallet.sso
-rw-------. 1 oracle oinstall 4886 Oct 29 11:14 /u01/app/oracle/oms24ai/oem-ssl/wallet/ewallet.p12

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/orapki wallet display \
  -wallet "$WALLET_DIR" \
  -summary
Oracle PKI Tool Release 19.0.0.0.0 - Production
Version 19.32.0.0.0
Copyright (c) 2004, 2026, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject:        CN=oem24ai.iaas.jewlab.oraclevcn.com
Trusted Certificates:
Subject:        CN=OEM Lab Root CA
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I backuped the current OMS configuration:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] mkdir oem-ssl/backup-20261029

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ld oem-ssl/backup-20261029/

drwxr-xr-x. 2 oracle oinstall 6 Oct 29 12:46 oem-ssl/backup-20261029/

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] cp -a /u01/app/oracle/oms24ai/gc_inst/em/EMGC_OMS1 ./oem-ssl/backup-20261029/

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh ./oem-ssl/backup-20261029/
total 0
drwxr-x---. 3 oracle oinstall 43 Oct 29 08:17 EMGC_OMS1

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh ./oem-ssl/backup-20261029/EMGC_OMS1/
total 4.0K
drwxr-x---. 10 oracle oinstall  110 Oct 21 05:33 sysman
-rw-r-----.  1 oracle oinstall 2.7K Oct 29 08:17 emgc.properties

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] ls -ltrh ./oem-ssl/backup-20261029/EMGC_OMS1/sysman/
total 12K
drwxr--r--.  2 oracle oinstall    6 Oct 14 13:52 state
drwxr--r--.  3 oracle oinstall   19 Oct 14 13:57 emcli
drwxr-x---.  2 oracle oinstall   89 Oct 14 14:07 ocm
drwxr-----.  2 oracle oinstall   43 Oct 14 14:09 backup
drwxr--r--.  2 oracle oinstall    6 Oct 15 09:56 tmp
drwxr-x---.  9 oracle oinstall 4.0K Oct 21 11:55 agentpush
drwxr--r--.  5 oracle oinstall 4.0K Oct 29 08:16 log
drwxr-x---. 11 oracle oinstall 4.0K Oct 29 08:17 config
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I added the wallet to the console (This targets the HTTPS Console certificate):

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/emctl secure console -wallet /u01/app/oracle/oms24ai/oem-ssl/wallet
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
Securing Console... Started.
Enter Enterprise Manager Root (SYSMAN) Password :
Securing Console... Successful
Run SSL Reload Command
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I ran the command to reload the certificate without having to restart the OMS:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] $OMS_HOME/bin/emctl secure ssl_reload
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
SSL Reload... Started.
SSL Reload completed successfully.
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

I verified the certificate actually presented by port 7803:

oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320] echo | openssl s_client \
  -connect oem24ai.iaas.jewlab.oraclevcn.com:7803 \
  -servername oem24ai.iaas.jewlab.oraclevcn.com \
  2>/dev/null | openssl x509 -noout -subject -issuer -dates
subject=CN=oem24ai.iaas.jewlab.oraclevcn.com
issuer=CN=OEM Lab Root CA
notBefore=Oct 29 10:19:27 2026 GMT
notAfter=Jan 31 10:19:27 2029 GMT
oracle@oem24ai:/u01/app/oracle/oms24ai/ [rdbms19320]

OEM is presenting the new certificate on port 7803

Trust the OEM Root CA on my local Mac

The OEM 24ai server now presents the new certificate on port 7803. Next, I need my Mac to trust the CA that signed it, so Codex can establish a TLS connection to the OEM MCP endpoint.

I checked the certificate again:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % pwd
/Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki

maw@dbi-servicess-MacBook-Pro-2 oem-pki % ls -ltrh
total 56
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   3.2K Oct 29 11:01 oem-root-ca.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.7K Oct 29 11:08 oem-root-ca.crt
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   2.4K Oct 29 11:16 oem24ai.key
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.3K Oct 29 11:16 oem24ai.csr
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   249B Oct 29 11:18 oem24ai.ext
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users    17B Oct 29 11:19 oem-root-ca.srl
-rw-r--r--@ 1 maw  DBI-SERVICES\Domain Users   1.8K Oct 29 11:19 oem24ai.crt

maw@dbi-servicess-MacBook-Pro-2 oem-pki % openssl x509 -in oem-root-ca.crt -noout -subject -issuer -dates
subject= /CN=OEM Lab Root CA
issuer= /CN=OEM Lab Root CA
notBefore=Oct 29 10:08:18 2026 GMT
notAfter=Oct 26 10:08:18 2036 GMT
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

And finally I added the Root CA to macOS trusted certificates:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % sudo security add-trusted-cert \
  -d \
  -r trustRoot \
  -k /Library/Keychains/System.keychain \
  /Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki/oem-root-ca.crt
Password:
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

Successful. This installed the CA into the system keychain and marks it as trusted.

Test connection with curl command

Let’s test the connection with curl command activating TLS verification, so omitting -k option.

I explicitly tested providing the CA file:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % curl -v \
> -cacert /Users/maw/Documents/Dokument/pem_ssh_key/OEM-Lab/oem-pki/oem-root-ca.crt \
> https://oem24ai.jew.lab:7803/em/api/mcp
* URL rejected: No host part in the URL
* Closing connection
curl: (3) URL rejected: No host part in the URL
* Host oem24ai.jew.lab:7803 was resolved.
* IPv6: (none)
* IPv4: 10.0.10.250
*   Trying 10.0.10.250:7803...
* Connected to oem24ai.jew.lab (10.0.10.250) port 7803
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / [blank] / UNDEF
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=oem24ai.iaas.jewlab.oraclevcn.com
*  start date: Oct 29 10:19:27 2026 GMT
*  expire date: Jan 31 10:19:27 2029 GMT
*  subjectAltName: host "oem24ai.jew.lab" matched cert's "oem24ai.jew.lab"
*  issuer: CN=OEM Lab Root CA
*  SSL certificate verify ok.
* using HTTP/1.x
> GET /em/api/mcp HTTP/1.1
> Host: oem24ai.jew.lab:7803
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Date: Thu, 29 Oct 2026 13:01:26 GMT
< Date: Thu, 29 Oct 2026 13:01:26 GMT
< X-ORACLE-DMS-ECID: d5217381-6141-4190-9a60-0a7f609ee162-00000beb
< X-ORACLE-DMS-RID: 0
< X-ORCL-EM-APIGW-ERR: 0
< X-ORCL-EM-APIGW-GUID: 9a1bdced-2748-8952-d070-7f568cfcb7fe
< X-ORCL-EMOA: true
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
<
* Connection #0 to host oem24ai.jew.lab left intact
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

Working now. All ok. HTTP 401 Unauthorized is expected because we’re testing without credentials.

I also tested with default certificate trust configuration:

maw@dbi-servicess-MacBook-Pro-2 oem-pki % curl -v https://oem24ai.jew.lab:7803/em/api/mcp
* Host oem24ai.jew.lab:7803 was resolved.
* IPv6: (none)
* IPv4: 10.0.10.250
*   Trying 10.0.10.250:7803...
* Connected to oem24ai.jew.lab (10.0.10.250) port 7803
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / [blank] / UNDEF
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=oem24ai.iaas.jewlab.oraclevcn.com
*  start date: Oct 29 10:19:27 2026 GMT
*  expire date: Jan 31 10:19:27 2029 GMT
*  subjectAltName: host "oem24ai.jew.lab" matched cert's "oem24ai.jew.lab"
*  issuer: CN=OEM Lab Root CA
*  SSL certificate verify ok.
* using HTTP/1.x
> GET /em/api/mcp HTTP/1.1
> Host: oem24ai.jew.lab:7803
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Date: Thu, 29 Oct 2026 13:03:09 GMT
< Date: Thu, 29 Oct 2026 13:03:09 GMT
< X-ORACLE-DMS-ECID: d5217381-6141-4190-9a60-0a7f609ee162-00000bf3
< X-ORACLE-DMS-RID: 0
< X-ORCL-EM-APIGW-ERR: 0
< X-ORCL-EM-APIGW-GUID: 23898c58-68b4-bb57-a5fb-1baf10253a9e
< X-ORCL-EMOA: true
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Content-Length: 0
<
* Connection #0 to host oem24ai.jew.lab left intact
maw@dbi-servicess-MacBook-Pro-2 oem-pki %

My Mac now trusts the new OEM certificate.

Run a discovery test with Codex

And now I could successfully list the tools exposed by the MCP server using codex, as you can see in previous picture.

The MCP connection is working and exposes 113 tools.

Make some tests with Visual Studio Code and Codex as MCP client

Let’s test and request to list the targets that are up.

I can request which MCP server Codex is using and he will give me the tool that was used to retrieve the information.

I queried to list the databases targets that are up:

And I finally queried to show the databases by tablespace allocated space. Here we can see that Codex will use the ExecuteSql tool and run SQL queries on OMR:

To wrap up…

In this blog I showed how to configure client connection to MCP server built directly in OEM and how to use and query it. We can see that this solution is more powerful and flexible than using Oracle AI Database Assistant.

Here additional interesting websites:

https://docs.oracle.com/en/enterprise-manager/cloud-control/enterprise-manager-cloud-control/24.1/emadm/enterprise-manager-model-context-protocol-server.html

https://blogs.oracle.com/observability/oracle-enterprise-manager-mcp-server-bringing-enterprise-manager-data-to-ai-agents

L’article AI meets OEM, use native OEM MCP server est apparu en premier sur dbi Blog.

For some prompts I was getting the error “No metrics data were found”. In this blog, I would like to explain how I resolved the problem.

Problem description

Running for example the prompt: “Show Oracle databases by tablespace allocated space in MB” I was getting the message “No metrics data were found. Check that metric collection is enabled”.

Explanation

This prompt will need “Tablespace Allocation” and “Tablespace Allocated Space (MB)” metrics.

Let’s check for one of the target DB configured in OEM and see if the metric is enabled.

The metrics is enabled and refresh rate is “every 24 hours”.

Let’s see if there is some data already stored in OMR.

As we can see there is no data for this metrics even if the metric is enabled.

This comes from the fact that it is our lab and the nodes are not always up and running.

Resolution

Let’s connect to the DB System node hosting the database (OEM Agent) and check available targets:

[oracle@oradb19c ~]$ /opt/oracle/agentem24ai/agent_24.1.0.0.0/bin/emctl config agent listtargets
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
[oradb19c.dbaas.jewlab.oraclevcn.com, host]
[oradb19c.dbaas.jewlab.oraclevcn.com:3872, oracle_emd]
[agent24c1_3_oradb19c.dbaas.jewlab.oraclevcn.com_8659, oracle_home]
[CDB19C_SITE1.dbaas.jewlab.oraclevcn.com, oracle_database]
[ASMNET1LSNR_ASM_oradb19c.dbaas.jewlab.oraclevcn.com, oracle_listener]
[LISTENER_oradb19c.dbaas.jewlab.oraclevcn.com, oracle_listener]
[CDB19C_SITE1.dbaas.jewlab.oraclevcn.com_CDBROOT, oracle_pdb]
[CDB19C_SITE1.dbaas.jewlab.oraclevcn.com_PDB19C, oracle_pdb]
[OraDB19000_home1_2_oradb19c.dbaas.jewlab.oraclevcn.com_6741, oracle_home]
[OraGI19Home1_1_oradb19c.dbaas.jewlab.oraclevcn.com_9873, oracle_home]

So we have the target name to be used: [CDB19C_SITE1.dbaas.jewlab.oraclevcn.com_PDB19C, oracle_pdb]

Let’s trigger the tablespace allocation metric collection on the Agent for this target:

[oracle@oradb19c ~]$ /opt/oracle/agentem24ai/agent_24.1.0.0.0/bin/emctl control agent runCollection "CDB19C_SITE1.dbaas.jewlab.oraclevcn.com_PDB19C:oracle_pdb" "tbspAllocation"
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD runCollection completed successfully

[oracle@oradb19c ~]$

We now need to upload the pending collected data to OMS:

[oracle@oradb19c ~]$ /opt/oracle/agentem24ai/agent_24.1.0.0.0/bin/emctl upload agent
Oracle Enterprise Manager 24ai Release 1
Copyright (c) 1996, 2024 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD upload completed successfully
[oracle@oradb19c ~]$

Now check if we have some data metrics for tablespace allocation for this target:

Yes we have!

Let’s change the metric settings to have it run on hourly bases:

To wrap up…

I did the same for the other DB System nodes. Let’s check with the AI prompt to see if we have now some result.

And it’s now working!

So in case of such error, “No metrics data were found. Check that metric collection is enabled”, need to find the corresponding metrics and check if the metric is enabled and data are existing. What is most difficult is to find a documentation providing the corresponding metrics according to the prompt family. If I find anything further, I will update this blog.

L’article Oracle AI Database Assistant showing “No metrics data were found” est apparu en premier sur dbi Blog.

During one patching hop I faced an issue where /root became full. One of the consequences was that I had to manually patch local controller firmware. Through this blog I would like to share with you my experience about this problem and the solution I applied.

Problem description

I was at the step to run the update-servercomponents patching:

[root@node0 ~]# /opt/oracle/dcs/bin/odacli update-servercomponents -v 19.30.0.0.0
{
  "jobId" : "03c81877-9d76-426d-b6c8-a7da09ffca66",
  "status" : "Created",
  "message" : "A server update job has been initiated. If a reboot is required, it will be performed as part of the update process. The final status will be provided once the update operation has been completed on all nodes.",
  "reports" : [ ],
  "createTimestamp" : "September 24, 2026 10:11:14 CEST",
  "resourceList" : [ ],
  "description" : "Server Patching to 19.30.0.0.0",
  "updatedTime" : "September 24, 2026 10:11:14 CEST",
  "jobType" : null,
  "cpsMetadata" : null,
  "ecJobId" : null
}

The job patching activity was on-going correctly:

[root@node0 ~]# odacli describe-job -i 03c81877-9d76-426d-b6c8-a7da09ffca66

Job details
----------------------------------------------------------------
                     ID:  03c81877-9d76-426d-b6c8-a7da09ffca66
            Description:  Server Patching to 19.30.0.0.0
                 Status:  Running
                Created:  September 24, 2026 10:11:14 CEST
                Message:

Task Name                                Node Name                 Start Time                               End Time                                 Status
---------------------------------------- ------------------------- ---------------------------------------- ---------------------------------------- ----------------
Validate ILOM server reachable           node0                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validate ILOM server reachable           node1                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validating GI user metadata              node0                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validating GI user metadata              node1                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Modify BM udev rules                     node0                 September 24, 2026 10:11:25 CEST         September 24, 2026 10:11:36 CEST         Success
Modify BM udev rules                     node1                 September 24, 2026 10:11:25 CEST         September 24, 2026 10:11:36 CEST         Success
Stop oakd                                node1                 September 24, 2026 10:11:40 CEST         September 24, 2026 10:11:43 CEST         Success
Creating local repository                node0                 September 24, 2026 10:11:43 CEST         September 24, 2026 10:11:45 CEST         Success
OSPatchBaseRepo
Creating local repository                node1                 September 24, 2026 10:11:43 CEST         September 24, 2026 10:11:45 CEST         Success
OSPatchBaseRepo
Updating versionlock plugin              node0                 September 24, 2026 10:11:45 CEST         September 24, 2026 10:11:48 CEST         Success
Updating versionlock plugin              node1                 September 24, 2026 10:11:45 CEST         September 24, 2026 10:11:48 CEST         Success
Applying OS Patches                      node0                 September 24, 2026 10:11:48 CEST                                                  Running
Applying OS Patches                      node1                 September 24, 2026 10:11:48 CEST                                                  Running

[root@node0 ~]#

I’m used to let a tail -f running on the DCS alert log agent, just to watch all things are on-going:

tail -f /opt/oracle/dcs/log/dcs-agent.log

And I could see, the following error:

2026-09-24 10:34:50,775 ERROR [DcsJobScheduler-Worker] [] c.o.d.c.e.ExceptionFactory: Failed to write to file: /tmp/dcsfiles/asm/asm_2026-09-24_10-34-50.0766_303.sql
DcsException{errorHttpCode=InternalError, msg=failed to write to file /tmp/dcsfiles/asm/asm_2026-09-24_10-34-50.0766_303.sql: No space left on device, msgId=12022, causedBy=java.io.IOException: No space left on device}
     at com.oracle.dcs.commons.exception.DcsException$Builder.build(DcsException.java:143)
     at com.oracle.dcs.commons.exception.ExceptionFactory.failWriteFile(ExceptionFactory.java:477)
     at com.oracle.dcs.commons.exception.ExceptionFactory.lambda$failWriteFileSupplier$18(ExceptionFactory.java:493)
     at com.oracle.dcs.commons.exception.ExceptionFactory.throwFailWriteFile(ExceptionFactory.java:832)
     at com.oracle.dcs.commons.utils.CommonsUtils.writeToSqlFile(CommonsUtils.java:2006)
     at com.oracle.dcs.agent.utils.db.SqlPlusHelper.runASMStatements(SqlPlusHelper.java:996)
     at com.oracle.dcs.agent.schedule.jobs.RebalanceMonitoringHelper.getRebalanceInfo(RebalanceMonitoringHelper.java:169)
     at com.oracle.dcs.agent.schedule.jobs.RebalanceMonitoringHelper.monitorRebalance(RebalanceMonitoringHelper.java:293)
     at com.oracle.dcs.agent.schedule.jobs.RebalanceMonitorJob.executeJob(RebalanceMonitorJob.java:100)
     at com.oracle.dcs.agent.schedule.jobs.RebalanceMonitorJob.run(RebalanceMonitorJob.java:62)
     at com.oracle.dcs.agent.schedule.DcsJobScheduler$DcsExecutionTask.run(DcsJobScheduler.java:572)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
     at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: No space left on device
     at java.io.FileOutputStream.writeBytes(Native Method)
     at java.io.FileOutputStream.write(FileOutputStream.java:326)
     at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
     at sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:316)
     at sun.nio.cs.StreamEncoder.close(StreamEncoder.java:149)
     at java.io.OutputStreamWriter.close(OutputStreamWriter.java:233)
     at com.oracle.dcs.commons.utils.CommonsUtils.writeToSqlFile(CommonsUtils.java:2004)
     ... 9 more
2026-09-24 10:34:50,776 DEBUG [DcsJobScheduler-Worker] [] c.o.d.a.s.DcsJobScheduler: afterExecute() : Begin

Resolution

I immediately checked Root File System:

[root@node0 tmp]# df -h /tmp
Filesystem                          Size  Used Avail Use% Mounted on
/dev/mapper/VolGroupSys-LogVolRoot   30G   30G     0 100% /

And root file system was full! No space left of device!

How is it possible, knowing one of my pre-patching steps is to check /, /u01, /opt, and /boot file systems occupancies, to ensure that for /, /u01 and /opt I have at least 20 GB available space, running:

df -h / /u01 /opt /boot /opt/oracle/oak/pkgrepos/orapkgs/clones

Checking what could have taken all this space, I could see that a folder named oraapp was taking 23GB space in local root file system:

[root@node0 /]# du -sh *
0    bin
257M boot
3.7M core.oclumon.bin.89762
20M  dev
35M  etc
4.0M home
0    lib
0    lib64
16K  lost+found
4.0K media
92G  mnt
0    null
25G  opt
4.0K oracle-ahf-2511100.zip
23G  oraapp
0    proc
575M root
4.1G run
0    sbin
4.0K srv
0    sys
341M tmp
32G  u01
20K  u02
12K  u03
20K  u04
56K  u99
2.3G usr
1.5G var
[root@node0 /]#

This folder with the name of the application was certainly not generated by the ODA patching. Contacting the customer we could realise that he has been writing some hourly synchronization script to sync application data between primary and standby, which is more than ok. And the data were copied on an ACFS File System that was created for this purpose. Which is totally ok. The problem is that the synchronization job was still running during the patching, and the patching will close all ASM resources. Bingo, the synchronization job created then of course locally a directory in root file system. With customer agreement I immediately deleted that directory.

Lesson learned, we updated the patching MOP to have the new synchronization job stopped before patching.

At the end the patching could anyhow run successfully.

[root@node0 ~]# odacli describe-job -i 03c81877-9d76-426d-b6c8-a7da09ffca66

Job details
----------------------------------------------------------------
                     ID:  03c81877-9d76-426d-b6c8-a7da09ffca66
            Description:  Server Patching to 19.30.0.0.0
                 Status:  Success
                Created:  September 24, 2026 10:11:14 CEST
                Message:

Task Name                                Node Name                 Start Time                               End Time                                 Status
---------------------------------------- ------------------------- ---------------------------------------- ---------------------------------------- ----------------
Validate ILOM server reachable           node0                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validate ILOM server reachable           node1                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validating GI user metadata              node0                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Validating GI user metadata              node1                 September 24, 2026 10:11:24 CEST         September 24, 2026 10:11:24 CEST         Success
Modify BM udev rules                     node0                 September 24, 2026 10:11:25 CEST         September 24, 2026 10:11:36 CEST         Success
Modify BM udev rules                     node1                 September 24, 2026 10:11:25 CEST         September 24, 2026 10:11:36 CEST         Success
Stop oakd                                node1                 September 24, 2026 10:11:40 CEST         September 24, 2026 10:11:43 CEST         Success
Creating local repository                node0                 September 24, 2026 10:11:43 CEST         September 24, 2026 10:11:45 CEST         Success
OSPatchBaseRepo
Creating local repository                node1                 September 24, 2026 10:11:43 CEST         September 24, 2026 10:11:45 CEST         Success
OSPatchBaseRepo
Updating versionlock plugin              node0                 September 24, 2026 10:11:45 CEST         September 24, 2026 10:11:48 CEST         Success
Updating versionlock plugin              node1                 September 24, 2026 10:11:45 CEST         September 24, 2026 10:11:48 CEST         Success
Applying OS Patches                      node0                 September 24, 2026 10:11:48 CEST         September 24, 2026 10:19:41 CEST         Success
Applying OS Patches                      node1                 September 24, 2026 10:11:48 CEST         September 24, 2026 10:19:24 CEST         Success
Creating local repository HMPPatchRepo   node0                 September 24, 2026 10:19:41 CEST         September 24, 2026 10:19:42 CEST         Success
Creating local repository HMPPatchRepo   node1                 September 24, 2026 10:19:41 CEST         September 24, 2026 10:19:42 CEST         Success
Applying HMP Patches                     node0                 September 24, 2026 10:19:42 CEST         September 24, 2026 10:19:45 CEST         Success
Applying HMP Patches                     node1                 September 24, 2026 10:19:42 CEST         September 24, 2026 10:19:44 CEST         Success
Setting SELinux mode                     node0                 September 24, 2026 10:19:45 CEST         September 24, 2026 10:19:45 CEST         Success
Setting SELinux mode                     node1                 September 24, 2026 10:19:45 CEST         September 24, 2026 10:19:45 CEST         Success
Starting the clusterware                 node1                 September 24, 2026 10:21:16 CEST         September 24, 2026 10:22:34 CEST         Success
Deconfigure AFD                          node1                 September 24, 2026 10:24:12 CEST         September 24, 2026 10:25:52 CEST         Success
Oda-hw-mgmt upgrade                      node0                 September 24, 2026 10:25:53 CEST         September 24, 2026 10:27:50 CEST         Success
Patch location validation                node0                 September 24, 2026 10:25:53 CEST         September 24, 2026 10:25:53 CEST         Success
Patch location validation                node1                 September 24, 2026 10:25:53 CEST         September 24, 2026 10:25:53 CEST         Success
Oda-hw-mgmt upgrade                      node1                 September 24, 2026 10:27:50 CEST         September 24, 2026 10:29:47 CEST         Success
Post Deconfigure AFD                     node1                 September 24, 2026 10:31:22 CEST         September 24, 2026 10:32:43 CEST         Success
OSS Patching                             node0                 September 24, 2026 10:32:43 CEST         September 24, 2026 10:32:44 CEST         Success
OSS Patching                             node1                 September 24, 2026 10:32:43 CEST         September 24, 2026 10:32:44 CEST         Success
Installing SQLcl software                node1                 September 24, 2026 10:32:44 CEST         September 24, 2026 10:32:47 CEST         Success
Applying Firmware local Disk Patches     node0                 September 24, 2026 10:32:47 CEST         September 24, 2026 10:32:51 CEST         Success
Applying Firmware local Disk Patches     node1                 September 24, 2026 10:32:51 CEST         September 24, 2026 10:32:55 CEST         Success
Applying Firmware local Controller Patch node0                 September 24, 2026 10:32:58 CEST         September 24, 2026 10:33:01 CEST         Success
Applying Firmware local Controller Patch node1                 September 24, 2026 10:33:01 CEST         September 24, 2026 10:40:45 CEST         Success
Checking Ilom patch Version              node0                 September 24, 2026 10:40:46 CEST         September 24, 2026 10:40:46 CEST         Success
Checking Ilom patch Version              node1                 September 24, 2026 10:40:46 CEST         September 24, 2026 10:40:46 CEST         Success
Patch location validation                node0                 September 24, 2026 10:40:46 CEST         September 24, 2026 10:40:46 CEST         Success
Patch location validation                node1                 September 24, 2026 10:40:46 CEST         September 24, 2026 10:40:46 CEST         Success
Save password in Wallet                  node1                 September 24, 2026 10:40:47 CEST         September 24, 2026 10:40:49 CEST         Success
Disabling IPMI v2                        node0                 September 24, 2026 10:40:49 CEST         September 24, 2026 10:40:50 CEST         Success
Disabling IPMI v2                        node1                 September 24, 2026 10:40:50 CEST         September 24, 2026 10:40:51 CEST         Success
Apply Ilom patch                         node0                 September 24, 2026 10:40:51 CEST         September 24, 2026 10:52:26 CEST         Success
Apply Ilom patch                         node1                 September 24, 2026 10:52:26 CEST         September 24, 2026 11:03:45 CEST         Success
Copying Flash Bios to Temp location      node0                 September 24, 2026 11:03:45 CEST         September 24, 2026 11:03:45 CEST         Success
Copying Flash Bios to Temp location      node1                 September 24, 2026 11:03:45 CEST         September 24, 2026 11:03:45 CEST         Success
Start oakd                               node1                 September 24, 2026 11:04:04 CEST         September 24, 2026 11:04:20 CEST         Success
Add SYSNAME in Env                       node1                 September 24, 2026 11:04:21 CEST         September 24, 2026 11:04:21 CEST         Success
Cleanup JRE Home                         node0                 September 24, 2026 11:04:21 CEST         September 24, 2026 11:04:21 CEST         Success
Cleanup JRE Home                         node1                 September 24, 2026 11:04:21 CEST         September 24, 2026 11:04:21 CEST         Success
Starting the clusterware                 node1                 September 24, 2026 11:05:51 CEST         September 24, 2026 11:07:08 CEST         Success
Update lvm.conf file                     node0                 September 24, 2026 11:07:09 CEST         September 24, 2026 11:07:09 CEST         Success
Update lvm.conf file                     node1                 September 24, 2026 11:07:09 CEST         September 24, 2026 11:07:09 CEST         Success
Generating and saving BOM                node0                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:39 CEST         Success
Generating and saving BOM                node1                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:37 CEST         Success
Update System full patch version         node0                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:10 CEST         Success
Update System full patch version         node1                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:10 CEST         Success
Update System rebootless patch version   node0                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:10 CEST         Success
Update System rebootless patch version   node1                 September 24, 2026 11:07:10 CEST         September 24, 2026 11:07:10 CEST         Success
PreRebootNode Actions                    node0                 September 24, 2026 11:07:39 CEST         September 24, 2026 11:08:53 CEST         Success
PreRebootNode Actions                    node1                 September 24, 2026 11:08:53 CEST         September 24, 2026 11:10:03 CEST         Success
Reboot Node                              node1                 September 24, 2026 11:10:04 CEST         September 24, 2026 11:29:09 CEST         Success
Validate DCS Agent is up                 node1                 September 24, 2026 11:29:08 CEST         September 24, 2026 11:29:08 CEST         Success
Reboot Node                              node0                 September 24, 2026 11:29:10 CEST         September 24, 2026 11:48:15 CEST         Success
Validate DCS Agent is up                 node0                 September 24, 2026 11:48:15 CEST         September 24, 2026 11:48:15 CEST         Success

[root@node0 ~]#

Checking the system, there was no really impact, expect some difference in old kernel in /boot between the two nodes (the synchronization script only run on node0), which was having no impact and would be resolved with next hop patching, and local controller firmware not been updated. The patching process had no space to copy the firmware patches locally on the appropriate directory.

The other patching steps, grid and storage went successfully at well. At the end the components versions were as following:

[root@node0 ~]# odacli describe-component
System Version
--------------
19.30.0.0.0

System Node Name
----------------
node0

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home3                    19.26.0.0.250121     19.30.0.0.260120
     [cdbx9]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167569     up-to-date

BIOS                                     62150200             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.4030           16.35.8002
}

SHARED CONTROLLER FIRMWARE {
     [c1, c2]                            16.00.08.00          up-to-date
}

FIRMWAREEXPANDER {
     [c1x0, c2x0]                        0310                 up-to-date
}

LOCAL DISK FIRMWARE {
     [c0d0, c0d1]                        D3MU001              up-to-date
}

SHARED DISK FIRMWARE {
     [c1d0, c1d1, c1d2, c1d3, c1d4,      RXG0                 up-to-date
     c1d5, c1d6, c1d7, c1d8, c1d9,
     c1d10, c1d11, c1d12, c1d13,
     c1d14, c1d15, c1d16, c1d17,
     c1d18, c1d19, c1d20, c1d21,
     c1d22, c1d23, c2d0, c2d1, c2d2,
     c2d3, c2d4, c2d5, c2d6, c2d7,
     c2d8, c2d9, c2d10, c2d11, c2d12,
     c2d13, c2d14, c2d15, c2d16,
     c2d17, c2d18, c2d19, c2d20,
     c2d21, c2d22, c2d23]
}

HMP                                      2.4.10.1.600         up-to-date

System Node Name
----------------
node1

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home3                    19.26.0.0.250121     19.30.0.0.260120
     [cdbx9]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167569     up-to-date

BIOS                                     62150200             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c1, c2]                            16.00.08.00          up-to-date
}

FIRMWAREEXPANDER {
     [c1x0, c2x0]                        0310                 up-to-date
}

LOCAL DISK FIRMWARE {
     [c0d0, c0d1]                        D3MU001              up-to-date
}

SHARED DISK FIRMWARE {
     [c1d0, c1d1, c1d2, c1d3, c1d4,      RXG0                 up-to-date
     c1d5, c1d6, c1d7, c1d8, c1d9,
     c1d10, c1d11, c1d12, c1d13,
     c1d14, c1d15, c1d16, c1d17,
     c1d18, c1d19, c1d20, c1d21,
     c1d22, c1d23, c2d0, c2d1, c2d2,
     c2d3, c2d4, c2d5, c2d6, c2d7,
     c2d8, c2d9, c2d10, c2d11, c2d12,
     c2d13, c2d14, c2d15, c2d16,
     c2d17, c2d18, c2d19, c2d20,
     c2d21, c2d22, c2d23]
}

HMP                                      2.4.10.1.600         up-to-date

[root@node0 ~]#

As we can see the local controller firmware is not up to date on node 0 when updated on node1.

Manual patching of local controller firmware

Let’s patch manually the local controller firmware on node0.

Local controller firmware needs to be patched on node0 only:

[root@node0 ~]# odacli describe-component | grep 'LOCAL CONTROLLER FIRMWARE' -A2
LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.4030           16.35.8002
}
--
LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}
[root@node0 ~]#

Listing the controller, we can see that we need to patch c3 and c4 Mellanox controllers from 16.35.4030 to 16.35.8002 version for the fiber channel network cards:

[root@node0 ~]# fwupdate list controller

WARNING: Due to strict MMIO memory settings in the running kernel some network controllers may not be accessible.
         See Hardware Management Pack documentation regarding iomem kernel settings required for
         firmware update of these devices.


==================================================
CONTROLLER
==================================================
ID    Type   Manufacturer   Model     Product Name               FW Version       BIOS Version   EFI Version    FCODE Version  Package Version  NVDATA Version    XML Support
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
c0    HDC    Intel          0x2826    0x4873                     -                -              -              -              -                -                 N/A
c1    SAS    LSI Logic      0x0097    ORCL-EXT-SAS3              16.00.08.00      08.37.00.00    18.00.00.00    01.00.68.00    -                14.01.00.13       N/A
c2    SAS    LSI Logic      0x0097    ORCL-EXT-SAS3              16.00.08.00      08.37.00.00    18.00.00.00    01.00.68.00    -                14.01.00.13       N/A
c3    NET    Mellanox       0x1017    Oracle 25GbE Dual-Port SFP 16.35.4030       -              14.29.15       -              -                -                 N/A
c4    NET    Mellanox       0x1017    Oracle 25GbE Dual-Port SFP 16.35.4030       -              14.29.15       -              -                -                 N/A
[root@node0 ~]#

Let’s go in the appropriate mellanox firmware controller directory from the repository and check that version 8002 file is available:

[root@node0 ~]# cd /opt/oracle/oak/pkgrepos/firmwarecontroller/mellanox/0x1017/16.35.8002/

[root@node0 16.35.8002]# ll
total 16392
-rw-r--r--. 1 root root      290 Feb 11  2026 componentmetadata.xml
-rw-r--r--. 1 root root 16777216 Feb 11  2026 fw-ConnectX5-rel-16_35_8002-8201339_Ax_Bx-UEFI-14.29.15.signed.bin
-rw-r--r--. 1 root root     2463 Feb 11  2026 metadata.xml
[root@node0 16.35.8002]#

Let’s update c3 controller:

[root@node0 16.35.8002]# fwupdate update controller -n c3 -x metadata.xml

The following actions will be taken:
==========================================================
ID        Priority Action      Status      Old Firmware Ver.   Proposed Ver.       New Firmware Ver.   System Reboot
---------------------------------------------------------------------------------------------------------------------------
c3        1        Check FW    Success     16.35.4030          16.35.8002          N/A                 N/A
Do you wish to process the above actions? [y/n]? y
Update of c3 to 16.35.8002.
Updating c3: Success

Verifying all priority 1 updates

Execution Summary
==========================================================
ID        Priority Action      Status      Old Firmware Ver.   Proposed Ver.       New Firmware Ver.   System Reboot
---------------------------------------------------------------------------------------------------------------------------
c3        1        Validate    Success     16.35.4030          16.35.8002          16.35.8002          N/A
[root@node0 16.35.8002]#

And c4 controller:

[root@node0 16.35.8002]# fwupdate update controller -n c4 -x metadata.xml

The following actions will be taken:
==========================================================
ID        Priority Action      Status      Old Firmware Ver.   Proposed Ver.       New Firmware Ver.   System Reboot
---------------------------------------------------------------------------------------------------------------------------
c4        1        Check FW    Success     16.35.4030          16.35.8002          N/A                 N/A
Do you wish to process the above actions? [y/n]? y
Update of c4 to 16.35.8002.
Updating c4: Success

Verifying all priority 1 updates

Execution Summary
==========================================================
ID        Priority Action      Status      Old Firmware Ver.   Proposed Ver.       New Firmware Ver.   System Reboot
---------------------------------------------------------------------------------------------------------------------------
c4        1        Validate    Success     16.35.4030          16.35.8002          16.35.8002          N/A

Let’s check and confirm that all is ok, listing the controllers:

[root@node0 16.35.8002]# fwupdate list controller

WARNING: Due to strict MMIO memory settings in the running kernel some network controllers may not be accessible.
         See Hardware Management Pack documentation regarding iomem kernel settings required for
         firmware update of these devices.


==================================================
CONTROLLER
==================================================
ID    Type   Manufacturer   Model     Product Name               FW Version       BIOS Version   EFI Version    FCODE Version  Package Version  NVDATA Version    XML Support
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
c0    HDC    Intel          0x2826    0x4873                     -                -              -              -              -                -                 N/A
c1    SAS    LSI Logic      0x0097    ORCL-EXT-SAS3              16.00.08.00      08.37.00.00    18.00.00.00    01.00.68.00    -                14.01.00.13       N/A
c2    SAS    LSI Logic      0x0097    ORCL-EXT-SAS3              16.00.08.00      08.37.00.00    18.00.00.00    01.00.68.00    -                14.01.00.13       N/A
c3    NET    Mellanox       0x1017    Oracle 25GbE Dual-Port SFP 16.35.8002       -              14.29.15       -              -                -                 N/A
c4    NET    Mellanox       0x1017    Oracle 25GbE Dual-Port SFP 16.35.8002       -              14.29.15       -              -                -                 N/A
[root@node0 16.35.8002]#

We can check and see that now all both nodes local controller firmware are up to date:

[root@node0 16.35.8002]# odacli describe-component | grep 'LOCAL CONTROLLER FIRMWARE' -A2
LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}
--
LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}
[root@node0 16.35.8002]#

To wrap up…

And finally all our ODA components are up to date.

[root@node0 16.35.8002]# odacli describe-component
System Version
--------------
19.30.0.0.0

System Node Name
----------------
node0

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home3                    19.26.0.0.250121     19.30.0.0.260120
     [cdbx9]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167569     up-to-date

BIOS                                     62150200             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c1, c2]                            16.00.08.00          up-to-date
}

FIRMWAREEXPANDER {
     [c1x0, c2x0]                        0310                 up-to-date
}

LOCAL DISK FIRMWARE {
     [c0d0, c0d1]                        D3MU001              up-to-date
}

SHARED DISK FIRMWARE {
     [c1d0, c1d1, c1d2, c1d3, c1d4,      RXG0                 up-to-date
     c1d5, c1d6, c1d7, c1d8, c1d9,
     c1d10, c1d11, c1d12, c1d13,
     c1d14, c1d15, c1d16, c1d17,
     c1d18, c1d19, c1d20, c1d21,
     c1d22, c1d23, c2d0, c2d1, c2d2,
     c2d3, c2d4, c2d5, c2d6, c2d7,
     c2d8, c2d9, c2d10, c2d11, c2d12,
     c2d13, c2d14, c2d15, c2d16,
     c2d17, c2d18, c2d19, c2d20,
     c2d21, c2d22, c2d23]
}

HMP                                      2.4.10.1.600         up-to-date

System Node Name
----------------
node1

Local System Version
--------------------
19.30.0.0.0

Component                                Installed Version    Available Version
---------------------------------------- -------------------- --------------------
OAK                                      19.30.0.0.0          up-to-date

GI                                       19.30.0.0.260120     up-to-date

DB {
     OraDB19000_home3                    19.26.0.0.250121     19.30.0.0.260120
     [cdbx9]
}

DCSCONTROLLER                            19.30.0.0.0          up-to-date

DCSCLI                                   19.30.0.0.0          up-to-date

DCSAGENT                                 19.30.0.0.0          up-to-date

DCSADMIN                                 19.30.0.0.0          up-to-date

OS                                       8.10                 up-to-date

ILOM                                     5.1.5.29.r167569     up-to-date

BIOS                                     62150200             up-to-date

LOCAL CONTROLLER FIRMWARE {
     [c3, c4]                            16.35.8002           up-to-date
}

SHARED CONTROLLER FIRMWARE {
     [c1, c2]                            16.00.08.00          up-to-date
}

FIRMWAREEXPANDER {
     [c1x0, c2x0]                        0310                 up-to-date
}

LOCAL DISK FIRMWARE {
     [c0d0, c0d1]                        D3MU001              up-to-date
}

SHARED DISK FIRMWARE {
     [c1d0, c1d1, c1d2, c1d3, c1d4,      RXG0                 up-to-date
     c1d5, c1d6, c1d7, c1d8, c1d9,
     c1d10, c1d11, c1d12, c1d13,
     c1d14, c1d15, c1d16, c1d17,
     c1d18, c1d19, c1d20, c1d21,
     c1d22, c1d23, c2d0, c2d1, c2d2,
     c2d3, c2d4, c2d5, c2d6, c2d7,
     c2d8, c2d9, c2d10, c2d11, c2d12,
     c2d13, c2d14, c2d15, c2d16,
     c2d17, c2d18, c2d19, c2d20,
     c2d21, c2d22, c2d23]
}

HMP                                      2.4.10.1.600         up-to-date

[root@node0 16.35.8002]#

We could see how easy it is to manually patch local controller firmware on the ODA, which helps to mitigate patching issues impacted by customized script. This is why, knowing we are speaking about appliance, it is always good to avoid the most we can any customization, and if doing some to well document it.

L’article Manual local controller firmware update during ODA Patching est apparu en premier sur dbi Blog.

In this blog post we will analyze much more the internal mechanisms to find the *.ora files, which offered a few surprises. You will see many tracing output. If you are only interested in the summary, scroll down to the end

Local and remote connections

To connect to an Oracle database, you can do that locally

sqlplus appuser/password
sqlplus appuser    #asks for password
sqlplus / as sysdba

In this case, it connects locally to the current environment of ORACLE_HOME and ORACLE_SID. No listener is involved for that.

For a remote connection, you have to connect to a listener (defined by hostname and port), and specifying the desired SID or SERVICE_NAME for which the listener should start a process for you. The easiest form is EZCONNECT: host:port/service, but usually you will use an alias that will be resolved to the destination. These aliases are defined in the tnsnames.ora file or an ldap server specified in ldap.ora. For how to setup an ldap server, see my blog post Setup an LDAP server for Oracle naming on Enterprise Linux 9

MYAPP.DBI-SERVICES.COM =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = host01)(PORT = 1521))
    (CONNECT_DATA =
      (SERVICE_NAME = PDBMYAPP.dbi-services.com)
    )
  )

To connect to that database remotely, you can use

sqlplus appuser/password@//host01:1521/pdbmyapp.dbi-services.com
sqlplus appuser/password@myapp.dbi-services.com
sqlplus appuser/password@myapp  #if names.default_domain=dbi-services.com

Location of the Oracle network configuration (*.ora files)

Without TNS_ADMIN

And where is sqlplus or tnsping looking for the configuration files? As an old, experienced administrator, I would expect it in the directory of the $TNS_ADMIN variable, or if unset, in $ORACLE_HOME/network/admin. Or if you are using the nice feature of the Read-Only ORACLE_HOME (ROOH) where configuration and logs are relocated to ORACLE_BASE (roohctl -enable), it will be in $ORACLE_BASE/homes/<home_name>/network/admin. I will call the directory as $DEFAULT_TNS and it points, depending if ROOH or non-ROOH, to the corresponding network/admin directory.

Hint: My home for the tests is such a ROOH home. If you have a non-ROOH home, simply set DEFAULT_TNS to $ORACLE_HOME/network admin.

Tracing with strace

But is my expectation of name resolution really correct? On Linux, we can see with the strace binary what the tool will look for:

strace sqlplus -S -L a/b@xyz 2>&1 |grep -E '\.ora'

I will use an alias that does not exist (@xyz). strace prints the output on STDERR, so I will redirect it to STDOUT (2>&1) and filter the whole output for “.ora”. I use the ORACLE_HOME=/u00/app/oracle/product/19.32.0 where no Oracle newtork configuration is present:

lstat("/u00/app/oracle/product/19.32.0/env.ora", {st_mode=S_IFREG|0644, st_size=852, ...}) = 0
openat(AT_FDCWD, "/u00/app/oracle/product/19.32.0/env.ora", O_RDONLY) = 3
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/home/oracle/.sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/intchg.ora", F_OK)         = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/intchg.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/tnsnav.ora", F_OK)         = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnav.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/home/oracle/.tnsnames.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/tnsnames.ora", F_OK)       = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora", F_OK) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/u00/app/oracle/product/19.32.0/ldap/admin/fips.ora", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/u00/app/oracle/product/19.32.0/ldap/admin/fips.ora", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", 0x7fff8d79e530) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/product/19.32.0/network/admin/ldap.ora", 0x7fff8d79e530) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/ldap.ora", 0x7fff8d79e620) = -1 ENOENT (No such file or directory)

• $ORACLE_HOME/env.ora : First surprise, a file that was not known to me. In this file you can set environment variables. Here you can set a TNS_ADMIN variable as a fallback if it is not yet set in the environment.
• /u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora: The next one is the $DEFAULT_TNS location which is expected for me
• /home/oracle/.sqlnet.ora : Did you know that it looks for a “.sqlnet.ora” (mind the dot!) in your home-directory?
• intchg.ora / tnsnav.ora : seems to be related to network “communities” and routing, not naming resolution.
• /home/oracle/.tnsnames.ora : It also looks for a .tnsnames.ora (mind the dot!) in the user’s home.
• /etc/tnsnames.ora : I have never seen that before …
• /u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora : expected $DEFAULT_TNS location
• fips.ora : Related to cryptography (US government Federal Information Processing Standards)
• /u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora : It is a home configured for ROOH. Why it looks here? Probably a bug…
• /u00/app/oracle/homes/OraDB19Home2/network/admin/ldap.ora : That is what I expected.

Summary: 1st, it looks for an sqlnet.ora (to determine name resolution order and default domain); Then, it looks for a tnsnames.ora, then for an ldap server

Next, I will create a /home/oracle/.sqlnet.ora and trace what happens:

$ strace sqlplus -S -L a/b@xyz 2>&1 |grep -E '\.ora'
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/home/oracle/.sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/home/oracle/.sqlnet.ora", O_RDONLY) = 6
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)

It opens my .sqlnet.ora, but it also tries to open the sqlnet.ora of the $DEFAULT_TNS. I assume it will use it as fallback for parameters not present in $HOME/.sqlnet.ora; let’s try:

Create a test configuration

I create a configuration with

# /u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora
names.default_domain = dbi-services.com
SQLNET.EXPIRE_TIME = 10

# /home/oracle/.sqlnet.ora
names.default_domain = dbi.internal
SQLNET.EXPIRE_TIME = 10

#  /u00/app/oracle/homes/OraDB19Home2/network/admin/
DBI.dbi-services.com = ...

# /home/oracle/.tnsnames.ora
EXT.dbi-services.com = ...
INT.dbi.internal = ...

Then, I try to connect:

sqlplus a/b@EXT  # ORA-12154: TNS:could not resolve the connect identifier specified
sqlplus a/b@INT   # it can resolve it and connects (default_domain of $HOME/.sqlnet.ora)

The domain of $HOME/.sqlnet.ora is used. If I remove the default domain from the $HOME/.sqlnet.ora:

sqlplus a/b@EXT  # it can resolve it and connects (using default_domain of $DEFAULT_TNS/sqlnet.ora)
sqlplus a/b@INT   # ORA-12154: TNS:could not resolve the connect identifier specified

That means, values in $HOME/.sqlnet.ora will override values of the default $DEFAULT_TNS/sqlnet.ora. The same applies to $HOME/.tnsnames.ora, it overrides entries in the default tnsnames.ora

And what about the /etc/tnsnames.ora? Create it:

# /etc/tnsnames.ora
ETC.dbi-services.com = ...

sqlplus a/b@ETC.dbi-services.com
sqlplus a/b@INT.dbi.internal 
sqlplus a/b@EXT.dbi-services.com   # ORA-12154: TNS:could not resolve the connect identifier specified
sqlplus a/b@ETC

If it is present, the default tnsnames.ora in the $DEFAULT_TNS is ignored (it will not find the EXT.dbi-services.com), but the $HOME/.tnsnames.ora has precedence (it can resolve INT.dbi.internal). And we can also see that the default-domain for “@ETC” is taken from the $DEFAULT_TNS/sqlnet.ora (in $HOME/.sqlnet.ora it is not defined).

With TNS_ADMIN

TNS_ADMIN is the variable to explicitly specify the location of the *.ora configuration files. If it is not set in your shell, Oracle looks for it in $ORACLE_HOME/env.ora;
It is usually used to centralize the configuration for all ORACLE_HOME’s in one directory. We recommend ${ORACLE_BASE}/network/admin.

First, I will use an empty TNS_ADMIN directory with no *.ora files

mkdir /tmp/empty
export TNS_ADMIN=/tmp/empty
strace sqlplus -S -L a/b@abc 2>&1 |grep -E "\.ora"|grep -vE 'tnsnav|intchg|fips'

lstat("/u00/app/oracle/product/19.32.0/env.ora", {st_mode=S_IFREG|0644, st_size=852, ...}) = 0
openat(AT_FDCWD, "/u00/app/oracle/product/19.32.0/env.ora", O_RDONLY) = 3
access("/tmp/empty/sqlnet.ora", F_OK)   = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", O_RDONLY) = 6
access("/home/oracle/.sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/home/oracle/.sqlnet.ora", O_RDONLY) = 6
access("/tmp/empty/sqlnet.ora", F_OK)   = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/u00/app/oracle/homes/OraDB19Home2/network/admin/sqlnet.ora", O_RDONLY) = 8
access("/home/oracle/.tnsnames.ora", F_OK) = 0
access("/tmp/empty/tnsnames.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/tnsnames.ora", F_OK)       = 0
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
openat(AT_FDCWD, "/home/oracle/.tnsnames.ora", O_RDONLY) = 9
stat("/etc/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=579, ...}) = 0
openat(AT_FDCWD, "/etc/tnsnames.ora", O_RDONLY) = 9
stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", 0x7ffde1f5b730) = -1 ENOENT (No such file or directory)
stat("/tmp/empty/ldap.ora", 0x7ffde1f5b730) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/product/19.32.0/network/admin/ldap.ora", 0x7ffde1f5b730) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/ldap.ora", 0x7ffde1f5b820) = -1 ENOENT (No such file or directory)
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
stat("/etc/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=579, ...}) = 0
...

If TNS_ADMIN is set, but the directory is empty, it will fallback to the order as above.
Now I will set it to a directory with an sqlnet.ora file and no tnsnames.ora, and I delete the /etc/tnsnames.ora:

mkdir /tmp/admin; cp $ORACLE_BASE/network/admin/sqlnet.ora /tmp/admin/
export TNS_ADMIN=/tmp/admin
strace sqlplus -S -L a/b@abc 2>&1 |grep -E 'tnsnames|sqlnet'

access("/tmp/admin/sqlnet.ora", F_OK)   = 0
openat(AT_FDCWD, "/tmp/admin/sqlnet.ora", O_RDONLY) = 6
access("/home/oracle/.sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/home/oracle/.sqlnet.ora", O_RDONLY) = 6
access("/tmp/admin/sqlnet.ora", F_OK)   = 0
openat(AT_FDCWD, "/tmp/admin/sqlnet.ora", O_RDONLY) = 8
access("/home/oracle/.tnsnames.ora", F_OK) = 0
access("/tmp/admin/tnsnames.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/tnsnames.ora", F_OK)       = -1 ENOENT (No such file or directory)
access("/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora", F_OK) = 0
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
openat(AT_FDCWD, "/home/oracle/.tnsnames.ora", O_RDONLY) = 12
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=432, ...}) = 0
openat(AT_FDCWD, "/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora", O_RDONLY) = 12
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=432, ...}) = 0
...

It takes the $TNS_ADMIN/sqlnet.ora and ignores the $DEFAULT_TNS/sqlnet.ora. It will not find the $TNS_ADMIN/tnsnames.ora (does not exist) and not the /etc/tnsnames.ora (I removed it before), so it uses the default in $DEFAULT_TNS.

For the next try I will add the tnsnames.ora to the TNS_ADMIN.

cp $ORACLE_BASE/network/admin/tnsnames.ora /tmp/admin
export TNS_ADMIN=/tmp/admin
strace sqlplus -S -L a/b@abc 2>&1 |grep -E 'tnsnames|sqlnet'

access("/tmp/admin/sqlnet.ora", F_OK)   = 0
openat(AT_FDCWD, "/tmp/admin/sqlnet.ora", O_RDONLY) = 6
access("/home/oracle/.sqlnet.ora", F_OK) = 0
openat(AT_FDCWD, "/home/oracle/.sqlnet.ora", O_RDONLY) = 6
access("/tmp/admin/sqlnet.ora", F_OK)   = 0
openat(AT_FDCWD, "/tmp/admin/sqlnet.ora", O_RDONLY) = 8
access("/home/oracle/.tnsnames.ora", F_OK) = 0
access("/tmp/admin/tnsnames.ora", F_OK) = 0
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
openat(AT_FDCWD, "/home/oracle/.tnsnames.ora", O_RDONLY) = 12
stat("/tmp/admin/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=11319, ...}) = 0
openat(AT_FDCWD, "/tmp/admin/tnsnames.ora", O_RDONLY) = 12
read(12, "# tnsnames.ora Network Configura"..., 4096) = 4096
stat("/home/oracle/.tnsnames.ora", {st_mode=S_IFREG|0644, st_size=716, ...}) = 0
stat("/tmp/admin/tnsnames.ora", {st_mode=S_IFREG|0644, st_size=11319, ...}) = 0
...

As expected, it finds the $TNS_ADMIN/tnsnames.ora and it no longer looks for it in the $DEFAULT_TNS/tnsnames.ora. And this was not quite clear to me: it will no longer look for the /etc/tnsnames.ora file. It is only used if TNS_ADMIN is NOT set!

ldap.ora

And what about the LDAP connections (defined in ldap.ora)? Currently there is no ldap.ora present. Without TNS_ADMIN:

stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", 0x7ffdb464fd30) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/product/19.32.0/network/admin/ldap.ora", 0x7ffdb464fd30) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/ldap.ora", 0x7ffdb464fe20) = -1 ENOENT (No such file or directory)

“Works as designed” or a bug: it first looks in $ORACLE_HOME, instead of the ROOH admin direcory.

And with TNS_ADMIN set, but without ldap.ora

stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", 0x7ffcb23acdb0) = -1 ENOENT (No such file or directory)
stat("/tmp/admin/ldap.ora", 0x7ffcb23acdb0) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/product/19.32.0/network/admin/ldap.ora", 0x7ffcb23acdb0) = -1 ENOENT (No such file or directory)
stat("/u00/app/oracle/homes/OraDB19Home2/network/admin/ldap.ora", 0x7ffcb23acea0) = -1 ENOENT (No such file or directory)

Now I set in TNS_ADMIN the names.directory_path=(LDAP) and add the ldap.ora file

stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", 0x7ffdb55680b0) = -1 ENOENT (No such file or directory)
stat("/tmp/admin/ldap.ora", {st_mode=S_IFREG|0644, st_size=500, ...}) = 0
stat("/tmp/admin/ldap.ora", {st_mode=S_IFREG|0644, st_size=500, ...}) = 0
openat(AT_FDCWD, "/tmp/admin/ldap.ora", O_RDONLY) = 12
...

And it works! But what if I create an ldap.ora in the ORACLE_HOME? I create it with an invalid ldap host.

stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", {st_mode=S_IFREG|0644, st_size=501, ...}) = 0
stat("/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", {st_mode=S_IFREG|0644, st_size=501, ...}) = 0
openat(AT_FDCWD, "/u00/app/oracle/product/19.32.0/ldap/admin/ldap.ora", O_RDONLY) = 12

We can see that it only uses the $ORACLE_HOME/ldap/admin/ldap.ora, and not the $TNS_ADMIN/ldap.ora! For me, that is a bug. For all other *.ora files the TNS_ADMIN variable is respected, but not for ldap.ora!

Summary

To resolve an sql*net alias, which configuration files sqlplus will use?

My stone-age knowledge that the *.ora files in $TNS_ADMIN are used, and if TNS_ADMIN is not set, then it uses the files in the $DEFAULT_TNS. That’s mostly correct (or at least not completely wrong), but there are some differences. My most important findings:

If a file in TNS_ADMIN does not exist, it looks in other locations. If a file .sqlnet.ora or .tnsnames.ora (with leading dot) in $HOME exists, the normal *.ora files will be used, but entries in the dot files has precedence. And if there is a file /etc/tnsnames.ora and TNS_ADMIN is not set, it will use it. And it ignores a $TNS_ADMIN/ldap.ora if there is one in $ORACLE_HOME/ldap/admin/.

sqlnet.ora (Network defaults)

The relevant parameters for name resolution are NAMES.DEFAULT_DOMAIN and NAMES.DIRECTORY_PATH.
It uses the sqlnet.ora in:

• $HOME/.sqlnet.ora : (mind the dot!): if it exists, the following files will be used, but values in this file has precedence.
• $TNS_ADMIN/sqlnet.ora : It uses this file and stops. If it does not exist or TNS_ADMIN is not defined, then continue with the next point:
• $default_tns/sqlnet.ora

tnsnames.ora

• $HOME/.tnsnames.ora : (mind the dot!): if it exists, the following files will be used, but aliases in this file has precedence.
• /etc/tnsnames.ora : if TNS_ADMIN is NOT set! It uses this file and stops
• $TNS_ADMIN/tnsnames.ora : if TNS_ADMIN is set and tnsnames.ora is present, it uses it and stops; if the file is not present, it continues with the next point.
• $default_tns/tnsnames.ora

ldap.ora

• $ORACLE_HOME/ldap/admin/ldap.ora : if it exists, It uses this file and stops (TNS_ADMIN is ignored!)
• $TNS_ADMIN/ldap.ora : if TNS_ADMIN is set and ldap.ora is present, it uses it and stops; if the file is not present, it continues with the next point.
• $default_tns/ldap.ora

These are the findings with an sqlplus of version 19.32.0; For other releases/versions it is probably the same, but you can simply verify it yourself with the strace utility.

For Oracle 26ai (23.26) it is a litte bit easier to see what tnsnames.ora is used

SQL*Plus: Release 23.26.1.0.0 - Production on Wed Oct 28 14:10:19 2026
Version 23.26.1.0.0

Copyright (c) 1982, 2025, Oracle.  All rights reserved.

ERROR:
ORA-12154: Cannot connect to database. Cannot find alias db1970 in
/home/oracle/.tnsnames.ora,
/u00/app/oracle/product/23/23.26/network/admin/tnsnames.ora.
Help: https://docs.oracle.com/error-help/db/ora-12154/


SP2-0751: Unable to connect to Oracle.  Exiting SQL*Plus
Help: https://docs.oracle.com/error-help/db/sp2-0751/

or with an ldap configuration, you will see that it tried it, but not which ldap.ora was used (so strace may be still needed).

ORA-12154: Cannot connect to database. Cannot find alias db1970 in
/home/oracle/.tnsnames.ora, /tmp/admin/tnsnames.ora and LDAP.

L’article Name resolution of Oracle est apparu en premier sur dbi Blog.

For an application server that connects to Oracle, you probably would not like to install gigabytes of a full Oracle client software? For that, Oracle offers the so-called “Instant Client” you can download for free at oracle.com. It is available for different platforms and in different package formats (rpm, zip, …)

There are different components you can download. Always required is the “basic” or “basic lite” package with the core libraries. Additionally you can install sqlplus, tools (sqlldr, datapump), sdk and the odbc component.

Analyzing connection problems

Analyzing with the “normal” client

If you have issues with the naming resolution, you will probably start with the tnsping utility:

$  tnsping DB01

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 27-OCT-2026 17:25:09

Copyright (c) 1997, 2026, Oracle.  All rights reserved.

Used parameter files:
/u00/app/oracle/network/admin/sqlnet.ora

TNS-03505: Failed to resolve name

It will show you which TNS_ADMIN (the location of the sqlnet.ora and the other *.ora files) is used. It will look in $TNS_ADMIN and if it is not set, in $ORACLE_HOME/network/admin.

And why “Failed to resolve name”? Have a look at the names.directory_path parameter in the sqlnet.ora file, e.g.

NAMES.DIRECTORY_PATH=(LDAP, TNSNAMES)
# default is (tnsnames, ezconnect, ldap)

In this case the alias “DB01” is unknown in LDAP and the tnsnames.ora. For LDAP, check that the ldap.ora is correctly defined and the ldap server running and reachable, otherwise it will be skipped and only the tnsnames.ora is evaluated. See also my Oracle LDAP Timeouts blog.

If the alias exists, but the connection fails, you will get the resolved connect-string you can analyze.

Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = server01)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TEST.dbi-services.com)))
TNS-12545: Connect failed because target host or object does not exist

Analyzing with the Instant Client

If you are using the instant client, unfortunately there is no tnsping available. How we can debug the connection? The only instantclient binary we have is sqlplus.

First, check if the variable TNS_ADMIN is set. Then, the configuration in this directory is used.

If not, where does it look for?

If you extract the instant client from the .zip file, binaries and libraries are all in the same directory (.rpm is different). To run sqlplus, the libraries of the instantclient have to be in $LD_LIBRARY_PATH, otherwise we get

sqlplus: error while loading shared libraries: libsqlplus.so: cannot open shared object file: No such file or directory

To see where sqlplus will look for the network configuration (*.ora) we can use the strace utility of Linux.

$ strace sqlplus x/y@DB01  2>&1 |grep sqlnet.ora
access("/u00/app/oracle/product/instantclient/instantclient_21_7/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)

Is this the path from the binary, or the path from LD_LIBTRARY_PATH? To find it out, I will write the path for the libraries relative.

export PATH=$PATH:/u00/app/oracle/product/instantclient/instantclient_21_7
cd /u00/app/oracle/product/instantclient/instantclient_21_7
LD_LIBRARY_PATH=../instantclient_21_7 strace sqlplus x/y@DB01  2>&1 |grep sqlnet.ora

access("../instantclient_21_7/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("../instantclient_21_7/network/admin/sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)
access("/home/oracle/.sqlnet.ora", F_OK) = -1 ENOENT (No such file or directory)

As you can see, it looks for the configuration below the library location in network/admin. And that is different from an sqlplus in a “normal” ORACLE_HOME, where it looks for $ORACLE_HOME/network/admin!

Now you can check the *.ora files in this directory and analyze why the name can’t be resolved.

If the name is resolved, but the connect fails, you have to get the resolved connect string. Note the order of names.directory_path!

If LDAP is one of the naming methods, it is a little bit more complicated. You can use the ldapsearch binary. Depending on the version from openldap or oracle, the syntax is slightly different:

# openldap 
/usr/bin/ldapsearch -o ldif-wrap=no -h localhost -p 389 -D '' \
  -b "cn=DB1970,cn=OracleContext,dc=dbi-services,dc=com" \
  -s sub "objectclass=orclNetService"

# oracle
 ldapsearch  -x -LLL -h localhost -p 389 \
  -b "cn=DB1970,cn=OracleContext,dc=dbi-services,dc=com" \
  -s sub "objectclass=orclNetService"

cn=DB1970,cn=OracleContext,dc=dbi-services,dc=com
objectClass=top
objectClass=orclNetService
cn=DB1970
orclNetDescString=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=server01)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=DB1970A.dbi-services.com)))

the value of orclNetDescString is the connect string sqlplus uses.

In case of the tnsnames.ora, it is easy: search in the file for an alias “DB01”.

Update: Good news! If you use a 26ai instant client, it shows you the tnsnames.ora files it used

ERROR:
ORA-12154: Cannot connect to database. Cannot find alias z in
/home/oracle/.tnsnames.ora, /u00/app/oracle/network/admin/tnsnames.ora and
LDAP.
Help: https://docs.oracle.com/error-help/db/ora-12154/

Instant Client on Windows?

Same but differnt It also evaluates the TNS_ADMIN variable and the network\admin directory below the instant client directory with the *.dll; But additionally, it also evaluates the *.ora files in the current directory.

L’article tnsping with Oracle instant-client? est apparu en premier sur dbi Blog.

After working on several Oracle Fusion Middleware upgrades, involving WebLogic Server, ADF, Oracle Forms, Oracle Reports, Oracle HTTP Server, and Identity Management components, I have noticed that the most challenging issues rarely come from the installation itself.

They usually come from existing configurations, dependencies, database schemas, or assumptions inherited from the previous environment.

Oracle provides comprehensive upgrade documentation, but every environment has its own history and technical constraints.

Here are six areas I now pay particular attention to when preparing and executing an Oracle Fusion Middleware 12c to 14c upgrade.

1. Compatibility is more than an Operating System check

Before starting an upgrade, the first challenge is understanding the full compatibility matrix.

Oracle FMW 14.1.2 introduces a new technology baseline, including Java 17, which can expose dependencies that worked perfectly under 12c.

Operating system certification is only one part of the equation.

The complete assessment should cover:

• Operating system and database versions
• JDK certification and compatibility
• Middleware component versions and interoperability
• Third-party libraries and custom Java applications
• JDBC drivers, authentication providers, and external integrations

A particularly important point is to inspect custom Java code for dependencies on internal WebLogic APIs or libraries whose behavior may have changed.

My recommendation: Do not limit your compatibility assessment to Oracle’s certification matrix. Include the applications and integrations that depend on the middleware.

A supported platform does not automatically mean a compatible application.

2. Database schemas: Small configuration differences, major consequences

One of the most important lessons concerns the relationship between WebLogic domains and their RCU schemas.

For example, Oracle FMW 14c introduces an important distinction between the following database users:

• <PREFIX>_WLS
• <PREFIX>_WLS_RUNTIME

In environments where WLSSchemaDataSource still uses the runtime schema, the configuration must be reviewed before upgrading.

The Upgrade Assistant expects the appropriate WLS schema owner. A mismatch can prevent the schema upgrade from completing successfully.

Oracle explicitly documents this requirement, but it is easy to overlook when working with multiple domains and different RCU prefixes.

Beyond this specific example, I systematically verify schema ownership, registry versions, database connectivity, account status, and the consistency between configured data sources and database metadata.

My recommendation: Treat database schema validation as an independent upgrade workstream, not merely a preliminary check performed by the Upgrade Assistant.

3. Readiness checks are essential, but they are not a guarantee

Oracle’s Upgrade Assistant provides readiness checks to identify potential upgrade problems.

They are extremely valuable and should be executed before the actual migration.

However, a successful readiness check does not guarantee that the upgrade will succeed.

Some issues only appear during schema modification, domain reconfiguration, server startup, or application initialization.

For this reason, I distinguish between three levels of validation:

• Pre-upgrade readiness: Are the schemas and configurations eligible for upgrade?
• Technical upgrade validation: Have the schemas and domains been successfully upgraded?
• Functional validation: Do applications, integrations, security services, and business processes actually work?

My recommendation: Never interpret a successful readiness check, or even a successful domain reconfiguration, as proof that the environment is operational. The real validation begins when the applications start communicating with their dependencies.

4. Domain reconfiguration deserves its own risk assessment

Domain reconfiguration is one of the most sensitive phases of an Oracle FMW upgrade.

The Reconfiguration Wizard modifies existing domain configurations to align them with the target middleware version. These changes can affect security settings, JDBC resources, deployment configurations, and product-specific components. The challenge becomes greater in environments containing multiple domains, custom configurations, or legacy integrations.

One important consideration is that domain reconfiguration cannot simply be reversed. Oracle explicitly recommends restoring the original domain from a backup if reconfiguration fails.

My recommendation: Take complete, verified backups of the Domain Homes and document custom configurations before starting reconfiguration. More importantly, treat schema upgrade and domain reconfiguration as coordinated operations, with clearly defined recovery points. Rollback must be designed before the upgrade begins, not after something goes wrong.

5. Patch strategy matters as much as the target version

Installing Oracle FMW 14c does not necessarily mean the environment is ready to be upgraded. Depending on the products involved, additional Bundle Patches, Stack Patch Bundles, or product-specific fixes may be required or recommended.

This is particularly important in environments combining components such as:

• WebLogic Server and ADF
• Oracle Access Management
• Oracle Internet Directory
• Oracle HTTP Server
• Oracle Reports

The objective is not simply to install the latest available patch. It is to establish a consistent, certified, and supportable patch baseline across all Oracle Homes. An incomplete or inconsistent patch strategy can introduce unnecessary complexity into an already sensitive migration.

My recommendation: Define and validate the target patch inventory before executing the upgrade. Keep installation, patching, schema upgrade, and domain reconfiguration as distinct, controlled phases.

6. The most important upgrade test is not whether WebLogic starts

A running AdminServer is encouraging, but it proves very little about the overall application platform. The same applies to Managed Servers reporting a RUNNING state.

A complete post-upgrade validation must consider the entire application chain:

• JDBC connectivity and database transactions
• Authentication, authorization, and identity integrations
• Application deployments and critical business transactions
• HTTP routing, SSL/TLS, and reverse proxy integration
• Scheduled processes, reporting services, and external dependencies

This is particularly relevant in environments where ADF applications, Oracle Reports, and Identity Management components depend on one another. It is entirely possible to have a technically healthy WebLogic domain while critical application functionality remains unavailable.

My recommendation: Define application-level acceptance criteria with the customer before scheduling the upgrade. The upgrade is complete when the expected business services have been validated, not when the last Managed Server reaches the RUNNING state.

Final Thoughts

After several Oracle Fusion Middleware upgrade projects, one conclusion has become clear to me: the actual upgrade commands are rarely the most difficult part of the job.

The real complexity lies in understanding the existing architecture, identifying hidden dependencies, anticipating compatibility issues, and controlling the sequence of changes.

Oracle’s documentation provides the technical foundation. Experience helps identify where additional verification is necessary.

A successful migration from 12c to 14c is not measured only by the absence of errors during the upgrade. It is measured by the ability to deliver a stable, supportable, and fully functional platform with a controlled operational impact.

Related blogs

L’article Oracle Fusion Middleware 12c to 14c: 6 Critical Lessons est apparu en premier sur dbi Blog.

Today, I would like to consider what it means when we decide to start a migration project.

We have 2 million documents in our current ECM. We need to migrate them to the new platform.

It sounds simple.

At least, it does when you put it in a project plan:

• Extract the documents
• Transform the metadata
• Load everything into the new system
• Validate the result
• Go live

Then you start looking at the contents of those 2 million documents and the questions begin:

• Which documents are still relevant?
• Which ones are duplicates?
• Which ones are obsolete?
• Which ones are legally required to be retained?
• Which ones are just old drafts?
• Who owns them?
• Are the permissions still correct?
• Is the metadata meaningful?
• Does anyone even know why some of these documents exist?

Suddenly, migration doesn’t look like a technical project anymore.

It looks more like an archaeological excavation!

And that is the uncomfortable reality about ECM migrations:

Migration is the moment when you inherit everyone’s mess.

Migration doesn’t create the mess

One of the biggest misconceptions about ECM migration is that the migration project is responsible for creating the information problems.

This is usually not the case. These problems have been accumulating for years:

• A document was created for a project.
• Someone copied it to another folder.
• Another person downloaded it and changed the file name.
• A new version was uploaded elsewhere.
• The project ended.
• The people involved moved to other departments.
• The folder structure changed.
• The retention rules changed.
• The application that created the document was replaced.
• Nobody went back to clean up what was left behind.
• Multiply this by thousands of users and years of activity, and the result is what I call ‘information debt’.

This debt isn’t visible on a daily basis; people simply work around it. Day after day, searching for information takes longer and longer.
They create this mess through the way they work: keeping several copies of documents, asking colleagues where things are, creating additional folders and saving documents ‘just in case’.

Until one day, someone says: “We’re migrating to a new ECM.”

Suddenly, all that accumulated debt becomes part of the migration scope. This is why migration projects often reveal problems that have existed for years. Migration does not create the mess. It simply makes the mess impossible to ignore.

The “just migrate everything” strategy

When faced with a large repository, it is understandable to have a certain reaction.

“We should migrate everything. We don’t want to lose anything.”

This approach sounds safe and responsible.

And sometimes it is also the easiest political option, because nobody has to make difficult decisions, explain why 15-year-old documents were deleted, or decide whether old project folders still have value.

You just move everything, But the problem is that you haven’t solved the information problem.

You’ve just moved it!

It’s like moving house: your old basement is full of boxes.

Rather than opening the boxes and deciding what you need, you simply load them onto the truck and store them in the new basement.

Technically, the move was successful. But you haven’t tidied up at all.
Ultimately, you continue to have years of old stuff that your family doesn’t need, but now it’s in a new building.

That’s exactly what happens with many ECM migrations.

Old repository → migration → new repository.

A few months later: “Why is the new ECM already so difficult to navigate?”

Because you migrated the old information environment instead of redesigning it.

The three questions

Before deciding how to migrate content, I think every organization should ask three simple questions:

• Should we migrate it?
• Should we keep it somewhere else?
• Or should we delete it?

Although these questions may sound obvious, they are surprisingly difficult in practice.

Migrate it

Clearly, some information is still valuable.

For example:

• active contracts;
• current customer documentation;
• ongoing projects;
• current procedures;
• active employee records;
• current policies;
• documents actively used by business processes;
• information required for day-to-day operations.

This content is suited to the new information environment. Ideally, it should not simply be copied as it is.

The migration provides an opportunity to improve the classification, metadata, relationships and lifecycle of the content.

The objective is not just to put this document into the new system, but to ensure that this information is useful within it.

These are very different objectives.

Archive it

The second category is more subtle.

While some information still needs to exist, it doesn’t need to remain part of the active working environment.

Consider:

• closed projects
• expired contracts
• historical financial records
• old customer files
• completed cases.
• previous versions that must be retained
• records subject to regulatory retention

These documents may have little operational value today.

However, this does not necessarily mean that they can be deleted.

Organizations often confuse two very different concepts here. Keeping something is not the same as actively using it.

Retention requirements may mean that information must remain available for several years.

However, this does not automatically mean that it belongs alongside the documents that people use every day.

Archiving can therefore be an important part of a migration strategy.

The aim is not to ignore historical information. Rather, the goal is to prevent historical information from overwhelming current information.

Delete it

Then comes the most uncomfortable option. Delete it.

Some information simply isn’t worth migrating.

• duplicates
• temporary exports
• old drafts
• test documents
• abandoned documents
• obsolete versions
• documents created for a one-off purpose that no longer exists
• files with no business value or retention requirement
• documents that nobody can explain

This is where migration projects can become political.

Someone will inevitably say: “We might need it someday.”

Maybe! However, “we might need it someday” is not an information management strategy.

If every document is considered potentially useful forever, nothing can ever be deleted.

If nothing can ever be deleted, your information environment will eventually resemble a historical archive rather than a working system.

The fourth category: nobody knows

However, in (almost) every migration project, there will always be a category that doesn’t fit neatly into ‘migrate’, ‘archive’ or ‘delete’.

Nobody knows.

You find a document, it looks important.

But nobody knows:

• Who owns it?
• Whether it is still valid?
• Why it exists?
• Whether it is a record?
• Whether there is a newer version?
• Whether it can be deleted?

So, what do organizations usually do?

They migrate it, because migration feels safer than making a decision.

This creates what I call the “just in case” migration.

“Just in case” is one of the most expensive phrases in information management.

Unknown content should not automatically be migrated.

The right answer is often to investigate before migrating.

The hidden problem

As I have already mentioned, documents are only one aspect of the migration process.

The other part is the information surrounding those documents: metadata.

Legacy ECM systems often contain years of accumulated classifications, fields, document types, folders and custom properties.

Some of these are useful and some aren’t. Sometimes they were created for a specific department, others were created because the old system required them.

Some have values such as:

• Other
• Miscellaneous
• Unknown
• N/A
• Old
• New
• Final
• Final_final

Then, as with documents, there are fields that nobody remembers either.

Yet during migration, there is often pressure to map everything.

• Old field → new field.
• Old document type → new document type.
• Old folder → new folder.
• Old classification → new classification.

This looks like the secure way to proceed. However, this can result in something worse than a bad legacy system.

A new system that incorporates a legacy model that has been successfully migrated.

Ultimately, no solution or improvement has been offered, the new ECM has the same issues as the previous one.

Do not migrate the structure, migrate the meaning

This is one of the most important lessons I have learned from working with various ECM environments.

A migration does not necessarily have to reproduce the inherited structure.

The important thing is to preserve the information.

Imagine a legacy repository containing: Customers / Switzerland / Active / Contracts / 2026

This folder tells you something, but it also tells you very little.

In a metadata-driven environment, the same document could be described using properties such as:

• Document class = Contract
• Customer = Customer X
• Country = Switzerland
• Status: Active
• Contract year: 2026

The document is not dependent on a particular folder structure.

It has context.

This context can be used for:

• Search
• Views
• Security.
• workflows.
• Reporting.
• Automation
• AI

This is one of the reasons I prefer working with a metadata-driven ECM solution such as M-Files.

However, there is an important warning here: M-Files won’t magically clean up your information.

Migrating bad information with bad metadata simply creates a better system for finding bad information.

The technology can support a better information model, but it cannot make decisions for you.

Migration is not a copy operation

I think it is important to distinguish between technical and information migration.

Technical migration asks: “Can we move the documents successfully?”

An information migration asks: “Are we moving the right information, with the right context, into the right structure?”

These are completely different questions.

It is possible to have a technically perfect migration and still have a terrible outcome.

Don’t forget the relationships

Another aspect that is often overlooked during migration is relationships.

Documents rarely exist in isolation, for example:

• A project document belongs to a project.
• An invoice belongs to a supplier, and potentially to a contract.
• An employee document belongs to an employee.

When migrating, we often focus on the document itself.

However, the more important question may be: “Did the context arrive with it?”

If not, you have migrated the document but lost part of its meaning.

This is particularly important when transitioning from folder-centric repositories to object and metadata-driven approaches.

The goal isn’t simply to preserve files, it’s to preserve information and context.

An opportunity to reduce information debt

This is probably the most important link to my previous article on information debt.

Information debt builds up when organizations repeatedly postpone decisions about their information.

Migration offers a rare opportunity to ask the right questions:

• What do we have?
• What do we need?
• Who owns it?
• How long should we keep it?
• What does it mean?
• How should it be classified?
• Who should have access?
• What can we get rid of?

These questions make migration painful.

But they also make migration valuable.

If you only use migration to move information, you miss the opportunity.

Instead, you can use it to reduce your information debt.

Do not migrate your history

Every ECM migration eventually raises an uncomfortable question:

Does this information still deserve to exist?

The goal shouldn’t be to recreate yesterday’s information environment using tomorrow’s technology. Rather, it should be to move the information that the organization still needs, retain what must be kept and remove what no longer has a reason to exist.

After all, technology can move millions of documents remarkably quickly. The real migration project is deciding which of those documents deserve to survive.

A successful migration isn’t about moving everything.

It’s about starting tomorrow with less information debt than you have today.

So, please stop asking: “How are we going to migrate everything?”

Ask a very different question: “What is actually worth taking with us?”

That’s where a successful ECM migration begins.

L’article ECM Migration: The moment you inherit everyone’s mess est apparu en premier sur dbi Blog.