After working on several Oracle Fusion Middleware upgrades, involving WebLogic Server, ADF, Oracle Forms, Oracle Reports, Oracle HTTP Server, and Identity Management components, I have noticed that the most challenging issues rarely come from the installation itself.

They usually come from existing configurations, dependencies, database schemas, or assumptions inherited from the previous environment.

Oracle provides comprehensive upgrade documentation, but every environment has its own history and technical constraints.

Here are six areas I now pay particular attention to when preparing and executing an Oracle Fusion Middleware 12c to 14c upgrade.

1. Compatibility is more than an Operating System check

Before starting an upgrade, the first challenge is understanding the full compatibility matrix.

Oracle FMW 14.1.2 introduces a new technology baseline, including Java 17, which can expose dependencies that worked perfectly under 12c.

Operating system certification is only one part of the equation.

The complete assessment should cover:

• Operating system and database versions
• JDK certification and compatibility
• Middleware component versions and interoperability
• Third-party libraries and custom Java applications
• JDBC drivers, authentication providers, and external integrations

A particularly important point is to inspect custom Java code for dependencies on internal WebLogic APIs or libraries whose behavior may have changed.

My recommendation: Do not limit your compatibility assessment to Oracle’s certification matrix. Include the applications and integrations that depend on the middleware.

A supported platform does not automatically mean a compatible application.

2. Database schemas: Small configuration differences, major consequences

One of the most important lessons concerns the relationship between WebLogic domains and their RCU schemas.

For example, Oracle FMW 14c introduces an important distinction between the following database users:

• <PREFIX>_WLS
• <PREFIX>_WLS_RUNTIME

In environments where WLSSchemaDataSource still uses the runtime schema, the configuration must be reviewed before upgrading.

The Upgrade Assistant expects the appropriate WLS schema owner. A mismatch can prevent the schema upgrade from completing successfully.

Oracle explicitly documents this requirement, but it is easy to overlook when working with multiple domains and different RCU prefixes.

Beyond this specific example, I systematically verify schema ownership, registry versions, database connectivity, account status, and the consistency between configured data sources and database metadata.

My recommendation: Treat database schema validation as an independent upgrade workstream, not merely a preliminary check performed by the Upgrade Assistant.

3. Readiness checks are essential, but they are not a guarantee

Oracle’s Upgrade Assistant provides readiness checks to identify potential upgrade problems.

They are extremely valuable and should be executed before the actual migration.

However, a successful readiness check does not guarantee that the upgrade will succeed.

Some issues only appear during schema modification, domain reconfiguration, server startup, or application initialization.

For this reason, I distinguish between three levels of validation:

• Pre-upgrade readiness: Are the schemas and configurations eligible for upgrade?
• Technical upgrade validation: Have the schemas and domains been successfully upgraded?
• Functional validation: Do applications, integrations, security services, and business processes actually work?

My recommendation: Never interpret a successful readiness check, or even a successful domain reconfiguration, as proof that the environment is operational. The real validation begins when the applications start communicating with their dependencies.

4. Domain reconfiguration deserves its own risk assessment

Domain reconfiguration is one of the most sensitive phases of an Oracle FMW upgrade.

The Reconfiguration Wizard modifies existing domain configurations to align them with the target middleware version. These changes can affect security settings, JDBC resources, deployment configurations, and product-specific components. The challenge becomes greater in environments containing multiple domains, custom configurations, or legacy integrations.

One important consideration is that domain reconfiguration cannot simply be reversed. Oracle explicitly recommends restoring the original domain from a backup if reconfiguration fails.

My recommendation: Take complete, verified backups of the Domain Homes and document custom configurations before starting reconfiguration. More importantly, treat schema upgrade and domain reconfiguration as coordinated operations, with clearly defined recovery points. Rollback must be designed before the upgrade begins, not after something goes wrong.

5. Patch strategy matters as much as the target version

Installing Oracle FMW 14c does not necessarily mean the environment is ready to be upgraded. Depending on the products involved, additional Bundle Patches, Stack Patch Bundles, or product-specific fixes may be required or recommended.

This is particularly important in environments combining components such as:

• WebLogic Server and ADF
• Oracle Access Management
• Oracle Internet Directory
• Oracle HTTP Server
• Oracle Reports

The objective is not simply to install the latest available patch. It is to establish a consistent, certified, and supportable patch baseline across all Oracle Homes. An incomplete or inconsistent patch strategy can introduce unnecessary complexity into an already sensitive migration.

My recommendation: Define and validate the target patch inventory before executing the upgrade. Keep installation, patching, schema upgrade, and domain reconfiguration as distinct, controlled phases.

6. The most important upgrade test is not whether WebLogic starts

A running AdminServer is encouraging, but it proves very little about the overall application platform. The same applies to Managed Servers reporting a RUNNING state.

A complete post-upgrade validation must consider the entire application chain:

• JDBC connectivity and database transactions
• Authentication, authorization, and identity integrations
• Application deployments and critical business transactions
• HTTP routing, SSL/TLS, and reverse proxy integration
• Scheduled processes, reporting services, and external dependencies

This is particularly relevant in environments where ADF applications, Oracle Reports, and Identity Management components depend on one another. It is entirely possible to have a technically healthy WebLogic domain while critical application functionality remains unavailable.

My recommendation: Define application-level acceptance criteria with the customer before scheduling the upgrade. The upgrade is complete when the expected business services have been validated, not when the last Managed Server reaches the RUNNING state.

Final Thoughts

After several Oracle Fusion Middleware upgrade projects, one conclusion has become clear to me: the actual upgrade commands are rarely the most difficult part of the job.

The real complexity lies in understanding the existing architecture, identifying hidden dependencies, anticipating compatibility issues, and controlling the sequence of changes.

Oracle’s documentation provides the technical foundation. Experience helps identify where additional verification is necessary.

A successful migration from 12c to 14c is not measured only by the absence of errors during the upgrade. It is measured by the ability to deliver a stable, supportable, and fully functional platform with a controlled operational impact.

Related blogs

L’article Oracle Fusion Middleware 12c to 14c: 6 Critical Lessons est apparu en premier sur dbi Blog.

The root cause was not an incorrect OAM registration or a missing configuration file. It was a stale ObAccessClient.xml in the OHS/WebGate runtime cache.

WebGate Initialization Failure After Upgrade

After upgrading OHS and WebGate to 14.1.2, the application returned HTTP 500 errors. The WebGate logs reported:

Exception thrown during WebGate initialization

The Access Server has returned a fatal error with no detailed information.

Additional errors included:

OAP over REST libcurl operation failed

These messages initially suggested an OAM connectivity or agent configuration issue.

However, the OAM endpoint was reachable:

curl -sv \
  http://<oam-host>:14100/iam/access/binding/api/v10/oap \
  -o /dev/null

Response:

HTTP/1.1 405 Method Not Allowed

This confirmed HTTP connectivity to the endpoint, although it did not validate the complete OAP authentication exchange.

The next step was to verify whether WebGate was using the expected configuration.

Root Cause Analysis: Active Configuration vs. Runtime Cache

WebGate uses ObAccessClient.xml and wallet files for its OAM agent configuration. I identified three relevant locations:

# OAM-generated configuration
$IDM_DOMAIN/output/<agent_name>/

# Active WebGate configuration
$ADF_DOMAIN/config/fmwconfig/components/OHS/instances/ohs1/webgate/config/

# WebGate runtime cache
$ADF_DOMAIN/servers/ohs1/cache/

First, I compared the SHA-256 checksums of the OAM-generated files against the active OHS configuration.

All three files were identical between the OAM-generated configuration and the active OHS configuration.

This was important: the deployed WebGate configuration and wallets were consistent with OAM.

I then compared the active ObAccessClient.xml with the cached version.

sha256sum \
  $ADF_DOMAIN/servers/ohs1/cache/ObAccessClient.xml \
  $ADF_DOMAIN/config/fmwconfig/components/OHS/instances/ohs1/webgate/config/ObAccessClient.xml

Actual output:

a1a5932a32b1c178461ab4cb373a14ff88446a6ad0cd9d2cdade90e5aecd4800  cache/ObAccessClient.xml
b110e7f0c8b153c1d17b7b79d4fc8273623fdb42dabafe0054d7c13e025d3f07  webgate/config/ObAccessClient.xml

The checksums are different. The cache directory also contained:

-rw-r----- 7618 Oct 20 16:26 ObAccessClient.xml
-rw-r-----    0 May 25  2021 ObAccessClient.xml.lck
-rw-r-----    0 May 25  2021 polltracking.lck

While different checksums do not automatically indicate corruption, the cached configuration was inconsistent with the verified active configuration. This became my primary suspect.

Resolution: Removing the Stale WebGate Cache

I stopped OHS using the domain management script:

$ADF_DOMAIN/bin/stopComponent.sh ohs1

Output:

Successfully Connected to Node Manager.
Killing server ohs1 ...
Successfully killed server ohs1

I confirmed that no OHS HTTP processes remained:

ps -ef | grep '[h]ttpd'

No processes were returned. Next, I preserved the existing cached configuration:

cd $ADF_DOMAIN/servers/ohs1/cache/
cp -p ObAccessClient.xml \
  ObAccessClient.xml.pre14c_20261023

I removed only the cached configuration and its associated lock file:

rm -f ObAccessClient.xml ObAccessClient.xml.lck

The active WebGate configuration and wallets were left untouched. I then restarted OHS:

$ADF_DOMAIN/bin/startComponent.sh ohs1

Output:

Successfully Connected to Node Manager.
Starting server ohs1 ...
Successfully started server ohs1 ...

An interesting observation occurred immediately after the restart. The initial directory listing showed that ObAccessClient.xml had not yet been recreated:

ObAccessClient.xml.pre14c_20261023
polltracking.lck

A subsequent check showed that WebGate had automatically regenerated its runtime cache:

-rw-r----- 7618 Oct 23 10:39 ObAccessClient.xml
-rw-r-----    0 Oct 23 10:39 ObAccessClient.xml.lck
-rw-r----- 7618 Oct 20 16:26 ObAccessClient.xml.pre14c_20261023

The newly generated files had fresh timestamps, while the backup preserved the original cached configuration.

Following this operation, WebGate initialized successfully and the protected application became accessible again through the expected authentication flow.

No modification to the active configuration, wallet replacement, agent re-registration or WebGate reinstallation was necessary.

Conclusion

This case demonstrates an important troubleshooting consideration after upgrading Oracle HTTP Server and WebGate.

Even when the active ObAccessClient.xml and wallet files match the configuration generated by OAM, an outdated WebGate runtime cache can still interfere with initialization.

For future Oracle FMW upgrades, comparing the active and cached WebGate configurations is a valuable diagnostic step when authentication fails despite apparently correct OAM integration.

Related blogs.

L’article Oracle OHS 14.1.2 Upgrade: Cache Broke Authentication est apparu en premier sur dbi Blog.

Context

During a recent Oracle Fusion Middleware upgrade from 12.2.1.4 to 14.1.2, I encountered an unexpected startup failure affecting Oracle Reports Server.

The installation, domain upgrade and reconfiguration had completed successfully. The following components were operational:

• WebLogic AdminServer
• ReportsTools
• WLS_REPORTS
• Node Manager

However, the standalone Reports Server components, RptSvr_1 and RptSvr_2, failed to initialize.

The underlying exception was reported as: REP-50125.

Oracle documents REP-50125 as a generic internal exception. The error code alone does not identify the failing subsystem.

In this case, the root cause was neither a WebLogic configuration problem nor a database connectivity issue.

It was caused by an unexpected interaction between REPORTS_JVM_OPTIONS and the internal Oracle Reports CORBA ORB initialization code.

Identifying the real exception

The Reports Server diagnostic logs exposed the following execution path:

java.lang.NullPointerException
    at java.util.concurrent.ConcurrentHashMap.putVal(
        ConcurrentHashMap.java:1011)
    at java.util.concurrent.ConcurrentHashMap.put(
        ConcurrentHashMap.java:1006)
    at java.util.Properties.put(
        Properties.java:1301)
    at oracle.reports.utility.Utility.getOrbProperties(
        Utility.java:2561)
    at oracle.reports.utility.Utility.createORB(
        Utility.java:2490)
    at oracle.reports.server.RWServer.startServer(
        RWServer.java:912)
    at oracle.reports.server.RWServer.jniMain(
        RWServer.java:289)

The important frame is:

oracle.reports.utility.Utility.getOrbProperties(
    Utility.java:2561)

The failure occurs when Oracle Reports initializes its ORB properties, before the Reports Server becomes operational.

A NullPointerException inside ConcurrentHashMap.putVal() strongly suggests that a null key or value is being inserted into a map that does not accept null entries.

The question was therefore straightforward:
Which ORB property was receiving a null value, and where did that value originate?

Investigating Oracle’s implementation with javap

To investigate the issue beyond the available log messages, I inspected the compiled Oracle Reports classes using the JDK javap utility.

The objective was to identify the implementation behind:

oracle.reports.utility.Utility.getOrbProperties()

The relevant commands were:

# Locate the Oracle Reports class
find "$ORACLE_HOME" -type f -name "*.jar" \
  -exec sh -c '
    for jarfile do
      jar tf "$jarfile" 2>/dev/null |
        grep -q "^oracle/reports/utility/Utility.class$" &&
        printf "%s\n" "$jarfile"
    done
  ' sh {} +

Once the corresponding JAR is identified:

javap -c -p -l \
  -classpath /path/to/oracle-reports.jar \
  oracle.reports.utility.Utility

To focus on the relevant methods:

javap -c -p -l \
  -classpath /path/to/oracle-reports.jar \
  oracle.reports.utility.Utility |
grep -A 100 -E \
  'getOrbProperties|extractOrbWaitTimeout'

The bytecode inspection revealed that getOrbProperties() reads the environment variable: REPORTS_JVM_OPTIONS

It then calls an internal method: extractOrbWaitTimeout(…)

The extracted value is subsequently assigned to the CORBA property:

com.sun.corba.ee.transport.ORBWaitForResponseTimeout

This provided the missing link between a JVM configuration variable and the ORB initialization exception.

The hidden behavior

The investigation exposed two different execution paths.

Case A: REPORTS_JVM_OPTIONS is not defined

When the environment variable is absent, Oracle Reports assigns a hardcoded fallback value: “999999999”

This behavior prevents a null timeout value from being inserted into the ORB properties.

Case B: REPORTS_JVM_OPTIONS is defined without the timeout parameter

Consider this configuration:

REPORTS_JVM_OPTIONS="-Xmx1024M"
export REPORTS_JVM_OPTIONS

The variable exists, so the fallback branch is not used.

Oracle Reports invokes extractOrbWaitTimeout() to search for: -DORBWaitForResponseTimeout=

However, that parameter is missing.

The method returns null.

The resulting behavior is equivalent to:

Properties properties = new Properties();

String timeout = null;

properties.put(
    "com.sun.corba.ee.transport.ORBWaitForResponseTimeout",
    timeout
);

This triggers the NullPointerException observed during ORB initialization.

The significant finding is that a variable containing valid JVM options can introduce an ORB initialization failure simply because one additional property is missing. It is an internal Oracle Reports configuration-parsing behavior.

These conclusions are based on the inspected Oracle Reports 14.1.2 implementation and should not be generalized to every Oracle Reports release or patch level without verification.

Implementing the correction

The correction was applied to the Oracle Reports environment configuration:

$ORACLE_HOME/reports/bin/reports.sh

The JVM options were updated as follows:

REPORTS_JVM_OPTIONS="-Xmx1024M -DORBWaitForResponseTimeout=999999999"
export REPORTS_JVM_OPTIONS

The timeout value was not chosen arbitrarily. It corresponds to the hardcoded fallback identified in Oracle’s bytecode.

The property value is expressed in milliseconds: 999999999 ms ≈ 11.57 days

This is a technical reproduction of the implementation’s fallback, not a general recommendation to configure such a long timeout for every environment.

Before restarting Reports Server, the configuration can be verified using:

grep -n "REPORTS_JVM_OPTIONS" \
  "$ORACLE_HOME/reports/bin/reports.sh"

Expected configuration:

REPORTS_JVM_OPTIONS="-Xmx1024M -DORBWaitForResponseTimeout=999999999"
export REPORTS_JVM_OPTIONS

The Reports Server was then restarted through Node Manager.

Result: RptSvr_1 started successfully after the correction.

This confirmed that the configuration change resolved the observed ORB initialization failure on that instance. The separate application-level report-generation tests and second-instance validation remain distinct checks.

Verifying the runtime configuration

After the restart, the effective environment should be checked against the actual Reports Server process.

First, identify the process:

ps -ef | grep -E 'rwserver|RWServer' | grep -v grep

Then inspect the inherited environment:

tr '\0' '\n' < /proc/<PID>/environ |
grep -E '^(REPORTS_JVM_OPTIONS|ORACLE_HOME|JAVA_HOME)='

Expected value:

REPORTS_JVM_OPTIONS=-Xmx1024M -DORBWaitForResponseTimeout=999999999

his check is important because the Node Manager startup environment may differ from an interactive shell.

A successful restart should also be followed by:

• Confirmation that the Reports Server remains running.
• Verification that the original exception no longer appears in new diagnostic entries.
• Validation of all Reports Server instances.
• Execution of representative report-generation requests.

Conclusion

This issue illustrates why troubleshooting Oracle Fusion Middleware upgrades sometimes requires going beyond configuration files and diagnostic logs.

The visible symptom was a generic REP-50125. The underlying failure was a NullPointerException in Utility.getOrbProperties(), caused by the interaction between REPORTS_JVM_OPTIONS and Oracle Reports’ internal ORB timeout extraction.

Oracle documentation describes the precedence of jvmOptions in rwserver.conf over the environment variable for Reports Engine startup. However, the implementation detail identified during this investigation is different. The getOrbProperties() method also consumes REPORTS_JVM_OPTIONS while constructing ORB configuration properties.

Bytecode inspection established the missing connection and identified the hardcoded fallback used by Oracle.

Adding the corresponding JVM property restored successful startup of the affected Reports Server instance.

Related blogs.

L’article Oracle Reports 14.1.2 REP-50125 – A Hidden REPORTS_JVM_OPTIONS Requirement est apparu en premier sur dbi Blog.

There was no obvious error message, and the wizard could not proceed.

The solution turned out to be a single Java parameter.

The Fix

Before launching the Configuration Wizard, disable the Java 2D XRender pipeline:

export _JAVA_OPTIONS="-Dsun.java2d.xrender=false"

Then launch the wizard:

$ORACLE_HOME/oracle_common/common/bin/config.sh -log=/u00/app/oracle/product/14c/reconfig_logs/oid_domain_20261019/reconfig_oid_domain_20261019.log -log_priority=ALL

In my environment, this resolved the issue immediately.

Why Does It Work?

Java graphical applications use different rendering pipelines on Linux, including X11 and XRender.

XRender is a Java 2D rendering pipeline that uses the X Render extension to accelerate certain graphical operations.

However, in remote graphical environments using SSH X11 forwarding, compatibility problems between the JVM, XRender, and the X server can sometimes cause graphical applications to freeze or become unresponsive.

Setting:

-Dsun.java2d.xrender=false

disables the XRender pipeline, allowing Java 2D to fall back to the traditional X11 rendering pipeline.

This avoids the problematic rendering path without modifying the Oracle installation or domain configuration.

Confirming the Fix

Initially, I tested two Java parameters together:

export _JAVA_OPTIONS="-Dsun.java2d.xrender=false -Dsun.java2d.opengl=false"

The wizard worked, but it was unclear which parameter made the difference.

During the reconfiguration of another domain, I repeated the test using only -Dsun.java2d.xrender=false.

The Configuration Wizard worked correctly again.

This confirmed that disabling XRender alone was sufficient in this environment. Disabling OpenGL was unnecessary.

Oracle Already Documents This Behavior

After resolving the issue, I checked Oracle’s official Java documentation and found that this behavior is already documented.

In its Java 2D Properties on Linux documentation, Oracle describes the sun.java2d.xrender property and explains that setting it to false disables the XRender pipeline and falls back to the basic X11 pipeline.

This is exactly the behavior we wanted to achieve:

-Dsun.java2d.xrender=false

Oracle does not specifically associate this parameter with the WebLogic Configuration Wizard freezing during domain reconfiguration. However, the documented rendering mechanism explains why this workaround can help in an environment using remote X11 forwarding.

Conclusion

If the Oracle WebLogic Configuration Wizard freezes when running through X11 forwarding on Linux, try disabling the Java 2D XRender pipeline before investigating more complex causes.

A single JVM parameter is enough:

export _JAVA_OPTIONS="-Dsun.java2d.xrender=false"

This workaround was successfully validated during two domain reconfigurations.

It does not prove the exact underlying XRender incompatibility, but it demonstrates that the rendering pipeline was a contributing factor.

After completing the configuration, remove the temporary setting:

unset _JAVA_OPTIONS

Related blogs

L’article Oracle WebLogic Configuration Wizard Freezes on Linux est apparu en premier sur dbi Blog.

I was already aware of the documented change to WLSSchemaDataSource in 14.1.2. Before modifying the existing domain, however, I wanted to establish whether the failure matched that prerequisite and whether the readiness checks exposed anything else that needed attention.

I am sharing the findings because the same message can appear in other Fusion Middleware environments. The useful part is connecting what the assistant selected, what the database actually contains, and what Oracle expects for the upgrade.

The findings below were collected before changing the data sources, upgrading the schemas, or reconfiguring the domains. Schema prefixes are retained to make the evidence readable; database hosts and connection strings are omitted.

The Oracle requirement

Oracle documents this prerequisite in Upgrading to the Oracle Fusion Middleware Infrastructure 14c, under Verify the Database User for the WLSSchemaDataSource Data Source.

For a domain-based schema upgrade, Upgrade Assistant uses WLSSchemaDataSource to obtain schema information. If it references <PREFIX>_WLS_RUNTIME, Oracle requires changing the user to <PREFIX>_WLS. The WLS schema page must also identify the correct owner.

Oracle recommends making the data source change through the existing 12c Administration Console so that subsequent upgrade and reconfiguration screens receive the appropriate values. In 14.1.2, the runtime account has a separate data source, WLSRuntimeSchemaDataSource, created during domain reconfiguration.

The intended mapping is therefore:

Knowing this requirement gave me a strong starting point. I still wanted evidence from the environment before applying it.

What the ADF readiness screen actually showed

The readiness check reached 100%, with the WebLogic schema marked as failed. OPSS, MDS, IAU and STB were marked ready for upgrade.

The failed WebLogic entry referenced DEVADF_WLS_RUNTIME. That was the first useful clue: the account selected for the check was the runtime account.

The screen did not establish an ADF application defect or a failed schema migration. It showed a failed readiness check for a shared infrastructure component. That distinction matters when deciding where to investigate.

Checking the database before changing the configuration

The database checks established the following:

For readers reproducing these checks, the following read-only queries show the relevant account and registry information. Execute them with an appropriately privileged account in the database service or container used by the domain:

SELECT username, account_status
FROM dba_users
WHERE username IN ('DEVADF_WLS', 'DEVADF_WLS_RUNTIME')
ORDER BY username;

SELECT comp_id, owner, version, status
FROM schema_version_registry
WHERE comp_id = 'WLS'
AND owner IN ('DEVADF_WLS', 'DEVADF_WLS_RUNTIME');

These queries are provided as a reproducible check; the table above summarizes the observed findings rather than reproducing a terminal transcript.

The registered component owner and the data source user differed in exactly the way described by Oracle. Both accounts existed and were open, so a missing or locked account did not explain the observed state. An OPEN account does not, by itself, prove that its configured password is correct or that a JDBC connection succeeds. Similarly, VALID in the registry is not a complete object-integrity or application-health check.

Those limits matter: the evidence supported the documented owner mismatch without justifying broader claims about the entire environment.

Applying the correction

The following steps describe the correction to apply to the existing 12c domain before continuing with the upgrade.

1. Open the existing WebLogic 12c Administration Console. Navigate to Services → Data Sources → WLSSchemaDataSource. If configuration locking is enabled, select Lock & Edit.
2. Update the database user. Under Configuration → Connection Pool, change the user property from DEVADF_WLS_RUNTIME to DEVADF_WLS. Keep the existing JDBC URL unless a separate connection issue requires changing it.
3. Set the corresponding credentials. Enter and confirm the password for DEVADF_WLS. Do not assume that the component owner and runtime account share the same password.
4. Save and activate the change. Click Save, then Activate Changes where applicable. Follow any restart requirement displayed by the console. With the target server running, use Monitoring → Testing to test the data source connection on its target. A successful connection validates connectivity and authentication; readiness still needs to be checked separately.
5. Rerun Upgrade Assistant in readiness mode. Start the tool from the new 14.1.2 Oracle Home:"$ORACLE_HOME/oracle_common/upgrade/bin/ua" -readiness Here, ORACLE_HOME must point to the target 14.1.2 installation. Select the same domain and check that the WLS Schema page identifies DEVADF_WLS. Review the new report and resolve any remaining failures before proceeding with the schema upgrade.

This change applies to WLSSchemaDataSource. Keep the DEVADF_WLS_RUNTIME database account: Oracle uses it with the separate WLSRuntimeSchemaDataSource introduced in 14.1.2.

Conclusion

The findings were consistent with the prerequisite I had already identified in Oracle’s documentation. Checking the database and domain configuration first gave me a clear basis for applying the change. The next step was to update WLSSchemaDataSource to use DEVADF_WLS with the corresponding credentials, then rerun readiness before proceeding with the upgrade.

If you encounter the same error, comparing the selected schema user, the data source configuration and the registered component owner is a useful starting point. It helps establish whether the documented change applies to your environment before you modify it.

See related blogs

L’article Oracle FMW Upgrade – ADF Failed (INVALID_WLS_COMPONENT_SCHEMA_OWNER) est apparu en premier sur dbi Blog.

Yet report generation failed.

Even a simple request to display the Reports job list returned an error:

REP-50125: An internal exception occurred:
java.security.PrivilegedActionException:
javax.security.auth.login.LoginException:
java.lang.NoClassDefFoundError:
Could not initialize class
oracle.security.jps.service.policystore.info.BuiltInFunctionsHandler

The investigation led somewhere else: a Reports Server JVM running with a maximum heap of only 256 MiB, while the equivalent environment used 1 GiB.

Here is how we moved from a misleading symptom to an evidence-based diagnosis.

Separate process availability from application health

The initial service check showed every component as running:

AdminServer             is running.
WLS_REPORTS             is running.
RptSvr_app-dev_1         is running.
RptSvr_app-dev_2         is running.

This established that the processes existed. It did not establish that they could authenticate requests, evaluate security policies, or execute reports.

The failing showjobs request provided a more useful functional check. It showed that Reports could receive a request but failed while processing it.

The first lesson is straightforward:

A running process is an infrastructure observation, not an application health check.

Find the first exception, not the most frequently repeated one

The browser displayed:

NoClassDefFoundError: Could not initialize class ...

The wording matters. A class that cannot initialize is not necessarily a missing class.

I searched the Reports Server diagnostic logs for the first occurrence and the exceptions preceding it:

grep -n -B 30 -A 70 -E \
  'BuiltInFunctionsHandler|ExceptionInInitializerError|OutOfMemoryError|Caused by:' \
  rwserver_diagnostic.log

Earlier in the same process lifetime, the log contained:

ava.lang.OutOfMemoryError: GC overhead limit exceeded
...
oracle.security.jps.service.policystore.info.BuiltInFunctionsHandler.<clinit>

The sequence was now clear:

• The JVM encountered memory exhaustion during initialization of a security class.
• That initialization failed.
• Subsequent requests reported that the class could not initialize.

The security error was therefore a consequence of the earlier memory failure.

The first exception explained the incident; the later exceptions described its persistent symptoms.

Identify the JVM that actually failed

An Oracle Reports installation involves several processes with different responsibilities:

Each can have its own JVM configuration.

Increasing the heap of WLS_REPORTS would not necessarily help a failure occurring inside the standalone rwserver process.

I first mapped the processes:

ps -ww -eo pid,user,rss,args |
  grep -E '[b]in/rwserver|[w]eblogic.Name=WLS_REPORTS|[o]racle.reports.engine.RWEngine'

Then I checked which JVM library the affected native rwserver process had loaded:

grep 'libjvm.so' /proc/<REPORTS_PID>/maps

This revealed another useful detail: the JVM loaded from the Oracle home differed from the JDK referenced by JAVA_HOME.

The Reports startup script populated LD_LIBRARY_PATH with the bundled JVM location.

That distinction did not prove a Java compatibility problem. It established which JDK tools we should use and prevented us from diagnosing the wrong runtime.

For a running process, inspect what is loaded not just what the shell environment suggests.

Measure the heap instead of inferring it from available RAM

The operating system reported more than 20 GiB of available memory.

That did not contradict the Java memory error. A JVM can exhaust its configured heap while the host still has substantial free RAM.

Using the matching JDK, I inspected the effective JVM flags:

"$REPORTS_JDK/bin/jcmd" <REPORTS_PID> VM.flags

The relevant output was:

-XX:InitialHeapSize=268435456
-XX:MaxHeapSize=268435456

The maximum heap was 256 MiB.

Next, I sampled garbage collection statistics:

"$REPORTS_JDK/bin/jstat" -gcutil <REPORTS_PID> 1000 10

The affected process showed:

The Full GC counter did not increase during that short sample. However, the nearly full heap, accumulated collection activity, and recorded OutOfMemoryError together provided strong evidence of heap exhaustion.

A single memory percentage would have been insufficient. The diagnosis came from combining the logs, effective configuration, and runtime measurements.

Compare with a working environment

I inspected the environment inherited by the working process:

tr '\0' '\n' < /proc/<REPORTS_PID>/environ |
  grep -E '^(REPORTS_JVM_OPTIONS|JAVA_TOOL_OPTIONS|_JAVA_OPTIONS)='

The reference environment contained:

REPORTS_JVM_OPTIONS= -Xmx1024M

Comparing the Reports environment scripts then revealed the missing lines in the affected domain’s reports/bin/reports.sh:

REPORTS_JVM_OPTIONS=" -Xmx1024M"
export REPORTS_JVM_OPTIONS

Validate the correction

The client aligned the Reports memory configuration with the reference environment and subsequently confirmed that report generation and application checks worked again.

A separate question remained: why had the problem become visible after this particular operating system upgrade and restart?

We did not have the previous process’s effective heap settings or a historical memory profile. We therefore could not prove whether the restart changed the inherited environment, whether memory demand had increased, or whether another condition exposed an existing limitation.

The timing made the OS upgrade a reasonable starting hypothesis. It did not make it the demonstrated root cause.

Conclusion

This incident followed a useful progression. The most valuable habit was to follow the evidence backward, from the visible error to the first failure, and then forward again, from the configuration difference to a verified recovery.

When critical applications stop working, you need more than a restart… You need a team that understands the platform. At dbi services, our Service Desk combines responsive support with deep technical expertise to investigate incidents, identify their causes, and guide recovery with clear, evidence-based recommendations.

Looking for expert support for your Oracle Fusion Middleware environment? Contact us to discuss how we can help.

More related blogs

L’article Oracle Reports: REP-50125: An internal exception occurred est apparu en premier sur dbi Blog.

These discussions usually involve critical environments based on Oracle WebLogic Server, Oracle Forms, and sometimes Oracle Reports. The objective is rarely just to “move to the cloud”. It is about understanding what can realistically be migrated, what should be modernized, what must be validated, and where the main technical, operational, and licensing risks are.

Based on these discussions, I would like to share a few practical considerations that may be useful when evaluating a cloud migration strategy for Oracle Fusion Middleware workloads.

Lift and shift is not a shortcut, it is a controlled migration strategy

For Oracle WebLogic, Forms and Reports platforms, a cloud migration should not automatically be presented as an application modernization project.

In many cases, the first realistic step is a lift-and-shift migration. The goal is not to rewrite Forms modules, redesign the full application, or replace every legacy component immediately. The goal is to rebuild a controlled target platform in the cloud while preserving the application behavior as much as possible.

However, lift and shift does not mean copying virtual machines and hoping that everything will work.

A serious migration requires a detailed comparison between the existing platform and the target one: WebLogic domain configuration, Managed Servers, Datasources, Wallets, Certificates, JVM version, Patch level, Forms runtime configuration, Reports configuration, File system dependencies, Fonts, Printers, Batch jobs, Monitoring, Backup, Restart procedures, and Operational scripts.

Oracle documents several WebLogic migration approaches to OCI, including manual deployment, WebLogic Deploy Tooling, and WLST-based deployment scripts targeted to a new domain. Oracle also describes WebLogic Server for OCI as a way to provision a WebLogic domain in OCI more quickly than a traditional on-premises setup.

This is why lift and shift should be seen as an engineering exercise, not as a low-effort infrastructure move.

Why OCI is naturally aligned with Oracle Fusion Middleware

It would not be accurate to say that Oracle Fusion Middleware can only run on OCI. Oracle’s own cloud licensing policy lists AWS, Azure, and Google Cloud Platform as “Authorized Cloud Environments” for specific Oracle software licensing rules. The same policy also explains how vCPUs must be counted in these environments and states that the Oracle Processor Core Factor Table is not applicable there.

So the real point is not that other clouds are impossible. The point is that OCI is naturally aligned with Oracle Fusion Middleware workloads.

For WebLogic, Oracle provides OCI-specific deployment models, Marketplace-based options, and documented migration patterns. For Kubernetes-based deployments, Oracle also documents WebLogic Server for OKE using a Marketplace stack, including lifecycle-management components such as Jenkins and the WebLogic Kubernetes Operator.

From an operational and licensing perspective, OCI also gives Oracle customers options such as BYOL and Universal Credits-based models. Oracle’s Support Rewards FAQ states that BYOL services on the Universal Credits Model rate card are eligible for the Support Rewards program.

This does not remove the need for a proper license review. But it does mean that for Oracle middleware workloads, OCI often provides a more Oracle-native discussion: infrastructure, middleware, licensing, support model, and target architecture can be addressed within the same ecosystem.

Oracle Reports must be treated as a specific risk

For Oracle Forms and WebLogic, a structured lift-and-shift approach to OCI can be a realistic option. For Oracle Reports, the discussion must be more careful.

Oracle Reports is still present in many enterprise Forms platforms, but it has technical characteristics that must be validated before any cloud migration. Reports cluster members or clients use multicast to discover other nodes, and that there is no workaround for using multicast in that context.

This is a critical point because multicast is generally not available in all cloud network architectures.

In practical terms, Oracle Reports should not be treated as a simple checkbox in the migration plan. It requires a dedicated technical validation: Reports Server configuration, naming service, generated output, cache behavior, fonts, printers, file paths, batch execution, restart behavior, and integration with the Forms application.

Conclusion

The conclusion is therefore simple: WebLogic and Forms can be good candidates for a controlled lift-and-shift migration to OCI. Oracle Reports must be assessed separately, tested carefully, and challenged architecturally.

OCI can reduce complexity for Oracle Fusion Middleware migrations, but it does not remove the need for deep product expertise. A successful migration is not only a cloud project, it is a WebLogic, Forms, Reports, licensing, operations, and validation project.

If you are planning to migrate an Oracle Fusion Middleware platform to the cloud, or if you simply want to assess whether such a migration is realistic for your environment, feel free to contact me. I would be happy to help you challenge the architecture, identify the risks, and define a pragmatic migration approach.

More related blogs here.

L’article Oracle Fusion Middleware to the Cloud est apparu en premier sur dbi Blog.

In this context, the question is not simply: “Should we replace JBoss EAP?”

The real question is: “How can we modernize these workloads without creating unnecessary risk?”

JBoss EAP still has a future, but its role is changing. It should no longer be seen only as a traditional application server running on long-living virtual machines. Its future is as a supported enterprise Java runtime that can help customers modernize progressively, especially when moving toward OpenShift.

But in large enterprise environments, technology relevance is not measured only by hype. It is measured by stability, supportability, lifecycle, risk, operational maturity, and the ability to run critical workloads for many years.

From that perspective, JBoss EAP still has a future!

Not because every new Java application should be deployed on a traditional application server. That would not be realistic. Its future is different: JBoss EAP remains a strategic runtime for enterprise Java systems that require continuity, long-term support, and controlled modernization.

The future of EAP is linked to application lifecycle, not hype

Many enterprise applications are not replaced every three years. They live for ten, fifteen, sometimes twenty years. They support billing, public administration, banking processes, insurance workflows, logistics, healthcare platforms, or internal business systems.

For these applications, the most important question is not whether the runtime is fashionable. The question is whether the platform remains supportable, secure, upgradeable, and aligned with the enterprise roadmap.

This is where JBoss EAP has value.

Its future will be driven by customers who need a stable enterprise Java platform for existing workloads, not by teams looking for the lightest runtime for a new microservice.

Red Hat’s direction: continuity with modernization

Red Hat’s strategy around JBoss EAP is not to freeze the platform in the past. The move to EAP 8 and Jakarta EE 10 shows that the product continues to follow the evolution of enterprise Java.

This is a major point. Jakarta EE 10 is not just a cosmetic version change. For many customers, it forces a real application review, especially because of the namespace migration from javax.* to jakarta.*, dependency updates, library compatibility, and testing effort.

That means the future of JBoss EAP is not passive. Customers will still need migration projects, compatibility assessments, application testing, dependency cleanup, and platform upgrades.

In parallel, Red Hat also positions EAP in the OpenShift ecosystem, with supported container images and deployment approaches for OpenShift. This confirms that EAP is not limited to the traditional VM model anymore.

The message is clear: JBoss EAP is evolving, but in an enterprise way. It does not try to become Quarkus. It keeps its role as a supported Jakarta EE application platform while becoming more compatible with modern operational models.

The real challenge: reducing dependency on the application server

Some applications depend too much on the application server. Over the years, many systems have accumulated runtime-specific assumptions:

• server-specific modules,
• configuration hidden in the application server,
• hardcoded JNDI names,
• deployment scripts known only by operations teams,
• local filesystem dependencies,
• legacy security mechanisms,
• clustering assumptions,
• old libraries bundled with the application.

These are not only technical details. They are modernization blockers.

A good EAP strategy should therefore not only be “upgrade the server.” It should also reduce unnecessary coupling between the application and the runtime.

The objective is not to leave EAP. The objective is to make the application more portable, more testable, easier to deploy, and less dependent on hidden server configuration.

This is where consulting expertise brings value: identifying which dependencies are legitimate enterprise platform usage, and which ones are accidental complexity created by years of operations.

Conclusion

The future of JBoss EAP is not about competing with every modern Java runtime.

Its future is to remain a reliable enterprise platform for applications that still need the Jakarta EE model, long-term support, and controlled modernization.

At the same time, customers should not ignore Quarkus, Spring Boot, or other modern runtimes. But Quarkus does not replace JBoss EAP in all cases. The right strategy is not to force a single runtime everywhere. The right strategy is to understand the application lifecycle, reduce unnecessary runtime coupling, and choose the platform that matches the workload.

In the future, JBoss EAP will remain strategic wherever enterprise Java continuity, supportability, and risk control matter.

JBoss EAP Blogs

L’article JBoss EAP: From Traditional Application Server to Modern Enterprise Runtime est apparu en premier sur dbi Blog.

Yet modern distributed systems introduced a shift in mindset: monitoring is no longer enough. Enter observability, not as a buzzword, but as an operational necessity.

This blog explains the difference clearly, with practical engineering examples and a real-world perspective.

Monitoring: Knowing When Something Is Wrong

Traditional monitoring answers a very specific set of questions:

• Is the service up?
• Are resources within acceptable limits?
• Are error rates crossing thresholds?
• Are we respecting SLAs?

Monitoring is fundamentally about predefined signals.

What Monitoring Does Well

Monitoring excels at:

• Incident detection
• Alerting and operational response
• Capacity tracking and trend analysis
• Infrastructure health visibility

Example: Classic Monitoring Workflow

You monitor an API using:

• Prometheus metrics (http_requests_total, latency histograms)
• Grafana dashboards
• Alert rule: 5xx error rate > 2% for 5 minutes

You immediately know something is wrong. But the next question appears: Why is it wrong?

And this is where traditional monitoring starts to hit limits.

The core limitation

Monitoring depends on:

• Metrics you anticipated
• Thresholds you defined in advance
• Scenarios you already imagined

When failures fall outside those assumptions, which is common in distributed systems, troubleshooting becomes exploratory and slow.

Observability: Understanding Why Something Is Wrong

Observability is not just more dashboards. It is the ability to understand internal system behavior from external outputs, even when you don’t already know what to look for.

Instead of asking: “Is the system healthy?”

Observability lets you ask:

• Why are users experiencing latency?
• Which service introduced the regression?
• What changed right before the incident?
• Which request path is affected?

This is a fundamentally different operational capability.

Modern observability relies on correlated telemetry:

• Metrics – aggregated system behavior
• Logs – detailed contextual events
• Traces – end-to-end request journeys

Individually, they are useful. Together, they provide causal understanding.

Practical Evolution: From Monitoring to Observability

You don’t replace monitoring, you evolve it. A pragmatic path:

1. Define meaningful metrics
   Focus on user-centric signals: latency, traffic, errors, saturation.
2. Structure logs properly
   Include fields like service, environment, trace_id, deployment_version.
3. Instrument distributed tracing
   Start with critical journeys (checkout, authentication, search).
4. Enable cross-linking
   Alert -> metric -> trace -> logs must be one flow.
5. Shift toward SLO-based thinking
   Alert on user impact, not raw CPU.

Conclusion

Observability isn’t about collecting more data, it’s about turning telemetry into understanding.

And that is where mature engineering organizations truly differentiate themselves: not by how much they monitor, but by how quickly and confidently they can understand, diagnose, and resolve complex system behaviors in production.

If your organization is starting this journey or aiming to accelerate its maturity, we would be happy to help. From strategy and tooling selection to implementation and operational enablement, feel free to reach out and connect with us.

Related blogs : Zabbix, monitoring, Elastic

L’article Observability vs Monitoring est apparu en premier sur dbi Blog.

The answer is not to rush into a rewrite. In many cases, the most realistic and valuable first step is to move the existing JBoss EAP workloads to OpenShift, while modernizing the way they are built, configured, deployed, monitored, and operated.

This is not a simple lift-and-shift. It is a controlled modernization of the runtime and operating model.

The customer scenario: existing JBoss EAP, new platform expectations

A typical customer already has several applications running on JBoss EAP. Some are packaged as WAR files, others as EAR files. They use datasources, JNDI names, security realms or Elytron, JMS, transactions, batch jobs, custom modules, shared libraries, and sometimes clustering or HTTP session replication.

At the same time, the infrastructure team wants to move to Kubernetes or OpenShift to standardize deployments, improve automation, reduce manual operations, and align with a container-based platform strategy.

In this situation, recommending an immediate migration to another runtime such as Quarkus would often be too aggressive. The customer would have to manage two major changes at the same time: a platform migration and an application architecture migration.

A better approach is to keep JBoss EAP as the first target runtime and move to OpenShift.

Red Hat provides specific documentation for using JBoss EAP on OpenShift, including build and deployment approaches based on Helm charts for EAP 8.x. This matters because the customer is not moving to an unsupported or exotic model. They are moving an existing Red Hat middleware platform into a Red Hat container platform.

Why JBoss EAP still makes sense on OpenShift

JBoss EAP remains a strong choice when the application already depends on enterprise Java capabilities. For existing applications, EAP provides continuity. The customer can keep familiar application patterns which reduce risk compared to a full rewrite.

The real value of OpenShift is not simply “running EAP in a container”. The value comes from changing the operational model.

Instead of manually maintaining application servers on virtual machines, the target model should be:

• One application image per application or application group,
• Configuration externalized through ConfigMaps, Secrets, and environment variables,
• Immutable deployments,
• Health checks and probes,
• Centralized logs,
• Controlled resource requests and limits,
• Automated deployments through CI/CD or GitOps,
• Rolling updates and rollback capability,
• Platform-level security and image governance.

Red Hat documents that EAP applications can be built from compiled WAR or EAR archives and deployed onto a JBoss EAP runtime stack for OpenShift. That is important for real-world migration projects because many customers cannot rewrite their packaging model immediately.

For EAP 7.4 customers, there are also Red Hat-provided OpenShift runtime images, such as the JBoss EAP 7.4 OpenJDK 11 runtime image, described as a base image to run an EAP server and application.

The recommendation should therefore be clear: do not reproduce the old VM model inside Kubernetes. Use the move to OpenShift to clean up the runtime model.

A good migration should review:

• EAP and Java versions,
• WAR/EAR packaging,
• Standalone.xml or domain configuration,
• Datasources and JDBC drivers,
• JMS usage and MDBs,
• Security integration,
• Filesystem dependencies,
• Clustering and session handling,
• JVM tuning,
• Startup and shutdown behavior,
• Logging format;monitoring and alerting,
• CI/CD pipeline,
• Backup and recovery constraints.

Some applications will be easy to containerize. Others will need technical adaptation. A few may not be good candidates without refactoring.

Recommended approach

For a customer already using JBoss EAP, my recommendation is:

Start with JBoss EAP on OpenShift. Stabilize the container model. Then identify which applications or components deserve deeper modernization.

This approach avoids unnecessary risk. It gives the customer a supported and realistic path to Kubernetes while preserving the value of the existing EAP investment.

OpenShift does not automatically make an application cloud-native. The right approach is to understand the existing EAP landscape, modernize the operating model first, and only then decide where a lighter, Kubernetes-native runtime brings measurable value.

Conclusion

In the end, the decision should not be driven by trends, but by application reality: architecture, dependencies, operational constraints, risk, and business value.

If you would like more information or need support with JBoss EAP, from troubleshooting and performance optimization to upgrades and patching, feel free to contact me. I can also support you in assessing and planning the migration of your JBoss workloads to OpenShift or Kubernetes.

L’article Moving JBoss EAP Workloads to OpenShift est apparu en premier sur dbi Blog.