A client contacted me after an operating system upgrade and the restart of an Oracle Fusion Middleware environment. The application servers were running. The ADF applications appeared healthy. Both Oracle Reports Server processes were also reported as running.

Yet report generation failed.

Even a simple request to display the Reports job list returned an error:

REP-50125: An internal exception occurred:
java.security.PrivilegedActionException:
javax.security.auth.login.LoginException:
java.lang.NoClassDefFoundError:
Could not initialize class
oracle.security.jps.service.policystore.info.BuiltInFunctionsHandler

The investigation led somewhere else: a Reports Server JVM running with a maximum heap of only 256 MiB, while the equivalent environment used 1 GiB.

Here is how we moved from a misleading symptom to an evidence-based diagnosis.

Separate process availability from application health

The initial service check showed every component as running:

AdminServer             is running.
WLS_REPORTS             is running.
RptSvr_app-dev_1         is running.
RptSvr_app-dev_2         is running.

This established that the processes existed. It did not establish that they could authenticate requests, evaluate security policies, or execute reports.

The failing showjobs request provided a more useful functional check. It showed that Reports could receive a request but failed while processing it.

The first lesson is straightforward:

A running process is an infrastructure observation, not an application health check.

Find the first exception, not the most frequently repeated one

The browser displayed:

NoClassDefFoundError: Could not initialize class ...

The wording matters. A class that cannot initialize is not necessarily a missing class.

I searched the Reports Server diagnostic logs for the first occurrence and the exceptions preceding it:

grep -n -B 30 -A 70 -E \
  'BuiltInFunctionsHandler|ExceptionInInitializerError|OutOfMemoryError|Caused by:' \
  rwserver_diagnostic.log

Earlier in the same process lifetime, the log contained:

ava.lang.OutOfMemoryError: GC overhead limit exceeded
...
oracle.security.jps.service.policystore.info.BuiltInFunctionsHandler.<clinit>

The sequence was now clear:

  • The JVM encountered memory exhaustion during initialization of a security class.
  • That initialization failed.
  • Subsequent requests reported that the class could not initialize.

The security error was therefore a consequence of the earlier memory failure.

The first exception explained the incident; the later exceptions described its persistent symptoms.

Identify the JVM that actually failed

An Oracle Reports installation involves several processes with different responsibilities:

ComponentRole
WLS_REPORTSHosts the Reports web application
rwserverHandles Reports Server requests and job management
rwEngExecutes report jobs

Each can have its own JVM configuration.

Increasing the heap of WLS_REPORTS would not necessarily help a failure occurring inside the standalone rwserver process.

I first mapped the processes:

ps -ww -eo pid,user,rss,args |
  grep -E '[b]in/rwserver|[w]eblogic.Name=WLS_REPORTS|[o]racle.reports.engine.RWEngine'

Then I checked which JVM library the affected native rwserver process had loaded:

grep 'libjvm.so' /proc/<REPORTS_PID>/maps

This revealed another useful detail: the JVM loaded from the Oracle home differed from the JDK referenced by JAVA_HOME.

The Reports startup script populated LD_LIBRARY_PATH with the bundled JVM location.

That distinction did not prove a Java compatibility problem. It established which JDK tools we should use and prevented us from diagnosing the wrong runtime.

For a running process, inspect what is loaded not just what the shell environment suggests.

Measure the heap instead of inferring it from available RAM

The operating system reported more than 20 GiB of available memory.

That did not contradict the Java memory error. A JVM can exhaust its configured heap while the host still has substantial free RAM.

Using the matching JDK, I inspected the effective JVM flags:

"$REPORTS_JDK/bin/jcmd" <REPORTS_PID> VM.flags

The relevant output was:

-XX:InitialHeapSize=268435456
-XX:MaxHeapSize=268435456

The maximum heap was 256 MiB.

Next, I sampled garbage collection statistics:

"$REPORTS_JDK/bin/jstat" -gcutil <REPORTS_PID> 1000 10

The affected process showed:

MetricObservation
Old generation occupancy99.85%
Eden occupancyApproximately 98%
Full GC count since startup185

The Full GC counter did not increase during that short sample. However, the nearly full heap, accumulated collection activity, and recorded OutOfMemoryError together provided strong evidence of heap exhaustion.

A single memory percentage would have been insufficient. The diagnosis came from combining the logs, effective configuration, and runtime measurements.

Compare with a working environment

I inspected the environment inherited by the working process:

tr '\0' '\n' < /proc/<REPORTS_PID>/environ |
  grep -E '^(REPORTS_JVM_OPTIONS|JAVA_TOOL_OPTIONS|_JAVA_OPTIONS)='

The reference environment contained:

REPORTS_JVM_OPTIONS= -Xmx1024M

Comparing the Reports environment scripts then revealed the missing lines in the affected domain’s reports/bin/reports.sh:

REPORTS_JVM_OPTIONS=" -Xmx1024M"
export REPORTS_JVM_OPTIONS

Validate the correction

The client aligned the Reports memory configuration with the reference environment and subsequently confirmed that report generation and application checks worked again.

A separate question remained: why had the problem become visible after this particular operating system upgrade and restart?

We did not have the previous process’s effective heap settings or a historical memory profile. We therefore could not prove whether the restart changed the inherited environment, whether memory demand had increased, or whether another condition exposed an existing limitation.

The timing made the OS upgrade a reasonable starting hypothesis. It did not make it the demonstrated root cause.

Conclusion

This incident followed a useful progression. The most valuable habit was to follow the evidence backward, from the visible error to the first failure, and then forward again, from the configuration difference to a verified recovery.

When critical applications stop working, you need more than a restart… You need a team that understands the platform. At dbi services, our Service Desk combines responsive support with deep technical expertise to investigate incidents, identify their causes, and guide recovery with clear, evidence-based recommendations.

Looking for expert support for your Oracle Fusion Middleware environment? Contact us to discuss how we can help.

More related blogs