At first, the issue looked like a Java or JDK problem. After a full investigation, it turned out to be different.

This blog explains the full story from beginning to end.

Introduction

Initial Objective

The initial goal was simple:

• apply the latest Oracle Fusion Middleware patch bundle
• restart the environment
• validate Forms and Reports services

The patch installation completed without any obvious errors, and patch inventory looked clean.

Issue after patching

Once the environment was started again, Oracle Reports was no longer stable.

Symptoms:

• Reports engine failed to start correctly
• Report execution was failing
• Oracle Reports processes were crashing shortly after startup
• The environment became unstable after patching

This immediately confirmed that the issue was introduced after the Oracle patching activity.

Investigation

The first checks focused on the usual areas:

• WebLogic status
• Reports server startup
• Reports engine diagnostic logs
• process status
• environment variables
• classpath and Java configuration

At this stage, the failure appeared during Reports engine initialization.

Crash Analysis

A deeper review showed JVM crash files (hs_err_pid) being generated.

The important finding was that the crash was not pointing to standard Java code.
It was occurring in the Oracle Reports native library:

• librw.so
• function around kguinit

This was an important clue because it suggested the problem was not a normal Java exception, but a failure in Oracle Reports native components.

JDK Validation

Because the issue started after patching, one possible explanation was a JDK problem.

Several validations were performed:

• Checked active Java paths
• Rolled back JDK version
• Retested with the previous JDK
• Compared behavior before and after JDK rollback

The same crash still happened even after JDK rollback.

That confirmed the issue was not caused by the JDK itself.

Rollback and Reproduction Test

To make sure the Oracle patch was really the trigger, I performed a controlled validation:

• Roll back Oracle patch
• Start environment
• Test Reports
• Reapply Oracle patch
• Retest Reports

Before patch: environment worked fine

After patch: issue was reproduced again.

This confirmed that the problem was patch-related and reproducible.

Real Root Cause Direction

After further analysis, it became clear that the problem was not just a startup configuration issue.

The patch had left the Oracle Reports native layer in an inconsistent state, and the environment needed a relink of Oracle FMW Reports and related shared libraries. I already had the case in the past with older versions (11 and 12), which normally has been solved since (related Oracle KB)!

That explained why:

• Patching succeeded
• Startup partially worked
• But Reports engine crashed in native code

Final Resolution

The final resolution was to relink Oracle FMW Reports.

First, the Reports environment was loaded:

cd $DOMAIN_HOME/reports/bin
. ./reports.sh

Then the Oracle Reports libraries and binaries were rebuilt:

cd $ORACLE_HOME/reports/lib
make -f ins_reports.mk install

This recreated and relinked key Reports components such as:

• librw.so
• librwu.so
• rwserver
• rwrun
• rwclient
• rwbuilder
• rwconverter
• rwcgi
• rwproxy
• rwrqv

After that, the Oracle client shared libraries were rebuilt as well:

cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk client_sharedlib

This regenerated the Oracle client shared libraries and symbolic links required by the middleware binaries.

After the relink activity:

• Oracle Reports started correctly
• the engine no longer crashed
• report execution worked again
• the patched environment became stable

So the environment could remain patched and operational, without needing to stay rolled back.

Lessons Learned and key takeaway

This case was a good reminder that post-patching issues are not always caused by:

• Java
• Classpath
• Managed server configuration
• OS changes

Sometimes the real issue is lower-level and sits in:

• Oracle native binaries
• Shared libraries
• Incomplete or inconsistent linking after patching

So, please note that when Oracle Reports crashes inside librw.so after patching, and rollback proves the patch introduced the issue, a relink of Reports and Oracle client shared libraries should be considered as a serious corrective action.

Read more blogs

L’article Oracle Forms & Reports 14.1.2 Patching Issue est apparu en premier sur dbi Blog.

Introduction

Within WebLogic 14, out-of-the-box tools such as the Remote Console and WLST scripts provide some visibility, but they are limited when it comes to real-time metrics, historical analysis, or predictive monitoring.

This is where the Elastic Stack (Elasticsearch, Logstash, Beats, Elastic Agent, Kibana, and Elastic Machine Learning) delivers real value. By integrating WebLogic 14 logs, traces and metrics with Elastic Stack, you can build a unified platform that provides:

• Real-time observability of servers, threads, JDBC pools, and JVM health.
• Powerful visualization and analysis in Kibana.
• Predictive monitoring through anomaly detection and forecasting.

In this blog, we’ll walk through the integration of metrics step by step.

Step 1: Export WebLogic 14 Metrics

WebLogic does not natively expose metrics in formats suitable for Elastic or Prometheus. The solution is the WebLogic Monitoring Exporter, which translates JMX metrics into Prometheus-style output.

Download the exporter:

wget https://github.com/oracle/weblogic-monitoring-exporter/releases/download/v2.3.0/weblogic-monitoring-exporter.jar

Configure it (config.yaml):

domains:
  - name: mydomain
    url: t3://localhost:7001
    username: monitoruser
    password: XXXXXXXXXX
    metrics:
      - name: ServerHealth
      - name: ThreadPool
      - name: JDBCConnectionPool
      - name: JVMRuntime

Start the exporter:

java -jar weblogic-monitoring-exporter.jar --config=config.yaml

Verify metrics are exposed at:

http://localhost:8080/metrics

Step 2: Collect Metrics with Metricbeat

Elastic provides Metricbeat with a Prometheus module, making it easy to scrape WebLogic exporter metrics and send them to Elasticsearch.

Enable and configure the module:

metricbeat modules enable prometheus

prometheus.yml configuration:

- module: prometheus
  period: 10s
  hosts: ["http://localhost:8080"]
  metrics_path: /metrics

Start Metricbeat:

sudo metricbeat setup
sudo service metricbeat start

Now WebLogic metrics are flowing into Elasticsearch.

Step 3: Collect WebLogic Logs with Filebeat or Elastic Agent

Metrics give you system health, but logs provide context. To correlate performance issues with application errors, you should ingest WebLogic logs into Elasticsearch.

Option A: Using Filebeat

Install and configure Filebeat on the WebLogic server.

Example configuration (filebeat.yml):

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /u02/app/weblogic/domains/mydomain/servers/*/logs/*.log
    fields:
      application: weblogic
    multiline.pattern: '^\<'
    multiline.negate: true
    multiline.match: after

output.elasticsearch:
  hosts: ["http://localhost:9200"]

And start Filebeat.

Option B: Using Elastic Agent (Recommended)

Elastic Agent simplifies deployment by combining logs, metrics, and security data collection into a single agent managed centrally via Kibana Fleet.

Steps:

1. Enroll the agent from Kibana – Fleet.
2. Attach the System + Custom Log integration.
3. Configure the WebLogic log path.
4. Deploy the agent on your WebLogic server.

This approach reduces operational overhead and centralizes configuration.

Next steps & Conclusion

Next steps:

• Build Observability Dashboards in Kibana
• Add APM for Transaction-Level Visibility
• Enable Machine Learning for Predictive Monitoring
• Configure Alerts

While WebLogic 14 provides essential administrative tooling, it does not offer full observability capabilities out of the box.

By integrating it with the Elastic Stack, you gain:

• Real-time operational insight
• Deep log and metric correlation
• Predictive monitoring with machine learning
• Actionable alerting
• Transaction-level visibility

This approach scales from a single WebLogic instance to enterprise clusters and hybrid cloud deployments.

In short: you move from reactive troubleshooting to proactive performance management.

See my Elastic blogs.

L’article WebLogic 14 & Elastic Stack: From Metrics to Predictive Insights est apparu en premier sur dbi Blog.

With the release of WebLogic 14.1.2, Oracle has introduced significant updates that directly affect stability, security, and modernization. Below is an overview of what’s new and what it means for organizations planning to upgrade.

A Quick Stroll Through WebLogic History

What’s New in WebLogic 14

1. Modern Java Support

WebLogic 14.1.2 now officially supports JDK 17 and JDK 21. This provides access to performance improvements (e.g., G1GC, ZGC) and modern Java language features such as records, sealed classes, and switch expressions.

Impact for organizations:

• Applications gain performance and security improvements.
• Stricter Java module system may expose hidden dependencies.
• Legacy libraries or reflection-based solutions may fail.
• Outdated JDBC drivers and frameworks may require upgrading.

2. Strengthened Security Defaults

Security has been significantly improved in WebLogic 14:

• OpenID Connect support for integration with modern identity providers.
• TLS 1.0 and 1.1 removed; only strong cryptographic protocols are supported.
• Domain-specific demo certificates using PKCS12 keystores by default.

Impact for organizations:

• Easier integration with enterprise authentication and identity management.
• Legacy systems that depend on outdated SSL/TLS protocols will need upgrades.

3. Administration Console Evolution

The traditional WebLogic Admin Console has been retired and replaced with the Remote Console, a lightweight web application that communicates via REST APIs.

Advantages:

• Manage WebLogic securely from anywhere.
• Console can be upgraded independently from the server.

Considerations:

• Administrators must adapt to a new interface.
• Existing procedures, documentation, and training materials will require updates.

4. Migration and Refactoring Tools

Oracle introduced two new tools to assist with upgrades:

• Migration Analysis Tool (MAT): Scans applications and reports compatibility issues.
• OpenRewrite Recipes: Automates some application refactoring tasks.

Considerations:

• These tools provide useful guidance, but results often require expert interpretation.
• Automated refactoring may only cover part of the necessary work.

5. Clustering and Database Enhancements

Enhancements in high availability and database integration include:

• Health-based routing: Routes requests to the healthiest available node.
• Database client modules: Simplify integration in Kubernetes and containerized environments.
• Improved failover and multi-data center support: Reduce complexity in HA deployments.

Impact for organizations:

• More reliable clustering.
• Simplified operations in cloud and hybrid environments.
• Better resilience in multi-DC architectures.

Should You Upgrade?

Short answer: yes. Long answer: yes, but carefully.

Sticking with 12c in 2025 is like still using Internet Explorer – technically possible, but also technically embarrassing. At some point, Oracle’s support matrix will drop you, and then you’re one zero-day away from chaos.

In another world, continuing to run WebLogic 12c in 2025 is increasingly difficult to justify:

• Security vulnerabilities accumulate as older versions leave support.
• Integration with modern Java, Kubernetes, and identity providers becomes more challenging.
• Oracle’s support matrix is moving forward, and legacy environments are becoming costly liabilities.

Final Thoughts

WebLogic 14 is not just an incremental update. It modernizes the platform with stronger security, cloud-native capabilities, and support for the latest Java standards. At the same time, it introduces changes that require careful planning to avoid disruption.

This is where experienced guidance is valuable. With years of WebLogic consulting experience, I support organizations by:

• Auditing WebLogic installation for compatibility issues before migration.
• Planning and executing upgrades with minimal downtime.
• Configuring new security features and administration tools correctly.
• Coaching development and operations teams to adapt to the changes.

If your organization is considering upgrading to WebLogic 14, let’s discuss how to make the transition smooth, secure, and future-proof.

Happy to share,

David

Have a look to all my blogs

L’article WebLogic 14: What’s New and Why It Matters est apparu en premier sur dbi Blog.

Introduction and Symptoms

An important step in the installation process is the Oracle Fusion Middleware Metadata repository creation using the RCU (Repository Creation Utility) which creates the necessary schemas for the components.

But when running the Repository Creation Utility (RCU) to load schemas on Oracle Database, schema creation fails with the below errors:

In fact, the problem is seen as RCU tries to set up VPD stripes. In another world, RCU relies on fine-grained access control (FGAC) to manage schema creation and access within the Oracle database.

The ORA-00439 error “feature not enabled: Fine-grained access control” in Oracle RCU (Repository Creation Utility) indicates that the database being used for RCU schema creation does not have the fine-grained access control feature enabled. This feature is only part of the Enterprise Edition of Oracle Database and is not available in Standard Edition!

Let’s check

So, no way to do it with Standard Edition? No worries, there is a solution

Solution

Oracle here is a patch to workaround this issue, so the steps will be:

• Download the patch
• Apply the patch
• Run the RCU to load the schemas
• Continue with Domain configuration

Download the patch

Go to Patch 37506854 and download it, then move it to your working folder on the server.

Apply the patch

Nothing really special here, apply the patch as any patch

Unzip the patch downloaded, go inside the folder 37506854, and apply the patch:

[oracle@fmwserver working]$ cd 37506854
[oracle@fmwserver 37506854]$ opatch apply
Oracle Interim Patch Installer version 13.9.4.2.17
Copyright (c) 2025, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/midw
Central Inventory : /u01/app/oracle/oraInventory
   from           : /u01/app/oracle/product/midw//oraInst.loc
OPatch version    : 13.9.4.2.17
OUI version       : 13.9.4.0.0
Log file location : /u01/app/oracle/product/midw/cfgtoollogs/opatch/opatch2025-07-10_11-22-38AM_1.log


OPatch detects the Middleware Home as "/u01/app/oracle/product/midw"

Verifying environment and performing prerequisite checks...
OPatch continues with these patches:   37506854

Do you want to proceed? [y|n]
y
User Responded with: Y
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/midw')


Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...
Applying interim patch '37506854' to OH '/u01/app/oracle/product/midw'

Patching component oracle.rcu.mds, 14.1.2.0.0...

Patching component oracle.rcu.mds, 14.1.2.0.0...
Patch 37506854 successfully applied.
Log file location: /u01/app/oracle/product/midw/cfgtoollogs/opatch/opatch2025-07-10_11-22-38AM_1.log

OPatch succeeded.

Retry the failed step

Now, the RCU should work fine and the domain creation could be done without issue.

Have a look on all FMW blogs, more blogs to come about FMW 14 to highlight new features!

If you have any questions don’t hesitate, please ask

Happy to share,

David

L’article Oracle FMW 14 Installation – ORA-00439: feature not enabled: Fine-grained access control est apparu en premier sur dbi Blog.

What is the vault password?

The vault encrypts sensitive strings, stores them in an encrypted keystore, and decrypts them for applications and verification systems. To avoid leaving sensitive credentials readable in the standalone.xml or domain.xml files, you can store passwords or other attributes in the vault and then reference the vault from within the server configuration file.

Using a vault creates a level of abstraction and obfuscates data which could otherwise be read by anyone who has access to the configuration files. Please note that the access to the configuration files should be anyway restricted.

Create a Java Keystore to store sensitive strings

The vault utilizes a keystore by using the certificate stored in the keystore as an encryption key for the vault as a whole. To initialize the vault, users need to first create the JavaSE keystore. The following is the syntax used to create a private key, a certificate and to store them in a keystore:

keytool -genseckey -alias <alias> -keyalg <algorithm> -storetype <type> -keysize size -keystore <filepath>

• alias: The alias is a unique identifier for the vault or other data stored in the keystore.
• keyalg: The algorithm to use for encryption.
• storetype: The keystore type. jceks is the recommended type.
• keysize The size of the encryption key which determines the difficulty in brute forcing the
  key.
• keystore: The file path and file name in which the keystore’s values are stored.

The below is an example:

[jboss@vmjboss jboss]# keytool -genseckey -alias vault -keyalg AES -storetype jceks -keysize 128 -keystore /app/jboss/vault.keystore
Enter keystore password:  
Re-enter new password: 
Enter key password for <vault>
	(RETURN if same as keystore password):  

Warning:
The JCEKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /app/jboss/vault.keystore -destkeystore /app/jboss/vault.keystore -deststoretype pkcs12".
[jboss@vmjboss jboss]# ls -rtl /app/jboss/
total 4
-rw-r--r-- 1 jboss jboss 500 Sep  4 10:56 vault.keystore

After running the command, you will be prompted for a keystore password and a keystore file will be created at /app/jboss/vault.keystore.

Using the vault

The vault can be initialized either interactively, by providing each parameter one at a time, or by providing all of the parameters initially.

The following is an example of the syntax used to initialize the vault:

vault.sh --keystore KEYSTORE_URL --keystore-password KEYSTORE_PASSWORD --alias KEYSTORE_ALIAS --vault-block VAULT_BLOCK --attribute ATTRIBUTE --sec-attr SEC-ATTR --enc-dir ENC_FILE_DIR --iteration ITERATION_COUNT --salt SALT

The following parameters are required to initialize the vault:

• KEYSTORE_URL: The path to the previously created keystore file.
• KEYSTORE_PASSWORD: The password to access the keystore that was used at the keystore’s
  creation.
• SALT: A random string of eight characters used to encrypt the attribute stored in the vault.
• KEYSTORE_ALIAS: The alias that identifies the certificate stored in the keystore.
• ITERATION_COUNT: The number of times encryption is run.
• ENC_FILE_DIR: The path where the encrypted files are stored.
• VAULT_BLOCK: The name to be given to the block in the vault.
• ATTRIBUTE: The name of the attribute being stored. For example, “password” as an attribute
  name when storing a password value.
• SEC-ATTR: The value being stored.

The following is an example with the parameters populated:

[jboss@vmjboss bin]# vault.sh --keystore /app/jboss/vault.keystore --keystore-password password --alias vault --vault-block blog --attribute password --sec-attr blogpass --enc-dir /app/jboss/ --iteration 50 --salt 12345678

After running the vault.sh command, an XML definition of the vault is displayed, which needs to be added to the server configuration files. The following is an example of the XML that needs to be added to either the standalone.xml or host.xml configuration file directly before the section:

<vault>
<vault-option name="KEYSTORE_URL" value="/app/jboss/vault.keystore"/>
<vault-option name="KEYSTORE_PASSWORD" value="MASK-31x/z0Xn83H4JaL0h5eK/N"/>
<vault-option name="KEYSTORE_ALIAS" value="vault"/>
<vault-option name="SALT" value="12345678"/>
<vault-option name="ITERATION_COUNT" value="50"/>
<vault-option name="ENC_FILE_DIR" value="/app/jboss/"/>
</vault>

The final step is to replace the sensitive data with a reference to the attribute in the vault. The vault.sh command provides the exact syntax necessary to reference the secured attribute after all of the parameters are provided. Using the previous example, the vault command generated the following:

VAULT::blog::password::1

In order to use this reference, use the following syntax within the server configuration:

${VAULT::VAULT_BLOCK::ATTRIBUTE_NAME::1}

The following can replace the previous password in the server configuration for the blog datasource password:

${VAULT::blog::password::1}

After replacing the password for the datasource, the server configuration for the datasource will look similar to the following:

<datasource jndi-name="java:jboss/datasources/blogope" ...>
<connection-url>...</connection-url>
<driver>mysql</driver>
<security>
<user-name>blogadmin</user-name>
<password>${VAULT::blog::password::1}</password>
</security>
</datasource>

In this blog we saw how to protect passwords or any sensitive data in configuration files. Don’t forget to restrict the access to your configuration files also!

Don’t hesitate to comment this blog, for any question or to share your experience with vault within JBoss EAP.

Some blogs around JBoss EAP:

• JBoss EAP 7 – Modules, Extensions, Subsystems, and Profiles
• JBoss EAP 7 – Domain creation
• JBoss EAP 7 – Cluster in Standalone mode

L’article JBoss EAP 7 – Use vault to protect passwords est apparu en premier sur dbi Blog.

Aug 10, 2023 3:53:47 AM oracle.glcm.fmw.chghost.framework.plugin.impl.t2p.T2PPluginExecutorImpl startAdminServer
INFO: /u02/weblogic/config/12c/domains/fr-domain  ias_admin  password : *** srv-linux-wls1.it.dbi-services.com  7001  /u00/app/weblogic/12cR4/Middleware
java.io.IOException: Cannot run program "/tmp/CLONINGCLIENT2006552337794141616/coredb_start_script1691632427319.sh": error=13, Permission denied
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
        at java.lang.Runtime.exec(Runtime.java:593)
        at java.lang.Runtime.exec(Runtime.java:423)
        at java.lang.Runtime.exec(Runtime.java:320)
        at oracle.as.t2p.sdk.util.CommandInvokationUtil$ThreadDaemon.run(CommandInvokationUtil.java:259)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: error=13, Permission denied
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
        at java.lang.ProcessImpl.start(ProcessImpl.java:134)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
        ... 5 more
java.io.IOException: Cannot run program "/tmp/CLONINGCLIENT2006552337794141616/coredb_admin_pid_script1691632427336.sh": error=13, Permission denied
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
        at java.lang.Runtime.exec(Runtime.java:593)
        at java.lang.Runtime.exec(Runtime.java:423)
        at java.lang.Runtime.exec(Runtime.java:320)
        at oracle.as.t2p.sdk.util.CommandInvokationUtil$ThreadDaemon.run(CommandInvokationUtil.java:259)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: error=13, Permission denied
        at java.lang.UNIXProcess.forkAndExec(Native Method)
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
        at java.lang.ProcessImpl.start(ProcessImpl.java:134)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
        ... 5 more

Investigating on the error, I find out that the chghost.sh script writes some scripts in the /tmp folder and execute them to start the WebLogic Administration Server. This fails because the tmp file system was mounted with the noexec flag for security reasons. When it comes to security reasons, it is very difficult to change the systems configuration. I took me some time to found the solution and thought it would be helpful for others if I publish it in a blog.

The workaround is to set the “java.io.tmpdir” in the chghost.sh environment as follow:

export CHGHOST_JAVA_OPTIONS="-Djava.io.tmpdir=/u01/app/oracle/tmp/”  

After setting this variable, the chghost tool will use the new tmp directory to run the scripts generated to start the WebLogic Admin Server.

The other problem encountered was that the chghost tool was not able to connect to an SSL enabled port. I had to enable the http port 7001 during the run of the tool.

L’article Unable to start WebLogic Admin Server during host reconfiguration est apparu en premier sur dbi Blog.

The Fusion Middleware repository database migration has been done following the Oracle Fusion Middleware 12.2.1.4 – Administering Oracle Fusion Middleware documentation. The following restrictions applies to this kind of move:

• The service name and SID of the database on the target host must be the same as on the source host.
• The port on the target host must be the same as on the source host.
• The database on the target host must contain the same schemas as on the source host and the schemas must have the same prefix as on the source host.

The Repository Creation Utility (RCU) gives the possibility to create SQL scripts to create the system tables and information plus the Fusion Middleware product repository schemas and tables. We will use this functionality to configure the new database to host the migrated repository.

Using the RCU located in the $ORACLE_HOME/oracle_common/bin directory of the Fusion Middleware product I want to migrate the repository, I will generate the scripts that will create all needed objects in the target database for this database to become a Fusion Middleware repository database. Below are the steps I run to archive this.

I start the RCU and select “Create repository” and “Prepare Scripts for System loads”.

I provide the connection information to the target database.

RCU checks if the target database fulfill the prerequisites.

I choose the same prefix as on the source database and the same list of components.

RCU checks the prerequisites for the components selected.

Once all is validated, I click on the “Generate” button to create the DBA script.

This raises a process to generate the DBA scripts.

Once finalized, the “Completion Summary” screen shows the scripts location and the RCU log file name

Scripts : /tmp/RCU2023-03-13_13-38_739854221/logs/

Run the script “script_systemLoad.sql” on the target database as user having the DBA privileges to create the Fusion Middleware system objects and the product metadata schemas.

Once this is done, the rest of the steps are the same as in my previous blog Changing Fusion Middleware Forms and Reports repository database“. Those are:

Export all Fusion Middleware repository schemas from the source database

Before exporting the schemas, you need to make sure the export will be done in the right character set. In my case, the recommended character set for Fusion Middleware was used: AL32UTF8

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
expdp system/***** schemas=VM40FR_STB,VM40FR_IAU_APPEND,VM40FR_IAU_VIEWER,VM40FR_MDS,VM40FR_WLS,VM40FR_WLS_RUNTIME,VM40FR_IAU,VM40FR_OPSS dumpfile=VM40FR.dpdmp

Import those schemas in the target database

Same as for the database schemas export, the character set has to be correctly set in the NLS_LANG to avoid any character conversion or loose.

export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
impdp system/***** file=VM40FR.dpdmp

Reconfigure the Fusion Middleware to use the new repository database

The reconfiguration of the Fusion Middleware Forms & Reports has been done using the chghost.sh tool but this one requires two files. The wallet containing the WebLogic domain administrator password and the mapping file describing the type of changes that the chghost.sh tool will apply. Before running the change host script, make sure no process is running for the Fusion Middleware WebLogic Domain. For example, if the node manager is started as Unix service, make sure to stop it or the change host tool will hang stopping the WebLogic managed servers.

Generate the wallet file for an administrator user named weblogic on my test environments.

 $MW_HOME/oracle_common/common/bin/configWallet.sh -walletDir /home/oracle/CHGHOST_TEST weblogic

Create the mapping file. db_host_move.txt in this case.

[ARGUMENTS] [DATABASE_MAPPING] 
#  You can only change the database host name. 
vm07.it.dbi-services.com=vm09.it.dbi-services.com

Finally run the chghost.sh script.

cd /app/oracle/12c/Middleware/oracle_common/bin/ 
./chghost.sh -chgHostInputFile /home/oracle/CHGHOST_TEST/db_host_move.txt  -javaHome /app/oracle/Java/jdk1.8.0_321 -domainLoc /data/config/12c/domains/vm12fr -domainAdminUserName weblogic -walletDir /home/oracle/CHGHOST_TEST  -logPriority FINEST  -logDir /home/oracle/CHGHOST_TEST/logs

L’article Changing Fusion Middleware repository database est apparu en premier sur dbi Blog.

This registry contains many Oracle products, but we will focus on WebLogic in this blog.

Prerequisites

To be able to pull an image, you must be sign in and accept the license of the image.

The license agreement is per image product (or repository as they called, but it might be confusing). Before you accept, it will look like that:

Note that Oracle also provides patched version of the images with latest CPU. I will go for the standard one.

To accept license, click on Continue button and read “Oracle Standard Terms and Restrictions”, then you can accept. After that, repository line will look like this:

Login

Before pulling the image, I have to log into the registry:

$ docker login container-registry.oracle.com
Username: <username@domain.ch>
Password:
WARNING! Your password will be stored unencrypted in /home/ols/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
$

Tags

Before pulling the image, let’s see how tagging is done. Tags will allow you to select among many types of container images provided by Oracle:

• WebLogic server version: 14.1.1, 12.2.1.4, 12.2.1.3, …
• Java version: 11.0.17, 8u351, …
• Linux version: Oracle Linux 7u9, 8u4, …
• Installation type: Generic, Slim, Developer

Almost all of these could be combined to obtain the perfect image that fit your needs and constraints. Oracle even provides a nice table view with search capabilities and the associated pull command:

Pulling

Let’s download image of the tag 14.1.1.0-11-ol8:

$ docker pull container-registry.oracle.com/middleware/weblogic:14.1.1.0-11-ol8
14.1.1.0-11-ol8: Pulling from middleware/weblogic
65da81875cb9: Pull complete
91a7039d5d86: Pull complete
1bd80e398fe0: Pull complete
a3367028d133: Pull complete
7d59a6e587a1: Pull complete
48b74505173e: Pull complete
Digest: sha256:9b9032fb01aa88c381fd49f0c029289f2b1db21b4e92793ed410c3e3064909f8
Status: Downloaded newer image for container-registry.oracle.com/middleware/weblogic:14.1.1.0-11-ol8
container-registry.oracle.com/middleware/weblogic:14.1.1.0-11-ol8
$ 

Note that if license was not accepted, it will fail like if I was not logged in (I took Oracle Unified Directory as example):

Error response from daemon: pull access denied for container-registry.oracle.com/middleware/oud, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Usage of Image

To be able to start this image, I need at least to set a WebLogic console administrator. To do that, I must create a domain.properties file with following content:

username=weblogic
password=********

And map current directory to /u01/oracle/properties when starting the container:

$ docker run -d -p 7001:7001 -p 9002:9002 \
      -v $PWD:/u01/oracle/properties \
      container-registry.oracle.com/middleware/weblogic:14.1.1.0-11-ol8

It will take a few minutes as domain is being created. Use docker logs to check status:

$ docker logs -f zealous_gagarin
Domain Home is:  /u01/oracle/user_projects/domains/base_domain
Create Domain

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

domain_name                 : [base_domain]
admin_listen_port           : [7001]
domain_path                 : [/u01/oracle/user_projects/domains/base_domain]
production_mode             : [prod]
admin name                  : [AdminServer]
administration_port_enabled : [true]
administration_port         : [9002]

After a minute, you can see that it is started:

...
<Apr 13, 2023, 7:47:57,260 AM Coordinated Universal Time> <Notice> <WebLogicServer> <BEA-000329> <Started the WebLogic Server Administration Server "AdminServer" for domain "base_domain" running in production mode.>
<Apr 13, 2023, 7:47:57,291 AM Coordinated Universal Time> <Notice> <WebLogicServer> <BEA-000360> <The server started in RUNNING mode.>
<Apr 13, 2023, 7:47:57,328 AM Coordinated Universal Time> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to RUNNING.

Image Customizations

This will bring up an empty domain without any managed server or deployed web application. Oracle suggests extending images with a Dockerfile. Here is how it could look like:

FROM container-registry.oracle.com/middleware/weblogic:14.1.1.0-11-ol8
COPY --chown=oracle:root container-scripts/* /u01/oracle/
WORKDIR /u01/oracle
RUN ["/u01/oracle/createAndStartEmptyDomain.sh"]

createAndStartEmptyDomain.sh is embedded in the image, but it could be extended with customization.

L’article Oracle WebLogic Docker Images est apparu en premier sur dbi Blog.

WebLogic Monitoring Exporter

Oracle is providing, as open source, a Prometheus-compatible exporter in this github repo. It can be run either in a web application or as a separate process. In this blog, I will focus on the web app method.

Prepare Config File

The web application must be configured with help of a config.yml file. It can be loaded from the web app URL once it has been deployed but changes will not be persistent. Nevertheless, it is a good way to fine-tune parameters.

Once your configuration file has been validated, you can even use the script the provide to integrate the config file into the war file so that web app is deployed directly with the right configuration.

Let’s take an example of the configuration file provided by Oracle (jvm.yml) with some slight’s modifications:

metricsNameSnakeCase: true
queries:
- key: name
  keyName: server
  JVMRuntime:
    #prefix: jvm_
    prefix: wls_jvm_
    key: name
    values: [heapFreeCurrent, heapFreePercent, heapSizeCurrent, heapSizeMax, uptime, processCpuLoad]

This file only focuses on Java Virtual Machine memory and CPU. We will use this file in next steps, but for now, we must deploy the web application.

Deploy Web Application

The deployment of wls-exporter.war is not different to any other web application deployments:

1. Log into WebLogic console
2. Go in Deployments section
3. Click Lock and Edit
4. Click Install and target it to whatever server you want to monitor.
5. Once uploaded, click Activate Changes
6. Still in Deployments section, select Control tab
7. Select wls-exporter and click start Servicing all requests

At this point, you should be able to open the web application http://<HOSTNAME>:7001/wls-exporter/

As you can see, the queries section is empty, because there is no default configuration.yml file applied. If I open the metrics web page (http://<HOSTNAME>:7001/wls-exporter/metrics), I will not see any metric:

# No configuration defined.

Let’s upload the jvm.yml file. When submitting, I can either Append or Replace configuration file. Then, we will see the running configuration:

When I refresh the metrics page, now I can see some metrics:

wls_jvm_heap_free_current{server="AdminServer",name="AdminServer"} 283142888
wls_jvm_heap_free_percent{server="AdminServer",name="AdminServer"} 58
wls_jvm_heap_size_current{server="AdminServer",name="AdminServer"} 481820672
wls_jvm_heap_size_max{server="AdminServer",name="AdminServer"} 481820672
wls_jvm_process_cpu_load{server="AdminServer",name="AdminServer"} 0.5
wls_jvm_uptime{server="AdminServer",name="AdminServer"} 4674528
wls_scrape_mbeans_count_total{instance="192.168.33.12:7001"} 6
wls_scrape_duration_seconds{instance="192.168.33.12:7001"} 0.02
wls_scrape_cpu_seconds{instance="192.168.33.12:7001"} 0.03
exporter_version{instance="192.168.33.12:7001",version="v2.1.2"} 1

Now, let’s add it into Prometheus configuration.

Prometheus Configuration

Prometheus metrics are pulled from exporter, so I must declare it in prometheus.yml configuration file. In scrap_configs section of the file, I added the following lines:

  - job_name: "WebLogicAdmin"
    metrics_path: /wls-exporter/metrics
    static_configs:
    - targets: ['192.168.33.12:7001']
      labels:
        weblogic_clusterName: 'Standalone'
        weblogic_domainName: 'wls-domain'
        weblogic_domainUID: 'wls-domain'
        weblogic_serverName: 'admin'
    basic_auth:
      username: weblogic
      password: ******
      
  - job_name: "WebLogicMS"
    metrics_path: /wls-exporter/metrics
    static_configs:
    - targets: ['192.168.33.12:8003']
      labels:
        weblogic_clusterName: 'Standalone'
        weblogic_domainName: 'wls-domain'
        weblogic_domainUID: 'wls-domain'
        weblogic_serverName: 'msWS-1'
    basic_auth:
      username: weblogic
      password: ******

Metrics are labelled so that we can use the Grafana dashboard example provided in the github repo under samples\kubernetes\end2end\dashboard. This is not the best as it is hard-coded whereas everything could be deduced from metrics labels.

As password is not encrypted (only base64), it is strongly recommended to use https encryption between Prometheus and WebLogic server.

Results and Next Steps

Here is how it looks in the provided Grafana dashboard:

It looks great and it was quickly provisioned.

As you might have noticed “Thread Pool” panel is empty. To have it populated, we must modify exporter configuration to include threads metrics. The extra queries will look like this:

- threadPoolRuntime:
    prefix: wls_threadpool_
    key: name
    values: [executeThreadTotalCount, queueLength, stuckThreadCount, hoggingThreadCount, standbyThreadCount, executeThreads, executeThreadIdleCount]

“Et voilà !”:

L’article WebLogic Monitoring in Prometheus est apparu en premier sur dbi Blog.

Oracle Fusion Middleware 12c Forms Services
 FRM-93131: Cannot find base HTML file basejpi.htm
 Contact your system administrator.

Doing some googling and searches on the Oracle support site, I came to the note below where it states that solution is to remove the WebLogic domain, cleanup the database repository and restart the repository and WebLogic domain creation from the beginning.
FRM-93131: Basejpi.html Not Found –> Cannot Access Forms 12.2.1.2.0 (Doc ID 2199628.1)

I decided to take some time and see if there is not an other solution to avoid dropping the WebLogic domain and repository. After some tests I found out that the issue is due to WebLogic domain system component configuration step missed running config.sh.

I tried to simply update the existing WebLogic domain and run the system configuration missed step.

cd /u00/app/oracle/product/fmw_reports/oracle_common/common/bin/
config.sh

Update the existing domain selecting the WebLogic Domain of the forms and reports.

Later in the steps, select the system components

Add the forms system component to the list of available system components if not already declared.

Assign the forms system component to the machine. This is the missed step during the initial configuration.

Select and assign the forms component using the horizontal arrow.

Once this step is done, go through the steps until the update button appears and click on it.

Once the WebLogic domain has been updated, the forms component configuration on the file system is up to date and the FRM-93131 error does not raise anymore.

A simple update of the WebLogic Domain solved this issue !!

L’article FRM-93131 after a brand new Oracle Fusion Middleware Forms installation est apparu en premier sur dbi Blog.