Archives des Migration - dbi Blog

Oracle AutoUpgrade? Easy way to upgrade your Oracle Database! Part I

Xavier Franck Godard — Thu, 17 Jul 2025 10:21:33 +0000

Introduction

The AutoUpgrade utility is designed to automate the upgrade process, both before starting upgrades, during upgrade deployments, and during postupgrade checks and configuration migration

The Oracle AutoUpgrade Tool is part of Oracle Home since the version 12.1.0.2.
It’s a java-based program which can be download on MySupport Oracle : ID 2485457.1
It’s recommanded to use the lastest version (as usual, like for opatch by example).

The most important point is that Oracle desupport DBUA (Database Upgrade Assistant) and other manual upgrade to Oracle 23ai. AutoUpgrade is the only supported method.

This tab show the supported sources database version and the supported target database version for AutoUpgrade

Source Database Version
11.2.0.4	12.2.0.1 (12.2 + DBJAN2019RU and newer)
12.1.0.2	18c (18.5 and newer)
12.2.0.1	19c (19.3 and newer)
18c	21c (21.3 and newer)
19c	23ai (23.4 and newer)
	Any future release

The AutoUpgrade workflow

Using AutoUpgrade is running four simple steps :

Prepare
Analyze
Fixups
Deploy

Prepare

The first step is about downloading all binairies that we need to patching or upgrade our databases. And to create a configuration file used by AutoUpgrade for known what to do and where do it.

Download lastest version of AutoUpgrade -> My Oracle Support ID 2485457.1
Download lastet version of opatch -> My Oracle Support ID 224346.1
Download DBRU + OJVM+ DPBP+ Most Important One-off Patches -> My Oracle Support ID 2118136.2 + ID 555.1

The configuration file used by AutoUpgrade must contain some informations like :

Source Oracle Home
Target Oracle Home
Patches repository
Source Database SID
and other advanced option

We will see that later in this blog, no worries!

Analyze

It’s the classic precheck used during all patch, upgrade process lead by Oracle.

Fixups

Automatic or manual Fixups, if necessary.

Deploy

It’s the main step where AutoUpgrade run the upgrade process on your source database.
In case of failure, AutoUpgrade is able to restart where it fails on… After have fixed the errors by yourself!

A simple example

For this labs, we have :

Oracle Linux Server 8
Oracle Database Home 19.24 installed in /u01/app/oracle/product/19.24/dbhome_1
Oracle standalone database : DB19

We want to patch this database to the 19.28 version, according to the Best Pratices (for this check Mike Dietrich blogs : https://mikedietrichde.com/ ).

Preparation

We download the following patches :

Patch 37960098 : DBRU 19.28
Patch 38170982 : Data Pump Bundle Patches (My Oracle Support ID 2819284.1)
Patch 37847857 : OJVM 19.28
Patch 34672698 : [VOS] DB50: ORA-800: soft external error, arguments: [set priority failed], [vktm] , dism(16)
Patch 34774667 : [AQ] ORA-7445 in Purge Queue Table / High CPU usage in SVCB Service
Patch 29213893 : [QRY OPTIMIZER] DBMS_STATS Failing With Error Ora-1422 When Gathering Stats for User$ Table

We save them in the repository /home/oracle/autoupgrade/patches (for example).

We download in the same manner the lastest version of Opatch following the Oracle MySupport ID 224346.1 :

How to install OPatch utility ?

An overview of “Oracle Cloud Infrastructure: everything for an efficient migration” event by dbi services

Oracle Team — Wed, 17 May 2023 12:19:51 +0000

Is the Oracle Cloud a hot topic for you?
dbi services organized an event about the Cloud today in Lausanne: “Oracle Cloud Infrastructure: everything for an efficient migration“.

After a welcome coffee to meet each other, Pascal Zanette shortly introduced the event and the agenda of the day.

Our CTO Hervé Schweitzer presented then YaK, developed by dbi services.
The idea was born 2 years ago, to find a solution to deploy Oracle instances on several Cloud providers (Oracle, Microsoft Azure, AWS) and On-Premises:
– in the same heterogeneous way
– avoiding constraints of each Cloud providers solution
– automated through Ansible.
YaK is made of 2 parts:
– YaK Core – as an Open Source collection (for the central inventory and instances deployment)
– YaK Components – based on a subscription (for database deployment and with dbi services best practices integration).
Hervé showed how YaK works more in details through a demo, that you can play again here.
YaK is already available for Oracle and PostgreSQL. YaK for SQL Server, MariaDB, Kubernetes, WebLogic, MongoDB and Alfresco is under development and will come soon, so stay tuned!

Marc Wagner presented how to migrate from On-Premises to Oracle OCI using Zero Downtime Migration tool (ZDM).
Marc talked about advantages to go to the Cloud (such as scalability and flexibility, no dedicated hardware, less responsibility), but also alerted the audience about things to think about (such as performances and latency, costs monitoring, lock-in risk, sensitive data). Once we decided to make this change, what about downtime to migrate to the Cloud? That’s why ZDM is interesting: it’s high available (applying MAA Best Practices), it offers a very short downtime and a minimal impact on production systems and reduces human errors. ZDM allows to migrate to Oracle OCI but also to On-Premises Exadata, Exadata Cloud@Customer (ExaCC), Autonomous Database, etc…
And finally Marc presented through a live demo how to use ZDM with an Offline Logical Migration workflow to migrate from On-Premises to ADB (you can find a summary here).

As a last session of the event, Marc Keller, with Frank Zeindler remotely, presented Cloud migration made easier and safer with data validation. Marc explained why an extra validation after a data migration is helpful and how the OMrun framework can compare data and orchestrate various processes for this purpose. Some OMrun use cases were then explored:
– Quality assurance for reporting
– Data integrity monitoring for system changeover and production
– Data crawling and data cleansing
– Data masking
– Data quality assurance during migration to the Cloud.

The event is over. After a nice lunch break with colleagues, customers and attendees, we can come back home keeping in mind the magic triple jump to go to the Cloud (as Marc mentioned during his session):
– dbi services YaK
– Oracle ZDM
– OMrun.

Last thing: don’t forget that another “Oracle Cloud Infrastructure: everything for an efficient migration” event will take place on May 23 in Basel.
And some other dbi services events will be organized during the year about DevOps, Cloud, Data and much more. So again, stay tuned!

By Elisa Usai

L’article An overview of “Oracle Cloud Infrastructure: everything for an efficient migration” event by dbi services est apparu en premier sur dbi Blog.

Migrating Non-CDB to Multitenant

Oracle Team — Tue, 14 Feb 2023 20:28:53 +0000

By Mouhamadou Diaw

Dealing these last days to database migration with a customer, I write this blog to describe the main tasks

The context is to migrate databases to a new infrastructure. Below the configuration

Source : Linux Suse in a non-multitenant architecture

Target : Oracle Enterprise Linux 8 with multitenant

The database version is Oracle 19c

Installing Oracle Binaries on the OEL

The first step is to install the Oracle binaries. You can do it by many ways. I created a Gold Image from the source server.

The advantage is that I will have the exact patch level than the source environment.

The steps are described below

1-Create the Gold Image on the source

[oracle@sourceserver ~]$ echo $ORACLE_HOME
/u00/app/oracle/product/19.14/dbhome_1
[oracle@u027 ~]$ /u00/app/oracle/product/19.14/dbhome_1/runInstaller  -silent -createGoldImage -destinationLocation /u01/software/gold_image/rdbms19.14
Launching Oracle Database Setup Wizard...

Successfully Setup Software.
Gold Image location: /u01/software/gold_image/rdbms19.14/db_home_2023-02-06_09-03-15AM.zip

2-Copy the Image to the OEL target server and unzip it to the new Oracle_Home

[oracle@targetserver ~]$ unzip -d /u00/app/oracle/product/19.14/dbhome_1 /u01/software/gold_image/rdbms19.14/db_home_2023-02-06_09-03-15AM.zip

3-Launch the installation process on the target OEL server

The installation can be graphical or silent method. In my case I used the silent method with a response file

With Oracle Enterprise Linux 8, you will need to export the CV_ASSUME_DISTID variable otherwise the install will fail

oracle@targetserver ~]$ export CV_ASSUME_DISTID=OEL7.8

[oracle@targetserver ~]$ /u00/app/oracle/product/19.14/dbhome_1/runInstaller -silent  -responseFile /u01/software/gold_image/db_1914.rsp
Launching Oracle Database Setup Wizard...

The response file for this session can be found at:
 /u00/app/oracle/product/19.14/dbhome_1/install/response/db_2023-02-06_10-00-42AM.rsp

You can find the log of this install session at:
 /tmp/InstallActions2023-02-06_10-00-42AM/installActions2023-02-06_10-00-42AM.log

As a root user, execute the following script(s):
        1. /u00/app/oraInventory/orainstRoot.sh
        2. /u00/app/oracle/product/19.14/dbhome_1/root.sh

Execute /u00/app/oraInventory/orainstRoot.sh on the following nodes:
[targetserver]
Execute /u00/app/oracle/product/19.14/dbhome_1/root.sh on the following nodes:
[targetserver]

Successfully Setup Software.
Moved the install session logs to:
 /u00/app/oraInventory/logs/InstallActions2023-02-06_10-00-42AM
[oracle@targetserver ~]$

Then run the 2 scripts as asked

[root@targetserver ~]# /u00/app/oraInventory/orainstRoot.sh
Changing permissions of /u00/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u00/app/oraInventory to oinstall.
The execution of the script is complete.
[root@targetserver ~]# /u00/app/oracle/product/19.14/dbhome_1/root.sh
Check /u00/app/oracle/product/19.14/dbhome_1/install/root_targetserver.sig-ge.ch_2023-02-06_10-02-03-988526328.log for the output of root script
[root@targetserver ~]#

4-Relink Oracle binaries (optional)

This step is optional but I recommend to do it when cloning your environment

[oracle@targetserver ~]$ export ORACLE_HOME=/u00/app/oracle/product/19.14/dbhome_1
[oracle@targetserver ~]$ cd $ORACLE_HOME/bin
[oracle@targetserver ~]$./relink as_installed

And that is all for the binaries. If you have different Oracle version, do the same

Migrating the Data to multitenant architecture

You have may options to do that, we can for example list following

-datapump

-unplug and plug data

-Creating the PDB via DB Link

I used the third method, i.e using database link

1-On the target server create an empty container

I used the silent method. You will have to create your responsefile accordingly.

oracle@u027:/home/oracle/ [rdbms1914] /u00/app/oracle/product/19.14/dbhome_1/bin/dbca -silent -createDatabase -responseFile /home/oracle/dbca_1914.rsp

2-On the target server create a database link to the source database

Let’s say my source database is PROD and the port used is 1521

On the CDB$ROOT with SYS or a user with required privilege

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO

SQL> CREATE DATABASE LINK "MIGLINK" CONNECT TO "SYSTEM" IDENTIFIED BY secret USING 'sourceserver:1521/PROD';

Database link created

And now test the DB Link

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO

SQL> select name from v$database@MIGLINK

3-On the source server, give following privilege to the user used in the DB Link (i.e SYSTEM)

SQL> grant create pluggable database to system;

Grant succeeded.

SQL>

4-On the target server create the PDB using the DB Link

This can be done while the non-cdb database PROD is opened in Read Write mode but it is recommended to open the database in Read Only mode

SQL> CREATE PLUGGABLE DATABASE PROD from non$cdb@MIGLINK;

Pluggable database created.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         5 PROD                           MOUNTED
SQL>

Of course you can change the PDB name .

5-On the target server run the noncdb_to_pdb.sql on the newly created PDB

Connect as SYS

SQL> alter session set container=PROD;

Session altered.

SQL> show con_name;

CON_NAME
------------------------------
PROD
SQL> @?/rdbms/admin/noncdb_to_pdb.sql
…
…
  8      END IF;
  9    END;
 10  END;
 11  /

PL/SQL procedure successfully completed.

SQL>
SQL> WHENEVER SQLERROR CONTINUE;
SQL>
SQL>
SQL>
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         5 PROD                          MOUNTED

There should not be any error. And then you can now open you PDB in a R/W mode

SQL> alter pluggable database PROD open;

Pluggable database altered.

SQL>

To automatically open the PDB the next time the instance is restarted.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         5 PROD                           READ WRITE NO

SQL> alter pluggable database all save state;

Pluggable database altered.

SQL>

Conclusion

As you can see migrating a Non-CDB database to multitenant one is not very difficult. Read Oracle documentation and you will see many information about converting database to CDB architecture

L’article Migrating Non-CDB to Multitenant est apparu en premier sur dbi Blog.

GitLab Migration and Upgrade

DevOps — Wed, 08 Feb 2023 17:01:10 +0000

I’ve worked with my mate Donovan Winter (we are called Thomson and Thompson!) on a very interesting project for one of our customer that’s worth sharing. They had an old version of GitLab (release 13.3.4) running in a pod in OpenShift and wanted to migrate it into a Virtual Machine under Rocky Linux 8.

They then wanted to upgrade it to the latest GitLab release (release 15.7.3). I’ll share here all the steps required for doing such migration and upgrade.

Setting up the new GitLab Server

The first step is to prepare a new Virtual Machine and install the prerequisite packages and services for GitLab:

$ sudo dnf install -y curl openssh-server perl checkpolicy policycoreutils-python-utils python3-audit python3-libsemanage python3-setools python3-policycoreutils policycoreutils-python-utils

$ sudo systemctl enable sshd
$ sudo systemctl start sshd

$ sudo firewall-cmd --permanent --add-service=http
$ sudo firewall-cmd --permanent --add-service=https
$ sudo systemctl reload firewalld

Then install GitLab 13.3.4 on this new server as this is the same release as the one in OpenShift:

$ curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh| sudo bash

$ sudo dnf install gitlab-ce-13.3.4-ce.0.el8

The curl command above adds the GitLab repository in our system so we can then install the GitLab package in release 13.3.4 (in our case). Note also that there are 2 flavours of GitLab: ce (Community Edition) and ee (Enterprise Edition). We installed the ce one because this is the one used in OpenShift and it has to match in order to be able to backup/restore data between these 2 instances.

Then, set the initial GitLab configuration by editing the file /etc/gitlab/gitlab.rb and removing the # in front of the line # gitlab_rails[‘initial_root_password’] = “password”

We also changed in this file the URL of the GitLab Server on which to connect for the graphical interface: external_url ‘http://’

Each time this GitLab configuration file is changed, you’ll need to reconfigure GitLab to take those modifications into account with the following command:

$ sudo gitlab-ctl reconfigure

In our case we had the following errors after running the reconfigure command:

Running handlers:
There was an error running gitlab-ctl reconfigure:
execute[/opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8] (postgresql::enable line 75) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of /opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8 ----
STDOUT: The files belonging to this database system will be owned by user "gitlab-psql".
This user must also own the server process.
STDERR: initdb: invalid locale settings; check LANG and LC_* environment variables
---- End output of /opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8 ----
Ran /opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8 returned 1

This can be resolved as follows:

$ export LC_ALL="en_US.UTF-8"
$ export LC_CTYPE="en_US.UTF-8"

$ sudo gitlab-ctl reconfigure

And then the reconfigure was successful.

Backup of GitLab in OpenShift

Connect to the OpenShift cluster, backup GitLab and transfer the backup file to the new GitLab server.

$ oc login --server=https:// -u admin
$ oc project gitlab-prod

$ oc get pods
NAME                READY   STATUS    RESTARTS   AGE
gitlab-ce-9-p5xzx   1/1     Running   0          9d
redis-2-jwxzl       1/1     Running   0          1y

$ oc describe pod/gitlab-ce-9-p5xzx

In an OpenShift cluster, you first login to it and then enter into a project (which is the equivalent of a namespace in Vanilla Kubernetes).

There are only 2 pods in this project, one that contain the GitLab server and another one with the Redis database. There is no GitLab operator pod that would have helped creating a backup of this instance.

Using describe on this GitLab pod will show all the environment variables that we may need in our new GitLab server. I’ll come back to this later on.

We’ll then have to connect to the GitLab pod and do the backup as shown below:

$ oc rsh gitlab-ce-9-p5xzx
# su git
git@gitlab-ce-9-p5xzx:~/gitlab$ /home/git/gitlab/bin/rake gitlab:backup:create

This GitLab pod didn’t contain any of the usual GitLab binaries in /usr/bin so we can only rely on the rake tool that is part of the core of GitLab in order to do a backup. The backup file is then stored into the folder /home/git/data/backups of the pod as configured in its configuration file /etc/gitlab/gitlab.rb

We then just need to transfer this backup file using the OpenShift command below and transfer it to the new GitLab server:

$ oc rsync gitlab-ce-9-p5xzx:/home/git/data/backups/1673909524_2023_01_31_13.3.4_gitlab_backup.tar ./

$ scp

Restore data in the new GitLab Server

First, set the backup path into the file /etc/gitlab/gitlab.rb by uncommenting and editing the following line : gitlab_rails[‘backup_path’] = “/data/gitlab/backups”

Create this new backup repository and make git the owner :

$ sudo mkdir -p /data/gitlab/backups && sudo chown -R git:git /data/gitlab

Copy the backup file to this new repository and set the owner of this file to git:

$ sudo cp 1673909524_2023_01_31_13.3.4_gitlab_backup.tar /data/gitlab/backups/

$ sudo chown git:git /data/gitlab/backups/1673909524_2023_01_31_13.3.4_gitlab_backup.tar

Stop only the GitLab processes that are connected to the database:

$ sudo gitlab-ctl stop unicorn 
$ sudo gitlab-ctl stop puma 
$ sudo gitlab-ctl stop sidekiq

Reconfigure to apply changes in the gitlab.rb file (the backup path set above)

$ sudo gitlab-ctl reconfigure

As the backup file has been copied into the backup folder of the new GitLab Server, we just have to restore it and restart everything:

$ sudo gitlab-backup restore BACKUP=1673909524_2023_01_31_13.3.4

$ sudo gitlab-ctl reconfigure 
$ sudo gitlab-ctl restart
$ sudo gitlab-rake gitlab:check SANITIZE=true

At this stage we have successfully migrated GitLab 13.3.4 from OpenShift to a new VM. Let’s now continue with the upgrade process to the latest release.

GitLab Upgrade

GitLab provides a great tool (thank you to my mate Donovan Winter – one of our rising star in DevOps and our GitLab guru – for finding this out!) to see at a glance the upgrade path to follow by using the following link https://gitlab-com.gitlab.io/support/toolbox/upgrade-path

Below is the path to follow:

[`13.1.11`](https://docs.gitlab.com/ee/update/#1310)

-> [`13.8.8`](https://docs.gitlab.com/ee/update/#1388)

-> [`13.12.15`](https://docs.gitlab.com/ee/update/#13120)

-> [`14.0.12`](https://docs.gitlab.com/ee/update/#1400)

-> [`14.3.6`](https://docs.gitlab.com/ee/update/#1430)

-> [`14.9.5`](https://docs.gitlab.com/ee/update/#1490)

-> [`14.10.Z`](https://docs.gitlab.com/ee/update/#14100)

-> [`15.0.Z`](https://docs.gitlab.com/ee/update/#1500)

-> [`15.1.Z`](https://docs.gitlab.com/ee/update/#1510) (for GitLab instances with multiple web nodes)

-> [`15.4.0`](https://docs.gitlab.com/ee/update/#1540)

-> [latest `15.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases)

Below are all the commands of this upgrade path:

# Upgrade to 13.8.8
$ sudo dnf install -y gitlab-ce-13.8.8

# Upgrade to 13.12.15
$ sudo dnf install -y gitlab-ce-13.12.15

# Upgrade to 14.0.12
$ sudo dnf install -y gitlab-ce-14.0.12

# Upgrade to 14.3.6
$ sudo dnf install -y gitlab-ce-14.3.6
$ sudo gitlab-ctl restart redis
$ sudo gitlab-ctl restart postgresql
$ sudo gitlab-rake db:migrate

# Upgrade to 14.9.5
$ sudo dnf install -y gitlab-ce-14.9.5

# Upgrade to 14.10.5
$ sudo dnf install -y gitlab-ce-14.10.5

# Upgrade to 15.0.5
$ sudo dnf install -y gitlab-ce-15.0.5

# Upgrade to 15.4.6
$ sudo dnf install -y gitlab-ce-15.4.6
$ sudo gitlab-ctl restart redis

# Upgrade to 15.7.3
$ sudo dnf install -y gitlab-ce-15.7.3
$ sudo gitlab-ctl restart redis

When Redis or Postgresql needs to be restarted, it is mentioned at the end of output of the release installation. For some release, this is required because the running service version is different than the new one.

The new GitLab server is now upgraded to release 15.7.3

Final configuration

You may remember that at the very beginning we’ve looked at the environment variables of the GitLab pod. It is now time to transfer the required parameters to this new shiny GitLab server to complete our migration. In our case we only had to set back the parameters used for SMTP. For everything else, we kept the default settings of the new GitLab server. Below are the SMTP parameters of the file /etc/gitlab/gitlab.rb

gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = 'no-reply@'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "mail."
gitlab_rails['smtp_port'] = 25
gitlab_rails['smtp_domain'] = "smtp_domain"

As usual we reconfigure GitLab to take those parameters into account:

$ sudo gitlab-ctl reconfigure

To test that SMTP is working properly, you can do a test and send an email to yourself, if it works (and it did for us) you are all set!

$ sudo gitlab-rails console
> Notify.test_email('benoit.entzmann@dbi-services.com', 'Benoit Test', 'Test ok :)').deliver_now

Migration and Upgrade done!

This is it! Those were all the steps required to migrate and upgrade a GitLab instance from a pod in OpenShift to a Virtual Machine.

I hope this will help you if you have to do the same operation. Note that this procedure is also valid if you want to migrate and upgrade from another server (or Virtual Machine) instead of a pod in a Kubernetes cluster or if you just want to upgrade a GitLab instance. Just follow the steps in the section that match your needs!

For a migration from another GitLab server, the backup is easily done with the command sudo gitlab-backup create on that instance and then you can follow this procedure for the restore (and upgrade if required).

Have fun with it!

L’article GitLab Migration and Upgrade est apparu en premier sur dbi Blog.

Import tnsnames.ora in LDAP directory

Oracle Team — Fri, 04 Feb 2022 17:00:26 +0000

This post gives a short intro to directory naming, shows how to import from tnsnames.ora to an LDAP directory. Finally, as an alternative, you get an example how a TNS connection string looks like in ldif file format. LDIF can be used universally to export and importing data to LDAP directories.

What is directory naming?

In order to connect to a database, you either need to pass a DB connection string or you are using an alias to lookup up the string. When using an alias, there are two ways to connect to an Oracle database:

Local naming
Lookup DB connection strings locally in a tnsnames.ora file
There is a variant when the connection is “directly” established using local application configuration in application or a jdbc string:
Directory naming
Lookup DB connection strings remotely in a LDAP directory
LDAP lookup is also possible via jdbc:

This diagram summarises all the possibilities :

Directory naming can be compared to DNS, all you aliases are in one central LDAP server (like in a DNS server).
If you do local naming, you use a local configuration file that can be compared to /etc/hosts.
The data structure of both tnsnames.ora file or LDAP server used is a very simple key-value-store, it’s actually =

Directory naming has some benefits over local naming:

There is a single source of “truth” in one central directory. Much easier to manage. You don’t have to distribute tnsnames.ora files to the clients.
It’s advisable to run LDAP highly availabe, for example configuring replication between several LDAP servers and having load balancers distribute traffic.
On every client, all connections are available. Useful when accessing a DB from remote, for example remote PDB cloning, using DB links, Data Guard Observer, doing remote RMAN backup & restore/recovery or cloning, etc.

Mass import from tnsnames.ora to LDAP Server

In the following example we are using Oracle Unified Directory as an LDAP directory. You are free to choose other LDAP directories to use for directory naming: OpenLDAP or Active Directory to name just a few.
Once you setup OUD for directory naming, you may want to do an initial load with contents from local tnsnames.ora file. In the directory your environment variable TNS_ADMIN points to, make sure to have these three files present:

sqlnet.ora
contains information on SQL*net configuration, important parameters are:

NAMES.DEFAULT_DOMAIN = company.com # default domain if domains are used
NAMES.DIRECTORY_PATH = (LDAP) # Where to lookup aliass, multi values possible, for example (TNSNAMES, LDAP) to first look locally, then on LDAP

ldap.ora
contains information about your LDAP directory, important parameters are:

DIRECTORY_SERVERS     = (loadbalancer.company.com:1389) # Adress of the LDAP server
DEFAULT_ADMIN_CONTEXT = "dc=company,dc=com" # Context where alias entries are stored
DIRECTORY_SERVER_TYPE = OID # Type of LDAP server

tnsnames.ora
contains aliases and connections strings, example:

TESTDB = (DESCRIPTION=
           (ADDRESS_LIST=
              (ADDRESS=
                  (PROTOCOL=TCP)
                  (Host=testdb.company.com)
                  (Port=1521)
              )
           )
           (CONNECT_DATA=
              (SERVICE_NAME=testdb.company.com)
           )
        )

On the client you would like to do the mass import, make sure to have either have the Oracle client or RDBMS software installed. Here, I’m using Oracle Client 12.2 64Bit on Windows. It’s quite an old version, but it fits our purpose. Both in Oracle client or RDBMS software, there is a tool called “Net Manager”. By reading the configuration files mentioned above, it’s able to connect both to remote LDAP servers and local tnsnames.ora file.

Startup Net Manager. In the menu, choose “COMMAND – DIRECTORY – EXPORT NET SERVICE NAMES”:

After authenticating to the LDAP server, a wizard shows up on which you are able to choose which aliases to add to LDAP directory:

Alternative: Use LDIF to export and import LDAP data

LDIF file format is easiest if you want to transfer data between LDAP directories. It’s also possible to create LDIF files containing TNS data. See the following example LDIF record for alias “TESTDB”:

dn: CN=TESTDB,cn=OracleContext,dc=company,dc=com
orclNetDescString: (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(Host=testdb.company.com)(Port=1521)))(CONNECT_DATA=(SERVICE_NAME=testdb.company.com)))
objectClass: top
objectClass: orclNetService
CN: TESTDB

Aliases are stored in LDAP context “cn=OracleContext,dc=company,dc=com”.
“orclNetDescString” contains the actual DB connection string.
“CN” contains the alias value and is used to lookup a connection string.

L’article Import tnsnames.ora in LDAP directory est apparu en premier sur dbi Blog.

Oracle autoupgrade on Windows and plugin to a Container DB with virtual accounts

Clemens Bleile — Mon, 08 Feb 2021 10:49:27 +0000

In a project I recently had to upgrade an Oracle 12.2.-DB to 19.9. and at the same time migrate from the non-container architecture to the container architecture. The interesting part here is to do this on Windows. Actually both steps (upgrade and plugin) are possible using the autoupgrade tool , which is the preferred tool to do Oracle upgrades today.

Before doing the upgrade at the customer, I performed 2 tests in my personal environment:

– Upgrade 12.2. to 19.9. and the plugin to a container-DB in 2 separate steps
– Upgrade 12.2. to 19.9. and the plugin to a container-DB in 1 step (i.e. calling “autoupgrade -mode deploy” only once)

Doing the upgrade and the plugin in 2 steps requires to manually create the Windows services again after the upgrade finished using the oradim-utility. I.e. first step is the upgrade, which automatically uses a temporary Windows service. At the end of the upgrade new services have to be created manually, because autoupgrade has no clue about the password of the user running the Oracle services. I.e. I used the following config file upgrade_config_122_199_SIMC.cfg.txt:


# Global parameter
# ================

global.autoupg_log_dir=D:Downloadsoracle19cautoupgradelogs
global.target_home=D:appcblvirtualproduct19.9.0dbhome_1
global.target_version=19


# Database parameter SIMC
# =======================

upg1.dbname=simc
upg1.start_time=NOW
upg1.source_home=D:appcblvirtualproduct12.2.0dbhome_1
upg1.sid=simc
upg1.upgrade_node=dbi-pc-cbl
# upg1.add_after_upgrade_pfile=D:Downloadsoracle19cautoupgradepfileadd_after_init.ora

The commands for the upgrade were


%ORACLE_HOME%jdkbinjava -jar autoupgrade.jar -config configupgrade_config_122_199_SIMC.cfg.txt -mode analyze
%ORACLE_HOME%jdkbinjava -jar autoupgrade.jar -config configupgrade_config_122_199_SIMC.cfg.txt -mode fixups
%ORACLE_HOME%jdkbinjava -jar autoupgrade.jar -config configupgrade_config_122_199_SIMC.cfg.txt -mode deploy

And finally executed the oradim-commands on the cmd-prompt:


oradim -delete -sid SIMC
oradim -new -sid SIMC -startmode auto -spfile -shutmode immediate

The second step then was to plugin the non-container-DB with the following config file:


# Global parameter
# ================

global.autoupg_log_dir=D:Downloadsoracle19cautoupgradelogs3
global.target_home=D:appcblvirtualproduct19.9.0dbhome_1
global.target_version=19


# Database parameter SIMC
# =======================

upg1.dbname=simc
upg1.start_time=NOW
upg1.source_home=D:appcblvirtualproduct19.9.0dbhome_1
upg1.sid=simc
upg1.upgrade_node=dbi-pc-cbl
# upg1.add_after_upgrade_pfile=D:Downloadsoracle19cautoupgradepfileadd_after_init.ora
upg1.target_cdb=cdbsimc
upg1.target_pdb_name=PDBSIMC
upg1.target_pdb_copy_option=file_name_convert=('D:APPCBLVIRTUALORADATASIMC','D:APPCBLVIRTUALORADATACDBSIMCSIMC')

As the source and target ORACLE_HOME are the same, no upgrade actually happens, but just the plugin.

During installation of Oracle I did choose to use the virtual account. This worked fine for the migration in 2 steps. Later on we’ll see that the virtual account causes issues during plugin when doing the upgrade and plugin in 1 step.

See this blog with more information about the virtual account on Windows.

Doing the upgrade and plugin in one step has the following advantage:

There’s no need to manually create the Windows service, because the services of the pre-created container-DB are already there.

To do the upgrade and plugin in one step I used the following autoupgrade-config-file after creating an empty container db CDBSIMC:


# Global parameter
# ================

global.autoupg_log_dir=D:Downloadsoracle19cautoupgradelogs2
global.target_home=D:appcblvirtualproduct19.9.0dbhome_1
global.target_version=19


# Database parameter SIMC
# =======================

upg1.dbname=simc
upg1.start_time=NOW
upg1.source_home=D:appcblvirtualproduct12.2.0dbhome_1
upg1.sid=simc
upg1.upgrade_node=dbi-pc-cbl
# upg1.add_after_upgrade_pfile=D:Downloadsoracle19cautoupgradepfileadd_after_init.ora
upg1.target_cdb=cdbsimc
upg1.target_pdb_name=PDBSIMC
upg1.target_pdb_copy_option=file_name_convert=('D:APPCBLVIRTUALORADATASIMC','D:APPCBLVIRTUALORADATACDBSIMCSIMC')

The upgrade went through, but the plugin failed with the following error:

createpdb_simc_PDBSIMC.log:


Start of SQL*Plus output

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Jan 29 22:54:33 2021
Version 19.9.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

SQL> SQL> Connected.
SQL> SQL> old   1: create pluggable database "&pdbName" &asClone using '&xmlFilePath' &fileNameConvertOption tempfile reuse
new   1: create pluggable database "PDBSIMC" as clone using 'D:Downloadsoracle19cautoupgradelogs2simcsimc105noncdbtopdbPDBSIMC.xml' COPY file_name_convert=('D:APPCBLVIRTUALORADATASIMC','D:APPCBLVIRTUALORADATACDBSIMCSIMC') tempfile reuse
create pluggable database "PDBSIMC" as clone using 'D:Downloadsoracle19cautoupgradelogs2simcsimc105noncdbtopdbPDBSIMC.xml' COPY file_name_convert=('D:APPCBLVIRTUALORADATASIMC','D:APPCBLVIRTUALORADATACDBSIMCSIMC') tempfile reuse
*
ERROR at line 1:
ORA-19505: failed to identify file
"D:APPCBLVIRTUALORADATASIMCSYSTEM01.DBF"
ORA-27041: unable to open file
OSD-04002: unable to open file
O/S-Error: (OS 5) Access is denied.

REMARK: As you can see the plugin is actually a “Clone”. I.e. the original datafiles are kept to be able to go back.

The reason for the issue was that the 12.2.-Window-service runs under permission of the group ORA_OraDB12Home1_SVCACCTS and hence permission to access the associated DB-files are also granted to that group. This does not change during the upgrade to 19c, but of course can be adjusted with oradim. As we skipped the manual step with oradim, the 19c-services for the container DB CDBSIMC (which runs in group ORA_OraDB19Home1_SVCACCTS) do not have access to the DB-files to clone them.

The workaround in my environment was to grant full rights on the db-files to group ORA_OraDB19Home1_SVCACCTS prior to the upgrade/plugin.

Afterwards my upgrade and plugin went through with the following content in the createpdb_simc_PDBSIMC.log:


Start of SQL*Plus output

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Jan 30 01:15:55 2021
Version 19.9.0.0.0

Copyright (c) 1982, 2020, Oracle.  All rights reserved.

SQL> SQL> Connected.
SQL> SQL> old   1: create pluggable database "&pdbName" &asClone using '&xmlFilePath' &fileNameConvertOption tempfile reuse
new   1: create pluggable database "PDBSIMC" as clone using 'D:Downloadsoracle19cautoupgradelogs5simcsimc100noncdbtopdbPDBSIMC.xml' COPY file_name_convert=('D:APPCBLVIRTUALORADATASIMC','D:APPCBLVIRTUALORADATACDBSIMCSIMC') tempfile reuse

Pluggable database created.

The best solution here is of course to use the same local account for all Oracle DB-services and not the virtual account. If you plan to use the virtual account then it is a good idea from the beginning to use the same value for the registry entry ORACLE_SVCUSER for all installed versions. E.g. I used ORA_DBSVCACCTS instead of the defaults ORA_OraDB12Home1_SVCACCTS (for 12.2.) and ORA_OraDB19Home1_SVCACCTS (for 19c) during my tests successfully.

L’article Oracle autoupgrade on Windows and plugin to a Container DB with virtual accounts est apparu en premier sur dbi Blog.

A typical ODA project (and why I love Oracle Database Appliance)

Jérôme Dubar — Tue, 08 Dec 2020 16:47:43 +0000

Introduction

You can say everything about Oracle infrastructure possibilities but nothing compares to experience based on the realization of real projects. I did quite a lot of ODA projects during the past years (on other platforms too), and I would like to tell you why I trust this solution. And for that, I will tell you the story of one of my latest ODA project.

Before choosing ODAs

A serious audit of the current infrastructure is needed, don’t miss that point. Sizing the storage, the memory, the licenses takes several days, but you will need that study. You cannot take good decisions being blind.

The metrics I would recommend to collect for designing your new ODA infrastructure are:
– the needed segregation (PROD/UAT/DEV/TEST/DR…)
– the DBTime from actual databases consolidated for each group
– the size and growth forecast for each database
– the memory usage for SGA and PGA, and the memory advised by Oracle inside each database
– an overview of future projects coming in the next years
– an overview of legacy applications that will stop to use Oracle in the next years

Once done, this audit will help you to choose how many ODAs, which type of ODA, how many disks on each ODA and so on. For this particular project, the audit was done several months before and led to an infrastructure composed of 6 X8-2M ODAs with various disk configurations.

Before delivery

I usually provide an Excel file to my customer for collecting essential information for the new environment:
– hostnames of the servers
– purpose of each server
– public network configuration (IP, netmask, gateway, DNS, NTP, domain)
– ILOM network configuration (IP, netmask, gateway – this is for the management interface)
– additional networks configuration (for backup or administration networks if additional interfaces have been ordered)
– the DATA/RECO repartition of the disks
– the target uid and gid for Linux users on ODA
– the version(s) of the database needed
– the number of cores to enable on each ODA (if using Enterprise Edition – this is how licenses are configured)

With this file, I’m pretty sure to have almost everything before the server are delivered.

Delivery day

Once the ODAs are delivered, you’ll first need to rack them into your datacenters and plug the power and network cables. If you are well organized, you’ll have done the network patching before and the cable would be ready to plug in. Racking an ODA S and M is easy, less than one hour. Even less if you’re used to racking servers. For ODA HA it’s a little bit more complicated, because 1 ODA HA is actually 2 servers and 1 DAS storage enclosure, or 2 DAS enclosures if you ordered the maximum disk configuration. But these are normal servers, and it shouldn’t be too long or too complicated.

1st Day

The first day is an important day because you can do a lot if everything is prepared.

You’ll need several zipfiles to download from MOS to deploy an ODA, and these zipfiles are quite big, 10 to 40GB depending on your configuration. Don’t wait to download them from the links in the documentation. You can choose an older software version than current one, but commonly it’s a good idea to deploy the very latest version. You need to download:
– the ISO file for bare metal reimaging
– the GI clone
– the DB clones for the database versions you planned to use
– the patch files, because you probably need them (we will see why later)

During the download, you can connect to the ILOM IP of the server and configure static IP address. Gathering the IP of the ILOM will need the help from the network/system team, because each ILOM first looks up for a dynamic IP from DHCP, therefore you need to change it.

Once everything is downloaded, you can start the reimaging of the first ODA from the ILOM of the server, then a few minutes later on the second one if you’ve got multiple ODAs. After the reimaging, you will use the configure-firstnet script on each ODA to configure basic network settings on public interface to be able to connect to the server itself.

Once the first ODA is ready, I usually prepare the json file for appliance deployment from the template and according to the settings provided in the excel file. It takes me about 1 hour to make sure nothing is wrong or missing, and then I start the appliance creation. I always create the appliance with a DBTEST database to make sure everything is fine up to database creation. During the first appliance creation, I copy the json file to the other ODA, change the few parameters that differ from the previous one, and also start the appliance creation.

Once done, I deploy the additional dbhomes if needed on both ODAs in parallel.

Then, I check the version of the components with odacli describe-components. You may know that a reimage does not update the microcodes of the ODAs, you should be aware of that. If firmware/bios/ILOM are not up-to-date, you need to apply the patch on top of your deployment, even if the software side is OK. So copy the patch on both ODAs, and apply it. It will probably need a reboot or two.

Once done, I usually configure the number of cores with odacli update-cpucores to match the license. If my customer only has Standard Edition licenses, I also decrease the number of cores to make sure to benefit from maximum CPU speed. See why here.

Finally, I do a sanity check of the 2 ODAs, checking if everything is running fine.

At the end of this first day, 2 ODAs are ready to use.

2nd Day

I’m quite anxious about my ODAs not being exactly identical. So the next day, I deployed the other ODAs. For this particular project, it was a total of 6 ODAs with various disk and license configuration, but the deployment method is the same.

Based on what I did the previous day, I deployed quite easily the 4 other ODAs with their own json deployment file. Then I applied the patch, configured the license, and did the sanity checks.

At the end of this second day, my 6 ODAs were ready to use.

3rd day

2 ODAs came with hardware warnings, and most of the time these warning are false positive. So I did a check from the ILOM CLI, reset the alerts and restart the ILOM. That solved the problem.

As documentation is something quite important for me and for my customer, I spent the rest of the day on consolidating all information and provide a first version of the documentation to the customer.

4th Day

The fourth day was actually the next week. During this time, my customer created a first development database and put it on “production” without any problem.

With every new ODA software release, new features are available. And I think it’s worth it to test these features because it can bring you something very usefull.

This time, I was quite curious about the Data Guard feature now included with odacli. Data Guard manual configuration is always quite long, it’s not very difficult but a lot of steps are needed to achieve this kind of configuration. And you have to do it for each database, meaning that it can takes days to put all your databases under Data Guard protection.

So I proposed to my customer to test the Data Guard implementation with odacli. I created a test database dedicated for that purpose, and followed the online documentation. It took me all the day, but I was able to create the standby, to configure Data Guard, to do a switchover, to do the switchback and to make a clean and complete procedure for the customer. You need to do that because the documentation has 1 or 2 steps that need more accuracy, and 1 or 2 others that need to be adapt to the customer’s environment.

This new feature will definitely simplify the Data Guard configuration, please take a look at my blog post if you’d like to test it.

5th Day

The goal of this day was to make a procedure for configuring Data Guard between the old platform and the new one. An ACFS to ASM conversion was needed, as well. So we worked on that point, made a lot of tests and finally provide a procedure for most cases. A DUPLICATE DATABASE FOR STANDBY with a BACKUP LOCATION was used in that procedure.

This procedure is not ODA specific, most of the advanced operations on ODA are done using classic tools.

6th, 7th and 8th days

These days were dedicated to the ODA workshop. It’s the perfect timing for this training, because the customer already had quite a lot of information during the deployment, and the servers are not yet on production meaning that we can use them for demos and exercises. At dbi services, we make our own training material. Regarding the ODA workshop, it’s start from the history of ODA until the lifecycle management of the plaform. You need 2 to 3 days to have a good overview of the solution and to get ready to work on it.

9th and 10th days

These days were actually extra days in my opinion, extra days are not useless day. Most often, it’s a good addition because there’s always the need to go deeper for some themes.

This time, we need to refine the Data Guard procedure and test it with older versions: 12.2, 12.1 and 11.2. We discovered that it didn’t work for 11.2, and we tried to debug. Finally, we decided to use only the ODA Data Guard feature for 12.1 and later versions, it was OK because only a few databases will not be able to go higher than 11.2. We also found that configuring Data Guard from scratch only takes 40 minutes, including the backup/restore operations (for the smallest possible database), it definitely validated the efficiency of this method over manual configuration.

We also studied the best way to create additional listeners, because odacli does not include the listener management. Using srvctl to do that was quite convenient and clean, so we provided a procedure to configure these listeners, and we also tested the Data Guard feature with these dedicated listeners, it was OK.

Last task was to provide a procedure for ACFS to ASM migration. Once migrated to 19c, 11gR2 databases can be moved to ASM to get rid of ACFS (as ASM has been chosen by the customer for all 12c and later databases). odacli does not provide a mechanism to move from ACFS to ASM, but it’s possible to restore an ACFS database to ASM quite easily.

Actually, these 2 days were very efficient. And I also had enough time to send the v1.1 of the documentation with all the procedures we elaborated together with my customer.

The next days

For this project, my job was limited to deploying/configuring the ODAs and training my customer plus giving him the best practices. With a team of experimented DBAs, it will be quite easy now to continue without me, the next task being to migrate each database to this new infrastructure. Even on ODA, the migration process takes days because there is a lot of databases coming from different versions.

Conclusion

ODA is a great platform to optimize your work, starting from the migration project. You don’t loose time because this solution is ready to use. I don’t think it’s possible to be as efficient with another on-premise platform. For sure, you will save a lot of time, you will have less problems, you will manage everything by yourself, the performance will be good. ODA is probably the best option for achieving this kind of project with minimal risk and maximal efficiency.

L’article A typical ODA project (and why I love Oracle Database Appliance) est apparu en premier sur dbi Blog.

Documentum – Some things to take care of when migrating/upgrading

Morgan Patou — Tue, 14 Jul 2020 17:21:50 +0000

Over the years I have been working with Documentum, I participated in many migrations and/or upgrades. Just like everything else, this obviously builds up some experience but what if you are pretty new to Documentum and it is your first time working on one? Sure, there is the Documentation from OpenText, which is several dozens of pages (~140, depending on versions) of intense fun so prepare a lot of coffee, you will definitively need some. Except for the documentation, there isn’t much information available so I was thinking about writing a rather short blog to list some key things to absolutely look at if you don’t want to lose all your work or worse. This is obviously not going to be an exhaustive list, it’s not really possible, else it would be in the documentation directly.

The definition of a migration and an upgrade can be found on the documentation. If these two keywords aren’t clear to you, you should definitively go take a look because it can be very confusing, especially some elements of the definition from OpenText. Alright so here are a few key takeaways:

Read the documentation. Yeah, I know it’s annoying and you will have to excuse me but I had to. There is a lot of crucial information in the documentation, so you don’t really have the choice. This will really help.

Look at the supported versions early (clients included!). A migration or an upgrade, for Documentum, is a rather complex project and the process that you will need to apply to attain your goal will heavily depend on supported versions. The result will obviously always be the same (the repository is migrated/upgraded) but the path to reach it can be very different. It’s not so easy to define the correct & supported path because there are many possibilities and many things to take into consideration. In addition, there might also be version specific requirements. For example if you are planning to upgrade a Content Server to 16.7 or above, then all DFC Clients (xPlore, DA, D2, …) will also need to be 16.7 or above so this needs to come into the picture.

Think about the data availability. In the migration/upgrade process, you could opt to keep the source system intact and perform the upgrade on a staging environment before moving to the target system: for example to move from a VM to Docker/Kubernetes. Technically, you don’t need the data on the staging to do an in-place upgrade of the repository, all you need is the Database (export the DB from the source and import it into the DB of the staging). However, performing the in-place upgrade on the staging will create some objects in the repository and these objects will therefore have their data created on the staging data folder. Therefore, if the target system isn’t the staging one, you will then be missing some data because it stayed in the staging while the data that you have on the target is the data from the source (or a copy of the data from the source). If you are following me, you can obviously just perform an additional copy of the staging upgraded data (it’s mainly jar, txt and xml files created by DARs installation), that’s simple because you can easily find which data were created by the upgrade. The purpose of the staging is usually to have the binaries for the upgrade so the initial data will be almost empty, most probably still using the folder “content_storage_01//80/00/0x” with x usually equal to 3/4/5. While the data created by the upgrade process will have a much higher id (the next available id, based on the source DB) like “content_storage_01//80/8d/6x” for example. I didn’t test it so that might just be a wild guess but maybe re-installing the DARs would have the same effect? That would be something to test…

Think about encryptions. Whether you are using an AEK key or a Lockbox, you will need to take care about the repository encryption (same for data encryption if you have a TCS license). The simpler is usually to copy the original AEK/Lockbox with its associated repository(ies). However, you might not have this luxury, especially if the source is an old version (AEK needs to be upgraded?).

Think about passwords (can also be seen as related to the point about encryption). You can change all passwords if you want between the source and the target but that will obviously add an overhead. It’s usually simpler to change passwords (if required) once the migration/upgrade is completed but for that, you will probably want to first copy the AEK/Lockbox. That reminds me of blogs I wrote several years ago about password changes on various Documentum components/layers, like for the Lockbox.

Think about the Global Registry Repository. In case of migration, the GR doesn’t really need to be migrated, that is only if the Repository that you want to migrate isn’t the GR, obviously ;). However, there might be some data specific for the normal Repository inside the GR, like D2 Keystore information (D2 4.7+) for example. It’s usually faster to just recreate the needed objects/configurations on the target system, compared to the need to duplicate all efforts to migrate another repo (the GR).

Use the same docbase id. This might sound silly but I prefer to say it, to be sure. If your process involves a different target system, then you will need to install an empty repository on it. It’s not strictly mandatory to use the same docbase id because you can always manually create the needed folder structure on the target system (log folders, data folders, aso…) as well as the needed files (server.ini, dbpasswd.txt, start/stop scripts, ServerApps/ACS config for the repository, aso…). However, doing this manually would take a lot of time to make sure nothing is forgotten while creating the repository, in silent at least, just needs one command to be executed and that’s it. Since you are going to create a repository anyway, you might as well just use directly the correct docbase id from the source. Starting with the CS 7.3, if you want to change the docbase id of a Repository, that’s possible. You don’t have to remove it completely, you can just use the “MigrationUtil” utility (See this blog for an example).

Make sure there are no locked objects. As far as I remember, the documentation only specify that the dm_server_config objects must not be locked. In reality, you must really check all dm_sysobject. It’s not a problem if some user’s documents are locked since the migration/upgrade process will not update them but Documentum OOTB objects shouldn’t otherwise some steps might fail, even sometimes silently which is then pretty hard to detect. I’m saying this for the dm_ objects but it also applies to other types of objects. For example, if you have a d2_dictionary_archive object locked, then running a D2 Config Import will fail and this is just one of many many cases so better be safe than sorry.

Look for version specific steps. I already gave an example above about the CS 16.7+ that needs all DFC clients to be 16.7+ as well but there are other things. For example, if you are trying to upgrade to 16.7+ and if you are using a Lockbox, you will first need to extract the AEK Key from the Lockbox since it has been deprecated (See this blog for guidance).

Verify the components and the logs after. That’s another point that sounds silly but it’s so important… Documentum is a very complex software and there are so many things happening. I have seen so many times things that apparently went well (even according to the main logs) and when you actually start digging into it (looking into Repository objects, looking into sub-logs generated by sub-methods and such), you start seeing a whole new world. Some examples:
1. DARs missing or not installed at all (don’t forget the preserve_existing_types=F during upgrades)
2. Duplicate ACS & JMS objects
3. Lockbox not upgraded (corrupt) (new parameter in 16.4 binaries upgrade)
4. Wrong target_server for the dm_DataDictionaryPublisher job
5. Locked Castor objects due to bug in EBS (introduced in previous patch)
6. JMS or ACS URLs reset to http://localhost
7. JMS that lose its HTTPS configuration
8. aso…

Stay patient and open-minded. Don’t worry, the migration/upgrade will probably fail but that’s OK, you are here to fix it ;).

These are just some key points I like to keep in mind. If there are some you would like to share, feel free and I can also add them to the list.

L’article Documentum – Some things to take care of when migrating/upgrading est apparu en premier sur dbi Blog.

Some myths about PostgreSQL vs. Oracle

Open source Team — Wed, 24 Jun 2020 15:42:04 +0000

By Franck Pachot

.
I originally wrote this as a comment on the following post that you may find on internet:
https://www.2ndquadrant.com/en/blog/oracle-to-postgresql-reasons-to-migrate/
but my comment was not published (many links in it… I suppose it has been flagged as spam?) so I put it there.

You should never take any decision on what you read on the internet without verifying. It is totally valid to consider a move to Open Source databases, but doing it without good understanding is a risk for your migration project success.

In italics are the quotes from the article.

Kirk,
As you do a comparison and link to a list of PostgreSQL features, let me refine the name and description of the Oracle features you compare to, so that people can find them and do a fair comparison. I’m afraid they may not recognize the names and descriptions you provide, at least in current versions. As an example, nobody will get search hits for “Federation”, or “plSQL”, or “HTML DB”… in the Oracle documentation but they will find “Oracle Gateway”, “PL/SQL”, “APEX”…

Federation vs. Foreign Data Wrappers

There is no feature called “Federation”.
The closest from your description is Database links and Heterogeneous Services through Database Gateway. They go further than FDW in many points. But anyway, I would never use that for ETL. ETL needs optimized bulk loads and there are other features for that (like External Tables to read files, and direct-path inserts to fast load). If your goal is to federate and distribute some small reference tables, then Materialized Views is the feature you may look for.
https://docs.oracle.com/en/database/oracle/oracle-database/20/heter/introduction.html#GUID-EC402025-0CC0-401F-AF93-888B8A3089FE

plSQL vs. everything else

“Oracle has a built-in programming language called plSQL.”
PL/SQL is more than that. It is compiled (to pcode or native), manages dependencies (tracks dependencies on schema objects), optimized for data access (UDF can even be compiled to run within the SQL engine), can be multithreaded (Parallel Execution). That’s different from PL/pgSQL which is interpreted at execution time. You mention languages as “as plug-ins” and for this, there are other ways to run different languages (external procedures, OJCM, External Table preprocessor,…) but when it comes to performance, transaction control, dependency tracking,… that’s PL/SQL.
https://docs.oracle.com/en/database/oracle/oracle-database/20/lnpls/overview.html#GUID-17166AA4-14DC-48A6-BE92-3FC758DAA940

Application programming

Providing an “API to communicate with the database” is not about open source as the main goal is: encapsulation and hide implementation details. In order to access internal structures, which is what you mention, Oracle provides relational views (known as V$ views) accessible with the most appropriate API for a relational database: SQL
https://docs.oracle.com/en/database/oracle/oracle-database/20/refrn/dynamic-performance-views.html#GUID-8C5690B0-DE10-4460-86DF-80111869CF4C

Internationalization and Localization

The “globalization toolkit” is only one part of the globalization features. You can also use any “any character encoding, collation and code page” but not relying on the OS implementation of it makes it cross-platform compatible and OS upgrade compatible (see https://wiki.postgresql.org/wiki/Locale_data_changes)
https://docs.oracle.com/en/database/oracle/oracle-database/20/nlspg/overview-of-globalization-support.html#GUID-6DD587EE-6686-4802-9C08-124B495978D5

Web Development

“Oracle acknowledges the existence of HTML through HTML DB. PostgreSQL natively supports JSON, XML and plugs in Javascript”. HTML DB can be found in paper books, but the name is “APEX” since 2006. And it is not (only) about HTML, JSON, or XML but is a low-code Rapid Application Development with no equivalent for other databases.
Support for the structures and languages you mention are all there. The latest trend being JSON: https://docs.oracle.com/en/database/oracle/oracle-database/20/adjsn/index.html

Authentication

“Oracle has a built-in authentication system.”
Yes, to be platform-independent, and has many other External Authentication: https://docs.oracle.com/en/database/oracle/oracle-database/20/dbseg/configuring-authentication.html#GUID-BF8E5E84-FE7E-449C-8081-755BAA4CF8DB

Extensibility

“Oracle has a plug-in system”. I don’t know what you are referring to. Oracle is multi-platform proprietary software. Commercial, which means with vendor supported. There are a lot of APIs for extensions, but the vendor must have to control what runs in the engine in order to provide support.

Read Scalability

“PostgreSQL can create a virtually unlimited read cluster”. Oracle has active/active cluster (called RAC) and read replicas (called Active Data Guard). For horizontal scalability, you use the same as for vertical (Parallel Execution) across multiple nodes (in sync, with instance affinity on partitions,…)
https://docs.oracle.com/en/database/oracle/oracle-database/20/vldbg/parallel-exec-intro.html#GUID-F9A83EDB-42AD-4638-9A2E-F66FE09F2B43

Cost

“they don’t mind charging you again for every single instance.”
No, that’s wrong, license metrics are on processors (CPU) or users (NUP). You run as many instances as you want on your licensed servers for your licensed users: https://www.oracle.com/a/ocom/docs/corporate/oracle-software-licensing-basics.pdf
“jamming everything into a single instance just to reduce costs”.
No, database consolidation is recommended to scale the management of multiple databases, but not for licensing costs. If you go there, there are a lot of features to allow isolation and data movement in consolidated databases: Multitenant, Resource Manager, Online Relocate, Lockdown Profiles,…

Performance

“differentiate the tuning parameters for your warehouse to OLTP to reporting to the data lake”: I already mentioned the point about read replicas and about multiple instances in a server. But with oracle, all the parameters I want to set different for OLTP or reporting do not require another instance. They can be set at session or PDB level. As Oracle does not need the filesystem buffer cache, there’s no need to separate on different servers to avoid noisy neighbours.

I hope this helps to look further at the features. There are many reasons to migrate and the main one is the will to move from a commercial model (with license and support) to an open-source one (start with low cost, help from community). But decision must be made on facts and not rumours.

Franck.

L’article Some myths about PostgreSQL vs. Oracle est apparu en premier sur dbi Blog.

Migrating an Oracle instance to AWS Aurora – 4 – AWS DMS Change Data Capture (CDC)

Daniel Westermann — Mon, 23 Mar 2020 09:41:44 +0000

This is the last post in this little series about migrating on Oracle instance to AWS Aurora with PostgreSQL compatibility. For the previous one you can check here, here and here. In this post we’ll look at how you can configure change data capture so that changes on the source Oracle instance are automatically replicated to the target Aurora instance. In a real life migration this is what you probably need as long down times are usually not acceptable. We’ll be starting from exactly the same state we left of in the previous post.

We’ve initially loaded the source “SH” schema from Oracle to AWS Aurora and now we want to replicate all the changes that are happening on the source to the target as well. Remember that AWS DMS sits between the source and the target and the we need a source end point and a target end point:

We can use exactly the same end points as in the previous post (nothing to add or modify here):

What we need to configure is a new database migration task. We still have the task we used to do the initial loading which could be modified but I prefer to keep it separated and will go ahead with a new task:

We’ll be using the same replication instance and the same end points but “Migration type” will be “Replicate data changes only”. We already enabled “supplemental logging” in the last post so the warning can be ignored:

We’ll go with the default task settings:

As I know that nothing happened on the source I do not need to specify a specific start point for the replication. The same is true for the tables on the target: I already know that the tables are there and that the content of the tables on the target is the same as on the source. For the LOB stuff I’ll go with the defaults as well.

The table mappings section is configured the same as we did it for the initial load previously. We want to replicate all tables in the “SH” schema:

Finally all the defaults for the remaining parameters of the task:

While the task is creating:

… you’ll see that LogMiner is starting on the source Oracle instance:

2020-03-23T08:44:08.261039+01:00
Thread 1 advanced to log sequence 39 (LGWR switch)
  Current log# 3 seq# 39 mem# 0: /u03/oradata/DB19/redog3m1DB19.dbf
  Current log# 3 seq# 39 mem# 1: /u04/oradata/DB19/redog3m2DB19.dbf
2020-03-23T08:44:08.398220+01:00
ARC2 (PID:4670): Archived Log entry 15 added for T-1.S-38 ID 0x2c7a7c94 LAD:1
2020-03-23T08:54:41.598409+01:00
LOGMINER: summary for session# = 2147484673
LOGMINER: StartScn: 1084978 (0x0000000000108e32)
LOGMINER: EndScn: 1085221 (0x0000000000108f25)
LOGMINER: HighConsumedScn: 0
LOGMINER: PSR flags: 0x0
LOGMINER: Session Flags: 0x4000441
LOGMINER: Session Flags2: 0x0
LOGMINER: Read buffers: 4
LOGMINER: Region Queue size: 256
LOGMINER: Redo Queue size: 4096
LOGMINER: Memory LWM: limit 10M, LWM 12M, 80%
LOGMINER: Memory Release Limit: 0M
LOGMINER: Max Decomp Region Memory: 1M
LOGMINER: Transaction Queue Size: 1024
2020-03-23T08:54:41.627157+01:00
LOGMINER: Begin mining logfile for session -2147482623 thread 1 sequence 39, /u03/oradata/DB19/redog3m1DB19.dbf
2020-03-23T08:54:46.780204+01:00
LOGMINER: summary for session# = 2147485697
LOGMINER: StartScn: 1085020 (0x0000000000108e5c)
LOGMINER: EndScn: 1085288 (0x0000000000108f68)
LOGMINER: HighConsumedScn: 0
LOGMINER: PSR flags: 0x0
LOGMINER: Session Flags: 0x4000441
LOGMINER: Session Flags2: 0x0
LOGMINER: Read buffers: 4
LOGMINER: Region Queue size: 256
LOGMINER: Redo Queue size: 4096
LOGMINER: Memory LWM: limit 10M, LWM 12M, 80%
LOGMINER: Memory Release Limit: 0M
LOGMINER: Max Decomp Region Memory: 1M
LOGMINER: Transaction Queue Size: 1024
2020-03-23T08:54:46.788212+01:00
LOGMINER: Begin mining logfile for session -2147481599 thread 1 sequence 39, /u03/oradata/DB19/redog3m1DB19.dbf
2020-03-23T08:54:51.861617+01:00
LOGMINER: summary for session# = 2147486721
LOGMINER: StartScn: 1085087 (0x0000000000108e9f)
LOGMINER: EndScn: 1085362 (0x0000000000108fb2)
LOGMINER: HighConsumedScn: 0
LOGMINER: PSR flags: 0x0
LOGMINER: Session Flags: 0x4000441
LOGMINER: Session Flags2: 0x0
LOGMINER: Read buffers: 4
LOGMINER: Region Queue size: 256
LOGMINER: Redo Queue size: 4096
LOGMINER: Memory LWM: limit 10M, LWM 12M, 80%
LOGMINER: Memory Release Limit: 0M
LOGMINER: Max Decomp Region Memory: 1M
LOGMINER: Transaction Queue Size: 1024
2020-03-23T08:54:51.868450+01:00
LOGMINER: Begin mining logfile for session -2147480575 thread 1 sequence 39, /u03/oradata/DB19/redog3m1DB19.dbf

The task itself will report it’s status as “Replication ongoing” a few moments later:

Having a look at the table statistics section of the tasks of course all is reported as zero at the moment:

Time to do some changes in Oracle and check if these changes will be properly replicated to the Aurora target instance. Let’s create a new table on the Oracle side:

SQL> create table sh.test ( a number );

Table created.

This table will show up in the AWS DMS console quite fast:

… and is also available on the target Aurora side:

postgres=> \d "SH"."TEST"
           Table "SH.TEST"
 Column |      Type      | Modifiers 
--------+----------------+-----------
 A      | numeric(38,10) | 

postgres=> select * from "SH"."TEST";
 A 
---
(0 rows)

Inserting data on the source:

SQL> insert into sh.test values(1);

1 row created.

SQL> commit;

Commit complete.

… and nothing happens on the target side. Why does DDL succeed but DML not? Well, first of all it takes some time for the changes to show up in the console. Doing some more inserts on the source and waiting some time:

SQL> insert into sh.test values(2);

1 row created.

SQL> commit;    

Commit complete.

SQL> insert into sh.test values(3);

1 row created.

SQL> commit;

Commit complete.

… the changes are actually recorded:

… but nothing arrives on the target:

postgres=> select * from "SH"."TEST";
 A 
---
(0 rows)

Why that? Let’s do some more inserts:

SQL> insert into sh.test select -1 from dba_objects;

23522 rows created.

SQL> commit;

Commit complete.

This is reported as “1” insert in the console as the number of inserts switched from 6 to 7:

Some picture, nothing there on the target. The issues was, that supplemental logging needs to be enabled for each table or on the database level, and this is what I did:

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;

Re-creating the table on Oracle side:

SQL> drop table sh.test;

Table dropped.

SQL> create table sh.test ( a number primary key );

Table created.

SQL> insert into sh.test values(1);

1 row created.

SQL> insert into sh.test values(2);

1 row created.

SQL> commit;

Commit complete.

SQL>

From now on new data is showing up in the target Aurora instance:

postgres=> select * from "SH"."TEST";
      A       
--------------
 1.0000000000
 2.0000000000
(2 rows)

postgres=>

So, again, quite easy to setup. Of course the usual bits apply as for every logical replication: You should have primary keys or at least unique keys on the source tables for the replication to run performant and smooth. Another point to add here: When you check the settings of the replication task there is one important parameter that should be enabled (you need to stop the task, otherwise it can not be modified):

Enabling this will create additional tables in the target Aurora instance and these give you more information on what is going on with the replication:

postgres=> select * from pg_tables where tablename like 'awsdms%';
 schemaname  |        tablename        | tableowner | tablespace | hasindexes | hasrules | hastriggers | rowsecurity 
-------------+-------------------------+------------+------------+------------+----------+-------------+-------------
 public      | awsdms_apply_exceptions | postgres   |            | f          | f        | f           | f
 aws_postgis | awsdms_apply_exceptions | postgres   |            | f          | f        | f           | f
 public      | awsdms_history          | postgres   |            | t          | f        | f           | f
 public      | awsdms_status           | postgres   |            | t          | f        | f           | f
 public      | awsdms_suspended_tables | postgres   |            | t          | f        | f           | f
(5 rows)

Here is an example of the replication history:

postgres=> select * from awsdms_history;
      server_name      |         task_name          |   timeslot_type   |      timeslot       | timeslot_duration | timeslot_latency | timeslot_records | timeslot_volume 
-----------------------+----------------------------+-------------------+---------------------+-------------------+------------------+------------------+-----------------
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 09:42:29 |                 0 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 09:47:29 |                 5 |                0 |            70592 |              28
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 09:52:29 |                 5 |                0 |                6 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 09:57:30 |                 5 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 10:02:31 |                 5 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 10:07:31 |                 5 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 10:12:32 |                 5 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 10:17:32 |                 5 |                0 |                0 |               0
 localhost.localdomain | XEBQFBY6EH6SQNN6CBBMO25364 | CHANGE PROCESSING | 2020-03-23 10:22:39 |                 0 |                0 |                0 |               0
(9 rows)

These tables can be used to monitor the replication, especially the apply exceptions.

To conclude this final post: The AWS Schema Conversion Tool is a great help for converting the schema and even comes with some Oracle compatibility. Use it, it saves a lot of manual work. AWS DMS on the other side is really easy to implement, the initial load is really easy to setup and change data capture works as expected. Of course this was only a playground and real issues will pop up when you do a real migration, especially when you have to migrate business logic inside the database.

L’article Migrating an Oracle instance to AWS Aurora – 4 – AWS DMS Change Data Capture (CDC) est apparu en premier sur dbi Blog.