This is part V of the article, see Part I here, Part II here, Part III here and Part IV here.

Timings

As said before, absolute timings vary a lot because they depends on many factors, starting with the hardware in use. Thus, the given values are just examples of what can be achieved in a simple setup such as a laptop, and they are nonetheless quite impressive. The relevant information here is to compare the creation times with the instantiation times when 1, 2, … 5 and 10 docbases are created at the same time.

It is well understood that Oracle’s slower timings are not caused by Oracle itself but by the more complex procedure to duplicate the docbase’s schema, vs. nothing to duplicate in PostgreSQL since the database gets included with the content server and customized in place. Maybe, similar results as PostgreSQL could be obtained with a local Oracle Express Edition RDBMS, to be tried sometime if the stingy 2 Gb RAM and 12 Gb data limits are acceptable (note that Oracle XE is likely not certified for Documentum, likely because of its limits), although there is no way in Oracle to rename a schema barring exporting/importing it, which we already do here except mostly remotely.

The fastest test bed we had access to was composed of:

  • Hardware: DELL XPS15 2019, i9 9980HK 8 cores, 2TB SSD, 64GB RAM
  • Virtualization: Proxmox Virtual Environment 7.2-3

The following 2 linux container hosts were created:

  • oracle-8, 4 cores, 10 GiB RAM, Oracle Linux Server 8.6 and Oracle RDBMS v21.3.0.0.0
  • cs2, 8 cores, 20 GiB RAM, Ubuntu 20.04.5 LTS for the Content Server v22.4

In both machines, the amount of memory was large enough to prevent the 1 GiB large swap to be used with the ensuing dramatic slowness.

The table below gives the average and average maximum times to first create a seed repository and then instantiate 1 to 5 + 10 concurrent times per repository. Averages were based on 3 to 5 executions, depending on the variability of the timings (the more variable timings, the more executions were performed).

As an example, the values in bold means we instantiated 10 docbases with Oracle RDBMS as the database and it took on average 14m43s per docbase, whereas the longest time was 15m7s in average. We can therefore tell that in average it took 15m7s to create that set of 10 docbases on the same machine. Had we created them the normal and sequential way, it would have taken on average 10 * 21m20s, i.e. almost 3.5 hours or 14 times more. That’s a lot of long coffee breaks.

RDBMScreation11 concurrent2 instantiation2 concurrent2 instantiations3 concurrent2 instantiations4 concurrent2,3 instantiations5 concurrent2,3 instantiations10 concurrent2,3 instantiations
Oracle
max
21m20s 21m56s3m7s
3m17s
4m25s 4m29s4m38s 4m42s5m59s
6m5s
7m13s
7m31s
14m43s
15m7s
PostgreSQL max18m22s 18m25s0m57s
1m
1m4s
1m5s
1m11s
1m16s
1m18s
1m26s
1m30s
1m40s
2m49s
3m11s
timings for docbases’ creation and instantiation

When creating a single docbase by instantiating it, it took in average 3m7s. The concurrency placed a toll on the performances, but of course this could be alleviated somewhat by adding more resources to the containers.

With PostgreSQL RDBMS as the database, the gain is even better: it took as little as 57s to instantiate one single docbase on average, and 2m49s per docbase to create 10 docbases on the same machine, and overall 3m11s until the last one was ready. Had we created them the normal and sequential way, it would have taken 10 * 18m22s, i.e. almost 3 hours or more than 60 times more ! The instantiation procedure is between 3 and 5 times faster with PostgreSQL than with Oracle but both are so fast that it does not make any practical difference, at least for a few docbases.

As expected, measured times are quite variable so only the ratios between instantiation and creation times are really comparative. Still, at the very least, the absolute timings prove that is is possible to have excellent performance on some out of the box, run-of-the-mill hardware, such as developer’s laptops.

Notes

1The command was:

$ time ./create_docbase.sh DCTM0 2>&1 | tee /tmp/dctm0.log

with the settings for the stem DCTM0 defined in global_properties.

2The commands for n concurrent instantiations was for instance:

$ for i in {1..n}; do
   time ./instantiate_docbase.sh DCTM0 ORA_DCTM${i} 2>&1 | tee /tmp/ORA_DCTM${i}.log &
done

with the settings for the stems *DCTM1 to *DCTM10 defined in global_properties.

3After an increase of the number of cores from 4 to 20 and of the memory from 10 GiB to 20 GiB in oracle-8, and of the number of cores from 8 to 20 and of the memory from 20 GiB to 30 GiB in cs2. Also, make sure there is enough disk space in both oracle-8 and cs2, e.g. for 10 repositories, 50 GiB for the Oracle data files and 50 GiB for the documentum files.

Management commands

Now that creating docbases has become such an expedite and mundane task (docbases are a commodity at this point), chances are that they’ll start piling up in the same machine and therefore we need a way to manage them easily. At the minimum, such functions must be able to:

  • switch among the installed docbases
  • show the currently selected docbase and its instantiation parameters
  • start/stop/query the status each of a docbase’s component
  • restart a docbase, a docbroker or a method server
  • navigate to folders of interest in a docbase’s directory tree
  • view/tail a docbase’s logs if interest
  • connect to the currently selected docbase and to its underlying database using native tools
  • cleanly remove an instantiated docbase or all of them at once

Like swr() glimpsed precedently, several functions and bash aliases have been created to fulfill the above needs.

Here are they are, defined in each created or instantiated repository’s environment file ${dctm_root}.${repo}/${repo}.env:

# function shr, for SHow Repository;
# displays the currently selected or given repository's info;
# Usage:
# shr[repository_name]
shr()

# alias whr for WHich Repository to show the currently selected repository;
alias whr=shr

# function sur, for StartUp Repository, to start a repository's all processes;
# it depends on function swr above;
# Usage:
# sur [repository_name]
# when invoked without parameters, the currently selected repository environment is started up;
# when invoked with a parameter, it becomes the current selected repository and its environment is started up;
sur()

# function sdr, for ShutDown Repository, to stop a repository's all processes;
# it depends on function swr above;
# Usage:
# sdr [repository_name]
# when invoked without parameters, the currently selected repository environment is shut down;
# when invoked with a parameter, it becomes the current selected repository and its environment is started up;
sdr()

# function str, for STatus Repository, to show the status of a repository;
# it depends on functions swr;
# Usage:
# rst [repository_name]
# when invoked without parameters, the currently selected repository environment is queried;
# when invoked with a parameter, it becomes the current selected repository and its environment is started up;
rst()

# bounced, for bounce Docbase, stop/start the repository;
alias bounced

# bounceb, for bounce Broker, stop/start the docbroker by minimizing the lost connectivity to the repository;
alias bounceb

# bouncems, for bounce Method Server;
alias bouncems

# stop & start components;
alias stopr=dm_shutdown_\${ACTIVE_DOCBASE}
alias startr=dm_start_\${ACTIVE_DOCBASE}
alias stopb=dm_stop_${ACTIVE_DOCBROKER_NAME}
alias startb=dm_launch_${ACTIVE_DOCBROKER_NAME}
alias stopms=\${DM_JMS_HOME}/bin/stopMethodServer.sh
alias startms=\${DM_JMS_HOME}/bin/startMethodServer.sh

# cd to directories of interest;
alias croot
alias cdroot
alias cdctm
alias cdba
alias clog

# view files/logs of interest;
# less server.ini
alias lconf

# less repository’s log;
alias llog

# less tomcat’s catalina.out;
alias lcat

# less the method server log;
alias lms

# tails logs of interest; also doable from within the less utility with the F command;
# tail the repository’s log;
alias tlog

# tails tomcat’s catalina.out;
alias tcat

# tails the method server log;
alias tms

# interactive utilities;
# for the repositories;
alias iapi
alias idql
alias iiapi
alias iidql

# for PostgreSQL
alias iisql
alias ipsql

# for Oracle;
alias sqlplus
alias isqlp

# function rmr, for Remove Repository, to remove a repository by wiping off its working directory;
# it depends on functions swr and uses sdr defined above;
# Usage:
# rmr [repository_name]
# when invoked without parameters, the currently selected repository environment is removed;
# when invoked with a parameter, it becomes the current selected repository and is removed;
rmr()

Several global functions are also defined in ~/.profile:

# alias lsr for LS Repository;
# list the existing, instanciated repositories;
alias lsr

# function swr, for SWitch Repository;
# tool to switch between instanciated repositories on the same machine and ${dctm_root};
# current working directory is moved to the selected repository's ${DOCUMENTUM};
# Usage:
# swr [repository_name]
# when invoked without parameters, the currently sourced repository's environment is displayed, without first refreshing it;
#
swr()

# function sura, for Start Up Repository All;
# start all the existing, instantiated repositories;
# Usage:
#    sura
#
sura()

# function sdra, for Shut Down Repository All;
# shut down all existing, instantiated repositories;
# Usage:
#    sdra
#
sdra()

# function shra, for Show Repository All;
# show all the existing, instantiated repositories;
# Usage:
#    shra
#
shra()

# function stra, for STatus Repository All;
# show the status of all existing, instantiated repositories;
# Usage:
#    stra
#
stra()

# function rmra, for RM Repository All;
# remove all existing, instantiated repositories;
# Usage:
# rmra
#
rmra()

swr() [repo] is the first function to invoke when starting a new shell and no repository is selected yet. It will switch to the docbase repo. If unsure about repo, a list of existing instantiated repositories can be obtained through the alias lsr which simply list the directories in ${dctm_root}, e.g.:

$ lsr
repo02
repo03
repo04
repo05
repo06
repo07
repo08
repo09
repo10
repo12

$ swr repo12
Switching to repository repo12 ...
ACTIVE_DOCBASE=repo12
ACTIVE_ROOT=/u01/dctm/repo12
CLASSPATH=/u01/dctm/repo12/documentum/product/22.4/dctm-server.jar:/u01/dctm/repo12/documentum/dctm.jar:/u01/dctm/repo12/documentum/dfc/bcprov-jdk15on.jar:/u01/dctm/repo12/documentum/config:/u01/dctm/repo12/documentum/product/22.4/bin:
DM_HOME=/u01/dctm/repo12/documentum/product/22.4
DM_JMS_HOME=/u01/dctm/repo12/documentum/tomcat9.0.65
DOCUMENTUM=/u01/dctm/repo12/documentum
DOCUMENTUM_SHARED=/u01/dctm/repo12/documentum
JAVA_HOME=/u01/dctm/repo12/documentum/java64/JAVA_LINK
LD_LIBRARY_PATH=/u01/dctm/repo12/documentum/product/22.4/bin:/u01/dctm/repo12/documentum/dfc:/u01/dctm/repo12/documentum/fulltext/dsearch:/u01/dctm/repo12/postgresql/lib:
PATH=/u01/dctm/repo12/documentum/java64/JAVA_LINK/bin:/u01/dctm/repo12/documentum/product/22.4/bin:/u01/dctm/repo12/documentum/dba:/u01/dctm/repo12/postgresql/bin:/u01/dctm/repo12/documentum/tomcat9.0.65/bin:/u01/dctm/repo12/java/amazon-corretto-11.0.17.8.1-linux-x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
POSTGRESQL_HOME=/u01/dctm/repo12/postgresql
PS1=\[\033[0;32m\]\u@\h:\[\033[36m\][\w][repo12]\[\033[0m\] $ 
dfcpath=/u01/dctm/repo12/documentum/dfc
Repository repo12's environment is:
Active docbase name is                  : repo12
Active docbase id is                    : 100013
Active docbase service name is          : repo12
Active docbase service port is          : 50120
Active docbase host is                  : cs2
Active docbase version                  : 22.4.0000.0120
Active docbase root directory is        : /u01/dctm/repo12
Active installer owner is               : dmadmin
Active installer password is            : dmadmin
Active docbase docbroker name           : docbroker010
Active docbase docbroker port           : 1511
Active docbase http server base port    : 9800
Active docbase http server memory       : "-Xms512m -Xmx1024m"
JAVA_HOME                               : /u01/dctm/repo12/documentum/java64/JAVA_LINK
JAVA_VERSION                            :
openjdk version "11.0.17" 2022-10-18 LTS
OpenJDK Runtime Environment Corretto-11.0.17.8.1 (build 11.0.17+8-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.17.8.1 (build 11.0.17+8-LTS, mixed mode)
dctm_root                               : /u01/dctm
Scripts'dir is                          : /mnt/shared/blog-seed-docbase
CLASSPATH                               : /u01/dctm/repo12/documentum/product/22.4/dctm-server.jar:/u01/dctm/repo12/documentum/dctm.jar:/u01/dctm/repo12/documentum/dfc/bcprov-jdk15on.jar:/u01/dctm/repo12/documentum/config:/u01/dctm/repo12/documentum/product/22.4/bin:
DM_HOME                                 : /u01/dctm/repo12/documentum/product/22.4
DOCUMENTUM                              : /u01/dctm/repo12/documentum
DM_JMS_HOME                             : /u01/dctm/repo12/documentum/tomcat9.0.65
PATH                                    : /u01/dctm/repo12/documentum/java64/JAVA_LINK/bin:/u01/dctm/repo12/documentum/product/22.4/bin:/u01/dctm/repo12/documentum/dba:/u01/dctm/repo12/postgresql/bin:/u01/dctm/repo12/documentum/tomcat9.0.65/bin:/u01/dctm/repo12/java/amazon-corretto-11.0.17.8.1-linux-x64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LD_LIBRARY_PATH                         : /u01/dctm/repo12/documentum/product/22.4/bin:/u01/dctm/repo12/documentum/dfc:/u01/dctm/repo12/documentum/fulltext/dsearch:/u01/dctm/repo12/postgresql/lib:
Active database owner is                : repo12
Active database password                : repo12
Active database connection string       : repo12
Active database server port             : 5422
DATABASE_TYPE                           : Postgresql

dmadmin@cs2:[/u01/dctm/repo12][repo12] $ 

It may help to compare functions defined in ~/.profile with static methods in OO programming, whereas functions and aliases defined in ${dctm_root}.${repo}/${repo}.env are similar to instance methods: the former apply to any repository and the latter to the current one, unless a different one is explictly specified. Note that if one of ~/.bash_profile or ~/.bash_login is present, ~/.profile is ignored; Ubuntu tends to use ~/.profile whereas Red Hat derivatives use ~/.bash_profile, so adapt this as needed in your linux variant.

The repository-related functions takes optional docbase names, not stems, because the latter don’t apply to existing docbases and are exclusively used in the create_docbase.sh and instantiate_docbase.sh scripts.

These functions are mostly useful when several repositories have been instantiated on the same machine and one needs to switch quickly among them (see ~/.profile’s swr() above) to perform certain tasks such as show the current one (shr()), start it up (sur()), shut it down (sdr()), check the status of its services (str() with colorization), restart services (bounced(), bounceb() and bouncems()), check the logs and configuration (lconf, [lt]log, [lt]cat, [lt]ms), navigate directories of interest (croot, cdroot, cdctm, cdba, clog), launch utilities (iiapi, iidql, iisql, ipsql, isqlp), or wipe it off altogether (rmr()). In effect, removing an instantiated repository should not be any more complicated than instantiating a new one. rmr() also takes care of the Oracle schema if one is used; to do so, it first transfers the script rm_user.sh to the Oracle host and then invokes it. This script removes the database user ${ACTIVE_DATABASE_OWNER}, and drops its tablespaces and datafiles in a safely manner, or so. For PostgreSQL, no such script is needed since the embedded database is removed when the ${ACTIVE_ROOT} directory tree is deleted. Moreover, the command-line utilities iapi, idql, sqlplus, isql and psql have been aliased to use the great rlwrap tool; refer to the article here for more details.

The alias bounceb that stops/starts a docbroker is quite clever and deserves some explanation. When a docbroker is stopped, the docbase that targets it becomes unreachable and stays so until its next projection period, even up to several seconds after the docbroker is back on-line. In order to minimize that unavailability delay, an iapi session is first started while listening to a temporary named pipe, connects to the docbase and is sent to the background; then, the docbroker is restarted, a reinit command is written into the pipe and gets processed by iapi so the docbase projects immediately to the freshly started docbroker. Finally, the quit command is sent to iapi through the named pipe to exit the tool, and the pipe is removed. This works because iapi keeps its connection to the docbase even when the docbroker is stopped. In effect, the docbroker is only used to allow clients to establish a session with the docbases that project onto it; once connected, they don’t need it any more. In that respect, the docbroker has a function similar to an Oracle listener.

Conclusion

While the original idea was simply to speed up the creation of docbases, some of its deliverables can be used as stand-alone utilities, e.g. the script create_docbase.sh, the management aliases and functions.

Those scripts can be used as the basis or greatest common divisor of a more ambitious tool to streamline the installation and operation of Documentum software, e.g. xPlore, DA, etc., on any environment such as VMs, linux or OCI containers.

However useful, the scripts’ main purpose is only to give a practical example of the approach. They were written with a clear context in mind: an individual with full control of their infrastructure (e.g. a powerful enough laptop with virtualization software such as Proxmox), which is not always the case, especially with commercial software. Thus, be prepared to do some adaptations to suit different, likely more rigid contexts, and fix a few bugs at the same time too.

Many enhancements are possible, e.g. use OCI containers, convert any docbase installation into a seed one, etc.; there is enough work to keep one busy during those long, freezing winter nights. Incidentally, it is not impossible that some of them will be covered in future articles, so watch this space.