This is part III of the article, see Part I here and Part II here.
Creating the seed docbase
See dbi services for its source. Hereafter, only some salient steps are presented, refer to the script for the details.
As its name implies, the script create_docbase.sh is used to create a docbase optionally to be used later as a seed. Its usage is:
$ ./create_docbase.sh [stem]
where the optional stem in global_parameters defaults to DCTM0 and points to the docbase’s required settings. It is executed as the installation owner, usually dmadmin.
After the global_parameters file is sourced with the given stem, the installation directory is created and all the needed software packages (the JDK, the Oracle Instant Client and SQL*Plus if the Oracle RDBMS is chosen, the PostgreSQL source tarball global_parameters.${postgresql_package} otherwise) are downloaded into the directory global_parameters.${dctm_software}, only if they are not there yet. After their downloading, the packages are expanded and the software installed and configured as advised in the OpenText documentation. For example, the anonymous algorithm for secure communication with the content server is reinstated in the JDK from its default disabled state.
If the selected RDBMS is PostgreSQL, its installation is completely customized. In effect, the platform’s standard PostgreSQL package is normally installed system-wide by root with pieces scattered in several locations, e.g. /var/lib and /usr/lib. As we want to embed this RDBMS with the repository and under the dmadmin ownership, this is not convenient; we want all the pieces in one directory tree root. In order to make this possible, we need to compile the source code specifying the installation directory ${ACTIVE_DOCBASE_ROOT}/postgresql. The binary and data files will all be installed under this directory, which will later be included in the compressed tar ball when the script completes executing. As the compilation takes several minutes and even though it only needs to be done once, a pre-compiled tarball global_parameters.${postgresql_custom_package} is available so this step can be skipped, unless global_parameters.${postgresql_compile} is set to “yes“.
After the PostgreSQL binaries have been extracted or compiled, the ODBC connectivity is configured. Surprisingly, Documentum’s configuration program imperatively needs a connect string named postgres and defined in /etc/odbc.ini as if it were hard-coded. Apparently, it reads this file explicitly instead of relying on the ODBC API which looks for any specified connect string first in ~/.odbc.ini and lastly in /etc/odbc.ini. Also, it requires the running PostgresSQL server to be listening on the default port 5432 instead of any port defined in the ini file. Strangely enough, and this is visible when using the configuration program interactively, ~/.odbc.ini is correctly used and validated but only until the program effectively starts creating the docbase, where it switches to /etc/odbc.ini and the weird behavior. The documentation mentions changes to be done in ODBC.INI but does not say where it is (it’s in /etc but ~/.odbc.ini should be used preferably). Also, it incorrectly says that the connect string should be [MyPostgres] whereas it should be anything, although only [postgres] works at this stage. All this suggests that the ODBC part is not well mastered by developpers of the installation program, or that it was ported without much attention from some other O/S. Fortunately, these idiosyncrasies are abandoned after the docbase is created and all the parameters set in the installer owner’s ~/.odbc.ini are honored, e.g. custom connect string and database server port.
Here is how /etc/odbc.ini must look for the docbase creation to start:
[postgres]
Description = PostgreSQL connection to postgres
Driver = PostgreSQL Unicode
Database = postgres
Servername = ${db_server_host_alias}
UserName = ${dctm_owner}
# hard-coded in the server configuration tool;
Port = 5432
and here is the final ~/.odbc.ini file defined after the docbase creation, including some of the recommended database settings:
[${db_connect_string}]
Description = PostgreSQL connection to ${ACTIVE_DOCBASE}
Driver = PostgreSQL Unicode
Database = ${ACTIVE_DOCBASE}
Servername = ${db_server_host_alias}
UserName = ${dctm_owner}
Port = ${ACTIVE_DB_SERVER_PORT}
Protocol = $(echo ${postgresql_custom_package} | sed -E 's/.+-([0-9]+).*$/\1/')
ReadOnly = No
RowVersioning = No
ShowSystemTables = No
ShowOidColumn = No
FakeOidIndex = No
UpdateableCursors = Yes
The shell variables get expanded when the file is saved.
Another requirement with the postgreSQL RDBMS is that the installer demands a directory named db_${ACTIVE_DOCBASE}_dat.dat for the datafile to be created beforehand; if not found, the installer fails.
When using the Oracle RDBMS, a global_parameters.${dctm_root}/docbase/oracle/instantclient_21_7/network/admin/tnsnames.ora file gets created with the following content:
${db_connect_string} =
(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)
(HOST = ${db_server_host_alias})(PORT = ${db_listener_port}
)
)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = ${db_service_name})
)
)
Next, the data and index tablespaces are created and finally the schema account with the required grants. Here is a summary of the database requirements for Oracle:
select TABLESPACE_NAME, ALLOCATION_TYPE from dba_tablespaces order by 1;
TABLESPACE_NAME ALLOCATION_TYPE
------------------------------ ---------------------------
DCTM1_DATA SYSTEM
DCTM1_INDEX SYSTEM
...
SEED_DATA SYSTEM
SEED_INDEX SYSTEM
SQL> select privilege from dba_sys_privs where grantee = upper('dctm1');
GRANTEE PRIVILEGE ADMIN_OPT COMMON
------------------------------ ------------------------------ --------- ---------
DCTM1 CREATE VIEW NO NO
DCTM1 CREATE ANY VIEW NO NO
DCTM1 CREATE SEQUENCE NO NO
DCTM1 CREATE PROCEDURE NO NO
DCTM1 CREATE TABLE NO NO
SQL> select * from dba_role_privs where grantee = upper('dctm1');
GRANTEE GRANTED_ROLE ADMIN_OPT DEFAULT_R COMMON
------------------------------ ------------------------------ --------- --------- ---------
DCTM1 CONNECT NO YES NO
DCTM1 RESOURCE NO YES NO
DCTM1 SELECT_CATALOG_ROLE NO YES NO
SQL> select TABLESPACE_NAME, STATUS, ALLOCATION_TYPE, SEGMENT_SPACE_MANAGEMENT from user_tablespaces;
TABLESPACE_NAME STATUS ALLOCATION_TYPE SEGMENT_SPACE_MANA
------------------------------ --------------------------- --------------------------- ------------------
DCTM1_DATA ONLINE SYSTEM AUTO
DCTM1_INDEX ONLINE SYSTEM AUTO
After the database steps have been completed, the documentum’s installer program serverSetup.bin is launched with the following generated on-the-fly response file:
INSTALLER_UI=silent
KEEP_TEMP_FILE=true
common.installOwner.password=dmadmin
SERVER.SECURE.ROOT_PASSWORD=root
# documentum binaries;
SERVER.DOCUMENTUM=${ACTIVE_ROOT}/documentum
APPSERVER.SERVER_HTTP_PORT=${ACTIVE_SERVER_HTTP_PORT}
APPSERVER.SECURE.PASSWORD=tomcat
# java;
PATH_TO_JAVA=$(ls -d ${ACTIVE_ROOT}/java/amazon-corretto-11*-linux-x64)
This step takes several minutes to complete but it does not matter; as said previously, the real optimization occurs in the instantiation step, not in the installation of the binaries or the creation of the model docbase; those steps are done only once (unless it is done outside the context of this project).
Method servers are dedicated to each docbase and receive a distinct base port number defined as global_properties.${stem_SERVER_HTTP_PORT}. Later during the instantiation, their memory sizing will be configured as set in global_properties.${stem_SERVER_HTTP_MEMORY}.
It has been noticed that the method server creates several cache directories (e.g. felix-cache and activemq-data) in the directory it is started from; to move them out of the way and prevent cluttering, the command cd ${DM_JMS_HOME}/temp is later inserted in startMethodServer.sh before tomcat’s startup.sh script is invoked.
After the binaries have been installed, a connectivity check using documentum’s dmdbtest is performed to confirm that the database is reachable via SQL*Net for Oracle or via ODBC for PostgreSQL. This test differs from the ones previously done directly at the database level in that it is performed by documentum. Its output resembles the following one:
if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then
${DOCUMENTUM}/product/*/bin/dmdbtest -Dxx -S${db_connect_string} -U${ACTIVE_DOCBASE} -P${ACTIVE_DOCBASE}
else
${DOCUMENTUM}/product/*/bin/dmdbtest -Dpostgres -Spostgres -U${dctm_owner} -Pxxx
fi
Database successfully opened.
Test table successfully created.
Test view successfully created.
Test index successfully created.
Insert into table successfully done.
Index successfully dropped.
View successfully dropped.
Database case sensitivity test successfully past.
Table successfully dropped.
Note that the above test needs an extra step when using Oracle RDBMS because although the sqlplus executable is installed in Instant Client’s ${ORACLE_HOME} and ${ORACLE_HOME} is in the ${PATH}, the installer expects it in the non-existing ${ORACLE_HOME}/bin. So, that directory must be created beforehand and sqlplus symlinked there. Evidently, sqlplus’ expected location is hard-coded and the ${ORACLE_HOME} environment variable is either not passed to the java programs invoked by the installer script, or ignored when looking for it. This quirk can be half-forgiven as it is a documented requirement.
Since several docbases can potentially be instantiated and run on the same machine, an environment file named docbase_name.env for each of them is prepared to allow switching easily between them. This file notably defines the ${DOCUMENTUM} and ${PATH} variables, invokes documentum’s own dm_set_server_env.sh, and should be sourced prior to working with a given docbase. To simplify this, the bash function swr() (which stands for SWitch Repository) has been defined in the installation owner’s ~/.profile. Other useful management functions such as sur()and sdr(), have also been defined in docbase_name.env, see down below.
Next, another response file is prepared to install the docbroker. docbrokers too are dedicated to each docbase and defined in the global_properties file with parameters stem_DOCBROKER_NAME and stem_DOCBROKER_PORT. Make sure there is no port conflict when creating/instantiating new docbases. Here is an example of this response file:
KEEP_TEMP_FILE=true
PATH_TO_JAVA=$(ls -d ${ACTIVE_ROOT}/java/amazon-corretto-11*-linux-x64)
INSTALLER_UI=silent
common.aek.algorithm=AES_128_CBC
# docbroker;
SERVER.CONFIGURATOR.BROKER=TRUE
SERVER.DOCBROKER_ACTION=CREATE
SERVER.DOCBROKER_PORT=${ACTIVE_DOCBROKER_PORT}
SERVER.DOCBROKER_NAME=${ACTIVE_DOCBROKER_NAME}
SERVER.PROJECTED_DOCBROKER_HOST=$(hostname)
SERVER.PROJECTED_DOCBROKER_PORT=${ACTIVE_DOCBROKER_PORT}
SERVER.DOCBROKER_CONNECT_MODE=native
SERVER.USE_CERTIFICATES=false
Make sure the docbroker name does not contains any non alphanumeric characters as the installer is quite restrictive in this regard. For example, the name docbroker_01 is rejected because it contains an underscore.
Documentum’s dm_launch_server_config_program.sh is then invoked with that response file and the docbroker gets created and started along with the method server.
Next, another response file is prepared for the repository. Its most complicated part is the conditional definitions for Oracle and PostgreSQL:
$ cat - <<eot > ${ACTIVE_DOCBASE}.properties
INSTALLER_UI=silent
KEEP_TEMP_FILE=true
...
############################### database stuff #######################################
SERVER.USE_EXISTING_DATABASE_ACCOUNT=$(if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then echo "true"; else echo "false"; fi)
$(if [[ "${ACTIVE_RDBMS}" == "postgres" ]]; then echo "SERVER.DATABASE_NAME=${ACTIVE_DOCBASE}"; fi)
SERVER.INDEXSPACE_NAME=$(if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then echo "${ACTIVE_DOCBASE}_INDEX"; else echo "db_${ACTIVE_DOCBASE}_log"; fi)
# postgres is hard-coded in the server configuration script;
SERVER.DATABASE_CONNECTION=$(if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then echo "${db_connect_string}"; else echo "postgres"; fi)
SERVER.DOCBASE_OWNER_NAME=${ACTIVE_DOCBASE}
SERVER.SECURE.DOCBASE_OWNER_PASSWORD=${ACTIVE_DOCBASE}
SERVER.DATABASE_ADMIN_NAME=$(if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then echo "${ACTIVE_DOCBASE}"; else echo "${dctm_owner}"; fi)
SERVER.SECURE.DATABASE_ADMIN_PASSWORD=$(if [[ "${ACTIVE_RDBMS}" == "oracle" ]]; then echo "${ACTIVE_DOCBASE}"; else echo "${dctm_password}"; fi)
$(if [[ "${ACTIVE_RDBMS}" == "postgres" ]]; then echo "SERVER.POSTGRES_USE_DEFAULT_SPACE=false"; fi)
...
Refer to the code on dbi services‘s git for the complete settings. Detailed explanations about the content server’s response file parameters can also be found here Documentum – Silent Install – Docbases/Repositories. There are articles on our blog site about the response files for other Documentum components too, so it may be worth giving them a look. Links to all of them can be found here.
After the configuration program is invoked with the above response file and completes, the documentum processes are shut down and, if using Oracle, the resulting seed’s database schema will be extracted using the traditional exp utility. Although the exp and imp utilities have been superseded by the data pump for a while now, they do their job well enough for repositories’ schemas. Of course, the more modern data pump can also be used instead under certain conditions, but it may need some preliminary work on the database’s side to accommodate it. The same applies if using the Instant Client’s exp and imp utilities when these tools’ version differs from the database’s. For example, one of our test database is a v12.1.0.1.0 one but there are no downloadable exp and imp tools with that version for the Instant Client as they started being included only in v12.2.0.x (cf. https://download.oracle.com/otn/linux/instantclient/122010/instantclient-basic-linux.x64-12.2.0.1.0.zip and https://download.oracle.com/otn/linux/instantclient/122010/instantclient-tools-linux.x64-12.2.0.1.0.zip). And since those versions differ from the database’s, a PL/SQL package must be executed so they can be used. All this is quite complicated and to simplify the data extraction and import tasks, we will use the exp and imp utilities bundled with the RDBMS; those will always work as-is so no additional download nor configuration are necessary. However, those tools need to be invoked from within a remote session, hence the settings global_parameters.${db_server_host_*}, unless those steps are delegated to the DBAs. As already discussed, Oracle RDBMS is a proprietary, closed-source and quite complex software. Starting with the licensing, it requires some careful planning. Sometimes, databases are even shared between several applications. Therefore, it is mostly managed by specialized people in an organization and installed in dedicated machines. For those reasons, the Oracle part takes up a large chunk of the procedure dedicated to the RDBMS and, for faster and more agile configuration, the leaner but sufficient PostgreSQL RDBMS is preferred. As precedently written, the postgreSQL source is compiled so it can be embedded with the docbase that uses it and the whole binaries and database get copied into the tar ball, which is a considerable simplification at a measly cost of 276 Mb.
After the compressed tar ball is produced, it is renamed to ${ACTIVE_DOCBASE}_${ACTIVE_RDBMS}.tgz and moved to the ${scripts_dir} if one with the same name does not exist there yet (otherwise, the move is not performed), ready to be instantiated as needed.
A useful by-product of this project, although it won’t benefit from the optimized provision time, the generic script create_docbase.sh creates any docbase whose parameters are set in global_parameters. Once created, this docbase can be used stand-alone, or taken a snapshot of and used as a seed for future instantiations, or both. It can even be uncompressed in several places on the same machine or on different machines as distinct clones, provided resource conflicts are resolved and the usual well-known adjustments post-cloning are applied. The script can create several docbases to be used as models for different purposes after further customizations; edit the script as required.
It is possible to specify as many docbases to be created (or instantiated, see later) in global_parameters.${dctm_machine} as needed by specifying their stems on the command-line. So, if several docbases need be created, define them all in global_parameters and invoke the creation script as follows:
# Example:
$ for i in {1..10}; do
./create_docbase.sh "DCTM${i}"
done
where DCTM1_* to DCTM10_* are the respective docbases's stem "pointing" to the settings defined in the global_properties file.
# Another example with unrelated stems:
./create_docbase.sh bookstore
./create_docbase.sh product_catalog
./create_docbase.sh research_papers
Concurrent docbase creation would be another useful by-product of the project and should be possible because each created docbase is normally stand-alone (i.e. no dependency on other services such as the docbroker or the method server) and has a distinct ${DOCUMENTUM} directory under global_parameters.${dctm_root}/docbase_name. Unfortunately, Documentum’s installer program does not permit more than one running instance of itself:
Installing the CS binaries documentum_server_22.4_linux64_oracle.tar ...
Preparing to install
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...
Launching installer...
Multiple launches of this installer is not allowed. It will now quit.
The configuration program dm_launch_server_config_program.sh, however, does not have this limitation, probably because it is enough for it to run in its own ${DOCUMENTUM}.
Granted, when using PostgreSQL , since the docbase creation demands a server listening on port 5432, there may be a short period of time when a server gets commonly used by several instances of dm_launch_server_config_program.sh. Also, multiple database server instances get started on port 5432, but with only the first one succeeding. Still, this should not be an issue since the databases and their locations are distinct. This glitch does not happen at instantiation time as the PostgreSQL server is always started from the beginning with a custom port.
The additional cost is 2 Gb of disk space since the documentum binaries are not shared between the repositories (i.e. each repository has its own ${DOCUMENTUM} and its own copy of the binaries), but disk space is so cheap these days that this is not prohibitive. When using Oracle, the above concurrency behavior does not apply.
As mentioned before, create_docbase.sh sets up a sourceable script to define a working docbase’s environment and several useful functions, global_properties.${dctm_root}/docbase/docbase.env. In particular, it defines:
${ACTIVE_DOCBASE}, ${ACTIVE_ROOT}, ${JAVA_HOME}, ${DOCUMENTUM} and ${PATH}, and sources documentum’s own dm_set_server_env.sh. ${ACTIVE_DOCBASE} is the name of the current repository and ${ACTIVE_ROOT} the root directory that contains that docbase’s ${JAVA_HOME}, ${DOCUMENTUM} and the postgres root directory if that RDBMS is used. ${ACTIVE_ROOT} is also equal to global_properties.${dctm_root}/docbase.
In addition to those environment variables, several management functions are defined too; see the paragraph Management Commands in Part V.
For more comfort, the environment file also changes the shell’s prompt to something nicer, e.g.:
[email protected]:[/u01/dctm/repo03][repo03] $
The prompt shows the currently logged user and the machine it is logged on, the current working directory, and the current active docbase, which is useful to prevent manipulating the wrong docbase when several of them coexist in the machine.
See Part IV here
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/CEC_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/GME_web-min-scaled.jpg)