{"id":21853,"date":"2023-01-28T12:51:16","date_gmt":"2023-01-28T11:51:16","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/?p=21853"},"modified":"2025-10-24T09:38:32","modified_gmt":"2025-10-24T07:38:32","slug":"a-quick-repository-creation-utility-part-iv","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/a-quick-repository-creation-utility-part-iv\/","title":{"rendered":"A quick repository creation utility (part IV)"},"content":{"rendered":"\n

This is part IV of the article, see here for Part I here<\/a>, Part II here<\/a>, and Part III here<\/a>.<\/p>\n\n\n\n

Instantiating a new docbase<\/h3>\n\n\n\n

See dbi services<\/a> for the script’s source.<\/p>\n\n\n\n

Roughly speaking, the instantiation process uncompresses a seed, or model, docbase from its compressed tar ball and runs the Migration Utility against it to adapt its name and id. Those actions are performed by the script instantiate_docbase.sh<\/code>. Its usage is:<\/p>\n\n\n

\n$ instantiate_docbase.sh seed_repo new_repo\n<\/pre><\/div>\n\n\n
where seed_repo<\/em> and new_repo<\/em> are stems pointing to the settings of the respective docbase defined in the global_parameters<\/code> file. The easy-to-remember syntax is the same as that of the cp<\/code> command, for a similar concept.<\/p>\n\n\n\n
Instantiation is always performed locally so if it needs to be done on a machine different from the one where the model docbase was created, the new machine needs first to be enrolled (see script pre-requisites.sh<\/code>) and the script invoked from there.<\/p>\n\n\n\n
The procedure looks in the ${DOCUMENTUM}<\/code> tree and changes all the references to the seed docbase so they match the target docbase, including the settings such as port numbers, docbroker names, host names, http server memory sizing, sym links, etc, and then tackles the metadata\u2019s big part. For this to be successful, make sure the seed docbase\u2019s definition in the global_parameter<\/code> file has not changed and still matches its installation in the compressed tar ball.<\/p>\n\n\n\n
For an Oracle RDBMS, a new schema is created with the required tablespaces and grants, the dump file that was produced by the exp<\/code> utility is copied onto the database\u2019s host, slightly edited (references to the seed\u2019s schema are changed to the new instance\u2019s schema) and imported into the new schema. Next, as the creation of some views\u2019 fails with \u201cIMP-00008: unrecognized statement in the export file:<\/code>\u201d errors, the view creation statements (almost 1000 of them) are extracted from the dump file (although this is a binary file, the SQL statements it contains can be extracted as text and edited), corrected and re-executed separately, as shown here:<\/p>\n\n\n
\n# extract the view creation statements from export file and create the views;\nstrings ${seed_dmp_file} | sed -r 's\/^[ \\t\\$]*(\\(*)(SELECT)(.+)\/\\1\\2\\3\/I' | gawk '{if (sub(\/^CREATE VIEW \/, "CREATE or REPLACE VIEW ", \\$0)) {print; getline; pos = 2000; if (length(\\$0) > pos) {while (substr(\\$0, --pos, 1) != ","); print substr(\\$0, 1, pos); print substr(\\$0, pos + 1)}else print; print "\/"}}END{print "quit"}' >> create_views.sql\n\n# count the views to be created;\ngrep "CREATE or REPLACE VIEW" create_views.sql | wc -l\n# 977 in CS 22.2;\n# 983 in CS 22.4;\n\n# create the views;\nsqlplus ${ACTIVE_DATABASE_OWNER}\/${ACTIVE_DATABASE_PASSWORD}@${db_remote_connect_string} @create_views.sql\n<\/pre><\/div>\n\n\n
The view text needs to be reformatted to work around the weird legacy format of the dump file.<\/p>\n\n\n\n
For good measure, all the new schema\u2019s compilable objects are recompiled and the schema\u2019s statistics computed. A verification of the schema’s object validity is performed:<\/p>\n\n\n
\nset echo on\nset pagesize 10000\nset linesize 100\ncol object_name format a30\ncol object_type format a30\nselect object_name, object_type from user_objects where status = 'INVALID';\nselect object_name, object_type, status from user_objects;\nquit\neoq\n# output:\nno rows selected\nOBJECT_NAME                    OBJECT_TYPE                    STATUS\n------------------------------ ------------------------------ -------\nD_1F00271080000195             INDEX                          VALID\nD_1F00271080000241             INDEX                          VALID\nDMI_CHANGE_RECORD_S            TABLE                          VALID\nD_1F002710800001A8             INDEX                          VALID\nDMI_DD_ATTR_INFO_R             TABLE                          VALID\n...\nDM_INDEXES                     VIEW                           VALID\nDM_RESYNC_DD_ATTR_INFO         VIEW                           VALID\nDM_RESYNC_DD_TYPE_INFO         VIEW                           VALID\nDMI_DD_ATTR_INFO_DDEN          VIEW                           VALID\n\n1931 rows selected.\n<\/pre><\/div>\n\n\n
For a PostgreSQL RDBMS, things are much less complicated. The server port is adapted and the server started. Next, the names of the schema, database and user are adapted and the ownership of each of the schema\u2019s objects is displayed as a verification. Example of output:<\/p>\n\n\n
\n  datname  \n-----------\n postgres\n template1\n template0\n repo02\n(4 rows)\n\n                                   List of roles\n Role name |                         Attributes                         | Member of \n-----------+------------------------------------------------------------+-----------\n dmadmin   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}\n repo02    | No inheritance                                             | {}\n\nYou are now connected to database "repo02" as user "dmadmin".\nALTER SCHEMA\nYou are now connected to database "postgres" as user "dmadmin".\nALTER DATABASE\nNOTICE:  MD5 password cleared because of role rename\nALTER ROLE\nALTER ROLE\nALTER ROLE\n\nYou are now connected to database "repo03" as user "repo03".\n                                   List of roles\n Role name |                         Attributes                         | Member of \n-----------+------------------------------------------------------------+-----------\n dmadmin   | Superuser, Create role, Create DB, Replication, Bypass RLS | {}\n repo03    | No inheritance                                             | {}\n\n current_user \n--------------\n repo03\n(1 row)\n\n                                                version                                                \n-------------------------------------------------------------------------------------------------------\n PostgreSQL 15.1 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit\n(1 row)\n\n  datname  \n-----------\n postgres\n template1\n template0\n repo03\n(4 rows)\n\n schemaname |              objectname               | len | objectowner | objecttype \n------------+---------------------------------------+-----+-------------+------------\n repo03     | adm_turbo_size                        |  14 | repo03      | TABLE\n repo03     | creation_date_message_id_ui           |  27 | repo03      | INDEX\n repo03     | d_1f00271280000001                    |  18 | repo03      | INDEX\n repo03     | d_1f00271280000002                    |  18 | repo03      | INDEX\n...\n repo03     | dm_federation_log                     |  17 | repo03      | TABLE\n repo03     | dm_federation_r                       |  15 | repo03      | TABLE\n repo03     | dm_federation_rp                      |  16 | repo03      | VIEW\n repo03     | dm_federation_rv                      |  16 | repo03      | VIEW\n repo03     | dm_federation_s                       |  15 | repo03      | TABLE\n...\n repo03     | status_no_idx                         |  13 | repo03      | INDEX\n(1932 rows)\n<\/pre><\/div>\n\n\n
The above output shows the repo03<\/code> account and its roles, a database named repo03<\/code> and docbase’s objects owned by user repo03<\/code>. Also, repo03<\/code> was instantiated from repo02<\/code> as the seed.<\/p>\n\n\n\n
As for Oracle, the connectivity is also adapted here but instead of the SQL*Net\u2019s tnsnames.ora<\/code>, the installation owner\u2019s ~\/.odbc.ini<\/code> file is adapted to match the target docbase\u2019s settings and the reference to it in the server.ini<\/code> file is adapted too. To accommodate instantiation concurrency,  the file ~\/.odbc.ini<\/code> must be protected against simultaneous editions by a mutex. We apply the simple trick based on the atomicity of the mkdir<\/code> command:<\/p>\n\n\n
\n   bodbc_done=0\n   mutex=\/tmp\/odbc_mutex_in_instantiate_docbase\n   mutext_iterations=10\n   while [[ ${mutext_iterations} -gt 0 ]]; do\n      if mkdir ${mutex}; then\n         # edit ~\/.odbc.ini\n...\n         rmdir ${mutex}\n         bodbc_done=1\n         break\n      else\n         sleep 1\n         ((mutext_iterations--))\n      fi\n   done\n   if [[ ${bodbc_done} -ne 1 ]]; then\n      echo "could not get a mutex on ~\/.odbc.ini, aborting current instance of $0 ..."\n      exit 1\n   fi\n<\/pre><\/div>\n\n\n
So, the script attempts to get the mutex for up to 20s, which should be enough as the editions are very fast. That concludes the RDBMS part.<\/p>\n\n\n\n
The new instance\u2019s password is also saved using the documentum\u2019s dm_encrypt_password<\/code> utility, and tested using dmdbtest<\/code>. At this point, the Migration Utility step can be started.<\/p>\n\n\n\n
The Migration Utility<\/h4>\n\n\n\n
The Migration Utility is located in ${DOCUMENTUM}\/product\/22.2\/install\/external_apps\/MigrationUtil<\/code>. This tool takes its instructions from the xml configuration file ${DM_HOME}\/install\/external_apps\/MigrationUtil\/config.xml<\/code>. The following relevant attributes\u2019 values are edited to match the target docbase\u2019s settings:<\/p>\n\n\n\n