You have finally completed that migration and need to restart all the Documentum processes. So, you shut down the docbroker and move on to the repositories but then you receive an error message about them not being reachable any more. Or conversely, you want to start all the Documentum processes and you start first the repositories and later the docbrokers. Next, you want to connect to one repository and you receive the same error message. Of course, you finally remember, since the docbroker is a requirement for the repositories, it must be started first and shut down last but it is too late now. How to get out if this annoying situation ? You could just (re)start the docbroker and wait for the next repostories’ checkpoint, at most 5 minutes by default. If this is not acceptable, at first sight, there is no other solution than to “kill -9” the repositories’ processes, start the docbroker and only next the repositories. Let’s see if we can find a better way. Spoiler alert: to stop this insufferable suspens, I must say up front that there is no other way, sorry, but there are a few ways to alleviate this inconvenience.<\/p>\n
Let’s first clarify a point of terminology here: there is a difference between docbases\/repositories and content servers. A docbase encompasses the actual content and their persistent data and technical information whereas the content server is the set of running processes that give access to and manage one docbase. It is very similar to Oracle’s databases and instances, where one database can be served by several instances, providing parallelism and high availability. A docbase can be served by more than one content server, generally spread over different machines, with its own set of dm_start_docbase<\/em> and dm_shutdown_docbase<\/em> scripts and server.ini. A docbase knows how many content servers use it because they each have their own dm_server_config object. If there is just one content server, both docbase and content server can be used interchangeably but when there are several content servers for the same docbase, when one says “stopping the docbase” it really means “stopping one particular content server”, and this is the meaning that will be used in the rest of the article. If the docbase has more than one content servers, just extend the presented manipulations to each of them.<\/p>\n If one could connect to a repository without a running docbroker, the situation that triggered this article, things would be much easier. In the ancient, simpler times, the dmcl.ini parameters below could help working around an unavailable docbroker:<\/p>\n and they used to work. Unfortunately, they don’t work any more. Actually, although they are still documented in the dfcfull.properties, they have not been implemented and will never be according to OTX. Moreover, they will be removed in the future. Too bad, that would have been such a cheap way to extricate oneself from an uncomfortable situation.<\/p>\n The best solution is obviously to prevent it to happen. This can be easily realized by using a central script for stopping and starting the Documentum stack. And, while we are at it, inquiring its status. See lines 11 to 29 for its usage.Connecting to the repositories without a docbroker<\/h2>\n
\n[DOCBROKER_DEBUG]\ndocbase_id = <id of docbase as specified in its server.ini file>\nhost = <host's name the docbase server is running on>\nport = <docbase's port as specified in \/etc\/services>\nservice = <docbase's service name as specified in \/etc\/services>\n<\/pre>\n
\nAfter the switch to the dfc.properties file, those parameters were renamed as follows:<\/p>\n\ndfc.docbroker.debug.docbase_id=<id of docbase as specified in its server.ini file>\ndfc.docbroker.debug.host=<host's name the docbase server is running on>\ndfc.docbroker.debug.port=<docbase's port as specified in \/etc\/services>\ndfc.docbroker.debug.service=<docbase's service name as specified in \/etc\/services>\n<\/pre>\n
Preventing the situation<\/h2>\n
\nDocumentum already provides such a script, e.g. see here Linux scripts for automatic startup and shutdown of Documentum Content Server<\/a>. Here is another more sophisticated implementation:<\/p>\n\n#!\/bin\/bash\n#\n# See Usage() function below for explanations; \n# cec - dbi-services - April 2019\n#\n\ngeneral_status=0\n\nUsage() {\n cat <<EoU\nUsage:\n start-stop.sh [(help) | start | stop | status] [(all) | docbases | docbrokers | docbase={,} | docbroker={,} | method_server]\n E.g.:\n display this help screen:\n start-stop.sh\n start all:\n start-stop.sh start [all]\n stop all:\n start-stop.sh stop [all]\n status all:\n start-stop.sh status [all]\n start docbroker01:\n start-stop.sh start docbroker=docbroker01\n start docbases global_registry and dmtest01:\n start-stop.sh docbase=global_registry,dmtest01\n start all the docbases:\n start-stop.sh docbases\n start all the docbrokers:\n start-stop.sh docbrokers\nEoU\n}\n\nstart_docbroker() {\n docbroker=$1\n echo \"starting up docbroker $docbroker ...\"\n .\/dm_launch_${docbroker}\n}\n\nstart_all_docbrokers() {\n echo \"starting the docbrokers ...\"\n DOCBROKERS=`ls -1 dm_launch_* 2>\/dev\/null | cut -f3 -d_`\n nb_items=0\n for docbroker in $DOCBROKERS; do\n start_docbroker $docbroker\n (( nb_items++ ))\n done\n echo \"$nb_items docbrokers started\"\n\n}\n\nstart_docbase() {\n docbase=$1\n echo \"starting $docbase\"\n .\/dm_start_${docbase}\n}\n\nstart_all_docbases() {\n echo \"starting the repositories ...\"\n DOCBASES=`ls -1 config 2>\/dev\/null `\n nb_items=0\n for docbase in $DOCBASES; do\n start_docbase $docbase\n (( nb_items++ ))\n done\n echo \"$nb_items repositories started\"\n}\n\nstart_method_server() {\n echo \"starting the method server ...\"\n cd ${DOCUMENTUM}\/${JBOSS}\/server\n nohup ${DOCUMENTUM}\/${JBOSS}\/server\/startMethodServer.sh 2>&1 > \/tmp\/nohup.log &\n echo \"method server started\"\n}\n\nstart_all() {\n echo \"starting all the documentum processes ...\"\n start_all_docbrokers\n start_all_docbases\n start_method_server\n}\n\nstatus_docbroker() {\n docbroker_name=$1\n docbroker_host=$(grep \"^host=\" \/app\/dctm\/dba\/dm_launch_${docbroker_name} | cut -d= -f2)\n docbroker_port=$(grep \"dmdocbroker -port \" \/app\/dctm\/dba\/dm_launch_${docbroker_name} | cut -d -f3)\n dmqdocbroker -t $docbroker_host -p $docbroker_port -c ping 2> \/dev\/null 1> \/dev\/null\n local_status=$?\n if [ $local_status -eq 0 ]; then\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): successfully pinged docbroker $docbroker_name listening on port $docbroker_port on host $docbroker_host\"\n else\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): docbroker $docbroker_name listening on port $docbroker_port on host $docbroker_host is unhealthy\"\n general_status=1\n fi\n echo \"status for docbroker $docbroker_name:$docbroker_port: $local_status, i.e. $(if [[ $local_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstatus_all_docbrokers() {\n DOCBROKERS=`ls -1 dm_launch_* 2>\/dev\/null | cut -f3 -d_`\n DOCBROKERS_PORTS=`grep -h \".\/dmdocbroker\" dm_launch_* | cut -f3 -d `\n for f in `ls -1 dm_launch_* 2>\/dev\/null `; do\n docbroker_name=`echo $f | cut -f3 -d_`\n docbroker_port=`grep \".\/dmdocbroker\" $f | cut -f3 -d `\n status_docbroker $docbroker_name $docbroker_port\n done\n echo \"general status for all docbrokers: $general_status, i.e. $(if [[ $general_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstatus_docbase() {\n docbase=$1\n timeout --preserve-status 30s idql $docbase -Udmadmin -Pxx 2> \/dev\/null 1> \/dev\/null <<eoq\n quit\neoq\n local_status=$?\n if [[ $local_status -eq 0 ]]; then\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): successful connection to repository $docbase\"\n else\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): repository $docbase is unhealthy\"\n general_status=1\n fi\n echo \"status for docbase $docbase: $local_status, i.e. $(if [[ $local_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstatus_all_docbases() {\n DOCBASES=`ls -1 config 2>\/dev\/null `\n for docbase in $DOCBASES; do\n status_docbase $docbase\n done\n echo \"general status for all docbases: $general_status, i.e. $(if [[ $general_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstatus_method_server() {\n # check the method server;\n curl --silent --fail -k http:\/\/${HOSTNAME}:9080\/DmMethods\/servlet\/DoMethod 2>&1 > \/dev\/null\n local_status=$?\n if [ $local_status -eq 0 ]; then\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): method server successfully contacted\"\n else\n echo \"$(date +\"%Y\/%m\/%d %H:%M:%S\"): method server is unhealthy\"\n general_status=1\n fi\n echo \"status for method_server: $local_status, i.e. $(if [[ $local_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstatus_all() {\n status_all_docbrokers\n status_all_docbases\n status_method_server\n echo \"General status: $general_status, i.e. $(if [[ $general_status -eq 0 ]]; then echo OK; else echo NOK;fi)\"\n}\n\nstop_docbase() {\n echo \"stopping $docbase\"\n docbase=$1\n .\/dm_shutdown_${docbase}\n echo \"docbase $docbase stopped\"\n}\n\nstop_all_docbases() {\n echo \"stopping the repositories ...\"\n DOCBASES=`ls -1 config 2>\/dev\/null `\n nb_items=0\n for docbase in $DOCBASES; do\n stop_docbase $docbase\n (( nb_items++ ))\n done\n echo \"$nb_items repositories stopped\"\n}\n\nstop_docbroker() {\n echo \"stopping docbroker $docbroker ...\"\n docbroker=$1\n .\/dm_stop_${docbroker}\n echo \"docbroker $docbroker stopped\"\n}\n\nstop_all_docbrokers() {\n echo \"stopping the docbrokers ...\"\n DOCBROKERS=`ls -1 dm_stop_* 2>\/dev\/null | cut -f3 -d_`\n nb_items=0\n for docbroker in $DOCBROKERS; do\n stop_docbroker $docbroker\n (( nb_items++ ))\n done\n echo \"$nb_items docbrokers stopped\"\n}\n\nstop_method_server() {\n echo \"stopping the method server ...\"\n ${DOCUMENTUM}\/${JBOSS}\/server\/stopMethodServer.sh\n echo \"method server stopped\"\n}\n\nstop_all() {\n echo \"stopping all the documentum processes ...\"\n stop_all_docbases\n stop_method_server\n stop_all_docbrokers\n echo \"all documentum processes stopped\"\n ps -ajxf | egrep '(PPID|doc|java)' | grep -v grep | sort -n -k2,2\n}\n\n# -----------\n# main;\n# -----------\n [[ -f ${DM_HOME}\/bin\/dm_set_server_env.sh ]] && . ${DM_HOME}\/bin\/dm_set_server_env.sh\n cd ${DOCUMENTUM}\/dba\n if [[ $# -eq 0 ]]; then\n Usage\n exit 0\n else\n while [[ $# -ge 1 ]]; do\n case $1 in\n\t help)\n\t Usage\n\t exit 0\n\t ;;\n start|stop|status)\n\t cmd=$1\n\t shift\n\t if [[ -z $1 || $1 = \"all\" ]]; then\n\t ${cmd}_all\n\t elif [[ $1 = \"docbases\" ]]; then\n\t ${cmd}_all_docbases\n\t elif [[ $1 = \"docbrokers\" ]]; then\n\t ${cmd}_all_docbrokers\n\t elif [[ ${1%%=*} = \"docbase\" ]]; then\n\t docbases=`echo ${1##*=} | gawk '{gsub(\/,\/, \" \"); print}'`\n for docbase in $docbases; do\n\t ${cmd}_docbase $docbase\n\t done\n\t elif [[ ${1%%=*} = \"docbroker\" ]]; then\n\t docbrokers=`echo ${1##*=} | gawk '{gsub(\/,\/, \" \"); print}'`\n for docbroker in $docbrokers; do\n\t ${cmd}_docbroker $docbroker\n\t done\n\t elif [[ $1 = \"method_server\" ]]; then\n ${cmd}_method_server\n fi\n exit $general_status\n ;;\n *)\n echo \"syntax error\"\n\t Usage\n\t exit 1\n\t ;;\n esac\n shift\n done\n fi\n<\/pre>\n
\nNote on line 110 the timeout command when attempting to connect to a docbase to check its status; see the article Adding a timeout in monitoring probes<\/a> for an explanation.
\nWe couldn’t help but adding the option to address each component individually, or a few of them, in addition to all of them at once. So, the script lets us stop, start and inquire the status of one particular docbroker or docbase or method server, or a list of docbrokers or a list of docbases, or everything at once.
\nAfter a maintenance task, to stop all the Documentum processes, the command below could be used:
\n
\n$ start-stop.sh stop all
\n<\/code>
\nSimilarly, to start everything:
\n
\n$ start-stop.sh start all
\n<\/code>
\nThus, the proper order is guaranteed to be used and human error is prevented. By standardizing on such script and using it as shown, the aforementioned problem won’t occur anymore.<\/p>\n