A few months ago, I tripped over a very mysterious problem while attempting to connect to a 7.3 CS docbase from within dqMan.
We had 3 docbases and we could connect using this client to all of them but one ! Moreover, we could connect to all three using a remote Documentum Administrator or the local idql/iapi command-line tools. Since we could connect to at least one of them with dqMan, this utility was not guilty. Also, since all three docbases accepted connections, they were all OK in this respect. Ditto for the account used, dmadmin or nominative ones; local connections were possible hence the accounts were all active and, as they could be used from within the remote DA, their identification method and password were correct too.
We tried connecting from different workstations in order to check the dqMan side, we cleared its caches, we reinstalled it, but to no avail. We checked the content server’s log, as usual nothing relevant. It was just the combination of this particular docbase AND dqMan. How strange !
So what the heck was wrong here ?
As we weren’t the only administrators of those repositories, we more or less suspecting someone else change some setting but which one ? Ok, I sort of gave it away in the title but please bear with me and read on.
I don’t remember exactly how, we were probably working in panic mode, but we eventually decided to compare the docbases’ dm_docbase_config object side by side as shown below (with some obfuscation):
paste <(iapi bad_boy -Udmadmin -Pxxx <<eoq | awk '{print substr($0, 1, 80)}'
retrieve,c,dm_docbase_config
dump,c,l
quit
eoq
) <(iapi good_boy -Udmadmin -Pxxx <<eoq | awk '{print substr($0, 1, 80)}'
retrieve,c,dm_docbase_config
dump,c,l
quit
eoq
) | column -c 30 -s $'t' -t | tail +11 | head -n 48
USER ATTRIBUTES USER ATTRIBUTES
object_name : bad_boy object_name : good_boy
title : bad_boy Repository title : good_boy Global Repository
subject : subject :
authors []: authors []:
keywords []: keywords []:
resolution_label : resolution_label :
owner_name : bad_boy owner_name : good_boy
owner_permit : 7 owner_permit : 7
group_name : docu group_name : docu
group_permit : 5 group_permit : 5
world_permit : 3 world_permit : 3
log_entry : log_entry :
acl_domain : bad_boy acl_domain : good_boy
acl_name : dm_450xxxxx80000100 acl_name : dm_450xxxxx580000100
language_code : language_code :
mac_access_protocol : nt mac_access_protocol : nt
security_mode : acl security_mode : acl
auth_protocol : auth_protocol :
index_store : DM_bad_boy_INDEX index_store : DM_good_boy_INDEX
folder_security : T folder_security : T
effective_date : nulldate effective_date : nulldate
richmedia_enabled : T richmedia_enabled : T
dd_locales [0]: en dd_locales [0]: en
default_app_permit : 3 default_app_permit : 3
oldest_client_version : oldest_client_version :
max_auth_attempt : 0 max_auth_attempt : 0
client_pcaching_disabled : F client_pcaching_disabled : F
client_pcaching_change : 1 client_pcaching_change : 1
fulltext_install_locs [0]: dsearch fulltext_install_locs [0]: dsearch
offline_sync_level : 0 offline_sync_level : 0
offline_checkin_flag : 0 offline_checkin_flag : 0
wf_package_control_enabled : F wf_package_control_enabled : F
macl_security_disabled : F macl_security_disabled : F
trust_by_default : T trust_by_default : T
trusted_docbases []: trusted_docbases []:
login_ticket_cutoff : nulldate login_ticket_cutoff : nulldate
auth_failure_interval : 0 auth_failure_interval : 0
auth_deactivation_interval : 0 auth_deactivation_interval : 0
dir_user_sync_on_demand : F dir_user_sync_on_demand : F
check_client_version : T check_client_version : F
audit_old_values : T audit_old_values : T
docbase_roles []: docbase_roles [0]: Global Registry
approved_clients_only : F approved_clients_only : F
minimum_owner_permit : 2 minimum_owner_permit : 0
minimum_owner_xpermit : minimum_owner_xpermit :
dormancy_status : dormancy_status :
The only significant differences were the highlighted ones and the most obvious one was the attribute check_client_version, it was turned on in the bad_boy repository. Now that we finally had something to blame, the universe started making sense again ! We quickly turned this setting to false and could eventually connect to that recalcitrant docbase. But the question is still open: check against what ? What criteria was applied to refuse dqman access to bad_boy but to allow it to good_boy ? That was still not clear, even though we could work around it.
Now, who and why turned it on, that had to remain a mystery.
While we were at it, we also noticed another attribute which seemed to be related to the previous one: oldest_client_version.
Was there any other client_% attribute in dm_docbase_config ?
paste <(iapi good_boy -Udmadmin -Pdmadmin <<eoq | grep client retrieve,c,dm_docbase_config dump,c,l quit eoq) <(iapi bad_boy -Udmadmin -Pdmadmin <<eoq | grep client retrieve,c,dm_docbase_config dump,c,l quit eoq) | column -s $'t' -t oldest_client_version : oldest_client_version : client_pcaching_disabled : F client_pcaching_disabled : F client_pcaching_change : 1 client_pcaching_change : 1 check_client_version : F check_client_version : T approved_clients_only : F approved_clients_only : F
Yes, but they looked quite harmless in the current context.
Thus, the relevant attributes here are check_client_version and oldest_client_version. Let’s discover a bit more about them.
Digging
As usual, the documentation is a bit scketchy about these attributes:
check_client_version Boolean S T means that the repository
servers will not accept connections
from clients older than the
version level specified in the
oldest_client_version property.
F means that the servers accept
connections from any client version.
The default is F.
oldest_client _version string(32) S Version number of the oldest
Documentum client that will access
this repository.
This must be set manually. It is used
by the DFC to determine how to
store chunked XML documents. If
check_client_version is set to T,then
this value is also used to identify the
oldest client version level that may
connect to the repository.
But what is the client version ? Logically, it is the version of its DfCs or, for older clients, the version of the dmcl shared library.
So, if check_client_version is true, the client version is checked and if it is older than the one defined in oldest_client_version, the client is forbidden to connect. That makes sense except that in our case, oldest_client_version was empty. Maybe in such a case, the client has to match exactly the content server’s DfC version ? As dqMan was either using the dmcl40.dll library or an old Dfc version, it was rejected. Let’s verify these hypothesis with a 16.4 target repository.
Connecting from an ancient 5.3 client
We exhumed an old 5.3 CS installation to use its client part with the default configuration in the target docbase:
[email protected]:~/documentum53$ idql dmtest -Udmadmin -Pdmadmin
Documentum idql - Interactive document query interface
(c) Copyright Documentum, Inc., 1992 - 2004
All rights reserved.
Client Library Release 5.3.0.115 Linux
Connecting to Server using docbase dmtest
[DM_SESSION_I_SESSION_START]info: "Session 0100c35080003913 started for user dmadmin."
Connected to Documentum Server running Release 16.4.0080.0129 Linux64.Oracle
Fine so far.
Let’s activate the dm_docbase_config.check_client_version in the target:
retrieve,c,dm_docbase_config
...
set,c,l,check_client_version
SET> T
...
OK
API> save,c,l
...
[DM_DCNFG_E_CANT_SAVE]error: "Cannot save dmtest docbase_config."
[DM_DCNFG_E_SET_OLDEST_CLIENT_VERSION_FIRST]error: "The docbase_config object attribute oldest_client_version has to be set before setting attribute check_client_version to T."
Interesting. At that time, this attribute was empty and yet the check_client_version was active. Is this constraint new in 16.4 or did the unknow administrator hack around this ? As I don’t have a 7.x repository available right now, I cannot test this point.
Let’s play by the rules and set oldest_client_version:
reset,c,l
set,c,l,oldest_client_version
16.4
save,c,l
OK
set,c,l,check_client_version
SET> T
...
OK
API> save,c,l
...
OK
Try connecting from the 5.3 client: still OK.
Maybe a reinit is necessary to actuate the changes:
reinit,c
Try again:
[email protected]:~/documentum53$ idql dmtest -Udmadmin -Pdmadmin
&nbps;
&nbps;
Documentum idql - Interactive document query interface
(c) Copyright Documentum, Inc., 1992 - 2004
All rights reserved.
Client Library Release 5.3.0.115 Linux
Connecting to Server using docbase dmtest
Could not connect
[DM_SESSION_E_START_FAIL]error: "Server did not start session. Please see your system administrator or check the server log.
Error message from server was:
[DM_SESSION_E_AUTH_FAIL]error: "Authentication failed for user dmadmin with docbase dmtest."
"
So a reinit it required indeed.
Note the misleading error, it is not the authentication that is wrong but the client version validation. It is such wrong messages that make diagnosis of Documentum problems so hard and time-consuming. Anyway, let’s revert the check_client_version to F:
set,c,l,check_client_version
F
save,c,l
reinit,c
Try connecting: OK. So, the client version filtering is effective. Let’s try it with a 5.3 client version:
API> set,c,l,oldest_client_version
SET> 5.3
...
OK
API> save,c,l
...
OK
API> set,c,l,check_client_version
SET> T
...
OK
API> save,c,l
...
OK
API> reinit,c
...
OK
Try connecting: OK, that’s expected.
Let’s try it for a minimum 5.2 client version: it still works, which is expected too since the test client’s version is 5.3 and in my books 5.3 > 5.2.
Let’s try it for a miminum a 5.4 client version: the connection fails, so client version checking works as expected here.
Let’s try it for a miminum a 20.0 client version: the connection fails as expected. No check on the version’s value is done, which is quite understandable programmatically speaking, although a bit optimistic in the context of the turmoil Documentum went through lately.
Let’s go back to a more realistic value:
API> set,c,l,oldest_client_version
SET> 7.2
...
[DM_SESSION_E_AUTH_FAIL]error: "Authentication failed for user dmadmin with docbase dmtest."
API> save,c,l
Oops, interestingly, the last change did not make it because with the current setting so down the way into the future, the present client’s session was disconnected and there is no way to reconnect in order to revert it !
Let’s do the rollback from the database level directly:
sqlplus [email protected]
SQL*Plus: Release 12.2.0.1.0 Production on Mon Jun 10 16:25:56 2019
Copyright (c) 1982, 2016, Oracle. All rights reserved.
Enter password:
Last Successful login time: Mon Jun 10 2019 16:25:40 +02:00
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
SQL> update dm_docbase_config_s set check_client_version = 0;
1 row updated.
SQL> commit;
Commit complete.
quit;
Try to connect:
iapi [email protected]:1489
Please enter a user (dmadmin):
Please enter password for dmadmin:
OpenText Documentum iapi - Interactive API interface
Copyright (c) 2018. OpenText Corporation
All rights reserved.
Client Library Release 16.4.0070.0035
Connecting to Server using docbase dmtest
[DM_SESSION_E_AUTH_FAIL]error: "Authentication failed for user dmadmin with docbase dmtest."
Still not ok because the reinit is missing but for this we need to connect which we still cannot because of the missing reinit. To break this catch-22 situation, let’s cut the gordian knot and kill the dmtest docbase’s processes:
[email protected]:~$ ps ajxf | grep dmtest
1 27843 27843 27843 ? -1 Ss 1001 0:00 ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27849 27843 27843 ? -1 S 1001 0:00 _ /app/dctm/product/16.4/bin/mthdsvr master 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 27843 dmtest /app/dctm/dba/log
27849 27850 27843 27843 ? -1 Sl 1001 0:03 | _ /app/dctm/product/16.4/bin/mthdsvr worker 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 0 dmtest /app/dctm/dba/log
27849 27861 27843 27843 ? -1 Sl 1001 0:03 | _ /app/dctm/product/16.4/bin/mthdsvr worker 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 1 dmtest /app/dctm/dba/log
27849 27874 27843 27843 ? -1 Sl 1001 0:03 | _ /app/dctm/product/16.4/bin/mthdsvr worker 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 2 dmtest /app/dctm/dba/log
27849 27886 27843 27843 ? -1 Sl 1001 0:03 | _ /app/dctm/product/16.4/bin/mthdsvr worker 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 3 dmtest /app/dctm/dba/log
27849 27899 27843 27843 ? -1 Sl 1001 0:03 | _ /app/dctm/product/16.4/bin/mthdsvr worker 0xe901fd2f, 0x7f8a50658000, 0x223000 50000 5 4 dmtest /app/dctm/dba/log
27843 27862 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27863 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27875 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27887 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27901 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
27843 27944 27843 27843 ? -1 Sl 1001 0:06 _ ./dm_agent_exec -docbase_name dmtest.dmtest -docbase_owner dmadmin -sleep_duration 0
27843 27962 27843 27843 ? -1 S 1001 0:00 _ ./documentum -docbase_name dmtest -security acl -init_file /app/dctm/dba/config/dmtest/server.ini
and:
kill -9 -27843
After restarting the docbase, the connectivity was restored.
So, be cautious while experimenting ! Needless to say, avoid doing it in a production docbase or in any heavily used development docbase for that matter, or the wrath of the multiverses and beyond will fall upon you and you will be miserable for ever.
Connecting from a 7.3 client
The same behavior and error messages as with the precedent 5.3 client were observed with a more recent 7.3 client and, inferring from the incident above, later clients behave the same way.
Conclusion
We never stop learning stuff with Documentum ! While this client version limiting feature looks quite exotic, it may make sense in order to avoid surprises or even corruptions when using newly implemented extensions or existing but changed areas of the content server. It is possible that new versions of the DfCs behave differently from older ones in dealing with the same functionalities and Documentum had no better choice but to cut the older versions off to prevent any conflict. As usual, the implementation looks a bit hasty with inapt error messages costing hours of investigation and the risk to cut oneself off a repository.
![Thumbnail [60x60]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/CEC_web-min-scaled.jpg)
![Thumbnail [90x90]](https://www.dbi-services.com/blog/wp-content/uploads/2022/08/GME_web-min-scaled.jpg)