If you are familiar with Documentum or if you already read some of my blogs, then you probably already know that EMC has sometimes issues with libraries. In a previous blog (this one), I talked about the Lockbox versions which caused us an issue and in this blog, I will talk about DFC versions.
Whenever you install a CS patch or another patch, it will probably have its own DFC libraries simply because EMC fixed something in it or because it was needed. Whenever you install D2, it will also have its own DFC libraries in the JMS and the WAR files. The problem is that the DFC libraries are everywhere… Each and every DFC client has its own DFC libraries which come when you install it, patch it, aso… Basically that’s not a wrong approach, it ensure that the components will work wherever they are installed so it can always talk to the Content Server.
The problem here is that the DFC libraries are changing at every patch almost and therefore it is kind of complicated to keep a clean environment. It already happened to us that two different patches (CS and D2 for example), released on the exact same day, were using different DFC versions and you will see below another example coming from the same package… You can live with a server having five different DFC versions but this also means that whenever a bug impact one of your DFC library, it will be hard to fix that because you then need to deploy the next official patch which is always a pain. It also multiplies the number of issues that impact your DFC versions since you are running several versions at the same time.
I’m not saying that you absolutely need to always use only the latest DFC version but if you can properly and quickly perform the appropriate testing, I believe it can brings you something. A few weeks ago for example, one of the Application Teams we are supporting had an issue with some search functionalities in D2. This was actually caused by the DFC version bundled with D2 (DFC 7.2P03 I think) and we solved this issue by simply using the DFC version coming from our Content Server (DFC 7.2P05) which was only two patch above.
To quickly and efficiently see which versions of the DFC libraries you are using and where, you can use:
find <WHERE_TO_FIND> -type f -name dfc.jar -print0 | while read -d $'' file; do echo "DFC: `$JAVA_HOME/bin/java -cp "$file" DfShowVersion` === Size: `ls -ks "$file" | awk '{print $1}'` kb === Location: $file"; done or find <WHERE_TO_FIND> -type f -name dfc.jar -print0 | while IFS= read -r -d '' file; do echo "DFC: `$JAVA_HOME/bin/java -cp "$file" DfShowVersion` === Size: `ls -ks "$file" | awk '{print $1}'` kb === Location: $file"; done
You can execute these commands on a Content Server, Application Server (Note: dfc.jar files might be on the D2/DA war files if you aren’t using exploded deployments), Full Text Server or any other Linux Servers for what it matters. These commands handle the spaces in the paths even if normally you shouldn’t have any for the dfc files. To use them, you can just replace <WHERE_TO_FIND> with the base folder of your installation. This can be $DOCUMENTUM for a Content Server, $XPLORE_HOME for a Full Text Server, aso… Of course you still need to have the proper permissions to see the files otherwise it will be quite useless to execute this command.
A small example on a Content Server 7.3 (no patches are available yet) including xCP 2.3 P05 (End of February 2017 patch which is supposed to be for CS 7.3):
[dmadmin@content_server_01 ~]$ find $DOCUMENTUM -type f -name dfc.jar -print0 | while read -d $'' file; do echo "DFC: `$JAVA_HOME/bin/java -cp "$file" DfShowVersion` === Size: `ls -ks "$file" | awk '{print $1}'` kb === Location: $file"; done DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/product/7.3/install/composer/ComposerHeadless/plugins/com.emc.ide.external.dfc_1.0.0/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/temp/installer/wildfly/dctmutils/templates/dfc/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/dfc/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/modules/system/layers/base/emc/documentum/security/main/dfc.jar DFC: 7.2.0210.0184 === Size: 15212 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/deployments/bpm.ear/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/deployments/ServerApps.ear/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/deployments/acs.ear/lib/dfc.jar DFC: 7.2.0210.0184 === Size: 15212 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/tmp/vfs/deployment/deploymente7c710bab402b3f7/dfc.jar-7ac143a725d0471/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/tmp/vfs/deployment/deploymente7c710bab402b3f7/dfc.jar-bc760ece35b05a08/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $DOCUMENTUM/shared/wildfly9.0.1/server/DctmServer_MethodServer/tmp/vfs/deployment/deploymente7c710bab402b3f7/dfc.jar-35c79cfe4b79d974/dfc.jar
As you can see above, it looks like there are two different versions of the DFC library on this Content Server which has just been installed: one coming from the CS 7.3 which is therefore in 7.3 P00 build number 205 and another version which is still in 7.2 P21 build number 184. This second version has been put on the Content Server by the xCP 2.3 P05 installer. Therefore using a 7.2 library on a 7.3 Content Server is a little bit ugly but the good news is that they are both in a pretty recent version since these two libraries were released almost at the same time (end of 2016/beginning of 2017). Therefore here I don’t think it would be a big problem even if as soon as the CS 7.3 P01 is out (normally end of this month), we will replace all dfc.jar files with the 7.3 P01 versions.
Another example on a Full Text Server using xPlore 1.6 (same as before, no patches are available yet for xPlore 1.6) including one Primary Dsearch and two IndexAgents for DOCBASE1 and DOCBASE2:
[xplore@fulltext_server_01 ~]$ find $XPLORE_HOME -type f -name dfc.jar -print0 | while read -d $'' file; do echo "DFC: `$JAVA_HOME/bin/java -cp "$file" DfShowVersion` === Size: `ls -ks "$file" | awk '{print $1}'` kb === Location: $file"; done DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/temp/installer/wildfly/dctmutils/templates/dfc/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/dfc/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_Indexagent_DOCBASE1/tmp/vfs/deployment/deployment5417db9ca7307cfc/dfc.jar-aa1927b943be418f/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_Indexagent_DOCBASE1/deployments/IndexAgent.war/WEB-INF/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_Indexagent_DOCBASE2/tmp/vfs/deployment/deploymentbb9811e18d147b6a/dfc.jar-7347e3d3bbd8ffd/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_Indexagent_DOCBASE2/deployments/IndexAgent.war/WEB-INF/lib/dfc.jar DFC: 7.3.0000.0196 === Size: 15220 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch/tmp/vfs/deployment/deployment5fd2cff2d805ceb2/dfc.jar-29edda1355c549b8/dfc.jar DFC: 7.3.0000.0196 === Size: 15220 kb === Location: $XPLORE_HOME/wildfly9.0.1/server/DctmServer_PrimaryDsearch/deployments/dsearchadmin.war/WEB-INF/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/wildfly9.0.1/modules/system/layers/base/emc/documentum/security/main/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/watchdog/lib/dfc.jar DFC: 7.3.0000.0205 === Size: 15236 kb === Location: $XPLORE_HOME/setup/qbs/tool/lib/dfc.jar
Do you see something strange here? Because I do! This is a completely new xPlore 1.6 server which has just been installed and yet we have two different versions of the DFC libraries… It’s not a difference on the minor version but it’s a difference on the build number! As you can see above, it looks like the PrimaryDsearch uses a DFC 7.3 P00 build number 196 while all other DFC versions are in 7.3 P00 build number 205 (so just like the Content Server). The problem here is that each xPlore modules (IA, Dsearch, aso…) are built by different xPlore Teams. Therefore the team that package the Dsearch libraries isn’t the same that the one that package the IndexAgent libraries.
Since there is a difference here, it probably means that the Dsearch team built their package some days/weeks before the other teams (from IA, CS, aso…) and therefore the DFC libraries included in the Dsearch are older… Is it an issue or not? According to EMC, it’s not, BUT I wouldn’t be so categorical. If EMC built this library 9 additional times, it’s not for nothing… There must be a reason behind those builds and therefore not having the latest build seems a little bit risky to me. Since this is just a sandbox environment, I will most probably just wait for the P01 of the xPlore 1.6 which will be release in a few days and I will implement it to have an aligned version of the DFC for all components.
Have fun finding issues in the EMC releases :).