Documentum Administrator does not display the correct “Index Agent” status

At one of my customers, I experienced a problem with the Documentum Administrator (DA), which did not display the correct Index Agent status. In this blog post, I will explain how I found the source of this problem and of course also how I solved this issue.

But first, some background facts: The Documentum Administrator (DA) is displaying “not responding” for the Index Agent, however the Index Agent is running well. No solutions are mentioned in the Documentum support notes.

Analysis

1) I have to use the API code to find the status.

Command to use:

API> apply,c,NULL,FTINDEX_AGENT_ADMIN,NAME,S,,AGENT_INSTANCE_NAME,S,,ACTION,S,start

In my case:

API> apply,s0,NULL,FTINDEX_AGENT_ADMIN,NAME,S,TEST_ftindex_01,AGENT_INSTANCE_NAME,S,ftserver.i_9200_IndexAgent,ACTION,S,status
...
q0
API> next,c,q0
...
OK
API> get,c,q0,status
...
200
API>

“200” means that there is a problem with the Index Agent.

2) I can find errors in two locations:

• by setting the trace for DA, I can find the following error in the trace file:

[!com.documentum.fc.common.DfException: [DM_METHOD_E_JMS_APP_SERVER_NAME_NOTFOUND]error: "The
 app_server_name/servlet_name 'ftserver.i_9200_IndexAgent' is not specified in dm_server_config/dm_jms_config."!]

• In the user session logfile I see the following:

2013-07-18T09:23:06.132697 22991[22991] 0101e2ee80ee8913 [DM_FT_INDEX_T_INIT_INDEX_AGENT_MSG]info: "Attempting to status Index Agent Instance ftserver.i_9200_IndexAgent"
2013-07-18T09:23:06.132766 22991[22991] 0101e2ee80ee8913 [DM_METHOD_E_JMS_APP_SERVER_NAME_NOTFOUND]error: "The app_server_name/servlet_name 'ftserver.i_9200_IndexAgent' is not specified in dm_server_config/dm_jms_config."
2013-07-18T09:23:06.132791 22991[22991] 0101e2ee80ee8913 [DM_FT_INDEX_T_INIT_INDEX_AGENT_MSG]info: "HTTP_POST with args -command status -docbase TESTREPO -user dm_fulltext_index_user -ticket DM_TICKET=T0JKIE5......UkxZeXc2cHd3SVc0dz09Cg== -instance ftserver.i_9200_IndexAgent -details false to Index Agent ftserver.i_9200_IndexAgent failed."
2013-07-18T09:24:05.032297 22991[22991] 0101e2ee80ee8913 [DM_FT_INDEX_E_NO_FTINDEX_AGENT_CONFIG_OBJECTS_FOUND]error: "There were no dm_ftindex_agent_config object(s) found for the specified arguments"

3) Let’s use the previous output to perform a deeper analysis

• In the dm_jms_config the Index Agent app_server_name is not defined – which is normal
• In DA I can see the name of the Application Server – which is is fine
• The dm_server_config object is correct:

retrieve,c,dm_server_config
dump,c,l
the relevant data of the output:
app_server_name [0]: do_method
 [1]: do_mail
 [2]: ftserver.i_9200_IndexAgent

app_server_uri [0]: http://localhost:19080/DmMethods/servlet/DoMethod
[1]: http://localhost:19080/DmMail/servlet/DoMail
[2]: http://ftserver.dummy.com:19200/IndexAgent/servlet/IndexAgent

But then, what is wrong?

4) Reproducing the error

As all seems to be fine, I have to reproduce the error.

• By purposely choosing a wrong AGENT_INSTANCE_NAME – ftserver.9200_IndexAgent instead of ftserver.i_9200_IndexAgent (–> I have removed the “i” ) – I am able to reproduce the error:

API> apply,s0,NULL,FTINDEX_AGENT_ADMIN,NAME,S,TEST_ftindex_01,AGENT_INSTANCE_NAME,S,ftserver.9200_IndexAgent,ACTION,S,status
...
q0
API> next,c,q0
...
OK
API> get,c,q0,status
...
[DM_FT_INDEX_E_NO_FTINDEX_AGENT_CONFIG_OBJECTS_FOUND]error: "There were no dm_ftindex_agent_config object(s) found for the specified arguments"
[DM_METHOD_E_JMS_APP_SERVER_NAME_NOTFOUND]error: "The app_server_name/servlet_name 'ftserver.9200_IndexAgent' is not specified in dm_server_config/dm_jms_config."

• The FTINDEX_AGENT_ADMIN method issues an HTTP_POST method that then invokes the servlet used by the IndexAgent Admin Web utility. So using the HTTP_POST method I can also reproduce the error:

1> execute http_post WITH app_server_name='ftserver.i_9200_IndexAgent'
2> go
[DM_METHOD_E_JMS_APP_SERVER_NAME_NOTFOUND]error: "The app_server_name/servlet_name
'ftserver.i_9200_IndexAgent' is not specified in dm_server_config/dm_jms_config."

• As I see the same name in the DA, I have to add a blank behind the app_server_name value, hoping this is the char I cannot see. Again, I am using the http_post

1> execute http_post WITH app_server_name='ftserver.i_9200_IndexAgent '
 2> go
 result http_response_status request_failed response_doc_id time_taken 
 timed_out time_out_length app_server_host_name app_server_port 
 app_server_uri 
 ----------------------------------------------------------------------------------
 0 HTTP/1.1 200 OK F 0000000000000000 0 
 F 60 ftserver.dummy.com 9200 /IndexAgent/servlet
 /IndexAgent
 (1 row affected)
 1>

With this positive result, I have found the source of my problem: a wrong app_server_name!

Solution

The issue was solved by creating a new app_server_name entry (without the blank) in the docbase config object.
The final test with API proved successful:

API> apply,s0,NULL,FTINDEX_AGENT_ADMIN,NAME,S,TEST_ftindex_01,AGENT_INSTANCE_NAME,S,ftserver.i_9200_IndexAgent,ACTION,S,status
...
q0
API> next,c,q0
...
OK
API> dump,c,q0
...
USER ATTRIBUTES name [0]: ftserver.i_9200_IndexAgent
 status [0]: 0
SYSTEM ATTRIBUTES
APPLICATION ATTRIBUTES
INTERNAL ATTRIBUTESAPI>

Once logged off and signed in again, DA displays the correct status (running) for the Index Agent.
I hope this solution will help other people in a situation where the EMC powerlink solution does not work.