{"id":14647,"date":"2020-10-05T22:40:56","date_gmt":"2020-10-05T20:40:56","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/"},"modified":"2025-10-24T09:32:06","modified_gmt":"2025-10-24T07:32:06","slug":"enhancing-idql-iapi-with-rlwrap-part-i","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/","title":{"rendered":"Enhancing idql\/iapi with rlwrap (part I)"},"content":{"rendered":"<h2>The rant<\/h2>\n<p>I have a confession to make: I&#8217;m obsessed with idql\/iapi. These tools so central to Documentum administrative tasks are so lame for interactive work that they have probably disgusted more than their share of administrators. The problem is that I have to use them almost daily, and everyday I grumble: there must be a better way. Sure, there were third-party utilities such as Samson and Repoint in the past, and now <a href=\"https:\/\/www.fme-us.com\/products\/dqman\/\">dqman<\/a> and <a href=\"https:\/\/www.informedgroup.com\/dqlbuddy\/\">DQLBuddy<\/a>, when they are allowed on the desktop, but my work occurs mostly server-side, on Unix or Linux machines. How come those tools received no update during all these years ? It&#8217;s been almost 30 years if senility has not confused my memory. What was the company doing all these years ? They sure deserve some enhancements as the current ones go through the roof on the Suckness scale.<\/p>\n<p>If you ever have been working as an administrator of Documentum installations and repositories, you inevitably have used the idql &amp; iapi command-line utilities. While there are several GUI-based third-party alternatives, e.g. the already mentioned dqman for Windows, idql and iapi are still very useful to automate administrative tasks in repositories. As they don&#8217;t need a graphical interface, they are are much lighter weight and can work server-side from a GUI-less terminal. However, the least that can be said about them is that they are extremely crude. They are effective for batch execution but they are so spartan that they even lack the very basic functionalities for a comfortable interactive work. Under Unix for example, idql doesn&#8217;t provide command-line history (although they do under Windows). Some functionalities, e.g. spooling its output into a file, are only available at launch time and no longer once it is running (this is true for idql but not for iapi which has the little known <em>$<\/em> command). Also, no facilities for formatting the output are available (there is the <em>-W|w<\/em>  command-line argument and command but they just silently truncate the columns), resulting in a totally messed up display when long rows are output. In addition, some frequently used functionalities, e.g. the dump, get and set commands, are only accessible from within the iapi tool (although there is the describe command in idql) and it would be more effective to have these two tools merged into a unique one. Other functionalities quite common in Unix tools, e.g. shelling out an external command without quitting the tool, are missing.<\/p>\n<p>Despite all this, there is hope. One could still develop an enhanced alternative tool using the DFCs from within a JVM-based language (e.g. java, jython, groovy, etc..) or use the dmcl dmapp.h interface from within C or any language that can link to the libdmcl.so library such C, gawk, perl or python. See my previous blog posts <a href=\"https:\/\/www.dbi-services.com\/blog\/adding-a-documentum-extension-to-gawk-part-i\/\">here<\/a>, <a href=\"https:\/\/www.dbi-services.com\/blog\/adding-a-documentum-extension-into-python\/\">here<\/a> and <a href=\"https:\/\/www.dbi-services.com\/blog\/a-few-scripting-languages-for-documentum\/\">here<\/a>.<br \/>\nIf you are fluent in C\/C++, the source code of an ancient version of idql is available in any Documentum installation&#8217;s folder $DOCUMENTUM\/share\/sdk\/example\/code and could be the basis for a much better enhanced version.<\/p>\n<p>But is there a way to enhance idql\/iapi without coding ? As surprising as it sound, there are several tools that can help here, one of these is <a href=\"https:\/\/github.com\/hanslub42\/rlwrap\">rlwrap<\/a> . This tool is really amazing and can address several of the shortcomings of the Documentum command-line tools. Let&#8217;s list the peskiest ones.<\/p>\n<h2>The shortcomings<\/h2>\n<p>Here is what idql and iapi lack the most, in no particular order:<\/p>\n<ol>\n<li>command-line history: recall and execute past commands;<\/li>\n<li>save the history so it is available the next time the tools are launched;<\/li>\n<li>command-line editing using vi or emacs commands, like in the shells;<\/li>\n<li>start vi from the input line;<\/li>\n<li>multi-lines commands or statements;<\/li>\n<li>easy piping so filters can be used to further customize the output, e.g. for grepping and colorization;<\/li>\n<li>really nice to have: macros;<\/li>\n<li>better visualization of output;<\/li>\n<li>execute external programs<\/li>\n<li>name completion<\/li>\n<\/ol>\n<p>Well, believe it or not, all of them can be alleviated by rlwrap without writing a single line of code !<br \/>\nAs its author Hans Lub says it in the readme file:<br \/>\n<em>rlwrap is a &#8216;readline wrapper&#8217;, a small utility that uses the GNU<\/em><br \/>\n<em>readline library to allow the editing of keyboard input for any<\/em><br \/>\n<em>command. (&#8230;) You should consider rlwrap<\/em><br \/>\n<em>especially when you need user-defined completion (by way of completion word<\/em><br \/>\n<em>lists) and persistent history, or if you want to program &#8216;special<\/em><br \/>\n<em>effects&#8217; using the filter mechanism.<\/em><br \/>\nSo what rlwrap basically does is use the GNU readline library to read input from the keyboard (and therefore opening up all the features of the library) and pass along the input to the specified program that it invoked. Its syntax is:<\/p>\n<pre class=\"brush: actionscript3; gutter: false; first-line: 1\">$ rlwrap [-options] &lt;command&gt; &lt;args&gt;<\/pre>\n<p>Example:<\/p>\n<pre class=\"brush: actionscript3; gutter: false; first-line: 1\">$ rlwrap idql dmtest72 -Udmadmin -Pdmadmin<\/pre>\n<p>Now, idql behaves as if it had been linked with the readline library, and the result is utterly impressive.<br \/>\nTo install it, just use your distribution&#8217;s usual package manager. After that, a man page is available for reference, but it can also be read from <a href=\"https:\/\/manpages.debian.org\/buster\/rlwrap\/rlwrap.1.en.html\">here<\/a> if your prefer a web page.<\/p>\n<h2>The miracle<\/h2>\n<p>rlwrap has been much publicized to enhance Oracle sql*plus, even though sql*plus is already light-years ahead of idql\/iapi in terms of flexibility and power. For example, sql*plus&#8217; built-in small line editor allows to simply and easily edit multi-line statements; column formatting is also easy, it just lacks command-line history under Unix.<br \/>\nAll that follows in this article also applies to any rlwrapped executable, not just to sql*plus, idql or iapi. Thus, what you learn here can be leveraged anywhere. Furthermore, as it is possible to specify distinct configuration files separately for each executable, and even define a <em>$RLWRAP_HOME<\/em> directory, conflicts and namespace pollution are prevented.<br \/>\nrlwrap works beautifully with idql\/iapi and enhances them tremendously. Here is the simplest usage:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1]\">$ rlwrap idql dmtest72 -Udmadmin -Pdmadmin\n...\nEMC Documentum idql - Interactive document query interface\n...\nConnected to Documentum Server running Release 7.2.0000.0155 Linux64.Oracle\n1&gt;<\/pre>\n<p>Now type a few DQL statements followed by &lt;enter&gt; and use the arrow keys to navigate the history. A dream comes true !<br \/>\nvi or emacs editing mode can be switched to forth and back as follows:<br \/>\nvi &#8212;&gt; emacs: &lt;esc&gt; (back to command mode) &lt;esc&gt;-ctrl-e<br \/>\nemacs &#8212;&gt; vi: &lt;esc&gt; ctrl-j<br \/>\nGenerally, there is little need to switch editing mode once the initial one has been selected from the configuration file.<br \/>\nAs rlwrap uses readline, it takes its configuration from the same default file as readline, i.e. <em>~\/.inputrc<\/em> and adds a few settings of its own. Here is an example of this file:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1\">set convert-meta Off\n\n# editing mode, either emacs or vi;\n# I prefer vi mode by default;\n#set editing-mode emacs\nset editing-mode vi\n\nset history-size -1\nset horizontal-scroll-mode Off\nset input-meta On\nset keymap vi\n\n# this is for rlwrap's rlwrap-call-editor:\n\"\\C-x\\C-e\": rlwrap-call-editor<\/pre>\n<p>If you exit idql now and re-start it via rlwrap, all the commands typed so far are still available; by default, they were saved in <em>~\/.&lt;wrapped_program&gt;_history<\/em>, here <em>~\/.idql_history<\/em>.<br \/>\nNow, while on the rlwrap&#8217;s command-line, type &lt;esc&gt; ctrl-x ctrl-e if in vi editing mode, ctrl-x ctrl-e if in emacs editing mode; this will start the editor defined in <em>$RLWRAP_EDITOR<\/em>, <em>$VISUAL<\/em> or <em>$EDITOR<\/em> environment variables, typically vi. rlwrap&#8217;s internal command <em>rlwrap-call-editor<\/em> is bound to the key sequence ^X^E, as shown above; it is possible to remap it as you fancy.<\/p>\n<p>So far, through the simplest syntax, rlwrap already covered items 1 to 4, quite remarkable.<\/p>\n<p>In order to use multi-line edit, it is necessary first to define a line separator; we cannot use the &lt;enter&gt; key as it causes the line input to be terminated and sent to the rlwrapped executable so we must choose a rarely used character string; it is a choice that can be done on a per-execution basis so it doesn&#8217;t need to be the ultimate, definitive, unique string in the history of mankind. Let&#8217;s choose the string &#8220;__&#8221; for instance and launch rlwrap with the proper option named, you guesses it, <em>&ndash;&ndash;multi-line<\/em>:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,6,18]\">rlwrap --multi-line='__' idql dmtest72 -Udmadmin -Pdmadmin\n...\nEMC Documentum idql - Interactive document query interface\n...\nConnected to Documentum Server running Release 7.2.0000.0155 Linux64.Oracle\n1&gt; select __ count(*) __ from __dm_user\n-- While still on this line, invoke the editor, &lt;esc&gt; ^x^e:\nselect\ncount(*)\nfrom\ndm_user\n:q\n1&gt; select\n2&gt; count(*)\n3&gt; from\n4&gt; dm_user\n&lt;up arrow&gt;\n5&gt; select __ count(*) __ from __dm_user<\/pre>\n<p>Thus, the original line with the line separator strings is still in the command history, unchanged, but when editing it, the editor splits it on multiple lines as specified, and sends those multiple physical lines to idql on exiting. This makes it pleasing to edit a complex statement with an external editor, and also easy to reach for the statement in the history as one line.<br \/>\nAnd that was item 5 in the list. Before tackling the big issue of output formatting, let&#8217;s first see a few nice goodies, a kind of intermission if you will.<\/p>\n<h2>Prompts, colors and macros<\/h2>\n<h3>Prompts<\/h3>\n<p>rlwrap also allows to change the prompt of the launched executable (idql here) through the <em>&ndash;&ndash;substitute-prompt<\/em>, e.g.:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1;  highlight: [1,9]\">\nrlwrap --substitute-prompt=\"howdy, mate ?&gt;\" iapi dmtest72 -Udmadmin -Pdmadmin\n...\n\tEMC Documentum iapi - Interactive API interface\n...\nConnecting to Server using docbase dmtest72\n...\nConnected to Documentum Server running Release 7.2.0000.0155  Linux64.Oracle\nSession id is s0\nhowdy, mate ?&gt; quit \nBye\n<\/pre>\n<p>However, sometimes it does not work flawlessly because idql already displays its own prompt and even completes it with a line counter if a statement spans multiple lines; if rlwrap&#8217;s prompt takes over, it overwrites the numbering and the output can be confusing. It is said to nonetheless be useful for programs with no prompt but when I tried with iapi and the <em>-V-<\/em> command-line option, the substitution prompt was not displayed.<\/p>\n<p>Another oddity of the substitution prompt is that sometimes rlwrap starts displaying the prompt too early while iapi is not ready yet, e.g. it still connecting or displaying the banner, which results in the prompt being pushed up by the incoming text and printed several times, as shown below:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [2,10,12]\">\n$ rlwrap --substitute-prompt=\"howdy, mate ?&gt;\" iapi dmtest72 -Udmadmin -Pdmadmin\nhowdy, mate ?&gt;\n\nEMC Documentum iapi - Interactive API interface\n(c) Copyright EMC Corp., 1992 - 2016\nAll rights reserved.\nClient Library Release 7.3.0000.0205\n...\nConnecting to Server using docbase dmtest72\nhowdy, mate ?&gt;[DM_SESSION_I_SESSION_START]info: \"Session 0100c39880002253 started for user dmadmin.\"\n...\nhowdy, mate ?&gt;Connected to Documentum Server running Release 7.2.0000.0155 Linux64.Oracle\nSession id is s0\nhowdy, mate ?&gt;<\/pre>\n<p>The option <em>&ndash;&ndash;wait-before-prompt<\/em> exists to prevent this; it tells rlwrap to wait until no output is displayed, presumably meaning that the rlwrapped executable finished outputting stuff (e.g. its banner), displayed its prompt and is waiting now for an user input.<br \/>\nExample of use with a more useful prompt:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,4]\">\n$ user_name=dmadmin repo=dmtest72 rlwrap --substitute-prompt=\"${user_name}@${repo} &gt;\" --wait-before-prompt=-1000 iapi $repo -U${user_name} -Pdmadmin\n...\nConnected to Documentum Server running Release 7.2.0000.0155  Linux64.Oracle\ndmadmin@dmtest72&gt; quit\nBye\n<\/pre>\n<p>Here, <em>&ndash;&ndash;wait-before-prompt<\/em> is set to 1 seconds, but this does not always work. Worst, this delay slows down every line of the subsequent output so the cure is worse than the disease. Nevertheless, it could still be useful in e.g. non-interactive demos, preparation of slides or documentation, etc., i.e. anything that doesn&#8217;t make one fall asleep behind the keyboard. So this rlwrap&#8217;s feature may cause some problem sometimes, and sometimes it works as expected. Use it with caution.   <\/p>\n<p>In order to add more visibility to the prompt, either the original or the substituted one, let&#8217;s colorize it:<br \/>\n<a href=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_1.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_1.png\" alt=\"\" width=\"300\" height=\"61\" class=\"alignnone size-medium wp-image-42999\" \/><\/a><\/p>\n<p>Colors are the usual 8 ones written in lowercase for normal intensity, i.e. black, red, green, yellow, blue, magenta, cyan and white, and in uppercase for bright intensity, i.e. BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN and WHITE, a clever notation I think. E.g.:<br \/>\n<a href=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_19.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_19.png\" alt=\"\" width=\"300\" height=\"58\" class=\"alignnone size-medium wp-image-43023\" \/><\/a><\/p>\n<h3>Macros<\/h3>\n<p>Another powerful feature is macros; they are implemented by the filter <em>simple_macro<\/em>.<br \/>\nTo enable macros, rlwrap must be started with the option <em>-z<\/em> or <em>&ndash;&ndash;filter=simple_macro<\/em>. Then, to define a macro <em>name<\/em>, use the syntax <em>$name((text))<\/em>. To use the macro <em>name<\/em>, type <em>$name<\/em>. Example:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [2,10]\"># define a shortcut c for clear:\n1&gt; $c((clear))\n1&gt; \n...\n# use it;\n1&gt;\n2&gt;\n3&gt;\n4&gt;\n5&gt; $c\n1&gt;<\/pre>\n<p>The prompt counter has been reset to 1, which shows that the input buffer has been emptied by the clear command.<br \/>\nUnfortunately, there is a small peculiarity:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [5]\">\n1&gt;\n2&gt;\n3&gt;\n4&gt; $c((clear))\n1&gt;<\/pre>\n<p>Defining a macro executes it too. While this looks harmless, although it can produce spurious error messages, it prevents commands such as <em>quit<\/em> to be abbreviated because idql is exited right after the definition:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,8]\">\n1&gt; $q((quit))\nBye\nInstead of:\n1&gt; $q((quit))\n2&gt;\n3&gt;\n....\n&gt; $q\nBye<\/pre>\n<p>But completion (see below) provides an even better solution here. Nevertheless, macros are a nice addition. Their usefulness stands in that they allow to shorten long, regularly used commands, just like bash aliases do.<br \/>\nOther examples:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [2,9,17,23,29]\">\n-- count users;\n$cu((select count(*) from dm_user ^Mgo))\ngo\n2&gt; count(*)\n----------------------\n54\n(1 row affected)\n\n1&gt; $cu\ngo\n2&gt; count(*)\n----------------------\n54\n(1 row affected)\n\n-- count documents;\ncd((select count(*) from dm_document ^Mgo))\ngo\n2&gt; count(*)\n----------------------\n814\n(1 row affected)\n1&gt; $cu\ngo\n2&gt; count(*)\n----------------------\n54\n(1 row affected)\n1&gt; $cd\ngo\n2&gt; count(*)\n----------------------\n814\n(1 row affected)<\/pre>\n<p>It is quite visible here that the macros were executed right after their definition; I guess, we can live with that.<\/p>\n<p>In iapi:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,3,14]\">\n$ rlwrap --no-warnings --filter=simple_macro --ansi-colour-aware --prompt-colour=RED --multi-line='__' iapi dmtest72 -Udmadmin -Pdmadmin\n# show dm_server_config;\n$sc((retrieve,c,dm_server_config ^M dump,c,l^M))\n3d00c39880000102\nAPI&gt; ...\nUSER ATTRIBUTES\n\n  object_name                     : dmtest72\n  title                           : \n  subject                         : \n  authors                       []: \n...\n# show a few config's infos;\n$sci((retrieve,c,dm_server_config ^M get,c,l,object_name^Mget,c,l,r_server_version^Mget,c,l,r_host_name^M))\n...\n3d00c39880000102\nAPI&gt; ...\ndmtest72\nAPI&gt; ...\n7.2.0000.0155  Linux64.Oracle\nAPI&gt; ...\ndmtest.cec\n<\/pre>\n<p>^M is entered by typing ctrl-v ctrl-m as usual in vi editing mode (i.e. like in the vi editor).<\/p>\n<p>Macros are saved in the same history file rlwrap uses for program <em>&lt;wrapped_program&gt;<\/em>: <em>~\/.&lt;wrapped_program&gt;_history<\/em>, e.g. <em>~\/.idql_history<\/em>. Thus, they can be shared, recalled and used between idql sessions.<\/p>\n<p>Filters can be chained using the &#8220;pipeline&#8221; filter and the syntax:<\/p>\n<pre class=\"brush: actionscript3; gutter: false; first-line: 1\">--filter='pipeline filter{:filter}'<\/pre>\n<p>E.g.:<\/p>\n<pre class=\"brush: actionscript3; gutter: false; first-line: 1\">--filter='pipeline pipeto:simple_macro'<\/pre>\n<p>A common user error that happens when using macros is to forget to include <em>simple_macro<\/em> in rlwrap&#8217;s <em>pipeto<\/em> argument. No error message will be output when defining a macro or using it, but no result will be produced either, as illustrated below:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1]\">\nrlwrap --no-warnings -f idql-completions.txt --filter='pipeline pipeto' --ansi-colour-aware --prompt-colour=RED\n...\n\tEMC Documentum idql - Interactive document query interface\n...\nConnecting to Server using docbase dmtest73\n...\nConnected to Documentum Server running Release 7.3.0000.0214  Linux64.Oracle\n1&gt;$qq((gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {printf(\"%s%s%s\\n\", !match($0, \/^[0-9a-f]{16}\/) ? col[l%2] : col[++l%2], $0, reset)}'))\n2&gt; clear \n1&gt; select * from dm_user \n go| $qq\n\n<\/pre>\n<p>This silent behavior can cause some frustration while troubleshooting but is easily circumvented by systematically including the macro on rlwrap&#8217;s command-line.<\/p>\n<p>Another hard-to-find error occurs when double closing parentheses &#8220;))&#8221; are present in a macro definition; rlwrap considers them to be the closing macro delimiters and removes them from the definition which subsequently invalidates the script, e.g.:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,2,6]\">\n1&gt; $qq((gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/)) printf(\"%s\", col[l%2]); else printf(\"%s\", col[++l%2]); printf(\"%s%s\\n\", $0, reset)}'))\n1&gt; gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/ printf(\"%s\", col[l%2]); else printf(\"%s\", col[++l%2]); printf(\"%s%s\\n\", , reset)}'))\n2&gt; clear \n1&gt; select * from dm_user __ go| $qq\n1&gt; select * from dm_user \n go| gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/\n\nsh: 1: Syntax error: Unterminated quoted string\n<\/pre>\n<p>On line 1, the <em>if (!match(&#8230;))<\/em> statement has been digested into if <em>(!match($0, \/^[0-9a-f]{16}\/<\/em> and the new line has replaced the line with the macro definition (line 2) which now invalidates the statement; in effect, when invoking the macro in line 4, a syntax error is raised (line 6).<br \/>\nTo fix this, just insert a space between both closing parentheses, as follow:<\/p>\n<pre class=\"brush: actionscript3; gutter: false; first-line: 1; highlight: [1]\">\n1&gt; $qq((gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/) ) printf(\"%s\", col[l%2]); else printf(\"%s\", col[++l%2]); printf(\"%s%s\\n\", $0, reset)}'))\n<\/pre>\n<p>Another quirk is that the order of the filters matters; e.g. if <em>pipeto<\/em> precedes <em>simple_macro<\/em> (<em>&ndash;&ndash;filter=&#8217;pipeline pipeto:simple_macro&#8217;<\/em>) and an in-line gawk script is used, the macro definition and invocation will be error-free but nothing will be produced as a result and no error will be raised, just as if <em>simple_macro<\/em> were missing. The obvious solution is to specify <em>simple_macro<\/em> before <em>pipeto<\/em> to avoid this situation, e.g. <em>&ndash;&ndash;filter=&#8217;pipeline simple_macro:pipeto&#8217;<\/em>. Use a shell alias to call rlwrap with its filter arguments in the correct order to get rid of this issue once for all.<\/p>\n<p>However, even when the filters are correctly ordered, if the macro contains a pipe character, e.g. aimed at the shell or as part of gawk&#8217;s || logical operator, the shell will report a syntax error, e.g.:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1,4]\">\nrlwrap --no-warnings -f idql-completions.txt --filter='pipeline simple_macro:pipeto' --ansi-colour-aware --prompt-colour=RED --multi-line='__' -q \\' idql dmtest73 -Udmadmin -Pdmadmin\n...\n1&gt; $qq((gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/) ) printf(\"%s\", col[l%2]); else printf(\"%s\", col[++l%2]); printf(\"%s%s\\n\", $0, reset)}' | less))\n1&gt; $qq((gawk 'BEGIN {col[0]=\"\\033[1m\\033[93m\"; col[1]=\"\\033[1m\\033[94m\"; reset=\"\\033[0m\"; l = 0} {if (!match($0, \/^[0-9a-f]{16}\/) ) printf(\"%s\", col[l%2]); else printf(\"%s\", col[++l%2]); printf(\"%s%s\\n\", $0, reset)}' \n\nsh: 1: Syntax error: \")\" unexpected\n<\/pre>\n<p>One line 3, the macro is defined with the pipe symbol | to invoke the less command on the output; on line 4, the offending &#8220;| less&#8221; part is removed and the definition fails with a syntax error from the shell on line 6. Fortunately, this error message can be ignored because the macro invocation works just fine. I guess the error comes from the fact that rlwrap prematurely executes the macro by passing it to the shell as part of its definition.<\/p>\n<p>rlwrap comes with several pre-installed filters; for a complete list, use the command:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1]\">rlwrap -z listing\nThe following filters can be found in  \/usr\/share\/rlwrap\/filters\ncount_in_prompt                replace prompt by simple counter\nftp_filter                     run plain Netkit ftp with completion for commands, local and remote files\nhistory_format                 Append  to every history item, and strip it off again when input is accepted\nnull                           a filter that does nothing\npaint_prompt                   paint the prompt in colour gradient between X11 colours  and \npipeline                       combines the effects of 2 or more filters\npipeto                         Allow piping of  output through pagers or other shell commands\nscrub_prompt                   removes all junk from prompt\nsimple_macro                   simple on-the-fly macro processing\ntemplate                       filter template\nunbackspace                    remove backspaces from output\n<\/pre>\n<p>Help for a filter can be requested thusly, e.g.:<\/p>\n<pre class=\"brush: actionscript3; gutter: true; first-line: 1; highlight: [1]\">\n$ rlwrap -z pipeline\nUsage: rlwrap [-options] -z 'pipeline filter_1:filter_2:..., filter_n' \ncombines the effects of 2 or more filters\nmessages will be passed through filter_1, ..., filter_n.\nUse a backslash to pass a ':' that is not meant as a pipe symbol, e.g:\nrlwrap -z 'pipeline prompt hello\\: : logger out' command\n<\/pre>\n<p>This is the end of Part I. Please, turn to <a href=\"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-ii\/\">Part II<\/a> for even more rlwrap&#8217;s goodies.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The rant I have a confession to make: I&#8217;m obsessed with idql\/iapi. These tools so central to Documentum administrative tasks are so lame for interactive work that they have probably disgusted more than their share of administrators. The problem is that I have to use them almost daily, and everyday I grumble: there must be [&hellip;]<\/p>\n","protected":false},"author":40,"featured_media":14650,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[525],"tags":[2102,1841,2103],"type_dbi":[],"class_list":["post-14647","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-enterprise-content-management","tag-command-history","tag-macros","tag-rlwrap"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.2 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Enhancing idql\/iapi with rlwrap (part I) - dbi Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Enhancing idql\/iapi with rlwrap (part I)\" \/>\n<meta property=\"og:description\" content=\"The rant I have a confession to make: I&#8217;m obsessed with idql\/iapi. These tools so central to Documentum administrative tasks are so lame for interactive work that they have probably disgusted more than their share of administrators. The problem is that I have to use them almost daily, and everyday I grumble: there must be [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/\" \/>\n<meta property=\"og:site_name\" content=\"dbi Blog\" \/>\n<meta property=\"article:published_time\" content=\"2020-10-05T20:40:56+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-10-24T07:32:06+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1694\" \/>\n\t<meta property=\"og:image:height\" content=\"271\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Middleware Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Middleware Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"17 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/\"},\"author\":{\"name\":\"Middleware Team\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/8d8563acfc6e604cce6507f45bac0ea1\"},\"headline\":\"Enhancing idql\\\/iapi with rlwrap (part I)\",\"datePublished\":\"2020-10-05T20:40:56+00:00\",\"dateModified\":\"2025-10-24T07:32:06+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/\"},\"wordCount\":2405,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/04\\\/screenshot_18.jpg\",\"keywords\":[\"command history\",\"Macros\",\"rlwrap\"],\"articleSection\":[\"Enterprise content management\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/\",\"name\":\"Enhancing idql\\\/iapi with rlwrap (part I) - dbi Blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/04\\\/screenshot_18.jpg\",\"datePublished\":\"2020-10-05T20:40:56+00:00\",\"dateModified\":\"2025-10-24T07:32:06+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/8d8563acfc6e604cce6507f45bac0ea1\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/04\\\/screenshot_18.jpg\",\"contentUrl\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/wp-content\\\/uploads\\\/sites\\\/2\\\/2022\\\/04\\\/screenshot_18.jpg\",\"width\":1694,\"height\":271},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/enhancing-idql-iapi-with-rlwrap-part-i\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Enhancing idql\\\/iapi with rlwrap (part I)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/\",\"name\":\"dbi Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/#\\\/schema\\\/person\\\/8d8563acfc6e604cce6507f45bac0ea1\",\"name\":\"Middleware Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g\",\"caption\":\"Middleware Team\"},\"url\":\"https:\\\/\\\/www.dbi-services.com\\\/blog\\\/author\\\/middleware-team\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Enhancing idql\/iapi with rlwrap (part I) - dbi Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/","og_locale":"en_US","og_type":"article","og_title":"Enhancing idql\/iapi with rlwrap (part I)","og_description":"The rant I have a confession to make: I&#8217;m obsessed with idql\/iapi. These tools so central to Documentum administrative tasks are so lame for interactive work that they have probably disgusted more than their share of administrators. The problem is that I have to use them almost daily, and everyday I grumble: there must be [&hellip;]","og_url":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/","og_site_name":"dbi Blog","article_published_time":"2020-10-05T20:40:56+00:00","article_modified_time":"2025-10-24T07:32:06+00:00","og_image":[{"width":1694,"height":271,"url":"http:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg","type":"image\/jpeg"}],"author":"Middleware Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Middleware Team","Est. reading time":"17 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#article","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/"},"author":{"name":"Middleware Team","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d8563acfc6e604cce6507f45bac0ea1"},"headline":"Enhancing idql\/iapi with rlwrap (part I)","datePublished":"2020-10-05T20:40:56+00:00","dateModified":"2025-10-24T07:32:06+00:00","mainEntityOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/"},"wordCount":2405,"commentCount":0,"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg","keywords":["command history","Macros","rlwrap"],"articleSection":["Enterprise content management"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/","url":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/","name":"Enhancing idql\/iapi with rlwrap (part I) - dbi Blog","isPartOf":{"@id":"https:\/\/www.dbi-services.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#primaryimage"},"image":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg","datePublished":"2020-10-05T20:40:56+00:00","dateModified":"2025-10-24T07:32:06+00:00","author":{"@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d8563acfc6e604cce6507f45bac0ea1"},"breadcrumb":{"@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#primaryimage","url":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg","contentUrl":"https:\/\/www.dbi-services.com\/blog\/wp-content\/uploads\/sites\/2\/2022\/04\/screenshot_18.jpg","width":1694,"height":271},{"@type":"BreadcrumbList","@id":"https:\/\/www.dbi-services.com\/blog\/enhancing-idql-iapi-with-rlwrap-part-i\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.dbi-services.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Enhancing idql\/iapi with rlwrap (part I)"}]},{"@type":"WebSite","@id":"https:\/\/www.dbi-services.com\/blog\/#website","url":"https:\/\/www.dbi-services.com\/blog\/","name":"dbi Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.dbi-services.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.dbi-services.com\/blog\/#\/schema\/person\/8d8563acfc6e604cce6507f45bac0ea1","name":"Middleware Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ddcae7ba0f9d1a0e7ae707f0e689e4a9c95bb48ec49c8e6d9cc86d43f4121cb6?s=96&d=mm&r=g","caption":"Middleware Team"},"url":"https:\/\/www.dbi-services.com\/blog\/author\/middleware-team\/"}]}},"_links":{"self":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/14647","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/users\/40"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/comments?post=14647"}],"version-history":[{"count":1,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/14647\/revisions"}],"predecessor-version":[{"id":41177,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/posts\/14647\/revisions\/41177"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media\/14650"}],"wp:attachment":[{"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/media?parent=14647"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/categories?post=14647"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/tags?post=14647"},{"taxonomy":"type","embeddable":true,"href":"https:\/\/www.dbi-services.com\/blog\/wp-json\/wp\/v2\/type_dbi?post=14647"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}