Overcome jboss-cli Limitations with Ansible

The work on YaK JBoss/WildFly component helped me to improve my knowledge of Ansible, but not only. I also learned a lot on jboss-cli scripting and especially limitations that comes with it. For instance, followings are not supported by jboss-cli language:

• nesting if
• incrementing/modifying variables
• no while loop

Even if there are some possibilities to overcome them with programming language like Java, as I am on Ansible, I decided to use it.

Introduction to Credential Store

To encrypt any sensitive data in the configuration, we can use expression feature of Elytron security module. This is done in several steps:

1. Create a secret key credential store:

/subsystem=elytron/secret-key-credential-store=initial:add(path=/data/certs/initial.cs)

2. Create an expression and associated resolver which will use the key just created:

/subsystem=elytron/expression=encryption:add(resolvers=[{name=initial-resolver, credential-store=initial, secret-key=key}])

3. Create the main credential store with a password encrypted via initial-resolver:

/subsystem=elytron/credential-store=CredentialStore:add(path=/data/certs/CredentialStore.jceks,create=true,credential-reference={clear-text="${ENC::initial-resolver:RUxZAUMQf3QXsesbrrM0YGbcv1ayxSymgPQrc3Ye5gohjDrAHVs=}"})

Encrypting key is done with this command:

/subsystem=elytron/expression=encryption:create-expression(clear-text="key-string")

4. Create a new key in the encrypted CredentialStore:

/subsystem=elytron/credential-store=CredentialStore:generate-secret-key(alias=main-key)

Then, we are good to encrypt any string either via CredentialStore or expression.

Idempotency with pure jboss-cli

Even if not perfect, it is possible to implement idempotency with pure jboss-cli scripting language.

For example, I want to add secret in a credential store only if it is not already present, but method will be slightly different as my previous blog as read-resource() method does not apply here.

To get existing aliases, the following method can be executed with a result of type array:

[[email protected]:9993 /]  /subsystem=elytron/credential-store=CredentialStore:read-aliases()
{
    "outcome" => "success",
    "result" => [
        "key",
        "jks_password",
        "datasource_password"
    ]
}

Fortunately, for loop exist and can be used on that result:

set found=no
for obj in /subsystem=elytron/credential-store=CredentialStore:read-aliases()
    if (result == jks_password) of :resolve-expression(expression=$obj)
        set found=yes
    end-if
done

At line 1, I set found to no.
Line 2 is the beginning of the for loop.
Line 3 is testing the value of the current cell ($obj) of the array. The way of testing variable is a bit weird in jboss-cli, but it works fine like that. If result of resolve-expression matches condition (equal to jks_password), then found will be set to yes.

Second part of the script is to test if found is equal to yes or no.

if (result == "yes") of :resolve-expression(expression=$found)
    echo nochange
else
    /subsystem=elytron/credential-store=CredentialStore:add-alias(alias=jks_password,secret-value=MyPassword)
    echo changed add
end-if

If it is yes, I only print “nochange”. If test is false (found not equal to yes), I will trigger the add command and echo changed so that Ansible is aware and task display accurate state.

One might think this could be applied to any operation or attribute, but sadly it is not that easy. We will see an example in next section.

Ansible to the Rescue

As I wanted to make everything idempotent, I wanted also encryption expression creation to be.

for loop can’t be used as in previous chapter because I did not find any way to compare object value ({name=CredentialStore-resolver,credential-store=CredentialStore,secret-key=main-key}) with expected value. Also resolve-expression on this fails with "WFLYCTL0097: Wrong type for 'expression'. Expected [EXPRESSION, STRING] but was OBJECT".

My idea here was to first find the index in the LIST with the same name with a trick: Do the for loop in Ansible and not in jboss-cli scripting language. As in Ansible, I can’t know how many items LIST will contain I must decide of a maximum amount of tries. For this, I am creating a variable named jboss_cli_list_max_iterations.

Here is the code for the search in Jinja template:

set found=no
set position=0
# jboss_cli_list_max_iterations: {{ jboss_cli_list_max_iterations }}
try {# BEGINNING of try-catch block #}
{% for index in range(jboss_cli_list_max_iterations) +%}
  # index {{ index }}
  if (result == "{{ query_name }}") of {{ operation.address }}:read-attribute(name={{  operation.name }}[{{ index }}]{{ point }})
  set found=yes
  set position={{ index }}
  ls exit # fake command to raise exception
  end-if
  if (result == undefined) of {{ operation.address }}:read-attribute(name={{  operation.name }}[{{ index }}]{{ point }})
  set found=no
  ls exit # fake command to raise exception
  end-if
  set position={{ index }}
{% endfor %}

As there is no while loop and no way to exit the for loop earlier (when index has been found), I am using try-catch option. Whenever item has been found (line 7), I am updating required variables (found and position) and trigger a failing command that will exit the for loop (line 10).

Catch section will look like:

catch
  if (result == "7") of :resolve-expression(expression=$position)
    echo error: jboss_cli_list_max_iterations reached (8)
    exit
  end-if
end-try

In this section, I check if we reached the maximum defined loop or not (exiting with error).

Then, in a second phase, I must check each field of the resolver individually:

if (result != "CredentialStore-resolver") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[$position].name)
    ls exit # fake command to raise exception
end-if
if (result != "CredentialStore-resolver") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[$position].name)
    set different=yes
end-if
if (result != "CredentialStore") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[$position].credential-store)
    set different=yes
end-if
if (result != "main-key") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[$position].secret-key)
    set different=yes
end-if

If any field is different from target value, I update different variable to yes. If it is different, I remove the item at position I set earlier in template:

if (result == "yes") of :resolve-expression(expression=$different)
    /subsystem=elytron/expression=encryption:list-remove(name=resolvers,index=$position)
    set found=no
end-if

Then, simply add it:

if (result == "yes") of :resolve-expression(expression=$found)
    echo nochange
else
    /subsystem=elytron/expression=encryption:list-add(name=resolvers,value={name=CredentialStore-resolver,credential-store=CredentialStore,secret-key=main-key})
    echo changed list-add
end-if

The code could be difficult to read as it mixes jinja templating as well as jboss-cli scripting language. Generated code will look like:

set found=no
# jboss_cli_list_max_iterations: 8
try
# index 0
if (result == "CredentialStore-resolver") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[0].name)
    set found=yes
    set position=0
    ls exit # fake command to raise exception
end-if
# index 1
set position=1
if (result == "CredentialStore-resolver") of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[1].name)
    set found=yes
    ls exit # fake command to raise exception
end-if
if (result == undefined) of /subsystem=elytron/expression=encryption:read-attribute(name=resolvers[1].name)
    set found=no
    ls exit # fake command to raise exception
end-if
...

Generated code can grow very quickly. For example, SSL hardening script is more than 6k bytes.

How to Enhance This

One big advantage of this solution is that generated script are idempotent without Ansible, so they can be used on any JBoss-EAP/WildFly installation directly without Ansible.

The disadvantage is the complexity of the code brought by the mix of Jinja and jboss-cli scripting language and the syntax differences (ie. endif vs end-if) which could be confusing.

To avoid this complexity, we could move all this to the HTTP Management API via REST.