dbi Blog

Reduce the output

If you do a loop in Ansible, it always prints the item variable for each loop. “item” is the current list element. If you are looping over a list with small values, then the output is OK.

    - set_fact:
        myList: [ "a", "b", "c", "d" ]

   - name: simple loop
      debug:
        msg: "value of item is {{ item }}."
      with_items: "{{myList}}"
      register: loop_result

TASK [simple loop] ***********************************************
ok: [localhost] => (item=a) => {
    "msg": "value of item is a."
}
ok: [localhost] => (item=b) => {
    "msg": "value of item is b."
}
ok: [localhost] => (item=c) => {
    "msg": "value of item is c."
}
ok: [localhost] => (item=d) => {
    "msg": "value of item is d."
}

But if the loop variable is a list of dict, especially with many or big values, then the log-output of Ansible gets very unreadable.

Example: you have to deploy more than one trusted certificate to an Oracle wallet (a PKCS#12 container file). The size of a certificate is usually 1-4 KB. That means, if you deploy 4 certificates à 3KB, your screen is spammed by approx. 12’000 characters. And I think you are not interested in the certificate content. Isn’t it?

Variable trusted_certs (certificate shortened for better readability)

       trusted_certs:
          - { issuer: "letsencrypt",
              certificate: "-----BEGIN CERTIFICATE-----\nMIIFBjCCAu6gAwIBAgI....MEZSa8DA\n-----END CERTIFICATE-----" }
          - { issuer: "verisign",
              certificate: "-----BEGIN CERTIFICATE-----\nMIICkDCCAXgCAQIwHDq....kMJVW2X=\n-----END CERTIFICATE-----" }

With the following task we add the certificate to the Oracle wallet

    - name: Add trusted certificates to wallet
      ansible.builtin.shell:  |
        echo "{{ item.certificate }}" > {{ item.issuer }}.cert.pem
        orapki wallet add -wallet . -cert {{ item.issuer }}.cert.pem -trusted_cert -pwd {{ walletpw }}
      with_items: "{{trusted_certs}}"

But how we can get rid of this annoying output?

One possibility is to use the no_log: true option. Then, item will be replaced by “None” in the output.

- name: Add trusted certificates to wallet
  ansible.builtin.shell:  |
    echo "{{ item.certificate }}" &gt; {{ item.issuer }}.cert.pem
    orapki wallet add -wallet . -cert {{ item.issuer }}.cert.pem -trusted_cert -pwd {{ walletpw }}
  with_items: "{{trusted_certs}}"
  no_log: true

Nice 🙂 But in case of errors, we have a problem to see what went wrong 🙁

Another approach is: Instead of looping over the certificate list, create a list with the index numbers of your certificate list.

    - name: Add trusted certificates to wallet
      ansible.builtin.shell:  |
        echo "{{ trusted_certs[item|int].certificate }}" > {{ trusted_certs[item|int].issuer }}.cert.pem
        orapki wallet add -wallet . -cert {{ trusted_certs[item|int].issuer }}.cert.pem -trusted_cert -pwd {{ walletpw }}
      with_sequence: start=0 end={{ trusted_certs|length-1 }}

We will create a sequence to identify the list elements using with_sequence: The 1st list element is [0] (start=0) and the last one is the number of elements -1 (`end={{ trusted_certs|length-1 }}`).

For example, to access the 1st certificate (index number 0) we can use {{ trusted_certs[0].certificate }}. In the loop we replace the [0] by [index|int] to access all elements. Caution! Ansible is not able to recognize the sequence as numbers and treats it as string, so you have to explicitly convert it to int.

And now, the output is very compact and readable 🙂

Replace loops

Loops are easy to implement and to understand. It is nice for a few list elements, but if you have thousands of elements, then you get a big amount of output, and it is slow.

If you want to see what certificates are added (only the issuer field), you can do it similar to the task to add certificates:

    - name: Show trusted certificates to add
      ansible.builtin.debug:  var=trusted_certs[item|int].certificate
      with_sequence: start=0 end={{ trusted_certs|length-1 }}

Or you can use json_query

    - name: Show trusted certificates to add (with json_query)
      ansible.builtin.debug:  var=trusted_certs|json_query('[].issuer')

json_query: [] is the filter criteria. In this case, we will not filter the list elements. And .issuer means, to only return the issuer field and not the certificate field.

If you run a playbook for many servers (e.g. hosts: [dbservers], a group with all database hosts) and you need information for the hosts from an ldap directory (or a REST web-service), then it is often faster to get once the whole data and then to extract the information for the hosts with json_query, instead of querying each host individually in ldap.

Example: If you have to know all databases on a server and you have stored all the connect strings (orclNetDescString) in an ldap directory, then you can get the information from there. For simplicity, we assume that:

orclNetDescString: contains no spaces, all keywords are in uppercase and HOST=...contains the {{inventory_hostname}}
cn: contains the database/instance name

    - name: "get databases on the server"
      community.general.ldap_search:
        server_uri: "{{ ldap_server }}"
        bind_dn: ""
        dn: "cn=OracleContext,dc=domain,dc=com"
        filter: "(&(orclNetDescString=*HOST*{{inventory_hostname}})(objectclass=orclNetService))"
        scope: "children"
        attrs:
          - cn
          - orclNetDescString
        register: ldap1
    - debug: var=ldap1.results

It will return all databases for this host, in this example exactly one:

ldap1.results: [
  {
    "cn": "DB1"
    "dn": "cn=DB1,cn=oracleContext,dc=domain,dc=com",
    "orclNetDescString": "(DESCRIPTION =(ADDRESS=(PROTOCOL=TCP)(HOST=srv01)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=DB01.domain.com)))"
  }
]

You have to do that for the [dbservers] hosts. If you have 1000 database servers, you will run this task 1000 times.

Instead of querying LDAP for each hostname, we can query it for all hosts, and then extract the required data from the result for the current host with json_query.

    - name: "get databases of all servers"
      run_once: true
      community.general.ldap_search:
        server_uri: "{{ ldap_server }}"
        bind_dn: ""
        dn: "cn=OracleContext,dc=domain,dc=com"
        filter: "(objectclass=orclNetService)"
        scope: "children"
        attrs:
          - cn
          - orclNetDescString
      register: ldap2

With run_once it will only be executed once on the 1st host, but the result is available for all hosts. All the hosts then can query this result to get the records of the own host:

- name: show all databases on this host
  debug: var=ldap2.results|json_query(jquery)
  vars:
    jquery: "[? contains(orclNetDescString, `HOST={{inventory_hostname}}`)].cn"

Summary

The output of a loop (loop, with_items, …) is annoying if the list item contains many or big values. To reduce the output you can:

use the no_log: true option. Then the item-values will no longer be printed. But in case of an error, you will no longer get the error-message.

Use the index of the list-elements. In the output you will see the index-number, and in case of errors, you will also see the error-message.

For filtering data, it is more efficient to use json_query instead of looping over all list items.

If you get data from an LDAP directory or a Webservice, try to fetch once all data and then to extract from this result the host-specific data with json_query.

Post Views: 12