Since the beginning of the year, several vulnerabilities have been discovered in the Linux Kernel as well as in others important and widely-used packages. Among them, there was the famous CVE-2021-3156 affecting the sudo package and allowing any unprivileged user to gain root privileges. This one had a base score of 7.8, which is considered as high.
This kind of events demonstrate the importance of having a strong patching strategy to ensure up-to-date softwares and operating systems (even when it’s Linux 😎 ).

When your inventory is composed of few servers, you can easily and quickly patch all of them manually using “dnf/yum update” for RHEL family OS, “apt update” for Debian based OS or with “zypper update” for SUSE servers.
But when you have to update dozens of machines, doing it manually could be time consuming… and boring.
For Enterprise distributions, there are tools that can help you (Red Hat Satellite, SUSE Manager, aso.). But if you want to go with an open source solution, you can use Ansible to create playbooks and roles to patch your servers automatically.

The goal of this blog is not to explain what Ansible is and how does it work. If you have no knowledge on Ansible, it would be better to start by reading the official documentation, which is written in a very simple way and contains a lot of useful exemples.

Playbook

The content of my playbook file is very limited. Its purpose is only to call a role :
---
- name: OS update
  hosts: dev
  gather_facts: yes
  tasks:
    - name: OS update - all packages or security fixes only
      include_role:
        name: os_update 
...

 

Role

The main task of my role is also very short :
---
- include_tasks: redhat.yml
  when: ansible_os_family == "RedHat"
...
Its aim is to call the correct task file depending which operating system the playbook is running for.
The role gives the user the possibility to choose between a full patching or to apply security fixes only (thanks to an extra-var “ev_security_only” you’ll see below). Nothing else in this file, as all the steps are described in the task.

 

Task

The first step is to list all packages that will be modified. Thus, the user can double-check what will be installed before moving on, or cancel the process if desired :
---
- name: Get packages that can be upgraded
  become: yes
  ansible.builtin.dnf:
    list: upgrades
    state: latest
    update_cache: yes 
  register: reg_dnf_output_all
  when: ev_security_only == "no" 

- name: List packages that can be upgraded
  ansible.builtin.debug: 
    msg: "{{ reg_dnf_output_all.results | map(attribute='name') | list }}"
  when: ev_security_only == "no" 


- name: Get packages that can be patched with security fixes
  become: yes
  ansible.builtin.dnf:
    security: yes
    list: updates
    state: latest
    update_cache: yes
  register: reg_dnf_output_secu
  when: ev_security_only == "yes"

- name: List packages that can be patched with security fixes
  ansible.builtin.debug: 
    msg: "{{ reg_dnf_output_secu.results | map(attribute='name') | list }}"
  when: ev_security_only == "yes" 


- name: Request user confirmation
  ansible.builtin.pause:
    prompt: | 

      The packages listed above will be upgraded. Do you want to continue ? 
      -> Press RETURN to continue.
      -> Press Ctrl+c and then "a" to abort.
  when: reg_dnf_output_all is defined or reg_dnf_output_secu is defined

 

Next step is to start the full upgrade or to install the security fixes only :
- name: Upgrade packages
  become: yes
  ansible.builtin.dnf:
    name: '*'
    state: latest
    update_cache: yes
    update_only: no
  register: reg_upgrade_ok
  when: ev_security_only == "no" and reg_dnf_output_all is defined

- name: Patch packages
  become: yes
  ansible.builtin.dnf:
    name: '*'
    security: yes
    state: latest
    update_cache: yes
    update_only: no
  register: reg_upgrade_ok
  when: ev_security_only == "yes" and reg_dnf_output_secu is defined


- name: Print errors if upgrade failed
  ansible.builtin.debug:
    msg: "Packages upgrade failed"
  when: reg_upgrade_ok is not defined

 

If the Kernel has been updated, it’s strongly recommended to reboot the server. To check if a reboot is required, we can use the command “needs-restarting” provided here by the package dnf-utils :
- name: Install dnf-utils
  become: yes
  ansible.builtin.dnf:
    name: 'dnf-utils'
    state: latest
    update_cache: yes

- name: Check if a reboot is required
  become: yes
  command: needs-restarting -r
  register: reg_reboot_required
  ignore_errors: yes
  failed_when: false
  changed_when: reg_reboot_required.rc != 0
  notify:
    - Reboot server 
...

 

Handler

As you can see above, an Handler is called by the “notify” directive.  Using an Handler is very suitable here, as we want to reboot the server once the patching is done, but only when the Kernel has been updated.
---
- name : Reboot server
  ansible.builtin.reboot:
    msg: "Reboot initiated by Ansible after OS update"
    reboot_timeout: 3600
    test_command: uptime
...

 

Execution

Let’s start the playbook to perform a full update (ev_security_only=no) on a single host :
$ ansible-playbook playbooks/os_update.yml --extra-vars "ev_security_only=no"

PLAY [OS update] *****************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************************
ok: [192.168.22.101]

TASK [OS update - all packages or security fixes only] ***************************************************************************************************************************************************************************************

TASK [os_update : include_tasks] *************************************************************************************************************************************************************************************************************
included: .../roles/os_update/tasks/redhat.yml for 192.168.22.101

TASK [os_update : Get packages that can be upgraded] *****************************************************************************************************************************************************************************************
ok: [192.168.22.101]

TASK [os_update : List packages that can be upgraded] ****************************************************************************************************************************************************************************************
ok: [192.168.22.101] => {
"changed": false,
"msg": [
"tuned",
"hwdata",
"libstdc++",
"libgomp",
"libgcc",
"openssl-libs",
"openssl",
"tuned",
"hwdata",
"python36",
"libstdc++-devel",
"tuned"
]
}

TASK [os_update : Get packages that can be patched with security fixes] *****************************************************************************************************************************
skipping: [192.168.22.101]

TASK [os_update : List packages that can be patched with security fixes] *********************************************************************************************************************************************************************
skipping: [192.168.22.101]

TASK [os_update : Request user confirmation] *************************************************************************************************************************************************************************************************
[os_update : Request user confirmation]

The packages listed above will be upgraded. Do you want to continue ?
-> Press RETURN to continue.
-> Press Ctrl+c and then "a" to abort.
:
ok: [192.168.22.101]

TASK [os_update : Upgrade packages] **********************************************************************************************************************************************************************************************************
changed: [192.168.22.101]

TASK [os_update : Patch packages] ************************************************************************************************************************************************************************************************************
skipping: [192.168.22.101]

TASK [os_update : Print errors if upgrade failed] ********************************************************************************************************************************************************************************************
skipping: [192.168.22.101]

TASK [os_update : Install dnf-utils] *********************************************************************************************************************************************************************************************************
ok: [192.168.22.101]

TASK [os_update : Check if a reboot is required] *********************************************************************************************************************************************************************************************
ok: [192.168.22.101]

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************
192.168.22.101 : ok=8 changed=1 unreachable=0 failed=0 skipped=4 rescued=0 ignored=0

$


Possible improvements

At this stage, this role is rather basic. It could therefore benefit from some improvements, such as :

  • Create tasks to patch Debian based and SUSE servers
  • Add pre-task to send the list of packages by email before starting the patching
  • Logging
  • ….

Perhaps in a next blog…

Post Views: 13,753
Thumbnail [60x60]
by
Joël Cattin