We all know the pain of an “enforced” SELinux that is overly suspicious and blocks our applications. Typically, when issues arise, we tend to search for solutions everywhere except within the /var/log/audit/audit.log or /var/log/audit directories. Within the audit logs, we might discover that SELinux is actively preventing our applications from running. Some people opt to set SELinux to “Permissive” mode and consider the issue resolved. I, too, was once that kind of engineer.
However, after working with numerous high-security clients, such as banks and other financial institutions, the necessity of SELinux has become indisputable. If we examine the OVAL or CIS Security scan, we’ll find a check that ensures SELinux is not only active but also set to “enforcing” mode.
So thanks to the fact that we can’t life without it anymore, it’s worth taking the time to get better acquainted with it. That’s the focus of this blog post.
Since when is SELinux available?
Good question… SELinux has been around since the 2000s and was officially introduced to the open-source community during that same period. Notably, it became an integral part of the new Linux Upstream Kernel in 2003, marking its availability for widespread usage. You might be thinking, “But I only became aware of it in CentOS/RHEL/OL 7 and later versions… How can it have been in existence since the 2000s?” That was precisely my reaction when I first learned about the release date of SELinux. I too, couldn’t recall a time when I had to disable or configure it before CentOS 7.
The reason is straightforward: SELinux defaulted to being disabled or set to ‘permissive’ until and including CentOS/RHEL/OL 6. As Linux distributions evolved and security became more critical, there was a gradual shift towards enabling SELinux by default. CentOS/ RHEL/ OL 7 and later versions, including RHEL 8, do enable SELinux by default as part of their security strategy.
How does SELinux work (simplified)?
SELinux, or Security-Enhanced Linux, is like an extra guard for your computer. It works by adding an extra layer of security to the heart of the system, the kernel. It’s like a bouncer at a club. When something tries to access a file, SELinux double-checks if it’s allowed, even if the regular security measures say it’s okay.
Here’s how it works: When a program or process wants to open a file, SELinux steps in. It looks at its list of who’s allowed to do what (the Access Vector Cache or AVC), kind of like a guest list at a party. It checks if the program is on the list for that file. If it is, great, the door’s open. If not, SELinux says, “Sorry, you can’t come in.” This extra layer of security makes your computer a lot safer.
How to analyse the SELinux context?
Now, onto my favorite part of this blog post: The action.
Without further ado, let’s dive into the commands necessary for checking and listing all SELinux security contexts:
# To list the SELinux context of a specific file/ folder you can execute:
$ ls -Z /u01/test/file1
# To view the SELinux context of a specific process:
$ ps -Z <PID>
# To view the SELinux context of network services:
$ netstat -Z
# This command displays the SELinux user and role for the current user (or a specific with UID/ username)
$ id -Z <username>
# Copy and preserve all SELinux context definitions of folder/ files:
$ cp -rZ /u01/test/folder1
With these commands one can check the SELinux setup and furthermore find out what blocks an application from running. Important to note here is that, in any case there is some unknown error blocking an service from running: The first place to check for the issue is inside the audit logs:
# View the audit log in RPM-Based OS:
$ less /var/log/audit/audit.log
# SELinux denials will be logged inside the system-logs as well
$ less /var/log/messages
$ less /var/log/syslog
If you have the tool “setroubleshoot” installed, you can check for further logs inside “/var/lib/setroubleshoot/setroubleshoot_database.xml”.
SELinux blocking DBVisit
I’m using DBVisit to give you an example on how to use SELinux and how to configure it. Thanks to the fact that dbi-services and DBVisit are partners, it might be helpful to find out what it needs for DBVisit to run with SELinux. To make sure we do not have to know SELinux by heart, we can use the amazing tool: “audit2allow”. This tool can make it much easier and faster to find out what application needs what kind of SELinux context.
In my case our senior DBA Marc Wagner installed the DBVisit on our new OL8 server. After starting the DBVisit service Marc noticed that SELinux is blocking it from starting properly. This meant that there has to be some errors inside the audit.log:
# Check the audit.log and grep for "avc:" which is usually used for a SELinux denial:
$ grep "avc:" /var/log/audit/audit.log
# The result was expected and we found many dbvisit errors like:
type=AVC msg=audit(1694780570.100:132): avc: denied { execute } for pid=3019 comm="(tmanager)" name="dbvagentmanager" dev="dm-6" ino=33554769 scontext=system_u:system_r:init_t:s0 tcontext=unconfined_u:object_r:default_t:s0 tclass=file permissive=1
type=AVC msg=audit(1694780570.160:139): avc: denied { map } for pid=3019 comm="dbvagentmanager" path="/u01/app/dbvisit/standbymp/db/AgentManager.db-shm" dev="dm-6" ino=100678254 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file permissive=1
...
Sadly there where 150 lines full of errors concerning our DBVisit service. Changing each configuration file/ executable/ binary file by hand with “semanage” would take ages. And that is where the command “allow2audit” enters the game.
Is “audit2allow” really that amazing?
Yes. I will explain why.
First of all, here is the man-page definition of the command “allow2audit”:
“
audit2allow
– generate SELinux policy allow rules from logs of denied operations” – ManPage
How does “audit2allow” work? When SELinux denies an execution or access for a process, it writes down all the information regarding the error inside the audit.log. These information can be used afterwards with “audit2allow” to generate new SELinux context for the errors.
# Generate a human readable (-w) description of why access was denied. The -a option causes all audit logs to be read:
$ audit2allow -w -a
type=AVC msg=audit(1695294455.597:278): avc: denied { open } for pid=1429 comm="dbvagentmanager" path="/u01/app/dbvisit/standbymp/db/dbvagentmanager.pid" dev="dm-6" ino=100678250 scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=file permissive=1
Was caused by:
Missing type enforcement (TE) allow rule.
You can use audit2allow to generate a loadable module to allow this access.
...
# We can now check what security context would need to be in place to solve the given issue inside audit.log:
$ audit2allow -a
#============= init_t ==============
allow init_t default_t:file { append create execute execute_no_trans ioctl link lock map open read unlink write };
...
After we executed the command “audit2allow -a” we get the correct context which we can use to correct the deny rules. To make it usable by SELinux we need to create a “.pp” file (also known as policy package) out of these. Before we can get a “.pp” file we need to have a “.te” file (also known as type enforcement file) which will be compiled into a “.pp” file. After compilation we can easily set the new context like so:
# Create a .te file with -M <filename> which will generate a .pp file:
$ audit2allow -a -M policyname
# After that we will have two files in our current location, one ending with .te and the other with .pp. These files can be used to install the new modules that ausit2allow created:
$ semodule -i policyname.pp
How to define new SELinux context?
It is rather easy to solve issues with “audit2allow”, but there is an easy manual way of doing that. In my example I had a “etcd” service that was manually installed and which was blocked by my SELinux. Using our “audit2allow” would have been a successful solution too, but in that case there is a much smarter and smaller solution. We can just add and remove context definitions with the command “semanage”:
# Define the etcd.service file as an executable file and add a new SELinux security context:
$ sudo semanage fcontext -a -t bin_t /etc/etcd/etcd.service
# Define the etcd.conf file as a etc config file so the service can access it:
$ sudo semanage fcontext -a -t etc_t /etc/etcd/etcd.conf
Here is a list of the most used SELinux security context types:
- user_u: Represents the user context. Each user is assigned a unique user context.
- system_u: Represents the system context. It’s typically used for system processes and files.
- root:root: Denotes a file or process owned by the root user and the root group. This is the default context for many system files.
- user_home_t: Used for user home directories.
- user_tmp_t: Used for user-specific temporary directories.
- bin_t: Represents binary executables, including most command-line utilities.
- sbin_t: Represents system binary executables, often related to system administration.
- etc_t: Represents configuration files and directories under
/etc
. - var_t: Represents variable data files under
/var
. - usr_t: Represents user-specific data under
/usr
. - lib_t: Represents library files under
/lib
. - log_t: Represents log files and directories.
- dev_t: Represents device files.
- home_root_t: Represents the root directory of user home directories.
- tmp_t: Represents system-wide temporary directories such as
/tmp
. - proc_t: Represents
/proc
and process-related information. - init_t: Represents the init process.
- unconfined_t: Denotes processes and files that are not confined by SELinux policies.
As we can see, there is almost for every file a specified security context. This makes SELinux a unreachable security wall inside every Linux machine (when configured correctly).
SELinux does not check for changes until we execute the command “restorecon” on the folder/ files that have been changed.
# To refresh the SELinux security context of changed files you can run:
$ sudo restorecon -R -v /etc/etcd/*
Summary
Setting up a system with numerous services can be challenging, especially when SELinux is enforced. Fortunately, “audit2allow” allows us to generate SELinux definitions for all services and files simultaneously. This streamlines the process of configuring your server with the necessary permissions, as you can grant access to all of them in one go.
There are still many more topics to explore in the realm of SELinux. However, for most SELinux issues, this blog should equip you with the skills to analyze and resolve them. If you’d like to deep dive into subjects like SELinux booleans or SELinux networking, be sure to explore the following two blogs:
- SELinux boolean
- https://access.redhat.com/documentation/de-de/red_hat_enterprise_linux/7/html/selinux_users_and_administrators_guide/sect-security-enhanced_linux-working_with_selinux-booleans
- SELinux networking
- https://wiki.gentoo.org/wiki/SELinux/Networking
- SELinux cheat sheet
- https://opensource.com/sites/default/files/gated-content/cheat_sheet_selinux_v2.pdf