Introduction

Some of us are using CIFS mountpoints on the ODA. This is mostly for sharing files with Windows application servers. On ODA, as on other Linux setup, it works like a charm. But it ends working starting from patch 19.11. Now, it’s no more possible to use these kind of mount.

What is CIFS?

CIFS (Common Internet File System) also called SMB (Server Message Block) is a file sharing protocol created by Microsoft a long time ago. It was implemented for Linux using reverse engineering under the name Samba. It’s still updated more or less frequently, and it has been known for some security issues over the time. CIFS relies on user/password authentication, most often as clear text in /etc/fstab.

CIFS is only used when sharing files from Windows to Linux or vice-versa. Sharing files between Linux servers is much more common. For this purpose, NFS protocol (Network File System), an open standard created by Sun Microsystems (now part of Oracle), is broadly used. One of the main difference compared to CIFS is that is does not rely on authentication but on user and group ids. NFS server explicitely decides which client is able to connect, and both machines are supposed to use the same ids for users and groups.

Error mounting CIFS shares on ODA >= 19.11

When mounting a CIFS share on your ODA >= 19.11, the following error will raise:

mkdir /WinShare
echo "//10.36.0.250/winshare /WinShare cifs user=winuser,password=* 0 0" >> /etc/fstab
mount -a
mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Message is not that clear, and after investigating I found that this may due to FIPS being enabled on my ODA (recently patched from 19.9 to 19.14).

What is FIPS?

FIPS (Federal Information Processing Standards) is a new security standard from the USA. It prevents security flaws on a system, thus making a system more secure. One of the features is to disallow unsecured authentications like NTLM, NTLMv2 and NTLMSSP which are CIFS standards.

How to solve this problem?

There is multiple ways of dealing with this problem. From the best to the worst.

Configuring a Windows NFS share

This first solution is the best one in my opinion: use NFS instead of CIFS. You may say that NFS is not compatible with Windows, but it’s not true anymore. Starting from Windows 2008, this OS is able to create a NFS share very easily. And this is definitely much cleaner because Microsoft implemented an open standard. I would recommend using Windows 2012 or later for NFS v4.1 support. Here is a blog post I would recommend if you need to configure a NFS share on Windows Server 2016.

Using Kerberos on top of CIFS

Using CIFS is not the problem for FIPS, the problem is using basic authentication method with login/password being sent through the network. Kerberos is a much more elaborated mechanism based on keys. But it needs a more complex setup. Thankfully, Active Directory can act as a Kerberos server. I would love to test it but for now I don’t have the adequate lab environment for that.

Disabling FIPS

This is definitely not recommended. If you choose ODA, you must accept system changes including the inclusion of FIPS protocol. If you don’t accept these changes, consider using old versions of ODA patches, but it’s also not recommended.

I found the method to disable FIPS in this blog post.

First of all, check if FIPS is enabled on your system (should be true if your ODA runs 19.11 or later patch):

cat /proc/sys/crypto/fips_enabled
1

FIPS is configured as a kernel option in /etc/grub.conf. You first need to remove some packages, do a backup of the initramfs, generate a new initramfs (dracut -f) modify the grub options, recompile the grub config file and reboot the server (to be done on each node with an HA ODA):

yum remove dracut-fips*
Loaded plugins: langpacks, priorities, ulninfo, versionlock
Resolving Dependencies
--> Running transaction check
---> Package dracut-fips.x86_64 0:033-572.0.9.el7 will be erased
---> Package dracut-fips-aesni.x86_64 0:033-572.0.9.el7 will be erased
--> Finished Dependency Resolution
ol7_UEKR6/x86_64 | 3.0 kB 00:00:00
ol7_UEKR6/x86_64/updateinfo | 507 kB 00:00:00
ol7_UEKR6/x86_64/primary_db | 40 MB 00:00:00
ol7_latest/x86_64 | 3.6 kB 00:00:00
ol7_latest/x86_64/group_gz | 136 kB 00:00:00
ol7_latest/x86_64/updateinfo | 3.4 MB 00:00:00
ol7_latest/x86_64/primary_db | 40 MB 00:00:00
Dependencies Resolved
===========================================================================================================================
Package Arch Version Repository Size
Removing:
dracut-fips x86_64 033-572.0.9.el7 @OSPatchBaseRepo 8.1 k
dracut-fips-aesni x86_64 033-572.0.9.el7 @OSPatchBaseRepo 18 k
Transaction Summary
Remove 2 Packages
Installed size: 26 k
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
** Found 3 pre-existing rpmdb problem(s), 'yum check' output follows:
oda-hw-mgmt-19.15.0.0.0_LINUX.X64_220530-1.x86_64 has missing requires of perl(GridDefParams)
oda-hw-mgmt-19.15.0.0.0_LINUX.X64_220530-1.x86_64 has missing requires of perl(s_GridSteps)
perl-RPC-XML-0.78-3.el7.noarch has missing requires of perl(DateTime::Format::ISO8601) >= ('0', '0.07', None)
Erasing : dracut-fips-aesni-033-572.0.9.el7.x86_64 1/2
Erasing : dracut-fips-033-572.0.9.el7.x86_64 2/2
Verifying : dracut-fips-033-572.0.9.el7.x86_64 1/2
Verifying : dracut-fips-aesni-033-572.0.9.el7.x86_64 2/2
Removed:
dracut-fips.x86_64 0:033-572.0.9.el7 dracut-fips-aesni.x86_64 0:033-572.0.9.el7
Complete!
cp -p /boot/initramfs-$(uname -r).img /opt/dbi/initramfs-$(uname -r).with_fips
dracut -f
vi /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="pci=noaer loglevel=3 panic=60 transparent_hugepage=never ipv6.disable=1 intel_idle.max_cstate=1 nofloppy numa=on console=ttyS0,115200n8 console=tty0 crashkernel=256M@64M rd.lvm.lv=VolGroupSys/LogVolRoot rd.md.uuid=10e67471:4b600fe1:d970d513:c635edf6 rd.md.uuid=1e334e65:aea7e87e:516f7dfe:8d79ccfb rd.lvm.lv=VolGroupSys/LogVolSwap biosdevname=1 boot=UUID=9cb4c7c1-e87a-4ae1-9c22-fcc5e3460ce1 fips=0 nvme.nvme_io_queues=32 nvme_core.multipath=0"
GRUB_DISABLE_RECOVERY="true"
grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
Generating grub configuration file …
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
Found linux image: /boot/vmlinuz-4.14.35-2047.512.6.el7uek.x86_64
Found initrd image: /boot/initramfs-4.14.35-2047.512.6.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-4.14.35-2047.510.5.4.el7uek.x86_64
Found initrd image: /boot/initramfs-4.14.35-2047.510.5.4.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-4.14.35-2047.505.4.3.el7uek.x86_64
Found initrd image: /boot/initramfs-4.14.35-2047.505.4.3.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-b7d66bf7abc14b359559ec75c7415cbc
Found initrd image: /boot/initramfs-0-rescue-b7d66bf7abc14b359559ec75c7415cbc.img
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
WARNING: Ignoring duplicate config value: global_filter
done
shutdown -r now
mount -a
df -h /WinShare
Filesystem Size Used Avail Use% Mounted on
//10.36.0.250/winshare 100G 85G 15G 86% /WinShare
echo "It works now" > /WinShare/test.txt
cat /WinShare/test.txt
It works now

If you need to reverse to FIPS enabled mode, then it’s possible.

Revert to FIPS enabled mode

yum install dracut-fips*
Loaded plugins: langpacks, priorities, ulninfo, versionlock
Excluding 111 updates due to versionlock (use "yum versionlock status" to show them)
Resolving Dependencies
--> Running transaction check
---> Package dracut-fips.x86_64 0:033-572.0.9.el7 will be installed
---> Package dracut-fips-aesni.x86_64 0:033-572.0.9.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================================================
Package Arch Version Repository Size
Installing:
dracut-fips x86_64 033-572.0.9.el7 ol7_latest 64 k
dracut-fips-aesni x86_64 033-572.0.9.el7 ol7_latest 68 k
Transaction Summary
Install 2 Packages
Total download size: 132 k
Installed size: 26 k
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/dracut-fips-033-572.0.9.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Public key for dracut-fips-033-572.0.9.el7.x86_64.rpm is not installed
(1/2): dracut-fips-033-572.0.9.el7.x86_64.rpm | 64 kB 00:00:00
(2/2): dracut-fips-aesni-033-572.0.9.el7.x86_64.rpm | 68 kB 00:00:00
Total 300 kB/s | 132 kB 00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
Userid : "Oracle OSS group (Open Source Software group) [email protected]"
Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03
Package : 7:oraclelinux-release-7.9-1.0.9.el7.x86_64 (@anaconda/19.12)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : dracut-fips-033-572.0.9.el7.x86_64 1/2
Installing : dracut-fips-aesni-033-572.0.9.el7.x86_64 2/2
Verifying : dracut-fips-033-572.0.9.el7.x86_64 1/2
Verifying : dracut-fips-aesni-033-572.0.9.el7.x86_64 2/2
Installed:
dracut-fips.x86_64 0:033-572.0.9.el7 dracut-fips-aesni.x86_64 0:033-572.0.9.el7
Complete!
dracut -f
vi /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="pci=noaer loglevel=3 panic=60 transparent_hugepage=never ipv6.disable=1 intel_idle.max_cstate=1 nofloppy numa=on console=ttyS0,115200n8 console=tty0 crashkernel=256M@64M rd.lvm.lv=VolGroupSys/LogVolRoot rd.md.uuid=10e67471:4b600fe1:d970d513:c635edf6 rd.md.uuid=1e334e65:aea7e87e:516f7dfe:8d79ccfb rd.lvm.lv=VolGroupSys/LogVolSwap biosdevname=1 boot=UUID=9cb4c7c1-e87a-4ae1-9c22-fcc5e3460ce1 fips=1 nvme.nvme_io_queues=32 nvme_core.multipath=0"
GRUB_DISABLE_RECOVERY="true"
grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
shutdown -r now
cat /proc/sys/crypto/fips_enabled
1
mount -a
mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

Conclusion

Switching your CIFS shares to NFS is the best approach, even if it needs some work on the Windows side. But continuously increasing security level should also be a task for the DBA.