By Mouhamadou Diaw

In this blog we are going to install an Oracle Restart 19c with ASM Filter Driver (AFD). I am using following configuration
-Oracle 19c
-Oracle Linux Server 7.6
-Kernel 4.14.35-1902.2.0.el7uek.x86_64

Once the Oracle Restart installed, I will create an ACFS filesystem

We will have to configure following disks for AFD
-/dev/sdc
-/dev/sdd

The Oracle downloaded software was unzipped to the GRID_HOME

1
[oracle@oraadserver u01]$ unzip -d /u01/app/19.0.0.0/grid LINUX.X64_193000_grid_home.zip

After setting the variables ORACLE_HOME and ORACLE_BASE, we then use with user root the command asmcmd afd_label to provision disk devices for use with Oracle ASM Filter Driver.

1
2
3
4
[root@oraadserver dev]# export ORACLE_HOME=/u01/app/19.0.0.0/grid
[root@oraadserver dev]# export ORACLE_BASE=/tmp
[root@oraadserver dev]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_label DATA /dev/sdc --init
[root@oraadserver dev]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_label CRS /dev/sdd --init

We can verify the status of the disks with the command afd_lslbl

1
2
3
4
5
6
7
8
9
10
11
[root@oraadserver ~]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdc
--------------------------------------------------------------------------------
Label                     Duplicate  Path
================================================================================
DATA                                  /dev/sdc
[root@oraadserver ~]# /u01/app/19.0.0.0/grid/bin/asmcmd afd_lslbl /dev/sdd
--------------------------------------------------------------------------------
Label                     Duplicate  Path
================================================================================
CRS                                   /dev/sdd
[root@oraadserver ~]#

In my case with the kernel 4.14.35-1902.2.0.el7uek.x86_64, I had to apply the following
Patch 27494830: BUILD UEK5U2 COMPATIBLE ACFS GRID KERNEL MODULES
To avoid this error during the installation

1
2
Action - To proceed, do not specify or select the Oracle ASM Filter Driver option.  Additional Information:
 - AFD-620: AFD is not supported on this operating system version: '4.14.35-1902.2.0.el7uek.x86_64'

The patch was applied before the installation as described in the following documentation
How to Apply a Grid Infrastructure Patch Before Grid Infrastructure Configuration (before root.sh or rootupgrade.sh or gridsetup.bat) is Executed (Doc ID 1410202.1)
The opatch version was verified

1
2
3
4
5
[oracle@oraadserver u01]$ /u01/app/19.0.0.0/grid/OPatch/opatch version
OPatch Version: 12.2.0.1.17
OPatch succeeded.
[oracle@oraadserver u01]$

After the patch was unpacked

1
[oracle@oraadserver u01]$ unzip p27494830_193000ACFSRU_Linux-x86-64.zip

And then the patch was applied

1
2
3
4
5
[oracle@oraadserver u01]$ /u01/app/19.0.0.0/grid/gridSetup.sh -silent -applyRU /u01/27494830/
Preparing the home to patch...
Applying the patch /u01/27494830/...
Successfully applied the patch.
The log can be found at: /tmp/GridSetupActions2020-10-20_09-32-54PM/installerPatchActions_2020-10-20_09-32-54PM.log

And now we lunch the install. Note that only some screenshots will be shown. And for the non shown pictures the default was kept.

1
2
3
[oracle@oraadserver grid]$ pwd
/u01/app/19.0.0.0/grid
[oracle@oraadserver grid]$ ./gridSetup.sh

For the ASM diskgroups configuration

The groups of my installation

I got some warnings that Idecided to ignore (just a test environment)



The two scripts are executed

1
2
3
4
5
6
7
8
[root@oraadserver oraInventory]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@oraadserver oraInventory]#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[root@oraadserver oraInventory]# /u01/app/19.0.0.0/grid/root.sh
Performing root user operation.
The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.0.0.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/19.0.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/oraadserver/crsconfig/roothas_2020-10-20_09-58-51PM.log
2020/10/20 21:58:55 CLSRSC-363: User ignored prerequisites during installation
LOCAL ADD MODE
Creating OCR keys for user 'oracle', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node oraadserver successfully pinned.
2020/10/20 22:00:43 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
oraadserver     2020/10/20 22:03:14     /u01/app/oracle/crsdata/oraadserver/olr/backup_20201020_220314.olr     2451816761
2020/10/20 22:03:49 CLSRSC-327: Successfully configured Oracle Restart for a standalone server
[root@oraadserver oraInventory]#


After the installation we can validate that the ASM was up

1
2
3
4
5
6
[root@oraadserver oraInventory]# crsctl check has
CRS-4638: Oracle High Availability Services is online
[root@oraadserver oraInventory]# ps -ef | grep pmon
root     10153  1730  0 22:10 pts/0    00:00:00 grep --color=auto pmon
oracle   26037     1  0 22:05 ?        00:00:00 asm_pmon_+ASM
[root@oraadserver oraInventory]#

Now using asmca, let’s configure an ACFS filesystem . Basically we
-Create a CRS diskgroup
-In CRS diskgroup we create a volume
-In this volume we create an ACFS filesystem and mount it

1
2
3
[oracle@oraadserver ~]$ which asmca
/u01/app/19.0.0.0/grid/bin/asmca
[oracle@oraadserver ~]$ asmca

The ACFS filesystem is now mounted

1
2
3
4
[root@oraadserver ~]# df -h /share_acfs/
Filesystem           Size  Used Avail Use% Mounted on
/dev/asm/acfs_vol-3  5.0G  319M  4.7G   7% /share_acfs
[root@oraadserver ~]#

Now let’s reboot the system and verify that the acfs filesystem is mounted.

1
2
3
4
5
6
7
8
9
10
[root@oraadserver ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             1.8G  8.0K  1.8G   1% /dev
tmpfs                1.8G  637M  1.2G  35% /dev/shm
tmpfs                1.8G  8.8M  1.8G   1% /run
tmpfs                1.8G     0  1.8G   0% /sys/fs/cgroup
/dev/mapper/ol-root   47G   14G   34G  29% /
/dev/sda1            497M  230M  268M  47% /boot
tmpfs                368M     0  368M   0% /run/user/54323
tmpfs                368M     0  368M   0% /run/user/0

No. When trying to manually mount it, I got this error

1
2
3
4
5
6
[root@oraadserver ~]# /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs
mount.acfs: CLSU-00107: operating system function: open64; failed with error data: 2; at location: OOF_1
mount.acfs: CLSU-00101: operating system error message: No such file or directory
mount.acfs: CLSU-00104: additional error information: open64 (/dev/ofsctl)
mount.acfs: ACFS-00502: Failed to communicate with the ACFS driver.  Verify the ACFS driver has been loaded.
[root@oraadserver ~]#

I then load the acfs drivers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@oraadserver ~]#  /u01/app/19.0.0.0/grid/bin/acfsload start -s
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9309: ADVM/ACFS installation correctness verified.
[root@oraadserver ~]#

When checking the volume, it was disabled

1
2
3
4
5
6
7
8
9
10
11
12
13
ASMCMD> volinfo --all
Diskgroup Name: CRS
         Volume Name: ACFS_VOL
         Volume Device: /dev/asm/acfs_vol-3
         State: DISABLED
         Size (MB): 5120
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage: ACFS
         Mountpath: /share_acfs

So I enable enable it

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
ASMCMD> volenable –all
ASMCMD> volinfo --all
Diskgroup Name: CRS
         Volume Name: ACFS_VOL
         Volume Device: /dev/asm/acfs_vol-3
         State: ENABLED
         Size (MB): 5120
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage: ACFS
         Mountpath: /share_acfs
ASMCMD>

And then I run the mount command

1
2
3
4
5
6
[root@oraadserver trace]# /bin/mount -t acfs /dev/asm/acfs_vol-3 /share_acfs
[root@oraadserver trace]# df -h /share_acfs/
Filesystem           Size  Used Avail Use% Mounted on
/dev/asm/acfs_vol-3  5.0G  319M  4.7G   7% /share_acfs
[root@oraadserver trace]#

It was strange to manually mount the ACFS filesystem at system reboot but it is mentioned in Oracle documentation
Starting with Oracle Database 12c, Oracle Restart configurations do not support the Oracle ACFS registry.
This means that the Oracle ACFS file system resource is supported only for Oracle Grid Infrastructure cluster configurations; it is not supported for Oracle Restart configurations. When using ACFS on Oracle Restart, we have to mount the filesystem manually mount and umount the ACFS filesystem

Conclusion

When using ACFS with Oracle Restart, we have to manually mount and umount the ACFS filesystem when the system stops or starts. This can be done by some customized scripts.

Post Views: 5,221
Thumbnail [60x60]
by
Oracle Team