{"id":2515,"date":"2012-03-09T05:40:00","date_gmt":"2012-03-09T04:40:00","guid":{"rendered":"https:\/\/www.dbi-services.com\/blog\/configuration-of-tmpfs-on-sles-11-for-oracle-112-and-amm\/"},"modified":"2012-03-09T05:40:00","modified_gmt":"2012-03-09T04:40:00","slug":"configuration-of-tmpfs-on-sles-11-for-oracle-112-and-amm","status":"publish","type":"post","link":"https:\/\/www.dbi-services.com\/blog\/configuration-of-tmpfs-on-sles-11-for-oracle-112-and-amm\/","title":{"rendered":"Configuration of tmpfs on SLES 11 for Oracle 11.2 and AMM"},"content":{"rendered":"

Even if we (dbi services) usually advise to setup Oracle Enterprise Linux for Oracle deployments on Linux, it might appear that some customers still prefer other certified distributions such as SuSE Linux Enterprise Server (SLES). This post describes the configuration of the tmpfs filesystem size on SuSE Linux Enterprise Server 11.0 (SLES 11, without any service pack) in order to use the Oracle 11.2 Automatic Memory Management feature (AMM) through the Oracle instance parameter memory_target.<\/p>\n

In order to configure the MAA, let’s have a look at the documentation: http:\/\/download.oracle.com\/docs\/cd\/E11882_01\/install.112\/e24321\/pre_install.htm#autoId2<\/p>\n

We can see that the documentation focuses on OEL and Red Hat like systems which are slightly different on SuSE 11:
\nPer default on Linux, the tmpfs filesystem is sized to half of the server memory:<\/p>\n

server1:~ # free<\/span><\/span>
\ntotal\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 used\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 free\u00a0\u00a0\u00a0\u00a0 shared\u00a0\u00a0\u00a0 buffers\u00a0\u00a0\u00a0\u00a0 cached<\/span>
\nMem:\u00a0\u00a0\u00a0\u00a0\u00a0 74100380\u00a0\u00a0\u00a0\u00a0 510908\u00a0\u00a0 73589472\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 9024\u00a0\u00a0\u00a0\u00a0 124608<\/span><\/p>\n

server1:~ # df -h \/dev\/shm\/<\/span><\/span>
\nFilesystem\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Size\u00a0 Used Avail Use% Mounted on<\/span>
\nudev\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 36G\u00a0 128K\u00a0\u00a0 36G\u00a0\u00a0 1% \/dev<\/span><\/p>\n

However, for a pure Oracle server it might be worth to increase this size, for instance on a server having 72 GB of memory, 50 GB might be allocated to the Oracle database.<\/p>\n

For this purpose, the size of the tmpfs must be explicitly set in the \/etc\/fstab. If not, starting the database with a larger size (memory_target) will result in the following error messages at startup:<\/p>\n

ORA-00845: MEMORY_TARGET not supported on this system
\nORA-01078: Failure in processing system parameters<\/p>\n

On SLES 11, in order to increase the tmpfs, we need to proceed differently as on Red Hat. Per default, the tmpfs filesystem is mounted through the following command in \/etc\/init.d\/boot.localfs:<\/p>\n

mount -fv -t tmpfs udev \/dev<\/span><\/p>\n

This mount results in the following output:<\/p>\n

server1:~ # df -h | grep udev<\/span><\/span>
\nudev\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 36G\u00a0 128K\u00a0\u00a0 36G\u00a0\u00a0 1% \/dev<\/span><\/p>\n

In contrary to the Oracle documentation (Red-Hat focused), we do not mount a “shmfs” device but a “udev” device on SLES. In order to mount this tmpfs filesystem with another size than half of the memory, for instance 50 GB, we need to adapt the \/etc\/fstab:<\/p>\n

udev\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \/dev\/shm\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 tmpfs\u00a0\u00a0\u00a0\u00a0\u00a0 noauto,size=50g\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0 0<\/span><\/p>\n

Note that \/dev\/shm has to be explicitely used, since the Oracle kernel checks (hard coded) for a mount point called \/dev\/shm on the filesystem. A missing \/dev\/shm will result in the following error while starting Oracle:<\/p>\n

————————————–
\nWARNING: You are trying to use the MEMORY_TARGET feature. This feature requires the \/dev\/shm file system to be mounted for at least 10737418240 bytes. \/dev\/shm is either not mounted or is mounted with available space less than this size. Please fix this so that MEMORY_TARGET can work as expected. Current available is 0 and used is 0 bytes. Ensure that the mount point is \/dev\/shm for this directory.<\/span>
\nmemory_target needs larger \/dev\/shm<\/span>
\n————————————–<\/strong><\/p>\n

We prevent the automatic mount (through umount\/mount -a) with the “noauto” parameter (see the fstab entry).<\/p>\n

For an immediate mount, we can use the “mount \/dev\/shm” command:<\/p>\n

server1:~ # mount \/dev\/shm<\/span><\/span><\/p>\n

server1:~ # df -h \/dev\/shm<\/span><\/span>
\nFilesystem\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Size\u00a0 Used Avail Use% Mounted on<\/span>
\nudev\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50G\u00a0 128K\u00a0\u00a0 50G\u00a0\u00a0 1% \/dev<\/span><\/p>\n

The question now is how to make this change persistent. We decided to avoid changing the SuSE boot scripts, therefore we had to mount \/dev\/shm in the oracle boot procedure:<\/p>\n

server1:~ # cat \/etc\/init.d\/oracle.linux<\/span><\/span><\/p>\n

——————–<\/span>
\n….<\/span>
\ncase “$1” in<\/span>
\n‘start’)<\/span><\/p>\n

if [ $(mount | grep \/dev\/shm | wc -l) -lt 1 ]; then<\/span>
\necho “$0 : mounting \/dev\/shm”<\/span>
\nmount \/dev\/shm<\/span>
\nret=$?<\/span>
\nfi<\/span><\/p>\n

if [ ${ret} -ne 0 ]; then<\/span>
\necho “error $0 : \/dev\/shm could not be mounted”<\/span>
\nexit 1<\/span>
\nfi<\/span><\/p>\n

su – oracle -c “${DMK_HOME:-\/u00\/app\/oracle\/local\/dmk}\/bin\/service_start_stop.ksh start”<\/span>
\ntouch \/var\/lock\/subsys\/oracle<\/span>
\n;;<\/span>
\n‘stop’)<\/span>
\nsu – oracle -c “${DMK_HOME:-\/u00\/app\/oracle\/local\/dmk}\/bin\/service_start_stop.ksh stop”<\/span>
\nrm -f \/var\/lock\/subsys\/oracle<\/span>
\n;;<\/span>
\n*)<\/span>
\necho “usage $0 {start|stop}”<\/span>
\n>exit 1<\/span>
\n;;<\/span>
\nesac<\/span>
\n….<\/span>
\n——————–<\/span><\/p>\n

We also decided not to “unmount” the \/dev\/shm filesystem while stopping Oracle. As a result, this is what we see after reboot (and after an Oracle instance start):<\/p>\n

server1:~ # df -h \/dev\/shm<\/span><\/span>
\nFilesystem\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 Size\u00a0 Used Avail Use% Mounted on<\/span>
\nudev\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 50G\u00a0\u00a0 29G\u00a0\u00a0 22G\u00a0 58% \/dev\/shm<\/span><\/p>\n

During our tests, we also tried to modify the \/etc\/init.d\/boot.localfs script in order to mount \/dev\/shm directly within this script. However, this prevented the network from being started correctly since SuSE itself uses tmpfs for \/dev during the boot. After the boot procedure, if \/dev\/shm is not used (mounted) by another process\/script, the following files remain available in \/dev\/shm :<\/p>\n

server1:~ # ls -lrt \/dev\/shm\/sysconfig\/<\/span><\/span>
\ntotal 28<\/span>
\n-rw-r–r– 1 root root\u00a0 5 Oct 24 15:52 network<\/span>
\n-rw-r–r– 1 root root\u00a0 3 Oct 24 15:52 config-lo<\/span>
\n-rw-r–r– 1 root root 27 Oct 24 15:52 if-lo<\/span>
\n-rw-r–r– 1 root root\u00a0 7 Oct 24 15:52 ifup-lo<\/span>
\n-rw-r–r– 1 root root\u00a0 5 Oct 24 15:52 config-eth0<\/span>
\n-rw-r–r– 1 root root 29 Oct 24 15:52 if-eth0<\/span>
\n-rw-r–r– 1 root root\u00a0 7 Oct 24 15:52 ifup-eth0<\/span>
\ndrwxr-xr-x 2 root root 60 Oct 24 15:52 tmp<\/span>
\n-rw-r–r– 1 root root\u00a0 0 Oct 24 15:52 ready-lo<\/span>
\n-rw-r–r– 1 root root\u00a0 0 Oct 24 15:52 ready-eth0<\/span>
\n-rw-r–r– 1 root root\u00a0 0 Oct 24 15:52 ready-eth1<\/span>
\n-rw-r–r– 1 root root\u00a0 0 Oct 24 15:52 ready-eth2<\/span>
\n-rw-r–r– 1 root root\u00a0 0 Oct 24 15:52 ready-eth3<\/span><\/p>\n

After a correct \/dev\/shm configuration and after a reboot, the shm devices on SLES11 should be mounted and visible through:<\/p>\n

server1:~ # mount | grep shm<\/span><\/span>
\nudev on \/dev\/shm type tmpfs (rw,size=50g)<\/span><\/p>\n

Note that this information is also available on the Novell website and in the Oracle forum under:<\/p>\n