Pages

Wednesday, April 6, 2016

Not enough space for Oracle memory under /dev/shm

=================
General
=================
Consider this real scenario:
When trying to connect to Oracle, there are this errors:

Incident A.
ERROR:
ORA-01034: ORACLE not available
ORA-27123: unable to attach to shared memory segment
Linux-x86_64 Error: 2: No such file or directory
Additional information: 4456455
Additional information: 84
Additional information: 10
Process ID: 0
Session ID: 0 Serial number: 0


Incident B.
ERROR:
ORA-01034: ORACLE not available
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 1
Additional information: 65537
Additional information: 8
Process ID: 0
Session ID: 0 Serial number: 0

When trying to start Oracle, it is throwing an error of not enough memory.
When starting up Oracle, there is an error in alert.log

something like: not enough memory under /dev/shm


When trying to stop Oracle, it is not responding
SQL> shutdown immediate
ORA-01034: ORACLE not available
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 1
Additional information: 65537
Additional information: 8
SQL> shutdown abort

ORACLE instance shut down.

But in the server pmon process is still running.
oracle@my_server:/dev/shm>% ps -ef | grep pmon
oracle    5858     1  0  2013 ?        00:27:28 ora_pmon_igt

oracle   31756 25262  0 06:43 pts/4    00:00:00 grep pmon

Checking files under /dev/shm
There are many such files
ls -l /dev/shm
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_92
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_93
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_94
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_95
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_96
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_97
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_98
-rw-r----- 1 oracle dba 4194304 Nov 10  2013 ora_igt_65537_99 

Oracle uses /dev/shm for allocating chunks of memory.
There are many files, which belong to Oracle. 
These are chunks of shared memory dedicated to Oracle.
The problem is that most of the memory area is used up by Oracle zombie process.

=============
Solution:
=============

Reboot the host server.
For some reason, a "zombie" Oracle process was occupying memory under  /dev/shm.


==========================
How to increase size of /dev/shm
==========================
root@devdb-1-aps-01:~>% df -hP | grep shm
tmpfs                          4.0G  658M  3.4G  17% /dev/shm

root@devdb-1-aps-01:~>% vi /etc/fstab
tmpfs           /dev/shm        tmpfs   size=4g,nodev,nosuid,noexec     0 0

Edit /etc/fstab change from 4g to 6g

tmpfs           /dev/shm        tmpfs   size=6g,nodev,nosuid,noexec     0 0

Save and exit

root@devdb-1-aps-01:~>% mount -o remount /dev/shm

root@devdb-1-aps-01:~>% df -hP | grep shm

tmpfs                          6.0G  658M  5.4G  11% /dev/shm


No comments:

Post a Comment