Pages

Tuesday, December 16, 2014

Linux Memory commands.

Most useful commands to see memory status on a Linux server.

1. top
2. free -m
3. less /proc/meminfo
4. vmstat -s
5. sar
6. ps
7. smem
8. process level info - pmap
9. scripts to gather memory info on Linux


==================
1. top
==================
top command displays dynamic real-time view of system information and running tasks.

top
top - 14:58:12 up 8 days, 22:01,  6 users,  load average: 10.34, 10.31, 10.35
Tasks: 406 total,  10 running, 396 sleeping,   0 stopped,   0 zombie
Cpu(s): 58.4%us,  4.3%sy,  0.0%ni, 36.9%id,  0.4%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  14364508k total, 14315820k used,    48688k free,   153900k buffers
Swap:  2097144k total,  2097136k used,        8k free,  1090736k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
11041 iu        15   0  240m 4416 2844 S 100.6  0.0 492:23.53 notification.ex
 1978 root      25   0  729m 681m  664 R 99.2  4.9   4791:17 aide
15690 root      25   0  931m 883m  664 R 99.2  6.3   7658:11 aide
15875 root      25   0 1102m 1.0g  664 R 99.2  7.5  10527:19 aide
29496 root      25   0 1178m 1.1g  668 R 99.2  8.1  11962:01 aide
31509 root      25   0 1019m 972m  668 R 99.2  6.9   9091:25 aide
12776 root      25   0  607m 559m  664 R 98.9  4.0   3358:35 aide
12936 root      25   0  834m 786m  664 R 98.9  5.6   6212:07 aide
29103 root      25   0  456m 408m  664 R 98.9  2.9   1927:31 aide
16361 root      25   0  232m 184m  720 R 98.2  1.3 425:50.84 aide
11779 iu        22   0 5247m 2.8g 6860 S  5.3 20.5  35:42.43 java
32115 oracle    15   0 2256m  73m  70m S  2.0  0.5   0:16.31 oracle
 4377 root      34  19     0    0    0 S  1.0  0.0 166:01.26 kipmi0
11072 iu        15   0  241m 6104 2796 S  0.7  0.0   0:48.57 notification.ex
11386 oracle    15   0 90120 1788 1044 S  0.3  0.0   0:00.14 sshd


PID – ID of the process
USER – The user that is the owner of the process 
PR – priority of the process 
NI – The “NICE” value of the process 
VIRT – virtual memory used by the process
RES – physical memory used from the process
SHR – shared memory of the process
S – indicates the status of the process: S=sleep R=running Z=zombie
%CPU – This is the percentage of CPU used by this process 
%MEM – This is the percentage of RAM used by the process
TIME+ –This is the total time of activity of this process
COMMAND – And this is the name of the process

to run top in batch:
top -b -n1 > top.log
top does one iteration, and exists.

To sort by memory:
SHIFT+M

To sort by any field:
SHIFT+F => Select field => Enter
==================
2. free -m
==================

free
free command displays amount of total, free and used physical memory (RAM) in the system, as well as showing information on shared memory, buffers, cached memory and swap space used by the Linux kernel.

By default the output is in Kb

free -m The output is in Mb
free -g The output is in Bb
free -m The output is in Mb
free -mt The output is in Mb with totals
free -mto The output is in Mb with totals
free -ms 5 Run free -m every 5 seconds
watch free - Run free every 2 seconds (default)
watch -n 1 -d freeRun free every 1 second, and highlight difference between runs


Understanding free output
~>free -m
             total       used       free     shared    buffers     cached
Mem:         14027      13984         43          0        166       1077
-/+ buffers/cache:      12739       1288
Swap:         2047       2047          0

total: Server total (physical) RAM (excluding a small bit that the kernel.
used: memory in use by the OS.
free: memory not in use.

shared, buffers, cached: This shows memory usage for specific purposes. 
                                               These values already included in "used".
shared - ?
buffers - Memory used for file system metadata.
cached  - Pages with actual contents of files or block devices.
Pre allocation of bufferes and cached memory RAM helps the system to run faster because disk information is already in memory which saves I/O operations.


-/+ buffers/cache: Adjustment of values in first line. These values are more meaningful than the first line.
12739 = 13984 - (166+1077)
1288 = 43 + (166+1077)

Swap: gives information about swap space usage

Linux (like most OS) will always try to use free RAM for caching.
So Mem: free will almost always be very low. 
Therefore the line -/+ buffers/cache:free shows how much memory is free when ignoring caches.
caches will be freed automatically if memory gets scarce, so they do not really matter.

A Linux system is really low on memory if the free value in -/+ buffers/cache: line gets low.

==================
3. less /proc/meminfo
==================
The /proc file system does not contain real files. 
They are rather virtual files that contain dynamic information about the kernel and the system.

less /proc/meminfo
MemTotal:     14364508 kB
MemFree:       3994220 kB
Buffers:        551096 kB
Cached:        2424648 kB
SwapCached:     379240 kB
Active:        6037092 kB
Inactive:      3469056 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     14364508 kB
LowFree:       3994220 kB
SwapTotal:     2097144 kB
SwapFree:       619160 kB
Dirty:            2340 kB
Writeback:          64 kB
AnonPages:     6157432 kB
Mapped:         750808 kB
Slab:           675316 kB
PageTables:     137172 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   9279396 kB
Committed_AS: 14622668 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    264892 kB
VmallocChunk: 34359473231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

==================
4. vmstat
==================

vmstat reports report virtual memory statistics.
The information is about processes, swap, free, buffer and cache memory, paging space.

vmstat
The command will display report based on averages since last reboot.

vmstat 5


The command will pool average system resources usage level for a sampling period of 5 seconds at interval of 5 seconds, except the first result that is averages since the last reboot.

vmstat -s
The output of vmstat -s is similar to /proc/meminfo

Default units are Kb

vmstat
procs -----------memory------------ ---swap-- -----io---- --system----- ---cpu----
 r  b   swpd    free   buff   cache   si   so    bi    bo  in  cs us sy id  wa  st

 2  0 2835188  38428  16336 6560160    0    0    51    70   0   0  7  6 85   2   0

vmstat -s
     14364508  total memory
     10392404  used memory
      6060160  active memory
      3469716  inactive memory
      3972104  free memory
       553064  buffer memory
      2439024  swap cache
      2097144  total swap
      1465904  used swap
       631240  free swap
    377010091 non-nice user cpu ticks
         2975 nice user cpu ticks
     59577453 system cpu ticks
    788782124 idle cpu ticks
     18921537 IO-wait cpu ticks
        29664 IRQ cpu ticks
       176165 softirq cpu ticks
            0 stolen cpu ticks
    312184009 pages paged in
    371292265 pages paged out
      2711222 pages swapped in
      3370874 pages swapped out
    917256990 interrupts
   1512285390 CPU context switches
   1417539417 boot time
     14021020 forks


==================
5. sar
==================
SAR stands for System Activity Reporter
It is part of sysstat installation.
It is running at intervals by crontab, samples system information, and logs that information into files. 
One file per day. 
sa01 for first day of the month, sa22 - for 22nd day of month, and so on.
By default - it is running every 10 minutes, and stores information for 7 days.

configuration file: /etc/sysconfig/sysstat
crontab file: /etc/cron.d/sysstat
files are written to: /var/log/sa/ folder


cd /var/log/sa
>% ls -1
sa18
sa19
sa20
sa21
sa22

The actual output of sar would depend upon sar version/configuration.

When invoking sar - it is reading gathered information from pre generated binary files.

In the above output most of the columns are self explanatory(most of the outputs are in KB).

Values
Meaning
kbmemfree
Amount of free memory available in kilobytes
kbmemused
Amount of used memory in kilobytes
%memused
Percentage of used memory
kbbuffers
Amount of memory used as buffers by the kernel in kilobytes
kbcached
Amount of memory used to cache data by the kernel in kilobytes
kbswpfree
Amount of free swap space in kilobytes
kbswpused
Amount of used swap space in kilobytes
%swpused
Percentage of used swap space
kbswpcad
Amount of cached swap memory in kilobytes.

Additional fields:
kbcommit: Memory in kilobytes needed for current workload.
%commit: Percentage of memory needed for current workload in relation to the total amount of memory (RAM+swap).


Some examples:
sar with no parameter - fetch memory info for current day.

>% sar
Linux 2.6.18-274.el5 (my_server)  03/26/2015
12:00:01 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle
12:10:01 AM       all      0.33      0.00      0.31      0.64      0.00     98.72
12:20:01 AM       all      0.31      0.00      0.30      0.33      0.00     99.06
12:30:01 AM       all      0.29      0.00      0.30      0.28      0.00     99.13

sar -f sa<day> fetch memory info for that day.

>% cd /var/log/sa>% sar -f sa22
Linux 2.6.18-274.el5 (my_server)  03/22/2015
12:00:01 AM       CPU     %user     %nice   %system   %iowait    %steal     %idle
12:10:01 AM       all      0.36      0.00      0.31      0.67      0.00     98.65
12:20:01 AM       all      0.35      0.00      0.30      0.36      0.00     98.99
12:30:01 AM       all      0.31      0.00      0.29      0.27      0.00     99.12
12:40:01 AM       all      0.31      0.00      0.29      0.30      0.00     99.10

sar -r sa<day> fetch extended memory info for that day.
sar -r -f sa22
Linux 2.6.18-274.el5 (my_server)  03/22/2015

12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached kbswpfree kbswpused  %swpused  kbswpcad
12:10:01 AM    459976   9765036     95.50    195912   8156376   8889112   1596640     15.23    285264
12:20:01 AM    418156   9806856     95.91    199908   8195956   8889132   1596620     15.23    285312
12:30:01 AM    399384   9825628     96.09    203832   8205748   8889172   1596580     15.23    285272
12:40:01 AM    356488   9868524     96.51    207860   8248572   8889172   1596580     15.23    285272

sar -r
12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
05:40:01 AM    515704  65157200     99.21         8  52898780  22709820     27.90  18171748  40886116       932
05:50:01 AM    546036  65126868     99.17         8  52869684  22709452     27.90  18144272  40884768       984
06:00:01 AM    515536  65157368     99.21         8  52838316  22783136     27.99  18149564  40889020      1720
06:10:01 AM  37281288  28391616     43.23         8  16407268  22695028     27.88  18133764   4423772      1064
Average:     25449404  40223500     61.25         8  28122914  22756175     27.96  18436755  15869233      1245

kbmemfree - Amount of free memory available in kilobytes.
kbmemused - 
Amount of used memory in kilobytes.
                       This does not take into account memory used by the kernel itself.

%memused  - 
Percentage of used memory.
kbbuffers - 
Amount of memory used as buffers by the kernel in kilobytes.  
kbcached  - Amount of memory used to cache data by the kernel in kilobytes.
kbcommit  - 
Amount of memory in kilobytes needed for current workload. 
                       This is an estimate of how much RAM/swap is needed to guarantee that there never is out of memory.  
%commit   - Percentage of memory needed for current workload in relation to the total amount of memory (RAM+swap). This number may be greater than 100% because the kernel usually overcommits memory.
kbactive  - 
Amount of active memory in kilobytes (memory that has been used more recently and usually not reclaimed unless absolutely necessary).   
kbinact   - Amount of inactive memory in kilobytes (memory which has been less recently used. It is more eligible to be reclaimed for other purposes).   


For a specific day, say 22:
sar - r -f /var/log/sa/sa21

See SWAP memory usage:
sar -r for current day
sar -r -f sa14  for specific day: 

root@my_server:>%  sar -r -f /var/log/sa/sa14
Or
root@my_server:/var/log/sa>%  sar -S -f sa14
Linux 2.6.32-504.8.1.el6.x86_64 (ind-ida-6-dbu-1)       05/14/2016      _x86_64_        (12 CPU)

12:00:01 AM kbswpfree kbswpused  %swpused  kbswpcad   %swpcad
12:10:01 AM  10485756         0      0.00         0      0.00
12:20:01 AM  10485756         0      0.00         0      0.00
12:30:01 AM  10485756         0      0.00         0      0.00
12:40:01 AM  10485756         0      0.00         0      0.00
12:50:01 AM  10485756         0      0.00         0      0.00
01:00:01 AM  10485756         0      0.00         0      0.00
01:10:01 AM  10485756         0      0.00         0      0.00
01:20:01 AM  10485756         0      0.00         0      0.00
01:30:01 AM  10485756         0      0.00         0      0.00
01:40:01 AM  10485756         0      0.00         0      0.00
01:50:01 AM  10485756         0      0.00         0      0.00
02:00:01 AM  10485756         0      0.00         0      0.00
02:10:01 AM  10485756         0      0.00         0      0.00
02:20:01 AM  10485756         0      0.00         0      0.00
02:30:01 AM  10485756         0      0.00         0      0.00
02:40:01 AM  10485756         0      0.00         0      0.00
02:50:01 AM  10485756         0      0.00         0      0.00
03:00:01 AM  10485756         0      0.00         0      0.00
03:10:01 AM  10485756         0      0.00         0      0.00
03:20:01 AM  10485756         0      0.00         0      0.00
03:30:01 AM  10485756         0      0.00         0      0.00
03:40:01 AM  10485756         0      0.00         0      0.00
03:50:01 AM  10485756         0      0.00         0      0.00

04:00:01 AM  10485756         0      0.00         0      0.00


For more options, there is this nice reference: sar reference

==================
6. ps
==================
Useful ps References:
www.computerhope.com 

ps options may have three formats:
1   UNIX options, which may be grouped and must be preceeded by a dash.
2   BSD options, which may be grouped and must not be used with a dash.
3   GNU long options, which are preceeded by two dashes. 

Some useful options

ps command reports a snapshot on information of the current active processes.

ps

ps -a
   List all running processes of current user. (Only processes that are attached to a terminal.)

ps -au
   List all running processes of current user. (Whether or not they're attached to a terminal.)

ps -ax
   List all running processes of all users.

ps -ax<user> or ps -ax <user>
   List all running processes of user <user>

ps -aux
   List all running processes, with additional information about their resource usage.

ps -eo pid,ppid,rss,size,vsize,pcpu,pmem,cmd -ww --sort=vsize
List all running processes, with additional information about their resource usage, and sort by virtual memory.


ps -eo pid,ppid,user,args,rsz,vsz,cmd --sort user
ps -eo pid,ppid,user,args,rsz,vsz,cmd --sort rsz
List all running processes, with additional information about their resource usage, and sort by user.

To see only the memory resources occupied by each category of processes, such as Apache httpd, MySQL mysqld or Java, use the following command:
ps -aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr

pid = Process ID
ppid = Parent Process ID
rssResident Set Size. It is the portion of memory occupied by a process that is held in main memory (RAM).
vsize = Virtual Memory size of entire process, in Kb.
size =  The virtual size of the data section of the process.
pmem = ratio of the process’s rss to the physical memory on the machine.
pcpu = It is the CPU time used divided by the time the process has been running.
args = Command command with all its arguments as a string.

==================
7. smem
==================
smem is a utility to view physical memory usage.
smem reports physical memory usage.  
It does not include memory that has been swapped out to disk.
It can report USS or PSS
USS - Unique Set Size - Unshared memory
PSS - Proportional Set Size - Unshared memory + process's proportion of shared memory

USS is the process unshared memory.
PSS is process USS + process's proportion of shared memory

usage:
smem -t -s uss

====================================
8. process level info - pmap
====================================
When ps is reporting on a memory,rss,size,vsize it is not possible to determine the actual process memory, from the loaded shared libraries.
To see the actual process memory usage, there is a command pmap
This is a sample of pmap -d <pid> output


root@hostname:~>% pmap -d 411
411:   /starhome/igate/hostname ig1 vip_monitor.real.exe --PROCESSNAME=AUT-MOBOV-99_MON_IG2_1
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000     736 r-x-- 0000000000000000 0fd:0000c vip_monitor.real.exe
00000000006b8000      12 rw--- 00000000000b8000 0fd:0000c vip_monitor.real.exe
00000000006bb000       4 rw--- 00000000006bb000 000:00000   [ anon ]
000000001d600000    1672 rw--- 000000001d600000 000:00000   [ anon ]
00000032b3a00000     112 r-x-- 0000000000000000 0fd:00001 ld-2.5.so
00000032b3c1b000       4 r---- 000000000001b000 0fd:00001 ld-2.5.so
00000032b3c1c000       4 rw--- 000000000001c000 0fd:00001 ld-2.5.so
00000032b3e00000      80 r-x-- 0000000000000000 0fd:00002 libz.so.1.2.3
00000032b3e14000    2044 ----- 0000000000014000 0fd:00002 libz.so.1.2.3
00000032b4013000       4 rw--- 0000000000013000 0fd:00002 libz.so.1.2.3
00000032b4200000    1336 r-x-- 0000000000000000 0fd:00001 libc-2.5.so
00000032b434e000    2044 ----- 000000000014e000 0fd:00001 libc-2.5.so
00000032b454d000      16 r---- 000000000014d000 0fd:00001 libc-2.5.so
00000032b4551000       4 rw--- 0000000000151000 0fd:00001 libc-2.5.so
00002b82af0b8000      40 r-x-- 0000000000000000 0fd:00001 libnss_files-2.5.so
00002b82af0c2000    2044 ----- 000000000000a000 0fd:00001 libnss_files-2.5.so
00002b82af2c1000       4 r---- 0000000000009000 0fd:00001 libnss_files-2.5.so
00002b82af2c2000       4 rw--- 000000000000a000 0fd:00001 libnss_files-2.5.so
00002b82af2c3000     400 r---- 0000000000000000 0fd:00008 timezlrg.dat
00002b82af327000      32 r-x-- 0000000000000000 0fd:00008 libnque11.so
00002b82af32f000    1020 ----- 0000000000008000 0fd:00008 libnque11.so
00002b82af42e000       4 rw--- 0000000000007000 0fd:00008 libnque11.so
00002b82af42f000     332 rw--- 00002b82af42f000 000:00000   [ anon ]
00002b82af482000   55124 r---- 0000000000000000 0fd:00002 locale-archive
00007fff30766000      76 rwx-- 00007ffffffea000 000:00000   [ stack ]
00007fff30779000       8 rw--- 00007fffffffd000 000:00000   [ anon ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
...
...
mapped: 215452K    writeable/private: 22944K    shared: 0K

Each shared library is listed twice; once for its code segment and once for its data segment. The code segments have a mode of "r-x--", while the data is set to "rw---". 

The writeable/private part in the last line "mapped: 215452K   writeable/private: 22944K" is a summary of the process memory segments, without shared liraries segments.. This is the actual process memory consumption.

Compare this to the output of ps aux

root@hostname:~>% ps aux | grep 411 | grep my_user
USER          PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
my_user       411  0.0  0.1 207260 33076 ?        S    Jan19   0:00 /starhome/igate/aut-mob-ovm ig1 vip_monitor.real.exe --PROCESSNAME=AUT-MOBOV-99_MON_IG2_1


          RSS          VSZ
pmap      22944Kb      215452Kb
ps -aux   33076Kb      207260Kb


====================================
9. scripts to gather memory info on Linux
====================================
Collect data from top, free, ps commands into a log
#!/bin/bash

touch top_trace.log
touch free.log
touch sorted_processes.log

while [ 1 -eq 1 ] ; do
  date >> top_trace.log
  top -b -n 1 | head -30 >> top_trace.log

  date >> free.log
  free >> free.log  

  date >> sorted_processes.log

  ps -eo pid,ppid,rss,size,vsize,pcpu,pmem,cmd -ww --sort=vsize > sorted_processes.log

  sleep 10

  #------------------------------------------
  #prevent the log files from growing forever
  #------------------------------------------
  mv -f `find . -name "sorted_processes.log" -size +1000000k` sorted_processes.log_bak
  mv -f `find . -name "top_trace.log" -size +1000000k` top_trace.log_bak
  mv -f `find . -name "free.log" -size +1000000k` free.log_bak

done


==================
References
==================


No comments:

Post a Comment