Monitoring

Cleaning Up Memory Usage

Posted on by

I noticed my Ubuntu desktop was using a rather large portion of available memory.  I usually have a lot running on my system, multiple terminals, background jobs, etc so this is nothing unusual.  Today however I noticed my system was sluggish so I started digging.  Memory use was near 100%.  I closed all of my programs to see what effect that would have but the memory usage stayed very high ~90%.  I started to suspect a memory leak in one of the processes or programs I was running.  I really didn’t want to reboot the system since it isn’t a Windows desktop!  What to do.  I needed to force memory cleanup on the system.  How do I analyze the memory usage on a system?  I thought I would document a few of the ways to see memory use.

You can use commands like ‘top’ and ‘vmstat’ to get an idea of what your system is chewing on.  Specifically looking at memory I tend to use:

watch -n 1 free -m

For a more detailed look use:

watch -n 1 cat /proc/meminfo

If you suspect a program of having a leak you can use valgrind to dig even deeper:

valgrind --leak-check=yes program_to_test

‘valgrind’ is great for testing however not to helpful with currently running processes or without some experience.

So you analyze the system and determine there is memory that has not been properly freed, what do you do?  You can reboot but that isn’t always an option.  You can force clear the cache doing the following:

sudo sysctl -w vm.drop_caches=3

This frees up unused but claimed memory in Ubuntu a (and most linux flavors).  This command won’t affect system stability and performance, it will just clean up memory used by the Linux Kernel on caches.  That said I have noticed the system is more responsive (contradiction, you decide).  Here is an example of how much memory you can free up with this command:

$ free
             total       used       free     shared    buffers     cached
Mem:      16287672   15997176     290496       5432     404120   14415648
-/+ buffers/cache:    1177408   15110264
Swap:      4093884          0    4093884
[msaba@nfc ~]$ sudo sysctl -w vm.drop_caches=3
[sudo] password for msaba: 
vm.drop_caches = 3
[msaba@nfc ~]$ free
             total       used       free     shared    buffers     cached
Mem:      16287672     948076   15339596       5432       1268      92708
-/+ buffers/cache:     854100   15433572
Swap:      4093884          0    4093884

Another command that can free up used or cached memory (inodes, page cache, and ‘dentries’):

sudo sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

I have not seen any significant difference between the results of this or the first command.

I’ll add updates to this page as I think of them.  Good luck for now.

The Root of Missing Mail

Posted on by

Like all conscientious system administrator I like to keep tabs on my servers.  One way of doing this is checking root’s email daily.  This is a great idea if you have a few servers and never take vacation!  I manage close to 100 servers, so I need a more efficient way of “hearing” my servers when they complain to root about something.  Aside from monitoring solution (not covered here) the best way to do this is to redirect where email for the root user gets sent.

This seems pretty simple so I never thought of posting about this, until today.  Some facts , to forward mail for the root user leverage the /etc/aliases file.  Like always I added a line to /etc/aliases like this:

# vi /etc/aliases

     root:    myemailaddress@uconn.edu

Ideally you want to set the email address to a list serve so that your backup administrator receives these messages also, so you can take a vacation.

I made that change yesterday on a new server and didn’t give it a second thought.  Today no mail, and I know there was an error on the system?!

First thing I checked was if I could send mail from the server, I could have…or I just forgot because I am sleep deprived…  I was able to send mail from the command line to an email address but not to an alias.  OK, that is a big clue.

While I have never had to do this before, (perhaps I restarted all my other systems?), regardless to fix the problem I simply ran this command:

# newaliases

Bingo, mail started flowing!

If that doesn’t fix it for you, other things to check are:

– Include the following in your /etc/hosts.allow:

ALL: 127.0.0.1 : allow