Analysis

When did that change?

Posted on by

Trying to shutdown an old web server from the late 1990’s that had it’s guts transplanted onto a newer system around 2003 and again around 2009. As you can imagine there are accounts and files that are like those items in your junk drawer, they beg the question…why is this here?!

In an attempt to determine last use of accounts we combined some log analysis with some unix timestamp forensics to prove that no one really needs this anymore!

The log analysis was pretty easy, track non-robot traffic to determine which accounts were being accessed and at what frequency and volume. The timestamp wasn’t difficult just had to isolate which files we wanted to analyze. Using the `stat`, `find` and/or the `ls` commands make this easy. In case you are not aware of this Linux/Unix stores a number of timestamps for each file.  These timestamps store when any file or directory was last accessed (read from or written to),  changed (file access permissions were changed) or modified (written to).

Three times tracked for each file in Linux/Unix are:

  • access time – atime
  • change time – ctime
  • modify time – mtime

Aside from using atime, ctime or mtime, the easiest way to get the information we are looking for is using the `stat` command:

# stat /home/myhome/file1 
  File: `/home/myhome/file1'
  Size: 1498906   	Blocks: 2928       IO Block: 4096   regular file
Device: fd01h/64769d	Inode: 3414009     Links: 1
Access: (0664/-rw-rw-r--)  Uid: (  500/   myhome)   Gid: (  500/   users)
Access: 2016-01-26 12:53:01.309089993 -0500
Modify: 2013-07-15 10:28:05.241847000 -0400
Change: 2013-07-15 10:28:05.315848001 -0400

If you are looking for a large set of files that have been accessed/modified/changed before or after a specific date then using the `find` command is your best bet.

For single files or a small set of files the `ls` command is probably easier.

For information on how to use atime, ctime and mtime with `find` and `ls` refer to the man page for the specific command.

Cleaning Up Memory Usage

Posted on by

I noticed my Ubuntu desktop was using a rather large portion of available memory.  I usually have a lot running on my system, multiple terminals, background jobs, etc so this is nothing unusual.  Today however I noticed my system was sluggish so I started digging.  Memory use was near 100%.  I closed all of my programs to see what effect that would have but the memory usage stayed very high ~90%.  I started to suspect a memory leak in one of the processes or programs I was running.  I really didn’t want to reboot the system since it isn’t a Windows desktop!  What to do.  I needed to force memory cleanup on the system.  How do I analyze the memory usage on a system?  I thought I would document a few of the ways to see memory use.

You can use commands like ‘top’ and ‘vmstat’ to get an idea of what your system is chewing on.  Specifically looking at memory I tend to use:

watch -n 1 free -m

For a more detailed look use:

watch -n 1 cat /proc/meminfo

If you suspect a program of having a leak you can use valgrind to dig even deeper:

valgrind --leak-check=yes program_to_test

‘valgrind’ is great for testing however not to helpful with currently running processes or without some experience.

So you analyze the system and determine there is memory that has not been properly freed, what do you do?  You can reboot but that isn’t always an option.  You can force clear the cache doing the following:

sudo sysctl -w vm.drop_caches=3

This frees up unused but claimed memory in Ubuntu a (and most linux flavors).  This command won’t affect system stability and performance, it will just clean up memory used by the Linux Kernel on caches.  That said I have noticed the system is more responsive (contradiction, you decide).  Here is an example of how much memory you can free up with this command:

$ free
             total       used       free     shared    buffers     cached
Mem:      16287672   15997176     290496       5432     404120   14415648
-/+ buffers/cache:    1177408   15110264
Swap:      4093884          0    4093884
[msaba@nfc ~]$ sudo sysctl -w vm.drop_caches=3
[sudo] password for msaba: 
vm.drop_caches = 3
[msaba@nfc ~]$ free
             total       used       free     shared    buffers     cached
Mem:      16287672     948076   15339596       5432       1268      92708
-/+ buffers/cache:     854100   15433572
Swap:      4093884          0    4093884

Another command that can free up used or cached memory (inodes, page cache, and ‘dentries’):

sudo sync && echo 3 | sudo tee /proc/sys/vm/drop_caches

I have not seen any significant difference between the results of this or the first command.

I’ll add updates to this page as I think of them.  Good luck for now.