Version 0.1.6-beta
BETA version - first public release.
Memlog is an open source tool to log processes on a server when memory usage exceeds certain preset thresholds. It was originally written to run on a Dreamhost virtual private server but should work on any Linux based server.
It can sometimes be hard to figure out which processes caused a server to run out of memory and crash. Wouldn't it be cool to have list of processes that ran just prior to such a problem? That's exactly what Memlog attempts to do.
Memlog was developed as a result of trouble shooting sudden memory usage peaks that could not be anticipated well by the memory resource allocation tool PsManager.
Note: Not yet a Dreamhost customer? Save $97 by signing up to Dreamhost with our Dreamhost Promo Codes.
You can use any shell user on your server, or create a new user specifically for Memlog. On a Dreamhost VPS you can create a new user here: Manage Users - Add a new user.
Type of User Account: Shell Shell Type: /bin/bash Server: <select appropriate server>
Proceed to download the Memlog script and the configuration file:
memlog.pl and memlog.conf.
Both these files (memlog.pl and memlog.conf)
need to be placed in the same directory on your server,
for example in a "bin" directory of any user on the
private server you want to manage.
Avoid placing this script in your cgi-bin directory or under your site's
document root, it should not be accessible via your website.
The easiest way do the above is to do it directly from the command line of your VPS. Open an SSH connection to a user on your private server at type the following commands:
mkdir ~/bin 2>/dev/null; cd ~/bin
wget -N http://www.7is7.com/software/dreamhost/download/memlog.tar.gz
tar -zxvf memlog.tar.gz
If you did not follow the procedure above, check that the permissions are
in order: memlog.pl must be executable while
memlog.conf should only be readable by the owner (or type:
chmod 700 memlog.pl; chmod 600 memlog.conf).
When the above is done, edit the configuration file (memlog.conf)
setting your triggers.
Take care to choose appropriate values for your situation.
Memlog will log all processes every 5 seconds when any of these
conditions are met.
$mem_trigger_perc = 60;
And/Or
$mem_trigger_mb = 200;
To get Memlog running, you'll need to set up your crontab to run memlog.pl every minute (this example assumes the program is located in your "bin" directory):
* * * * * $HOME/bin/memlog.pl > /tmp/memlog_cron.log 2>&1
We recommend to edit your crontab yourself directly on your server, i.e. not to edit it via the DH Control Panel. This is because the Dreamhost control panel adds wrapper code that can disrupt the proper functioning of PsManager.
You can edit the crontab directly with crontab -e, while
crontab -l will list the contents of your crontab.
(Read more about
crontabs
on Dreamhost on their wiki page.)
When upgrading from an older version, one generally only needs to
replace memlog.pl,
as the configuration file can stay the same.
You can follow the procedure above but without replacing
your original memlog.conf file.
The instruction below will only extract the perl program from the tar-file after downloading it:
cd ~/bin
wget -N http://www.7is7.com/software/dreamhost/download/memlog.tar.gz
tar -zxvf memlog.tar.gz memlog.pl
By default Dreamhost servers use Californian time, as this is where they are located. However you may want to see PsManager report changes in an other timezone, like your own. Simply add a line like the one below to your configuration file:
$ENV{'TZ'} = 'Europe/Amsterdam';
To find your timezone, you can either browse through the subdirectories
of /usr/share/timezone/ or use the interactive command line
tool tzselect.
The running processes are logged whenever one of the conditions for
logging is met. The output is written to the memlog_process.log
file. This is the logfile you need to look at in order to see what
processes were running. For instance by doing a tail on the logfile:
tail -n 100 ~/bin/memlog_process.log
If you are encountering problems, these are a few starting points to check:
Check that the cronjob is set properly and runs every minute:
crontab -l | grep memlog
Check the output file for the Memlog cronjob. It should be empty if there are no problems. Errors will be sent here:
tail /tmp/memlog_cron.log
Check the usage log maintained by PsManager:
tail ~/bin/memlog.log
The columns are: <timestamp> <available memory> <used memory> <free memory> <cache memory> <logged flag> <number of processes>. All memory in MB. Here is a sample:
2013-02-16 23:50:02 330 44 285 159 n 58 2013-02-16 23:50:03 330 43 286 159 n 51 2013-02-16 23:50:07 330 40 289 159 n 44 2013-02-16 23:50:12 330 42 287 159 n 49 2013-02-16 23:50:17 330 40 289 159 n 45
To list all cases where processes have been logged
grep -e '\bY' ~/bin/memlog.log
A sample of what that would show:
2013-01-20 23:01:50 330 112 217 210 Y 55 2013-01-20 23:01:56 330 112 217 210 Y 53 2013-01-20 23:02:00 330 109 220 210 Y 55 2013-01-20 23:02:02 330 113 216 210 Y 54 2013-01-20 23:02:06 330 110 219 210 Y 49
If you are encountering problems, and need help, Memlog can generate a report, that you can copy and paste into an e-mail. This checks a number of settings and the things mentioned above, while not disclosing any sensitive information like account names. To generate this report run memlog.pl from the command line with the "check" argument:
~/bin/memlog.pl check
Guaranteed freedom of use and code transparency.
The Memlog program is released under an open source license (GNU-GPL v3), which means that you can use this script for free and make any modifications that you please. You are in no way dependent on the original author of this program for future maintenance and your right to use it never expires. The source code of the script is easily viewable (the script is written in perl), you can scrutinize it or have it scrutinized to convince yourself that it does what it says it does.
If you are using the Memlog script on Dreamhost, you may also be interested in using the Dreamhost VPS Manager script (PsManager) to automatically manage the memory allocated to your VPS.
There is a Dreamhost forum thread for Memlog where you can ask questions. If you are using or want to use Memlog feel free to contact me by E-mail about any problems or suggestions.
Note: If you want to be notified of important new releases of Memlog, or issues affecting the script, send me an E-mail and ask to be added to the mailing list. (It is still a small mailing list that I maintain manually.)
7is7.com | Software | Otto | Travel Stories | Countdown Clock | Firefox | StatEye
New | About | Contact | Connect | Friends | Promotions | Copyright | Advertise
