It's happened again. I set off to investigate one specific thing, and wind up learning a whole lot about things in which I wasn't the least bit interested originally.
I'm looking into installing Mailman for one of my virtual domains. First I set off looking at alternatives to Mailman, as I have an unnatural fear of Python. I looked at Enemies of Carlotta, SmartList, and a host of other solutions. They all had some (perceived) short-coming, and since I'm already modestly familiar with Mailman administration, I decided to install Mailman onto my workstation. (`apt-get install mailman` is such a wonderful thing!)
To properly support my environment, I need to configure Mailman to interoperate with Postfix virtual domains. For some reason, I became concerned that Mailman might be a performance drain on my system. The Mailman FAQ suggested mounting my mail spool with noatime, so off I went Google-ing for "Postfix noatime". I found this thread on the Postfix users' mailing list, which I followed to the end, re-learning much about SCSI and RAID filesystems that I'd forgotten (and none of which is directly appropriate to my paltry IDE-based server).
Another Mailman FAQ addressed memory consumption, and how to diagnose it. This re-introduced me to vmstat, and I learned that new Linux versions (kernel 2.5.41+) support a -a option to vmstat that displays active/inactive memory. Not quite sure of the difference, I Linux Google-ed for "vmstat inactive memory", which led me to this discussion, which helped me better understand why my server looks to be using so much memory:
$ free
total used free shared buffers cached
Mem: 1035568 974844 60724 0 34752 561812
-/+ buffers/cache: 378280 657288
Swap: 971924 0 971924
$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 0 58188 34752 561880 0 0 1 7 2 3 1 0 99 0
I'd been worried that my system was running dangerously low on RAM but the new -a switch to vmstat, with the previous explanation about "inactive memory", makes me feel much better:
$ vmstat -a procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free inact active si so bi bo in cs us sy id wa 0 0 0 58396 229728 586144 0 0 1 7 2 3 1 0 99 0
This whole process happens to me again and again. I set off to investigate mrtg to monitor my network performance, and that leads me to rrdtool, which in turns has me pondering what kind of script to call from cron: sh, perl, command-line php? Do I really want to run an SNMP daemon on my system, or can I get by with some fancy shell scripts? Heck, while I'm monitoring, why stop at network throughput? Why not monitor Apache performance (on a vhost basis, no less!), or Postfix performance?
I really do enjoy reading your posts. Unfortunately I was unable to finish this one, as my head exploded around the 2nd paragraph. Thanks anyway.
If you're interested in all the little kernel statistics you can get from your Linux system, check out Sysstat. Set that up to run every minute (it logs to a binary file, which you can then view with "sar"), and load all the data into RRD's with rrdtool. "sar -h" gives you output that makes rrdtool loading easy.
You might be interested in postfix-to-mailman if you've not already uncovered it. It's a small python script that sets up all the virtual aliases you need to run a mailman list, and with a little tweaking you can make lists.yourdomain.com (or whatever you feel like) run completely free of any user interaction on the side of postfix, you just setup a new transport method, the postfix-to-mailman handles the rest.