lo down

My server had a small hiccup this morning. I’m not sure what happened to it, but I couldn’t connect from remote. I ran home just before lunch, and connected a keyboard and monitor. The monitor showed the standard console output of lots of messages about packets that the firewall was blocking; but the keyboard wouldn’t work. Ugh, I needed to physically reboot my system.

It’d been quite some time since I last rebooted the machine. Sometime along the way I was assigned new IP addresses from Speakeasy, and I had configured the new addresses as aliases on the ethernet interface (eth0:0), so I could use the new IPs without having to reboot the server. I thought I had updated the /etc/network/interfaces file to reflect the changes, but memory and reality don’t always coincide, unfortunately.

So when I rebooted the server, the networking was fouled up. It had been awhile since I had fiddled with the interface aliases, so I quickly skimmed the man pages for interfaces, ifconfig, and ifup. It was here that I found the ifdown -a command, to down all the configured interfaces in one go. What a time saver! I executed that, fixed up the interfaces file, and then brought up eth0, which also brought up eth0:0.

After that I restarted the services that I felt needed restarting. Everything looked okay, but I decided to check my mail, just to make sure. The IMAP connection succeeded, but the connection never finished and the client was never able to open the mail folder. Hrm. A quick look through /var/log/mail.log showed mail being deferred, too. Not good. ps ax revealed that amavisd was not running. I tried to start it with /etc/init.d/amavis start, and it looked like it was working, but it never showed up in the process list. Finally, running /sbin/amavisd debug revealed that it couldn’t bind to 127.0.0.1:10024. That coincided with some log messages being generated by the firewall about not permitting access to 127.0.0.1:10024, so I looked over the firewall configuration. I spent rather longer than I should have double-checking my firewall settings, and trying things that I knew wouldn’t help the situation.

I finally decided to use the advice my old man always offers: always check the simple things first. So I backed up, and started from scratch. I took down all the interfaces (ifdown -a), and then brought up eth0, which conveniently also brings up eth0:0. I executed ifconfig to review the list of interfaces, and there was my problem, staring at me: there was no lo loopback adapter! I had brought down all the configured interfaces, but only brought up eth0 and eth0:0.

I didn’t consciously recognize that that was the problem, but some part of me saw that the listing of interfaces did not look like it should, and forced me to double- and triple-check the output line-by-line. Once I saw that the loopback interface was missing, it made perfect sense why amavisd wasn’t starting!

ifup lo fixed everything. As the joke goes, there’s no place like 127.0.0.1!