I acquired a Pogoplug device awhile back, and on the whole I find it a neat little gadget. The user interface is okay, but not great; but really I don't need another web-based storage location. I've already got Dropbox and Ubuntu One accounts that I hardly use, so being able to store more data online -- albeit on hardware I control -- just isn't that useful. What I would like, though, is a small, low-power Linux server that I can use for a variety of useful tasks.
The Pogoplug has 256 megabytes of RAM, so anything I might do with it needs to have minimal system requirements. I decided to try to install Habari, mostly just to see if I could. Habari can use SQLite as its datastore rather than a complex relational database server, and requires only an http process that can handle PHP. This means it can use lighttpd or nginx rather than Apache. Using SQLite severely restricts the potential performance and scalability of Habari, but with only 256MB RAM I'm already severely limited in how much I might be able to scale.
It was trivially easy to install Debian on the Pogoplug device. I booted from a USB stick which contains the root filesystem. I added a 500GB external drive, and gave it three partitions: swap, /var and /home. The Debian install had basically no services running other than sshd -- no syslog, no cups, no avahi, no nothing. I installed nginx and PHP. In order to get nginx to handle the PHP processes, I needed to install the spawn-fcgi package.
Then I just followed the Habari nginx instructions, and I had a blog running! With nginx, spawn-fcgi and a screen session connected to IRC, I still have 7 megs of unused memory, and I haven't dipped into swap at all!
skippy@debian:~$ free -m
total used free shared buffers cached
Mem: 249 242 7 0 11 205
-/+ buffers/cache: 25 224
Swap: 2047 0 2047
skippy@debian:~$ vmstat -a
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free inact active si so bi bo in cs us sy id wa
0 0 0 7328 197228 37972 0 0 0 25 36 39 0 0 99 0
Of course this little blog isn't actually handling any traffic, but it's nice to know that I have a self-contained, self-hosted Habari environment on which I can hack.
At the 2011 Ohio LinuxFest, I gave a presentation entitled "Real World Job Skills the Free Software Way." In it, I detailed a number of ways in which free software improves not only one's hard technical skills, but also the softer social skills that are so often absent from professional development discussions. I focused on three hard skills -- programming, documentation, and packaging -- and three soft skills -- communication, mentoring, and professional networking -- and gave practical examples of how using free software and participating in free software projects can demonstrably improve one's image to potential employers.
One of the most interesting moments in my presentation occurred early when I asked for a show of hands of who had a GitHub or SourceForge or similar code sharing account. Most of the hands in the room went up. I then asked for a show of hands of who provided links to those accounts on their resumes. Only one hand remained up.
"Why not provide a link to your code on your resume?" I asked. I was, honestly, surprised that no one was doing it. Here's an excellent way to show to potential employers "This is my work", where that work includes not only code contributions but commit messages, bug fixes, community engagement, and much more. Such a link allows a potential employer to see what you're doing, and gives a great starting point for a variety of interview questions. This applies as much to sysadmins as programmers: automation scripts and time-saving solutions that you may have created go a long way to demonstrating your value to potential employers.
After my prepared remarks, I asked for comments from the audience. In particular, two people were currently hiring for new positions, so I wanted to hear from them as to what specific things they looked for in potential candidates. This proved illuminating, and largely mirrored the items in my presentation. Then one attendee asked, essentially, "I do PHP stuff on my own at home. How can I really put that on my resume in a way that makes sense to a potential employer?"
My on-the-spot answer was okay, but not great. I encouraged the fellow to get involved with a project so that he could then also list his participation in that project along with his PHP experience. Rather than just goofing around with PHP on his own, he'd be using PHP within the framework of a real free software project, and enjoying all of the ancillary benefits that come from such participation. This takes time, though, which may not help someone looking for a job right now.
What I should have said was this: you should list PHP on your resume somewhere, and expand upon it in your cover letter with an explanation of your passion for learning new things, an explanation of your problem solving methodologies, and a direct statement that you'd like to apply your passion to the benefit of this would-be employer. You're unlikely to get a full-time PHP programming position based solely on personal use of PHP, for example, but listing the things you've learned on your own on your resume should prove instructive to potential employers when evaluating you against other candidates for the position. Taking the extra step to articulate these things in your cover letter is even better.
I quite enjoyed giving this presentation, and I'd love to find an opportunity to do it again!
As I mentioned on Google+, I destroyed my laptop's filesystem. This was entirely due to my own carelessness. As I sat there looking at an unhelpful GRUB error message, I reviewed my options. I knew that I had destroyed the partition table on my laptop, and created two new partitions: one 1GB ext4 partition and one 512MB Linux swap partition. Both had been formatted, meaning that I wouldn't be able to easily get my old partitions back, and even if I could the filesystem would be pretty corrupt.
I could spend some time using other computers in my house to research recovery options. This was a crap shoot at best, and I didn't really expect to be able to get back to a reliable state. I could cut my losses and install a fresh copy of Ubuntu atop a new partition table. Or I could try to tell my wife that the computer was beyond repair and that I'd need to purchase a brand new system.
After several deep breaths, I chose the second option. I quickly started the Ubuntu installation, and sat down to watch some television while waiting for the process to complete. I had performed a mental inventory of the data on my laptop, and felt reasonably comfortable with losing most of what was on there. After a couple of moments, though, I realized that I was losing all of the digital photos of my family. We've taken a lot of photos in the last two years, and only a small portion of those get published to Flickr. I had a tremendous sinking feeling in my gut as I realized the enormity of my loss.
The next day I made an effort to put a good face on things. It's just data. Just pictures. I still had my family, of course, and we could take new pictures. No big deal.
When relating all of the above to Mike, he shared his own digital loss experiences, and mentioned in passing PhotoRec, a "file data recovery software designed to recover lost files including video, documents and archives from hard disks, CD-ROMs, and lost pictures from digital camera memory." It's part of the TestDisk suite, which I saw in passing while briefly investigating my recovery options prior to re-installing Ubuntu. I thought to myself, what the heck: if it can recover some of the files, that's better than nothing.
So last night I booted the Ubuntu live CD, edited /etc/apt/sources.list to ensure that the Universe repository was enabled, and then executed sudo apt-get install testdisk. I installed an empty USB thumb drive, and then invoked sudo photorec /dev/sda, instructing PhotoRec to look at my laptop's entire hard drive.
The entire process was alarmingly simple. By default, PhotoRec finds and recovers a whole lot of file types: tarballs, executables, text files, and more. My first pass with the default options quickly filled the USB stick because it was recovering a lot more than just the JPG files I wanted.
I purged the USB stick and ran PhotoRec again, this time instructing it to only recover JPG files. Again the USB stick quickly filled up! I inserted another stick with twice the capacity and that was filled to capacity. I attached a 500GB USB drive, carved out a 10GB partition -- thinking that that would be more than enough to finish the job -- and even that was filled! So I made a second partition on the USB drive for ~490 GB and let PhotoRec run over night.
This morning, several thousand JPG files had been recovered. PhotoRec can't restore the original file names, but a quick skim through the various directories it creates shows that my photos -- and a lot more -- have been salvaged. Now I can go through the recovered files at my leisure and organize them as necessary for import back into Shotwell on my laptop, or archive to DVD.
A few quick thoughts:
- PhotoRec does a superb job of dealing with out-of-space situations. It doesn't fail, it simply stops what it's doing and asks you where it should store future files. This allowed me to successively provide additional media without having to start over or duplicate the files recovered. I was thoroughly impressed.
- PhotoRec recovered a bunch of images from the new installation of Ubutnu, as I expected. But it also found a lot of photos from my prior installation -- the photos I wanted. It also found a number of photos files from my browser cache, which shouldn't really surprise me but for some reason did. And I suspect that some of the files found were from an even earlier installation of Ubuntu that I had installed over some time ago!
- Data recovery is surprisingly effective. The ease with which this data was recovered has me more convinced than ever that I should start encrypting at least some stuff, and I should definitely be using a secure delete function when possible. For any hard drives I dispose of, it is absolutely imperative that I run them through DBAN first.
I can't praise PhotoRec highly enough.
In 1998 and 1999 I was working for a consulting company doing largely Windows desktop support. I wanted very much to advance my career to the point where I was working on servers and complex infrastructures, and to that end I pursued the Microsoft Certified Systems Engineer certification. I acquired various training materials, and studied hard. My employer paid for the exams, and I passed each of the five tests on the first try. I was delighted with my accomplishment.
Unfortunately, my employer never moved me up, preferring instead to bill me out to clients as a supremely qualified desktop support specialist. It wasn't all bad, of course, because I was assigned to a team that traveled throughout Europe converting a client's setup from Novell to Microsoft. Still, it would have been nice to use professionally that which I had studied for.
Of course, looking back, the late 90s were the time when Microsoft certifications were extremely easy to acquire. It was possible to earn advanced certifications without ever actually using the technologies involved. Indeed, I was living proof: I passed the various Windows Server exams without ever using Windows Server products professionally.
Eventually my MCSE certification expired. I didn't think much of it. It hadn't done much for me, professionally, and I began to view certifications in general with a fair amount of disdain. If you can successfully do something, your work record should speak for itself. You ought not need a certification to demonstrate that you can do a specific job or excel with a specific technology.
Of course, a certification may be a deciding factor between two otherwise similarly qualified candidates. I don't know that that should always be the case, but I know that it often is. And a certification of any kind can help impress hiring managers enough to at least get an interview, from which one's interpersonal skills and work experience should then carry you through.
In the years since my MCSE expired, I began using Linux -- at first personally and then, much to my delight, professionally. I've used Linux at every job since I left that consulting company. I never really thought much about obtaining any of the various Linux certifications, mostly because my previous experience with certifications had been such a non-starter. But also because I have acquired a healthy body of professional Linux experience to which I can point. In addition, I've been a long-time member of the Central Ohio Linux User Group, and have participated in and contributed to numerous open source projects.
I was somewhat surprised, then, when my employer sent me to RH255 - Red Hat System Administration III with RHCSA and RHCE Exams. The training itself was modestly useful. I entered the class already knowing the bulk of what was presented, but I did pick up a few useful tricks, and it was certainly useful to learn the official "Red Hat Way" for a few things. Most useful of all was getting prepared for the two tests I took on the last day of training.
Had I simply taken the exams without the class, I'm certain I would have failed. It's not that I didn't know the material, but rather I wouldn't have been prepared for the testing environment. I wouldn't have been prepared for the kinds of tasks I was asked to perform. Sure, I could have figured them all out given sufficient time, but the exams only afford about two hours each to complete the objectives.
I'm happy to report that I passed both EX200 and EX300, so I am officially a Red Hat Certified Engineer!
Puppet is "an enterprise systems management platform" that can be used to control the configuration of many servers, in theory substantially reducing the management effort of systems administrators. We use Puppet at work, and I was recently tasked with migrating the Puppet service to a new server. The actual migration was easy enough, thanks to a tip from a Puppet user I know.
While I was working with Puppet, I decided to make a couple of small improvements to the way we use it. The Puppet documentation recommends using version control for the Puppet files, and we'd been doing that but deploying the configuration from Subversion into /etc/puppet was an entirely manual process. I wanted to set up a Subversion post-commit hook so that simply checking new stuff into the repository would automatically get pushed to the live system.
The default location for the Puppet files is /etc/puppet. The Subversion repository we're using is stored on the same server, so I made a check-out of our files into /etc/puppet:
$ cd /etc/puppet
$ sudo svn co file:///usr/local/svn/puppet .
I checked out the Puppet repository onto my local workstation, so I can work on the files there, and then commit them back. That works fine.
So I set up the following post-commit hook for the Puppet repository:
#!/bin/sh
# update the local checkout of Puppet manifests
export HOME=/tmp
REPOS="$1"
REV="$2"
/usr/bin/svn up /etc/puppet
I access the repository from my workstation over https using Apache's mod_dav_svn, so the post-commit hook runs as the Apache user. This means the Apache user needs write access to /etc/puppet. That's an easy chmod. But even after that, the post-commit hook failed with a permission denied problem. So I chown'ed the files to Apache, but still no luck writing there.
The SELinux context on /etc/puppet is system_u:object_r:puppet_etc_t, and SELinux's audit log confirms that Apache, via httpd_sys_script_t, does not have permission to do the necessary operations inside /etc/puppet.
Through a series of iterations with ausearch -m avc -ts recent | audit2allow -r, I worked up the following SELinux policy module:
module local 1.0;
require {
type httpd_sys_script_t;
type puppet_etc_t;
class file { rename setattr unlink write };
class dir remove_name;
}
#============= httpd_sys_script_t ==============
allow httpd_sys_script_t puppet_etc_t:file rename;
allow httpd_sys_script_t puppet_etc_t:file setattr;
allow httpd_sys_script_t puppet_etc_t:file unlink;
allow httpd_sys_script_t puppet_etc_t:file write;
allow httpd_sys_script_t puppet_etc_t:dir remove_name;
As root, I built and installed the module using these commands:
# checkmodule -M -m -o local.mod local.te
# semodule_package -o local.pp -m local.mod
# semodule -i local.pp
Now my post-commit hook works as expected: /etc/puppet is updated whenever I commit something to the repository from my workstation.
My questions are:
- is my policy too permissive?
- what alternatives might exist to this solution?
- are there any subtle gotchas that might cause trouble?
This is my first time writing an SELinux policy, so I wouldn't mind some feedback on it. I posted this same question to the Central Ohio Linux User Group mailing list, so check there for follow-ups, too.