What happened was that I committed some changes. Pretty normal. Then I realised I forgot to take something out. I committed that, with an "Oops, I should have included this in the last commit too"-style message.
I'm using Ubuntu 7.10, which has an old version of git without interactive rebase (I'll switch soon, I swear!), so I couldn't use that. After a rather dopey hour of searching how to use rebase to collapse the latest two commits, I opted for a different tack.
Turns out all I had to do was the following:
git reset --soft HEAD^
git commit --amend
The first line sets the current HEAD to the second-last commit, while keeping the changes I wanted in the index cache (i.e. the removal of the stuff I wanted to take out). The second makes the correction. Result? The latest intermediate commit is collapsed into the second-latest, so I don't have a messy history.
Of course, this is rewriting history, so I wouldn't want to do this for a public branch, but I'm the only one working on this small project anyway, so it's fine.