13 August, 2007

Look, ma! Backup superblocks!


Given it's 640GB and USB2, it churned for quite some time. Note, however, the "classic" backup superblock of 32. One wonders exactly what Disk Utility.app was thinking when it initialized this drive.

Using newfs(1) is probably not the recommended method per Apple, but it looks to me to be a lot safer than their idiotproof clickety interface from Disk Utility. I wish Apple would provide a "... so you're using a UNIX workstation" guide with their computers. You know, when SGI sold Octanes and Fuels and even the Indy's and Indigo's, they distributed both the "how to use from the front end" guide and the "how to use from Unix" guide. That is, you could manage your Octane from the gui, or you could go roboinst and friends.

There's no reason the two have to be exclusive. Even the developer tools (and we developers ostensibly are capable of using you know, newfs) don't include a comprehensive manual for the Unixy end of things. Sure, there's a manual included (see man(1)), but it's not only woefully lacking in places, it's wrong – dangerously wrong – in others. I might be willing to chalk this up to there being a large degree of churn in their Unix backend (e.g., the default shell changing, versions of perl, various netinfo tweaks, filesystem changes – hfs to hfs+case – and so on), but it's just amateur. When I write code (and this isn't to say I'm the standard by which Apple should measure their operating system), I write the documentation first (this is a technique I learned from a guy named saucepan on perlmonks) and make sure that the code fits the documentation and not the other way around. Now, I was doing this years and years ago, but I understand it's now a sort of design paradigm, so I'm not so special as I was back then. However, back in 2001, people looked at you like you were some kind of loon if you wrote docs before code because, well, how could you document something that simply didn't exist? Anyways, I digress substantially. The point here is that Apple could be writing the documentation for their systems and binaries as they are developing their systems and binaries, rather than just wholesale importing them from 4.2BSD (as in the case of newfs.)

Documentation is actually pretty cheap to produce, especially when compared to what it saves you down the line. In this case, I'm bitching about a flaw in their product. In other cases, I might decide that I am simply not going to buy an XServe for my next webapp project because their Unix support sucks. All they'd have to do to fix it is have each developer who is responsible for their neck of the binary woods (e.g., I'm the guy responsible for find(1), therefore I need to make sure it behaves the way the manpage says) sign off on the documentation. In the event they find a discrepancy, while this would seem like a pain in the ass to somebody who was naïve, it's actually a blessing. If you find a bug in your documentation, it probably means that your programmers are assuming that their APIs or binaries are going to act one way when they're really acting another. This is how we get the "grey screen of death" on the Mac.

So here's a question that's not addressed in the newfs manpage: does newfs create an ffs filesystem? Or maybe ufs? HFS? HFS+? Maybe HFS+ with case sensitivity? It's entirely unclear. My guess is that because tunefs(8) is completely hosed:

that newfs, while it claims to be creating ffs:


M. McKusick, W. Joy, S. Leffler, and R. Fabry, "A Fast File System for
UNIX,", ACM Transactions on Computer Systems 2, 3, pp 181-197, August
1984, (reprinted in the BSD System Manager's Manual).


it is actually creating UFS of some sort or other. Which sucks, since Apple's UFS support is so incredibly slow I'd just rather be flayed and fed to hyenas than use it anyways. It does look like, however, that by using newfs to create your filesystem, and mount(8) to actually, you know, mount them, that you can have backup superblocks, and you can tell your filesystem to not reserve X percent for root (do we need this on an iTunes volume?). We can even tell it the expected average file size, number of files in a directory, and all kinds of things. You might even start to think that, under the hood, thar be Unix. However, if it were Unix, it would frickin have an /etc/vfstab where I could actually set up mounts and options and things like that.

As it is, while my original idea of having all my iTunes media on a 400gb single file was, I think, nominally a good thing, the worst case scenario happened: either the disk tanked or the filsystem deposited a fecal patch to the device driver. At any rate, it resulted in taking nineteen hours to copy 400 gig because it had a hard time finding every file. At least it found them.

This time round, I'm going to keep them separate, but have two volumes, and set up an hourly rsync between them. Sure, it means the disk(s) thrash a lot, but I have redundancy, and I don't have to worry, as I did this time, about a single disk losing its mind and having it take 400gb of media with it.

So after all this dicking around with filesystems for two days, we get this:

There's no easy way to tell what sort of filesystems you've got mounted (mount(1) won't tell you, and neither will df(1)). But, df -T fstype will in fact tell you which filesystems of fstype you have mounted. Nevermind that a df -Ta might be useful, because we have the lovely perl construct,


perl -le 'print map { qq,FSTYPE: $_ $/,.qx, df -aignT $_, } map { /^([a-z]+)/; chop and $1 } qx{lsvfs}'


Which only sort of does what I want it to do, and is largely opaque to anyone but me. So, long story short, I've managed to reformat the drive that lost its mind (and redundant superblock), re-fill it with the goodies I want on it (Steve hates DRM, so none of the stuff I have has DRM; nooooo, it's all just, uh, protected for my own good), and I'm now using rsync to make sure that the primary volume – that is the one that the shitfairy blessed recently – has somewhere to leave data, should its coprogenic tendencies emerge again. Only now it's UFS instead of ffs or HFS or HFS+ so it will be slow as hell. Which is what I've come to expect from having a couple hundred gigs of stuff in iTunes anyways. It's just that before, I had iTunes to blame. Now I have to blame their stupid broken Unix. Argh.

And so the score:
Alex: 1
Apple: 1
Shitfairy: 2

Lastly, I could use a tiramisu. Anyone feeling generous, stop on by. All this stress gets my coffee liquor and mascarpone nerves wiggling.

Please Donate To Bitcoin Address: [[address]]

Donation of [[value]] BTC Received. Thank You.
[[error]]

No comments: