27 September, 2008

Writer's block, only worse

Gupta is done. Mostly. I'm going to wrap it probably Tuesday this week, and then it's just a matter of sending it to people. I wanted to work on another short story, or, really, two or three, and get that SFWA membership started, but I've got the exact opposite of writer's block.

I complained about it in the past. When I was trying to finish the last 25% of Limits, this idea I had for a book, Foreigners, got about 15% of the way towards being a book, and of course, neither of them are fit to publish, or even casually hand around.

But here I am today, reading through my slush pile and a novel is eating at me. I swear, it's like I had some fierce Science Fiction Vindaloo and my guts are on fire with the need to write this fucking book. And, with deep guilt I admit this, I've been working on it all day. I've gotten a lot of work done.

I know I haven't got enough free time to write a book right now. I know this giant flow I have in my head right now will not be there tomorrow, or after a beer or two (which is what I normally use to get rid of this sort of distraction), but goddammit, It. Must. Get. Written.

I'm powerless to fight my own goddamn ideas and ambitions. I feel sick. You're my ideas, you can't tell me what to do!

...and I'm not alone. Ugh.

Spaces

I've actually gotten to the point where I'm using Spaces pretty effectively (with two monitors, I keep, for example, Word and Textmate on one space, Browser and email in another, iTunes and a Terminal in another, and so on, and lock them down in the control panel).

But Spaces really confuses Word, especially things like floating palettes. And iChat is pretty confused, too, since it may have a dozen windows open over several spaces. Which one does it go to when you alt-tab to it?

iTunes, Enterprise Edition, solved.


IM IN UR TOKYO3 STORIN UR CHOONZ

short: how to "fix" the "iTunes problem"

I've been bitching more or less incessantly about iTunes and file management, the way it stores data, and things like this. Originally, my complaint was that I had so much "stuff" that I wanted iTunes to manage it, make appropriate decisions about how to chunk and stream it across the network, how the underlying file structure should look, and its very, very serious lack of any robust query language (you can search for a word, or a tag, or whatever, and then make recursive searches of sets, rather like views in a database, but it's horribly coarse, and doesn't provide an interface to the filesystem, e.g., I cant get a cksum(1) of a file vs another). And, while it has a "show duplicates" function, it's too stupid to understand that I have, what, four different copies of No Phone by Cake, and they're all in fact different.

The other thing is, I was complaining I wanted to be able to store all my audio in one place, all my video in another place, all my iTunesU stuff in yet another place, audiobooks, and so on, down the line. You may not understand why I feel this way, but it boils down to essentially this. I need to know where stuff is so I can make decisions about how frequently to back something up, whether to back it up at all, what kind of media to put it on (RAID5 vs just some spare disk I've got lying around), whether to have Time Machine look at it, and so on. There are other reasons, but I think this is sufficient enough (I could even have network volumes store data for iTunes as I mentioned in my recent ZFS rant). Filesystems are very important to people who have lots and lots of data. How much is "lots"?

Well, after doing lots of manual and awk-aided cleaning and reaggregating of media files, the total is presently 891gb. That was a few days ago. We've since added all of Neon Genesis Evangelion, a few movies (the Mad Max movies, for example), a season of House, and so I would imagine if we haven't actually reached a terabyte, we will by next week.

But I've kind of solved my own problem, if you can believe that. iTunes has the option to "copy files to my iTunes library" when you add them, or to just leave them in place. It gets kind of interesting here, so pay attention or walk away because the details get tricky.

So, I had a lot of content attached to a RAID5 of 250gb LaCie disks (which have a very bad rap; ours have been running fine in hideous conditions for literally years), which was mixed. That is to say, video, audio – books, iTunesU, music, audible, etc – and I guess the other weird things that iTunes saves, like iPhone apps and iPod games, as well as PDFs that come with some albums from the iTunes Music Store.

This content worried me, because when I ran through it with awk looking for a way to get rid of duplicates, I noticed that I had not just duplicates, but in some cases quadruplicates (and in some cases, these "extra copies" were not showing up in the iTunes library, so while iTunes would tell me I had 339gb of content, I actually had a lot more on disk, and that extra stuff was duplicates – or was it?).

The problem I had was, well, once you've got duplicates noticed, how do you go and clobber them? I ran checksums on everything (this took a long time), and then used uniq(1) to show me the duplicate checksums. There were a lot. So, I created a new iTunes Library, this time letting it manage the files and structure, and using one monitor with vim open, ran through the duplicates, which roughly corresponded with some places in iTunes where duplicates were obvious. There were other cases where I had a 64kbit file and a 192kbit file, and of course since their checksums aren't going to be the same, having iTunes actually display the bitrate (among other tags) made the task of going through and eliminating lots of stuff pretty easy (as opposed to coming up with a tool to do this in the shell, I did it by holding down shift or command and zapping stuff with the backspace key). It took about eight hours, but I got all the duplicates out.

Another problem we had run in to is that, since we switched from the Mini to the Mac Pro, we've added a lot of content. Sandy's been adding content to her computers, I've been adding content to mine, and we've just sort of been storing them on disk, and had planned to drag and drop the whole deal into iTunes and just let it sort it out. Before doing this, however, I took my "clean" library, chown(1)ed it, and made it read-only. I then created a new iTunes library, this time telling it to not manage the file structure, and started cleaning up the content I wanted to add to the "new" library, segregating it into different folders, like "Sandy's Bleach episodes," "new itms purchases," "wmv files that need to be converted," and such. I kept these on a separate disk.

When I was satisfied with that stuff being relatively clean, I started adding each of these folders into iTunes so that they'd be available on the AppleTV, and began "cleaning" some of the new, imported content (like the anime) with the iTunes interface (which is, admittedly, pretty good for manipulating entire albums and ID3/attribute tags on files).

So, suddenly, we had all our old "clean" content, our new "clean" content, and our "imported, then cleaned" content available, in one single iTunes library. One day as I caught a duplicate I had missed, I used the "reveal in Finder" function to realize that, because iTunes is not managing the names and directory structure they're still (in the case of Bleach, named "[DB]...") or whatever. But, wait a minute. My library now spans the "original" raid, a new 640gb disk, a new 500gb disk, and the root volume, which is, uh, whatever it is, as well as Sandy's external pocketdrive.

I realized I'd gotten what I wanted: I could now store my media wherever I wanted to store it, add it to iTunes, and have iTunes organize it logically in its own little xml file. This is a huge plus, and it's a huge minus.

First, I got what I wanted. This is great. The problem is, since iTunes hasn't actually "corrected" the files, it's just corrected their entry in the xml file (the "iTunes Music Library"), the files on disk are still a mess (except the read-only library). So, if something were to happen to that xml file, I'd lose all the work I've been putting into this project for three weeks or so.

There is a solution to this, though. iTunes has a "consolidate library" function under File > Library, which will essentially take all these files on all these separate volumes, organize them the way it would prefer them to be organized (monolithically, on one disk, with "TV Shows" stupidly stored under "iTunes Music" along with movies and audiobooks, etc), but in the process it would also do what I was going to have the hardest time doing: writing a program to clean up all these files we had sitting around.

Ostensibly, I could pick up a LaCie Biggest Quadra, say, a 4tb model, tell iTunes to place my library there, and "consolidate" this library to that drive, and then keep up the same trick: triaging content into iTunes, groups of content at a time, using "consolidate library." This is kind of a problem, because I don't really trust iTunes, I don't trust Software Update to not fuck up that (presently very precious) xml file, and I don't especially want everything living in one gigantic 1+ tb directory. But despite all that "don't like" and "don't trust," I think I like the idea of iTunes cleaning up all that music, putting it on the (a new raid, the existing raid, whatever) raid, and making it available to Sandy and I (we use an "iTunes User" so that we don't have conflicting libraries) to manage iPods, iPhones, and the Apple TV.

So, I've kinda gotten my wish, but as is usually the case with Apple, it's fragile, consumer-grade (note I did not say pro-sumer), and Not Supported (imagine printing this post out and trying to explain it to a Mac Genius. Their eyes would bug out of their head.). The attitude of "if you want to use our hardware to do hard stuff, don't blame us when it breaks" sucks a lot, but at least I've got a workaround, a possible future solution, and "what I want" for the moment.

Motorcycle updates

quick: if you don't care about motorcycles, move along. there's some newbie wisdom here, if you can believe that, so if you're a new rider, you may learn something.

I rode the Ninja home yesterday, in the rain, because I got caught working a little later at work than I had planned to. It was mostly uneventful. I treated the Ninja just like I treat the STI: don't push it like there's dry asphalt under you (my plan for the STI is/was to run the Falken Azenis as winter tires and BFG slicks as summer tires; people think I'm insane, but I mostly think anyone who "needs" a better tire in the winter is being idiotic. You drive like it's wet, or snowy, and you won't lose traction. Tires ain't gonna help if you're stupid.). I had worried about my visor, but I found that, generally, the rain did a good job of flowing to the sides, and each time I checked a corner, it sort of "cleaned" that side of the visor (so check both your corners, and it's mostly rain-free afterwards). I suspect the visor is coated and I'll have to replace it at some point, but that's probably going to be a ways off. I hope. Gear is expensive.

So I leaned a little less, and kept my eyes open for puddles (motorcycles are real good at hydroplaning) and everything was mostly uneventful other than the fact I was wearing regular-ol-jeans, and those got kinda wet. The boots (Vasque, waterproof) were sufficient, although a bit clunky for operating a gearshift, and slippery when trying to get a good hold on the rear brake. The jacket, a Joe Rocket Phoenix II, with its vinyl insert (the jacket is otherwise a mesh textile, so air flows into it, keeping the rider cool) was sufficient to keep me dry, although it was hot as hell. I rode in to work with a sweater on under it (it was about 66F) and felt fine, but the ride home, I kept the sweater in my backpack and was way hot (ambient was 77F). For whatever that's worth.

I also performed, accidentally, my first "wheelie." I've learned a lot on the "little Ninja" in the time I've been commuting on it. First, it's not the STI. So driving around at 6,000 rpm is nowhere near as horrible as it is in the car. As a matter of fact, 6,000rpm is probably the least I want the engine running on city streets. If I need to throttle up (say, 55 to 70mph) and I'm starting at 6k, it's gonna be a slow curve until the bike "comes on" at 9k, and then it moves pretty durn quick.

So, from a stop, and pulling into traffic doing 60, I've learned that I just need to drive it hard. I shifted at about 13,000 rpm, which dumped me right back down at 9,000 rpm, and the front wheel (mind you, it's raining, and dark) lifted what felt like four inches. Maybe six. But not more. This, I'm told is called a "power wheelie," and should serve to indicate to new riders that even a 250cc Ninja, with a rider who weighs almost as much as it does, has a lot of power on tap if you operate it correctly. So I'll be paying a little more attention to when I'm shifting, and making sure not to do that through turns (thankfully, this was on the straightest part of the ramp).

The other thing I learned is that my buddy Colin is right. He says he never gets on the bike if he's angry, depressed, has had a drink in the last few ("few," not "one") hours, and so on, because the bike requires 100% of your attention, all the time, and if you don't have it all there, you can get killed. A personnel security issue at work was the culprit that kept me late, and I was irritated enough and preoccupied enough with the situation that I almost dropped the bike three times just coming out of the parking garage. By the time I got to the ramp, I was ready to call Sandy and ask her to just pick me up because I couldn't get into the focus I needed to ride. But, I stupidly plowed forward, into the rain, into the dark, and rode. I think what saved me in this instance is the fact that the new (I'd never ridden in the rain, and I try real hard to avoid driving home in the dark) challenges seriously engaged my attention. When I got home, I was still super angry, but when I was on the bike, I was checking corners, watching for cracks and puddles, minding my lean angle, and so on.

Next time I will just wait it out at the office, even if it means a few hours or some yoga, or indeed just ask for a ride home, because no argument or security issue is worth dying over.

We have purchased, this time with Hard Cash (rather than stock that's falling in a nauseated market), a Ninja ZX7-R (I'm not sure whether it's the R or the RR – I've been told the RR is "unstreetable," but there are few visual differences between the two). We're going to bring it home when the rain clears up. It's red, and looks a lot like this:

 
Which astute readers will notice is a hell of a lot more bike than our 250. It's also fully faired, and so should offer me a lot more protection from wind buffetting, and should also give me somewhere to "tuck in to." It's an unusally long bike, compared to everything else I've sat on or ridden, and I feel kind of strange reaching so far forward, but that gets me under the windscreen, and so on. I guess by design. It's a 750cc, makes about a hundred at the wheel, and kind of ironically, has nearly identical numbers to the 2008 CBR600RR "on paper."

The balance of the bike is a lot more forward than the CBR (which is magnificently balanced), with most of the weight being under the tank, or your chest if you picture yourself on the bike. It's also got phenomenal front brakes, which means one has to be careful when applying them. Of course, I haven't mentioned that it has a similar power curve to our little 250, and it comes on at about 8,000 rpm, and it will pull all the way to about 180mph. It's a fast bike.

The things I like most about the bike is that it's been taken extraordinarily good care of (still not sure how I feel about "persimmon red" – Airtech fairings in satin black may be in order), and it is 100% stock. Lots of the bikes on the used market have tons of modifications. This bike is cherry (no pun intended). It's also a very big bike. Sandy can't even get a leg over and triangle the bike, so there's no question of whether we'd be sharing this bike. And, while the power is frankly a little daunting, I don't think I'm going to have any more fears about getting into a fast-moving bit of traffic. She should be audible from a few blocks away if I get her up to 14k before I shift (and because all the weight is so far forward, it's actually pretty hard to wheelie the ZX-7R, although you can do it if you work at it). It will be nice having a sixth gear.

I've been thinking about getting a Yoshimura slip-on for it, just because it looks cool, but the bike really needs nothing. It's going to be interesting having my first "wet season"/winter riding experience being on what is essentially a MotoGP bike sold to the public for homologation purposes. At least I don't live in Pittsburgh or something.

As far as gear, since I never take the bike(s) out on the highway, I've not felt the need to pick up any real riding pants, but have ordered (arriving I guess this coming week) a set of Coretech denim "leathers" – they're actually denim, plastic, and leather – and a set of Sidi Doha boots. Don't want to come off the bike at all, but if I do come off her at 75, I want to make sure I've got ankle protection and I don't burn my ass off on the road.

And, lastly, the more I look at gear on Kneedraggers, the more I realize I don't really trust my Joe Rocket jacket, and I'd like to buy both supplemental back/kidney armor and a new jacket. But jeez, gear is expensive. Imagine, spending $500 on a new jacket, plus $500 for a new rain jacket, plus $250 for the armor, and another $150 for rain pants (that fit over your riding pants).

The Joe Rocket is functional and will work for commuting. But I'm under no illusions that either my Icon gloves or my JR jacket will offer me much protection at anything above surface-street speeds.

$6 tanks of gas are awful nice, but no matter what anyone tells you, riding a bike is not cheaper than driving a car. And nobody who rides a bike owns just one, either. It's an affliction.

I'll find somebody to get a picture of Sandy and I on the Ninjas so those who keep asking about it can see (her pink leathers, my "demon bike," the size difference, etc).

26 September, 2008

Media, stunning ignorance therein

A young black man has been paid by NASA to rap about physics. Apparently, even that paragon of journalism excellence, the BBC (well, okay, it was the Wales bureau...), has never heard of nerdcore, but even that's no excuse, as They Might Be Giants and The Dead Milkmen were recording such "nerdy" music when even I was a kid (wasn't my thing back then; fan of TMBG these days). I am undoubtably leaving things out, as people have been putting "learning material" to music for as long as there has been music, as far as I can tell.

In fact, if you want to be pedantic about it, rap started as oral history. That would mean that rap, by definition, is nerdy (is Ice Cube's It Takes a Nation, Hood Mentality, or Tomorrow nerdcore because it aims to educate its audience?).

And then, of course, there's Schoolhouse Rock, my favorite piece of which is actually a cover by Shannon Hoon (and you can listen to it right now!).

Holy interwebs batman! It's amazing what you can find if you look for it. I'm stunned that the BBC didn't even bother to note that this has been going on for as long as anyone cares to look back.

23 September, 2008

Interactions between Unix and Windows (or other directories)

short: it rant

Is it really necessary that Unix and Windows interoperate on authentication (the fabled "single sign on")? I would say that the answer is a definite "sort of." I'm real pleased with Solaris 5.11 and 5.10, and I've been thinking about running Windows, virtualized, within Virtualbox on Solaris on AMD64. This way, I get end-to-end support of my hardware, software, and virtualization environment from Solaris (the proverbial "one neck to choke"), and frankly I trust Solaris a lot more than I trust Windows, when running services.

And, let's be real. We don't run Windows machines as services. Services run on Windows machines. A Unix machine on the network is a service; it's a place people can shell into and do stuff. A Windows machine on a network is an IP address that might let you interact with it over metaframe or terminal services, but it's only barely multiuser and even then, it's just as if you were an ape at the keyboard.

So why not take your directory server, file server, and so on, and put them in very small, very lean Windows guests in Virtualbox, and then mete out the storage with ZFS?

I was initially impressed with our local EMC rep (who, incidentally, owns VMware, which I didn't know). But maybe that's because she's pretty. She had some pretty interesting things to say about Exchange and replication and things like this, but that's Sun's game, and has been (well, the replication part). If you want Exchange, great, run it in a virtual host with sixteen gigs of ram and a terabyte of disk on a ZFS volume, and then tune that volume from the Solaris side as necessary.

So all your guests can use Active Directory to talk to eachother (and AD itself is virtualized, remember) and all your users, whether they be Mac users or Windows users, get their "single sign on" to their email, machines, VPN, network storage, and so on, via AD (because, for whatever reason, nobody seems to like Sun's LDAP service). And your Windows admins (because you need em, you have lots of Windows "boxes") can all do their authentication and privilege management through Windows, which is what they want to do.

But, do you break down that last wall and make sure that anyone with a Windows account should be able to log into one of these Solaris (or dare I say Linux) machines? Maybe those Solaris machines should use krb or Sun's LDAP, or maybe even a shared set of keys (depending on your organization, how many 4x4 core Opterons do you need to run all your Windows services virutally?), to keep it simple.

Bottom line: Windows admins don't need to be on the Solaris machines, Solaris admins probably don't want to be on the Windows machines anyways, but understand what it means when the Windows guys say "Exchange needs three more terabytes." ZFS. Bang. "Sure, I'll get that done when I'm back at my desk."

I just don't trust VMware and I don't like it. It's burned me too many times, both on MacOS and Linux, as well as on Windows. Why let it continue to do so when Virtualbox is free?

Of course, I'm just me, an irascible systems admin who asks that systems be stable, scalable, and built well. Piddly shit and hacks piss me off, and that's earned me a reputation as an asshole. Sure, maybe I am. But then, the networks everyone else is making are falling apart at the fucking seams. I don't mean anyone in particular; I just mean, I've never met a Windows environment that seems to, you know, Just Work (except at Microsoft, which was blissful, I have to admit).

But even Microsoft would admit they'll happily sell you licenses to 2003R2 or 2008 to run virtually on Solaris – if you really want to. They make the money, and they support virtualization. They do, and their sales reps will tell you. So trust their OS to do what it does: one thing, at a time, with nothing else getting in the way (like, I dunno, a second nic driver or running three virtual guests).

Fucking Windows. I understand its place in the world, but I am so continually disappointed by its lack of administration tools, operator transparency, stability, and so on, that as I work with Solaris (now daily), I can't help but be disgusted – doubly – because my Mac treats me better, too.

21 September, 2008

OPSSEC and the family

It is unfortunate that one's professional life can become intermingled with ones personal life in such a way that said professional could have to explain operational security to un-read-in people.

When and how and where to talk. What to say, and not say, whether or never. What you can and can't say about your spouse at work. Where you can and can't go on vacation.

But these are facts of life, I guess, in the community. Your kids may not realize that torrent they're downloading only has one seed, and it's in Iran, and what that actually means. They may not understand that telling a friend of their at school that you're in Saudi Arabia for two weeks can literally filter into the intelligence-collection funnels of a "hostile" (at Microsoft, we used the term "coopetition" to describe Citrix – you may see the same sort of relationship here with, e.g., Israel) country or other entity.

After the lectures are done, and the cloak-and-dagger stuff is over, and the family is suitably scared but hopefully reassured Everything Is Okay, life returns to almost normal. But, it's sad that it has to happen at all.