17 May, 2003

Coding frustrations with Petunia

I've been trying to work on my pet today and this coming week since my free time is about to be gobbled up by my new job. But I'm running into the same problem I was last time I almost gave up with the project. Here's a rundown. 
The program receives stimulus from any number of inputs. The dispatcher breaks this down into three simple variables:

my ($thischan, $thisuser, $thismsg) = (@_);
 
This way, no matter what our output method is (or where our data came from), or even what language our code is written in, we have a very smooth API for how to process data. At present we have about 55 modules responding in this API. However, some of them take a long time. Herein lies the crux of the problem. IRC wants you to respond to pings every N interval (I think its 60 seconds). Some of these modules are: 
select_map.pm: allows free-form database queries
slashdot.pm: grabs slashdot headlines, greps for older stories on request.
shorten.pm: uses's metamark's url shortening service
translation.pm: attempts to translate text via babelfish
market.pm: all kinds of stock market quotes and the ability to track a portfolio
Sometimes yahoo's quote service times out. Sometimes metamark is down. Sometimes, I want to run a query like:
select count(quip) from log
    where substr( date_trunc( 'day', stamp ), 1, 10 ) = substr( date_trunc( 'day', now() ), 1, 10 )
    and channel = lower( ? )
which might take more than a minute to run. So here is how I'm doing it presently:
foreach my $module (@modules) {
  $module -> run( @args );
}
Each module has a set of "trigger" subroutines. It runs through them to see if it has seen anything it wants to reply to, and then replies if it does. If not, it doesn't, and it really doesn't take much time to make that choice. However, Imagine this interaction: 
 
:last AAPL
Hello, petunia!

Normally, I'd like petunia to run off and grab the quote for Apple, and then say hello. But in this case, she actually has to form the LWP agent, make the request of yahoo's servers, scrape it, and reply. What if yahoo is down? Then the bot never comes back from "last", and the server disconnects her. So what we're looking for is this:

eval 'undef &utility::spew';
eval '*utility::spew = \&async_spew';

 foreach my $module (@modules) {
  $module -> run( @args );
}

sub async_spew {
  # shovel data to the kernel
  my $filter = POE::Filter::Reference->new();
  my %result = (  spew => [@_] );
  my $output = $filter->put( [ \%result ] );
}
 
where spew() was her original output handler, and async_spew() now runs in its own little POE container. I'm having problems, though with POE forking them off, and no longer having access to critical variables. Like the global %config hash, and the $dbh that I had originally passed them. I'm almost getting to the point where a rewrite is necessary, and I really hate to do that.


&c: wha? he can code? I thought all he did was whine about not having a job!

16 May, 2003

PHP is SHIT

mrcsparker: PHP gets "so much shit" because it is a royal pain in the ass. I don't suppose you've ever tried to build php on ultrasparc, 64-bit, on Solaris. As it turns out, you need to build from a snapshot since the stable tarball has bugs in it. Once you get it actually compiled, the 'make test' fails miserably. Basically the problem with the project is it is highly x86-centric and highly windows-centric. When I look at a project like ADODB, I get those godawful every-line-ends-with-^M files, and there are explicit paths that include things like C:\BLAH\ ...

I don't personally have much against the language. I think it can be used properly, and I think it has some technical merits. It does however suffer from the same problem perl does. I'm sure many perl programmers will attest to having been grouped with "duhvelopers." People tend to hate perl because the code that comes out of many perl programmers is awful (see below for more). This isn't a reflection of the language, but rather of its community. PHP has a lot of users out there without any understanding of community responsibility and responsible coding.


arhuman once mentioned on perlmonks (where I am known as deprecated), the book Musashi. While some may disagree with the notion of programmers-as-samurai (there is an extensive thread on perlmonks about this, which I decline to link to), the principle of honorable, learned people who believed in their craft would benefit most programmers immensely.


But I digress.


Job: I now work for British Aerospace. Let me tell you, the last two weeks have been hell. I mean, I've been on so many interviews, every single pair of slacks I own now needs to be dry cleaned. I interviewed with a pile of companies, and it eventually came down to two offers. I wasn't thrilled about either of them. I had finally chosen one when the BAE position came along and just blew everyone out of the water. More pay, 3 minute indoor walking commute (I live in the same building I work in now), and no benefits. (thank you, but I'll invest for my own retirement and get my own credit union and buy my own health insurance thankyouverymuch)

So I'll be hacking perl. My first task involves mostly auditing and fixing 270 perl scripts that need to be running with -T but aren't. Wee. Apparently the software in question is almost a decade old, and needs to be run by the DISA Web Applications STIG (sorry, it is unclassified, but it looks like DISA isn't very friendly). Basically this means they need to be secured to obtain an "Authority to Operate" ("ATO") on the SIPRnet. The STIG (Security Technical Implementation Guidance) is essentially a step-by-step process to secure a

Sounds like a daunting challenge, and something that will keep my very busy, which is the way I like it. I'll also be making enough money to pay off the debt I accrued (it has been six months since I received a paycheck), and rebuild my savings. I'm also thinking about picking up a Honda S2000 (silver, red leather). [ed: not a fucking chance. i'm way too tall]

What a long, strange trip it's been.

job

Got a job. Hacking perl. Doing government stuff. More soon. Yay money.

06 May, 2003

Me and Tom Lane just aren't Friends.

logic: wow, cool. It is really great to see another geeky-type-person interested in cars.


hacking: So a friend of mine calls me last week and tells me that his home network has been compromised. Apparently some romanian guy exploited a hole in his stock suse box's samba (big surprise, that). The erstwhile cracker then showed up on the ircd (running on the box), and gloated about it. He explained that he was going to "patch samba" so that "nobody else could get in." The friend explained that he wanted the network secure, and he didn't know where to start, other than calling me.

The network in question is four machines sitting out in the open on a Speakeasy line, communicating through a hub. Eek. So, I installed the most secure OS I know. Despite my previous ranting about it, and my refusal to use it, my friend is Canadian, and could probably not care less about the politics of Open Source. I decided it would be better for him to be more secure than for me to force him to make some moral stand he didn't agree with. [ed: since I've used SuSE since then, if I'd known then what I know now, I'd have thrown a hamburger at him: Hi, Andy!]


I have to say I was impressed. OpenBSD 3.3 is real nice. I spent all weekend getting everything set up, and he now has a firewalled, natted, MAC-address based access, secure network. I'm very happy. I was so happy with openbsd, in fact, that I went home Sunday night and blew away the Solaris 9 install I had on my Ultra 1 soon-to-be-firewall, and installed an OpenBSD snapshot (network install, mind you, I'll keep my $50). The machine is up and running, and I'm real happy with it.


So, I guess in order to be happy, I really need to swallow the fact that Theo may suck, and the project may not be heading in the right direction (eg away from sparc/alpha to x86), but that some of it is nice (I happen to like Miod and Henning a lot), and the project does accomplish its goals. I've also done what I can to make things better. Three bug reports in the last 12 hours, as well as a docpatch sent in. Wee.

strife: what, more? Yeah. Stuff like this pisses me off. I'll be chrooting postgres to work with Snort and Acid on the aforementioned firewall. Tom has done things like this in the past, essentially saying that "what we have works, if you dont like it, foad." Open Source Software developers need a swift kick in the pants.


job: I have an interview at NIH (linux/solaris administration) on thursday, and may have an offer coming from the pentagon on friday (solaris/irix administration). Seems that putting "Currently possess DSS Secret clearance" adds credibility to a resume. Even for civilian positions. Go figure.

wtf?: mglazer has certified me as master. I'm going to assume he's just being retarded at this point.

02 May, 2003

OCTANE DEATH FROM DESKSIDES

mglazer: wtf? could you please do that somewhere else? 
 
code: I asked ask if he was going to do some statistics for metamark. He said that he had planned to, and that he had some sql in the TODO, but that he had not. He was kind enough to provide the code. However, it wants to talk to mysql, and I'm pretty much a postgres kinda guy.
 
So I know that its sort of one of those jihads among OSS geeks, postgres or mysql. Both camps seem firmly convinced that theirs is better (the postgres camp: "mysql isnt a real database", the mysql camp: "postgres is slow and hard to use"), but I've never seen what I considered an honest comparison of the two of them. So I've been doing that myself now I guess. My first impressions were not good. The build process is fairly easy. After getting it installed, I was appalled to know that mysql has a "root" user and it needs its password set. My first thought was, "oh, christ, I can only imagine how many users out there think that mysql root is system root and vice-versa." (note: I once gained root on a machine by finding the user's "root" user password in the database [for php nuke or something silly], and su'ing with it in the shell)


Next, after the software has been installed, it makes assumptions about where the data should go. In my case, I think this was /opt/alex/mysql/opt/var. This bugged me, because the assumption is that users should keep their data where their binaries are. That's just flawed. Additionally, when trying to find out how to pass it arguments (just passing it arguments did not get a 'usage' message), I had to pore through some bash source.

Ok, so after mysql was installed, and had its own directory (on /global/database/mysql64, naturally), it was fairly easy to get running and poke at in mysql's interpreter. It seems somewhat friendlier than postgres' psql(1), but it wasn't particularly useful to me, since I couldn't issue "help create" ("\h create" in psql). The sql differences are minor, but different enough to be a pain. Additionally, using mysql_setpermission, I was able to "create a user and database," but nothing really happened, so I have an xrl database which I created, and an xrl user that can't connect to it.

I'm pretty frustrated by all this, and the people in #mysql on freenode were not exactly helpful. In fact, I mostly just sat through arguments about why php did or did not suck. I'm probably going to poke on this some more today, but I've just got a bad taste in my mouth from it, and I'm pondering just porting metamark to postgres to be done with it


geek: my Octane showed up. I've been struggling with getting an OS on it. My only wide scsi cable [ed: I now have ultra 320 disks coming out of my ears. the world sucks...) is attached to a raid on my Ultra, so I can't attach a CDROM to it, and Octane's just don't have internal cd's. So I was trying to netboot it off my Challenge, but quickly tired of it. The heat produced by an Octane and a Challenge in my apartment is just obscene, and I could only go at it for a few hours at a time. A friend of mine fedex'd me a new scsi cable though, so I should be able to get media installed tonight without booting the challenge.


pudge is a cool dude. As it turns out, many large networks I frequents have large populations of Mac users, and I'm pondering going file poaching. I suppose its only a matter of time before Apple sets up some cryptographic challenge or something to discourage curl/wget users
 
Job: Well, we have a meeting with DISA for next Thursday, and I've been hearing good things about OSD, but I'm not sure what I'm going to do to come up with another month of money. I'm sick of being unemployed.

28 April, 2003

Why the clearance world can suck sometimes.

Job This has been an insane week. Or rather last week was insane. I got my interim Secret clearance. My Top Secret investigation has been underway for a little more than a month. I'm told it can take 3-9 months to get the TS. However, at the last moment, DISA pulled out and eliminated the position that I was supposed to be filling. I had changed plans to fly out to California, told my whole family I was supposed to be working again soon, and now I'm not working. But I have a clearance. Somebody should really kill -9 ironyd. 

Hi, vivekv.
 
djm: I think it had to be said. Had I not said anything, people would be under the impression that Theo is wrongly accused of being anti-American. In fact, this is a claim he publicly denies, and frequently. It really is amazing to watch him in public and interact with him in private. The act is so practiced you have these insipid fools on misc@ proclaiming their "shame" of being American. Heaven help them if Theo is enough to make you ashamed of being an American


It really is time somebody rescued OpenBSD from Theo.


And by the way, read this.