Wow. After I stopped directly replying to His Specialness over on lwn (because it just wasn't accomplishing anything), and then later stopped posting to the lwn.net thread at all (well, except for this) when Jonathan Corbet (editor of Linux Weekly News) asked us to stop. (Then I fell behind on email and twitter for a couple days because it's christmas and I was busy, so I'm just catching up now.)

But other people didn't stop posting, and I still get emailed the replies to replies because I checked the little boxes, and apparenty His Specialness has decided that anyone who disagrees with him must really be me in disguise.

I find that hilarious. When have I ever been reluctant to insult him to his face? The only reason I've stopped using His Specialness's name is I think he's a glory hound on the order of those white house party crashers, balloon boy's parents, and that toupee that used to govern Illionis. Getting his name repeated seems to be what he wants, with context in which it's repeated a very distant secondary consideration. I suspect the only reason he hasn't done one of those reality television shows where you eat a bug on camera is they wouldn't take him.

LWN doesn't seem to have user profiles, so out of morbid curiosity I spent the 30 seconds to type "site:lwn.net syspig" into google, which found a comment from 2006 on its first page of hits. His Specialness apparently did not bother to do this.

I repeat my comment from the last big post I made to the thread before the moratorium: "I'm tired of arguing with people who don't bother to do their homework". (And I note yesterday's squirrel analogy on the difference between clever and smart.)

I'm trying to break the toolchain generation out of root-filesystem.sh and give it its own script. Right now, build.sh can do up to four toolchains: the simple cross compiler, the static cross compiler, the static native compiler, and the dynamically linked native compiler that's part of the final root filesystem.

In theory, that should all be factored out into a single "build a compiler" script that can do all four variants based on config flags. In practice, the root filesystem might need to build uClibc even if it hasn't got a compiler, if stuff in it is dynamically linked. (But in that case it just needs the *.so files, not the *.a or *.o files or the headers. And don't get me started on libgcc_s.so.)

I suppose the static native compiler and the dynamic native compiler could be collapsed together. Static linking against uClibc isn't much overhead and we're talking 13 megabytes of compiler and headers and such already, so micro-optimizations are at the wrong scale to matter.

But that's still three separate roles for one script to perform, and they expose complexity in root-filesystem.sh that's been glossed over so far. So the question is, what user interface should all this have?

Once gain, easy to do, hard to make _clean_...

Ok, building the simple cross compiler is:

# Build binutils, gcc, and ccwrap

FROM_ARCH= PROGRAM_PREFIX="${ARCH}-" build_section binutils
FROM_ARCH= PROGRAM_PREFIX="${ARCH}-" build_section gcc
FROM_ARCH= PROGRAM_PREFIX="${ARCH}-" build_section ccwrap

# Build C Library

build_section linux-headers
HOST_UTILS=1 build_section uClibc

Building both the non-simple cross compiler and the native compiler are just:

# Build C Library

build_section linux-headers
build_section uClibc

# Build binutils, gcc, and ccwrap

build_section binutils
build_section gcc
build_section ccwrap

The difference being how PROGRAM_PREFIX and FROM_ARCH are set.

So basically, if FROM_ARCH is blank, build the C library first, otherwise build it afterwards, and leave the setting of those two variables to the calling script? Or should that information be on the compiler.sh command line? And then there's distinguishing the static cross compiler for the host arch (prefixed) from a native compiler (unprefixed).

Hmmm... Implementation's fairly straightforward, figuring out a sane UI is still the hard part.)

Got my big ccwrap.c update checked in yesterday, and today I taught the build_section logic to run package.build and package.sh (the first does setup/cleanup for "package" automatically, and the second you do your own setup and cleanup. Basically decided to automate only the easy cases, and let the others set themselves up manually. (So any package that uses a separate build directory sets itself up so I don't have to figure out how to pass in the extra data, for example.)

During the upcoming disentanglement of the download/extract/setupfor logic I might strip out the build directory logic from setupfor and just have the script do that by hand. (mkdir ../thingy isn't hard, and I already have to list the build directory explicitly when doing cleanup, so just having that be an rm -rf lets me remove that logic from the cleanup function.)

The old "repetition vs clarity" debate rages on...

Why am I Arguing with The Bruce again? Come on, I know better, I even own one of these.

Um, actuallly I think I gave my copy to Beth. Right, I need to get a _new_ one of those and hang it somewhere prominent. (Scan it in and use it as desktop wallpaper, perhaps. Or just use the original strip...)

I'm amused that Bruce and I can oppose the lawsuit for such completely different reasons. I'm also somewhat amused that Bruce has been flogging his rant (which I still haven't read more than the first few sentences of) more or less as a press release, of _vital_ importance for the community at large to read, and it's gotten all sorts of coverage about how he was never involved and still isn't involved and is all indignant about it.

Meanwhile my blog entry explaining that while I used to be involved I've stopped, and my reasoning for doing so (based on actual inside information), is not on slashdot or LWN or such. (I didn't expect it to be, but it's probably more informative than anything Bruce has to say.)

Oh well. Self-promotion is Bruce's thing. Me, I write code instead.

There are times I want to slap Ulrich Drepper.

The fix he claims doesn't exist is to "#undef _FORTIFY_SOURCE" before your #includes, because complaining that I'm not checking the return value of asprintf() is just _stupid_. (The malloc only fails if you run out of _virtual_ memory, not out of physical memory. Exhaustion of physical memory happens later when you dirty the pages, and triggers the OOM killer which you can't do much about. Yes, nommu can have a harder time of it but that can fail allocating your program's _stack_.)

And no, the suggested fix of introducing an unnecessary typecast to (void) on each occurrence no longer seems to work. I'd guess they found out that people were actually doing it, so they removed it.

(I had a macro wrapper that was going int unused=asprintf(), but then it complained about an unused variable. THERE IS NOTHING I WANT TO DO WITH THIS VALUE. GO AWAY.)

And you wonder why I want an alternative to glibc? You do not know what's best for me, and if you try to give me no choice you can't take away the choice to use a different implementation than yours. Idiots.

Some people are assuming I'm involved in the most recent round of BusyBox lawsuits. I'm not. I lost faith in the SFLC's judgement about what is and isn't a good lawsuit to file back when they filed suit against Cisco for a 5 year old toolchain that Cisco got from a vendor (Broadcom) and which predated the original BusyBox vs Linksys lawsuit circa 2003.

The result of that was that Cisco's big "Strategic investment in Linux" ended. Cisco had brought me in to consult on that, their plans to base every single new product on Linux, and schedule a week of license auditing and publication work between the time they got a golden master image and the time it actually went to manufacturing. It was an aggressive, "bet the company on Linux" move, and the week the suit was filed it went from "our first project using this new process will be the in-development MediaHub" to "our corporate use of Linux at all 'is under review'". A few weeks later I was told the CTO who had been behind the big Linux push was "spending more time with his family" or some such (never did get the details). And then a few months later, the entire Linksys division was reorganized out of existence (at the end of August). The brand name still exists, but the engineers I knew have all left the company (rather than be reassigned to work on non-Linux projects).

The remaining Linux support (for existing products that already shipped) seems to have been outsourced to Red Hat. Last I checked (a few months ago) I couldn't find any Linux work Cisco was still doing in-house, and that massive 180 seemed to have been in direct response to the "Mepis II: Even Dumber" lawsuit the SFLC filed on behalf of the FSF.

The toolchain in question came from Broadcom back around 2002. Cisco never _had_ the source code to it, because Broadcom refused to give it to them. The SFLC never made Broadcom a party to the suit, thus the company that was cooperating with the Linux community was punished and the one that wasn't was ignored.

The only reason that toolchain hadn't already been replaced was that after the first lawsuit back in 2003, Cisco washed their hands of Linux for five years and outsourced maintenance of existing projects to Taiwan, and switched their new routers to VxWorks instead. They had just gotten over that a few months before this lawsuit was filed, and their new projects were using toolchains built from source in-house (instead of supplied by an outside vendor).

The toolchain in question contained no interesting IP, projects like OpenWRT had been happily targeting that hardware with fully open source toolchains for _years_. From an engineering perspective, there was NO REASON to file the lawsuit that got filed, unless you wanted to STOP Cisco from cooperating with the Linux community and doing new Linux development. Which seems to be what happened. It was INCREDIBLY STUPID, and destructive, and massively counterproductive, and it made the community look bad.

That's why I'm not involved in the current round of Busybox suits. I have zero faith that the SFLC understands which lawsuits are in the interests of the community, and which are the equivalent of patent trolling. Humoring the more zealous and clueless excesses of the FSF's idealism division (the parts that call Tim O'Reilley a "parasite" for producing good documentation for sale, and say that having a day job writing proprietary code is "a sin")... That's not helpful to Linux. We have enough trouble distancing ourselves from the lunatic fringe as it is.

Erik Andersen's still going along with it, although based on the rounds I was involved in, I'd guess his involvement is limited to signing whatever paperwork they send him and mailing it back. Erik hasn't had any real involvement with BusyBox since about 2006, he's busy with other things. His last commit to the busybox repository was October 19, 2006. He hasn't posted to the busybox mailing list at all this year, and in 2007 and 2008 combined he posted a total of 5 messages: one about the lawsuit, twice under the title "New mailserver test -- ignore", and twice asking people to delete files off the server when its disk filled up. (I note that OSUOSL reinstalled that server and took over its administration a year ago, after it got cracked due to lack of maintenance.)

Last time I talked to the SFLC about this I suggested they get current Busybox maintainer Denys Vlasenko involved. They declined. The current Busybox project has no connection to these lawsuits that I am aware of, and the SFLC seems to prefer it that way.

Oh, one other fun detail: the code drops we did get from the rounds of suits I was involved with? As far as I know, it didn't result in a single line added to the BusyBox repository. From an engineering perspective, the entire exercise was _completely_useless_.

By the way, I got a blanket email monday sent out to all "Conservancy" members (I'm still on that mailing list, apparently), which contained this chunk:

However, if you do speak or write publicly about this, please be very clear in such communications that you only speak for yourself and/or your project, but not for the Conservancy as a whole. Also, we would appreciate it if you would avoid commenting specifically about the violations, products and the companies mentioned in the litigation. Finally, if you want to avoid this matter entirely, you are absolutely welcome to refer inquiries related to GPL enforcement and litigation directly to me or to SFLC's press coordinator, Lysandra Ohrstrom .

Obviously I'm not speaking for the SFLC, or for BusyBox. I haven't even read the new press release, so I don't know who they sued this time, or over what. I think the last time I had any communication with them was about 6 months ago, but they kept switching who it was I was talking to, so searching for the email would be more trouble than it's worth.

I dunno how "public" a blog entry is. (My sick cat is doing much better, thanks.) But yeah: not involved this time, done enough damage already. Driving companies away from Linux while giving the FSF's screwball division a bigger soapbox is not what I signed up for, so I didn't re-enlist for the current round.

Update: I have been pointed at this guy but haven't got the stomach to read it. That man is the reason I stopped doing BusyBox development.

As for why the SFLC chooses not to include him in the lawsuits, perhaps the fact he was proven not to have any code remaining in it years ago, let alone today might have something to do with it?

Bruce abandoned BusyBox two years before Erik Andersen picked up the dead code and re-launched it, and then Bruce didn't bother to post or subscribe to Erik's new list for most of a decade (until he suddenly showed up not to contribute code but to be a license troll, at which time Bruce's web page said that BusyBox was maintained by Lineo, a company that Erik had left something like seven years earlier).

It wouldn't help the SFLC to involve Bruce because he has nothing to do with BusyBox. That's a significant reason BusyBox has been as successful as it has: Bruce's _lack_ of involvement in it.

If any of the above comes off like I'm dissing Erik: I'm not. Erik Andersen is the reason BusyBox exists today. He could just as easily have grabbed Red Hat's Nash or started his own new codebase from scratch, that just wasn't his style. Erik wandered away from BusyBox development partly because uClibc and BuildRoot were eating his time, and then because he had his own company to run and a family to care for. (Erik continued to administer the server it ran on even after that, albeit somewhat distractedly.)

Erik found a new maintainer (me), just as I found a new maintainer (Denys) to hand the project off to. Bruce let his version die, and the current one is at least as separate from his version as modern BSD is from the Bell Labs code of 1975. Whatever else may be going on, the SFLC is right to keep itself away from Bruce. If only they'd kept themselves seperate from the rest of the extremist wing of the FSF...

Ha. Changing /etc/acpi/powerbtn.sh to this means the stupid hal/dbus layer doesn't get a chance to veto the suspend:

#!/bin/bash

chvt 1
sync
echo mem > /sys/power/state

I should probably swap that in for sleep.sh as well so the low battery stuff doesn't go "yesterday, VLC was playing some media, and it didn't notify HAL when it _stopped_, thus it seems to think it's playing three media streams simultaneously when in fact VLC isn't curently running. So refuse to suspend, instead just let it run out of power and have the hard drive do its emergency park."

And you wonder why the drive's got a bad sector on it...

I would like to find the developers responsible for this:

        char inFile[80]={0}, outFile[80]={0};
...
        sscanf(argv[2], "%s", inFile);
        sscanf(argv[3], "%s", outFile);
...
        // Read data and generate header
        fh = open(inFile, O_RDONLY);
        if ( fh == -1 ) {
                printf("Open input file error!\n");
                free( pHeader );
                exit(1);
        }
        lseek(fh, 0L, SEEK_SET);
        if ( read(fh, buf, status.st_size) != status.st_size) {
                printf("Read file error!\n");
                close(fh);
                free(pHeader);
                exit(1);
        }
        close(fh);

And I would like to ask them what they were smoking. The unnecessary lseek, printing error messages to stdout instead of stderr, closing and freeing data right before exiting the program... Those are just noise. But going out of your way to _introduce_ a buffer overrun like that is kind of impressively bad. (It's from cvimg.c by some guy at realtek, circa 2004.)

There are times you look at a program and just start over from scratch. This may be one of those times.

So the larger contract I'm working on right now is to come up with a decent board support package for a router that's using "not mips" processors. The relationship between Mips and the Lexra 5200 is a bit like m68k vs coldfire or PowerPC vs ppc-440, except with more historical acrimony.

Once upon a time a company called Lexra made a mips clone, based on the freely published MIPS instruction set. Unfortunately, four of those instructions had patents on them (unaligned loads and stores, because obviously nobody have ever done _that_ before 1986). So Lexra removed those instructions from the chips they designed, coming up with entirely original circuitry implementing a less efficient but patent-free subset of the original MIPS instruction set.

Alas, Mips still sued them, arguing that when the chip encountered these "illegal" instructions it threw an exception, and somebody could write an exception handler in software that would then implement the original behavior (even though the hardware didn't), and thus the chips violated the patents _anyway_.

The lawsuit dragged on for years. As far as I can piece together, Lexra made progress convincing the PTO to invalidate the patent, but MIPS still tied Lexra up in court until the smaller company ran out of money, then MIPS more or less bought them out with a large cash payment and a free MIPS license in exchange for Lexra voluntarily going out of business.

Really. Here's a quote from that last link, the joint press release Mips and Lexra put out about the settlement. It is an impressive piece of corporate-speak as you'll ever see:

Lexra will shift from the intellectual property (IP) licensing business and become a fabless semiconductor company while at the same time licensing the MIPS32 architecture for use in its upcoming network infrastructure chip. Under the terms of the license agreement, Lexra acknowledges the validity and enforceability of MIPS Technologies patents. As Lexra is exiting the IP licensing business, Lexra will assign its processor IP assets over to MIPS Technologies and both companies will work together to convert Lexra's current customers into MIPS Technologies' customers.

"We strongly support Lexra's new direction as a fabless semiconductor company," said John Bourgoin, chairman and CEO of MIPS Technologies.

Becoming a "fabless semiconductor company" while "exiting the IP licensing business" means you don't make chips and you don't license technology to anybody else who does make chips. Your customers have now been switched over to another company anyway, to which you assigned all your existing chip design intellectual property, after declaring that gosh, those patents we spent all that time fighting really _do_ cover using purely theoretical software you never actually wrote to implement things the hardware goes out of its way to avoid doing.

This is what the CEO of MIPS "strongly support"ed, with a straight face. Is it any wonder the rest of the industry switched en masse over to ARM chips instead?

But before all that happened, the Lexra 5200 processor design was licensed to Realtek, which used it in a pair of System On Chip designs, the RTL 8181 and the RTL 8186. These were a Lexra 5200 processor plus integrated serial, ethernet, wifi, PCI, and so on. (The two chips used the same "subset of mips" compiler, but different kernel .configs for the different peripheral device versions integrated onto the chip. Also, for some reason the /proc/cpuinfo in the kernel identifies this sucker as a "Philips Nino". Presumably another licensee of the Lexra chip designs?)

After Mips put Lexra out of business, Realtek discontinued the 8186 rather than be "converted" into a MIPS Technologies customer, but that didn't stop Chinese companies from manufacturing their own RTL-8186 variants, which they still do to this day. They weren't the ones who got sued, and the patent in question was filed in 1986 and presumably expired circa 2006 anyway. These chips are small, low power, and really, really cheap.

However, because of the FUD cloud surrounding the chip design, the patches to support this stuff in Linux never made it upstream. All the toolchain tweaks do is remove the four unimplemented instructions, so those changes are trivial to forward-port, but they never got cleaned up so they're a proper ./configure target instead of a hardwired #if 0-ing out of chunks of the source code. So the compilers available targeting this hardware are of the 3.3.3 vintage.

Worse, the integrated peripherals on the SoC are all hand-rolled Realtek stuff, the drivers for which never went upstream either. Thus the kernels that use them are circa 2.4.18.

I can't do much about the kernel right now (realtek only released binary only drivers for its SoC peripherals, although there seem to be some open source rewrites floating around out there on rtl8186.sourceforge.net and such). But the toolchain and user space I can clean up and upgrade.

Yay, Andre's routers arrived. (Paying work, always fun.)

Pushing perl patches upstream, somebody asked Peter Anvin to ack 'em. It did not end well. On the bright side, his current objection du jour is something I can actually address. (Admittedly it's useless and almost certainly another proxy for his objection to removing perl in the first place, but hey, I can call this bluff. If I can just clear my plate to make time and energy for it.

Ripping ccwrap.c a new one, as mentioned on the FWL list.

Why does everybody keep playing Zombie Sinatra? The man couldn't sing when he was _alive_, and they JUST WON'T STOP...

Yesterday's bisectathon went off into one of those "git bisect skip is stuck after 2 dozen attempts" cul-de-sacs, which was my cue to declare that particular build break "good" so I could track down the commit that introduced its fix. The fix was:

--- alt-linux/arch/arm/common/vic.c     2009-09-15 11:51:09.000000000 -0500
+++ git/arch/arm/common/vic.c   2009-12-09 03:26:49.000000000 -0600
@@ -22,6 +22,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 

 #include 

Which needed to be applied to bisect through a range of a thousand or so patches which otherwise wouldn't compile. Luckily, it _did_ apply to that range, and thus I could bisect through it.

That meant that the bisectathon wound up with this, on about the third attempt:

There are only 'skip'ped commit left to test.
The first bad commit could be any of:
0e0fe9219d2e5426d87a0f62db445010d4aba2c7
8b79d5f217d7fc7d6b85ee5b10899f2bd20b64ed
8bdca0ac2b1ec35091941c57b4202f7096291c5b
b32f3afe3c5c4ddd1c414d41845470a99c88475f
ab6494f0c96f8953c7ad3e35301b2de2db3add57
bdaaaec39792ee0035d6c5a5ad2520991e090a3c
2732f4b6f11689fa08e2db5689fc652d608936b5
412bb0a6224f96661c042a109c4978649c00ed52
3c4279750c7d3f88a0638a44e257d7d8d10732f2
61f4a10cb4e9447a85245c63b3e7f46e09299fed
13efdbecc65ef6ec4028551fb223dea5c5e3143c
We cannot bisect more!

So the first 9 of those 11 commits are a range of patches by Catalin Marinas committed on July 24. The other two are branch merges by Russell King (the arm maintainer), so shouldn't be introducing any new code.

Which means that the commit that introduced this bug should be one of Catalin's commits. According to the order "git log" shows commits in, those first nine are a contiguous block (of _course_ not in the order git bisect just dumped 'em in though, that would be too easy). The last commit before any of those was 05efde9d04ccc, which worked. The next one isn't by Catalin but by Hyok Choi of samsung, and it's introducing a new config symbol. There you go.

I'm not at all surprised by this. I actually suspected something like it, but it shouldn't be that much of a pain to bisect to find a simple tweak you need to make.

Ok, I need a config upgrade script. I need to expand a miniconfig against "stable", copy that .config to the alt- tree, do an oldconfig, and then compress it back down to a miniconfig... Hmmm, what UI should an automated script to do that have... Ok, that might work.

And lo, FWL 0.9.9 has shipped. (Using linux 2.6.31.6 rather than 2.6.32 because .32 still needs fixing.)

I spent bits of the weekend and the first part of this week tracking down what to do about the PowerPC pmac_zilog serial panic, and now I've checked in a one line fix. (It's a test for null, averting the null pointer dereference. Doesn't seem to hurt anything else.) Probably not the right fix (since it's in common code and not pmac_zilog.c), but I posted said one liner to the linuxppc-dev list for them to _properly_ fix it upstream, and it's small enough for me to deal with it myself in the meantime. Not _currently_ my problem. (I also posted the updated perl removal patches to linux-kernel, fairly _early_ in the merge window this time. Here's hoping.)

So why is git's user interface so horrible? Here's an example:

Now I'm trying to bisect why the arm targets no longer produce any serial console output in 2.6.32. The kind of random breakage you get each new release, only now in arm rather than ppc.

I start bisecting betwee 2.6.31 (good) and 2.6.32 (bad). I set up my build environment to yank all the patches I'm applying, and symlink perl from the build/host directory just so I don't have to deal with what applies to what. (I'm only running system-image.sh anyway, not trying to cross compile the root filesystem. No ./configures involved.)

The previous bisect hiccuped on some random file conflict, so I did a checkout -f and then restarted it. (git bisect reset, git bisect good, git bisect bad.) The first one is a git bisect skip, and the second is git bisect bad. And then it does this:

Bisecting: 3098 revisions left to test after this error: Untracked working tree file 'arch/arm/mach-w90x900/mach-w90p910evb.c' would be overwritten by merge.

I happen to know you can fix that with "git clean -fdx", which I probably should have done last time (although nothing anywhere _hints_ that you're supposed to do this, they just expect you to memorize the magic incantation). But here's my question: what state is the thing in now? Do I need to do "git bisect bad" again? Will another checkout -f put me before or after the bisect bad? It gives me no hints what to do next. It just expects me to _know_.

I do happen to know to do a "git bisect log", and I get this:

# good: [74fca6a42863ffacaf7ba6f1936a9f228950f657] Linux 2.6.31
git bisect good 74fca6a42863ffacaf7ba6f1936a9f228950f657
# bad: [22763c5cf3690a681551162c15d34d935308c8d7] Linux 2.6.32
git bisect bad 22763c5cf3690a681551162c15d34d935308c8d7
# skip: [73c583e4e2dd0fbbf2fafe0cc57ff75314fe72df] Merge branch 'omap-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap-2.6
git bisect skip 73c583e4e2dd0fbbf2fafe0cc57ff75314fe72df
# bad: [ae19ffbadc1b2100285a5b5b3d0a4e0a11390904] Merge branch 'master' into for-linus
git bisect bad ae19ffbadc1b2100285a5b5b3d0a4e0a11390904

Note that the comment lines repeat the information from the actual data lines, and of course all the identifiers are machine readable but mean absolutely nothing to humans. The "number of revisions left to test" data is not included in the comments, of course.

The common failure mode of code that's primarily used by the people who wrote it is you have to understand how it was implemented in order to make it work. You must know how it works internally to command it. They _assume_ you know how to do this, and it never occurs to them to list your options for what to do next, because obviously you have them memorized (or can look in the source code).

This is not true of cars, phones, refrigerators, or any of the other modern marvels that achieve widespread acceptance. You don't have to know how a light bulb works to flip a switch.

Linux's 20th anniversary is in a couple years. Its desktop penetration is expected to remain below 1%. There is a reason for this.

Poking at the PMAC_ZILOG issue on PowerPC.

At 4:30 security started letting people back into the area with the nice google beanbags and the promise of a foot court opening at 5 am where I could get a beverage. Unfortunately, yesterday's boarding pass no longer worked, so the security people directed me to the Alaska desk where they assured me someone would be there.

And there was someone, at every other airline's booth. But the Alaska people didn't show up until an hour after everyone else (including again the Frontier booth next door). In the meantime, their computer insisted I wasn't booked on that flight, and would not issue a boarding pass. So I spent an hour standing there in front of the only unstaffed desk, holding an orange "security" pass that would let me skip about 3 minutes of screening line. Whee.

Apparently the boarding pass they gave me yesterday was a specially coded sort of scarlet letter, telling every Alaska employee who saw it that I was to be treated like a transgressor. The new one looked completely different, today went much better, and the planes they exist to fly around are relatively pleasant once you're on them.

For a second time my safety information card was stuck together by something spilled into its little plastic pocket. This time it wasn't still damp with chunks stuck to it, nor was there wasn't a puddle of it on the floor in the space I was expected to put my backpack into, which was an improvement over the flight out. (Instead it tore in half when I tried to open it.) Didn't care. Slept on the plane, and got something to drink when I woke up. Bliss.

Not flying that airline again voluntarily, though. Ever.

Stuck in an airport theatre presents: my day as described in the email I sent to the person in charge of Alaska Airlines' twitter account:

I normally fly SouthWest, and admit I have been spoiled by them.

Southwest sees its mission as getting people to their destinations. If they have to cram you into an overhead bin to get you there, they will. (Southwest is proof positive you can get the contents of a Greyhound bus airborne.) But by Pharamond they will GET you there, generally on time. It's what they do. It's what Southwest is _for_. (I once had a southwest employee take me in their _own_car_ to another nearby airport when a flight was overbooked.)

In contrast, legacy carriers see their business as flying planes around from place to place. Whether or not you're _on_ them isn't really their problem, they have more important things to worry about. They take your money, which pays to fly the plane, and everything else is unimportant. I had forgotten this.

Human resources people tend not to book on southwest. It's apparently some side effect of the fact that they're not set up to give discounts to travel agents. The price on southwest's website is the cheapest price they offer to anybody. (They might as well just offer it directly to customers.) So it wasn't too surprising when a resources person in Europe booked me on Alaska Air between Austin and San Jose, and I managed to get here ok. (Although it was the first time I've deplaned onto the tarmac in years. Reminded me of Kwajalein.) My flight back was a red eye this morning. I even went to bed early.

The problems started when the phone in my hotel room did not ring for the wake up call I'd requested. Instead my phone alarm woke me up, but it was still set at the time I had to get up at _yesterday_. AAAHHH!!!

Still, it was 8am and the plane took off at 8:55 and the airport's 20 minutes away. I wasn't checking any bags, wasn't returning a rental car, and I'd made tighter time crunches before. (On Southwest.)

I tried to call the lobby to get them to pre-order me a taxi: the phone can't call the lobby. It is completely dead. Great.

Reluctantly skip the shower, shove everything in my backpack, head downstairs (elevator, cannot make it go faster), had the front desk call a taxi for immediate pickup (and informed them about the broken phone, which they confirmed wasn't working before the taxi arrived). The nice taxi guy only took about 5 minutes to get there and then did 80 getting me to the airport. So I arrived at 8:30, meaning I still had 25 minutes left to make the flight.

The last time I had a similar problem was about five years ago, back when the post-911 security theatre frenzy was even worse. I arrived at the airport with 12 minutes before my flight took off, and Southwest got me on the plane with almost a full minute to spare. Admittedly that was _awesome_ of them (and started my loyal-customer-ness), but this time I had over twice that much time. It was tight, but I might still make it!

Run to the alaska check-in machines... which wouldn't let me check in for the flight. Doesn't acknowledge that flight number exists. What? Ok, run to the Alaska counter, stand in line to speak to the bureaucrat at the desk, and the lady there _spent_time_arguing_with_me_ that it was too late to get on the plane. But she eventually agreed to let me try since the plane had not, in fact, taken off yet. (My argument was something like "The plane is right over there, it hasn't left yet, can I try to get on it please? I just need a boarding pass.")

Run to the security theatre line, spent a dozen or so minutes getting through that. (Yes of course my bag was pulled out for manual screening. I was in a hurry. Same bag that I took to SJC, containing the same stuff that didn't get screened on the way here, but oh well.)

Anyway, I got out of security theatre. RUN to the gate. There were still 6 minutes left (according to the clock at the gate) until the scheduled take-off time of the flight. Victory! And... the desk has nobody at it? Over to the next gate's desk... Where is the plane? They tell me the plane is "already in the air".

At this point, I refrained from asking the bureaucrat at this new desk what the time listed on my reservation actually represents. (That's when it achieves cruising altitude? If I was taking a bus, train, or SouthWest, making it physically to the vehicle boarding point six minutes before the scheduled departure time would work. But apparently, not in this context.)

Raising that sort of issue is not helpful. I know never to debate policy with the worker bees in a bureaucracy.

So instead I asked her the standard bureaucracy navigation question: "What should I do now?" And here is the most important difference between Alaska and Southwest: she did not, in fact, care. The plane was in good hands, I could go hang.

She books me on tomorrow's version of the same flight, hands me the pass, and turns away from a job well done. I ask her "So I should wait here in the airport for 24 hours?", and she replied (and I quote), "I guess so", and turned away again. (Note: there was nobody in line behind me, she was turning to face the other person at the desk.)

I'm aware at this point that the lady wants me to go away, presumably because she thinks I've committed some kind of sin arriving at the gate only 6 minutes before the time listed on my flight information printout. She clearly feels that she is the wronged party, and keeps turning away from me at the slightest opportunity. (Ok, I know I had to skip my shower-and-shave this morning, and I probably have hilarious gravity-defying bed hair, but would a little eye contact kill you?)

Nevertheless, I persist for a few follow-up questions, which clearly annoy her:

1) Was there a later flight? No, there are no later flights, Alaska Airlines only has one flight to Austin per day.

2) Do they fly to San Antonio? Oh no, of course not. (She seemed to think I was crazy for asking.)

3) Anywhere else in Texas at all, from where I could get a bus...? Not today.

At this point the lady makes her first non-prompted suggestion: I could buy a ticket on another airline, with my own money. She makes sure to clarify this last part of her suggestion, which again is the first actual suggestion she's made about what I can do next.

I take the hint. I wander to a corner of the waiting area near an outlet, find a chair, and plug in my laptop. In a few minutes the sign changes to a Sacramento flight. After the Sacramento flight boards and departs, there's a shift change and a new lady at the desk. I have a number of questions the previous lady obviously wasn't interested in answering, and I try one on the new person:

Can I get a transfer through another airport, three or four hops even? She checks her computer: It's not even 10am yet and Alaska Air does not have any flights, from anywhere in the country, going to anywhere in Texas, until tomorrow. (Apparently travel on this airline offers no fallback plans whatsoever. Good to know.)

I go back and sit in my corner, but this lady (on her own initiative) starts working on my case. After about 15 minutes on the phone she rebooks me on another airline. (Yay!) She gives me a little printout thing on a boarding pass, which I gladly accept, thank her _profusely_, and follow her directions back out through security and to the Frontier Airlines desk next to the Alaska desk. At this point, despite how the morning has gone, I am _happy_. It is FIXED! I get to go HOME!

The woman at frontier airlines doesn't know what to do with the Alaska stub. That's fine, I've got an hour and a half until the flight, there is plenty of time. She suggests they might be able to accept my ticket at face value and let me pay the difference? Sure, I just want to get home, thank you. She takes the it to the Alaska desk next door where she and two other people huddle over the computer.

At this point, a fourth person comes to the Alaska counter. It's the lady who wasted time arguing with me this morning while I was rushing to catch my flight. From over at the frontier desk, over the noise of the people in line, I hear her yell "No, he was late! He agreed to go on tomorrow's flight! We're not doing that!". Since this is news to me (not having had caffeine yet my contributions to our earlier interaction had been things like "Please, can you just let me go to the gate? I've still got X minutes", while looking at my watch still in Texas time).

I go over to the Alaska counter, where the lady from this morning is busily un-fixing my issue. Apparently she had unilaterally attached conditions to allowing me to attempt to get to my flight before it took off, and was angry for me altering a bargain I had not, in point of fact, noticed. (When wastes-time-with-arguments showed up, the Frontier lady went right back to her desk to serve the next people in line. I envy her escape.) So the lady from this morning returns things to her idea of the status quo (decided the moment she saw me, apparently) while trying to start a fresh argument about it (or perhaps continue the one from this morning).

At this point, I have meaningfully interacted with three Alaska Air employees. Two of them treated me like a species of bug that had done something nasty on the carpet, and the third (who actually tried to _help_) seems to have been cut off at the knees for having done so (I assume it's her being yelled at through the phone). Not wanting to get the only nice one in further trouble, I don't argue with the one in front of me as she switches my ticket back to what she feels is right and proper. Since she's already under the impression I must have agreed with her earlier, I now _do_ agree it must all be my mistake, and ask her the "So what do I do now" question, which is ignored.

I ask about a hotel. This gets a better response. She's obviously happy at the idea of me accepting my place, taking my punishment of waiting until tomorrow, and most importantly going away and not bothering her anymore. And thus she deigns to answer this one. She says that I can go to the rental car area and call a hotel from there, some of them have "distressed flyer" discount rates. She does not name a specific chain, despite repeated attempts at clarification.

At this point I decide that volunteering information is against company policy. She points me towards the baggage claim area. Our conversation was over.

I wasn't really interested in leaving for a hotel and potentially having this problem all over again tomorrow morning. I am at the airport. Leaving the airport is not progress. (Besides, my ability to get lost is _epic_, wandering around a city I don't know is not my idea of fun.)

I'm tired. I just want to go home.

I go back to wastes-time-with-arguments and ask her how I get to the SouthWest terminal. This elicits the first smile I've seen from an Alaska Airlines employee so far. Also the first truly helpful response. Dunno if she's happy at the idea of me paying twice for the same trip, or if it's merely the idea of losing me as a customer to another airline.

So I went over to Southwest to reassure myself that competent airlines do still exist. They were nice, they _cared_ (or faked it darn well), they listened to my sad story, and they had three more flights that day to Austin, but the best they could do was offer me a last minute ticket for about the same price Alaska costs with 2 weeks advance purchase. (No this wasn't a special deal, this was the price on southwest.com.) Called Fade and fiddled with my frequent flyer program to try to overcome the fact I'm 5 credits short of a free flight.

Alas, I really couldn't convince myself that my comfort was worth more than a new subnotebook, especially since I've got my laptop with me, free wireless, electrical outlets galore, half a tin of penguin mints, a food court (alas, at airport prices), and a flight I'm not paying for tomorrow morning. Refreshing as it was to hang around people who actually _wanted_ me to reach my destination, my journey had not been entrusted to their care this time around. (To both sides' clear regret.)

I eventually bit the bullet, went back, and sat at gate C10. Didn't interact with another Alaska Airlines employee for several hours, until the nice british lady working the gate for the incoming austin flight (around 5pm local time) struck up a conversation with me. (She's very nice, warning me that this area would close overnight around 11pm, and talking a bit about the weather. She's actually helping another customer too now, and being _helpful_. So the nice/nasty record is 2 and 2. Makes me feel better about the company in general, but I'm still sitting in Terminal C with no better option than to wait until morning.) And around 7pm there was a nice security guard, who let me stay in the semi-shuttered security area (the motion sensors kept first turned the lights off in the Alaska Airlines area around noon), until 11p when they were "closing the gates" and I finally had to go out to the lobby.

So Firmware Linux is currently creating two cross compilers. The first is a quick and dirty simple one called "cross-compiler" which has no thread support, incomplete c++ support, and is dynamically linked against the host's C library. But it's enough to build the root-filesystem, and to do distcc acceleration (for both C and C++).

The second cross compiler is called "cross-static". It has thread support, uClibc++, and it's statically linked against uClibc for portability. It's (optionally) produced as a convenience for those crazy people who want to cross compile extensive amounts of stuff.

There's little point in packaging up the simple cross compiler for the downloads/binaries directory. If you're downloading prebuilt binaries, the second one is more capable, more portable, and generally superior in all respects. It's also a lot more complicated to build, among other things requiring _TWO_ of the simple cross compilers to build with (one for the host, one for the target) using a modified canadian cross process to build uClibc binaries on a non-uClibc host. (The native compiler builds the same way, it just uses the same simple target compiler for both its "host" and "target".)

Thus the quick and dirty one isn't going away, since creating that is a necessary first step before creating the second more polished version. But the "cross-compiler" I pack up isn't the one created by "cross-compiler.sh". That creates the _simple_ cross compiler. The full-featured one is actually created by root-filesystem.sh with some evil build flags. Part of the Great Refactoring should allow me to untangle and rename this, but to what? Building a system image currently requires calling cross-compiler.sh, root-filesystem.sh, and system-image.sh, and that's a clear story. The more complicated cross compiler is a side branch.

Also, I'm building the native compiler twice, once as part of root-filesystem.sh and once standalone. I should clean that up. Maybe the new toolchain builder script should be in sources/more? Except build.sh isn't currently calling anything out of sources/more...

Argh. Dismantling code that works because it's a MESS. Always fun. And figuring out the technical infrastructure is easy, figuring out what to call it so other people can intuitively understand it without needing a lot of documentation they'll never read is the hard part.

So I missed the deadline to get a release out at the end of november. At this point I'm aiming for monday, I gotta catch a plane tomorrow.

The ongoing Great Refactoring horked the static toolchain, but I figured out what I screwed up (moving the headers to the install location before building libsupc++.a means the build can't find those headers). I'm not quite sure why the native toolchain builds and the static toolchain doesn't. Investigation proceeds, to the point where I've got several different changes backed up and needing to be untangled so I can check them in individually. Such is life...

Digging through some old files I found recordings of an old Linucon panel I never got to attend, called "This Topic Intentionally Left Blank" (part 1, part 2). This was Steve Jackson, Eric Raymond, and Howard Taylor put in a room for an hour to talk about whatever they felt like, back in 2004. (Note that Howard had just quit his day job at Novell to become a full-time webcomics guy about a month before this con, this was his first Guest of Honor spot.) Jay "the tron guy" Maynard was in the audience and participated a bit too.

Unfortunately, there's a lot of tape hiss nearly drowning out the participants (especially Steve, who speaks softly and was probably farthest away from the recorder), and what I really need to do is type up a transcript.

There's probably more of these buried in various backups...

I don't understand why anyone ever votes Republican. I don't get it.

Herbert Hoover was a Republican. He's the guy who put the "Great" in The Great Depression, and thus had Hoovervilles named after him. Joe McCarthy was a Republican. He led the most overwrought witch-hunts since Salem, and publicly equated being a Democrat with treason (yes really). Richard Nixon was a Republican. The whole Watergate scandal was about Nixon's cronies getting caught breaking into the offices of the Democratic National Committee.

How could anybody vote for these people? How could any political party survive that legacy? The guy who undid Hoover's Damage was a Democrat. Bill Clinton balanced the budget and paid down some of the debt between the first and second Bushes' historic deficits. Kennedy simultaneously faced down the Russians in Cuba and launched the space program that got us to the moon _while_ paying down huge amounts of our leftover debt from World war II.

The guy the Republicans hold up as their current idol, Ronald Regan, suffered from Altzheimer's while in office. He pursued communists with the same blind zeal Joe McCarthy had, and got himself hip deep in scandal because of it, and his _defense_ was Altzheimer's. Not that he hadn't done stuff, but that he didn't _remember_. What kind of stuff? Funding terrorist organizations to attack anybody we didn't like in south america or the middle east, the same nutballs terrorists that we're fighting today are the ones WE trained. (Yes, that includes the Taliban, and Saddam Hussein.) Oh, and pursuing Regan's agenda racked up trillions of debt, for the first time since World War II (including looting the Social Security trust fund), essentially inventing our current budget problems (which two Bushes exacerbated without even the excuse of facing down the Soviets, but it _started_ with the massive fiscal irresponsibility of the Regan administration). That's their hero.

Who else is left? Harry Truman (Democrat, FDR's vice president) used the atomic bomb to end World War II and the Marshall Plan to clean up after it, helped set up the United Nations and Nato to make sure it wouldn't happen _again_. His first executive order desegregated the military, his second made it illegal to discriminate in civil service hiring based on race. Truman was so successful that Republicans invoke his record all the time in their campaign speeches! Both the first and the second Bush repeatedly invoked Harry Truman in their speeches, despite voting against him, having Trumans' relatives ask them to stop, and so on. Truman himself equated "Republican" with "rapscallion" (his word, if Wikipedia is to be believed).

Eisenhower was a Republican, elected because he was a World War II general, and like Grant before him his administration was full of corruption (Nixon was his Vice President, and McCarthy rose to power on his watch unopposed). But the guy himself was more defined by his military career than his political party, and also had his hands full cleaning up after World War II.

And of course Carter shows the main failure mode of the democrats: their ability to be massively ineffective despite the best of intentions. (The current group has the presidency, large majorities in the house and the senate, and they still can't get anything done. Of course when the positions were reversed, they couldn't prevent the other side from passing their entire agendas virtually unopposed.)

I've said for years, the democrats are incompetent and the republicans are evil. But one side means well, and the other is doing things I'd go to great lengths to _prevent_. I can see wanting to replace most Democrats with somebody with a spine, but what I don't understand is why anybody would vote _for_ a Republican.

Working on granularity in FWL. Breaking up the various sources/sections scripts so each one handles exactly one package, and moving more package builds into sources/sections.

I introduced sources/sections to factor out common code, and if some package builds live there they might as well all live there (so they're all in one place). Again, the age-old tension between being explicit and not repeating yourself...

While I'm at it, time to audit sources/functions and see if some of those functions are specific to FWL (read_arch_dir) and some are more generic (download). Possibly split that file into two files? Have to work out what's best. I've already got some out-of-tree users of this code, and just discussed some more use cases. Trying to make it easy, the question is how?

So far the best I've come up with for beating the path to the standard headers out of gcc is:

echo '#include <stdio.h>' | gcc -E - | sed -n 's@.*"\(.*\)/stdio\.h".*@\1@p;T;q'

Poking at the sources/sections refactoring. Lots of today was spent frowning at why a gcc ./configure section was insisting on using /lib/cpp instead of "i686-cc -E", until I figured out it wanted to use "i686-c++ -E" (Um, ok?) and that I'd broken the C++ link names. (Horrible error reporting in this stuff. Horrible.)

Starbucks is open until 4 on thanksgiving! Yay!

Picking away at the sources/sections refactoring. I've mostly gotten binutils/gcc/ccwrap broken up into individual scripts, although making them reasonably orthogonal is a challenge. For one thing, gcc is always going to depend on binutils. For another ccwrap expects a very particular layout of the binutils and gcc output, and the logical thing to do is have the first two scripts install things where ccwrap expects them, rather than having ccwrap move them from one place to another. (Because otherwise ccwrap would just expect them in that _other_ place, which is silly.)

After I get that working again, the next step is to make ccwrap take an existing functional toolchain and wrap it. This is different from wrapping the output of a fresh unpatched binutils/gcc build, because that toolchain doesn't _work_. If I ask this gcc "where is your compiler include directory", it can't actually _find_ it. A functional toolchain could respond to "gcc --print-file-name=include" with a path. Adding the wrapper is what makes the FWL toolchain functional.

But if I have an existing functional toolchain, then I should be able to beat the six paths out of it. For compiler libraries, print-file-name of "crtbegin.o" and "crt1.o" and grab all the ".o" files out of thoe directories. Do the same for "libgcc.a", "libgcc_eh.a", and "libgcc_s.so" and grab all the .a and .so files. Then "libc.a" and "libc.so" into the non-cc lib directory.

I can also do "gcc --print-search-dirs", which gives "install", "programs", and "libraries" directories (remember to chop off the pointless prepended = sign from paths that start with that). This might be useful to find the linker and such to populate the "tools" directory.

But how do I find the system headers? It's not in --print-search-dirs, the --print-file-name won't find stdio.h...

Pondering an "expect" implementation for toybox. (Something written in C, not in tcl.)

When you look at "expect" scripts:

set timeout 30

expect "login:"
send "username"
expect "Password:"
send "password"

interact

It looks like all you need to do is write a command line utility that eats data until it matches a regex (and listens to an environment variable specifying a timeout), and then pipe data through a shell script. The "send" command is essentially just echo.

There are two problems with this. First, switching the tty in and out of "raw" mode (so you can match incomplete lines) is a bit of a pain when you keep relaunching commands. If you leave it in raw mode when your command exits, you inconvenience the next command, but if you don't input clumps weirdly.

The more serious problem is you need a circular pipe. The input of "expect" and output of "send" are some other program (though a pipe, character device, etc), which the expect script is processing. If you run that process, and pipe its output into the expect script, you then need to pipe the output of the expect script _back_ into the first process.

This means you need to run it as a child process of something _other_ than a shell, because shells don't do that. Some versions of the netcat command do that (toybox netcat -l), so I could extend netcat, but that means using "expect" as a normal command run from the shell doesn't work anymore because the shell isn't in control anymore...

Queue the Marvin the Martian voice, "Design, design..."

Ok, looking at the todo list from last week, Coleman's patch got integrated, I started a FAQ, Analyzed Grzegor'z issue and added it to the FAQ.

My vague poking at the hw-uml target has turned from adding futimes() to uClibc to fixing User Mode Linux to use futimens() instead. (Work in progress. I posted my futimes() patch to the uClibc list and it reignited the perennial "the uClibc development process is moribund" flamewar. Patching linux looks _way_ more attractive just now...)

Ben Zanin converted the presentation slides to text (all 260 of them), and cleaned up the HTML a bit. I added an index and trimmed a couple things, and put it up. Yay! (The pdf version is in the downloads directory now.)

Jean's ld search path issue and Vladimir's redone snapshots page are still pending.

At the moment, I'm working on Marc's arm-eabi issue. It turns out the problem he was seeing was an issue with his build scripts, which are based on top of the FWL scripts but extend them to do extensive cross compiling. (I had a "Wow, that's impressive! Why?" moment.)

The problem is he builds a new uClibc and adds it to the library path with -L, which doesn't remove the _old_ one from the linker path, and then the two fight.

The reason he used -L is A) It sort of worked, sometimes, when he was lucky, in the right light, with a tailwind. B) It was easy to do. But my solution is to use a wrapper script (sources/toys/ccwrap.c) which removes the existing headers and libraries from the $PATH and builds a whole new set of paths from from scratch. (You can't make gcc do this itself, gcc is totally incompetent when it comes to path logic. You can make it add stuff to its $PATHs, but the only way to get it to _remove_ stuff is to tell it to blank them and start over.)

For a while I've needed to write a script wrap an existing toolchain, building a copy of the wrapper and constructing a new wrapper directory to put it in. There are six main paths that compilers look at:

• 1) System headers (stdio.h, ~/include )
• 2) Compiler headers (varargs.h, ~/gcc/include)
• 3) System libraries (libc.so.0, ~/lib)
• 4) Compiler libraries (libgcc.a, ~/gcc/lib)
• 5) Tool search path. (ld, ~/tools/bin)
• 6) Input files on the command line (current directory, or paths upplied).

The only one that gcc _doesn't_ usually screw up and hardwire in strange behavior for is #6. (You'd think #5 would just use $PATH, but no.) So the wrapper has to force all of this, and it does so by assuming that the wrapper itself is installed at ~/bin for some ~, and then appends ../lib and such as needed to construct the other paths.

The wrapper has a WRAPPER_TOPDIR directory that lets you explicitly feed it a different value for ~, and Marc suggested I give him more granular environment variables so he could tell it where to find the things it's not finding. Unfortunately, if you start specifying this stuff yourself, you have to specify at least five different paths, and that's assuming that Path #4 above is where the compiler's "crtbegin.o" and friends live, and that path #3 is where the C library's "crtn.o" and friends live. (This is where my setup puts them, but in the wild they can wind up all over the place what with /lib and /usr/lib and /usr/lib/gcc/x86_64-linux-gnu/4.3/ and so on. On Ubuntu, /usr/lib/gcc/x86_64-linux/gnu/4.3/libgcc_s.so is a symlink to /lib/libgcc_s.so.1, so the name the linker is looking for isn't in _any_ of those locations. Wheee...)

So the fix to Marc's problem is to do a "wrap_compiler.sh" that takes an existing compiler and sets it up with ccwrap via lots and lots of symlinks. There's some design work to do, though...

Adrienne asked why I don't like the new "retweet mangling" misfeature of twitter, and since there's no possible way to express the horror in 140 characters...

If twitter added a button that quoted a tweet (to save a cut-and-paste), and enabled the same link to original tweet that "reply" had, that would be great. That's not what they did.

Instead, they made it so that it looks like tweets are showing up from people whose feeds you're not subscribed to. "Who the heck is this person and why should I care what they have to say?" You have to look out of line, at small print, to see who you _are_ subscribed to who actually posted this. (There's a funky "recycle" icon in front of the tweet, which is light grey on a white background, looks a bit like an @ sign, and is barely noticeable when you're _looking_ for it.)

This is not an improvement over the conventional RT behavior where "@whoiknow RT @newperson" is right at the start of the message where I expect it, and the relationship is very clear.

Context is important with retweets: if @neilhimself is retweeting someone it's most likely a literary reference and probably interesting. If @simonpegg is retweeting somebody it's usually some local brit I've never heard of, or an actor on a show I never saw, and I'm unlikely to care about it. One of those is consistently worth a read, one of those is consistently barely worth a glance, and lumping them together is WRONG.

I'm bad with names. Every time I see a name in my list that I don't immediately recognize I have to stop and think "did I subscribe to this person and they just haven't posted in a while? Did I misread the name? Did some spammer crack into my follow list?" It totally breaks the flow, and turns twitter from something I can quickly scan and only have to stop and think about if a tweet is actually _worth_ thinking about, to something that regularly grabs my attention to resolve problems with no payoff. This change makes it harder to evaluate my feed. It becomes _work_. Half the _point_ of twitter is it's quick and non-intrusive.

So the old way of presenting retweets is superior to the new way, I greatly prefer the old way, the change is pure loss... but there's no way to disable it. I can't opt out. I CAN'T MAKE IT STOP. It would be trivial to make these show up as "@persioniknow RT @personidunno" but they won't do it, they think their new presentation format is unquestionably superior, and they're going to cram it down my throat whether I want it or not. I would rather block _all_ retweets than have to deal with them in the new format, but they don't give me that option.

Add in the fact that the "retweet beta" introduced itself with pop-ups, to make sure I had the worst _possible_ first impression, and I couldn't opt out, and I couldn't figure out how to even send them feedback about it until Mark pointed out the link on the right (under "direct messages" and "favorites")...

And that's why I'm looking for a non-web UI for twitter. They broke the web UI. "Kill it with fire."

Hmmm, an email I got this morning points out that I need to write more documentation. My reply was:

On Wednesday 18 November 2009 08:43:41 drsunbo wrote:
> Hi, Rob,
>
>  I Have a router which with a busybox 0.61 pre fireware embeded.
>  I want to upgrade it to 1.* such hight version, But I don't know how to do
> . Could you help me something or give me some related links?Thanks . ^_^
> Yours, BOBS

The first problem is that busybox is just one part of your router firmware.

What you have in your router is a small Linux system, with three major 
components:

1) The Linux kernel

Usually a 2.4 or 2.6 version, "cat /proc/version" to see which.

2) A C library

Usually either glibc or uClibc.  Do an "ls -l /lib/libc.so.*" and if it says 
libc.so.0 it's uClibc, and libc.so.6 is glibc.  What that symlink points to 
should tell you the version number of the corresponding package.

3) BusyBox.  (Which you know.)

Busybox provides the posix command line utilities, your libc provides the 
system call interface, and your kernel handles resource management and 
interfacing with the hardware.

The second problem is that you need to be able to build code for your target 
hardware.  Your router's processor probably isn't an x86 chip.  It could be 
arm, mips, powerpc, or something else entirely.  You have to find out what, and 
then get an appropriately targeted cross compiler.  If you "cat /proc/cpuinfo" 
it should tell you what processor it's using.  (The uname -m command would 
also say.)

You might be able to get a cross compiler that can use your existing linux 
kernel and C library headers, but that's a fairly precise configuration.  If 
you can't, you'll have to replace the C library and the Linux kernel when you 
replace busybox.

The next question is 'what filesystem format is the router's root partition 
in"?  "cat /proc/mounts" should tell you that one.  You'll also need to figure 
out "how do I write to the router's flash"?  (Does it have a bootloader you can 
get a serial console for?  Or do you have a jtag?  This is another hardware-
specific question, it varies per router.)

Probably the best place to go to ask these sort of questions is the #edev 
channel on irc.freenode.net.  That's where a bunch of embedded developers hang 
out.  There's also a linux-embedded@vger.kernel.org mailing list, see 
http://vger.kernel.org/vger-lists.html for subscription information.

Rob

I suppose I could have pointed him to the OLF presentation (which Ben Zanin converted to html for me! Yay! I need to get those up), but it doesn't exactly answer his question. It doesn't even include the Introduction to Cross Compiling document I wrote way back.

Gotta integrate all this stuff into one big lesson plan, and then break it down into individual lessons that provide small enough chunks to deal with...

Ripping apart the http://impactlinux.com website so it's stylesheet based instead of wordpress. Much construction dust at the moment, and possibly for the next couple days as well.

The Gentoo From Scratch page needs updating, its navigation was dependent on the wordpress stuff, and I dunno how to put that back. I'm reading the html source that wordpress used to put out, and it's full of javascript and css (included from directories that don't actually seem to exist, some kind of apache rewrite going on there).

For the moment I just put a really simple index.html up directing to the FWL, GFS, and toybox pages. But only one of those is actually navigable at the moment. Still, better than a raw directory listing. :)

Updated the nav bars of FWL to provide more info. Trying to figure out why the cross-static compiler is having build issues the native compiler isn't having. (Basically trying to deal with todo items faster than they accumulate.)

So Grzegorz's issue is that when you type "qemu" on ubuntu when it's not installed, it says:

The program 'qemu' is currently not installed.  You can install it by typing:
sudo apt-get install qemu-kvm
qemu: command not found

Except that the qemu-kvm package only includes the i386 and x86_64 targets, not any of the others (arm, mips, ppc, sh4...). It installs the man pages for the other targets, but not the actual executables.

Heh. If you run qemu inside qemu, both tell you to hit ctrl-alt to exit the mouse capture. (Luckily, the external one is transmitted to the internal one as well, so you can click on the outer one to give it focus again but then have your mouse back in the internal one.)

Ubuntu's "qemu" package is just an alias for "qemu-kvm". The package you need to install to get qemu-system-mips and friends is "qemu-kvm-extras", which is crazy because these are pure qemu packages that have nothing to do with kvm. The Ubuntu package managers are nuts here, this package name is actively misleading.

Ok, time to start a FAQ. I dunno how "frequently" these questions are asked, but they _were_ asked and I haven't got a better place to put this info, so...

Ok, what are my current todo items...

Marc's arm-eabi issue (the__aeabi_unwind_cpp_pr0 thing) is still outstanding. That one's complicated. (Well, the fix might be simple but understanding it enough to figure out what's _wrong_ isn't.)

Jean's ld search path issue is still outstanding, but I need more info from him before I can address it.

Vladimir submitted a redone snapshots page, based on stylesheet instead of tables. He's got some ideas that could significantly improve the thing, but since this is largely cosmetic I'm holding off until I get the first two fielded.

Grzegorz's issue: write up documentation on installing qemu, with a note about kvm providing the host qemu only. (I'm actually installing xubuntu 9.10 i386 in a qemu image so I can test installing the debian kvm package and then installing qemu over it.)

Coleman's issue is probably also documentation: he's trying to patch uClibc to backport shm_open from uClibc-git, and the patch isn't taking effect. (Did he put it in sources/patches? Dunno.)

Possibly I need a FAQ. The _volume_ of documentation I've already got is enough that people don't read it. The hard parts are keeping it up to date, and properly indexing it so you can find the answers to questions.

On an unrelated issue, I started doing a script to download the HTML version of the OLF presentation and do better navigation for it... And the first slide was 1.6 megabytes. There are 260 of these! Downloading all the slides is most of a CD image! For text slides with a background!

So yeah, looks like what I should do is just retype the text on those slides...

So today's accomplishments: make a big todo list of the stuff I'm didn't get any significant work done on, and then waste hours fiddling with slides that are already online. Wheee...

So after shipping the 0.9.8 release, I got feedback that the c++ links were horked (turned into absolute host paths) by the Great Refactoring, which was easy enough to fix, and that arm eabi has something subtly wrong with it, which isn't so easy to fix.

The report of subtly wrong was while cross compiling dropbear, but I also ran into it while building current uClibc-git:

path-path-path/gcc/lib/libgcc.a(_divdi3.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
path-path-path/gcc/lib/libgcc.a(_udivdi3.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0

I'm having trouble reproducing this in a hello world program. The _divdi3.o stuff _looks_ like the support code for doing 64 bit math on a 32 bit host, although maybe it's something looking for a soft-float support function when it should be using VFP? Except that I tried a half-dozen different division variants, and couldn't get the build to fail. (Unless gcc's optimizer is considering the return code of printf("hello\n") a constant string and optimizing out the division via constant propogation even without any opimizer options? Maybe?)

At first I was blaming ccwrap, thinking it wasn't sucking in libgcc_eh.a properly (don't those missing functions live there?), but that turns out not to be the case. The library is getting linked in, and according to readelf -a, libgcc_eh.a seems to include an implementation of __aeabi_unwind_cpp_pr0, but it's marked hidden?

Relocation section '.rel.text' at offset 0x51cc contains 37 entries:
 Offset     Info    Type            Sym.Value  Sym. Name
000002fc  00002c18 R_ARM_GOTOFF32    00000cb4   __aeabi_unwind_cpp_pr0

Symbol table '.symtab' contains 71 entries:
   Num:    Value  Size Type    Bind   Vis      Ndx Name
    44: 00000cb4     8 FUNC    GLOBAL HIDDEN    1 __aeabi_unwind_cpp_pr0

Confusing.

Google found a message saying this bug manifests when an EABI toolchain tries to use "-mabi=apcs-gnu" instead of "-mabi=aapcs-linux". So I'd like to confirm I'm using the right one of those, except I'm not actually passing those flags in at compile time, they come from the those funky spec files where gcc invented its own language for no readily apparent reason. (I note that "gcc --dumpspecs" says "no input files", you have to say "gcc -dumpspecs" with a single dash in order for it to be recognized. Ah, gcc...)

So what _should_ it look like? The above message links to Debian's EABI page, which describes the two ABIs thusly:

OABI

• ABI flags passed to binutils: -mabi=apcs-gnu -mfpu=fpa
• gcc -dumpmachine: arm-unknown-linux

EABI

• ABI flags passed by gcc to binutils: -mabi=aapcs-linux -mfloat-abi=soft -meabi=4
• gcc -dumpmachine: arm-unknown-linux-gnueabi

So it should be getting -mabi=aapcs-linux, -mfloat-abi=soft, and -meabi=4.

This page explains that -mfloat-abi=soft doesn't mean we're _not_ using Vector Floating Point (VFP), it just means that the compiler uses the same ABI for hardware and software floating point. (Possibly this means wrapper functions around the floating point instructions? Weird.)

So anyway, the output of gcc -v isn't feeding any -mabi option to collect2/ld, and the only specs snippet that seems relevant is:

*subtarget_extra_asm_spec:
%{mabi=apcs-gnu|mabi=atpcs:-meabi=gnu;:-meabi=4}

Which looks wrong, and confusing. (The | means it'll accept two different ones and what the heck _is_ atpcs?) This string seems to come from gcc-core/gcc/config/arm/bpabi.h:

/* Tell the assembler to build BPABI binaries.  */
#undef SUBTARGET_EXTRA_ASM_SPEC
#define SUBTARGET_EXTRA_ASM_SPEC "%{mabi=apcs-gnu|mabi=atpcs:-meabi=gnu;:-meabi=4}"

How the HECK is that file getting #included? (What _is_ BPAPI, anyway? The comments at the top of the file don't expand the acronym. It could be "base port", or it could be some specific system. I note that nothing anywhere seems to actually _include_ bpapi.h, so this gets sucked in by magic build infrastructure somehow, somewhere. Have I mentioned recently that I DESPISE the gcc build infrastructure?)

Hmmm, only gcc/config/arm has bpabi, so it's not like each architecture has a "base" file... But yes, this seems to be a base file for arm, because gcc/config/arm/linux-eabi.h says:

/* At this point, bpabi.h will have clobbered LINK_SPEC.  We want to
   use the GNU/Linux version, not the generic BPABI version.  */

How it got #included, I have no idea, but this gets #included after it and is already overriding bits of it for other stuff.

Ok, what _should_ SUBTARGET_EXTRA_ASM_SPEC look like? The acknowledged experts in this area are the codesourcery guys, so download their current prebuilt i686-hosted toolchain, make sure it can build hello world...

$ ./arm-none-eabi-gcc hello.c
/home/landley/firmware/arm-2009q3/bin/../lib/gcc/arm-none-eabi/4.4.1/../../../../arm-none-eabi/bin/ld: warning: cannot find entry symbol _start; defaulting to 00008018
/home/landley/firmware/arm-2009q3/bin/../lib/gcc/arm-none-eabi/4.4.1/../../../../arm-none-eabi/lib/libc.a(lib_a-sbrkr.o): In function `_sbrk_r':
sbrkr.c:(.text+0x18): undefined reference to `_sbrk'
...

At a guess, that needs to be installed at a magic absolute path, or needs a special --sysroot command line argument or some such. I suppose the docs would tell me, but all I really need is dumpspecs...

*subtarget_extra_asm_spec:
%{mabi=apcs-gnu|mabi=atpcs:-meabi=gnu;:-meabi=5} %{mcpu=arm8|mcpu=arm810|mcpu=strongarm*|march=armv4:--fix-v4bx}

Ok, I officially don't understand what's going on here. The Debian page is saying one thing, the code sourcery implementation is saying another. And thus we're back to brute force: take apart the build in question to see what circumstances reproduce the problem (and that the symbol being hidden is the root of the problem) and then trace through the gcc build and try to figure out WHY this symbol is hidden.

More or less finished the screenshots page. Looks reasonable-ish. I still need to fill in everything but ARM for architectures, and I need to finish history.html, and completely redo documentation, but hey: progress.

I'm also banging on sparc, which currently isn't booting with uClibc-git. No idea why.

So vapier removed the readelf.c Erik Anderson wrote for uClibc almost a decade ago, apparently because he couldn't be bothered to understand it. So I grabbed a copy from the last release, spent the 5 minutes necessary to clean it up so it built under glibc, and checked it in.

Now that I've gotten the 0.9.8 release cut and uploaded (if not yet the website changed so you can actually _tell_), I'm taking a break and banging on this readelf implementation.

So what you do is you run readelf against a binary, and it tells you what its headers say, ala:

Type:		EXEC (Executable file)
Machine:	AMD x86-64 architecture
Class:		ELF64
Data:		2's complement, little endian
Version:	1 (current)
OS/ABI:		UNIX - System V
ABI Version:	0
Interpreter:	/lib64/ld-linux-x86-64.so.2
Dependancies:
	librt.so.1
	libselinux.so.1
	libacl.so.1
	libc.so.6

It's got code to automatically detect the endianness... but not 32/64 bit word size. For that, it segfaults, which is bad.

The other problem is the function describe_elf_hdr(), which is pages and pages of this:

    switch (ehdr->e_machine) {
        case EM_NONE:        tmp="No machine"; break;
        case EM_M32:        tmp="AT&T WE 32100"; break;
        case EM_SPARC:        tmp="SUN SPARC"; break;
        case EM_386:        tmp="Intel 80386"; break;
        case EM_68K:        tmp="Motorola m68k family"; break;
        case EM_88K:        tmp="Motorola m88k family"; break;
//        case EM_486:        tmp="Intel 80486"; break;
        case EM_860:        tmp="Intel 80860"; break;

Those EM_BLAH constants live in a header "elf.h", which uClibc supplies a copy of, and glibc supplies its own copy of.

Why is EM_486 commented out? Because glibc's elf.h hasn't got it. Even though this is presumably a standard file listing constants out of the ELF standard, the two have fallen out of sync. (i486 binaries aren't specifically tagged as such in the ELF header, it's "i386" from the 80386 until you switch to 64-bit with the Athlon. Early on some people did but did that mean with or without math coprocessor? It fell out of use in the face of pentium, pentium 2, pentium 3, with or without mmx or mmx2 or 3dnow or... The ELF header just isn't that detailed.)

So uClibc's elf.h still has the old EM_486 entry (which you basically never encounter binaries for anymore), and glibc just dropped it and skipped that number.

Note that the uClibc readelf.c hasn't built against glibc for years, due to this kind of mismatch (and some strange makefile magic that meant they didn't bother to #include the headers they needed at the top). What dropping "readelf.c" from uClibc implies is Mike couldn't be bothered to keep uClibc's _own_ elf.h in sync with its own readelf.c. That was too much effort.

Sigh.

So the release notes are finished, the repository is tagged, I've got about half the screenshots page done, and attempting to go tp "http://impactlinux.com/fwl/screenshots" gets redirected to "http://impactlinux.com/projects/firmware-linux.html/screenshots". (The mirror on my site is still there so http://landley.net/code/firmware/screenshots works, except that now the header and footer #includes have been taken out, that mirror has no navigation cues to get from page to page at all.)

I'm also adding an architectures.html page, and who knows what next time? Hmmm. Gotta get together with Mark and redo the site navigation entirely. Among other things, I'd like the left-side menu back. (Progressive disclosure is great if you want to hide information, but I'd rather not hide it just now. If anything I want it _more_ on display.)

Oh well, upload it and clean it up tomorrow. Only a little more than three weeks until I wanted to do the next release anyway, gotta get a move on as they say...

Working on the new snapshots directory, and it's slow going. The only sane way I know how to get it to look remotely how I want is tables, which don't wrap based on your screen size. (I suppose I could just have the whole thing stacked vertically with the links on the right side of the screenshot instead of under it? Hmmm...)

As always, writing documentation raises design questions. Next release should leverage the BINARY_PACKAGE_TARBALLS stuff to have lots of more granular tarballs, so should the root-filesystem tarballs still be the same as the system image tarballs, just packaged differently? Or should I not _have_ root filesystem tarballs anymore, and just include directions on how to assemble one by extracting the other tarballs?

The existence of the native-compiler tarballs suggests that root-filesystem should _not_ have a toolchain in it, but should just be the minimal base system you can extract the native-compiler into to get what we package up into the development system images. Except that sounded good back when root-filesystem could just be the minimal dynamically linked uClibc+busybox stuff, and then busybox switched to statically linked by default, so now this "minimal root filesystem" would just be the busybox tarball produced by BINARY_PACKAGE_TARBALLS. (Well, not _quite_ true. There's the empty directories (/tmp, /proc, /mnt...), and the contents of sources/native (the new /etc/passwd, /sbin/init.sh)...)

It's a granularity thing. Right now, native-compiler.sh doesn't include things like "make" and "bash", meaning turning a minimal environment into a development environment takes more than the native compiler tarball, which implies the native compiler tarball as currently packaged is sort of useless. I need to fix that. How?

Possibly once I've got all the granular per-package tarballs, I should then a HOWTO on assembling them into a system image. (It's almost "extract them all into the same directory", except that some of them assume things live under "/usr" and some don't. The ones that don't need to be extracted _into_ /usr. Is a HOWTO overkill?)

But having native-compiler as an analog to cross-compiler is appealing. And having a minimal chroot environment (which would be the sources/native stuff plus empty directories, and the reason the empty directories aren't just in sources/native is that mercurial only tracks files, not directories) would be nice. And the current root-filesystem tarball has users, people who chroot into a uClibc development environment instead of running a system image under an emulator...

Design issues! Always time consuming...

Anyway, this release is an interim release pricesly so I can get something out and worry about this stuff afterwards, so I just need to finish up the release notes and the screenshots page, and write all this other complication _down_ so I can deal with it later. (You get a simple result by working _through_ all the complicated bits. It doesn't mean the path to _get_ there was simple.)

Hey, and Ubuntu lost its marbles again:

[18916.659218] iwl3945: Microcode SW error detected.  Restarting 0x82000008.
[18916.659231] iwl3945: Error Reply type 0x00000005 cmd REPLY_TX (0x1C) seq 0x0000 ser 0x0000004B
[18916.660006] iwl3945: Can't stop Rx DMA.

So my network is dead, no packets getting routed through the bricked wireless card. Ubuntu 8.10 used to let me rmmod and insmod to get my network back, but 9.04 will kernel panic if I do that. So I have to reboot again. And the sad part is I just had to reboot last night because of the X11 hang, which is a _separate_ bug but one that's bitten me something like a dozen times this month.

(So after years of mocking Windows, Linux needs daily reboots to keep a desktop system running. Our big argument in _favor_ of Linux is better code quality, and after 18 years into Linux development every new release is still full of _regressions_ from the previous release. That's kind of sad.)

So I've got all the binaries I want to release, but writing the release announcement turns out to be kind of time consuming. (Muchly documentation needed.)

This is basically a new entry in news.html, which involves going through hg commits 810 through 876. As usual, I keep thinking I haven't done much until I try to document it, and then there's so much stuff gets lost in the noise. (New bug tracker with roadmap! IRC channel moving! Prebuilt static dropbear, strace, and busybox binaries! New sources/more and sources/sections directories! BINARY_PACKAGE_TARBALLS option. New /dev/hdc with /mnt/init.sh! Fixed powerpc and sh4 to run with stock qemu! New armv4tl-eabi target! Now with more exclamation points!)

Another thing is that Mark made the web page lots prettier, but I can't personally navigate it anymore. There's no mention there _are_ additional pages in the FWL project unless you notice two things that I didn't (Mark told me):

• If you hold down the button instead of just clicking, the navigation buttons at the top of the page turn into pulldown menus
• If you hover over the, they're _multi-level_ pulldown menus.

In addition, I have no idea how to _modify_ this setup. Now that the nav bar went away, there's no longer any mention of an IRC channel, or the RSS feeds hg produces for commits and releases. (And I'm adding a "history.html" page, which isn't done yet. And now I'm adding a screenshots directory.) How would I add any of those? Ask Mark to do it, I guess. Except if I didn't notice this hidden pulldown menu at the top, how do I expect anybody else to?

The new layout is _pretty_, but I personally have trouble navigating it, let alone modifying it. Not sure what to do about that. I _like_ the pretty, but...

Also, Mark got a new job. (The economy sucks, we haven't had enough clients to keep the company going as more than a part-time thing. I'm interviewing elsewhere myself. This is a great hobby project, but in a recession nobody's willing to pay for what they can get for free.) So I'm not sure how much time or interest Mark is going to have in managing the website once his new job gets up to speed and starts eating his time and energy. (I suppose the same is true for me, one of the Oxford gigs effectively mothballed this project for months because I just didn't have the energy to work on it when I got home from work. *shrug*)

Oh, I'd also like the news.html page to be more than _just_ release announcements. Possibly the release announcements need their own page? I've been posting [LINK] (oct 30, Oct 26, Oct 24, Oct 15, Oct 7 twice, Oct 3, Oct 1, August 26, August 21, August 20 (twice)) general status updates to the list. Those should really be in some kind of news blog with an rss feed.

This blog has a lot of me _working_ on stuff, but it's not a summary of _results_. Even if you focus on just the "span id=programming" stuff in this blog (someday I need to updated the rss feed generator to be able to select just material in specific span tags), lots of it is ruminations about directions I don't take, vague blue sky plans I won't get to for months, debugging that results in a three line fix but takes weeks to work out. What the project needs in a news page is actual _news_.

I suppose what I really want to do in such a news thingy is have posts to the mailing list that _also_ get posted to a web page (and sent out to an rss feed), but which you can then go to the mailing list to reply to if you want. (I suppose I could set up a second announcements only mailing list and cc: that, but that seems kind of silly. These aren't necessarily "announcements" of any import, that would be the news.html page we've got now. They're news blog postings along the lines of "what's new this week".)

Eh, I'll figure something out.

Putting together a release. This script renames the cross-static tarballs to cross-compiler, including redoing the directory prefix in the tarball:

for i in ../cross-static-*.tar.bz2
do
  X="$(echo $i | sed 's@^../cross-static-\(.*\)\.tar\.bz2@\1@')"
  tar xvjf $i
  mv cross-static-$X cross-compiler-$X
  tar cvjf ../cross-compiler-$X.tar.bz2 cross-compiler-$X
  rm -rf cross-compiler-$X
done

I ran a buildall on securitybreach, and afterwards smoketest-all said:

Testing armv4eb:FAIL
Testing armv4l:PASS
Testing armv4tl:PASS
Testing armv5l:PASS
Testing armv6l:PASS
Testing i586:PASS
Testing i686:PASS
Testing mips:PASS
Testing mipsel:PASS
Testing powerpc:PASS
Testing sh4:FAIL
Testing sparc:FAIL
Testing x86_64:PASS

The sh4 test fails due to the serial port thing (that's a case of the test failing, not necessarily the system image or emulator failing). But the interesting thing is that sources/targets has 15 non-hw targets, and that's only 13. The m68k and powerpc-440 targets aren't building a system image.

I should upgrade smoketest to add a "MISSING" category, but Mark's mostly written a new expect based version, so I'll wait for that to go in. (Yes, the new version is written in Python, but I have different portability standards for a test harness as for the actual build. Especially since the test it's running is so easy to run by hand.)

So the reason the uClibc-git build has been dying all this time is that vapier decided readelf.c was just too much effort to maintain, and yanked it. Sigh. Forked a version to maintain myself, but I'm actually thinking this sucker might be best done in lua. (The bulk of it is the big long list of EM_BLAH values and corresponding strings, possibly it should just parse the comment fields from the /usr/include/elf.h header.) Eh, we'll see.

And now the i686 build is segfaulting. All right, what's going on... It's the static host toolchain, linked against uClibc-git and really not working... Because ccwrap.c is using the malloc functionality of realpath() and despite arguing about it for a week they haven't actually applied it to the repository yet, and I didn't make an alt-symlink for the patch because I thought they had.

I expected "Ew, that's disgusting that the PATH_MAX=4096 size limit is still there even in the dynamic memory allocation version, you should fix it _properly_ so it can handle unlimited length paths which is the whole point of being able to pass in NULL and have it malloc the right length". I didn't expect a week of arguing about how to micro-optimize the hack. (THIS PATCH IS QUICK AND DIRTY. That's part of its _nature_. That's part of what it tries to _accomplish_. It's not an implementation detail, it's a "we've been missing this functionality for years and now SUSv4 requires it, this at least lets programs limp by". And yet, they can argue micro-optimizing the hack for days. I question the priorities here, but don't really want to argue with Mike about it.)

Darn it, ported the patch (yes, ported due to gratuitous churn du jour, in this case they re-wordwrapped the comments before the function so I had to fix it so the patch applied to the start of the function)... and it's still segfaulting. Diagnose it in the morning...

And xubuntu had the hang again, forcing me to reboot. It does this so often I don't always bother to mention it.

On a related note, Firefox is kind of stupid. It remembers the 8 gazillion tabs you had open before the last crash, but won't let you access or edit the list. All it lets you do is "would you like to re-open all these tabs now, or discard them forever, and no you _can't_ launch a separate instance of the browser even to look at something local without answering this question". When the current hotel wireless access has a login screen (itself stupid, but that's a side issue), this can get really unpleasant. Especially if xubuntu's wireless manager "helpfully" remembered the wireless association and brought it up when you thought the laptop was offline so they'd all redirect to "could not access the network" tabs.

In theory, if you're fast ennough you can do a ctrl-alt-backspace to kill the whole desktop when you see it doing this. Except Xubuntu 9.04 seems to have removed this key binding, it no longer kills X11.

It's not that it's broken. It's not even that this OS is now 20 years old and still riddled with breakage. It's that things that _used_ to work regularly stop working. There's no positive direction, it's just circling the drain.

I have the xubuntu-9.10 iso downloaded and sitting there, and when I get home from Oni-con I need to install it... but I expect it to break at least as much as it fixes, because they always do. (And this is _my_ fault for not running the betas and experiencing daily _expected_ breakage on my laptop. I still need to buy a Mac.)

Speaking of which, xubuntu popped up a window saying my laptop battery's down to 47% at maximum charge, which is defective. This one lasted a lot longer than the previous nine cell battery did, but it's still getting into the "put more money into it, or buy a mac" decision time...

Drove to Oni-con with Fade and Beth. (Mark decided against coming, we meet up with him for trick-or-treating on 6th street saturday night.) Got here after all the interesting panels, but early enough to catch Randy Milholland's "Bunnies and Burrows" game.

We not only got the halloween candy the king wanted (most of which _wasn't_ covered in blood) but after the monster had finished off all the local humans we A) set it on fire with the gasoline, B) hit it with the grenade, C) hit it with the rat poison, D) had poison ivy in the fire so it breathed the smoke, E) the police who'd showed up to investigate the grisly murders saw the burning monster and shot it repeatedly. Pretty much all at once. (The frightening part? This is what I was _trying_ to do. It came together more or less by accident, but it was my announced plan and none of the party members died pulling it off! Although my rabbit was down to 2 hit points left at the end there.)

Yes, this is a Gurps game loosely based on Watership Down. No, he didn't mix in the Cthulu gurps rulebook this time.

Good times, good times...

Went out with Beth to see the Toy Story double feature. (Fade was going to come but we missed the 3pm start time and she couldn't come to the 7:25 showing.)

It's kind of odd how primitive the Toy Story 1 graphics look by modern standards. The story holds up just fine (and it's hard to top "Wind the frog!" as a battle cry), but Toy Story 2 is actually the better movie.

Looking forward to the third movie.

The reason distcc was failing is that QEMU 0.11.0 switched its default network adapater type (for x86 and x86-64) to the intel gigabit ethernet thingy, which the kernel .config didn't have a driver for. Easy enough to fix... once you know what the darn problem is.

Went ahead and checked in the unfinished history thing just to get the hg diff noise down to a dull roar.

I'm trying to finish the little Firmware Linux History document I've been writing so it doesn't keep showing up as pages of noise every time I do an "hg diff", and that prompted me to track down the old series of initramfs columns I wrote back when I was at TimeSys. (I even managed to dig up the third one they never published; I'm not actually sure I completed it. I can think of material I'd want to add.) Anyway, they're mirrored here now.

The uClibc config rewrite mostly worked, except some platforms have DOPIC=y set and i686 builds fine but can't compile anything natively when that's set, and I don't know why. Oh, and I haven't written the USE_UNSTABLE support yet so my in-progress sparc tests of the current uClibc-git can't progress until I get that back in...

The "UCLIBC_HAS_MMU" and "UCLIBC_USE_MMU" dichotomy remains insane but the guy who wrote it defends it, and since A) he hates me, B) I think he has horrible technical judgement (I suspect the two are related, yes), it seems unlikely to change any time soon.

Ok, I'm officially sick of distcc:

  CC      procps/top.o
distcc[13524] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
  CC      libbb/wfopen.o
distcc[13532] (dcc_build_somewhere) Warning: failed to distribute, running locally instead

WHY IS IT FAILING? Give me a HINT, guys? Was the network not set up right? Could distccd on the host not find the right gcc binary? What actually went WRONG?

And while you're at it, DON'T RUN LOCALLY. _FAIL_. This is an error, you stop working now because this should not happen and it needs to be FIXED.

Started collating together the uClibc configs for the various targets. I'm making one big baseconfig-uClibc common to all architectures, and adding the 2-3 lines unique to each one to the settings file of each target, so it can be appended and then the result expanded into a full-sized .config.

Small and straightforward enough that I should be able to get it stable in time for the release at the end of the month...

It's ironic that the vim "undo" functionality is the most dangerous feature in the program in terms of data loss. If you accidentally wander from insert mode into edit mode and hit u (a vowel), entire paragraphs of recently written text can suddenly go away... and there's no way to get it back. (There is a "redo". It's not implemented.)

The reason yesterday's fun little debugging session went so long is that distcc is fiddly. The new stuff doesn't manually take a path argument to where your cross compiler lives, instead it searches the path for $ARCH-cc and sets up its own set of names for it distcc can use. Specifically, distcc wants unprefixed names for all the tools, so the new code searches for $ARCH-cc out of your $PATH (the "which" command again), figures out which directory it lives in, gets a list of all the tools with $ARCH- prefixes in that directory, and populates a directory of unprefixed symlinks to them, which it then points the distccd $PATH at.

Curning through all the variants of "these symlinks aren't being made right", we then get to "the symlink is fine, it just won't run because I've violated the expectations of ccwrap.c".

The point of the wrapper is to rewrite all the paths to point to header and library directories relative to where the executable lives. But this new directory of symlinks lives somwhere _else_, so rewriting all those paths relative to the new symlinks will give nonsense. I need to resolve the symlinks until I find out what it actually points to, and then make the paths relative to _that_.

The libc function to do this is "realpath()", but that function is deeply flawed in that its second argument (the new buffer to copy data into) hasn't got a length argument, and a path could be arbitrarily long. Once upon a time there was PATH_MAX, but these days that's only advisory.

The Linux guys fixed this long ago in libc5/glibc so passing in NULL for the destination buffer would malloc() a new one of the appropriate size, without which this function is insecure now that PATH_MAX is obsolete. Unfortunately, uClibc hasn't caught up.

So I sat down and spent the 5 minutes to come up with a quick uClibc patch to make the malloc behavior work, even if it is just a quick hack that does an alloca() of a PATH_MAX sized buffer and then strdup()s it on the way out. (The returned result can't be longer than PATH_MAX, but the bounds checking for that was already there.)

(What surprises me is that in the past 5 years nobody _else_ has bothered to do this. Oh well.)

I then spent a while debugging _that_, because although it seemed to work fine at first it turned out the new codepath wasn't triggering because the compiler was optimizing out the null test. (Wha...?) After much head scratching I traced this down to the function prototype, which had this strange little __nonnull((2)) annotation on it to let the compiler know this argument could never be null, although obviously it can. (At first I thought declaring it char blah[] instead of char *blah was doing that, but changing that didn't fix it. No, it's a funky __gcc_extension. Right.)

It's also strange that:

$DO_DEBUG PATH=/blah/blah:$PATH thingy

tries to run PATH=... instead of running thingy. Even when $DO_DEBUG isn't set and thus goes away, the resulting command line doesn't do the "set variables only for this run of the program" thing unless you remove the variable expansion, meaning you can't set it to "echo" to display the relevant commands instead of executing them. (Enabling tracing mode in bash isn't nearly so useful because it spits out 20 lines of garbage calculating port numbers and creating symlinks and so on, not the actual meat of what the script is doing.)

Must fight with that later...

Took advantage of the marvelous weather and biked to chick-fil-a, where I spent most of the day implementing and debugging the new ./run-emulator.sh script plumbing. It's touched the top level run-from-build.sh (most of which went away, it's trivial now), and run-emulator.sh (which got largely rewritten), and I added a new dev-environment.sh adjacent to run-from-build.sh in each system-image.sh.

I'm sometimes ambivalent about UI issues because every option sucks. Way back when I had three scripts which called each other and which you ran depending on what you wanted. (This script sets up distcc and calls this script which adds a home directory which calls this script to actually run the emulator.) I threw that out and smashed it all together into a single run-from-build.sh script with a bunch of command line options, which didn't actually turn out to be noticeably simpler.

It's also touched sbin/init.sh inside the emulator, although that's mostly better status reporting, and it also makes sure that /home is writeable space (mounting a tmpfs on it if you haven't got an hdb). Although if hda is writeable, that's not really necessarily ideal behavior either... (As I said, "ambivalent". Checking for and handling a half-dozen different special cases is too complicated and makes the scripts brittle and hard to read, but _not_ doing it means they won't always do the right thing. Lots of sitting down going "ok, what do I want to accomplish and what's steps cover the most ground for the least complexity". Yes, I often spend two weeks pondering before writing five lines of code, because otherwise they aren't the RIGHT five lines. Actually they usually still aren't, but they suck less.)

Largeish ripping-a-new-one of the run-emulator.sh code underway. It's always been easy to build the appropriate target and then use run-from-build.sh, but it's been a bit of a pain to download the right system-image (and cross-compiler) tarball and hook them up in the right way to get a development environment. The command line for that is too long and fiddly and not properly documented.

So the NEW way to do it is to download system-image-$ARCH.tar.bz2 and extract it and run dev-environment.sh out of that. That gives you the 256 megs of ram and 2 gigabyte hdb.img. If you want the distcc accelerator trick, you also need to download cross-compiler-$ARCH.tar.bz2 and add its bin directory to your $PATH. (Or extract that tarball into the system-image-$ARCH directory the first tarball created, or into your home directory. It'll automatically pick it up from either location even if it isn't in the $PATH.)

So I'm redoing the script so it sets up distcc based on the $PATH, and adding the new wrapper script dev-environment.sh that defaults 256 megs of memory and creates a 2 gigabyte hdb.img if it isn't there.

I'm also removing all the old command line arguments that used to just set environment variables. It's just as easy to set the environment variables directly.

The run-emulator.sh also announces when it DIDN'T find the components for the distcc accelerator trick, and why. (Either couldn't find distcc, couldn't find $ARCH-cc in $PATH.) And init.sh should also indicate whether or not it's using distcc.

(It would be nice if there was some kind of distcc smoke test it could do. I've had way too many "it thinks distcc is working, but it isn't really" occurrences. The static native build logs are a good way of noticing next time I break that, but it would be nice to get something properly automated.)

Heh. Over in another corner of the project, the reason that uClibc-git on sparc wasn't behaving like I expected is that the file you put in sources/targets/sparc is "miniconfig-alt-uClibc", not "alt-miniconfig-uClibc". Oops, my bad. That _would_ explain why the symbols in there weren't taking effect. :)

I think a better way to do the --with-distcc stuff would just be to grab the appropriate $ARCH-prefix compiler if it's in the $PATH. Then run-from-build.sh could just adjust the $PATH rather than passing in a magic command line option.

User Interface questions are always fiddly, there's lots of things that could work and you have to try to figure out what's easy to understand and remember. This has the advantage that you can set it up and have it work automatically. I should have run-emulator.sh spit out a line if it can or can't find the compiler, so you have some feedback. (It's not like there's a shortage of kernel boot messages at the moment anyway, although a --quiet option might not go amiss...)

Haven't heard back from the PPC guys. (Well, benh said he'd look when he got back from Japan in a week and Alan Cox's email address from the patch bounced. He's at Intel now, but is he still interested in fallout from his TTY patches?

Last time I dug into the tty later it took 3 days. Maybe I'll bang on it this weekend if nobody else gets to it before then.

Found my copy of "Wolf who Rules" and spent most of the day reading that.

So the powerpc thing is somewhere between 2.6.28 (which works) and 2.6.29 (which panics as soon as interrupts are enabled). Unfortunately, in between is a long stretch of null pointer dereferences in the IDE driver, and after calling :git bisect skip" over two dozen times it's still noddling around in there.

Why can't git bisect skip chip off 1/3 from either end and look _there_? Testing commits adjacent to the one that didn't work is highly likely to find another one that also don't work for the same reason. Testing 25 adjacent comments in a range of several thousand is ludicrous. Pick a point 1/3 from either end instead of halfway between them, which should at least get you away from the error, and then if you continue bisecting from there it should introduce enough jitter to bypass it entirely if it isn't a particularly huge range. But having to rebuilt dozens of commits is not helpful.

Also, when I "git bisect skip", the number of commits to test goes down by one. Honest and truly it does, because I TESTED that commit. The results may have been inconclusive, but that's not my fault.

Found it. The powerpc serial problem I reported turns out to have been introduced by commit f751928e0ddf54ea4fe5546f35e99efc5b5d9938 written by Alan Cox. If I check that out it fails, but if I rever the patch "git show" gives me, it then works.

If I revert that patch i686 builds and runs fine, but I haven't gotten powerpc building in-situ properly with the reverse of that patch in FWL yet. (I think I broke the .config going from 2.6.28 to 2.6.31.) Oh well, deal with that in a bit. First I need to report it to #mklinux on freenode and linuxppc-dev at lists.ozlabs.org. (And no, google can't easily find either of those.)

So the 2.6.24 kernel doesn't have the powerpc serial panic, but 2.6.31 does. Time to bisect.

Laptop had the X11 hang again. Essentially lost the day's work in the reboot. (Not all the work, but context and what I was doing was all in the open windows.)

Reading my LUA book. (I've got a job interview tuesday morning for a company that A) looks cool, B) is local, C) uses LUA. Yay! An excuse to give this the time it needs to properly learn it.

What impresses me about this language is how stunningly minimalistic it can manage to be while still accomplishing everything it needs to. It's so SIMPLE. For example, the book points out that "A and B or C" is almost equivalent C's "A ? B : C". (The and gets evaluated before the or, so it becomes (A and B) or C. If A is false B gets evaluated. If A is false B doesn't get evaluated, but C does.

The corner case is if A and B are both false C still gets evaluated, but in Lua "0" isn't false. Only nil and the special boolean typed value "false" are false, so "max = (x > y) and x or y" works as expected even when x is 0. Yeah, it's funky in that it _doesn't_work_like_C_. (The whole arrays starting from 1 instead of zero thing is also going to bite me a few times before I get used to it.) But by slightly adjusting the assumptions it's been able to jettison about half the complexity of other languages, it just doesn't _need_ a lot of the constructs you take for granted elsewhere...

This is a noticeable step up from Python, which is now a completely different language than when I started learning. (Python 2.1-ish, circa 2001, didn't even have generators. I've picked up a few of the new contructs as it went along, but by no means all. By Python 2.5 we were already well into "new language" territory, and then Python 3.0 made it explicit. They could easily have called it "anaconda" or something, they just wanted to move their existing userbase to the new language. Yeah, I need to learn Python 3.0 at some point. Knowing "Python 2.3 or so" isn't the same thing at all.)

Meanwhile, Mark is learning Ruby. He seems to be enjoying it immensely. I should probably learn that one too, I've heard good things about it, and oddly enough todaty I've been reading about a Ruby book, if not actually reading said Ruby book...

I am totally going to have to read the LUA runtime source code. I have theories about how they implemented some of this...

Spent most of the day out with Stu's daughter Beth, and with Mark. Visited a toy store, a bead store, and a "general goods" store that sold rubber ducks in the shape of celebrities, candles scented like flowers I couldn't name (but Beth could), and enough 1950's diner style plates and utensils and syrup pitchers and such to launch your own diner.

The north drafthouse location has already stopped doing the Toy Story special feature in 3D, so we saw Cloudy with a Chance of Meatballs (me for the second time, Beth for the first). It continues to be awesome.

Then went downtown to meet Mark and see Master Pancake Theatre mock "Nightmare On Elm Street" (which oddly is _not_ to the tune of "21 jump street", nor to "Life in the fast lane"). Their theory that Freddy and Michael Jackson are related has some merit (both wore a fedora, were badly burned, had a trademark single glove, peaked in the 80's, had disturbing interactions with children, and are now dead). Good to see Owen back with the group, even as a guest. It's a family thing.

Since the downtown drafthouse is on 6th street, we all went to see 6th street. Mark showed us several bars. There's one called "Pure" whose theme is antiseptic sterility. The place is scented with various cleaning agents, and their tables are solid sheets of plastic lit from underneath to glow blue. (The ground floor is all blue-lit. The top floor is all red-lit. As far as we can tell, the building is heading downwards at a significant fraction of the speed of light, but it seems to be ok with that.)

And I found the coffee shop on 4th street where you can order S'mores. You pay the money and they bring to your table a tray containing a pile of marshmallows, graham cracker pairs in little plastic packets, hershey bars, little wooden sticks, and a metal pot containing fire. (Sterno or something, they light it at the table and it burns for 10-15 minutes.) You then toast the marshmallows and make smores, there at the table. It's a wonderful thing. The marshmallows, graham crackers, and chocolate were surprisingly well balanced but we almost wound up asking for more fire. (It was close at the end, we talked for too long.) I probably shouldn't have eaten the third S'more, but it was really good, and Mark drove Beth home so I got to walk it off.

Ok, the next con I'm doing guests for I need to invite Diane Duane.

Over in FWL-land, I'm juggling lots of non-working platforms. Interest in sparc has resurfaced, and current uClibc-git has a lot of fixes. It also doesn't build busybox, because include/utime.h isn't getting installed. (I don't know why, I've got CONFIG_SCREW_SUSV4_CODE_ACTUALLY_USES_THIS set.)

The powerpc panic seems to be due to the Poppy Z. Brite PMACZILOG serial driver, which panics as soon as interrupts are enabled (I.E. right when the kernel launches init). The issue might be fixable by bisecting back in the history and finding the last time the driver worked, or it might be fixable by switching from qemu -M g3beige to the mac99 one, which is apparently less badly maintained?

No progress on Alpha or m68k recently. I think if qemu hasn't got a board emulation for it, then it's not a FWL 1.0 release target. That does mean I should dig up and dust off the xylinx/microblaze patches, though.

Last night, I went to an organizational (possibly too strong a term) meeting of the Texas LinuxFest. This is the second local austin convention that I could probably take over and turn into a "Linucon jr." if I really wanted to put in the effort, but again once again starting over from scratch managed to seem like the easier approach.

The meeting was scheduled for 7pm, I arrived about on time and sat outside with the other people who had been locked out: the building we were meeting in locks as 7pm. Luckily the guy delivering pizzas had the number to call, so they let us inside where we ate the free pizza and sodas.

I believe the total attendance at the meeting was 6 people, if you include me. They had about as many again on speaker phone, although only one (David, from his Dell office up in Round Rock) actually spoke that I recall. (On the bright side, this meant there was plenty of leftover pizza and we all got to take some home.) Apparently the google group has about 40 subscribers.

They have no con chair, but believe that things can get done via committee or general acclaimation. The guy whose idea it was attended and ran the meeting, but doesn't want to be con chair.

They're actually not very interested in a combo sf/tech event ala Penguicon or Linucon. They have an existing "Linux Fest" model that they're following, which I'm only passingly familiar with, largely due to a lack of interest on my part. (Note that actual technical events like Ottawa Linux Symposium don't fall under this model: that's not a "fest". Ohio LinuxFest counts, though.)

They hinted a few times that they were looking for people to step up and do stuff, but it really doesn't sound like my kind of fun, nor are they interested in me trying to add my kind of fun to their event. They know what they want to achieve (in an extremely vague sort of way), they just seem to want somebody else to do it for them.

They have no venue, no speakers, no preregistrations, but what they do have a name, an event date, and lots of verbal agreements with sponsors. Apparently five figures worth of sponsorship if everybody they have verbal agreements with comes through. I didn't ask about bank account or incorporation details, but my impression was right now they haven't bothered to set up either. (They did mention two existing 501c3 entities that might want to handle money for them, if/when they actually get some.)

Despite their lack of venue, their event date is in six months. No venue candidates were presented during the meeting (they haven't visited any sites or gotten any of the little information folders the places give out). Finding some is the first item on their todo list, and they hope to meet again in a month to see if they have something by then. (They can't change the event date because it's one of the only things anybody knows for sure about the event.)

They were interested in the new facility UT built on MLK, or getting space at UT or St. Edward's, but have no students or professors involved in organizing the event. I brought up doing a call for papers to attract interest from professors (and corporate researchers), but I'm not sure there's enough time left to do one properly. (I tried to work out a rough schedule for them starting _now_, but nobody wrote anything down. Doing a call for papers raises a lot of complexity I don't think they're prepared to deal with anyway.)

My venue candididates don't really help, because a one day (saturday-only) event means people are probably not going to travel to it, so doing the SF con thing of bartering room block for function space means that schedule deprives them of any significant number of room nights to barter with. They also have to work setup and teardown into their day, or rent the facility for extra time on adjacent days, for vendors only, not attendees. (I suggested they ask armadillocon for a run-down of the local hotel function space anyway, even if they're paying cash it gives them more options.)

Admission is free, which explains why there is no preregistration mechanism. (Why would you preregister for an event that's free at the door?) The downside of this is nobody's committed to coming until the last minute, at which point it's an impulse decision either way and easy to decide not to bother because you have nothing invested in it. (On the other hand, since they're not using attendee registrations to pay for the event and aren't bartering a room block, on a financial level it doesn't really seem to matter whether anybody actually shows up or not.)

When I asked about the focus of their event, they used the phrase "big tent" to confirm an intentional lack of focus. They said their intended focus is "open source" in general, but I pointed out the name they've chosen is more specific (Linux only, and again they can't change it because it's one of the only concrete things you can say about the event, plus their "Linux Fest" model). When I specifically asked about open source on the Macintosh (in theory built on Darwin), or users of Firefox and such on Windows, or even things like Jamie Zawinski's struggles to get Dali Clock running on his Palm Pre, they said they didn't see those as significant sources of open source development effort. So they have a certain amount of focus due to both lack of interest, and to the unintended consequences of their name. (Apparently non-Linux can go hang, except for the BSD guys that slavishly chase every occurrence of Linux to say "me too" and the FSF guys who try to stick a GNU/Dammit on it. Both were brought up at the meeting, although not quite phrased that way.)

I asked about tracks (embedded, supercomputing, server, desktop, kernel, network security, desktop usability, programming, performance, debugging...), and they seemed to think that was a bad idea. They said having lots of different panels might make more people come. (For the one panel that interests them? Ok.) But also that panels aren't really of much interest, because it's not what the half-dozen people in that room (except for me) went to other LinuxFests for. The main draw for them seems to be hanging out with the vendors. (They did mention their distro sponsorships might lead to a lot of distro guys coming, so presumably they could put together a track about... repository management and package integration, maybe? Not sure. It was at least a theme.)

I asked "where's the shiny?" and explained I wanted to know what excited them about this thing, which they might want to communicate to their potential attendees. They couldn't come up with anything specific. The idea was that their companies had sent them to other shows around the country (as vendors), and they didn't like having to travel out of Austin to go to Linux shows when there's so much high tech in Austin. So they want lots of vendors hanging out with each other, here in Austin. (I note that Door64 has arranged two of that sort of event in the past 6 months, neither of which I went to. As far as I know, the Texas LinuxFest guys and the Door64 guys are not aware of each other's existence.)

They specifically mentioned basic introductions and tutorials when asked for example panels, and they're looking forward to giving out lots of distro CDs, which they saw as an asset forming one of the big draws of the event, and I see as the modern version of AOL coasters. It seems to me that the people who would go to a LinuxFest are people who already using Linux, and thus not the target market for basic introductions and distro CDs?

I did get a _little_ accomplished. I pointed out that their google group was set so you have to log in to view the archive, and they confirmed it and promised to fix it. I explained about the "heartbeat blog" of Penguicon's first year, and they thought that was a good idea, and that they'd ask on the google group if anybody wanted to do that. (The guy currently doing the website was there and said he'd actually just volunteered to let them use his server, not to create and maintain the actual website for them. As far as I can tell, there wasn't a single position anybody there wanted to _do_.)

Oh well, at least they updated their twitter for the first time in almost 2 months, mentioning the organizational meeting less than 2 hours before it happened. (Wheee?)

I don't know what they want. I don't know what they're trying to do. I don't see how I can help, or why I'd want to put much effort into it. I note that Stu, Mark, and Linas all declined to come to this meeting, and they didn't need to see it to make this decision.

Sigh. If I want another Penguicon/Linucon-style event, it looks like I'm going to have to create one myself. Repurposing other events doesn't seem to work, whether it's a tired old event like Armadillocon actively looking for new blood and new ideas, or a barely-there event like Texas LinuxFest looking for somebody to take over and do it for them so they can be vendors at the result. It's just not what they _are_, it's too far from their self-image to bend that way without losing their identity. And maybe too weird an idea to explain until you see it in action.

Oh well. If it's free, I'll probably still go and see how it turns out.

Finished factoring out busybox and toybox, and checked them in. Several other small cleanups too. Not fully recovered, but definitely feeling better.

Trying to benchmark static vs dynamic busybox under qemu, using the good old make 3.81 "time ./configure" to do it. (Even in a host build of that package, the configure stage takes three or four times as long as the compile, tarball extraction, and install _combined_.) It's pretty much the patholigical case for emulated native builds, and thus a good thing to try to optimize.

First run, dynamic busybox:

real	2m 35.02s
user	1m 49.66s
sys	0m 42.62s

Second run, dynamic busybox:

real	2m 36.14s
user	1m 50.54s
sys	0m 43.64s

First run, static busybox:

real	2m 6.95s
user	1m 26.56s
sys	0m 37.86s

Second run, static busybox:

real	2m 3.66s
user	1m 25.53s
sys	0m 36.18s

So it shaved off 30 seconds from a 2.5 minute job, for just under a 20% speed improvement. Nothing to sneeze at.

Alas, 99% of what this ./configure seems to be doing is lots and lots of little calls to gcc (which is still dynamically linked, and brings in the distcc transaction overhead which is at its worst for really small compiles and also still dynamically linked). You'd think the compiler itself would have some way to go "is this header there?", "is this function there?"... but no.

Still, it does imply that if the _entire_ root filesystem was statically linked, we'd see more speed improvement. Worth a poke. (After all, BUILD_STATIC is there already... :)

First run, static everything:

real	1m 56.20s
user	1m 13.31s
sys	0m 40.79s

Second run, static everything:

real	1m 58.68s
user	1m 15.38s
sys	0m 41.61s

Right, so making busybox static is worth 20%, and making everything else static is worth another 5%. Good to know, but it looks like only busybox is worth building statically by default.

I should ask the list...

On the programming front, I managed to tell people _twice_ to test code that I hadn't checked everything in for. (I really hate having a cold.) Got a little done on factoring out BusyBox, but my brains weren't quite together enough to poke much on even that.

So the downside of the new online bug tracker is that when I'm not online, I can't check the roadmap to see what needs doing next. This is especially annoying when my laptop decided to do the X11 hang again (Ubuntu 9.04 is almost as stable as Windows 95 was; second forced reboot this week) so I had to reboot and lose all my state, so I don't remember half the things I was in the middle of doing becaue all the open windows are gone.

Oh well, I still have roadmap.txt.

Darn it, my repository is completely horked. I pulled from mirell's repository, and doing so reverted the last half-dozen commits of mine, because it had decided that "tip" was now mirell's branch without my changes. So I told it to merge, and it complained that I had local changes. What the heck does that have to do with anything, I'm telling it to fiddle with the _repository_, not with my locally checked out files. They're two separate things!

So I told it to "force" the merge, and this did something but didn't fix the problem. I tried doing it again and it said I had an outstanding uncommitted merge.

How the hell do you have an uncommitted merge? It's a merge commit! That's what a merge commit DOES, take two already committed commits and glue 'em together. Ordinarily when I tell it "hg commit" without arguments, it goes through and finds all my changed files and tries to commit them all, but I don't _want_ it to do that. I want it to commit this abstract "merge" object, which somehow didn't get commited when it got created for no readily apparent reason. How do I specify an abstract object that doesn't have a location in the filesystem? Why would I NEED to? What POSSIBLE use does having it in an uncomitted state have?

So I told it "hg rollback", hoping this would destroy the commit 844 from Mark and put me back at commit 843 where I understood the state of my repository. That's not what happened. Instead, the repository went away. Completely. When I go "hg log -v", I get _nothing_. The entire project history is gone.

This probably has something to do with branches, but "hg branches" says nothing either. I have once again gotten the repository into a state where the project's maintainers would probably go "how did you do that", and I want it to stop RIGHT NOW and I'm ready to take a hex editor to the repository to make it happen. (Luckily, I haven't rsynced yet.)

I'm never pulling from an external tree ever again. Mark's change was a one-liner, and I lost a day of work because of it (and confused poor wangji who was trying to use the new work I'd done that mark's commit erased), and I didn't even learn anything (other than "don't ever do that again").

Ah, figured it out. I'd forgotten that before I did the rollback (expecting trouble), I tarred up my working directory, blew it away, and did a fresh checkout. (In an attempt to make that darn uncommitted merge commit go away, and also to give me an empty working store to do the merge in.) The rollback didn't undo the last commit, it undid the _clone_. (Yay shell command line history showing the actual commands I'd typed, even when I don't remember them. I hate having a cold. I also hate having to try to figure out what the mercurial developers were thinking when they implemented a given non-obvious behavior. Still light-years ahead of git, but that's damning with faint praise in spades...)

Banging on powerpc. Laurent Vivier pointed me at a page with debian qcow images that run under qemu 0.11.0 just fine (and even manage to have -hda set /dev/hda). I grabbed the /boot/config-2.6.26-1-powerpc out of that, and did a "make oldconfig" under 2.6.31 to bring it up to date. Took a bit, lots and lots of of new symbols.

If you try to run the debian image -nographic, the bootloader figures it out and talks to the serial console, but the Linux kernel does not. I fought with said bootloader for five minutes trying to figure out how to feed it a new kernel command line. If I type "help" it says:

Enter the kernel image name as [device:][partno]/path, where partno is a
number from 0 to 16.  Instead of /path you can type [mm-nn] to specify a
range of disk blocks (512B)

(Of couse it also says that if I type "walrus".)

It doesn't say what by "device:" options are (or partno) for that matter. (Dos drive letters? hda?) If I hit enter it'll boot a default, but it won't tell me what string I would have typed in to _get_ that default. It does tell me that the kernel command line is "init=/dev/hda3 ro", but not how I'd go about supplying an alternative.

Not very impressed with this bootloader.

So I built a 2.6.31 kernel with the new config, using the powerpc cross compiler I've been using all this time (which built a kernel that booted up to userspace before), and it produced four different arch/powerpc/boot/zImage* files, all of which crashed instantly with an illegal instruction when I pointed qemu-system-ppc at 'em. A quick check of my sources/targets/powerpc/settings showed I was running the vmlinux (that's the unpackaged ELF image in the root directory), and sure enough that got as far as prom_init where openbios handed off to the kernel. Roughly where the original debian kernel was failing to talk to the serial console. Even when I append console=ttyS0. (It's looking like this kernel hasn't got serial console support built in? No, menuconfig says it does...)

Ah, there's more than one type, and the 8250 emulation appears to have been replaced by the zilog one. Right, I remember now, it's console=ttyPZ0 now. *shrug* As long as it works.

Ok, using Debian's kernel .config (more or less), and debian's root filesystem, and the exact same qemu, the kernel panics when using the serial console and doesn't when using the emulated graphics card. It's userspace trying to talk to the serial console that causes the problem.

It's always something.

So I've got the native static builds building, and I've got the timeout code working, and I decide I need one more cleanup because passing in QEMU_EXTRA="-hdc /path/to/hdc.sqf" has a whitespace problem.

An absolute path from root could have spaces in it. I don't know what your system's paths look like, you could have a windows background. Heck, Mark left spaces in the URL to the slides from OLF, because he's using a mac now and macs do everything through a pretty GUI. It's easy to let spaces sneak in, and I have to support them.

The problem is, shell scripts suck at dealing with spaces. They have the "reparse command lines" problem with whitespace the way perl has the "list context vs scalar context" re-parsing problem where _it_ loses track of similar grouping information unless you stand over it and hit it with a stick.

The -hdb "$HDB" (and -hdc "$HDC") argument(s) should be optional, therefore the easy thing to do is set WITH_HDB="-hdb $HDB" and let that resolve to nothing when it's not set. But that can't have quotes around it or -hdb and its argument become a single item that qemu can't parse. Unfortunately, if $HDB has a space, then _that_ becomes multiple command line entries. Doing WITH_HDB="-hdb \"$HDB\"" means the quotes become literal characters considered part of the filename, meaning "temp dir" would be the worst of both worlds: two separate arguments but each containing a quote character.

This constant wrestling with whitespace is pretty much the bane of shell programming, and the bugs it causes don't usually happen on _my_ system because the path from root to my firmware directory hasn't got any spaces in it. (Possibly I should move it into a subdirectory that _does_ have them, just so these problems are forced to manifest so I can fix 'em.)

Ordinarily I quote the heck out of everything to try to deal with this, but in this case I'm not quite sure how to go about it. Feeding -hdc "" to qemu is probably going to make it try to open an empty filename; even if I make $HDCARG a variable that doesn't need to be quoted and will thus drop out if empty, the quote arguments don't drop out when empty. So I want to group spaces, but not have empty arguments persist, and those aren't orthogonal in shell...

This is trivial to do in something like python. There's a _reason_ people don't try to do complicated stuff in shell anymore. But I'm not introducing more environmental dependencies.

What I really want is something like the "$@" behavior where an array turns into a block of arguments with its original grouping level (regardless of spaces). Unfortunately, "$@" is a magic one-shot special case the shell people put in to deal with command line arguments, not a general facility available to shell code in other contexts. (And dealing with arrays in shell is black magic anyway.)

I can re-quote the command line and then feed it through "eval", which always makes me think I'm opening a security hole, even though this is local code that doesn't take arbitrary third party input data. An advantage of that is I can echo the command line before running it. The result isn't remotely _simple_ anymore, but the echo might make up for that. (Cut and paste _this_ to run it yourself. Even if you have to run the script rather than reading the script to see how it calls qemu.)

Well, even the echo isn't trivial. I'd have to echo the un-evaled line in order to have the quotes show up in the output (without which cutting and pasting and re-running the line means things like -append's big long argument block won't be aggregated). But if I do that, the command line will get re-parsed based on spaces for echo, so any occurrences of multiple spaces between lines will get squashed to one space, tabs will get squashed to spaces, etc...

Sigh. The designers of shell really weren't thinking about general purpose programming when they did it.

I suppose I could put one big occurrence of single quotes around it and then use double quotes for everything I do inside, and hope none of the data contains any single quotes. No filenames with apostraphes. No, I'd have to process the data to escape them (and escape backslashes), and how do you process the output without echoing it through sed or some such? By using more bash extensions. Grrr...

Such a simple issue should not open a can of worms. That's poor language design, right there.

Yesterday I poked the qemu list about the curses problem again, and today it was completely ignored, zero responses and buried under a flood of new posts. (Just like last time.)

I also wrote up fairly extensive data dumps on why I expect to rename the project when the 1.0 release comes out (giving the reasoning and potential new names), and a way too long post about why using uboot under emulation is more trouble than it needs to be (by explaining how linuxbios, bochs bios, bootloaders like grub, and the early kernel boot actually work).

Unfortunately, I can't link to those because the mailing list archive is down. I've poked Mark, but he's not answering his phone at the moment. (It was working last night...)

Not a lot of programming yet today, I biked to the transmission place to help them reproduce the funky noise my car was making (it mostly only does it when it's cold, and it was 90 again today) and managed to make myself vaguely ill in the process (I'm not over this cold yet). Spent the rest of the day reading "Bloodhound", by Tamora Pierce, and doing laundry.

Exciting times.

Ah, I figured out what's going on. I stuck some printf() calls in the qemu source code to see why the curses code was triggering, and it isn't. The curses thing _was_ fixed during development, the current escape codes that were being so weird are busybox ash, trying to figure out how big the display attached to the serial console is. (Exporting LINES and COLUMNS stops it, but it doesn't actually cause the problem the curses thing did.)

Ok, that's a relief.

Mark and I are playing with a new bug generator. (Yes, I'm voluntarily submitting to one this time. Call it personal growth, call it "this isn't bugzilla".) I'm trying to beat behavior out of it that's an actual improvement over a text file roadmap. So far it seems nice, and in need of much configuration. Luckily, Mark is good at this sort of thing. (He can touch stuff without breaking it immediately!)

And Alpha needs the #ifndef inhibit_libc fix I've been applying to sh4. (The gcc-core-fix-inhibit-libc.patch thing.) GCC really isn't all that well-maintained, is it? The FSF doesn't regression test bootstrapping its old architectures... Hey, and the "-O2 causes internal compiler errors during the uClibc build" issue that m68k had _also_ applies to Alpha.

After you've gotten your first dozen platforms working, you really do start to spot patterns. "Anything maintained by the FSF is crap" is a pattern.

And I didn't get around to banging on the qemu -rc releases before 0.11.0 dropped, I'm hoping the curses issue got fixed. By default, qemu's new ./configure enables curses support if it detects you've got it on the host, and curses mode isn't scriptable. (It craps escape sequences randomly into the output, which confuses things like expect.)

During development, curses mode was enabled by default, but the current source code (in vl.c) looks like -nographic and -curses are unrelated. Here's hoping...

Nope. It's still broken. Time to ping the list...

The "um-fest" that is our Ohio LinuxFest talk is now online. We managed to compress an 8 hour talk into 54 minutes, which is about like doing the minute waltz in 11 seconds. We did it by eliminating all the hands-on parts, and the actual demonstrating the software parts, and skipping maybe 1/3 of the slides entirely, and skipping about half the material on the remaining slides, and then ZIPPING through the rest as quickly as possible.

The result was exhausting for me to listen to, and I _wrote_ it. It's fairly dense with insufficient breaks, but that's pretty much unavoidable when you compress a topic that much, even with such lossy compression....

The reason it turned into an "um-fest" is we didn't practice enough at the faster speed, so the normal delivery pace full of _pauses_ had to have all the pauses eliminated, meaning I went "um" between every point while waiting for the next slide because I _knew_I_couldn't_stop_talking_. (And the longer ums are us advancing past skipped slides. And I even got Mark doing it during one section...)

It's slightly less embarassing if you think of "um" as the "advance slide" signal, but still... :)

The first minute with a DJ trying to be funny is just inexplicable, but he came with Ohio LinuxFest and when he asked us for a summary of our company and I gave him the "Impact is how you get embedded" line and... well, listen for yourself...

Oh, the "Max Headroom" delivery is because OLF bought eeepcs to record the talk, and apparently didn't take flash write latency into account (nor test long enough for it to manifest). It's in the original flac, not an mp3 encoding artifact, alas. (Yes, realtime issues affect the real world. Who'da thunk?)

But hey, the recording is up! (And the slides are here.)

On an unrelated note, it occurs to me that Barack Obama uses the Moist von Lipwig style of project management: "Do everything at once, get it done before they can object". (Napoleon also followed this path.) Too bad the federal government doesn't work that way...

Beth Lynn Eicher (chair of Ohio LinuxFest) pointed me at the twitter of some organization calling itself Texas Linux Fest. That thing hasn't been updated in a month, and points at a google group with no public archive (you need to log in not only to post, but to read anything).

Another guy, Bryan Bishop, emailed me after my name was misspelled on slashdot. He's acquainted with the people running it, and pointed me at their new website, which is almost content free. (Other than that they've picked a date, but don't yet have a location... I thought it went the other way around?)

So I went to their contact page and tried to send them email to the contact address that listed:

: host ASPMX.L.GOOGLE.COM[209.85.212.76] said:
    550-5.1.1 The email account that you tried to reach does not exist. Please
    try 550-5.1.1 double-checking the recipient's email address for typos or
    550-5.1.1 unnecessary spaces. Learn more at
    550 5.1.1 http://mail.google.com/support/bin/answer.py?answer=6596
    8si4971359vws.69 (in reply to RCPT TO command)

They have a wiki. It has exactly one page, containing their banner graphic and nothing else.

So much for my attempts to volunteer to help out. I think I'll wait and see what the first year is like...

October 3, 2009

I actually posted to livejournal for the first time in ages, for reasons explained in the entry.

Due to a sudden interruption I actually posted to livejournal for the first time in ages, for reasons explained in the entry.

Where was I?

October 2, 2009

October 1, 2009

September 30, 2009

Taking the UT shuttle bus up to Far West to pick up my car from the mechanics (the engine is fine, they're guessing the weirdness is the transmission). I like Lamb's automotive, they didn't charge me for something they couldn't fix.

I'm amused by the number of different wireless networks that xfce keeps popping up and then disconnecting from. (Keep in mind it only attaches to ones I've previously given it permission to use. I bike around a lot. :)

September 29, 2009

September 28, 2009

Harper Collins put the entire text of Terry Pratchett's new "Unseen Academicals" online. That's not fair. It means I have to go to the bookstore today if I want to get anything done on my computer.

(Ah, it's not the full text, just 70+ pages that cut off abruptly. Oh good, that means I don't have to go buy the thing to stop reading it online if I get hooked, and can thus put off buying it until I get around to it.)

September 27, 2009

September 24, 2009

September 23, 2009

September 22, 2009

September 20, 2009

September 19, 2009

September 18, 2009

September 17, 2009

It's been nice out, so we've had the windows open. This has its downsides. The neighbor who can't park, and who has an even dozen bottles of alcohol in his window (yes I counted, although the only two I remember were vodka and everclear), was outside with a friend of his being loudly racist. Apparently secret service people finding a guy with a sniper rifle near the white house is a laugh line, although a bigger laugh line was "What did they think he was going to do? Shoot the black guy!" (This was after he insisted to his friend that he was going to get an NFL contact because he had a dream about it All done in a heavy rural southern accent, of course.)

This is the one neighbor who has remained constant since I bought the place. The others come and go, but this guy stays, because his mom owns the unit. (He was very proud of this fact back when he came over to yell at me for parking entirely within the lines of my space in a way that inconvenienced him.)

All this does make me seriously consider volunteering for one Obama's health insurance reform phone banks this weekend.

September 16, 2009

Slept 10 hours last night and managed to screw up my neck somehow. Ouch. Kinda distracting.

September 15, 2009

There's more, but that's the top of the todo list for now.

I spent rather a lot of last night trying to unify the binutils and gcc builds for The Great Refactoring, and I've made it up to the wrapper stuff. Now I'm trying to figure out if the wrapper and uClibc++ builds should be in a big binutils-gcc script, or if they should be split out.

The individual binary package tarballs complicates matters, of course. The package tarballs grab the new files between setupfor and cleanup, and since ccwrap.c isn't in packages (it's in sources/toys) it doesn't get packaged. (Currently the wrapper build is done before the gcc build's cleanup is called, so that's not an issue.) Plus building the wrapper shuffles around a lot of the gcc files, and there's no way to represent _moving_ files in a tarball. So packaging the wrapper by itself is problematic.

However, one of the goals of The Great Refactoring is being able to drop in replacements for various components: a glibc build instead of uClibc, a different compiler (tinycc, llvm/clang, pcc, or even a GPLv3 version of gcc) instead of the last GPLv2 version of gcc, and so on. The wrapper is currently tightly tied to the current gcc build, but other compilers might need it too. So if I want to do a drop-in replacement script to build binutils 2.18 and gcc 4.3 (for armv7l/beagleboard) obviously the script should only do that. But a binary package tarball of gcc that _doesn't_ contain ccwrap is kind of useless.

Perhaps I need to do the symlink approach, and do a null setupfor just so I can package the wrapper? I'd still need to move gcc->rawgcc, though. It MIGHT be possible to break off the wrapper build, but do I want to? Hmmm...

In any case, I should be able to eliminate the redundancy between cross-compiler.sh and root-filesystem.sh describing how to build the same packages in slightly different ways. That's the _other_ big goal here, not having to repeat the busybox build between host-tools.sh and root-filesystemm.sh, not having almost the exact same the uClibc build sequence in both cross-compiler.sh and root-filesystem.sh, and so on.

If I want to do a drop-in replacement script to built binutils 2.18 and gcc 4.3 (for armv7l/beagleboard) obviously the script should only do that. But a binary package tarball of gcc that _doesn't_ contain ccwrap is kind of useless...

And then there's the whole export of the funky static c++ support library so uClibc++ can suck it in and use it. That's a very ugly handoff between two different packages. I suppose I could just treat it as a very ugly prerequisite. And right now, uClibc is separated out from the toolchain build, so I suppose separating out uClibc++ makes as much sense.

Wheee. Design issues.

Another fun shell corner case, the "VAR=VALUE command" syntax works with source statements. Ala "VAR=VALUE . snippet.sh" will export VAR=VALUE, read in and execute shippet.sh in the current context, and then set VAR=VALUE back to whatever it had been before. Essentially, source statements have their own shell function context.

Good to know, and quite useful for the refactoring I'm doing. I'm considering making the snippets be sourced so I don't have to say #!/bin/bash at the top of all of 'em. (I'm still ambivalent about the bash dependency and would love it if busybox ash grew the extensions I'm using. I need to test that, and possibly fix it.)

Also poking at the BeagleBoard HOWTO, hoping to get it finished this weekend.

Trying to merge the cross-compiler.sh and root-filesystem.sh scripts (at least the toolchain portions) is educational, even though I wrote it. I'm looking back at these flags and asking things, like why is the cross binutils build explicitly specifying AR AS LD NM OBJDUMP and OBJCOPY, but the canadian binutils build only specifying CC and AR? Why are they _different_? Or for that matter, why is cross specifying --with-lib-path=lib but the canadian one isn't? (I think it has to do with that "lib64" but on x86-64, but don't quote me on that.)

I'm trying to squish this into one codepath, and it's _not_ a straight line. And binutils is only 1/10th as evil as gcc, the FSF hasn't _really_ maintained binutils for a decade now. (It was Cygnus, which got bought by Red Hat.)

Alas, this means I probably need to do more investigation. Possibly I need to look at my old blog entries, although I doubt I recorded what I was doing in enough detail. (For example, OBJDUMP all caps doesn't previously appear in this year, 2008, or 2007's blog entries.) I can dig through "hg annotate" to see where it was checked in, but most likely it was one big lump with a single comment. What I want to know is what happens when each of those isn't specified.

The thing is, over-specifying stuff can give me a build that works for all three of my use cases (creating a basic cross compiler from host to target, canadian crossing a native compiler for the target, and canadian crossing a cross compiler from an arbitrary host to an arbitrary target statically linked against uClibc on its host, ala i686->arm build on an x86-64 host). But it also accumulates cruft. How much of this is still needed for the 4.2.1 build, and how much was 4.1.2 only? What breaks if I try to spin a 4.3 toolchain (which, alas, armv7 needs and that's what the shiny new Cortex A8 in the BeagleBoard is using).

What I probably have to do is get the big overspecified one working, and then yank each symbol one at a time and run all three build variants without it to see what happens. And that's incredibly boring and time consuming. Oh well.

It's so easy to slap stuff together to make it work once, and always so much extra work to do it _right_.

Interesting post from a professional writer.

You are not owed a read from a professional, even if you think you have an in, and even if you think it's not a huge imposition. It's not your choice to make. This needs to be clear--when you ask a professional for their take on your material, you're not just asking them to take an hour or two out of their life, you're asking them to give you--gratis--the acquired knowledge, insight, and skill of years of work. It is no different than asking your friend the house painter to paint your living room during his off hours.

One of the rules about inviting convention guests is that you can't ask them to do whatever it is they do for a living for free. It's HORRIBLY impolite. To use an extreme example: if we invited Weird Al as a panelist, we couldn't ask him to sing. He makes tens of thousands of dollars for a concert, he worked years to achieve that, and no matter how much he may LIKE us, giving us even a little filk session for free would undermine that. Similarly, if we invited a professional author (defined as "one who makes their living by their income from writing"), we couldn't ask for a short story for our program book.

I vaguely recall that when we asked our web cartoonists to do covers for the program book, we bartered for it, giving them some advertising space elsewhere. (And we bought advertising on their website as well, so it was part of a larger commercial transaction where they got paid cash.)

Lots of hobbyist conventions don't understand this. Asking guests for their time so they can be admired and entertained is one thing (and it's entirely ok for them to say no simply because they don't want to, they don't _need_ a reason). Asking them to cheapen their professional services is something else entirely.

September 9, 2009

September 8, 2009

September 6, 2009

Ok, after about 3 days of sleeping 12 hours/day, I've at least gotten past the headache. Whether or not I can concentrate enough to do something useful is another question.

September 5, 2009

Still cold turkey on caffeine. Spent most of yesterday with a headache. I normally try to do this annually, but it's actually been about 3 years since I did this. Oops.

Fade was out visiting friends most of today, so I played Sims 3. (About all I felt up for.) There was a full tin of penguin mints on her desk the whole time, taunting me.

Of course I'd find a store on Guadalupe that sells the Orange Mango Passion Fruit Rockstar energy drinks I like _now_. Bought one. It's in the fridge, taunting me.

Gonna be a long couple weeks.

September 4, 2009

I ran out of tea yesterday, and emptied my current tin of Penguin mints at the same time, so I haven't had any caffeine for over 24 hours. That would explain why I both went to bed before midnight and slept until 2pm.

Visited Mark and started on a BeagleBoard HOWTO, but I'm just too out of it to finish right now.

September 3, 2009

Anyway...

Ooh, The biggest ball of twine in minnesota is 61 miles north of where my sister lives, pretty much a straight shot from new ulm to darwin along route 15.

According to Weird Al, since he did the song they've actually built a "Twine Ball Inn" which sells postcards that say "Greetings from the twine ball, wish you were here".

Next time in Minnesota, I may be required to go to this thing.

Time to play Sims 3 for a bit, I expect.

September 2, 2009

September 1, 2009

Fade was in class all day and her Big Shiny iMac Monolith was just sitting there unused, so I spent most of the day playing Sims 3. (My household now has a ghost baby, from a firefighter mother so she may have the hidden pyromaniac attribute letting her set stuff on fire when she gets older.)

August 31, 2009

August 29, 2009

August 28, 2009

August 27, 2009

Fade spent today in class and I did _not_ play Sims 3 on her computer! (This is an _accomplishment_. I am PROUD!)

August 26, 2009

Fade's classes started yesterday. Today, we did 8 gazillion errands.

The most expensive was that we bought yet another printer, because the Epson CX8400 we had turns out to need its color cartridges replaced when you buy new black ink for it, even when it wasn't complaining about them _before_ I put a new black ink cartridge in, and now it's complaining about all three. (We've printed basically nothing in color so far.)

Called Epsion tech support and the nice lady in India confirmed this is Broken As Designed (not _quite_ in so many words), and since this is too evil money grubbing bastard weasely for me to ever want to give money to Epson ever again in my _life_, and Fade needs a printer for class, we hit Fry's and bought a laser printer. It just does black and white, but even the cheesy "substandard free ink cartridge that comes with it" prints 1000 sheets. And a new ink cartridge is $50 and prints 3000 sheets. We should come out ahead in about two semesters.

I'd mention Mark, but apparently his relatives have found my blog and are using it to indirectly keep tabs on him. (I keep forgetting that people other than me read this. I also keep forgetting to update it. I'm spending a lot more time on twitter than here these days.)

August 25, 2009

August 24, 2009

August 23, 2009

August 22, 2009

Despite Rule 34, the internet does not do requests. (I.E. Googling for "entertain me" is kind of disappointing.)

August 21, 2009

August 20, 2009

August 19, 2009

Fade's birthday. She already bought herself an iPhone, and nothing I get her could top that, so I cleaned the house a bit instead, got her up early (9 am, ooooh), we went for kolaches (with coupon!), hit the library, went and saw Ponyo at the Barton Creek mall (it's Miyazaki, of _course_ it's good), wandered around the mall a bit window shopping, came home and I baked her a red velvet cake (well, cupcakes), and then we watched the second half of the "Due South" netflix DVD.

Good day. You can actually see the kitchen and dining room table again, which is kind of odd. Didn't start banging on the FWL release notes again until after she went to bed, but I finally ground through all the commits since last time.

August 18, 2009

Fade bought an iPhone. I need to go to the t-mobile store and update my phone plan so her line isn't on it anymore. (Yes, the t-mobile guys asked me _six_times_ whether or not I'd seen the new google phone. Not the g-1, the new "mycroft" or whatever it is. They still don't understand _why_ it's not the same thing. I tried asking them when Sims 3 would be ported to the google phone and they never acknowledged the _question_.

August 17, 2009

Phone interview for a job doing processor diagnostic software this morning. It went quite well. It might mean I'd stop having time for embedded Linux development again for the foreseeable future, but oh well.

August 16, 2009

August 15, 2009

August 14, 2009

Armadillcon's opening ceremonies are in half an hour. Trying to work up the enthusiasm to go.

Although I volunteered to help out in the con suite, and repeatedly poked them about it, I just checked my email and the last response I have from the con suite people was two months ago. I kind of doubt they even remember me, let alone have any sort of "counting on me to show up" kind of thing going on.

There's nothing in the panel schedule I want to go see. The closest is the "is Dr. Who speciesist?" panel, which is at least about something I'm a fan of, but do I really want to see a property I like grilled for political correctness for an hour?

Their guests are supposed to be the big draw, but I'm not a current fan of any of them and only even recognize two (one because Fade read one of his books, although she didn't recognize his name either until she looked it up. The other because she's Vernor Vinge's ex-wife, and I know who he is).

In their list of 100+ panel participants I recognize six names. (Seven if you count the fact that Lawrence Person is listed twice.) That's two authors I haven't read anything by since before I met Fade and have already seen at previous Armadillocons, two people I remember from Linucon, a local journalist who I briefly followed on twitter but don't anymore, and the convention's founder who I met at a party once. That's it, out of a list of over 100 people. (Several other names look familiar, but I couldn't tell you who they _are_. Probably the same panelists I saw in 2005, 2004, maybe even 1998...)

Their gaming is all Steve Jackson Games stuff (remember that big bag of games I dropped off at the concom meeting I attended? Either it was useless because they have a MIB supplying them, or it's forming the core of their game room this year). They're good games, but I don't have to leave home to play them.

Eh, maybe I'll go tomorrow. I have a coupon for a $5 haircut that expires today, and to be honest when comparing which I'd rather not miss, the haircut is winning.

August 13, 2009

Drove around a lot today, collecting books and supplies for Fade's ACC classes. Three consecutive office supply stores didn't have Moleskine notebooks, but Barnes and Noble did. (And a spanish dictionary. And a copy of "Endless Ocean", which would be a game about molesting fish if I was entirely convinced it was actually a game.)

August 12, 2009

August 11, 2009

Woah, water pressure today is up to full. I dunno if the city guy who came out did something, or because I didn't get out of bed until 3pm. (No, I didn't sleep through mark's morning call, I just went back to bed afterwards. He got 4 hours of sleep. Yay sleep.)

West campus is sort of dead right now since last year's students have moved out but the new ones haven't quite moved in yet. (Lots of ripping out of carpets, furniture winding up in dumpsters, and cleaning service vans lurking in alleyways looking suspicious.) The rezoning of everything from 3 stories to 5 stories back around 2005 means all the old houses got torn down and replaced by big apartment buildings, and I was assuming the water mains were overtaxed by the increased urban density. But according to the email I just got, they replaced the building's water meter, which was only putting out 20% of what it should have. (Yay, water pressure!)

August 10, 2009

Up until 5am playing Sims 3, slept past noon, the water went out during my shower (to the point where the pipe was sucking air _in_, which can't be good), but the condo association people say that the city of Austin water thingy isn't currently having known issues, and that the city is sending someone out to re-enact Flanders and Swann's "The Gas Man Cometh", but I probably don't have to be here for that.

Still felt tired, lay down to nap for a bit. It is now 9pm. Feels like I'm recovering from a sunburn. Wonder how I managed that?

Fighting with armv4l. This would be easier if I could pick an approach, but there are at least three horrible ways to fix this, all of which suck, and each time I step back to summarize the problem the grass looks greener in one of the _other_ approaches. (Well, not kernel-side soft float, that code's slow and buggy. It's buggy because it doesn't get much testing, because it doesn't get much use, because it's fundamentally so slow. If it was _just_ slow I might go for it, but add in the buggy...)

But I keep going back and forth on the "patch gcc to do this _right_" vs "reorder my scripts and hope I can sequence my way around the sheer horror of gcc" approaches, especially since the second makes it harder to fix the design flaw in my scripts. (Each layer is supposed to be orthogonal, but I recently figured out that root-filesystem.sh has some fairly deeply baked in assumptions that it's using a cross compiler with my ccwrap.c. Unless you BUILD_STATIC, anyway.)

Finally watching the most recent bleach disk (volume 18). This entire disk feels like filler. It's not that it's going slowly (this often happens in anime), it's that NOTHING IS HAPPENING. (Very energetically, of course, but there are reaction shots for everything. A scene went by where every single person in a group of about 8 people had a "stop and say somebody's name in shock" moment. I should fire up my cell phone and see if the "Overtaking the Manga" entry on tvtropes mentions this episode by name.)

Ah, an episode and a half in they finally get their theme music power up, and the Vampires of Gratuitous Renaming turned to dust about sixty seconds later. There were maybe fifteen minutes of plot between the two episodes, combined. Ok, episode three is back to being merely slow. The "long artistic pauses in semi-romantic dialog.

Ah, and the last episode on this disk seems to be the start of a new season in Japan. (There's no obvious correspondence between where Viz the importer split the DVDs and anything the Japanese are doing.) You can tell because the credits changed again, and now the opening theme music is in english. Must boggle.

The Japanese seem to treat english the way we treat latin and french. Throwing the occasional english phrase into Japanese produces a certain "Je na sais quois", QED. This isn't just a phrase or two, the lyrics to this entire song are in english, and it's not exactly playing to their strengths. The singer seems to be doing some kind of vague Bruce Springsteen impersonation that winds up sounding like he has has a mouth full of novacaine. It's english with an andalusian lisp (excuse me: lithp) and occasional glitched bit of syntax, ala "to start up new day". (For some reason the fact the lyrics are nonsense like "tonight love is rationed" doesn't bother me much, because they always were. So are most of ours, really.)

I saw a chinese article the other day about their mars probe, which had a sentence "the mars and the earth", and couldn't immediately explain why it's just mars and not "the mars", but it's ok to stick the in front of earth. I don't blame people for getting english wrong, we do all the time and the british (who invented it) point and laugh at us even when we think we're doing it right. It's an ad-hoc language at best. Latin again.

Wrote up a longish description of the current armv4l situation, then yanked it from here and am redoing it as a post to the FWL mailing list. I should really post this sort of thing somewhere people can _reply_ to it. This blog is really easy for me to do (especially when LJ is ddosed or I'm not on the net), but I've never sat down to do a reply infrastructure. (Heck, I've been putting "span" tags in the source for years, this paragraph is <span id=programming>, but I've never sat down to teach the rss feed creator to do searchable feeds. I know how, I just haven't done it yet. What was that about the cobbler's children having no shoes?

Went over to Mark's today so we could work on podcasts. Wrote up a couple dozen potential podcast topics, and he set up his little camera thingy and we found out we're both quite camera shy when the little red light is on. (No, turning the little red light off would not improve matters.) What happens during the recording is _not_ natural conversation, and I'd like to credit Terry Pratchett for being able to blame "bloody quantum" for it. We need to try again with scripts.

Charles Stevens says that the main problem with the FWL sparc target is dynamic linking, and if you statically link busybox you get a mostly working root filesystem (modulo some unaligned access errors and such which should be fixed upstream in busybox).

Unfortunately, setting BUILD_STATIC in sources/targets/sparc/details turns out to have a sequencing issue: STATIC_FLAGS is set at the end of sources/include.sh, but the architecture config file doesn't get read until after sources/include.sh returns.

Currently only cross-compiler.sh and root-filesystem.sh pay attention to that flag, so if I move setting STATIC_FLAGS to the check_for_base_arch function things will work. But that means that things that don't check for an architecture (I.E. host-tools.sh) can't have BUILD_STATIC support added in future. I think I'm ok with that, though.

I know XKCD does a better job mocking the very idea of Linux on the desktop than I'm likely to, but I just hit one more little bit of fun and I need to vent.

I try to back up a couple of directories to a 16 gigabyte USB key every once in a while. I'm using rsync, which means A) it's ignoring the existing files completely and copying the entire contents of the file over again, even when they're mbox files that just got appended to, B) it's doing so at 83k/second for a file that's about a gigabyte. I went out to lunch and it wasn't finished with the file when I came back.

Apparently, the new version of Ubuntu A) broke rsync somehow (so it's copying the whole file and can't be convinced to append to the existing one), B) regressed the USB drivers so it's mounting it as USB 1.x instead of 2.0 (thus being insanely slow).

Wheee. More stuff that (I think) used to work, and doesn't anymore. (We regret regression test nothing!)

Of course the behavior of the Linux kernel when memory is filled up with disk cache that flushes to a slow device is "fun". And its decisions about how much memory it will allow each device to pin assume they all write at the same rate.

The new "Fedora 11 doesn't work" bug seems to actually be "distcc and FWL don't play nice together", which actually makes sense if you stop and think about it. The clever things distcc is doing break when builds try to do strange things, and both host-tools.sh and uClibc's build are the height of strangeness in this regard. I'll see if I can work around it.

I seem to have actually recovered some enthusiasm for poking at toybox again, which is odd. There's no _point_ to it, I'm not working towards a goal. I just haven't fiddled with any actual programming stuff in C lately, and I'm feeling a bit rusty. It's like whittling a stick to make nothing but sawdust, or building a house of cards. It doesn't need a point, it's something to do.

I wonder if I should do the buffer applet I've been pondering for a while. (As with "count", which became busybox's pipe_progress when I submitted it, there wasn't one when I decided I needed this. Since then I've seen somebody write a halfway version that doesn't do everything I want it to do. A proper buffer command needs ratelimiting too.)

Ok, this is one of the reasons I despise gcc. If you write a file containing one line, "#include <zlib.h>", and you write it as temp.c and run "gcc -E temp.c", it spits out hundreds of lines of the contents of zlib.h. Which it should, because -E says spit out the preprocessor output. But if you go:

echo "#include <zlib.h>" | gcc -E -

It outputs:

# 1 "<stdin>"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "<stdin>"
include <zlib.h>

Why is the behavior different? The file is coming from stdin instead of from a filename, and the preprocessor completely melts down and doesn't do its job.

I mention this because squashfs requires zlib to build, and I want to test if it's installed and skip it cleanly (or die with a more explicit message) if it's not there. I also don't want to assume that the host's headers are at /usr/include, and there's no obvious way to get gcc to dump its header search path. ("gcc --print-search-dirs" shows the library search path, and the executable path (which should just be $PATH but this is gcc so of course they reinvented the wheel). "gcc -dumpspecs" doesn't show it, and note that "gcc --dumpspecs" doesn't work because it's gcc. The uClibc build uses a dirty trick, "gcc --print-file-name=include", to get the gcc internal #include directory. Note that in this case "include" isn't a file, it's a directory, and it's only coincidentally installed in a directory that's in the library search path, but it prints it anyway and all anybody cares is that the trick _works_. But that's not the directory that would contain zlib.h.

Have I mentioned I hate gcc with the flame of a thousand suns? Recently, I mean? This stuff is SIMPLE if you understand it. I can only assume the people who implemented the gcc front-end don't understand what a compiler _does_. It's not brain surgery!

Got another report on the mailing list of somebody else trying to build on Fedora 11 and not being able to. Fired up qemu and trying to reproduce it. Building fine for me so far. (It's only up to binutils, but they said it was the same distcc problem. I thought I'd rsynced my changes up to the server. It's the same script that updates my blog, which has admittedly been a bit patchy lately.)

Somebody else tried to use the m68k target (for which qemu doesn't work), and then fell back to using the powerpc target (qemu again). I pointed 'em at mips, which is a nice big endian test environment.

Got the powerpc kernel build and calling userspace, but it hangs two letters into outputting "hello world" when it gets to userspace. Pinged the qemu list, but I haven't got a _clue_ what's up with that. (This is with today's git version. This is a kernel derived from the supposedly working debian .config, minus all the modules, plus squashfs.)

Heard back from Charles Stevens about the status of m68k, some discussion on the list about that. I need to ping the qemu list about the status of the m68k work, but I need to catch up on reading it first.

Taught the rest of root-filesystem.sh to build static binaries. The only dynamic binaries now are hello-dynamic and ldd (which is a uClibc build issue that seems hardwired, might have to patch the makefile). This could help sparc, since dynamic linking seems to be the main problem with it.

Still haven't tackled armv4l-soft yet. An externally imposed schedule on this project would be nice, my release date's slipped over a month already. (Feature creep is no excuse.)

Poked at powerpc a bit, trying to get a kernel that works with the current qemu-git, rather than the a months-old repository snapshot that was between the bugs in different release versions. Aurelien Jarno was kind enough to send me the .config of a system that's booting for him, based on debian's powerpc kernel, but as with most distro .configs it's enormous, has 8 gazillion modules, and expects you to have an initramfs find the actual boot device so some fairly vital drivers aren't statically linked. (And for some reason, console=ttyS0 doesn't work with this even though I got at least that much working with current -git, now it's ttyPZ0 for some reason. *shrug* Ok?)

Progress has been made, but it's one of those whack-a-mole evenings. Might get more done at the laundromat this evening.

Trying to associate my cell phone with my new bluetooth adapter. (Lost the old one, went to Fry's a day or two back to get a new one.) Hitting the "oh wow Ubuntu's bluetooth support sucks" event horizon again.

Attempt 1: it saw my cell phone, associated, but couldn't talk to the modem. Went into my phone and told it to forget all the old devices. Attempt 2 saw a half-dozen other devices in the area, but not my phone. Obviously it needed to re-assocate with the phone, but the laptop thought it already was associated and the phone wasn't. Went in to the bluetooth GUI and found where the device association list was (this is progress; the Gnome and KDE guis for this didn't have such a list I could ever find), removed it, went back and ran the association wizard again: and now it's seeing no devices. Unplugged and re-plugged the USB bluetooth thingy, still seeing no devices. (No indication that it's scanning for them, no progress indicator, just a window that previously, after I let it sit for a while, had stuff magically appear in it. Now nothing is magically appearing in it.)

Let it sit there for five minutes, nothing appeared. I don't think it's scanning.

I need a mac.

Ah, if "visibility" is on in the laptop, it can't scan for devices. That's kind of stupid. Ok, now it's found the phone again, re-associated with it, and there's no old one of the same name to confuse things. So, only about three attempts to make it work, that's greatly improved, it merely sucks now.

The text mode dvd install went without a hitch. So the older infrastructure left over from the days before Red Hat spun off Fedora works fine, and the shiny new livecd which is the only thing the fedora project points you at doesn't work at all.

I note that the text mode install never let me set the system name or select packages to install, so it's lost some functionality since the old days, but oh well. It gave me something that boots. This is progress.

The Fedora 11 boot has three overlapping progress bars, presumably because it made the boot asynchronous to try to make it suck less. (Rather than trimming out gobs of unnecessary stuff, run 'em in the background. Then block and wait for them to finish. Right.)

The sad part is that all three progress bars have hit the right edge after 58 seconds, at which point the install sits there for multiple minutes, unchanging, before suddenly resizing the window to go into graphics mode and start bringing up X11 and the desktop. (Well, that's what the livecd does, the disk install is giving me a text mode login: prompt.)

Why even _have_ a progress bar that's spends more time waiting at "done" than it did advancing? Rehan showed me that you can hit escape here to replace the progress bar with the older list of unnecessary things Fedora is actually doing, such as launching exim, complaining about the "system message bus", setting firewall rules... So once again they've replaced "ugly and overcomplicated but functional" with "pretty, hides the complication without actually removing any of it, does not actually work".

Anyway, I have an installed system. Login as root and see what's installed:

[root@localhost ~]# gcc
-bash: gcc: command not found

Of course not. That would have been too easy.

Wow, I'd forgotten how utterly crappy installing rpms from an iso actually was. It doesn't autoresolve dependencies, it says things like:

error: Failed dependencies:
	kerhel-headers is needed by glibc-headers-2.10.1-2.i586
	kernel-headers >= 2.2.1 is needed by glibc-headers-2.10.1-2.i586

And then you just add more and more stuff to the rpm -i command line until it stops complaining. Each new thing you add has its own dependencies, which it'll complain about, and eventually, without warning, the whole mess will either just install or die with some strange internal error.

Luckily I remember from my Red Hat days (which ended back around 2003) that the naming is pretty consistent. For non-lib packages use kernel-headers-[tab], and for library packages strip off the lib, use the library name, add a dash, and tab complete. (Doesn't _always_ work, but it's a good first guess.

I have no idea why it's complaining about:

warning: gcc-4.4.0-4.i586.rpm: Header V3 RSA/SHA256 signature: NOKEY, key ID d22e77f2

It's both complaining there's no key, and giving a key id? Um... Ok. (Only package it has problems with, and this is Red Hat's own DVD iso. Oh well. It's not like I'm trying to use this thing long-term, just reproduce a bug with it.)

Heh, I note after all those insanely verbose init scripts, it didn't bother to run "dhclient eth0". But it set up exim by default (I vaguely recall that's an email server or something). Ok, stop making fun of the insane defaults, point established, moving on...

Installed gcc and binutils (which sucked in a dozen other packages). Installed wget. Ooh, Red Hat hasn't got "which", and my scripts react badly to that lack. (Dumped symlinks to "as" and such all over the current directory, that's bad. It didn't abort the script because the call to which was embedded in a "$(blah)" string, so it didn't set an error code I could test for.) Maybe I can move the busybox build up so which is already there? (Presumably they test on Fedora already...) Of course I need to install "make" before busybox will build...

Huh, that could explain the bug Rehan is seeing. The symlinks to the host toolchain were missing and the build didn't abort, so once it adjusts the $PATH to exclude the host tools the compiler isn't there, and thus can't make executables...

Yup, that's it.

So Rehan Khan posted a bug report to the FWL mailing list, which boils down to host-tools.sh failing under Fedora 11 because the distcc ./configure stage says gcc can't build usable executables (even though we're using the host toolchain at that point). I haven't used Fedora in years, so my first step trying to reproduce this is to install Fedora 11 under qemu.

Got Rehan on irc and confirmed that it's the 32-bit version, which is the only obvious option get.fedora.com seems to offer anyway. A livecd that acts as an installer. Ok, download that and fire it up under qemu.

Booting Fedora from the livecd takes about 7 minutes to give me a desktop. If you press escape you can watch it slowly set up LVM and user quotas, oblivious to being a livecd where such things are hilariously inappropriate. You can also watch the init scripts fire up the CUPS network printer daemon and NFS and so on, when the association with the wireless network is done by a GUI desktop app these days so it's launching servers before it has a network. This is a distro that clearly can't decide what it wants to be, so it tries to do everything, at once, very slowly.

Once you've got a desktop, you can double click on the installer whereupon it takes about 5 minutes to grind through painfully slow dialogs to the point it gives you a page which doesn't explicitly tell you it didn't find any hard drives, it just shows you an empty list and asks you to select from it. Some of the selections are what to _do_ with the nonexistent drive, such as "use entire drive", "replace existing linux install", "use free space", and so on. Note that you specify what to do with the drive before you select the drive to do it with, to add an extra little bit of confusion when the drive list is empty.

The reason it didn't find any drives is due to a qemu bug: current qemu-git only notices the first drive listed on the command line, and installing an OS requires both an -cdrom and an -hda argument. I git bisected it to 9dfd7c7a00dd within the qemu repository, mentioned it on the qemu list, and got pointed at a fix. This was quick and relatively painless, if I put the -hda before the -cdrom it either fires up the bootloader or it doesn't seconds after launching qemu, so reproducing is easy. And if you build qemu with ./configure --target-list=i386-softmmu it doesn't take very long, and you can even run ~/qemu/git/i386-softmmu/qemu without installing it first.

The next bug is that Fedora 11's mouse pointer wouldn't move in the current version, but in the one I booted earlier it did. I naturally assumed it was another qemu bug, but this was slow to bisect because it only shows on the desktop, so the test takes longer than the compile. Nevertheless, I very slowly bisected that one to a revision that couldn't possibly affect it (renaming a config symbol), and on a hunch booted the same qemu version three times and got the mouse frozen twice and working just fine the third time. So it's probably not a qemu bug, it's a Fedora bug. Of course.

It's not the only Fedora bug. Fedora seems to be constructed entirely out of bugs. Attempting to install Fedora from the livecd is an exercise in frustration. What a piece of garbage. I started with a 1 gigabyte virtual drive, and it complained it needed 3072 megs to install. Ok, make a 4 gig disk image (dd if=/dev/zero of=fedora-11.img bs=1M count=1 seek=4095) and reboot (7 more minutes, plus 5 to get back through the installer)... and the auto-partitioning ate itself. I think it wanted to create a swap file bigger than a gigabyte, when I was already feeding qemu 512 megs of ram to make sure it wouldn't have problems without _any_ swap. (For comparison, I first booted Red Hat 5.2 with 16 megs of ram. It ran fine.)

Checking the checkbox that I want to review and edit the partition layout apparently does nothing, you have to select custom partition layout from the pulldown. Ok, do that, new partition, mount at /, the default filesystem to format it with comes up as ext4 (huh, that's new) so accept that, hit next... and half a minute later I get a pop-up saying that the boot partition can't be ext4. (On irc, Rehan says grub doesn't support it yet.) Ok, back up, go through it again selecting ext3 this time, save it... and now it says that the root partition must match the "install image" type, which is ext4.

I hate Fedora.

Ok, presumably the "boot partition" above is /boot, and the root partition is /, so I need to make two partitions and format them with different filesystems to humor the hideously overcomplicated Fedora 11 installer. Go through the custom partition gui, make a /dev/sda1 that's 200 megs and ext3 and mounted as /boot, make a /dev/sda2 that's the rest of the space and ext4 and mounted as /, hit next, tell the stupid pop-up that I know I haven't got swap and I'm ok with this, tell the second pop-up that if I didn't want to write the partition layout to disk I wouldn't have hit "next"... It grinds for a bit... And it pops up an error message saying it couldn't save the partition layout. There's a "details" pulldown which says... that it couldn't save. The details provide no additional details. Great.

Ok, fire up a terminal window, become root (sudo /bin/bash doesn't work but su - does), run fdisk /dev/sda... It's managed to save the first partition but not the second. How do you do that, I created two primary partitions and the primary partition layout is a single structure in the boot sector of the hard drive! It's less than 100 bytes for all four of them combined, how do you get something that small _partially_ wrong?

Ok, kill the installer, use fdisk to create the partitions by hand, fire up the installer again, 5 more minutes of grinding, tell to use the existing partitions but format and mount each one appropriately. Re-confirm the various nanny dialogs (wow the Fedora UI is fond of pop-up dialogs)... Yay, it's started formatting. Five minutes later it's still formatting. Fifteen minutes later, it's still formatting. This is the 200 megabyte ext3 partition, even under qemu it should take _maybe_ 10 seconds to format.

Let's go back to that root window and run "top". That says 83% of the Fedora system CPU is beign consumed by "user", and the biggest user is X11 with 4%, and just about everything else is idle. Lemme guess, selinux is enabled, isn't it? Of course it is, this is Fedora. So root is even more crippled than a normal user. (Top doesn't show any processes that don't belong to root, because selinux won't let it see them. Isn't that special?) Ok, new tab running as the install guest user, run top in that, and that can still see all those root processes but now top's fourth and fifth processes are running as the install user... and they're gnome infrastructure crap using about 1% of the cpu each at the moment. What's eating 80% of the cpu? Top can't see it.

Right, run ps and look at the list. There is a mke2fs process, which is a zombie. Is the formatting progress bar still there? Yup. Check again, the zombie is still there.

I hate Fedora.

Ok, kill the installer AGAIN. Format the partitions by hand with mkfs.ext3 and mkfs.ext4 (takes about five seconds each)... And the gnome automounter pops up filesystem browser windows for them! I didn't ask for this, go away! Umount! Right.

On irc, Rehan points out a Fedora bug where certain types of video cards would cause the installer to hang while formatting drives. I stop to boggle at how do you get _that_ wrong? The video card is a piece of hardware the X11 server talks to, the install program is an x11 client. They're separate programs talking to each other through a library API. And the format itself is a child process of the X11 client. How do you build in an incestuous relationship which would _allow_ that to fail? The level of incompetence require to tangle that together and leak information between the two of them so a video card screws up your ability to format a hard disk is just AWE inspiring.

Ahem. Re-run the installer YET AGAIN. (By my count, this is the 13th time. Maybe it'll be lucky.) By about time 8 I was tempted to skimp on some of the details, such as setting the timezone to Chicago (there's no Austin setting) and leaving system name as localhost.localdesktop instead of "tiffanyhat" (what better name for a nonexistent Fedora system than Tiffany Aching's virtual witch hat from Wee Free Men/Hat Full of Sky?), but I fill it all out anyway YET AGAIN despite the incredible slowness of this GUI. (Qemu was fast enough to play "Frozen Penguin" under using a knoppix boot disk back in 2005 on my previous laptop. But it's not fast enough to comfortably navigate window dialogs Fedora 11 running on a system that's more than twice as fast and feeding the emulator 4 times as much memory. Hands up everybody who considers Fedora 11 an improvement over 2005-era Knoppix?)

At this point, the installer is no longer partitioning, because it can't competently do that. The installer is no longer formatting, because it can't competently do that either. (Here's hoping it can copy files! And install the bootloader, because I haven't installed grub by hand in a couple years and don't look forward to looking up how to do that again.) I still have to go into the manual partitioning thingy and tell it where to mount each partition, but I'm NOT MAKING ANY CHANGES so hopefully it... of course pops up a dialog warning me that it's saving the changes I didn't make. It's Fedora.

Yay, it's copying files! Leave it to that while watching "The Daily Show" webcast... And it died with another pop-up window. What this time? Couldn't mount /boot. It copied the files into root, and it couldn't mount /boot. The details say the reason was "input/output error".

Deep breaths. Deep breaths.

Rehan suggests yet again I use VirtualBox. This is A) not the problem, B) not happening. I'm not going to swap an emulator I'm familiar with (and can actually fix or get help with if something goes wrong) for one I've never used. Fedora has BOOTED TO THE DESKTOP with the livecd under qemu, if something was significantly wrong with the emulation that would have caused problems. No, these are Fedora bugs I'm hitting, switching emulators won't help.

Ok, screw this. The fedora install is too brittle to manually fiddle with anything. Every single step where it actually has to _do_ anything breaks. The only codepath that ever got any tested seems to be the fully automated one, and that needs a disk larger than 4 gigabytes. Ok. Kill qemu, zap the 4 gigabyte Fedora-11.img and create an 8 gigabyte Fedora-11.img. (I've got 40 gigs free, and it's a sparse file anyway.) Reboot (7 minutes), grind through the install to the part about hard drives (5 minutes), tell it to autopartion and "use entire disk", sit back...

And it's formatting /dev/sda1. It's been formatting that partition for 15 minutes. For a variant of "formatting" that does not involve the disk usage light coming on at any point. The barber pole progress bar (no idea how long it'll take, but here's a spinny thing!) keeps twirling, but nothing is happening. Been there, done that, not a new bug at this point. The autopartitioning codepath _doesn't_work_either_.

At this point, Rehan tells me that he couldn't get the livecd to work and he installed from the DVD image, using the text mode installer. Of course get.fedora.com makes no mention of a DVD iso, but mirrors.kernel.org/fedora has it if you know to look for it. Apparently, the DVD has more install options than the livecd does, and doesn't need to talk to the network while installing.

*sob*

I really, really, really hate Fedora. I'm going to bed now.

Another fun little bug in xfce's Terminal emulator: when you "detach tab" from an 80x25 terminal, the new one ain't 80x25. They're usually more like 77x21. (Took me a little while to figure out why terminals were spontaneously resizing themselves so small that menuconfig would refuse to run.)

I've mostly stopped noting the random bugs/regressions/UI deficiencies in Linux/Ubuntu/Xfce anymore. Yesterday it froze and I had to hard reboot it and it was just another day. Windows users don't really comment on it either. Linux will never have a significant number of nontechnical end users, just a few holdouts like me remembering the glory days of when it coulda been a contendah, just like OS/2 or the Amiga. Linux runs the servers and routers and so on that Unix has spent the past 40 years running, like Solaris and AIX and Xenix before it, and the desktop-inclined of us turn our attention to MacOS X.

Pondering yesterday's chicken-and-egg problem with uClibc, it occurs to me that the library search paths don't get modified until _after_ the new compiler gets built. So currently, building uClibc first doesn't do any good, the one out of the supplied compiler toolchain is the one getting used.

In theory the reason for building it first is because the supplied cross compiler toolchain could link against a different libc. But in practice, the way we redirect the library search paths is to set an environment variable that points the wrapper script at a different directory, and if the supplied toolchain isn't using our wrapper script doing that early isn't going to help. (Plus I _think_ you can feed it multiple directories with a colon separated search path, but it's been a while since I tested that. It needs to check the new directory and then fall back to the old one, and throwing canadian cross stuff in here with two compilers, potentially both using our wrapper script, gets ugly. I'd need to put the architecture name in the environment variable name to distinguish between them, and since you never need to canadian cross to build anything _other_ than a compiler, I'd rather not add that complexity to the infrastructure for this one case.)

The alternative is to accept that the canadian toolchains we build will be built with the --disable-shared toolchains we build. I.E. the uClibc we build for the canadian toolchain will have the same binary API as the uClibc in the target toolchain supplied to the canadian build. That's true now, and if it becomes an assumption we can _rely_ on, then the uClibc build can move back to after the binutils/gcc build...

Darn it, except there's NO GUARANTEE THAT NEW TOOLCHAIN RUNS ON THE HOST. (Whole point of canadian cross.) So building a new toolchain that's --enable-shared and thus has soft-float support doesn't help us then build a uClibc that depends on soft-float, because we can't run it as part of the build!

I continue to hate the gcc developers. And cross compiling in general. I'm motivated to provide a sufficiently generic solution to the cross compiling problem that it GOES AWAY and nobody has to do any more of it. I'm aware this is difficult, but cross compiling is an inherently horrible problem. I've been doing this for years and the tracking the multiple contexts still bites me repeatedly during the design phase, especially after a couple weeks away from the problem space.

Darn it, I know how to do this, and it's a special case. I don't want it to be a special case. Is there any way to make it NOT be a special case? I want a simple API, insane amounts of granularity, and a calling script that's simple to the point of being disposable. These goals conflict a bit. Hmmm...

Firing up Firmware Linux and looking at the code again. Where did I leave off? Fixing armv4l, that's right.

To get armv4 working again (an example soft-float target), I've been poking at adding the soft float stuff to libc.a like I was doing for 4.1.2. This is because gcc only puts soft float support in libgcc_s.so and libgcc_eh.a, and for some reason it doesn't build libgcc_eh.a for the --disable-shared version. Yes gcc is clearly broken, but patching each new version is a pain. The patch I have for 4.1.2 doesn't apply to 4.2.1, and trying to unwind its pathological makefiles already ate about three evenings with no obvious progress. (Horrible, horrible code.)

I've been looking at genericizing the uClibc miniconfig stuff, comparing the various target configurations. The uClibc configurations are mostly identical, which makes sense since they started out with a single hand-rolled one and got modified for each target. However, they do vary a bit.

The i586, i686, and x86_64 targets use the small MALLOC version, the others use the full one. The armv4eb target has UCLIBC_LINUX_MODULE_24 switched off, which is the old 2.4 kernel module support that doesn't apply to the 2.6 kernel anyway. The armv5l and armv6l targets have several additional symbols switched on: DOPIC, LDSO_LDD_SUPPORT, UCLIBC_HAS_ADVANCED_REALTIME, and DOSTRIP. Mark added those targets. And sparc has FORCE_SHAREABLE_TEXT_SEGMENTS, I remember needing to add that but forget why.

Other than that, the TARGET_blah symbol for each target (and corresponding CONFIG_586 and similar sub-symbols), ARCH_WANTS_LITTLE_ENDIAN, and UCLIBC_HAS_FPU vary per target, as you'd expect them to.

There are also ARCH_HAS_MMU and ARCH_USE_MMU variations per target, which is weird because none of the targets are NOMMU yet. Possibly some of that is miniconfig filtering out symbols that can't be switched _off_ for various targets, and thus there's no reason to record the symbol because it doesn't matter for that config. (I'm not quite sure what the difference between the two is, actually. I can understand the kernel caring about the difference between having an MMU and using it, but I don't think userspace does. The test/build/check_config_options.sh script has a test "Make sure nothing uses the ARCH_HAS_MMU option anymore", so apparently USE_MMU is the only one that matters? Except it hasn't been removed from current -git... I'll ask on the list.)

The actual breakage in FWL is building uClibc in cross-compiler.sh. Right after building a --disable-shared compiler, it builds uClibc with that new compiler so it has a C library to make a toolchain that can build actual executables. Building uClibc with a compiler that hasn't got _either_ hard or soft floating point support breaks, because the uClibc config for each target enables UCLIBC_HAS_FLOATS (and even DO_C99_MATH), which doesn't work on a no-FPU platform without soft float.

It looks like the _easy_ way to get armv4 working again is to build a stripped down uClibc just for the --disable-shared cross compiler, one which doesn't have floating point support enabled. If I factor out the target-specific bits of each uClibc miniconfig and append the generic ones to them, I can supply a slightly different generic bit based on context, hence looking at what would be involved in factoring all this stuff out. Then you can canadian cross an --enable-shared cross compiler right afterwards, and build the rest of userspace with it. (This would make armv4l require the canadian cross toolchain step to build its root filesystem, but it would avoid patching gcc.)

Darn it, no it wouldn't. When you're doing the canadian build of the --enable-shared compiler, it tries to build the uClibc build with the existing cross compiler, _before_ doing the new --enable-shared gcc build. (The canadian shared cross commpiler build is based on the root-filesystem.sh build, not the cross-compiler.sh build. The sequencing is different and it builds more stuff.) It _has_ to do this, so libgcc_s.so has a libc.so.0 to link against. So you still need soft float working before you can build uClibc, and you need uClibc built before you can build the shared compiler, and soft float is only enabled in the shared compiler. Throwing in the canadian cross stuff just makes the dependency _circular_. Great.

I really, really, really hate the gcc build. This would be _trivial_ if I could tell it "build libgcc_eh.a but not libgcc_s.so", but of _course_ that's not orthogonal. Sigh.

Attended the Armadillocon concom meeting today. Dropped off the big bag of games Fade culled from her collection (which still takes up an entire bookcase to itself, she mostly got rid of ones she playtested so many times she's sick of them). The con suite person wasn't there but I talked to the guy who was, who might be her brother? Nothing new there, but I showed the flag.

Then we went out to a restaurant and they started talking about how nobody's put in a proposal to chair next year's Armadillocon, and one thing led to another and I now have about half of such a proposal written up, with guests to invite and bits of a budget and list of positions that need to be filled and so on.

This may not be entirely advisable. (It certainly wouldn't mesh with spending the next few years in England, although I doubt that will actually happen...)

The cron job thingy seems to trigger when _any_ unstable package is selected. The "none" build works fine, but busybox, uClibc, and linux unstable builds all fail at the same place (binutils getting confused about makeinfo).

This is progress: it means it's not a busybox issue. (So it's not that the unstable busybox in build/host is screwing stuff up.) It also explains why I've been having trouble reproducing it: it doesn't happen on the first build. I have no idea WHY it doesn't happen on the first build, but I have a new place to look.

So the cron job on securitybreach isn't working:

WARNING: `makeinfo' is missing on your system.  You should only need it if
         you modified a `.texi' or `.texinfo' file, or any other file
         indirectly affecting the aspect of the manual.  The spurious
         call might also be the consequence of using a buggy `make' (AIX,
         DU, IRIX).  You might want to install the `Texinfo' package or
         the `GNU make' package.  Grab either from any GNU archive site.
mv: cannot rename '.am11916/bfd.info': No such file or directory
make[3]: *** [/home/nightly/firmware/build/temp-i686/binutils/bfd/doc/bfd.info] Error 1

I don't know why it's having this problem. It only crops up with the alt-busybox build, not with stable busybox, but when I run the alt-busybox build on my laptop, or from the command line as my normal user on securitybreach, it doesn't reproduce.

It's not a specific version of busybox-git with a transient error, because it's happened three nights in a row. It's not something to do with gentoo because the one without alt-busybox builds fine. It might have something to do with being the second build to run (after the build with all stable packages), but I don't see how since I move the build directory to triage.$STAGENAME after copying out the needed stuff so any leftover debris from a previous build would be in an entirely different directory (and the build does an rm -rf on the temp directories anyway). It might have something to do with running from cron context, since it's not happening when I run it from the command line, but what would that be, tty initialization causing a busybox command to abort? Seems unlikely.

Went to Mark's yesterday and got the cron jobs working. Only the "build all stable packages with the most recent scritps" build is doing anything useful, the unstable busybox, linux, and uClibc builds are all horked for various reasons. Need to fix that.

Being upset at XFCE because middle-click on a Terminal tab closes it, and there doesn't seem to be any way to stop that. My laptop has two mouse buttons and hitting them both at the same time emulates middle click, but they're right next to each other and meant to be hit with the thumb so accidentally clicking both happens a lot, such as when I'm trying to switch tabs and accidentally close the tab I'm trying to switch to instead. I want it to stop. It never occurred to the XFCE designers that this might be a problem. (Or the Mozilla designers either. There's already a little "X" icon on every tab to close it, but they had to make another "kill my program with no warning" option. That's just brilliant.)

I think this might be a window manager option, except the window manager thing only lets you change keyboard shortcuts, and specify what double-click does. It won't let you specify what anything _else_ does.

It's very Unixy that even after fifteen minutes of Googling I still can't figure out whether fixing this problem would require a change to X11, to the window manager, to the desktop program, to the actual Terminal program, or whether it's controlled by something else entirely like the touchpad driver or something. "Delegate it until it goes away" is still the Linux desktop motto, and I still need to buy a mac.

For some inexplicable reason, the busybox developers reverted netcat to the old upstream version in 2007. This means that -f went away, -l no longer announces its dynamically allocated port number, and so on. No idea why they did this, but my plans to switch from using toybox netcat to busybox netcat have been derailed by the fact that busybox netcat can't do the "tar c blah | netcat $(netcat -l tar x)" trick anymore. They broke it for no apparent reason.

Busy night last night. I was up until almost 6am, and didn't get up again until 1:30. But I got a lot of work done, and it's mostly working. And now I've got to disentangle it so I can check it in.

The overall point of the changes is to update build.sh so it can do static cross and native compilers via canadian cross. Previously buildall.sh could do that, but build.sh couldn't, which was wrong because really buildall.sh should be calling build.sh. (Implementing two versions of the same thing in two different places is wrong because it's unnecessarily complex, but it's sometimes hard to figure out how to simplify it.)

So once this is in, you should be able to do:

STATIC_CROSS_COMPILER_HOST=i686 ./build.sh i686

Or even:

./build.sh i686 && STATIC_CROSS_COMPILER_HOST=i686 ./build.sh powerpc

So all ./buildall.sh has to do is call ./build.sh for the host first (i686 or x86_64, or heck if you really _are_ building on a powerpc host...) And then do the FORK=1 thing to build for all the _other_ non-hw targets in parallel, and then build the -hw targets.

But the changes to build.sh are the _last_ checkin in the series. The other stuff is infrastructure changes to prepare for that. So what comes first:

Lots of the work was to teach the various build stages to use $STAGE_NAME as the target directory for the build. (Mostly root-filesystem.sh, which gets called repeatedly to create the static and native compilers as well as the root filesystem. Also update check_for_base_arch and create_stage_tarball to use STAGE_NAME for the tarballs, and change the === announcement lines to use STAGE_NAME.

I also taught root-filesystem.sh to check for both prerequisite compilers for the canadian cross stuff.

I removed the NATIVE_TOOLDIRS and /tools support, because it's just not being used (and updating it for the rest of this is too fiddly). The isolation purpose it serves for Linux From Scratch is done by host-tools.sh here, so it's somewhat redundant. I replaced it with a ROOT_NODIRS flag that just controls whether or not to create the full set of directories (/usr, /tmp, /etc, /mnt, /home, /proc, /sys, and so on), or just let the install create what's actually used and leave the rest out. (Disabling that behavior is good when canadian crossing the static compilers, the result is simpler. The compiler directory doesn't have to pretend to be a complete root filesystem.)

The changes to buildall.sh and smoketest-all.sh are still underway. (By the way, one of those files having a dash and the other not having a dash is ugly, but I'll worry about it later.) I broke up the old doforklog so the forking and the logging behavior are orthogonal. Added a maybe_fork to functions.sh, and a maybe_quiet that you pipe the output through to filter out everything but the === lines. (Alas, this bit's still fiddly because I want to log the output _before_ maybe_quiet, but that's just getting the sequencing right: tee to the log first, then pipe the output to maybe_quiet.)

Rushing to get this done before heading to Mark's, so I can get the cron job installed. (Step 1 is moving it over to the nightly.sh user so I have a clean environment with a known hook I can hang functionality on...)

The main reason buildall.sh isn't calling build.sh directly is that doing the canadian cross stuff has prerequisites that need to be completed before later stages can continue. Specifically, the two source toolchains (host and target) before you can canadian cross a static uClibc toolchain.

However, one possible way around that is to call build.sh for the host first, and then call build.sh for all the other targets afterwards.

Redoing build.sh to integrate the extra features buildall.sh had. I've wanted to do this all along, just couldn't figure out how at first.

Unfortunately, doing this has triggered a largeish untangling of things like NATIVE_TOOLSDIR. Right now's a fairly bad time to do this, of course, because I'm meeting Mark tomorrow to get the cron job stuff installed and I have to drive up to Minnesota on the 8th (although there's a homeowner's meeting on the 11th... Timing. Grrr. I couldn't go to the LAST one because I was out of town too. Highly tempted to just bump the visit to my sister to after sword camp instead of before it, but then I'd be dragging Fade along because she's flying up and driving back with me...)

Ok: combine $CROSS and $NATIVE_ROOT into STAGE_DIR. Make cross-compiler.sh and root-filesystem.sh use it. Then decouple the two things NATIVE_TOOLSDIR is doing: moving the top level directory into a subdirectory (changing the dynamic linker path and such), and populating a full set of directories (etc, tmp, proc, sys, home...)

What I really need for the static cross compiler is to not do _either_ of those things. No full set of directories, and no relocating the top level directory either.

Ok, replace NATIVE_TOOLSDIR with ROOT_TOPDIR and ROOT_NODIRS. Replace $TOOLS with ROOT_OUTPUT... Sigh. Do I really want to keep the old Linux From Scratch /tools directory support? If you really want one of those, you can build it yourself. And if you do a static busybox and static toolchain, you can move it anywhere you like. (Well, modulo the boot scripts saying #!/bin/bash instead of #!/tools/bin/bash...)

AAAAAH! CRAPECTOMY TIME!

Possily ripping out more functionality than I really should, but it got too complicated. I want simple. (You can tell it's 4am, can't you? Technically this should be the July 5th entry. I go pass out now.)

Took my last antibiotic pill last night. That's round 6 for this sinus infection. The fact I had a sore throat before I'd even run out of pills isn't filling me with confidence, but oh well. Hopefully it's an unrelated (viral) cold.

Woke up tired. Took a nap for 4 hours. Woke up tired again, but this time covered in cats. So there's that.

This episode of Bleach is trying to be comedy, and none of the cultural references translate unless you're japanese. I can tell they're parodying specific things, just not what. The entire episode is designed to be an aside and not affect the main plot at all. (I think it's a special for their 50th episode.)

Hmmm, episode 52 introduced new credits again. Ah, "if you continue you may die before you reach $PLOTPOINT". It's Ichigo being addressed. Hands up everybody who remembers his limit break.

Ooh, did red haired guy pick up Ichigo's limit break? Nope, apparently not... Ah, I see, he's channeling Ichigo. Well, that works. No, it didn't. I'm starting to think a flashback montage right before an attack is a bad sign in this universe.

Do Japanese people really throw random english phrases in conversation whenever they want to emphasize something? I suppose it's a bit like english speakers with Latin and french...

So where did I leave off yesterday? I found out that the reason the "FORK=1 ./smoketest-all.sh" script was failing was that formatting a dozen virtual partitions (to create the "hdb" images to mount on /home inside each qemu instance) was causing so much I/O contention that the timeout was elapsing before any of the actual tests could even start. At a guess, mke2fs has some fsync() calls in there, and having an 8-way server with 32 gigs of ram doesn't help if flushing to disk is the bottleneck.

The easy way to deal with this is to add an option to _not_ create the hdb images, because in this case they're not needed... Ok, done.

Ok, I like this president. Watching the video of Obama recording the voice for his Disney hall of presidents incarnation:

Obama: Are these like wax figures, or holograms?

Disney guy: Audio animatronics.

Obama: Animatronics.

Disney guy: Robots.

Obama: They're robots. Well that's kinda cool.

Geek president!

Hmmm... Ok, when I run "./download.sh --extract" I want to log both the stdout and stderr output, but for buildall.sh I only want to _display_ the stderr output. I can merge the streams to put them into the same log ala "./thingy 2>&1 | tee blah" easily enough, but can't separate the streams again afterwards to display only one.

Ok, Obama's speech at the Radio and TV correspondents dinner was totally, totally upstaged by John Hodgman's follow-up.

There really, really, really should be an easier way to do this:

function genocide()
{
  local KIDS=""

  while [ $# -ne 0 ]
  do
    KIDS="$KIDS $(pgrep -P$1)"
    shift
  done

  KIDS="$(echo -n $KIDS)"
  if [ ! -z "$KIDS" ]
  then
    genocide $KIDS
    kill $KIDS
  fi
}

I suppose I should probably call it something other than "genocide", even though that's what it does to a process tree.

The xfce terminal program lets you change font size (either with the preferences toolbar button, or the edit->preferences menu entry), and when you do this it affects every open terminal. Both KDE and Gnome's terminal programs let you change this per-window.

Having a dozen different implementations of each major piece of functionality isn't much use when they're not designed to mix and match. I'm using kmail with xfce but when I click on a link it pops up abiword, not a browser. (The way you change kde file type associations is through the web browser, konqueror. No, it doesn't make any sense, but it's what you do. Unfortunately, if you install Konqueror on a non-kde desktop, it crashes when you try to bring up its preferences dialog.)

Oh, and the first time I tried to change the font size, all my open terminal windows (with multiple tabs) closed instantly:

[145829.419872] xfce4-terminal[3417] general protection ip:7f7d8a9d4e9d sp:7fff93cb7700 error:0 in libvte.so.9.5.0[7f7d8a9b6000+11f000]

Wheee...

I learned a new thing about how shell scripts work! (Translation, I spent a couple hours trying to understand a really strange bug that bit me.)

In shell script, a for loop works like:

for i in one two three
do
  echo $i;
done

The problem with that is if you want multiple related arguments (for i in "one 111" "two 222" "three 333"), you have to split them yourself, which is fiddly. (You can do it with awk, with cut, and a couple other fiddly ways that tend to be brittle.

The dirty trick to get around it is to do this:

echo -e "one 111\ntwo 222\nthree 333" | while read one two
do
  echo $one $two
done

The problem is when you try to set a variable in the loop for use after the loop, or return from a shell function inside the loop. The problem is the second half of the pipe runs in a subshell (it's a separate process), and the entire body of the loop is considered the second half of the pipe. So you're setting variables in the subshell or returning from the subshell, and that has no affect on the parent process.

That's subtle and fiddly and evil. I understand why it's doing it, but it took me by surprise anyway.

Watching mythbusters season 1. Highly entertaining. (One of the episodes you can get on dvd through netflix mentioned the rocket car, but the dvds didn't actually contain that episode. Crazy, eh? It's such a chopped down subset, with the individual episodes edited differently than they actually aired. You have to go to bittorrent to get the real ones.)

When it gets to be 5:30 am, and meanings get reversed and then unreversed in the same bisect, it's hard to work out which of good or bad your next bisect should be. Screw up once and your result is useless.

Resuming the next morning isn't a whole lot of fun either. Where did I leave off? (And now you know why I keep this blog!)

For bonus points, the squashfs bug occurs intermittently. I've run a kernel with it twice and had it run to completion the first time and die with the squashfs bug the next. That's going to be fun to track down if 2.6.30 still has it once I've got the boot bug out of the way.

Tracked the bug down and posted about it on the list. It turns out that there's a new config symbol I need to switch on (which is described in the commit message). So all that work was to change a single config symbol. (Not even a code patch.)

Par for the course, pretty much. If it seems like I'm doing a huge amount of work for very little progress... It's pretty much all like this. (Pretty masochistic hobby if you stop and think about it. Oh well.)

Several people pointed out to me in email that I can go "git archive HEAD" to export the current version. The reason that didn't work when I tried it? Because "head" doesn't work, it must be all upper case, even though "master" is lower case. (Yeah, that's git.)

I also read the man page for git bisect. (git help bisect == man git-bisect, therefore if the git man pages aren't installed, which they aren't by default, git has no help.) Anyway, it turns out there is a way to specify a skip result from the run script, which is to exit with the magic error code 125. No rhyme or reason behind it, because this is git, but it's nice that there is a way. (I'm still not using it, since it's trivial to do by shell script.)

The part that isn't trivial is that my script has so far done bad, good, bad, skip, skip, skip, skip, skip, skip, skip... It may go through all 1398 remaining revisions individually? Now it's done it twelve times. It takes several minutes per run, I'm watching Bleach while it does this...

Bleach continues to be weird, not that I expected different. Up to disk 12 now. Episode 46 was a giant gratuitous flashback for secondary characters, and now 47 is apparently internal politics on the part of the antagonists, and now we're back to protagonist boy Ichigo dueling with the antropomorphic personification of his sword. (Yes, really. While werecat woman watches.) Episode 48, more internal politics with long dramatic swaths of recycled footage and the same three songs, and now the catgirl is wondering why Ichigo isn't levelling up during training (because she hasn't pushed him to limit break yet, you've got to beat him silly before he levels and the current session hasn't even drawn blood yet). Well of course they shortened the deadline, that's plot 101. And now episode 49, and it's flashback land again. "Don't worry, it's just a reconnaisance mission". Oh yeah, to quote ghostbusters "this chick is toast"...

And the video froze again... And the disk icon won't let me eject it because "mount volume" is where eject normally goes. (Why would that be related? I want to eject the unmounted volume, why does the gui not give me that option? Make it orthogonal, guys. Of course the eject button on the actual drive still doesn't work.)

I still need to buy a mac. It might also be buggy crap, but then if I complained about it there would be someone to complain to who wouldn't imply its lack of quality was somehow be _my_ fault. Also, the mac has had an impressive improvement trajectory since the introduction of MacOS X, while Linux has pretty much been treading water on the UI front.

Back to Bleach. Ah, whatserface didn't die, just the red shirts. Ok, hang on, she's posessed? And slaughtering everybody. It's kind of odd the huge power disparity amoung the soul reapers, the nameless ones are one hit kills. ("If we help him, what will become of his pride?" Sigh, they've gone all insanely japanese again. And he's dead because of it. Oh _now_ you're willing to kill him. Sigh.)

Ok, this bisecting isn't getting anywhere, it's gone through well over a dozen iterations of skip and every one it looks at still build breaks with:

/home/landley/firmware/thingy/build/temp-powerpc/alt-linux/arch/powerpc/include/asm/highmem.h: In function 'kmap_atomic_prot':
/home/landley/firmware/thingy/build/temp-powerpc/alt-linux/arch/powerpc/include/asm/highmem.h:98: error: implicit declaration of function 'debug_kmap_atomic'

What I probably need to do is grab the current revision it's looking at and the last known "bad" one (which reproduced the error I'm looking for, meaning _this_ error was fixed), find the fix, make a patch, and bisect between the last known "good" and "bad" applying the patch to each iteration that's horked by this bug.

Anyway, back to _this_ bisection. I've got the last commit bisect found that manifests the problem I'm actually interested in, and I can grab any of the commits it's been stuck on which show the problem I'm _not_ interested in (but which git bisect skip is too stupid to get past), and then bisect those two find the patch that _fixed_ it, then apply that patch to re-test broken versions in the original bisect and hopefully actually make _progress_.

This is another fun case of "good" and "bad" having to be redefined, since each is showing a different bug:

git bisect good 18b41f1cd537168a886c43237297692ba8d0a143 # shows build break
git bisect bad a0e0404fb06164100991cacf8e055f6b30f87cc9 # shows hang

So here, "good" means "the bug I'm trying to find the fix for in the short term", and "bad" means "the bug I'm ultimately trying to find a fix for". Wheee... I hate git.

And hey, one of the intermediate commits shows the squashfs bug, but not the build break. (It's e0cf8f045b2023b0b3f919ee93eb94345f648434 and yes, that's how git expects you to refer to it. You can use any arbitrary consecutive subset of those digits, but assuring you're using enough to avoid collisions with any of the tens of thousands of other hash codes the tree is made up from is your problem.)

Anyway, that commit shows _NEITHER_ of the bugs I'm looking for (it instead shows a third bug unrelated to either do; did I mention that the general testing level of non-x86 kernels is crap?), and that's progress! The last "good" showed the build break, so this is after that was fixed. The last "bad" showed the hang, and this is before that was introduced. So depending on what I answer here, I could track down either the fix for the build break or the introduction fo the hang, but since what I want is the hang and this proves that it wasn't introduced in the range of commits covered by the build break, I can answer "good" here and change what I'm looking for (mid-bisect) to track down that hang!

Did I mention git's user interface sucks rocks and it's essentially unusable if you don't know how it was implemented?

Late. Tired. Cranky. Feeling kind of nauseous actually (15 penguin mints in one night may be a bit much, even of the chocolate ones). Finish up in the morning...

Bisected a powerpc problem and reported it upstream. Found out I'd bisected it wrong, and that the list I was supposed to report it to is A) not on kernel.org but on ozlabs instead, B) unlisted in their index.

Wrote a script to automate the bisection. Somebody pointed me that git's had yet another random feature bolted to the side of it in a hap-hazard fashion that I might have been able to use instead. Decided I didn't care. (And I decided that _before_ I found out that there's no way to indicate a "maybe" response to git bisect run, which makes it kind of useless because I've hit two unrelated bugs already in the bisect. One build breaks the kernel, the other one screws up squashfs some time after the root filesystem is mounted. The one I'm looking for is distinguishable in the kernel boot messages, before init runs.)

Ok, current Ubuntu's uselessness factor continues to rise. Attempting to suspend my laptop, I get the pop-up "Request to do policy action: multiple applications have stopped the policy action from taking place. vlc: Playing some media. vlc: Playing some media."

First, no running software should be able to the system from suspending. Second, there are no copies of vlc running anymore, they exited before I tried to suspend but this status _leaked_.

Right, BIG HAMMER TIME. Ooh, it' s been a while, what do I need to do... switch to a text console, become root, "echo -n mem > /sys/power/state"...

Back from suspend. (It worked.)

Ok, here's my objection. It's not that the system was just complaining about nothing, it's that a sus

The status leaking might be because doing something simple like extracting a linux tarball paralyzes the rest of the system for a minute or more at a time. (For example, in the above sentence, vi hung after the "m" in system and nothing else I typed displayed until the period after "time". I could have typed _way_ more in that gap, but I stopped because cursoring back up to fix typos three sentences back is annoying.)

This means that a bisect script which uses "git archive" to feed a tarball into my build system can't just run in the background and be ignored, it makes the rest of the system INSANELY unresponsive. Firefox also hangs for upwards of a minute at a time (not only can I not scroll the page, but if I move another window over it it won't _redraw_). And it hangs like this about every 3 minutes. On a dual core 1.7 ghz laptop with 2 gigabytes of ram.

Yes, the "Linux on the Desktop" people have been trying for 15 years now, and this is what you've come up with. Moore's Law says they've now got 2^12th times as much memory, processor power, and disk space as they started out with and this is still the best they can do. (How many years did they waste insisting that the display driver (X11) needs to run as a normal process so normal system load can freeze your mouse pointer, making it hard to even queue up actions for the system to do when it stops playing with itself?)

An invocation I've found useful:

git archive $(git show | head -n 1 | awk '{print $2}') --prefix=linux/ | bzip2 > ~/firmware/thingy/packages/alt-linux-0.tar.bz2

It would of course be nice if "git archive" could just spit out the current version without having to be told what version that _is_, but this is git we're talking about. There's probably a way 300 lines into a seemingly unrelated man page, but you'll never figure it out yourself.

In that way it's very much in the spirit of vi or emacs. You don't learn how they work by playing around with it. You learn by grabbing a spellbook and reading through hundreds of pages of incomprehensible jargon in ancient dead languages to become worthy of discovering the mystic incantion you will NEVER figure out otherwise. And in reality, you learn about 5 functions it does and ignore the other million settings on the Sonic Screwdriver unless you're the guy who _built_ it.

And this is why I vastly prefer Mercurial.

The reason powerpc is giving me so much trouble is that too many different things have changes since the last time I had it working. The qemu version is different (largely because of laptop hard drive failure and subsequent reinstalls), the linux kernel version I'm building (and thus trying to boot) is different, the compiler I'm building it with is different (4.2.1 instead of 4.1.2), and my build scripts have changed. When I run across any particular symptom, _and_ of that could be the cause. (Or it could be one of the other things that changed like the busybox tools developing a subtle sed bug, been bitten by that before.

Dug up the mailing list thread from last time a random openbios change broke powerpc, and it says svn 6657 used to work. That translates to git 2d18e637e5ec, so let's fire that up and build it verbatim... And it works. Yay.

But in this case "works" means it boots the _old_ binaries. The ones I'm currently building don't give me any output at all, which implies the serial console is horked. Hmmm...

Now I'm iterating through different versions of my build scripts (much faster) to see where I introduced whichever tool broke it. (That way I'm only testing along one axis.) The 0.9.6 release worked. The last version right before I switched to gcc 4.2.1 worked, so it wasn't the infrastructure changes since then. Commit 752 (first "expected to work" version after the compiler switchover fallout had shaken out) worked. So it's not the compiler. Now we go more slowly. Next commit (753) updated busybox version... and that worked, so it wasn't busybox screwing up the build tools. The actual kernel switchover to 2.6.30 was 754... Which doesn't build because the powerpc config needs the gratuitous new symbol to avoid build breaks. Ok, tweak that, and...

invalid/unsupported opcode: 00 - 12 - 00 (00009024) 0000938c 0

Now it's failing with a _reason_, which is an improvement. But it's definitely the kernel update that's doing it.

Of course now we're back to "two changes which can't be separated" territory; is the new kernel horked or does the config need to be changed more? (These are sort of aspects of the same question, but still...)

Now I need to bisect through kernel versions, of course. (Bisecting through things that take several minutes to build is way slower than it seems, because I switch to other windows and get distracted and don't come back promptly. It can take me an hour to go through three versions if other things are interesting enough that I forget what I was doing...)

Ok. Dug up the release binaries of system-image-powerpc, which was known working with some previous version of qemu as of 3 months ago (albeit with some patching). The two new problems (the -hda argument actually setting /dev/hdc, and init segfaulting with the kernel trying to access memory it shouldn't) are both in the old binary, which means qemu is what changed.

Bisecting through qemu revisions that need to be patched to have any chance of booting anything sucks. Especially when different ranges of the history need different patches...

Ok, 179a2c1971 can't find _any_ hard drive with this bios, 2fbc409571 can. Looks like 513f789f6b18 might be the relevant change between those... And it is. That's where qemu switched from using hardwired layout info to querying openbios (and hence the device tree) for it. The new layout has hda/b and hdc/d swapped (the primary and secondary IDE controllers, apparently), and the panic due to accessing some invalid resource seems similar. (It's not the kernel, I'm testing a binary that used to work.)

Went to the doctor and got another cat scan (we have four at home, but the one they have is specially trained) which showed that I still have a sinus infection but A) not as bad, B) in a totally different place. I'm callin this progress! Got another 10 days of antibiotics. Feeling better already, although that's probably the placebo effect.

Mark had a series of videos called "Chi comes home" (and a second season called "Chi's new address") which are hard to describe. "Kitty porn" comes pretty close. Kitten porn in this case. Or perhaps "Kawaii diabetic attack". I DARE you to get through even the opening credits with a straight face, let alone a full episode. (There are 104 of those in the first season. That's episode 2. Yes, it's a series of 3 minute cartoon videos about a kitten, and it is horrible yet awesome. I have not yet shown Fade because she might go into shock.)

The first episode isn't quite representative of the series, since it's the kitten wandering away from its mother and hence three minutes of way too cute _angst_ instead of pure uncut refined cuteness like the others. Although if you want angst done inexplictably well, there's a blog telling the story of a homeless family of sims. It's a father and daughter pair of sims set up on an empty lot with no money and nothing but 2 park benches as "home", allowing Sims 3's "free will" option to dictate their behavior as much as it can (everything but aspirations and opportunity pop-ups), and writing up their story with screenshots and prose. It is FAR more emotionally involving than it has any right to be, although you really have to start at the beginning to appreciate it.

There's a weird sort of yin/yang in the above two paragraphs, now that I come to think of it.

Got the perl removal patches resubmitted for 2.6.31. (It's technically still the merge window, let's see if they make it in this time.)

Even though the changes from 2.6.29 were one minor cosmetic tweak and rediffing so the offsets matched up, it took hours to do this because I'd totally forgotten how (and I had to write up new descriptions). My own cannonicalize.sh script seems to have gotten lost sometime in the past 3 months, so I had to write a new one. (It puts a patch in the cannonical format for linux-kernel. Yes, it has my name and email hardwired into it, it's my script.)

Now at Starbucks. Still bisecting the qemu powerpc stuff. Building qemu is slow...

Hey, my daily google alert thingy just pointed me at my Firmware Linux page on the impactlinux website. Um, yay?

So bisect narrowed down 9d479c119b42b8a548f8d79a8e5a1c1ce2932d91 which was the openbios update that made the sucker start booting again, but which was _after_ whichever change swapped hda and hdc. (And presumably whatever caused the powerpc image to start panicing with a null pointer dereference right after userspace got control even when you work around that.)

It's possible the panic is a kernel change and not a qemu change, but lemme try 0.10.1 with the other openbios and make sure I can reproduce the problem here before trying to bisect the kernel build.

Last night my laptop did one of those "everything freezes but the mouse pointer" things so I had to power cycle my laptop and lost the debugging context for powerpc. (Ubuntu 9.04 is like Windows 95 when it comes to reliability. Oh well.)

This morning (up before noon!) I'm rebuilding powerpc with gcc 4.1.2, and if the behavior differs I'll build the various packages with each toolchain and see _which_ package is having a behavior difference. (It's also possible that it stopped working when I reinstalled Linux at the start of the month, which means the qemu it's using now and the one it used to have are very different. This is really to see which of those two ratholes to go down.)

And it's the qemu one. Ok, fire up git... And I have to go --disable-werror now because the Werror disease has spread to qemu. (Different gcc versions produce different warnings. Differently patched distros produce different warnings, such as not checking the return value of asprintf(). Apparently that's a warning now. So is ignoring the return value of chdir("/"). Maybe that's an Ubuntu thing, I dunno, but in general -Werror is _useless_ these days.

Repairing my desktop again while it builds. I don't know why some of the things I add to the xfce taskbar survive reboots, but others don't. This is the... 5th time I've added the CPU governor plugin, I think? (The battery icon doesn't show you current CPU speed, or let you change it, which is one of the main ways to control battery life. You have to go dig up a separate control for that.)

I suppose I don't really mind having a separate little fiddly plugin for everything. What is it about volume controls that Linux desktop developers have such trouble with? The KDE one was good: that had a little speaker in the tool bar that gave you a slider pop-up when you clicked on it, and went away when you clicked on anything else. The Gnome one had a slider pop-up that never went away unless you explicitly clicked on the speaker icon a second time to dismiss it, and displayed above everything else. That was annoying.

The Xfce one doesn't have a pop-up, instead it creates a new window every time you click on it. Did I mention that creating a new window takes about 5 seconds, during which you have no acknowledgement that anything is happening until the new window suddenly appears? Did I mention that you can have a half-dozen of these old volume windows lying around if they fall to the back of the z-order instead of getting closed? Did I mention they take up almost 1/4 of the screen for what is essentially a single control slider? Yeah, I know. Oh well.

I want the xfce general desktop with the kde volume control. Even if I could mix and match them (they don't seem to have a common docking API), the KDE thing would suck in 100 megabytes of shared libraries to twiddle the volume. The "let 1000 flowers bloom" approach only works if there's some way to integrate the result, and after 20 years of X-based GUI development we've got nothing in that department.

Ok, so the first problem is that -hda is showing up as /dev/hdc. That's odd. Is this because I'm using the -git version of qemu? I think I was using the 0.10.1 release before, lemme try building that...

Great, 0.10.1 dies in the bios (I traced this down once; doing so sucked), and in current -git the bios is fine but the emulated hardware has at least two obvious things wrong with it. Maybe I can git bisect a sweet spot where everything worked...

Meanwhile, upgraded the kernel to 2.6.30. The perl removal patches are unchanged except for applying at an offset now. I am officially nervous about this, but can respin and resubmit them Monday morning.

So, smoketest-all.sh is reporting that armv5l, armv6l, i586, i686, mips, mipsel, and x86_64 are passing. This leaves m68k, powerpc, powerpc-440fp, sh4, and sparc reporting failure.

Sparc's been broken forever, and m68k isn't supported by qemu yet. I checked sh4 by hand and the problem is it's so slow the new timeout logic is triggering (which is realy a minor qemu problem, very slow emulator for that target). Run by hand with no timeout, it works. It just takes over a minute to boot up and compile "hello world". (It seems to be some kind of timeout in the emulated hardware, actually. Once hello world is built, it runs essentially instantly.)

The powerpc target seems to be building fine, it just won't run. Or more accurately, the kernel boots but it can't find the hard drive to mount the root filesystem. This sounds familiar. But how did chaning the _compiler_ break this? (Build break due to pointless extra error checking rejecting a construct that used to compile just fine, I can at least understand. But this is a change in _behavior_.)

Hey, the sound just died on my DVD, 17 minutes into playing it. Backing up a bit didn't help, fiddling with the volume control didn't help, killing and relaunching VLC didn't help... Sigh. Time Invoke Knowledge a non-technical end user wouldn't have: dmesg has no clues, check and see if the youtube videos I predownloaded have audio... And the flash plugin rectangles are grey and unresponsive... but the netscape plugin isn't dead. Ok, killall npviewer.bin, relaunch vlc... yup, and the sound is back.

When a single failed program can stop any other program from producing sound (but the volume and mute controls insiste everything is fine), it's not really a "mixer", is it?

Hey, after half an episode of running around Ichigo got stabbed and is near death. Finally. That means he gets to use his limit break, which is to level up. This is why he constantly takes on purples. (And now a dream sequence training montage on the importance of levelling your equipment and not just your character. Yes, I'm up to disk 10.)

Beautiful, now most of an hour later the VIDEO died. (Frozen and not repainting, when I drag the window off the edge of the screen and back the obscured portion comes back solid black) But the sound is continuing fine! I'm actually giggling from the MST3K-style horribleness of this crap. I mean honestly, it's staging its failures to cycle through every aspect of its functionality. Luckily a simple restart of VLC fixed that one...

Slept horribly last night. My sinus infection is making me tired all the time (it's like swelling in my head is cutting off circulation or something), but when I lie down it hurts so much it wakes me up. I've been through a half-dozen courses of antibiotics on this thing already over the past couple years, but it looks like I need to go back and be insistent about it because this is ridiculous.

Right before I left home, I fought with my laptop for 20 minutes to get it to suspend so I could head out. It kept timing out the freezer after 20 seconds and booting back up to the desktop, first complaining that the nspluginviewer.bin wouldn't freeze. (Well, actually some white text flashed by on the display to quickly to read it, and then gave me the desktop back with no explanation. As with any user interface issue on Linux you have to know to fire up a shell prompt and run dmesg.)

So I figured out what dmesg was trying to say (the multi-page stack dumps didn't actually help here, they were just a distraction from the list of processes it couldn't suspend at the end) and killed all instances of nspluginviewer. Next it complaining cifsd wouldn't freeze, so I tracked down a dead samba mount that hadn't prevented it from freezing when I was at Mark's place earlier today, but was suddenly a problem now. (At a guess, the samba thread only blocks uninterruptibly waiting for a response from the server when it doesn't hear back from it, so when I was behind Mark's firewall it happily suspended because the server said it was ok, but when I wasn't it wouldn't. I gloss over 5 minutes of trying to get it to umount when one of my open terminal tabs had a shell prompt open with the current directory under that mount point. Well, actually the terminal tab had a root prompt at /, but if you exited _that_ the parent process was a shell prompt with its current directory under the /mnt in question. Linux: smell the usability.)

I'm gradually wandering back into BusyBox development, although this currently consists of posting on the list a lot and getting into arguments with the other developers. (Yeah, par for the course I know.)

Some prominent TODO items I have, off the top of my head:

• Busybox:
• Check toybox commands, check what can be ported to busybox. (Patch definitely. Netcat?)
• Test suite: Check toybox tests, see what an be merged. Covert rest of busybox tests from subdir to command.tests format.
• Update/break up toybox structural documentation, post each piece to busybox with rationale. Convince people it's a good idea, then do huge amounts of coverting work to move all the busybox applets over. (Figure out how to do this in stages; it's a huge amount of work.)
• Port busybox to the iPhone.
• FWL
• Get all architectures working again. (Yes, that includes sparc. And see if the m68k stuff that's gotten posted to the list actuallly works.)
• Get nightly builds happening and uploading for uClibc, busybox, and kernel tests. Figure out where to upload stuff. (To impactlinux.com of course, but should I also upload toolchains to my dir on kernel.org? System images with native toolchains on uClibc.org? Static busybox binaries for each target to busybox.org? How often, weekly? Run test suites?)
• Cut a release.
• Kernel
• Perl removal patches.
• Miniconfig.
• Hello world kernel.
• qemu
• Test m68k.
• QEMU weekly news.
• Qemu C Commpiler (qcc)
• Dig up the C++ paper and do another pass.
• Figure out how my desktop's once again grown 4 browser windows with a couple dozen tabs each and close 'em. (I'm slowly growing to tolerate Firefox. Yes it hangs for 20 seconds at random intervals, refusing to even repaint its windows until it's done. But the new version of kmail is doing that now too. #ineedtobuyamac)
• Watch the two netflix DVDs I have with me, and/or the accumulated podcasts, and/or the videos I downloaded from Mark.
• Learn LUA.
• Update the rss feed creation script to parse the "span" tags I've been putting in this blog for the past year or so, and create rss feeds with _just_ that content for each tag.
• Read "Kitty and the Midnight Hour" (bought 2 more books in this series at Hypericon and I'm starting it over from the beginning, so I threw that book in my backpack too).

There are 8 zillion other things on the todo list (biking here was "get exercise"), but those are the "might feel like working on them if I sit down to do something now.

So, attacking the FWL broken architectures. x86, x86_64, and mips (but not mipsel) are working. Let's see what's wrong with armv5l, that's usually a fairly reliable one:

  CC      networking/dnsd.o
networking/dnsd.c: In function 'process_packet':
networking/dnsd.c:428: error: lvalue required as unary '&' operand
networking/dnsd.c:430: error: lvalue required as unary '&' operand

Oddly enough, I'm pretty sure that's _not_ the same error that happened on securitybreach (the 8-way gentoo server), but let's get it working here first before debugging it there.

Step 1, try to build busybox from the command line with the cross compiler. So:

cd ~/busybox/git
mkdir ../temp
git archive master | tar xvC ../temp
make allyesconfig KCONFIG_ALLCONFIG=~/firmware/firmware/sources/trimconfig-busybox
PATH=~/firmware/firmware/build/cross-compiler-armv5l/bin:$PATH make CROSS_COMPILE=armv5l-

And it... built to the end just fine. Ok, what I built was more or less current -git, rather than the release version. (I only have one outstanding patch, for pgrep, and that's almost certainly not relevant here.) So maybe they fixed it, and maybe this just isn't reproducing it. (I note that git tag -l doesn't even show a 1_14_0 tag, the newest is 1_13_4, but I think that's an artifact of the svn->git conversion. I should ask on the list.) So, try again with the busybox 1.14.0 tarball the build was using, and...

Yes, it reproduces the error, and the git version of busybox doesn't. Do a diff between the git networking/dnsd.c and the one in 1.14.0 and... they're identical. So something in some header changed. The error is happening on a call to move_to_unaligned32(), and that's defined in include/platform.h so let's see what the diff is between those two...

And the file is _full_ of unnecessary whitespace changes. BRILLIANT. Ok, try again with diff -b...

Well, that explains why x86 and x86_64 work:

#if defined(i386) || defined(__x86_64__)

Now I'm wondering why mips worked. (Presumably the guts of gcc 4.2.1 didn't complain about the lvalue thing on mips.) Which brings up the question of why mipsel _didn't_ work... yup, it's complaining about the same lvalue thing.

Ok, all architectures brought down by the same busybox bug, which seems to be fixed in source control. It's git 3be23086 in the busybox repository thing, applied on April 17th. (I note that "git blame" is kind of useless if something else, like a pointless whitespace change, has touched the file since. With mercurial you can subtract one from the commit number you want to see before and do an "hg annotate -r commit filename". But you can't do previous/next math on git commit hashes. Smell the usability.

So yesterday I got x86_64 building all the canadian stuff with gcc 4.2.1, and checked it in. Then I told the 8-way server to build all the targets and went to bed, and today I find that half the architectures went boing.

Sigh. I _really_ need to get linux-2.6.30 building today so I can update whatever may have changed in the perl removal patches and submit that upstream, but this is being stroppy. Armv5l died with this during the cross-static build of uClibc:

libc/stdio/_READ.c: In function '__stdio_READ':
libc/stdio/_READ.c:39: error: 'LONG_MAX' undeclared (first use in this function)
libc/stdio/_READ.c:39: error: (Each undeclared identifier is reported only once
libc/stdio/_READ.c:39: error: for each function it appears in.)
make: *** [libc/stdio/_READ.os] Error 1

But usr/include/limits has it? So what exactly is it complaining about?

But first, I need to get smoketest.sh to kill its child processes. I want a "kill --recurseive 12345" that takes out all the child processes and their children and so on, but apparently I've hit yet another thing that the standard unix tools never bothered to implement.

I am so _sick_ of fighting the shell. The sparc target hangs when you try to test it, so testall.sh gets hung up on that one. I can run a background process that does "kill $(jobs -p)" after a timeout, or does a "pkill -P $$", I can set a trap "pkill -P $(jobs -p)" EXIT, and so on. But so far, nothing but "killall qemu-system-sparc" actually kills the qemu instance.

I'm running a script that's running a script that's calling qemu, and the child processes don't get killed when the parent process goes away. After the qemu instance hangs it doesn't get killed when its parent shell script gets killed by a signal, and the traps I'm putting in won't propogate down to it. (The buggy error messages don't help: "pkill -P pid" complains "no signal P" when the pid in question has already exited, which is _not_ helpful...

Such crap. Such total crap. This is not the bug I'm trying to fix, I keep hitting OTHER bugs that won't _let_ me get to the bug I'm trying to fix. And then I hit othe bugs when trying to debug that.

Linux is buggier now than it was under Red Hat 9. Why am I bothering?

I dug up my old copy of neverwinter nights, but character generation was such a slideshow (mouse position updating about once a second, which is kind of hard to use with a touchpad) I gave up. It worked fine on my previous laptop, which also ran Linux. (This game is so old that it was back when people bothered to put out Linux versions of games. World of Warcraft even had a Linux client during its beta. Back when people thought Linux on the desktop might actually matter someday.)

Alas, without 3D working, even with the faster processor, it's just unplayable.

So back to watching "bleach" dvds and banging on FWL, at The Donald's (where the apple pies seem to have fixed themselves, dunno if it was just a bad batch, if I was consistently getting them dehydrated, or if corporate changed its mind).

The prerequisite system-image.sh is checking for is the existence of the root-filesystem-$ARCH directory, and that's not a particularly useful test. Sure if the cross compiler didn't build, that shouldn't exist, but if the root filesystem build died halfway through...

The problem is, what specifically can I test for in the root filesystem directory that says it completed successfully? I _want_ to be able to pack up fairly arbitrary contents, so setting a flag file at the end of the build is exactly the kind of dependency between stages I _don't_ want them to have.

The root-filesystem.sh script has a natural dependeency: it can't build without the appropriate cross compiler. It doesn't care where that compiler came from, just that it's in the $PATH. The system images don't. I could add a flag to indicate that it _failed_, but that's a bit non-obvious for comfort. (When it's not asynchronous, return codes work fine, which is why this hasn't come up before.)

I'm torn between trying to properly fix this (although I have no idea how) or just hacking temporary tests into the scripts (yet again) to abort at the point I want to see the output. I guess it depends on what I'm trying to fix next. Eh, bop it on the head with a brick for now, get back to beating the behavior I want out of my old enemy, gcc.

So gcc/config.cache needs to contain "gcc_cv_as_hidden=yes" and "gcc_cv_ld_hidden=yes". I could just prepopulate the cache with this, but _why_ is the test failing? (It doesn't seem to log that part.) Hmmm...

Ah. It decided the assembler name it's using is "". That's just BRILLIANT. Right, explicitly set AS_FOR_TARGET, which for some reason 4.1.2 didn't need but 4.2.1 does. You'd think there would be a way to just specify the darn prefix for all these tools, but no...

I dug up my old copy of neverwinter nights, but character generation was such a slideshow (mouse position updating about once a second, which is kind of hard to use with a touchpad) I gave up. It worked fine on my previous laptop.

Banging away on FWL, moving everything over to gcc 4.2.1 (last GPLv2 release). Been needing to do that forever, yes it breaks stuff, trying to fix it. Need to refactor the hell out of the stage intro code to make debugging buildall.sh easier, though...

When did all this code become so _complicated_? It's just a darn build script. Where can I simplify...

Breakfast at Einstein's, then decided to bike south to the Chick-fil-a location near SJG to try to work off some of the calories from the con this weekend. (Making lots of peanut butter and jelly sandwiches, cutting them into quarters, and filling aluminum trays with them? Marvelous idea. A german restaurant with a live polka band indoors? Not so much.) Going down The Drag I got hit by a car that turned right out of the Schlotzky's parking lot without looking, but luckily I'd seen the "how the stuntmen do it" things (my behind the scenes at hollywood hobby came in useful) and rolled up on the hood like you're supposed to, and my laptop landed on top of me. The bike, alas, wasn't so lucky, and the front tire was all wobbyly afterwards, and I had to spend $60 getting it replaced. Didn't actually make it out towards Chick-fil-a until around 5:30 pm.

Reason #9437 why Linux on the Desktop sucks rocks: every couple hours, the wireless network will suddenly stop working, with no warning. Generally, it means this happened:

[15163.962304] iwl3945: Microcode SW error detected.  Restarting 0x82000008.
[15163.962326] iwl3945: Error Reply type 0x00000005 cmd REPLY_TX (0x1C) seq 0x0000 ser 0x0000004B
[15163.964337] iwl3945: Can't stop Rx DMA.

And under Ubuntu 9.04 _that_ means you're going to reboot soon. In previous Ubuntu versions, you could "modprobe -r iwl3945" and then insert the module back in, and you'd get your wireless back. But in 9.04, if you try it your kernel will panic. New and improved!

This is why I've given up on Linux on the desktop. Each new release breaks things that used to work, and this has been true for TWENTY YEARS now. On a personal level, why should I "get out and push" if the people steering wind up in a ditch or going off a cliff every single time?

Open source does not competently handle user interface issues because "shut up and show me the code" is not a useful response to aesthetic issues or usability problems. It's an inherent structural problem withour development model. Suppose a user notices something obviously STUPID, like the fact that the default action on battery drain is "shut down" and quickly suspending when you get the "system is about to shut down" pop-up just means that when you plug it into AC power and resume, you get to watch it shut down then. They complain about it. We filter out that kind of complaint as noise, and tell them "you fix it". Become an expert in our little area, then submit your fix to us and spend months iteratively convincing us to take it. Or else leave us alone you useless freeloader.

These problems are built-in to our development model. Coding disputes can be handled via objective tests, benchmarks, code size metrics, and so on, which can conclusively end arguments with numbers and proofs and logic. Dealing with more abstract issues, we fork ourselves to death with a thousand competing implementations, each of which does a couple things better than anyone else, meaning it does everything else worse.

This feeds into the way "this is a horrible default that bit me" is met with "here's how you can customize your personal copy, to fix it _for you_". No attempt at a general solution is made, because maybe somebody WANTS that behavior even if most people get bit by it. Creating a usable environment requires huge customization effort, but isn't it great how customizable our system is? You can build your own desktop. You must do so, since the default environment sucks so badly you really have no alternative. But there are so many different ones to choose from, if you just spend three years studying your available options you'll eventually find that special one built just for you. Don't like Totem? Try VLC! Don't like Pidgin? Try Jabber! Don't like Thunderbird? We've got _hundreds_ of alternatives...

So it's easy to _leave_, but hard to actually get anything _fixed_. Users who can't or won't back up their reports with scientific metrics are ignored. Or those who don't regularly perform regression testing, or who aren't going to spend six months following up on an issue, or who can't swap out individual components of their system with ones they built from source (after having upgraded everything to the latest version because fixes aren't made to the old versions; you need to be exposed to all the new bugs at once to test any fixes for your old bug)...

And even if you _do_ try to fix these things, you can't keep up. Pointing /bin/sh at dash was _new_ breakage that older Ubuntu didn't have. KDE 3.x->4.x was new breakage. Losing syntax highlighting in vi and the ability to cursor around in insert mode was new breakage. Kmail getting eaten by Kontact and then its user interface changing so hugely in the most recent release I spent 2 hours finding the "sort by thread" option: new breakage. The same network hardware this laptop has had for 3 years now breaking in new and exciting ways: new breakage. The fact 3D support no longer works for my intel graphics card (it worked fine in 8.10): new breakage. (Although a 64-bit dual core 1.73 ghz processor can just about keep up with software rendering to display video, as long as the system's otherwise idle and the window size is small enough. Even then youtube stuff flickers like mad with the "hidden" under-layer of youtube navigation buttons showing through intermittently due to the compositing happening in a single live buffer, and it tends to go all slideshow at the drop of a hat anyway. Oh, and swapping windows it sometimes leaves uninitialized memory garbage on the screen for several seconds, and sometimes the event that would redraw over it _times_out_ and it's left there in the window's buffer. Even if I drag another window over it, the garbage comes back because the redraw is from a double buffer that's got uninitialized memory in it. Kmail does this all the time with its toolbar buttons.)

There's no "steady improvement". There's no indication of clear agreed-upon _goals_, just random lateral thrashing. I keep having to either abandon existing infrastructure that stops being supported, or personally go to great lengths to keep the older stuff working in newer contexts.

Because of what I do, I mostly notice this with technical things that end users only ever notice as "black magic that failed". On a technical level it's epidemic, such as the way the IDE devices became part of the SCSI layer so device ordering is now fundamentally horked in a way it didn't used to be. (SCSI throws all devices in a big bucket and gives 'em a stir, so your USB devices and your SATA devices draw from the same pool of device names, in first come first serve order. Yes really. Back under the IDE world, /dev/hdc wouldn't move without a screwdriver.) Or the emergence of unnecessary layers like the whole HAL/dbus thing, so that your web server goes "HAL says I'm not connected to the network, so I can't let you talk to that web server you're got running on the loopback interface". (Why does it CARE? The TCP/IP layer has had a "no route to host" error for 30 years now, so if there's no network this fact should be instantly apparent when you try to use it. Adding a separate layer capable of vetoing network access is damn stupid, and whether it actually works or not is a side issue to it having no reason to _exist_ in the first place.) And don't get me started on how hideously unstable and undocumented the sysfs exports have been over the past 20 kernel releases. So on a purely technical level, the people steering Linux haven't filled me with confidence in the soundness of their judgement.

And of course there's social/process things too. The GPLv2 vs GPLv3 split didn't help anybody do anything: we're still paralyzed by infighting against our zealot arm which undermines everything the rest of us try to do, in the name of "Purity". (Similarly, I stopped cooperating with the SFLC after the FSF hijacked them.)

On the social front, the people who insist that openness is inevitable and in future all scarcity will vanish have apparently never studied history. Capitalists try to corner the market on things that were previously free _all_the_time_. Capitalism is a marvelous mechanism for managing scarcity, and in the absence of scarcity to manage, capitalism creates it artificially so it has something to manage. For example, Austin is now ringed with privately managed toll roads. I hate this, but they're still there, and they're building more. Individual instances come and it go as people play whack-a-mole with the latest incarnation, but there isn't going to be some magic day when people stop TRYING to corner the market and make a killing.

But technical and social issues aside, an awful lot of the time I notice that my Linux desktop simply doesn't work at an end-user level, and then have to invoke programmer-fu to try to work around it.

My desktop is Linux. My desktop doesn't work. Most of the time I just want to USE the thing to surf the web, watch videos, read email, edit text, and it breaks repeatedly and I have to duct tape it back together to get minimal functionality out of it. (I STILL haven't gotten the bluetooth association between my laptop and cell phone set back up. On Fade's Mac, it took about five minutes starting fromm scratch, most of which was figuring out where in the config menus to find the appropriate settings page.)

These little "end user issues" come from everywhere in the infrastructure. The network spontaneously dying so only a reboot can fix it is a kernel issue. The 3D support not working anymore is an X11 issue. When I press the eject button my my DVD, it gives me a pop-up message 'Failed to eject "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV". Given device "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV" is not a volume or drive." and that's HAL/dbus breakage. All of those are brand new failures in 9.04, on the exact same hardware that originally came running Ubuntu 6.10 from the factory. (It has more ram and a bigger hard drive since then, but that's it.) Installing a new version of ubuntu breaks stuff that used to work, every time.

The largest chunk of infrastructure I've had to abandon recently is KDE. I'm now on Xubuntu because the KDE developers did one of those "throw out all the progress we've made and start over" things that resulted in a KDE 4.x so hideously unusable that even Linus Torvalds (who has mocked the gnome developers for years as Just Not Getting It) has abandoned KDE and gone to Gnome. (In Kubuntu 8.10, KDE 3.x was no longer an option, and 4.x simply did not work.) I tried Gnome for a bit (couldn't stand it, and then the system's network configuration spontaneously ate itself so badly I had to reinstall to talk to the network again, and I went back to 8.04 for a while), and now I'm using XFCE instead.

This is just within Ubuntu, the only distro that's still _trying_ for the end-user desktop. I used to use Red Hat, but they abandoned the desktop and went off to the enterprise market. (When the call went out for a company to stand up and defend decss so Linux could play DVDs, they yanked MP3 playback support from Red Hat 8 and bogged off to the enterprise workstation market, discontinuing their consumer version entirely one version later.) I didn't so much abandon Red Hat as get abandoned by it. Then I used Knoppix, but while its "boot from CD" gimmic made it a great platform to demo Linux on various hardware it never actually transitioned to running from the actual hard drive at all well. So I abandoned Knoppix in favor of Ubuntu, then Kubuntu, then Xubuntu, which are all more or less from the same group. There are other distros I could use if I tire of Cannonical's offerings, but that's just another manifestation of "all of these suck, so we keep trying new ones".

This means I'm now using the third place desktop (gnome, kde, xfce) on the third place operating system (windows, macosx, linux). My configuration's market share doesn't even register, and NEVER WILL. I'd say XFCE was even in _fourth_ place behind the desktop the subnotebook distros (like Easy Peasy) are using for eeepc clones, except that we lost that niche (as it scaled up past rounding error, all the new seats it added were Windows) so it's comparing one flyspeck with another and the result just doesn't matter either way.

Since I've never been able to recommend Gnome with a straight face, and can no longer recommend current KDE, I'm now back in the OS/2 position where the desktop I use isn't one I can honestly recommend to anyone else. I use a weird custom system nobody else has ever heard of, and I may never actually meet another user of this particular setup in person. I might as well still be using an Amiga.

Embedded Linux is still moderately fun, but I used to do Java under OS/2 the same way. What the code you produce runs on, and the development environment you produce it in, are two different things. Linux itself will be around as long as Java (which is the new Cobol), but the Z80 processor lasted 30 years and DOS could do just about everything embedded Linux does. Cisco still sells IOS, and Solaris and even Xenix still have their users. In a way, the embedded niche is where old techologies go to die. (And every supercomputer worth the name is a hand-rolled bespoke job.) Maybe the openmoko Google phone will stop being "technology in search of a use case" and start to matter as much as the iPhone or blackberry one of these days. I'm not holding my breath, the cell phone companies have no incentive to give up control when they can sell _ring_tones_ for profit and people buy them.

It's been "the year of the Linux desktop" every single year since 1998. We are the boy who cried wolf, and even if it did start to get interesting in future nobody will ever care, or pay any attention.

This is why I need to buy a mac laptop to do my embedded Linux development on.

Poking at the FWL buildall.sh script. It's broken in several different ways, but the most fundamental one is that its debug output is horrible.

The main problem is that buildall.sh does things that build.sh doesn't, because doing a canadian cross requires multiple architectures, and build.sh builds one architecture through from start to finish. This means if I want to test any of the stuff that only buildall.sh builds, I need to test it in a different build environment that produces horrible debug output.

The horrible debug output is because it has to be able to background a lot of processes (to support FORK=1), and doing anything asynchronously makes logging and flow control fundamentally harder. Also, buildall shouldn't stop the whole build script because one architecture fails, and you have to go out of your way to make it build just _one_ architecture.

There are several ways I could address this. I could teach build.sh to handle the special case of building this extra stuff for the host architecture, which is the one instance where you don't need two architectures because the two match. I could add the option to buildall.sh to die after the first error. But ideally, I want buildall.sh to stop blanking the temp directory for later stages after earlier stages fail.

I taught the individual stages to exit when they haven't got their prerequisites, and even added an option to suppress that error. But they still blank the temp directory from include.sh, before I can make specific prerequisite tests. Moving the cleanup out of common code to solve the flow control issue would require me to duplicate it in every stage. It sucks either way, really.

The reason all all this is important is that the canadian cross stuff (or an equivalent set of prerequisites) is required to do an --enable-shared compiler, which is required to build uClibc++. (It's an unpleasant set of circular dependencies, based on the fact that libgcc_s.so is a completely horrible idea. The compiler should not add shared library prerequisites to its output, especially not for a library that links against libc.so. It wouldn't have been so bad if --disable-shared built libgcc_eh.a, but that would have been _smart_ and we're talking the gcc developers here.)

It also turns out that the reason I've been having such trouble with soft-float on arm is that they decided to add the soft float stuff to libgcc_eh.a, so it only gets built for --enable-shared. (I.E. the --disable-shared compilers have been gradually crippled by the gcc developers ever since the start of the 4.x series. Wheee...)

At a convention, and what do I do? Program on my laptop, of course.

Figured out why gcc 4.2.1 was going nuts (if you pair gcc-core in one version with gcc-g++ in another version, it gets unhappy). Now building everything with that, so I can beat soft-float out for armv4.

The mercurial git->hg converter is still broken (at least with the busybox repository). Need to bug the list about it...

And once again, buildall.sh is going "boing". Sigh. It's sad how brittle binutils and gcc are, what I'm trying to do is fairly straightforward and I have to supply a dozen separate overrides of its assumptions to do it.

Random bug in VLC: if you enable deinterlace, it kills the old window and pops up a new window. The new window has no subtitles in it, you have to disable subtitles and then re-enable them to get them back.

I'm used to Linux desktop software being buggy. This is normal.

Broke down and set kmail back up, and caught up with the accumulated week of back email. (Mostly spam, of course.)

And kmail has lost the ability to thread messages in 9.04? There used to be a folder->thread messages option in the pulldown menus, and now it's gone. And I can't find any replacement. That's just brilliant.

Reinstalling various things. I have no idea why the "hg convert" extension spontaneously disabled itself. I had the "hgext.convert=" line in my .hgrc, and it got commented out somehow. Apparently, installing mercurial from a dpkg goes through your existing .hgrc and comments out extensions you have enabled. That's deeply stupid; if I have an .hgrc possibly the data in it _means_ something, eh? Once again, Linux developers are _sure_ they know what I want to do better than I do. I need a mac. (They're probably as arogant, but far more likely to actually have good UI suggestions.)

Looking at the busybox "patch" command to see what would be involved in replacing it with the toybox one. (Either as a patch I maintain locally, or something I can push upstream to the busybox guys.) Their version is 254 lines, and mine is 361 lines. On the other hand, mine does a heck of a lot of stuff theirs doesn't (starting with supporting offsets, which is why I need it).

Ideally I'd make it config down to be as small as the old one, but it wasn't really designed that way. Hmmm...

Another thing is that after using the toybox infrastructure for a while, the busybox infrastructure looks HILARIOUSLY clunky. Put a prototype for the applet's main function on the line above the declaration of that main function. (Why?) And of course pass in argc and annotate the fact it isn't used. (Um... Ouch?)

So I found a new way to break toybox, and I don't want to put out yet another single patch bugfix dot release. If I don't want to bother with it anymore, then I need to push stuff upstream into busybox.

I pretty much just need patch, oneit, and netcat. I need my version of patch, because the busybox one can't apply anything at an offset. I need my version of netcat because the one they've got doesn't seem to have working -l support. And I boot the resulting system under oneit, which I suppose could be standalone.

The big part is that if I start working on busybox again, I'm going to have a huge urge to clean stuff up...

I tried to build qemu from source, but the version of the source I had died with a build break halfway through. (I have horrible luck with random source control snapshots, part of the "I break everything" schtick.) So I thought "ok, git update time"... except that it hung. Why did it hang? Because savannah.nongnu.org is down. That was yesterday. It's still down.

Luckily, the mailing list archive is still up (yeah, my personal email still isn't reinstalled yet. I need to break down and go back to kmail...) and there's good news. Now that the project has a new maintainer, not only did they migrate off cvs but they may finally leave savannah as well. Yay!

Watching Obama's weekly address. (He's really going to the well for his supreme court nominee.) What I'm really noticing the little flag pin on his suit and tie, and can't help but wonder if he's wearing bright red tennis shoes along with it to complete the ensemble. (Or is that just british?)

Attempting to listen to a podcast where Seannan McGuire (the lady whose livejournal I started following after she wrote a marvelous Barry Ween fanfic) is interviewing Jenn Brozek (the ex next door neighbor of Andy Weir, who wound up in his now completed webcomic as an international jewel thief with a time traveling daughter. Both Jenn and Seanan are also published science fiction authors. Jenn's an editor. Seanan's a filker with several albums out, and draws an intermittent webcomic. Interesting people, generally.

The two of them wound up at baycon together, and BayCon Podcast #4 is Seanan interviewing Jenn. It's a small internet, innit?

On an unrelated note, an interesting thing about VLC is its starting size is the actual width and height of the source video, in pixels, with 1:1 scaling. (The other players I've used just remembered the previous size relative to the rest of the desktop.) This makes it clear that the _reason_ Obama's podcasts look way the heck better than the msnbc ones is the white house podcasts take up the whole desktop, and the msnbc ones are a tiny little window in the corner. (DVDs take up maybe about half the screen.)

Mark poked me last night about several FWL todo items, so I guess I should tackle them...

Let's see, I got the nightly snapshots building with uClibc-git, I need to test those to make sure they're all working, then upload 'em to the binaries directory on uclibc.org. Once I've got the those working, I need to figure out which ones I can switch to nptl, probably adding new targets for those (ala armv5l-nptl).

And I should really deal with sparc. And I should switch over to gcc 4.2.1 as the stable version and make it always build libgcc_eh.a even when it's --disable-shared (which would be a _better_ way to fix the arm soft float issues, and also fix...

Ok, THAT was annoying. The display froze. The mouse still moved, and the audio of the video I was watching continued to play, but the video itself was frozen and nothing on the desktop responded to the mouse pointer or being clicked on, the keyboard was dead (ignoring ctrl-alt-F1 and ctrl-alt-backspace), and the power button didn't suspend. Luckily, holding the power button down for 10 seconds powered the sucker off, since that's not under Linux's control. And thus I was able to reboot.

That was crazy, the kernel was obviously still working because the audio playback would have stopped if it wasn't (and the mouse would have stopped moving), but all user I/O was frozen. Why on earth did this get redesigned? What idiot decided to route the power button (acpi event) and the desktop (X11) through a single point of failure? (Sounds like the darn "input core".)

Great, now my laptop can choose to just hang on me at any time. "Xubuntu 9.04: there are no nontechnical end users I hate enough to inflict it upon."

I need to buy a mac.

One of the main failures of Xubuntu, or possibly Desktop Linux in general, is that you can't easily pick and choose which apps you want to run. Yes, that "failures" and "can't" rather than "features" and "can".

Xubuntu comes with "movie player", which is actually a program called "totem" although it never anywhere tells you that. (You have to use ps to see the name of the running process to figure that out, and this is intentional. That's a bit like saying "web browser" and never revealing whether you're running IE, Firefox, Opera. Did I mention the Gnome developers consistently make horrible user interface decisions and then defend them to the death? Why the xfce guys are humoring the Gnome guys' poor decision making skills is an open question...)

If you press the eject button on the dvd drive, it pops up a window saying "Failed to eject "/org/freedesktop/Hal/devices/storage_model_CDRWDVD_DW224EV", and tells me that name is not a volume or drive. I have to go to a console and run "eject /dev/sr0" as root to get the disk back. This is unrelated breakage, it's because the "hardware abstraction layer" was a horrible, horrible idea Linux copied from Windows NT (see Gnome == stupid, above). But I can't tell modern software _not_ to use it.

Totem is an inferior dvd player to Kaffeine, the one Kubuntu used back in 8.04. (So the rest of what I'm talking about is a regression vs what they got working back in KDE 3, which was rendered useless when they moved to the unusable KDE 4.) Totem auto-launched when I inserted a DVD (not even popping up a "what do you want to do" menu), and then told me it couldn't access the dvd player. Turned out to be just a race condition, and on relaunch it accessed it fine. (Kaffeine did not have this race condition.)

Then it turns out that it doesn't seem to deinterlace the dvd by default. (Note I already went through the magic dance to get decss support working, which is non-obvious and something the mythical nontechnical Linux desktop users would have to phone a friend to get to work.) Google implies that back under 7.04 you could find a deinterlace option under the "view" menu, but there isn't one there under xubuntu 9.04. That menu has "fullscreen", "fit window to movie", "zoom in", "zoom reset", "zoom out", "aspect ratio", "switch angles", "show controls" (the play button and such at the bottom, no deinterlace option among them), "select text subtitles", subtitles (a sub-menu, not only redundant but it isn't aware of the actual subtitles this bleach disk has, which I navigated the DVD menu to switch on), and "sidebar" (which is read-only, showing things like the duration, resolution, and frame rate). Kaffeine puts the duration as a label ont he slider at the bottom.

Kaffeine not only had "interlace" actually as a labeled option in the menus, but when I hit "i" it said it was toggling interlace. I _think_ I enabled some kind of deinterlacing by pressing all the letter keys in sequence (accidentally fullscreening and quitting and such along the way), but A) it never notified me I was doing so, so who knows what key it was, B) the deinterlacing algorithm it's using is crap. It's better than nothing, but still full of artifacts.

Of course the problem with installing Kaffeine is it's bolted to the KDE desktop, using the QT widget set and kobjects and so on. We have so many redundant implementations of exactly the same infrastructure, and they're all slightly incompatible so can't be substituted for each other. It's like windows "dll hell", only built into the OS.

So I tried installing "vlc", another dvd player that google recommended, which presumably doesn't have these unnecessary dependencies on pieces of infrastructure it shouldn't care about. It installed fine. It seems to play the dvd. But when I put in the dvd, totem still pops up. I uninstalled the package "totem", which made no difference. There's 8 gazillion other sub-packages (totem-common) and such, several of which imply it's tied into the guts of mozilla. (This seems odd since mozilla's using the binary-only flash plugin to display essentially all video, because nothing else really works.) Ok, remove totem-common, let it randomly rip out three other packages to "resolve dependencies"...

Ok, now it does nothing when I insert a disk, and I can pull up vlc by hand from the menu to play the disk. I can live with this. (This has no menus, the whole window is displaying the video. You right click on it to get options, and video->deinterlace has seven options under it. It defaults to "disabled", yet even that seems to be deinterlacing the picture slightly better than totem was. I set it to "blend", although what I really want is "just deinterlace properly, don't bother me with implementation details since I'm not a DVD playback codec designer". (This is a really common failing of Linux software, you have to know how it's implemented in order to make it work.)

Except that with both kaffeine and totem I could use the cursor forward and back keys to skip ahead or go back in the disk, and also get slider bar along the bottom. With vlc, I can jump to the start of episodes, and that's it... Ah, I see. There's a progress bar on a separate window. I guess that's reasonable...

I still need to buy a mac, so I don't have this much upgrade hassle every 6 months. I have to make the "to kmail or not to kmail" decision again, and until then I don't have an email client...

How do the ubuntu developers get stuff this simple wrong?

When I fire up totem and tell it to play a podcast, it says it hasn't got the plugin and offers to search for it. It comes up with a window that says "GStreamer ffmpeg video plugin" and "GStreamer plugins for mms, wavpack, quicktime, musepack". Ok, fine. There's the pointless step of clicking checkboxes to select them, which pops up a dialog about legal disclaimers (clicking a checkbox should not pop up a dialog, that's horrible UI design), and then I tell it install, it grinds for a bit, and then says "Software installation failed". It won't tell me what the problem was, and there's no way to get extra information. The three buttons are "retry", "add/remove more software", and "close".

Oh, and did I mention it won't tell me the package _names_, just the description fields? If I want to do this from the command line (and thus have at least a _chance_ of better error messages), I have to figure out that "GStreamer ffmpeg video plugin" is _probably_ the "gstreamer0.10-ffmpeg" package. I still have no idea what the _second_ package is.

You could write a "user friendliness fail howto" out of this. Automation that completely hides how the underlying thing works is not useful if it means your car runs out of gas and it refuses to say so (or even that gas is a thing it might ever need).

May 27, 2009

Computer tied up most of yesterday backing stuff up, and today restoring it. Being able to do 10 megabytes/second through a USB cable isn't that impressive when you're backing up and restoring 300 gigabytes to a terabyte drive.

May 26, 2009

May 24, 2009

May 23, 2009

A little while ago I promised to explain:

So if open source used to be the norm back in the 1960's and 70's, how did this _change_? Where did proprietary software come from, and when, and how? How did Richard Stallman's little utopia at the MIT AI lab crumble and force him out into the wilderness to try to rebuild it?

Two things changed in the early 80's: the exponentially growing installed base of microcomputer hardware reached critical mass around 1980, and a legal decision altered copyright law to cover binaries in 1983.

Increasing volume: The microprocessor creates millions of identical computers

The shortage of customers was changed by the microcomputer, which was based on the microprocessor. Intel invented the microprocessor around 1971 (largely by accident; here are interviews with the engineer who designed it, the one who implemented, the customer they did it for, and their boss at the time who is most widely known for having a law named after him), but it took Intel three revisions (4004, 8008, 8080) to come up with a usable chip, which was at the heart of the MITS Altair, the first microcomputer (and the original home of the S/100 bus and CP/M operating system so widely cloned later, including somewhat indirectly by the IBM PC).

The first computer to sell 1 million units was Comodore's Vic 20, introduced in 1981 and beating the Apple II to the million unit mark by a few months. So within five years of the Apple II's introduction, you could potentially sell a million copies of the same software. This allowed the concept of "shrinkwrapped" software to emerge, which you could write once and then sell many identical copies of without even recompiling. Initially this supported small companies consisting of a programmer and perhaps a small support staff (WordStar, Richard Gariott's "Alkabeth"), but exponential growth in the hardware market led to matching growth in the potential software market.

The first "killer app" (a software program compelling enough to drive sales of the system it runs on) was Visicalc, the first spreadsheet. It was produced by a two person company, Dan Bricklin and Bob Frankston, operating out of Bob's attic. The first version shipped in October 1979, and was the main reason that Apple's less than 50k units its first two years were followed by a million more. (Another fun quote in that article, by Bob Frankston: "it's worth noting that back in 1979 people viewed the keyboard as an impediment to using computers. After all, only secretaries could type...") Oddly enough the book "On the Edge", which is about commodore, gives great behind the scenes information on early Apple and Visicalc stuff. The fact commodore bought the company that made the processor inside early Apple machines has a little to do with it, but mostly it was a darn small industry at the time and everybody knew everybody else...

So the reason something like Visicalc couldn't happen before 1980 is there was no customer base to sell shrinkwrapped software to, not even enough to support individual programmers operating out of their bedrooms unless they had a day job doing something else. You could make a living as a consultant writing bespoke software on commission, but your customers told _you_ what to write, and owned the result which of course included the source code.

The other thing about microprocessors is they were more uniform. The old "wire up a breadboard" computers weren't just produced in lower volume, they were more easily modified at a fundamental level.

The MIT AI lab where Richard Stallman learned to program was started in the 1950's with a pair of prototype computers, built by a recent graduate named Ken Olsen when he interned at Lincon Labs in Boston. Both were unique, each the only one of their kind ever built. The first was the MTC (Memory Test Computer), which only had only four instructions and was built for a single purpose: to stress test early "core memory" and prove it could work reliably under load. The second was the TX-0 (Transistor Experiment 0), Ken Olsen's pet project to prove that transistors could be made reliable enough to build a computer and memory out of them. (Early transistors were so sensitive to static they were even less reliable than vacuum tubes, Olsen came up with a way to buffer them so they could survive normal handling and operation.) Both prototypes were heavily modified and upgraded by the students at MIT, because although they weren't very powerful they weren't shared, either. Each belonged entirely to the AI lab 24/7, so the students there didn't have to make an appointment to use it or get permission to modify it.)

Olsen went on to found Digital Equipment Corporation to commercialize his new transistor memory, and hired a bunch of his fellow MIT graduates to do so, finding the most useful ones came from the AI lab that had received his cast-off prototypes. (The book "Hackers" by Steven Levy covers the founding of the MIT AI lab. This marvelous interview with Ken Olsen covers the TX-0 from the Lincoln Labs end, plus early transistor memory experiments, and goes on to chronicle the rise of DEC.) The TX-0 was the only one of its kind, and when Olsen founded DEC he hired a bunch of MIT graduates and sent one of the first PDP-1 systems to the MIT guys to train future graduates so he could hire them too. (The MIT guys soon wrote the first shoot-em-up video game, "spacewar" for the PDP-1, which became DEC's main sales tool for showing off the new system. DEC eventually sold around 50 PDP-1 systems.)

When DEC built its first mainframe (the PDP-6, they which shipped a total of 36 machines in its entire production run), MIT of course got one. When DEC upgraded the PDP-6 to the more powerful PDP-10 (about 700 total ever made), MIT's system was upgraded. That PDP-10 was the machine Richard Stallman did all his work on at MIT.

All of these systems, from the TX-0 to the PDP-10, were heavily modified by the MIT guys, to add new instructions and capabilities. The fact that MIT not only wrote its own operating system from scratch but customized its computer hardware to add extra instructions and registers, really wasn't that unusual under the circumstances; everybody did it back then. The computers were made by wiring together individual diodes and resistors and such on a bread board. Wiring in a few more wasn't a big deal. But it meant that programs written for a single machine ran _only_ on that machine, or perhaps one or two others that had been similarly hand-modified.

Microprocessors were different in that you couldn't customize their circuitry the way you could a breadboard full of chips and wires. There weren't just more of them but they were also more uniform, thus software didn't need to be ported between different machines of the same model, it could run out of the box. That was something new.

Only with the arrival of millions of identical microcomputers, based on mass-produced microprocessors instead of expensive hand-wired breadboards, (see the books "Soul of a New Machine" and "A few good men from Univac" for more descriptions of that era) could you write a program once and then sell the exact same program to lots of different people without porting (or even recompiling) it for each one of them. That's why the software in the 1960's and 1970's was either custom stuff individually comissioned from consultants, or it was produced by hardware manufacturers in an attempt to sell their hardware, or it was produced by the people who had the computers for their own use, and shared through user groups like DECUS and the CP/M Users Group Northwest. All this stuff came with source code.

But in 1980 Apple Inc. had a record-breaking IPO (the largest in history at the time) based entirely on sales of the Apple II. This triggered IBM to rush out its own microcomputer (the IBM PC) in hopes of squashing Apple before it grew too big to contain. (That's an interesting story in its own right, but a side track to the rise of proprietary software. Robert Cringely's book about it, "Accidental Empires", is fairly accessible and was made into an even more accessable PBS documentary called "Triumph of the Nerds".) In 1982, "The Personal Computer" was Time Magazine's Man of the Year. Exponential growth that continues long enough shakes the entire world.

Changing the law: Apple v Franklin extends copyright to cover compiled/assembled binaries instead of just source code.

The copyright issue changed in 1983, when the Apple v Franklin ruling extended copyright protections to binary code. (The writing was on the wall in that case over a year earlier, but the decision had to be upheld on appeal before commercial interests could seriously act on it.) Before that decision, source code was copyrightable but binaries weren't, so companies shipped source code to _increase_ their ownership of the code in the eyes of the law. If you just shipped precompiled binaries, you had no rights the law would recognize.

The first real "voice in the wilderness" about proprietary software was a kid named Bill Gates, with his 1976 "letter to hobbyists". Except that nobody took it seriously. (Richard Stallman claims not to have even heard about it at the time.) People only listened at the time because MITS paid to run the letter in a dozen magazines, and he wrote the letter because the world _wasn't_ like he was describing, like he wanted it to be so he could profit from it.

William H. Gates III ("Trey" to his friends) was a rich kid and the son of a lawyer who dropped out of Harvard (pre-law) to start a software company with a techie friend of his (Paul Allen). Gates's Micro-soft was founded (in 1975) in hopes of not merely selling software to the emerging microcomputer market, but explicitly cornering that market. Except that their tiny company could only afford starvation wages, to the point that both of its co-founders, Paul Allen and Bill Gates, took day jobs at their main client, MITS for the first few years of the company, and their third employee (the guy their "letter to hobbyists" credits with actually writing Altair BASIC, although their official history now differs), was a part-timer they had to lay off.

They referred to their strategy companies as "riding the bear", attaching themselves like a remora onto one dominant hardware manufacturer after another, grabbing the software crumbs falling from that hardware market and switching to the next when it lost its position. First they rode the Altair's manufacturer MITS, then switched to Tandy as their meal ticket when MITS went under. Finally they landed a lucrative contract with IBM (because Gates' mother was on the board of directors of the Red Cross with IBM's CEO and thus IBM could do business with "Mary's Boy" even though IBM wouldn't ordinarily approve such a tiny company as a vendor because it might not stay in business long enough to complete the contract; see the book "Big Blues" for details). Along the way, they sold their software (mostly BASIC) to Commodore and eery other company that would buy it.

Ironically, the first software product of micro-soft was Basic (a language developed at Dartmouth college in 1964 and freely appropriated by micro-soft). Because it was the only high level programming language MITS offered for the Altair, it quickly became the lingua franca of early 8-bit microcomputers, starting with the Altair clones from companies like Imsai but spreading to the Comodore 64 and Apple II and even IBM's PC. The thing about Basic is it was an interpreted language, where the source code _was_ the program and you ran it directly. Books and magazines distributed BASIC programs by printing source code could type into your own computer. Again, all code written in early 8-bit BASIC was open source, because _not_ being open siply wasn't an option. The language itself enforced it.

Gates was the type of person to not only see a market opportunity in bottled water, but then consider drinkable tap water as a threat to his business and respond by trying to make tap water undrinkable.

In his 1976 letter to hobbyists, Gates was incensed that microcomputer users mostly wrote their own software, and shared their programs freely with others. Users saw Gates as the type of person to not only see a market opportunity in bottled water, but then consider drinkable tap water as a threat to his business and complain about it. Back in 1976, he wasn't laughed off the stage: he was completely ignored.

It's a bit like the recent Time magazine article about how newspapers must start charging for online content by a guy named Walter Isaacson. It's considered a laughable idea today, when the norm for internet content is free access. But if that changes, Isaacson will look like a prophet.

What users didn't expect in 1976 was that Gates and others like him would respond not only with FUD (such as "shrinkwrap licenses" which were laughably unenforceable at the time) but by trying to change the law. Apple's 1980 IPO attracted Wall Street's attention to the booming computer industry, the sharks smelled money in the water, and proprietary interests have lobbied continuously to change the law for their own financial gain ever since. (Note it's still ongoing: the software patents blowup happened in the 1990's, and the Digital Millenium Copyright Act at least claimed to make shrinkwrap licenses enforceable in 1999. Today Rupert Murdoch (owner of Faux News) is trying to implement Isaacson's ideas.)

The thing about Gates is repeated failure didn't discourage him. Here's a 1980 phone interview with a young Bill Gates (audio, transcript, context) who was incensed that a recently published book about TRS-80 programming had printed an annotated dissassembly of the TRS-80 ROMs, so hobbyists could understand their machines better. Gates insisted those roms were his property, and nobody else had the right to look at them (let alone print out annotated disassembly and sell it for profit), and he lobbied congress to that effect. The entire interview is Gates talking about his attempts to change copyright law, back in 1980, into what he wanted it to be.

Similarly, when Microsoft belatedly noticed the internet in 1995, it tried to replace it with MSN, the Microsoft Network. (Remember that in Windows 95, the "unremovable" icon wasn't Internet Explorer, it was MSN. When that strategy didn't work, they bought a browser called Spyglass and renamed it Internet Explorer and made _that_ the unremovable icon, in a second slightly more successful effort to at least partially de-commoditize the internet.)

But back in the early 80's, it wasn't Microsoft that succeeded in changing the law, it was Apple. The Apple v Franklin lawsuit was Apple's response to a company (which became Franklin Ace, makers of the Speak-N-Spell) that cloned the Apple II. Franklin's computers ran Apple software, and they used Apple's ROM code verbatim in their competing product in order to do that. The courts decided that Franklin had gone too far, and that copyright must extend to cover binary code to prevent that sort of thing.

Part of the reason AT&T agreed to be broken up in 1983 (implemented in 1984, but agreed to in 1983) was to get out from under the antitrust regulation imposed on it by a 1956 consent decree, forbidding it from entering any other business beyond its telecommunications monopoly. It nominally owned Unix, by then a widely used operating system, but couldn't enter the booming computer industry as a heavily regulated telephone monopoly. When Unix first leaked out of Bell Labs in 1973, there was nothing to commercialize because there was no market. In between, the computer industry _changed_. Unix was open source from day 1, and developed a vibrant community in the 1970's based on this. (It didn't have a name for open source because it didn't need one, there wasn't any non-open-source software to speak of. When copyright first became an issue for software, the term commonly used for unencumbered software was "public domain".) When AT&T suddenly started enforcing its newly acquired copyrights, and charging _extra_ for source code, it was an enormous shocking change.

AT&T wasn't alone in this, of course. IBM's Object Code Only (OCO) policy was announced February 8, 1983 (more on this), but software predating that commonly came with source code, and often even that source wasn't copyrighted. This is why the Hercules S/370 emulator has plenty of old public domain software to run, including things like the MVS operating system (see also here). Yes, even IBM's mainframe operating systems worked by different rules back in the 1970's.

RMS tells his own story (in his own words, or biographical version) about a Xerox printer he couldn't get the driver source to. This came as a shock to him because previously, giving source code to any software was a matter of course. The world was changing under him, and he didn't like change.

So when Richard Stallman founded the FSF in 1983, the same year DOS 3.0 came out, he was reacting to the recent and ongoing rise of proprietary software. Microprocessors created an exponentially growing hardware market, the Apple vs Franklin decision gave software authors vastly expanded copyright powers, and a gold rush ensued (similar to the dot-com boom of the 90's) that hired away all the "hackers" from the MIT AI lab to work in a pair of start-up companies attempting to commercialize LISP. While his fellow students graduated and took jobs out in the booming commercial computer industry, Stallman stayed in academia as a perpetual grad student, and hid away from the problem as long as he could.

Kicked out of the nest: cancellation of the Jupiter project obsoletes the PDP-10 at the heart of the MIT AI lab.

The big change that forced Stallman out of his comfortable nest at the MIT AI lab was the obsolescence of the operating system he used, which was tied to specific hardware that was discontinued.

Over the course of three decades (1950's, 60's, and 70's), the MIT AI lab upgraded from the tiny Memory Test Computer all the way to a PDP-10 mainframe, keeping ahead of Moore's Law (which Gordon Moore first documented in 1965). Just as the PDP-6 became obsolete and was upgraded into the PDP-10, the PDP-10 itself was over a decade old by 1980, and ripe for replacement.

To replace it, DEC prototyped a new machine called the "Jupiter project", which was roughly compatible and would thus allow PDP-10 software to be easily ported to the new system. But there were only 700 PDP-10 systems ever sold, and their newest computer (the VAX, an upgrade to the PDP-11) was selling thousands of systems per _week_.

The VAX was completely incompatible with the PDP-10. The PDP-10 (like the PDP-6 before it) was a 36-bit machine with 6 bit bytes, the VAX (like the PDP-11 before it) was a 16 bit machine with 8-bit bytes. Porting assembly software from one to the other essentially required a complete from-scratch rewrite.

By 1980, Ken Olsen's DEC had now been producing and selling its own computers for 2 decades, and could hire programmers familiar with them from anywhere, so he no longer needed the MIT AI lab as a source of labor. The industry was standardizing on 8 bit systems and leaving 6 bit systems behind, and the 700 customers with PDP-10 systems were already a rounding error compared to the 600,000 PDP-11 systems eventually sold, and the VAX looked to be even more successful. Thus DEC decided to focus on 16-bit systems, told its PDP-10 customers to migrade to a VAX instead, and cancelled the Jupiter project in 1983.

With the cancellation of Jupiter, the operating system the MIT guys had written over the years, a giant pile of hand-coded assembly proudly named the "Incompatible Timesharing System" (which gives you a good idea of the AI lab's "hacker" culture), was now tied to a dying hardware architecture which would not be upgraded. The writing had been on the wall for years; no new students had bothered to learn the obsolete ITS or PDP-10 programming for a while, and the booming computer industry in Boston hired away the graduates who had been maintaining ITS, until eventually Richard Stallman was the last holdout clinging to the old ways as a perpetual grad student. When MIT announced they were retiring the now-obsolete and unmaintainable PDP-10/ITS system, Stallman's carefully insulated little bubble of ivory tower academia popped, forcing him to respond.

Richard Stallman is not and never was a visionary. He's a reactionary conservative trying to cling to the past, and when the rug was finally yanked out from under him he set about rebuilding that past as best he could. Some of the things he fought to preserve were indeed worth preserving, but looking at him as any sort of guide to the future is ludicrous. He's never been about the future, except to warn against it.

There are of course more questions (why would an ITS guy switch to Unix, why was the FSF as successful as it was, how do lisp and microkernels show the dude is totally totally not a visionary, who _else_ was doing this, and where did it all go wrong), but again they'll have to wait because this is already long enough.

So I wrestled with the design of the new qcc wrapper for a while, unsure quite how I wanted to proceed, but now it's turning into a variant of ccwrap that breaks down stages. It calls cpp to do preprocessing (and that's the stage that gets

The advantage of doing it this way is it both partitions the task nicely (eliminating duplicate code, so multiple different stages have to understand about library paths and such) but also lets me test against the existing gcc stuff before I've built the new replacements. (Writing a wrapper I can't _test_ makes plugging in other functionality a real pain, because if it doesn't work it's not immediately obvious _why_. It's also no fun preprocessing C code you can't compile, or building .o files you can't link...)

So I'm designing the top level "cc" wrapper to call cpp the -M options are all aspects of cpp, not cc1. At the moment, I'm just trying to get it to call all the gcc tools in order (cpp, cc1, as, ld, strip) without telling one of the multi-function tools to do anything one of the single function tools can do. (So even though cc1 can preprocess, and ld can strip, I'm having the specific tools that only do that do that.)

I'm also writing it to not pass on any option it doesn't understand, so if I haven't immplemented support to grab "-M" then it'll barf when given -M, instead of passing it on to cpp.

Then I can swap out this wrapper for the one in FWL and make sure it can build the whole system, and maybe gentoo from scratch on top of it.

At some point in the future, I'll probably want to coalesce stuff back together. It's possible to implement the compiler as up to six processes (cpp | cc1 | as | ld | strip, all called by cc) but passing preprocessed data from cpp to cc might not be a win vs just keeping each translated line in the processor's cache and dealing with it immediately, and outputting assembly source and then assembling it is _definitely_ slower than generating the machine code directly (although the _capability_ is still needed to support -S, which the kernel needs in order to create some of its header files).

But that's all future optimizations, which can at least wait until I get around to actually implementing the code that needs to be optimized. (I'm not too worried about anybody calling cc1 directly, and if calling cpp directly was particularly common Ubuntu wouldn't have split it out into its own package which I've never needed to install to build anything I've tried. Builds do call ld directly and expect it to strip, but that's a question of outputting _less_ stuff.)

Still vaguely uninspired about programming. Poking at qcc, but admittedly somewhat listlessly. (Currently Linux isn't seeming like a fun hobby at all; I deleted my old git howto rather than defend it against some idiot who insisted any portion he didn't understand needed to be removed from it. Apparently "I don't understand what you're saying, not that I've actually tried the technique you described to see if it worked" means "you must be wrong". *shrug* Deleted the whole thing, which seemed easiest way to give him what he was technically asking for. There are at least three other git howtos out there written by people who care. Talk to the 404.)

Followed Fade to the library this morning and got a book, Keith Olbermann's "Truth and Consequences". I was somewhat surprised he'd found the time to write a book, but apparently it's mostly transcripts of his special comments with introductions. (About 2/3 of the way through now.)

Various other errands during the day. Went out with Fade to buy a bathing suit, hit a mall.

Up to DVD 5 in bleach, which is the one where they loose track of astral form vs physical form. (Or at least he spends most of an episode with his body being doctored for wounds received as a soul reaper, despite never having established the Matrix-like "your body makes it real".)

Not that this is the first series that's needed a continuity advisor. (Red Dwarf's books tried to justify Lister having his appendix out twice, but glosses over the fact that the first Joke Lister makes upon being revived is "Three million years? I've still got that library book!" and later in the first season "I've never read... a book." Yeah, it's a comedy and these are jokes of opportunity, possibly self deprecating humor on Lister's part, and the MST3K mantra certainly applies to most of the series. Still...)

Bleach remains fun, when it doesn't go into DragonballZ territory. (Watching people yell at each other for 10 minutes isn't fun, and these people would do well to study the video game concept of "logarithmic difficulty increase", although World of Warcraft has some similar funky balance issues. I can't be _too_ upset that individual members of the strongest 10% of Soul Reaperdom could each slaughter the remaining 90%, so why the heck are they sending _those_ guys out into the human world to fight hollows? If you stop and realize that a single Stormwind guard could solve all of Westfall's problems in an afternoon, "it's required by the plot" is really the only explanation required, or offered.

There are a few tricks to understanding the series. They level by something like an XP system (or as they say, "spiritual power increases fastest when the soul is in danger"), but Ichigo's limit break is to level, so when he's almost killed he goes "ding" on his next attack (which heals him out of limit break territory). Orihime's power is based on the Mythbusters Mantra: "I deny your reality, and substitute my own" (which makes her potentially even more of a walking deus ex machina than Ichigo, although she levels more slowly). When cornered and otherwise close to levelling, all of 'em do big long "bribe the GM with a backstory solliloquy to get character development XP" to push 'em over the edge. Don't expect to get any of the jokes, the entire cultural background they're based on is different (although the ones parodying other anime series/tropes you at least have a chance on).

Currently watching the episode where Protagonist Boy regains soul-reaper-dom after starting his heartless hollow transformation. His limit break kicked in and screwed up the cutscene, and he wound up with the "hollow" and "soul reaper" flags both set. :)

So I'm writing a fresh implementation of ccwrap.c for qcc,

Wow the gcc option parsing is crap. -static and --static do the same thing, "gcc -cv hello.c" says that -cv is an unrecognized option but "gcc -c -v hello.c" is just fine... "-Lpath and -L path" are synonyms, but "-lpthread" and "-l pthread" are not...

Rather a lot of "hmmm" moments. I'm starting by reading ccwrap.c and writing a new front end "qcc" command line interpreter, which creates appropraite command lines to call cpp, cc1, as, and ld as necessary. (This means none of those commands need to know about standard paths for include files or libraries.) Note that the existing ccwrap.c calls the gcc front-end which does a lot of command line rewriting of its own. This is a harder problem, breaking up the command into stages and calling the sub-commands directly. Another issue is whether or not those sub-commands should ever call each other, or whether the way they're called should string together "cpp | cc1 | as" manually.

The issue I'm facing right now is since none of those sub-commands deal with standard includes or standard libraries, it seems silly to feed them --nostdinc and --nostdlib. (The qcc versions won't _have_ the logic for that stuff.) But the gcc version of cc1 does do that unless called in such a way that they don't have anything to do, whigh gets us bac to the "cpp | cc1 | as > out.o" thing. (On the bright side, that would distribute the work over more processors on smp systems. Is there enough work in preprocessing for this to have any benefit? The existence of ccache implies so. But the communications overhead might make it a net loss anyway. It's easy to program and keep it nicely modular, but it's not that hard to keep it in one process, either...)

Anyway, from command line option parsing standpoint, cpp needs to understand "-D -U -I -o -M* -d* -nostdinc" (and possibly also "-include -iprefix -withprefix -system"). And ld needs all the stuff from "-static -shared -L -l -Wl," and such. So presumably if qcc is dispatching right, the cc1 command line parser doesn't need to know about either...

The question is, of course, how to come up with command lines that gcc would understand, which don't require the same code to be written twice in qcc, and which doesn't impose unpleasant communication overhead between processes. (Partitioning the task! A design issue...)

I asked Mark to create a new qcc list so I can post design documents and such from the new project. Have not yet posted such a design document, but wrote a longish notes file yesterday.

Currently frowning at ccwrap.c and rewriting it from scratch in preparation of slotting the options.c material into place. There needs to be a cc applet that rewrites the command line (so "cc hello.c" becomes a much longer command line with -I and -L entries and mentions of crt1.o and such) and then calls a cc1 applet that does actual compiler thingies. But whether it should literally create a new command line and call another main function with its own command line parser, or whether it should just set up data structures internally which the other command line parser could also do, is an open question. (Leaning towards the first. Simplicity vs code duplication is one of those tradeoffs that makes you want to step back and think about the problem at a higher level because there should be a way to get them to not fight like that...)

Part of it is "should this wrapper still be able to call out to an external compiler, ala gcc?" The current wrapper tweaks $PATH because gcc won't be able to find its own sub-commands otherwise, which clearly _isn't_ required for a qcc thing calling another applet internally via a busybox built-in command style multiplexer pseudo-exec. But if the other 90% of it _does_ work that way, is losing that functionality good? (Especially if I'm going to be writing low hanging fruit like ld and ar and such first, and possibly handing off to some other cc for test purposes...)

Another thing is that ccwrap.c has c++ support, and qcc doesn't (and probably won't any time soon, since tinycc never did). If it's doing c++, it needs to call out to an external c++ compiler. (At least something like cfront.)

Eh, worry about it when I've got more code written...

I admit the #fixreplies thing is an impressive #twitfail. They spun it as a user interface improvement, except that it used to be a configurable setting which already defaulted to "off". They removed the _option_, with no warning, and called it a UI improvement. That's what we call rolling a crit-fail on your intelligence check: When you have Neil Gaiman retweeting Stephen Fry pointing to an article about how stupid your policy is (both of those people are in the 10 most followed twitterers list), you have officially screwed up.

It turns out that the change was due to technical limitation in the database scalability or some such (which means they can't put it _back_ easily), which in some ways makes it worse. This was actually a _marketing_ crit fail. If they'd said at the start "we had to yank it because it didn't scale, the code's unmaintainable or it bogs the servers, we're working on a new implementation", that's one thing. But the actual quote from their "small-settings-update" announcement was, "Today's update removes this undesirable and confusing option." I never even _used_ the option that was removed, so my config hasn't been affected by this, and I'm still kind of annoyed at being patronized like that. I don't want them fiddling with the UI "for my own good", nor do I want them insisting they know how I'd prefer to use their service better than I do. They earned the wall of flame coming back at them right now.

They've been dancing around an apology ever since, but have yet to actually do a Mea Cupla for treating their users like idiots. Nor have they promised not to crash around like tone-deaf elephants in future, perturbing the UI of a service millions of people use with no warning or beta test site. That's an easy way to get millions of people to find or set up other microblogging sites, which would mean I'd need an rss aggregator to follow microblogs. Once upon a time, the livejournal friends list was all you needed. Now as Jeph Jaques says, "Can you believe people still have Livejournals?"

It's inevitable that the idea of microblogging diffuse beyond a single website, the way free website hosting isn't just "geocities" anymore and free blog hosting no longer means "livejournal". In a lot of ways livejournal was doomed by RSS aggregation. But right now, there _isn't_ a way to do @replies across sites, and until we come up with one and spam-proof it, twitter still has a structural advantage. But making their entire userbase see them as poor custodians of the service they use probably just shortened their time of dominance by about a year.

Reading the tinycc source, collecting bits easy to yank for qcc. Got curious and browsed alloca.S. Wow, I need to brush up on my assembly.

I was a bit confused why "alloca.S" didn't compile from the command line, but then I wandered through the makefile and noticed there's no rule to build it. So I looked at the git version of tcc-grischka, and although they've got an alloca.S there too _it_ never gets compiled either.

So my original plan of "tweak and test build" to see if I can "pop %eax" instead of "pop %ecx" then "move %ecx,%eax" and so on... Kind off derailed by the fact that the original code I'm reading doesn't work, and the last time I actually tried to _author_ x86 assembly code (rather than figure out what debugger output is doing wrong) was... 1994, I think? Yeah, time to brush up...

(Hence my desire to get online and download an x86 assembly tutorial. I've got books on it at home, just not with me. And a reference != a tutorial...)

Oh _that's_ cute. Although gcc won't compile alloca.S, tinycc itself _will_. Ok, at some point I need to figure out what's wrong here, but for now...

And it died because I haven't got the 32-bit development files installed. Right. Come back to this when I have internet access...

Why am I *ahem* "not a fan" of the FSF? Ok, I'll explain.

I'm not sure you can be a computer historian and not hate the Free Software Foundation, because they're not who they say they are, they claim credit for things other people did, and these days their agenda has become counterproductive and harmful.

A revolution to preserve the status quo

The US was founded by revolutionaries, hence the name "Revolutionary War". The founding fathers tried out all sorts of new ideas, and often took multiple attempts to get it right; replacing the articles of the confederation with the constitution, then adding a big batch of amendments to the consitution in the bill of rights, swapping in the electoral college when the first method blew up in their faces, going back and forth about whether or not to have a central bank, eventually needing a civil war to finally resolve the whole slavery issue...) The US founding fathers would be the _first_ to say they were imperfect people just doing the best they could.

These days, the founding fathers are primarily invoked by ultra-conservatives, often while fighting against things like desegregation, women's rights, gay marraige, bikinis, and so on. They're about as far from revolutionaries as you can get. (As Mark Twain said, "Conservatism is the blind and fear-filled worship of dead radicals".) If you couldn't do it in their grandfather's day, you shouldn't be able to do it now (so throw out that cell phone and don't use antibiotics). You wouldn't think this attitude would come up much in the computer industry, but human nature is the same everywhere, and thus we have the Free Software Foundation.

The main reason I can't stand the FSF is they're ultra-conservatives masquerading as revolutionaries. While that's always a possibility when "the revolution" drags on for 30 years and the ideals they're fighting for fossilize in their proponents heads, that's not what happened here. The FSF actually started that way.

The FSF was founded as a conservative reactionary movement. Back in 1983 Stallman's goal was to stop (or at least personally avoid) the rise of the brand new proprietary software industry, and return to the glory days of the 1970's.

Whether or not a return to the past was a _good_ goal is a separate issue; there was nothing visionary about attempting to roll back the clock, so the people who see RMS as a "visionary" today are _badly_ misinformed. He is not a visionary leader, he was taking us back where we'd already _been_. If anything this makes him _less_ qualified to predict the future, because his most conspicuous trait is that he doesn't like change.

Unfortunately, this doesn't stop him from trying to predict the future, proclaiming dire warnings about it, and attacking others who have a different ideas about what's ahead. Nor does it prevent hordes of impressed followers (who don't know anything about computer history and think this future/past he's describing is great) from hanging on his every word.

Before there was a proprietary software industry, 1939-1979

So why is "free software" older than proprietary software? The proprietary software industry didn't exist back in the 60's and 70's for two reasons:

First, until the early 80's there wasn't a critical mass of customers to sell proprietary software to.

The MIT AI lab where Richard Stallman learned to program was started with a computer called the TX-0, a unique system built as a prototype by Ken Olsen, the guy who went on to found Digital Equipment Corporation. (The book "Hackers" by Steven Levy covers this in a lot of detail, as does this marvelous interview with Ken Olsen.) That was followed with a DEC PDP-6 mainframe, which the shipped a total of 23 machines in its entire production run. The lab was later upgraded to a PDP-10, of which about 700 total ever made. The fact that MIT modified its computers to add extra instructions and wrote its own operating system from scratch really wasn't that unusual under the circumstances; programs written for early mainframes and minicomputers often wouldn't run on any other machine in the world even if they hadn't been modified by their users, simply because their component selection and configuration was unique.

For most of the 1970's, the best-selling computer in the world was the PDP 8, which was introduced in 1965 and sold a grand total of 50,000 units during its entire production run. The PDP8 was displaced as bestseller by the Apple II, which was introduced 1977 and over the next two years sold a combined total of 43,000 units. But that was enough to put it on top: in 1980, when Apple had its IPO, it had 50% market share of the computer industry (according to the PBS series "Triumph of the Nerds").

So if you tried to sell "shrinkwrapped" binary software back in the 1970's, the largest possible market you could hope for on any platform was _less_ than 50,000 copies, and even that's only if you targeted the most popular platform in the world and somehow managed to sell a copy for every single computer of that type ever produced.

The second reason the proprietary software industry didn't exist in the 60's and 70's was that copyright law didn't cover compiled binaries. Human readable source code was copyrightable (as an original work of human authorship), but binaries produced by a compiler or assembler were "just numbers", and the law didn't explicitly allow them to be copyrighted. So if you wanted to claim any copyright on your program, you were _required_ to ship source code. Shipping source code gave someone _more_ control over the result than just shipping a binary.

Thus publishing source code (with copyright notices) became official policy throughout the computer industry (DEC, AT&T, IBM)... They actually had fewer rights if they _didn't_ publish their source code. (When Unix came out in 1974 what was special about it wasn't that it came with source code; that was normal. What was special is that the source code was in a high level language that minimized porting effort to move it from machine to machine, instead of the assembly language all the other operating systems were written in.)

Software in the 1960s and 1970s was either produced by hardware manufacturers to help sell their hardware (who happily gave out the source code, generally written in assembly language), or else it was produced by the people who bought the computers for their own use (who had no incentive to withold source code from _themselves_). Users sometimes hired consultants to a write custom software for them (in which case the source code was one of the deliverables), but back then the only way to use a computer was to learn how to program it, so users wrote most software themselves or had someone on staff to do it for them. The vast majority of software in the 1970's and earlier was written by the users, who naturally shared the results through early user groups like DECUS and the CP/M Users Group Northwest.

So how did this _change_? Where did proprietary software come from, and when, and how? How did Richard Stallman's little utopia at the MIT AI lab crumble and force him out into the wilderness to try to rebuild it?

I should probably hold off until next time for that. This has gotten a bit long. :)

Ok, I know what to do with my tinycc fork now. :)

The obvious way to replace gcc is with qcc, the QEMU C Compiler. Take tinycc, break it up the rest of the way, and slot it into the qemu source code (in a qcc subdirectory) so it can build an "$ARCH-qcc" from each current "qemu-$ARCH", using tcg (the Tiny Code Generator) as a backend.

I'd held off on gluing tcg to tinycc because forking tcg wouldn't be fun (or productive). It's still under active development, any changes I need to make to it must go upstream or they'll become a maintenance nightmare. Pushing the compiler front-end upstream into qemu makes noticeably more sense, if they'll take it. (But they already _do_ all the hard parts...)

Right now qemu application emulation can parse ELF executables and shared libraries, so qcc's linker needs to use that code (and teach it to handle .o and .a files). So in addition to redoing the code generation backend, the linker needs significant work to merge it into qemu.

I split my fork's option parsing logic off into its own file a year ago, and I still need to do the swiss-army-executable thing on it and merge it with the ccwrap.c in FWL.

That's a lot of work. Trying to figure out what order to do stuff in...

Anyway, providing a decent replacement for gcc means the only GPLv3 piece of software that actually matters is Samba, and the kernel has built in smb client support so a simple server could probably be exported from that. (It might not support Windows 95, but I can't really say I care. Current samba isn't exactly embedded code right now, is it? Oh, and the dirty trick to ignore all the case insensitivity issues: export a vfat filesystem. :)

Not quite back to my normal levels of enthusiasm (or at least bloody minded stubbornvindictive problem solving), but any realistic chance to rip out one of the foundations of the FSF's power base is always invigorating.

Still apathetic about programming in general.

Linux on the desktop is dead. Ubuntu 8.04 doesn't work with Netflix or Hulu or Youtube's television options. 8.10 was a disaster. And now 9.04 doesn't work with Intel 3D, not only the most common one but the one that used to be recommended for use with Linux. Meanwhile, Apple's market share finally hit double digits. (Whether or not Windows 7 sucking less than Vista counts as damning with faint praise or is actually significant, I reserve judgement on.)

GPLv3 has fatally split the community. The last GPLv2 version of gcc is 4.2.1 which is increasingly obsolete (doesn't do armv7 and up, meaning no thumb 2), and there's still no replacement compiler. Neither PCC nor LLVM/Clang has turned into anything useful yet, and tinycc development is just painful to watch.

Actually, tinycc development is _hilarious_ to watch in a black comedy way. Grischka admitted that he didn't take most of my code because he didn't understand it, and now _three_years_ years later he's slowly reproducing some of the obvious refactoring I did literally back in 2006. At this rate it might only take him 3 more years to get to the point my fork was at when I abandoned it.

It's painful to go back to work on BusyBox both because of lingering Bruce and because I like the toybox infrastructure so much better. And there's a lot of cleanup work to do, starting with removing the forest of #ifdefs littering the code now.

But convincing people that conceptual bloat and what you can measure with bloat-o-meter are not the same thing is exhausting. (A wrapper script around mdev is a better design decision than trying to shoehorn firmware loading into something that wasn't meant to do that. It's not mdev's job to do every possible hotplug activity, and complicating the hell out of the config file syntax is not the answer. Also, removing the mmap config file parser in favor of much slower and more complicated code that allocates memory for each line of text in the config file (and then frees it again), and does this again and again for every single device during "mdev -s"... That was wrong. I don't care if it's shared with other applets, it's still wrong.)

The reason for mdev in the first place was I disagreed with the design of udev, and that there should be an obvious and simple way to do it. I had to fight the kernel guys (Greg KH and Kay Sievers) who thought that sysfs was a private export and not a stable userspace API, so anybody trying to write their own code to do this rather than using udev was obviously crazy. More recentlly I had to fight the kernel guys (pretty much Peter Anvin) who are rewriting the kernel build system in perl. In addition to not needing an additional large prerequisite package which would rule out sane languages like Python or Lua, perl is a specific programm, not a language. It has no standard, merely a single implementation (just like Microsoft Word or Microsoft Excel). Whatever that implementation does is Perl, and it's never successfully been reimplemented (despite the Perl 6 guys trying for over a decade now). That's not a language, that's a legacy hairball.

Sigh. I feel like I'm fighting the entire Linux community, and I'm tired of it. Yeah, you can duck into the embedded world and avoid most of that, but in a proper embedded system you don't care what it's running. Might as well be DOS. The current most interesting embedded device (the iPhone) has MacOS X.

When this laptop dies, maybe I'll buy a mac. (About half the kernel developers are already doing their kernel development on macs, from what I can tell...)

Poked at build strace a bit in the evening. This is a weird program. Building natively under armv5l, it went:

In file included from syscall.c:129:
linux/arm/syscallent.h:435:3: error: #error fix me
linux/arm/syscallent.h:457:3: error: #error fix me

I'm guessing that strace 4.5.18 doesn't support the 2.6.29 kernel, at least not on an arm host. (Instead of failing by saying unknown syscall at runtime, it fails by refusing to build. That's nice. I note that 4.5.18 is the most recent version on strace.sf.net and that page has no link to a mailing list...)

The source code's README says:

You can get the latest version of strace from its homepage at
http://www.liacs.nl/~wichert/strace/ .

Which is 404 and according to archive.org has been since 2006.

I'm not sure "maintained" is a word that applies to this package.

Mark pointed out that the Gentoo guys have patches for this, and apparently have for some time. (Apparently there's been nobody upstream to send them _to_? Dunno.) Oh well, yay it's fixed, moving on...

Poking at a dropbear build script under the native root filesystem for a nightly cron job. Alas, dropbear went to autoconf for some inexplicable reason, so of course it sits there making stupid tests (very slowly):

checking for gcc... gcc

Yes, gcc is called gcc. If you were checking for "cc" (the standard name required by SUSv3; SUSv4 seems to require "c99" now), that might make some of the later tests less silly.

checking whether we are using the GNU C compiler... yes

Um, why do you care, and what was the first test for, then?

checking for C compiler default output file name... a.out

You can specify -o all the time so this never matters. Same with suffix of executables and suffix of objects files, but you can also just accept that there are exactly three interesting build platforms these days (Linux, MacosX, Windows) and you're under no obligation to support all of 'em anyway.)

checking whether the C compiler works... yes

During the build, you'll notice pretty quickly if it doesn't. You'd also get a better error message saying _why_.

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

A) This is required by the Single Unix Specification, B) there's a gcc that doesn't accept -g?

checking for gcc option to accept ISO C89... none needed

Again, your very first test was for gcc, then an explicit "are you really gcc and not another compiler pretending" (no idea why), and now it's doing this kind of crazy stuff.It just goes on and on like this. But that's not the funny part. Here's the funny part:

checking for deflate in -lz... no
configure: error: *** zlib missing - install first or check config.log ***

You have to explicitly say --disable-zlib. What exactly is the POINT of running ./configure if you have to explicitly tell it on the command line what your environment contains?

Of course when you do that it then goes on for a long time checking "is the strip command called strip" and so on, explicitly checking whether or not our C compiler has "memcpy"... and then at the end it tells us to edit options.h to select the options we want.

Sigh.

Saw Fade off the airport for Penguicon. (I'm not going this year, the con chair and I don't get along, it's probably better for the event if I stay out of his hair while he's wearing the con chair hat.) I feel bad I won't get to hang out with Garrett and Christian and Tracy and Heather and so on, but oh well. Maybe next year.

Ok, I know I'm weird, but I'd just _assumed_ the Penguicon guys had a video link set up so Wil could address opening ceremonies (and maybe even voip-phone in a panel or two) from home if he couldn't physically make it to the con again. USB web cameras are what, $15 these days? After a Red-sox-like streak had established itself, it just seemed like common sense to me. (Although I admit my money was on "meteor strike" this time. And no, the third time isn't "hindsight".)

Rather than turn this into armchair quarterbacking, here's a Plan For Getting Wil To A Future Penguicon. (I don't know who future Penguicon runners might want to invite, but in case they someday decide to try again with Wil, and he's up for the abuse, here's how I'd go about it.)

• Step 1: Go ahead and send Wil a cheap USB web camera, one that works with macintosh laptops. (This is a gift, he can keep it. Also note that most macs have one of these suckers built in. Send him a spare _anyway_.)

  Also send Wil a badge that says "Sir Not Appearing At This Con". (Make him promise he will wear it through the entire weekend of Penguicon, wherever he happens to be during that time. Demand pictures. Yes, this will be his badge if he does, by some fluke, wind up attending. Ribbons for him to hand out to Penguicon attendees, possibly by mail, are left as an exercise for the reader.)

• Step 2: Keep a second webcam on site in case Wil winds up bedridden in his hotel room (or a local hospital) instead of bedridden at home, so he can still occasionally say hi. This has the added benefit that it could potentially be used to record panels downstairs if it isn't otherwise needed. (Hey, gotta plan for everything, even the possibility of success...)

• Step 3: make a case for inviting him back a fourth time. Start by ripping the Monty Python and the Holy Grail clip about Swamp Castle, everything from "They said I was daft building a castle in the swamp" through "but the fooourrth one stayed oop!". (It's short enough it probably falls under fair use, especially with what we're going to do with it.) Have Wil use his web camera to record himself describing his first three attempts to get to Penguicon. Intercut the "burned down, fell over, and then sank into the swamp" bits with Wil's description of his previous Penguicon weekends. When competed, post this video on the Penguicon web site. (This will also be played during opening ceremonies. If necessary, use the corresponding scene from Babylon 5 about the previous stations for the next attempt.)

  Note that using the webcam for this rather than some other video camera has the advantage of testing out the webcam to make sure it works and is set up properly ahead of time.

• Step 4: Fly him in early.

  The first few years of Penguicon we flew all Guests of Honor in on Thursday wherever possible, so there was some failover in the plan. This gave the guests time to unwind, because "travel and perform the same day" is hugely stressful (as those of us who ever travel for business know). The travel itself is also less stressful when you're _not_ on an immediate deadline. (Yeah spending an extra couple hours waiting after a missed connection sucks, but there's no rush, you're not missing any panels, you can relax.)

  Given that this is a third fourth attempt, fly him in on _Wednesday_, and arrange some local event for him for Thursday so he isn't wasting his time. (Reading/signing at a local bookstore or two, perhaps? Is there a local radio program that interviews authors? Or if Detroit has anything at all like the Alamo Drafthouse he might want to introduce a special showing of one of his movies. It's a big city he doesn't get to much and we're springing for the extra hotel room day, ask him for suggestions. Maybe he wants to go to the zoo; up to him. The point is, it's ok to ask him to allocate us a little more time as long as he can make good use of it. I believe Terry Pratchett used the extra time to sit quietly in his hotel room and write the "you all meet in a tarvern" scene near the start of Monstrous Regiment, which he read for us the next day one of his talks, and we were all pretty happy with this arrangment...)

  (The Thursday night dinners with just the GoHs and concom are a nice fringe benefit of the "bring 'em in early" approach, but they're also entirely optional. I only ever actually made it to one of those myself, tended to be busy a lot right before the con for some reason...)

• Step 5: Offer odds on the web page. (Possibly Lloyds of London could help here, I dunno how that works.)

  Suspense! Drama!

• Step 6: Recuit help.

  Inviting Guests of Honor is always a question of getting good combinations that would work well together; people who are already friends, or fans of each other's work, or would go well on panels. (This part worked well this year, and Andrew Hackard stepped up to fill in for Wil.)

None of this would have actually prevented him from getting sick, of course. But it would allow the whole process to fail in the most entertaining possible manner. :)

(Actually, It occurrs to me that the logical way to get Wil to a future Penguicon is don't announce it. Do it in secret. Fly him in quietly, sneak him into the hotel, and then have Ninjas carry him into opening ceremonies (bound and gagged, if possible, and untie him on stage). Schedule either "Robert Wesley" or "Stephen E. Whitfield" (various pen names used in Star Trek) on Wil's panels and swap him in for those... Somebody on twitter suggested that we should declare he's been reconned into previous Penguicons and that this is actually his Xth appearance at the con...)

Eventually decided that building under emulation was the way to go after all, even for three small packages. For one thing, I'm not trying to get strace to cross compile. For another, it's nice to have a smoke test of the build environment to catch things like the fact that the current buildall.sh is leaving prefixes on the _native_ toolchain. (Sigh.)

So I want to build static busybox, dropbear, and strace binaries for each target. This is easy to do; I can think of at least three different ways off the top of my head. It's so easy I can sit down and write a script to do this with no references or research. Which means it's one of those class of problems where figuring out the _right_ way to do it makes my brain hurt and I get writer's block for days.

I can just fire up the native build environment, wget the source, untar it, cd into the new directory, make defconfig, make, and copy the binary back out. (Building dropbear first makes copying the binary back out noticeably easier.)

In practice, the config I want to build isn't _quite_ an actual defconfig, I switch a lot of stuff off that doesn't build on various targets (m68k hasn't got taskset) or without strange host support (selinux is evil). So I need to grab that trimconfig file out of FWL, but I can wget it straight from that URL if I like. (Put "tip" in for the version number and it auto-updates.) So I don't quite need to grab the full FWL infrastructure, but it's not entirely independent either.

Next, grabbing the dropbear source again every night is a bit impolite to the download servers because it's not going to change that regularly. Desite my desire for it to grow stunnel support, the project is mostly complete and in maintenance mode. The FWL infrastructure has cacheing already; the download and setupfor functions handle this transparently. So if I _do_ grab the FWL infrastructure, I get this pretty much for free.

I can also cross compile both of these packages, which is quick and easy and ignores the whole point of the project (that building natively under emulation is the way to go, and cross compiling is a transitional stage you get through as quickly as possible). But in each case, I've got a package that's already set up to cross compile and which produces a single static binary as its output, and the rest of the nightly cron job is a cross compiling so I've already _got_ a cross compiling build environment set up. It's even got the FWL stuff sitting there ready to use, and if I jump into the emulator I have to re-download FWL and set it back up if I want to use it.

So there's a simple way to do this which has obvious flaws, and a more complicated way to do this that probably isn't worth the complication, and this is the kind of thing that makes my brain hurt.

There are some actual fiddly bits. The download.sh script deletes everything out of packages that it doesn't know about. (This is to automatically zap old versions of the packages after you upgrade download.sh. It's actually a bit of a pain, because it means that things like qemu (which are speculatively downloaded) or alt-packages (which are sometimes downloaded and sometimes not) get deleted and then have to be re-downloaded, or else are downloaded and kept around when they don't need to be.)

I don't want to add dropbear to download.sh to unconditionally download it every time, and I'd like to move strace out too. Have some kind of cron.sh call download.sh stuff. But then I have to use a different download directory (not "packages") or the extra files will get wiped every time the original download.sh script runs. And the problem with _that_ is that the original download.sh is already downloading busybox, and having the second directory download a second copy of the busybox tarball is kind of icky.

Did I mention that root-filesystem.sh already builds busybox, and could easily be taught that BUILD_STATIC=1 applies to busybox? Except that it would build a lot of _other_ stuff (like uClibc), and would thus be an inefficient way of building that one binary (unless I wanted to have all the root filesystems it built be static), which I don't. So this is either going to be duplicated or is going to have too much fiddly configuration shoehorned into it, and both suck. Sigh...

Again, it's the _little_ things that screw me up. The ones that don't really _matter_ in the larger scheme of things, but are nevertheless _ugly_, that I go round and round on...

Don't mind me, I'm having an aesthetic reaction to a simple problem. FETCH THE CHAINSAW!

In a lot of ways programming is an art instead of a science. Debugging, especially. "I don't know what it's doing wrong" is pretty much a requirement for a bug: very few programmers intentionally write the code wrong, so it's misbehaving either because what you wrote didn't match what you had in your head or because what you had in your head didn't take something into account.

An awful lot of reworking is because "no, this isn't right" without being tied to a specific bug. You redesign some bit to make it better (often simpler and relying on fewer assumptions now that you understand which parts are actually important), and as a result bugs you haven't actually hit yet go away. And then when somebody hits a bug in an older version and ask you which patch fixed it you can't really answer because you never saw that bug. And sometimes "which patch fixed it" was a redesign, where the memory leak went away when you switched to mmap(), or the bug was in the translation module and now you've changed the other parts of the program to all talk the same language so there isn't a translation module anymore...

I bring this up because software suspend in Ubuntu 8.04 semi-regularly corrupts the vfs cache if the filesystem is active when it suspends (such as running a build that'll take another 15 minutes to complete and I don't want to wait that long just now, but I don't want to abort the build either). And I know the kernel in 8.10 didn't do that once in all the months I was using it (which is why I forgot about the problem), but doing the Ubuntu upgrades when I installed this laptop (at which point the release was almost a year old) didn't fix the bug; the old kernel is still susceptible to it. Which patch along the way to the new kernel fixed it? Probably not that simple...

I wonder what's involved in getting a Gentoo desktop with kmail on an xfce base and a working Flash plugin. Highly _unlikely_ to be particularly simple, but it's something I could much more easily tweak myself once I got it that far... (This is probably a bit like saying "It's hard to get a tank platoon 200 miles behind enemy lines, but once you do things get much easier".)

Wait until Mark gets back to town. Definitely.

So I'm fiddling with upgrading the system images to gigabit ethernet (e1000). I just did i686 and it's slightly faster, but not much. Maybe 3.5 megabytes per second. When I do a "dd if=/dev/zero of=blah bs=1M count=100" and time it, it seems to be getting maybe 11 megabytes/second. Hmmm, when I do a wget of a 120 megabyte file and add "-O /dev/null" it also gets about 11 megabytes/second. Right, there's some sort of emulated PCI bus contention going on. Whee. Still, sending stuff to/from distcc might go faster now, which would be good.

Finally wrestled canadian cross building of static toolchains into submission. Lots of darn fiddly little things to get right (one of the fiddlier, strangely enough, was making sure that the prefixes of everything wound up in the right places. For example, you can't get uClibc to produce an "i686-ld" or "i686-readelf", you have to rename 'em after the fact. And a "cc -> gcc" symlink actually has four possible states, and "i686-cc -> gcc" isn't the one you want. Some darn _weird_ errors result if you just try to run and debug this; visually inspecting the bin directory of the cross compiler is much easier. As opposed to debugging gcc, where visual inspection tells you nothing and the useful thing to do is find a failing command line and run it with -v and under strace to see what it's actually _doing_.

So now Mark can set up the cron job and build nightly binaries, except that I have to get uClibc's nightly snapshots working again. (Every time I ignore them for a month, when I come back they don't build anymore until I've tweaked the config of a half-dozen platforms and come up with fresh patches. Might be different this time, but it seems unlikely.)

Oh I'd forgotten about that bug. Occasionally software suspend on 8.04 will catch some weird race condition having to do with a dentry that's active during the suspend, and will wind up turning every access to that dentry into a segfault. (Sometimes it does one of those non-fatal panics and tells you it's a null pointer dereference in the vfs, sometimes it just kills your process with segfault and no explanation.) I remember the first time I tried to explain source control to Fade, and show her mercurial, Python spontaneously developed this affliction to screw up my demo. Now kaffeine's got it, so I can't catch up on my podcasts until I reboot. Sigh.

Hundreds of open browser tabs, unlikely to get 'em closed this week. Rebooting with the network off means they'll all be 404 and I won't be able to read cached data offline until I reload it. Rebooting with the net on will reload them all, meaning Konqueror will eat 100% of the CPU because tabs that aren't selected still run their javascript (which is a bad thing, but seems to be intentional).

And the dying thing is /usr/lib/libXdmcp.so.6 which is taking out amarok too, so I can't listen to mp3s either. Let's see, if I use Konqueror to navigate to an mp3 file and click on it... It tries to launch amarok. Idiot. It has built-in mp3 playing capability, and the flash plugin can also play mp3, but no, it wants to launch an external player. Brilliant.

Oh well, at least the stupid "Are we human, or are we dancers" song I was trying to drown out has gone off the radio now. (Because no human can dance you moronic whiny git.) Now it's gone to some other song by the same guy who did the original song Weird Al's "you're pitiful" is based on. It's a whiny falsetto theme on this station. The noise cancelling headphones, they do nothing.

The radio has now moved on to "Every time you go away, you take a piece of me with you". The frightening part is that this is is an enormous step _up_. (At the very least, I'm no longer contemplating surreptitious ways to damage the PA system here. And no, it's not the theme from the "Saw" movies. Don't think it is, anyway, despite the title I never saw 'em.)

Ooh, a Let's Play that Yahtzee did back before Zero Punctuation.

Watched the first disk of "Avatar, the Last Airbender" today. I previously thought it was an anime, but it's apparently a Nickelodeon production in a vaguely Japanese style (up to and including Japanese characters in the intro credits. I have no idea why. The practical effect of this is I can't turn the horrible english voices off and go to subtitles with Japanese voices that at least convey reasonable emotions whether or not I can understand a word. Nope, the english voice acting _is_ the original. I can only hope they improve as they get used to the characters...

I spent the first episode hoping the brother would die. The second episode _almost_ involved whatserface and the avatar guy leaving the rest of the cast behind (including the idiot brother) to go off on their own, but no. End of the disk, he's still there, which is sad. (I liked the voice actor when he was voicing Ron Stoppable, who was not a mysoginistic faux authority figure constantly blaming other people for his own screw-ups.) Still, Babylon 5, Star Trek TNG, and Star Trek the Motionless Picture (where "Nomad" has gone before) all started horribly and got better. The water bender girl and Aang's are both nice. And the bad guy's mentor is great.

Also watched the rest of the first Torchwood disk (episode 2, Death By Sex), the other commentary for Gosford Park (which is still way too complex for me to follow even half of it, and based on cultural details of a society farther from mine than the Yonomamo). And now I'm watching Bleach disk 3 (still episodes I've already seen back before Hulu stamped Watermarks all over the screen) which remains inexplicably Japanese in places, stopping for speeches about nature imagery in the middle of fight scenes.

Broke down and implemented a proper little setsid program in C, and made system-image.sh build it and call it. In theory I could have added this to toybox, but the build doesn't _depend_ on toybox being there. I could also have upgraded the busybox version to do this, and still might, but again the build shouldn't depend on the host setsid being buybox. (You can skip host-tools.sh.) And the host setsid on Ubuntu is forking a background process and returning immediately, which is useless for my purposes.

So yeah, spent a couple hours yesterday fighting with the shell in an attempt to avoid writing 9 lines of C code. (Because I shouldn't have NEEDED to, darn it.)

Ok, I haven't seriously poked at Firmware Linux in a week. Time to pick up the shovel again.

So where I left off is I rewrote buildall.sh to take advantage of the canadian cross stuff, and everything broke. The actual canadian cross part broke, but more fundamentally the old buildall functionality broke. One reason is that when system-image.sh dies, it tends to take down the script that ran it because it does a "kill 0" which takes out the current process group. Hence the first system-image.sh to fail stops buildall.sh from trying any of the later ones, and right how hw-uml is failing because user mode linux is still horked, as usual.

The way build.sh works around this is by running it via setsid (the "set session id" command explicitly creates a new process group), but needing to do that is icky. It would be nice if system-image.sh could do that internally somehow, I just need to figure out how.

I can't put system-image.sh's logic in a shell function and call that via setsid because setsid isn't a shell internal command, and a regular command can't do anything useful with the name of a shell function on its command line because it can't call _back_ into the shell that just forked and execed it as a child process. (I hit that one _all_the_time_.) I could do something incredibly ugly with "here" documents but the cure's worse than the disease there, especially with what it does to syntax highlighting, which admittedly has never worked in Ubuntu because Ubuntu hates vi for some reason and goes out of its way to break it; another reason to switch to Gentoo or something someday.

So my next attempt is to add this to the start of system-image.sh:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  export SYSTEM_IMAGE_SETSID=1
  exec setsid "$0" "$@"
fi

The reason that's unacceptable is now ctrl-c does nothing when system-image.sh is running. I took the exec out and added "exit $?" on the next line, but that made no difference.

The _odd_ thing is that when I call the sucker from build.sh via setsid, _that_ handles ctrl-c just fine. So what's the difference here? I tried 'setsid /bin/bash -i -- "$0" "$@"' and that confused the _heck_ out of the tty. (Ctrl-c gave me a shell prompt back, but didn't kill the running script.)

Tried calling setsid through the built-in "time" function, since that was the only difference I could see from what build.sh was doing vs this thing calling itself. No difference...

Ok, so I commented out the additions to system-image.sh and put the setsid back into build.sh and now _that_ isn't letting ctrl-c work anymore either. Which is strange because I'm pretty sure I tested that when it went in and it was providing the correct behavior. (I.E. kill system-image.sh and its children, but not the calling shell.)

I want "kill this process and all child processes, recursively". There should be a way to do that. (The gnu implementation of ps has --ppid, but the busybox ps is kind of a joke.)

Grrr. I need a way to call tcsetpgrp from the command line. Why isn't bash doing this when I call it from setsid? Because its process already is a session leader? Except that 'echo | setsid xargs -- /bin/bash blah' isn't interruptible either, and there bash is forced to be a child process of what setsid calls.

I'm using 'kill 0' because neither 'kill %1' nor 'kill `jobs -p`' work from signal context. It can't access environment variables either, so I can't save the PID and have it call that. I suppose I can reset the signal handler with an explicit list of PIDs to kill after every job I launch (make a function for it)... Except that _that_ doesn't work because killing the child process isn't recursive, it doesn't kill the sub-processes like "make" it was running (even though it wasn't a _background_ process for that shell, so you'd think the shell would do it, but no it depends on the terminal control group to propagate the signal).

Tried killing the pid of the background process as a negative number (which I believe should kill its process group), but it claims it isn't a process group. Tried running the background process through setsid to force it to _be_ a process group, but that didn't work either. (Hang on, does busybox know it's ok to do this? Does the bash builtin kill know? Trying $(which kill) and... no help. It's parsing the negative number as a signal number, so try "kill -11 -$(jobs -p)" and... it says no such pid, but with the negative number this time. Huh, I thought they'd added that support to the kernel years ago. (Try without the setsid on the child process... Nope.))

This _really_ should not be this hard. I really, really, really don't want to have to write C code for this.

Ok, spent another ten minutes installing an atexit handler in the outer instance of the script, sending a signal 11 to the child session running under setsid, catching that particular signal in there, and calling 'kill 0' if it gets it. Set up a little test where the child just waited 5 seconds, like so:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  echo Need to recurse
  export SYSTEM_IMAGE_SETSID=1
  setsid "$0" "$@" &
  trap "echo dying now, kill $(jobs -p); kill -11 $(jobs -p)" EXIT
  wait4background 0
  exit 0
else
  # If we exit before removing this handler, kill everything in the current
  # process group, which should take out backgrounded kernel make no matter
  # how many child processes it's spawned.
  trap "echo die lots now; kill 0 -QUIT; exit" 11

  echo do not recurse
  sleep 5
  echo done
  exit 0
fi

The above _WAITS_FOR_THE_SLEEP_TO_END_ before delivering the signal 11.

*BOGGLE* It sends the signal. And the shell doesn't process the signal until its child process exits. WHAT THE...?

Sigh. The reason this doesn't work:

if [ -z "$SYSTEM_IMAGE_SETSID" ]
then
  echo recurse now
  export SYSTEM_IMAGE_SETSID=1
  setsid /bin/bash -- "$0" "$@" &
  trap "kill $(jobs -p) 2>/dev/null" EXIT
  fg
  exit $?
fi

Is that it says "no job control" when it hits fg. Presumably because the shell isn't interactive. I can replace the last two lines of the if block with "wait4background 0" and "exit 0", but then I lose the error return code of the background process.

EPIC FLAIL!

(Well, actually as development sessions go for me, this is pretty tame. Still frustrating, though.)

Sigh. Domestic disputes, gotta love 'em. (You can stop reading now, nobody but me will care about this entry.)

My condo complex has designated parking spaces, and my neighbor's been parking over the line of their space for a while now. Annoying, but not a big deal. Today my neighbor came to yell at me because my car, parked entirely inside the lines of my space, was preventing them from pulling out.

I say "yell" because they did, and because they brought a friend with them, and claimed I'd had them towed twice (which was news to me; I asked Mark and he said he didn't tow anybody when he was here either).

I have towed cars that parked completely _in_ my space, but not for being over the line and not since last September. (It's an autumn ritual around here, as each new year starts at UT and the renters figure out which spaces are reserved and which aren't.) If somebody tows you without authorization, what you do is contact the towing company (only one services this garage, number's on the sign over the door), then you get the paperwork related to the tow which identifies who did it and has their signature. Then you sue them.

For once, the cell phone camera came in handy! (Ok, I had to borrow Fade's phone becuase my battery was low, and then I had to spend half an hour fiddling with bluetooth to figure out how to get the pictures off. But that's par for the course.) Here's the space they leave on the left, and the space they leave on my side. (This is after I moved my car to the right edge of my space, otherwise you couldn't see between 'em very well. I tried a close up, but the lighting makes it really hard to see. If you're looking for the line it's under the tires of the car on the left.) This was actually fairly mild "over the lines", he's been over by lots more, it's just this time he decided to demand that I move my car to give him more space. (Yeah, I know...)

The hilarious part was that they say they've been complaining about me to "Mrs. Thompson" for a while now, who I assume is the person they rent from. (I own my condo, they rent.) I took the pictures so I could send 'em to the condo association if it comes up again. (Which is McNeil property management, still no "Mrs. Thompson").

I suppose once somebody did a dance mix cover of "The Time Warp" it was inevitable that a music video made entirely out of clips from Dr. Who would be set to it and put on Youtube. This is the internet after all.

That's not even a particularly remarkable one. The weird science one makes better use of the Dr. Who material and matching it to the music, as do this one, and this one. There's often significant variation on the same songs, with straight takes and and ironic takes. And some were just extended versions of what wound up in the show, although you sometimes realize how carefully cut the original song was for broadcast TV. (Yes, the part of the song that wound up on TV was explicitly advocating murder, but didn't mention _sex_.) Bumped into some songs I'd totally forgotten, and I've even found some new music doing this.

Of course Dr. Who isn't the only show with with music videos. (Although Harley/Ivy is almost cannon, as is Bonnie Tyler/Hiro Nakamura really. (Although innocently looking for that brings up unwanted Luke Ski stuff, a bit like Penguicon really. Yes, I've had more than one panel scheduled next door to his "concerts", and after six years his material gets old. Nice guy in person.)

Saw "Monsters vs Aliens" today, which was entertaining but made me appreciate Coraline more. Coraline was _elegant_ in its use of 3D, when the tunnel expanded into the distance it was _supposed_ to be otherworldly, when the hands of the "piano that plays you" stepped out they _were_ intruding on coraline's space and being meanacing in a foreshadowing way. The needle coming out at the start and poking the audience in the eyes was supposed to be unsettling, as was the rest of that scene. But it would all work in 2D as well.

Today's movie wielded its 3D like a blunt instrument, and a lot of them would just be silly in 2D. Making a paddle ball fill the screen near the start of the movie was just pointless showboating. Monsters spent a lot of time using 3D very intentionally. Coraline simply _was_ 3D, and made it look effortless and natural.

Some of it's technical stuff. Several times Monsters had a bright 3D object on a dark background, and a halo filtered through the polarized lenses. I never noticed that _once_ with Coraline. (I'd also heard that one of the failure modes of 3D is that rapidly shifting depth of focus for an hour gives you eyestrain. Never came up in Coraline, happened a lot in Monsters.) A movie should makes me _notice_ the technical tricks it's doing, even when it's doing them reasonably well, because it defeats the purpose. Use the tricks to tell a story, not to show off the tricks.

I remember one of walt Disney's famous quotes was "Change the trick". This came up in an old a making of Who Framed Roger Rabbit documentary I watched, where they showed around 10 seconds of footage of a human actor riding in cartoon taxi (chase scene), and in one shot the human was on a crane with the buggy painted over it, in the next he was on a miniature vehicle with the buggy painted over it, and in the third both the buggy and the human were animated (but they were going down a dark alley so it was harder to notice the human wasn't the actor; and they still used his voice). All in the space of ten seconds. The point was so that you _couldn't_ figure out what they were doing because they kept doing different things. (See a man in a rubber suit, look again and it's now a puppet, look again and it's audioanimatronic...)

CGI eliminates a lot of this from a technical perspective, but not the need for it from a storytelling perspective.

By the way, either Ginormica's grandfathers are The Doctor and Death, or the name "Susan" attracts superpowers. Somebody should ask The Invisible Woman about this.

Poked at the canadian cross toolchain (translation: hit it with a rock) and made it work. Now I'm ripping build.sh a new one to add static cross compiler building to that.

So the problem with the canadian cross toolchains is that the pathing is horked. AGAIN. Yet another thing for the wrapper to EXPLICITLY WHACK WITH A HAMMER. Except that I probably need to patch the source to _not_ let it look at the wrong areas _first_ before falling back to $PATH. (And when it's _not_ canadian crossed, the path logic works DIFFERENTLY. Or maybe it's some targets.) Other than that, the sucker seems to work. Or at least it builds a hello world that runs under qemu.

(Ok, technically it doesn't run under qemu-mips, but that's because the current mips application emulation hangs attempting to run a hello world statically linked against uClibc. But that's not the toolchain's fault, and it runs fine under qemu system emulation just fine.)

April 15, 2009

I'd hoped to drop off Mark at the airport _before_ rush hour started, but it starts at 6am and his plane didn't leave until almost 8, so it was either deal with rush hour traffic or get him to the airport 3 hours early. Got home and slept for about two hours before Fade got up, then the phone rang with some guy who was unclear about whether or not he wanted to offer me a job, who said he'd sent me an email (and was calling to tell me he'd sent an email), but no actual email ever arrived. Email about existing work did though, so I dealt with that before going back to bed. It was noonish by that point so I didn't really sleep well, and wound up getting up earlier than I really wanted and being vaguely unproductive until after dinner. (I can work on a day schedule, or on a night schedule, but some variant of consistency does help. My internal clock flashing 12:00 leaves me a bit zombie-ish to focus on technical work. You can bury the problem with enough caffeine, but that's not a long-term solution.)

I'm so glad I got my taxes done last month. Today would really suck otherwise.

I need to re-rip Billy Joel's greatest hits, track down my genesis CD with "land of confusion" on it, and a bunch of other stuff.

"If and when the president does make his Afghanistan announcement, you can be assured that we here at The Rachel Maddow Show will cover it with an almost ridiculous, over the top, obsessive interest in the details. I'm sorry, I can't stop myself." - Rachel Maddow, 3/26/09.

I want to have Rachel Maddow's baby. I'm aware that there are a number of reasons this isn't likely: I'm male, I've never actually met her, I'm married to somebody else, Rachel's gay and in a stable relationship with a woman in the northeast somewhere... Little things like that...

April 14, 2009

Went in to the mortgage guy's office down off of Bee Caves to sign lots of paperwork for my sister's house. Hopefully, that's the last of it until closing. (Between paying taxes, funding IRAs, and the down payment and closing costs for my sister's house, I seem to have used up most of my ready cash, _and_ acquired a new $615/month bill. Oh well.)

Fade sounded distinctly forlorn when I called home around dinnertime, so I felt compelled to go make her less forlorn. (This involved sandwich, fruit, three different types of caffeine, and a cookie. She seems less forlorn now.)

More fiddling with transcoding software, banging on qemu summaries, and back to working on the canadian cross stuff too.

Gotta take Mark to the airport at 4am, so he can go visit Eric and Cathy for two weeks.

April 13, 2009

Went to vist the investment guy this morning to actually do something with the Roth IRAs Fade and I set up last week. The S&P 500 fund Wells Fargo has eats 5.75% of your investment up front, right off the top, as a management feet. (Ouch.) I wanted to buy a share of Berkshire Hathaway Class B stock (currently around $3k) and put the rest in the S&P 500 fund, but I wound up putting it all in the fund because to do that they'd have to transfer the money into a self-managed account which would take at least a day meaning I'd have to come back _again_. (Yeah, I know this is how they make their money.)

Fade went with the managed fund the guy recommended.

April 12, 2009

Ok, my working style's gotten some pretty deep groves in it. I just got to a good stopping point with the rebuild.sh and indexsections.py files for Qemu Weekly News (hey, I'm up to the end of January, 2008! Woot!). And I went, "ok, time to check these into source control"... and I haven't got source control for 'em. (Well they don't develop, except for those two pieces of infrstructure, one of which is from the old kernel docs I spent half of 2007 poking at, each data file gets finished and should never change again after that.)

Seems silly to set up a mercurial instance for two files, and/or to check in lots of static files.

Fade's been busy with online appointments for 6 hours so far today. (This is fairly normal.)

My attempt to come up with a paper for the Linux Plumber's Conference hit the snag that the scope of everything I tried to write was just too big. Even my _outlines_ spanned multiple pages, and just kept getting bigger. (I suppose I could try again tomorrow, but mostly I want to sit down and spend a couple _weeks_ writing documentation, not a couple hours. So many tangents...)

Spent rather a lot of the day doing qemu mailing list summaries instead. Lots of funky editorial decisions there, but they're mostly "should this thread be included or not, and how much". The scope of each week is "threads starting between this date and this date", and from there it's a question of _removing_ stuff, not adding it...

For example, currently I'm trying to figure out if this thread is worth covering. Four posts between two people, riffing off a LWN thread that already implemented something, wondering if qemu could do a better job and deciding "no". Long term result for qemu: none really. Is it likely to come up again? Dunno, probably not. Is it good background material explaining how qemu wound up the way it did? Well, sort of, but not very.

Or how about these two, where somebody asks a very good question... and never gets an answer.

Yeah, all this is totally subjective, but most editorial jobs are.

April 11, 2009

Good to know Moore's Law continues apace. Also nice to see idiots punished.

April 10, 2009

April 9, 2009

Of course this wouldn't have any application to brainwashing at all. (SO glad the shrub administration's worked its way through the guts of the nation.)

April 8, 2009

At home, awaiting the cable guy.

Ooh, I accidentally hit "i" in Kaffeine (I thought I the vi window had focus), and it turned "deinterlace" on, which is actually something I'd been trying to figure out how to do. (Otherwise the bleach dvd it's showing looks terrible whenever things move around on screen to fast.)

But why would deinterlace ever _not_ be on? (Isn't interlacing an artifact of CRT scanning which even modern LCD and plasma televisions need to compensate for?

Weird Al's Don't download this song remains a work of genius. (I admit I feel slightly guilty about listening to an mp3 I ripped from a legally purchased album instead of listening to the Youtube version or something, but I'm way too lazy to actually _investigate_ the file sharing networks...)

April 7, 2009

Distinctly under the weather today. Fade said she felt blah a couple days ago, now I've got it. (Very mild upset stomach, zero energy.) Not really feeling like doing anything specific, so I cleaned the kitchen. (Always a good option when nothing particularly seems worth doing at the moment.)

The cable modem's out. (There's that marvelous service Time Warner wants to charge extra for.) Somewhat inconvenient to look up time warner's number without internet access. I'm spoiled. Have to go dig up the phone book... And it's a "signal issue". A guy has to come look at it, some time tomorrow between 8 am and 9 pm.

Checked the other local wireless access points. I could associate with a couple but they route packets either. Dunno what that means. Heading down to Thundercloud, I haven't had breakfast yet anyway.

Removed mlocate and updated my "next time I reinstall" list. (Having a cron job start up and hog the disk when I resume my laptop is silly, especially for a service I never use.) I should probably poke at Xubuntu 9.04 when it comes out.

Hanging out at Epoch with Fade. Yay coffee shop internet...

April 6, 2009

April 5, 2009

This is what our new couch looks like, except for the actual couch part, and there are more cats, and the word balloon is merely implied.

April 4, 2009

April 3, 2009

April 2, 2009

It shipped.

Falling over now.

April 1, 2009

March 31, 2009

We have a couch. It is orange. I should take a picture of it.

Mark and I rented a u-haul, got the couch installed (a small phrase which contains a lot of incident), and then moved the rest of his stuff out of the old apartment into his new garage. (Most notably the washer and dryer.)

Now at Mark's pulling what's turning into another all-nighter, switching between working on the crosstool-ng patches Cisco wanted and getting FWL 0.9.6 ready for release. (I was hoping to have a FWL release out _before_ the end of the month, but as midnight approaches it's not looking likely.)

No I didn't pull two all-nighters in a row, I did an all-nigher sunday night, and now I'm pulling an all-nigher tuesday night. (Monday night was at home, Fade and I watched the rest of the Schoolhouse Rock DVD and )

It's a bit odd to be copying things I got working in FWL months ago into into a completely different open source toolchain project (which is also mostly a single person endeavor). The fact it took me about an hour to get FWL to produce such a toolchain and over a week to get crosstool-ng to do it doesn't enter into it, apparently. They want what they want, and are willing to pay to get what they want. I don't understand _why_ they want it, but it's their money...

I suspect I've been drinking too many energy drinks.

March 30, 2009

Very long day. Up overnight at Mark's last night, finishing up many things, then came home and slept a lot.

There's a release candidate set of binaries up at the new snapshots directory. We plan to start doing nightly builds, but it hasn't quite started yet.

March 29, 2009

It occurs to me that around half the eating I do is for social reasons. I went out to dinner with with Andrea at Whataburger last night (having just cooked and eaten chicken), I'm meeting Mark for dinner at Texadelphia (having just eaten leftover chicken and an apple), I regularly enjoy going out to restaurants with Fade, biking out after midnight the only places that are open are restaurants...

March 17, 2009

Biked to the north Chick-fil-a location last night. Adrienne called (locked out) while I was there. It's at least an hour and a half bike ride each way (closer to 2) so she drove up to get the key. I only got about 45 minutes of work done because they closed at 10, but yay exercise.

Thinking of biking to the south chick-fil-a location today. Made it as far as the Starbucks on 15th street.

Huh. So Dell responded to the Macbook Air by coming out with the Dell Adamo, a system which costs $200 more than the Macbook, has a slower processor, less powerful graphics chip, weighs a pound more, is slightly wider, /bin/bash: indent: command not found

I've been dropping "span" tags in my blog for something like a year now, meaning to write a cgi filter that'll actually use 'em, and for some reason today I keep doing "spam" tags instead. (Blame my email...)

Interesting. Time magazine estimates that if California legalized pot, it could balance their state budget by $2.3 billion/year. That's $1.3 in tax revenue and $1 billion saved in enforcement. (It's california's most lucrative agricultural crop, earning $14 billion/year.) Legalization would be personally inconvenient since I continue to be allergic to the stuff, but other than that it seems kind of obvious, possibly inevitable.

March 16, 2009

Dropped Adrienne off at SxSW this morning (she overslept; I simply hadn't set the alarm), then returned yet another fax to my sister's realtor, and now hanging out at the coffee shop next to Kinko's on 26th street. The barista is _stunningly_ cute. (Somewhat distracting, actually.)

Wendy at the credit union poked me about hazard insurance for Kris's new house, and I phoned and left a message with the Nationwide office on Far West.

Tested a new m68k tree from Greg Ungerer. Didn't fix the problem (uClibc build dies trying to generate the syscall list due to headers exported by make headers_install #including files it didn't install), but I got him the error message and hope for a reply.

Replied to an email about my tinycc fork (unrelated to resubscribing to the mailing list, this is a college professor at Tufts who says he and his postdoc wanted to hook my old fork up to a different backend. Could be interesting. (I still think qemu's tcg is the logical backend for the thing, but _any_ work to genericize it so the frontend and backend talk through some kind of defined interface can only help matters.)

Critiqued Pavel Machek's filesystem requirements documentation some more.

Wrote up a quick summary of last week's 440 work and sent it to Mark so he can do Impact Linux invoicing.

Heard back from the insurance guy, need to talk to an insurance guy in Minnesota. Right, called him, gave him the contact info for the mortgage lady so she can tell him what kind of insurance I need.

Tracking down how I broke the powerpc target, checking the revision history and building old versions until I find the one that doesn't work. (This would go faster if I didn't keep running out of disk space. Yeah, I managed to fill up the new bigger drive.)

I'd hoped to go to some of the free SxSW interactive parties tonight, but after being beaten to death by interrupts over the weekend I just haven't got the energy. Might go bike to someplace to huddle with my laptop...

March 15, 2009

Didn't get to update my blog yesterday. It's 8pm and I'm keeping Mark waiting while I stop at a restaurant to eat for the first time today.

Too overscheduled. Really in danger of burning out. I am now officially swap thrashing. I spend so much time being interrupted and context switching that I'm getting NOTHING DONE.

Adrienne's visiting and I have no time to spend with her. She's free for dinner tonight as soon as she finishes her nap; she came back from the conference exhausted. She's skipping the SxSW parties tonight, many of which I could go to if I had time because they're free. I really wanted to see the vintage coin-op video arcade they had (free to everybody, no badge required) at the convention center, but I never had time and it was only there for three days ending today.

Fade made it off to the airport early this morning. I didn't get to spend a lot of time with her this week, but she's only gone for a week.

Mark's wanted to meet me every evening for four days now (both because he needed help moving slightly _farther_ out of town, and because he needs help with a big documentation task Cisco assigned him with; Mark doesn't really do documentation). He's my friend and I want to spend time with him, and the technical stuff is even billable time, but with traffic it's an hour drive each way to his apartment. (Between errands, visiting Mark, driving Adrienne around, and driving Fade around, I've gone through an entire tank of gas this week. That's kind of unusual.)

I'd hoped Adrienne's visit would inspire me to clean the condo (which would have helped Fade's pre-trip stress too), and I did manage a couple hours, but I just didn't have time to finish it.

The mortgage on my sister's new house was approved, although the closing had to be rescheduled for sometime later this month (I forget when, but it's not this week and I think it's in my email). I need to go get some kind of house insurance ASAP (I forget what kind, but it's in my email). Still haven't manage to do so, maybe I'll find time tomorrow. (I need to call Kris and see how she's doing. Maybe I'll find time tomorrow.)

I've haven't had a chance to open the Lua book since the last time I wrote about it. Yes I know Gentoo From Scratch is blocked on a new useradd (because mark doesn't want to install an old version of shadow utils for some reason) and useradd is a toybox thing and toybox is blocked on me learning lua to rewrite it. I'd _love_ to have time to learn lua this week. It's very hard to do in 15 minute increments. Maybe I'll find time tomorrow.

Spamalot was in town all this past week and I missed it. I missed David Schwimmer mocking Showgirls at the drafthouse. I missed the start of the drawing class at UT I wanted to take this semester. Lawrence Lessig is in town for SxSW, maybe I'll find time to meet him tomorrow.

I've made trivial progress tackling my todo list for the FWL 0.9.6 release I wanted to do tomorrow. Maybe I'll find time tomorrow.

Went out to Whataburger last night and carved out a couple hours to poke at the 440 toolchain, because I'll have to do a progress report about that tomorrow. (I'm calling this week half-time because even with poking at two different crosstool variants I can't say I managed more than 20 hours.) I duct taped something into a FWL target that _might_ look like 440 toolchain in the right light, and fired up "qemu-system-ppc -M bamboo", and found out that the 440 processor isn't _implemented_. It's just a stub, the bamboo board just emulates the system-on-chip peripherals and otherwise uses a 405 processor emulation. (I'm embarassed this took me until yesterday to discover, but I didn't have anything to _test_ before then.)

I'd love to go visit the doctor to get antibiotics for the sinus infection that's preventing me from sleeping for more than a couple hours at a time. I've meant to go ever since I got back from Pennsylvania. (Of course the paperwork for better health insurance never got filled out anyway.) Maybe I'll find time for both tomorrow.

There are so many other things that have fallen through the cracks. Solar asked me to cut new FWL binaries on friday, and Vladmir Dronnikov's wanted those for months. Haven't found time. I need to set up the nightly build cron job, which means I need to fix static linking, see "FWL todo list" above. It's just a simple binary search regression test, I could sit down and figure it out in an hour or so. (Might be when I fiddled with the wrapper, might be when I started stripping the libraries.) And the m68k guys sent me a link to an atari emulutor that can apparently run a m68k kernel (aranym.org) but I haven't looked at it yet, nor have I followed up on the m68k headers_install issue in 2.6.29. Maybe I'll find time tomorrow.

I need to poke Eric about the C++ paper (he says he emailed me a draft earlier this week, but I didn't get it, and he's not getting my email either; sounds like a local delivery routing issue on grelber we need to track down). I haven't been to Starbucks in weeks. I need to get some exercise, and a haircut. My current car insurance card expired on the 5th (I think they sent us a new one, but dunno where it is). I need to hire a lawyer to show that Stafford idiots that when they gave me a "failure to show proof of insurance" ticket I did, in fact, have insurance (the card fell under the passenger seat). I need to deliver those shirts to Reese. I haven't spoken to Stu in weeks, I need to resubmit the perl removal patches, I may _never_ get QEMU weekly news updated...

Sigh. I don't even REMEMBER most of the things I have to do, they're just piling up into a big wave. But Mark called, and I guess I've kept him waiting long enough. Off to do documentation...

Ok, got some documentation edited over at Mark's, took Adrienne to dinner around 1 AM at the 24 hour IHOP (after visiting a 24 hour wal-mart and walgreens trying to find a supplemental battery pack for her smart phone), going to bed in hopes of getting up early in the morning and tackling backlog. May just turn my phone _off_ all day to avoid interruptions.

March 13, 2009

Lua books (2): check. Roxette "Look Sharp" album (to which I learned C): ripped, converted to mp3, and shoehorned into rockbox (which has the most horrible user interface, but oh well). Soda refills (Einstein's this time): check.

Not a bad language, really. It's firmly in the whitespace agnostic camp, not even requiring semicolons (which to be honest, aren't usually needed in C either; the compiler tells you when you've dropped one). It uses "end" in place of closing curly brackets, and makes it mandatory instead of having magic single line contexts after if statements and such. 21 reserved keywords. -- starts a single line comment, --[[ and --]] are multi-line comments. Variables are created by assigning to them, and reading a nonexistent variable return "nil" (instead of throwing an exception, ala Python).

I love this Album. Yeah it's old, but it's great to program to. (Joyride's ok too.)

Ok, I'm to section 1.3 in "Programming in Lua" and I've hit the first place I want to smack the author: "Global variables do not need declarations. You simply assign a value to a global variable to create it... if your variable is going to have a short life, you should use a local variable." Ok, _HOW_? YOU'VE NEVER MENTIONED LOCAL VARIABLES BEFORE NOW. This section offers no way to distinguish between local and global variables. I'm GUESSING it's a scope issue, but that's just a guess. Throwing me a forward reference here would be very reassuring; the author did not do this.

Wow Rockbox really is a horrible music player. (It works fine, it's just a pain to pause and unpause when I get a soda refill. Plus putting music _into_ it requires an act of congress.) I miss the KDE one. After several months of daily use, I think I can confidently say that I hate everything about Gnome. (I still have my little orange "iPod Klepto" around here somewhere, but apparently it's not in my backpack. Presumably in the car. Bit fiddly to use my giant noise cancelling headphones with a device that's maybe 1/20th their size, but it's got the "big prominent pause button" thing down. Hopefully there's a way to get it to play things in order rather than random shuffle. I'll have to look into it.)

Dear Gnome console developers: putting the close button for the last tab right above to the up arrow for the scroll bar is DEEPLY STUPID. The Konsole people figured out not to do that _years_ ago, and I just closed 3 tabs I wanted to keep in a slightly misaligned attempt to scroll up. (This is unrelated to the above statement that I hate everything about Gnome, which predated me doing this.)

The original crosstool only _claims_ to have support for the 440. If you actually try to build it, the thing breaks trying to build the first package (gcc 3.3.6, which is _really_ strange because I _told_ it to build gcc 4.1.1, which is the latest version it has). Looked at its config and it _seems_ the magic is --with-cpu=440 for gcc. Dunno if I need to tell binutils something (it didn't try to build binutils before trying to build gcc 3.3.6... I don't get it either).

Broke down and resubscribed to the tcc mailing list a couple days back to ask if CVS was dead yet. No official response so far, but I replied to the one guy who commented with a big long rant about what tcc SHOULD be doing. (Do the busybox swiss-army-executable thing and replace binutils, grab TCG from QEMU and use it as your code generator, clean up the path mess, try to build real packages... The usual. The kind of things I was working on when I left. Progress made so far: zero. Oh well.) They'll probably say CVS isn't dead, and then I'll go away for 6 months again. It's a bit like groundhogs day: I pop my head up, see CVS, and go away for 6 more months.

March 12, 2009