New Challenges Require New Solutions

SR-71 cockpit diagram

Guy Steele talks about language design in the January Doctor Dobb's.

I'm not saying we should throw structured programming out the window. I am saying that the trade-offs have changed and are likely to keep changing.

Anti-UAV UAVs

LOITER MODE—Using its long wings and a diesel prop engine, Peregrine will patrol an aerial grid looking for enemy UAVs

Noah Schactman and Dave Noland repond to a DARPA “request for proposals to contractors, asking for a dual-propulsion Peregrine UAV Killer drone to take down unmanned hostiles.”

Fresh CLiki Grease

The other day someone on #lisp was having issues editing a CLiki page, which led to my discovering that my Greasemonkey script that fixes CLiki's HTML entity handling, sane-cliki.user.js, wasn't working in Firefox 1.0. I've fixed the script (I should have listened to timb back when he first pointed out the problem) and it will let you enter your IRC quotes without any additional hassle.

I would have modified the script to support the ALU's wiki, which uses kiwi underneath and has the same HTML-handling bugs, but I couldn't figure out how to ask kiwi for a page's raw wiki source; the CLiki method of adding “?source” to the page URL doesn't work.

Given Daniel Barlow's other time commitments, I wonder what kind of future CLiki has. The HTML handling was reported as a bug a year and a half ago. The ALU wiki looks pretty nice, and it seems to be a locus of CL Gardener activity, but it has some serious bugs and I haven't seen much development there lately (and the source isn't yet available).

Grand Challenge, Urban Edition

DARPA wants feedback from anyone who was involved in or even attended a Grand Challenge event. It still isn't clear whether there would be any more Grand Challenges, but the questionnaire says that they're “soliciting information from the Grand Challenge community to improve the operation of similar events in the future,” and that “DARPA is considering a future Grand Challenge event centering on autonomous vehicle operations in an urban area.”

ASDF-INSTALL-TESTER

Gary King is doing the hard work of fleshing out my experiments with ASDF-INSTALLation testing.

Qwantz on Robots

Click on (or point or chord or speak or wheeze or whatever your preferred gesture is) the comic to see the full version [via Lori].

(How about I change “a mostly Lisp weblog” to “a canonical study in failed promise”?)

Rebels Without a Claus

My pal Rico's SantaCon story aired on public radio's Marketplace tonight.

CL-ZEROCONF 0.2

One of the ASDF-INSTALLable libraries that failed to install on OpenMCL was my own CL-ZEROCONF. It worked in OpenMCL 0.14.something and didn't work in OpenMCL 1.0, so I fixed that and a few other things.

It no longer works with SBCL, or at least newer versions of SBCL, but maybe once foreign callback support is added I can revisit this.

Can I Get a Ho

Lori and I contributed our santalaciousness to the coincidental gathering of hundreds of santas in LA this weekend. After breakfast at House of Pies (scene of the infamous Kirstie Alley backseat piejob photo that hit the tabloids last year) we marched directly to the Frolic Room, and then it was one bar after another with sex shops, bewildered onlookers, excited children, an art gallery, malls, strip joints, professional santas, wax museums, Japanese schoolgirls, Scientological Christmas displays, head shops, mechanical bulls and a million other things in between that could distract an easily distracted group of hopped up santas.

You'd think that would be enough for one day. But no, we had two parties to go to that night, including one boring office party. As often happens at these things, HR people used to having a margarita at Chili's once a month overindulge and begin acting inappropriately and stripping and rubbing and hugging. And of course Ron Jeremy was there. Duh, it's LA.

The next night the Museum of Jurassic Technology had their party. Steve Martin flaked this year, but Werner Herzog did strip down to his underwear and stroke his revolver.

Roomba Serial Interface API

iRobot has released a manual documenting the Roomba Serial Control Interface, along with instructions on how to determine whether your Roomba has the serial interface and how to upgrade it if it doesn't.

Here is a Python code fragment that illustrates this method (Device Detect is connected to the PC’s RTS line via a level shifter):

ser = serial.Serial(0, baudrate=19200, timeout=0.1)

ser.open()

# wake up robot
ser.setRTS (0)
time.sleep (0.1)
ser.setRTS (1)
time.sleep (2)

# pulse device-detect three times
for i in range (3):
  ser.setRTS (0)
  time.sleep (0.25)
  ser.setRTS (1)
  time.sleep (0.25)

Looks cool, and speaks of a respectful attitude on iRobot's part.

Battyani's Lisp Directory

Marc Battyani is trying to organize a new directory of Lisp resources, built on his Linkit engine, which is a Lisp copy of Reddit.

By the way, this URL scheme isn't just ugly, it blows:

Linkit front page: http://linkit.fractalconcept.com/asp/sCst/sdata/sdataQuEY-NQ=
Linkit lisp directory: http://linkit.fractalconcept.com/asp/sCst/sdata/wCstsdataQo5Y-1Mh9urk
Linkit hot links: http://linkit.fractalconcept.com/asp/sCst/sdata/uBstsdataQuEYnB==

I didn't click on anything that had side effects, I didn't even login. Might as well emit GUIDs.

Franz is a Tough Customer

Richard Fateman in comp.lang.lisp:

> I am just curious: did you or your company ever had to take
> action (by lawsuit) against a potential programmer when he was
> using ACL and making great profit out of it but never reported
> it to you.

Let me change your question to:

"Did Franz Inc ever take action (e.g. threaten a lawsuit)
against a company which was shipping ACL with its product, but
failed to properly report and pay royalties?"

Yes.

Comments are Back

An unfortunate Debian bug broke lemonodor for part of yesterday and today. Sorry for the hassle, especially if you tried to post a painstakingly crafted refutation of why ASDF-INSTALLable library quality matters and were rewarded with the following:

  Got an error: Unsupported driver MT::ObjectDriver::DBM: 
  DB_File needs compatible versions of libdb & db.h
    you have db.h version 4.3.29 and libdb version 4.3.28

My gracious admin Andreas has fixed the problem, so please try again.

ASDF-INSTALLable Library Status

Just for fun, I tried to ASDF-INSTALL all the ASDF-INSTALLable libraries listed on CLiki. I used OpenMCL 1.0, so obviously not every library was going to work. And I didn't start a new lisp for every installation attempt, so there could have been some interaction issues (and in fact, it looks like maybe some problems might have been caused by some library installing a reader macro). But the results were still interesting.

Of 202 ASDF-INSTALLable libraries on CLiki, 104 could not be installed.

19 failed due to HTTP 403 responses. This is because of a bug in ASDF-INSTALL whereby it sends an incorrect HTTP request (it does a GET http://domain/path instead of a GET /path).

26 failed due to HTTP 404s. 10 of these are due to Brian Mastenbrook's server, unmutual.info, being down. A few seem to be due to the module listing as a dependency some other module that is not listed anywhere on CLiki.

7 wouldn't install because of “missing components”. Most of these are modules that list another module that wouldn't install as a prerequisite.

13 were “no such package” errors. Most of these are packages that are non-portable and won't work in OpenMCL. Some are just simple mistakes (one module tries to use the "ASFD-INSTALL" package, for example).

One was an “unsupported lisp” error. One module (ZIP) explicitly detected that it wouldn't work in OpenMCL.

And finally, there were 38 other errors. These are primarily compilation errors, like “Unbound variable: CDR”. 9 might have been due to a freaky reader macro.

I was disappointed that over half the libraries wouldn't install. Those just aren't good odds. And in terms of perception, as well as actually getting work done, it doesn't matter what the reasons for the failures are.

Though, on the positive side, it did turn out that many of the problems are easily fixed—within a few minutes of my posting my findings to #lisp, Zach Beane had an ASDF-INSTALL patch that fixed all the HTTP 403 errors, and a couple other people had fixed minor bugs in their libraries.

The next step would be to automate this sort of testing for multiple Lisp implementations. This might be something the CL-janitors/CL-gardeners would be interested in. I can imagine a library validator that would check ASDF system definitions for all the helpful fields like :description and :maintainer, list the Lisp implementations in which the library cleanly installs, check for unit tests, etc. It might be a first cut at estimating the overall quality of a library. Ideally, eventually, such a tool might become less useful as every library is brought into compliance.

convert *.tiff out.pdf

Geese and clouds, taken by my old pal Eric.

I had twenty 2500x3300 TIFFs I wanted to turn into a single PDF, one image per page. I didn't want to load them one-by-one into a Word document. Trying the straightforward fink-installed ImageMagick command "convert *.tiff output.pdf", my 1.5 GB RAM machine immediately began swapping and 10 minutes later it still hadn't finished. I wrestled with Preview, but I couldn't figure out how to get it to print multiple images at once. I even looked at CL-PDF, but it didn't seem to support TIFF images.

I was reminded that Apple ships a Python CoreImage wrapper, with some example scripts that convert images to PDFs and turn collections of text documents into multi-page PDFs, so I combined a couple examples. The script took 9 seconds and gave me a beautiful PDF.

The final script, docs2pdf.py, should run on any later version of OS X, and will create a PDF from a sequence of text, HTML, RTF, .doc, and image files, one per page.

Bob Lost

Sometimes people post comments to this weblog asking what sort of work I do.

The diagram above is from a patent application related to the work I've been doing for Evolution for the past year and a half. LaneHawk is a vision-based system that uses that little camera (118a) to recognize what items are in the bottom of a shopping cart and ring it up at the cash register automatically. The idea is that a surprising number of people actually do forget to pay for what's down there, and cashiers forget to check. This results in what the grocery industry calls “bottom of basket loss,” or BOB loss.

Interestingly, BOB loss is cultural. Some American and European grocers have such a big problem with it that they remove the bottom racks completely. On the other hand, I've heard that in Japan it's basically a personal failure to leave a market with an unpaid-for item, therefore LaneHawk may not have too many customers over there.

Lanehawk is also a brand of bowling ball that includes an interesting line of novelty balls, but I'm not a huge bowler, and I have nothing to do with that.

LaneHawk is built on ViPR, which is the advanced vision system Evolution has developed as part of the Evolution Robotics Software Platform. ERSP itself (on which I used to work more directly) has been used most notably in the Sony Aibo robot dogs, but it gets used in quite a few prototype & research robots by some of the big robotics companies (the sort of robot that often show up in Yahoo News photos doing something goofy like wearing makeup or playing a guitar while a small person rides on its back).

Examples of ViPR working under a variety of conditions. The first column presents the two objects to be recognized. The other rows display recognition results.

It turns out, and I shouldn't be surprised, that there are companies that spam anyone whose name is on a patent application.

Dear Sir/Madam,

Your above invention is among the more than 80,000 new inventions which are published world-wide each month. That clearly exceeds 1 million patent applications annually.

Um. OK, that doesn't seem entirely obvious to me since 80,000 x 12 = 960,000, but whatever. I respectfully decline your offer, despite the “Current Introductory Price of US$ 0.00”

So that's what I do. That, and make transparently boasting posts to lemonodor so my dad will see that my name might be on a patent one of these days. Hi dad!

I Don't Even Wear a Bow-tie

“I want to paraphrase Jon Stewart and say to John Wiseman: Stop, stop, stop, stop hurting Lisp.”—Scott Reynen

Hi, Scott.

I figured I would send you a short note to give you some insight into my motivations, which I think it's possible you may have misjudged.

The reason that I posted a link a while back to your short review of that SLIME screencast is that I agreed with much of it. I love Lisp, and it took me months before I could actually finish that entire 55-minute video. And the bit at the end where he can't quit cleanly? That's kind of embarrassing.

I'm interested in how other people view Lisp, the things that lead them either to try it or to stay away from it, and the problems they have in using it. One reason I'm interested in those things is because I think there's lots of room for making Lisp better by fixing or minimizing the problems that even smart programmers run into when they consider using Lisp. And I selfishly want to benefit from those improvements myself.

“The idea that there is something better than Lisp is apparently inconceivable to some, judging from comments on the reddit blog. The Lispers instead quickly set about trying to find the real reason behind the switch. One assumed it must have been divine intervention, since 'there seems to be no other reason for switching to an inferior language.' Another figured something else must be going on: 'Could this be...a lie? To throw off competition? It's not as though Paul Graham hasn't hinted at this tactic in his essays...' Another chimed in: 'I decided it was a prank.' Another suggested the authors simply wanted more 'cut corners, hacks, and faked artisanship.'”—Aaron Swartz

I posted the stuff about Reddit because I thought that there may be something to learn by looking at the experience of a company that is pro-Lisp, is financed and mentored by a prominent Lisp-friendly figure, actually wrote version 1.0 of their code in Lisp, and still ran into trouble and rewrote their software in another language. Reddit's rewrite is no Lisp apocalypse, and it might not even turn out to be a very interesting event. But I think there's probably something to learn from their experience that can be used to improve Lisp. And Lisp can be improved.

“Despite my pride in my Lisp heritage, the time when I am closest to being ashamed of it is when I hear a Lisp advocate in the middle of a good rant.”—Tim Converse

I'm sorry you got a bunch of morons posting comments to your site. I don't think I'm responsible, though. It's true that I post snippets of things without much context. But those snippets are the things that my brain found interesting, and the idea is that you can click on the link and get the whole picture for yourself. I may emphasize, but I don't think I slant. In fact I'm pretty sure some people think of me as a Lisp zealot while others consider me a Lisp traitor.

And I guess I'm cool with that.

Reddit Chooses Python Over Lisp

Reddit's Lisp-based engine was completely rewritten. In Python.

Emacs and SLIME are a killer combination, but I develop on a Mac, and reddit.com is a FreeBSD box. On my Mac, my choices of threaded Lisp implementations was limited to OpenMCL, and in FreeBSD it's CMUCL. Because of the low-level socket and threading code we had to write, reddit would not run on my Mac, and I was always tethered to our FreeBSD development server. Not being able to program offline is a pain.

If Lisp is so great, why did we stop using it? One of the biggest issues was the lack of widely used and tested libraries. Sure, there is a CL library for basically any task, but there is rarely more than one, and often the libraries are not widely used or well documented. Since we're building a site largely by standing on the shoulders of others, this made things a little tougher. There just aren't as many shoulders on which to stand.

Emphasis mine.

HMPassphrase Anti-spam Plugin for Movable Type

Part of my recent Movable Type upgrade experience was discovering that the plugin that I previously used to protect lemonodor from spam comments had been crippled during its transition from third-party plugin to official, integrated-with-Movable-Type plugin. Spamlookup's passphrase feature, the only thing that kept me from having to waste any time at all on the 40 or so spam comment posts per hour that lemonodor gets, had disappeared.

None of Spamlookup's other anti-spam techniques are effective for me. For the past few days I've had to at least scan every comment as it came in to make sure it wasn't spam. Even that small amount of effort doesn't scale to several hundred spams per day, and I was getting cranky.

So I wrote a Movable Type plugin to do what I want, and everything is now humming along perfectly. I'm no longer cranky!

My plugin, HMPassphrase more or less replicates SpamLookup's old passphrase feature, though it offers the option to send comment submissions that don't have the passphrase to the new Movable Type Junk folder instead of just rejecting them completely (I think it's most useful to straight up reject them; Junk is just another thing you have to check manually).

I don't know Perl, but it only took a few hours to code using some existing plugins as a guide and with the #loafers always ready with expert advice.

Ruby As An Acceptable LISP

Eric Kidd writes the inevitable “Why Ruby is an acceptable LISP”, concluding that “Ruby is a denser functional language than Lisp” and “Ruby gives you about 80% of what you want from macros.” [via reddit, where 6 of 25 stories on this morning's front page are Lisp-related.]

The comparison is both narrow and shallow, but I think he makes some valid points and shouldn't be dismissed too quickly.

Now, given a choice between a powerful language, and popular language, it may make excellent sense to pick the powerful one. But if the difference in power is minor, being popular has all sorts of nice advantages. In 2005, I’d think long and hard before choosing LISP over Ruby. I’d probably only do it if I needed optimized code, or macros which acted as full-fledged compilers.

I don't find it implausible that for many people, doing certain kinds of work, Ruby is a better Lisp than Lisp. For others, it won't come close.

Rediscovering the State of the Art

491,000 year old ice core from Anarctica. “...the modern atmosphere is still highly anomalous. At no time in the past 650,000 years is there evidence for levels of carbon dioxide or methane significantly higher than values just before the Industrial Revolution.”

We're not so much building on the programming state of the art as continually have each generation of programmers rediscover it.

Avi Bryant:

As the stunning rise of Ruby and Rails makes clear, maybe there’s something to dynamic typing, metaprogramming, open classes and blocks after all. And so the Smalltalkers - especially, perhaps, ex-rubyists like me - are feeling a thrill of vindication once more. As a language, Ruby can be pretty accurately and completely described as a dialect of Smalltalk with Algol-inspired syntax and some scripting-friendly extensions.

[...]

So if Java is a victory for Smalltalk’s implementation choices, and Ruby is a victory for Smalltalk’s language choices, what do you do if you want both? There’s still only one option: use Smalltalk.

Online Lisp Books

Chimp head on a Roomba

Bill Bradford has collected links to various Lisp books online, some old, some new.

You Will Upgrade

I didn't do enough testing, shit broke, plugins failed, byte order bit me.

The end result was hundreds of spams per hour getting through the defenses. So I had to upgrade to the newest Movable Type, 3.2, which has integrated the previously independent Spamlookup plugin. Only, for my convenience, the one useful anti-spam measure, the passphrase checking, wasn't included.

WTF.

Anyway, it might all work now. But if you see something weird, better let me know.

Walrus EOL

walrus.boinkor.net, the server on which Andreas Fuchs generously hosts lemonodor.com, is going away. The site will be moving to a new server, during which there may be some downtime (either of the site as a whole, or just comments).