It seems to be in the air. Sam Ruby plants a stake in the ground, and isn’t too shy to point at his track record. My turn.

Right now is a good time for futures hog-calling; there are lots of changes that we’re in the middle of, where it’s hard to see what’s on the other side.

Item: Millions of people are directing online energy away from the wide-open Web into walled gardens like Facebook. At what seems like every social gathering I’ve attended in months, there’s a conversation that ends up with the FBers evangelizing the service and bugging others to get on. Some of the arguments make my blood run cold: “It’s like the Net, but safe because you know who everyone is.” I don’t think so. But still, I’m not immune to Facebook’s charms.

Item: Computers keep on getting wider. We just shipped this silicon morsel that’s really wide, and our rude boys are shouting to the world is just the fastest ever: shout shout shout.

Meanwhile, geeks everywhere are thinking Erlang. Yep, that book’s sitting here on my desk too, and since I used Webserver logfile analysis as my Beautiful Code testbed, I’m thinking of seeing how fast ten thousand Erlang processes can crunch my logs, compared to linear Ruby or Python.

Item: The WS-* project is imploding amusingly and while we can all stand in a circle and chant “REST! REST! REST!” as long as we want, the tools and best-practices landscape is distinguished mostly by its emptiness. What’s the RESTful future look like? Beats me.

Item: Nobody I know, who switches from mainstream static languages to Ruby or Python or equivalent, wants to go back. Yet as of today, the dynamic-language practitioners, compared to the Java/.NET mainstream, are off to the right of the decimal point, if you count heads. It’s nice to be the fastest-growing member of the ecosystem, but you still get the feeling that some big horny hoof could descend from on high and squash us like bugs, and nobody would notice.

Item: It is now within our power to equip every significant locus of digital creativity with a “Publish” button. So, what happens then?

Sam’s List · Surprisingly, while I would have agreed with probably four out his five futuristic bullet points, last time, I think I’m down to three at best on this one.

REST, well, yeah; I don’t invest three years of my life unless I’m pretty bought in.

Hadoop? Halfway there. If by Hadoop you mean “Large-scale functional-programming infrastructure that doesn’t rely on conventional database thinking, as first seen in Google’s Map/Reduce”, I’m down with that. But Hadoop itself? We still don’t know, it’s really the first attempt outside Google, and I’d be entirely unsurprised if it turns out to have some sort of horrible flaw that shows up under Internet-scale loads. I still think the direction’s important.

Erlang? I just don’t think so. Once again, if we’re really talking symbolically, saying its message-passing, lightweight-threading, functional-programming, live-patching virtues are important, well, then, yes. But Erlang itself? It’s too weird, and in my brief experiments, the implementation shows its age; we have in fact learned some things about software since way back then. And anyhow, I worry less about concurrency these days. The right way get the most mileage out of something like our T2 is load it up with a bunch of process-granular PHP or Rails or Django jobs. Which burns memory, but who cares? Or alternatively, to run something like Java EE, stay away from application-level threads, and let the JVM sweat the concurrency.

Jabber? Yep. Wire open a connection to wherever and send one start tag followed by an infinite number of XML chunklets. I seem to stumble across another good application for this every month or so, on average.

Microformats? Color me unconvinced. It’s like RDF and the SemWeb; sounds great in theory, but I’ve been hearing the theory for years and I’m still not using any applications. I could be convinced; but only by a killer app.

My List · A few of these are very specific. Hmm... risky.

  • Green infrastructure, both for public-spirited reasons and because the juice is pricey and we can’t swing the HVAC. (This may be bigger than the rest put together.)

  • The Atom Protocol, and REST.

  • Ruby and Python. The Web frameworks will lead them there, and they’ll get a taste, and they won’t come back.

  • AJAX, by which I mean the browser getting smarter and better and quicker and richer, incrementally month by month, until the increments become qualitative change. Which happened two or three times already in the last decade.

  • Jabber, like Sam said.

  • Ubiquitous functional programming. But I’m far from convinced we’ve seen the actual instantiation that gets us there from here.

Self-Indulgent Postscript · Writing this kind of thing makes me nervous, because last month Jeff Atwood asked Yes, But What Have You *Done*? and I don’t wanna be an airhead pundit. And it’s years since I spent a whole week doing nothing but code. And it’s worse than that; not only am a bloviating commentator, I’m also intermittently a standards iron-butt, writing rules from on high, the things that turn respectable geeks into morons and assholes.

Whatever... I do find time to code, even if it’s not getting Internet-scale deployment. And I’m an OK coder, but I’ve worked with a few of the great ones and more than a few who weren’t great but still better than me, so my engineering ego is well under control. And I’m a decent writer, but I doubt I’ll ever achieve my childhood dream of publishing a slim book of verse, or anything at all in a good serious magazine. I think I’ve basically lucked into following Scott Adams’ career advice.

At the end of the day, I think Atwood is overly harsh; the world needs loudmouths (at least on a part-time basis) as well as engineers. Having said that, he’s got a damn fine blog, for example Measuring Font Legibility, which is entirely irresistible to a typography geek named Bray.



Contributions

Comment feed for ongoing:Comments feed

From: Boudewijn Rempt (Aug 14 2007, at 00:07)

Well... You don't know me, but once mentioned the application I'm the maintainer of (the raster image app Krita). I went from Python to C++ without many regrets.

I was really into Python, even wrote a book on it, still use it sometimes. But it just didn't work for the application I wanted to write -- a natural painting application like Corel Painter, only more natural. So, when looking around for some existing project that suited my purpose, I found Krita. I learned enough C++ from the then-maintainer to be productive in a few months. Looking back, I find no difference in my productivity (measured in features done) between my large-ish Python application and Krita.

Of course, a large amount of that may be due to the Qt library that offers just about everything I ever used from either Python's or Java's standard library.

[link]

From: Danny (Aug 14 2007, at 01:32)

I'll be passing your comment re. SemWeb onto SWEO, but here's a quick response:

It doesn't make much sense to talk of the killer app of the Semantic Web, any more more than it does to talk about the killer app of the Web*. The point is the whole ecosystem that can be supported on a Web of Data. (And in answer to the point that a Web of Data is possible without RDF, I'd say yes, and syndication is possible without Atom/AtomPub, but similarly there are good reasons for choosing one route over another).

"I’ve been hearing the theory for years" - well yes, but with all due respect, to make progress in directions like this it'll take people listening and applying it.

Anyway slowly but surely things are continuing to develop, recently most notably recently being around the Open Linking Data project http://esw.w3.org/topic/SweoIG/TaskForces/CommunityProjects/LinkingOpenData (scroll down for a cool diagram).

* one possible answer to the killer app of the web is "the browser". To which I'll respond - Firefox uses RDF. Ok, it uses it in an old & clunky fashion, and is likely to lose it. But the fact remains, lots of people use it every day. While embedding SQLite may make a lot of apps a lot easier, long term the key part is in how the agent interfaces with the web. SQL isn't resource-oriented. Personally I anticipate a new RDF-like layer appearing within FF a year or two down the road.

[link]

From: Thomas Lindgren (Aug 14 2007, at 02:26)

"But Erlang itself? It’s too weird, and in my brief experiments, the implementation shows its age; we have in fact learned some things about software since way back then."

Interesting, could you say a bit more?

[link]

From: Jesper (Aug 14 2007, at 04:21)

I think Microformats are infinitely more likely to succeed because it doesn't rely on introducing an entirely new format. How many people understand RDF? Or, excluding formats altogether:

If you put some specific class names around parts of your address, you browser can present an "add to address book" button. if you put an inline element with a specific class around a measurement of temperature or currency or height or weight, your browser can convert it to your unit of choice.

People immediately see the benefits and it's a small, tiny investment. With the Semantic Web, it's more like "if everyone, absolutely everyone, maintains this metadata meticulously and nobody lies or withdraws information, we're closer to being able to build a search engine where you can say 'give me the addresses to the clubs my friend goes to'".

Even if Microformats *don't* take off, that at least explains why I think they're way more likely to do so than is the Capital Letter Semantic Web

[link]

From: Danny (Aug 14 2007, at 04:46)

"...even if it’s not getting Internet-scale deployment..."

- if the thing offers a half-decent HTTP interface and there's at least *one* deployment, it has Web-scale deployment - any client can interact. The code you mention here fares pretty well on this measure ;-)

[link]

From: Alex Miller (Aug 14 2007, at 07:23)

Interesting as always Tim! I posted some comments at my blog about Terracotta's approach being another direction in between Java EE scaling and something like Hadoop.

[link]

From: Steve Jenson (Aug 14 2007, at 10:33)

I agree with you that Erlang is 'too weird' to really win a mainstream audience. I spent most of my spare time last year programming in it, writing experiments to see if OTP and mnesia would fit my needs.

Scala, OTOH, is object-functional, has message passing, has pattern matching, and while statically typed uses type inference which keeps types mostly out of your way. I think Scala has a real chance.

Also, Scala is built on top of the JVM which means that I already know how to tune it to handle millions of requests.

[link]

From: Danny (Aug 14 2007, at 11:23)

Jesper, RDF is a data model, the format is secondary, e.g. microformats can be interpreted as RDF.

To get maximum benefit from Semantic Web technologies, sure, widespread deployment helps because of the network effect. Fortunately there's already a significant amount of Semantic Web-compatible data already out there, because it's an extension of the existing Web (think URIs). There are a lot of existing data sources that can be exposed on the Web relatively easily (e.g. http://tinyurl.com/27cah6 )

But your line "if everyone..." is way off the mark, accuracy issues about Semantic Web data are no different than with any other data representation (though having a description language like RDF can help if you want to determine the trustworthiness of data). Take your microformats example - what's to stop anyone lying about their address?

The Semantic Web is about getting interoperable data on the Web, and search is only one thing you can do with data and/or the Web. Think about the things you do with data already. Now think about the Web.

[link]

From: Doug Cutting (Aug 14 2007, at 11:42)

I think you're putting too much stake in functional programming as the key to large-scale, data-intensive distributed computing. Yes, MapReduce exploits some functional programming ideas, but it exploits these as a means to reliably process data at transfer rate, which is required for scalable performance, since seeks are getting relatively slower. Just as object oriented programming is no panacea, neither is functional programming. They're tools that should be used when they add value and not otherwise forced on folks. Most MapReduce programs are only functional at the highest level. Not forcing them to be functional everywhere when they don't need to be is a pragmatic, reasonable approach.

[link]

From: Ian Bicking (Aug 14 2007, at 14:00)

I hope if you try the Erlang processing, you also try a multiprocess form of Ruby or Python. Neither is as adept at processes as Erlang, but they are so much more adept at everything else that it probably makes up for it in terms of implementation time. As for processing speed... well, that's where it gets interesting, no?

I can't speak to what you'd use on Ruby, but on Python you might want to look at the processing module (http://pypi.python.org/pypi/processing/), PyLinda, or the new IPython stuff (http://ipython.scipy.org/moin/Parallel_Computing)

[link]

From: Jeff Carroll (Aug 14 2007, at 14:01)

Why are users of dynamic languages so far in the minority? I'd say that it's because users of scripting languages are by nature well-acquainted with habits that have to be taught under the guise of "extreme programming" and "agile methodologies" to people who use statically compiled languages.

Hence we're more productive, and consequently few of the projects we work on don't get massively hosed to the point that hordes of architects and programmers have to be hired to fix them. In turn, people don't seek out (or promote) skills in dynamic languages, because they're not buzzword-intensive and not perceived to pay as well.

[link]

From: Jesper (Aug 14 2007, at 16:31)

Danny: Understood. I meant no disrepect towards RDF - there's no doubt in my mind that it is well-engineered and well thought-out. However, the fundamental difference between RDF and microformat is that RDF is a new format (because it's going to be serialized into a format) and microformats are just new ways of using HTML.

There's a divide stretching from "this I know" to "this I don't". You have to be able to put things into relations in ways that are canny to RDF processing. The alternative is to put tags (or equip existing tags with class names) around your name and address. Which feels like the heavier task for those that know only HTML?

I did use the infamous "great big relationship search engine" as an example. You know why I did it? Because it's been touted as *the* application of the Semantic Web for ages. This also brings to mind the issue of accuracy as you said. Microformats accuracy is ideally as big of an issue as RDF accuracy. It's a smaller issue right now because they're on fewer sites and mostly put there by enthusiasts; not unlike RDF a few years ago.

However, in the great big search engine, accuracy is going to be an enormous issue. And what you have to ask yourself as an RDF supporter upset at my interpretation is: who promoted that this was going to make that search engine possible? Other RDF supporters. Constantly; beyond the basic structure of RDF, it's the single thing I've remembered hearing about RDF the most.

I admit that I may be wrong in really assessing how widespread RDF is today and what its goals are. But I'm sticking to my premise that microformats by and large are going to be more successful at accomplishing RDF's goals than is RDF itself (even if there's a level of overlap between RDF and microformat, as you note). That doesn't spell that I'm rooting against RDF, or that RDF isn't also going to be successful, but I can almost guarantee that if microformats aren't successful, RDF in its current guise isn't going to be successful either.

[link]

From: Aristotle Pagaltzis (Aug 14 2007, at 18:11)

Jesper,

you continue to misunderstand. RDF is not a format *at all*. It is a data model. You can process microformats as RDF. You can process pretty well anything on the web as RDF.

Your “if everyone, absolutely everyone, maintains this metadata meticulously” is utterly off the mark; there is no reason whatsoever that you couldn’t inject “a bit of metadata” into existing data in order to enhance its useful *exactly the same way you’d do with microformats*. I literally mean *exactly* the same way.

Check out GRDDL and RDFa someday.

Likewise, the “if nobody lies or withdraws information” chestnut is hogwash. People already do that; this very comment box is protected by a faux-CAPTCHA because of spammers. Somehow we deal with that. The issues differ not a wit when it comes to the semantic web.

This is not meant as a put-down, so please don’t take it as such: you’re repeating common misconceptions uttered by people who don’t know much about RDF. If you want to speak intelligently about it, please do at least some research beyond reading 20,000 foot view intros (I have yet to find one that wasn’t complete garbage), otherwise you risk embarrassment that you won’t even notice.

[link]

From: Jerome Lacoste (Aug 15 2007, at 00:41)

First thank for your chapter in the book, I've found it enjoyable to read.

As for micro-formats, I like them for what they are: standards that ease integration and code reuse.

Example (shameless plug): a little greasemonkey experiment where I combine my script with an existing one to allow integration of a realestate site with google calendar:

http://coffeebreaks.org/blogs/?p=77

I could have avoided to use micro-formats and used a single GM script, but I've found the 2 scripts solution to be more "beautiful". So in that regard, micro-formats were useful to me.

[link]

From: Morgan Collett (Aug 15 2007, at 04:02)

We're building the OLPC Sugar collaboration features on top of Jabber (XMPP) using Telepathy (http://telepathy.freedesktop.org/).

The spec is still being bedded down, but we've got application data channels called Tubes - D-Bus over XMPP. A taste of this is documented at http://wiki.laptop.org/go/Shared_Sugar_Activities

Collaborate on anything, with anyone in your buddy list. Watch this space.

[link]

author · Dad
colophon · rights
picture of the day
August 13, 2007
· Technology (89 more)

By .

The opinions expressed here
are my own, and no other party
necessarily agrees with them.

A full disclosure of my
professional interests is
on the author page.

I’m on Mastodon!