Amazon Payment System

I was looking at the Amazon Flexible Payment System service API and it's a whopping big set of docs. Unfortunately, I was sorely disappointed to see the operations in the HTTP based API (called a REST Request) seem to all be GET requests with an Action= query term. Ugh.
The documentation doesn't even mention what HTTP method to use, what content-type to submit or expect as a return, etc. I'm pretty sure that they simply don't know the difference. Sad but true. At least there are docs.

I suppose they didn't have the time to make it simple.

NASA Tech Briefs

A long time ago I used to subscribe to NASA Tech Briefs - a slim monthly magazine full of hard-core engineering articles and light scientific reporting.
A few months ago a friend had an issue and I decided to see what kind of online presence they have now - and they have a good one. No Atom feed, but still lots of techno-bits!

NASA Tech Briefs

Metaplace: open DIY virtual worlds

This I have to check out... it sounds very much like what I've wanted to build for the past ten years or so.

Metaplace: open DIY virtual worlds for everyone

Facebook - the end is near

Ah, yes. Facebook is on the decline already - when I logged in this week, I got a an ad for Zwinky. Animated, colorful, annoying. Completely not me. Except the annoying part.

CouchDB: Thinking beyond the RDBMS

From Assaf at labnotes,
CouchDB: Thinking beyond the RDBMS

It stores document in a flat space.

There are no schemas. But you do store (and retrieve) JSON objects. Cool kids rejoice.

And all this happens using Real REST (you know, the one with PUT, DELETE and no envelopes to hide stuff), so it doesn’t matter that CouchDB is implemented in Erlang. (In fact, Erlang is a feature)

Here’s where it gets interesting. There are no indexes. So your first option is knowing the name of the document you want to retrieve. The second is referencing it from another document. And remember, it’s JSON in/JSON out, with REST access all around, so relative URLs and you’re fine.

Hmm, Erlang. Hmm, no schemas. Hmm, no write consistency. Sounds perfect.

HTTP errors - a photoset on Flickr

(From BB)

HTTP - a protocol so simple, even a cartoonist can understand it. (and why haven't you bought a shirt yet?)

404 Arrgghh - (insert pirate code joke here)

SubAtomic future

I just had a thought - what if the Atom Publishing Protocol and Subversion had a mashup - we could call it SubAtomic!

Object serialization raises it's insidious head

Funny post from Dare Obasanjo decrying the horrible state of affairs created by the company he works for.

This part is about object serialization and version incompatibility.

However, the insidious thing is that the failure wasn’t because their application was improperly coded to fail if it saw a fruit it didn’t know, it was because the platform they built on was statically typed. Specifically, the Web Services platform automatically converted the XML to objects by looking at our WSDL file (i.e. the interface definition language which stated up front which types are returned by our service) . So this meant that any time new types were added to our service, our WSDL file would be updated and any application invoking our service which was built on a Web services platform that performed such XML<->object mapping and was statically typed would need to be recompiled. Yes, recompiled.

Insidious indeed! Who in their right mind could have predicted such things!

And this...

It’s sad that as an industry we built a technology on an eXtensible Markup Language (XML) and our first instinct was to make it as inflexible as technology that is two decades old which was never meant to scale to a global network like the World Wide Web.

Who's this industry of which you speak, Kemosabe?

Getting all meta

Dare Obasanjo seemingly always has time to create detailed and thoughtful posts, I've gone from being annoyed at early posts to reading them regularly.
His latest two posts are worth reading as well, but I can't help but comment on his strawman characterization of ReST.

I should probably start out by pointing out that the title of this post is a lie. By definition, RESTful protocols can not be truly SQL-like because they depend on Uniform Resource Identifiers (URIs aka URLs) for identifying resources. URIs on the Web are really just URLs and URLs are really just hierarchical paths to a particular resource similar to the paths on your local file system (e.g. /users/mark/bobapples, A:\Temp\car.jpeg). Fundamentally URIs identify a single resource or aset of resources. On the other hand, SQL is primarily about dealing with relational data which meansyou write queries that span multiple tables (i.e. resources). A syntax for addressing single resources (i.e. URLs/URIs) is fundamentally incompatible with a query language that operates over multiple resources. This was one ofthe primary reasons the W3C created XQuery even though we already had XPath.

That said, being able to perform sorting, filtering, and aggregate operations over a single set of resources via a URI is extremely useful and is a fundamental aspect of the Web today. As Sam Ruby points out in his blog post Etymology, a search results page is fundamentally RESTful even though its URI identifies a query as opposed to a specific resource or set of resources [although you could get meta and say it identifies the set of resources that meet your search criteria].

I don't think there's any harm in the mis-characterization of ReST and the detail of the post about the two query systems from Google and Microsoft is rather interesting, but I was surprised at his misunderstanding since I thought he knew better. Dare does have an out - as any good program manager does - by saying "... you could get meta...", so I will take that opportunity and get meta.

• "URIs on the Web are really just URLs and URLs are really just hierarchical paths to a particular resource [...]" - no, they are not really just hierarchical paths. They are an identifier. Sometimes they are a 'path', but usually not.
  
• "Fundamentally URIs identify a single resource or a set of resources." - always just a single resource.
  
• "A syntax for addressing single resources (i.e. URLs/URIs) is fundamentally incompatible with a query language that operates over multiple resources." - this is core missed direction in the explanation of ReST. The term 'resource' as applied within RDBMS isn't the same as 'resource' as applied within ReST. A ReST 'resource' is the view from the outside, and datbase 'resources' are the data elements on the inside. Read up on Data on the Outside vs. Data on the Inside by Pat Helland (not exactly relevant to this post, but a catchy title).
  
• "a search results page is fundamentally RESTful even though its URI identifies a query as opposed to a specific resource or set of resources [although you could get meta and say it identifies the set of resources that meet your search criteria]." - the URI actually does identify the results (as Dare suggests) and not the query. The URI is the query. This means you can operate on the set of data that are the results - you could delete them, retrieve them, replace them, etc.
  

I'm not suggesting that URI are better than or a replacement for queries, just that they are different. They do different things and are both very useful, and when you think in terms of "a URI is a query" you think of the processing of the implementation and miss out on some aspects of Web architecture which naturally arise when you think of "a URI identifies the results".

Ruby on Rails Job Trends

From the indeed.com job search site (which looks very nice by the way), the trend for Ruby on Rails mentions in job postings is on a steep growth curve. The absolute percentage is smaller than other terms, and there may be other factors that contribute to the trend but it's pretty telling.

But to keep things in perspective, compare Ruby on Rails with J2EE job postings

More happy fun job trends for Ajax, Web2.0, et al.

Branding ads and ad listings

I just saw this post on BoingBoing about this ars technica post on the psychology of banner ads which is interesting in light of Google's and Microsoft's recent acquisition of 'creative/banner ad' networks.

"The research concludes that repeated exposure to a product via banner ads generates a positive feeling towards that product. The good news for consumers is that a critical reevaluation of the product can make these positive feelings vanish." and
"This suggests that familiarity-based advertising may work best for impulse buys, where more detailed evaluations aren't likely to occur."

I found these two papers yesterday when reading up on economic theory and internet advertising. This one Internet Advertising and the Generalized Second-Price Auction - Selling Billions of Dollars Worth of Keywords is the most readable of the two and talks about the bidding process for online ad placement. They cover the history and describe the shift from "pay what you bid" to the current "pay the next lower bid" (called a 'generalized second price auction'). Really very interesting stuff.

The second paper Brand and Price Advertising in Online Markets looked at different fundamental forms of advertisements - brand advertising and 'price' advertising. I had high hopes for learning from it, but it was very dense with too much lingo specific to this research area for me to fully understand. Here's an example: In contrast to models where loyalty is exogenous, these crosschannel effects lead to a continuum of symmetric equilibria. Yeah, I, uh, was thinking the same thing.

Their models and assumptions also seem questionable and so I don't know that their results apply to the world we live in, as opposed to the simplified models they used to prove their theory. In any case, the questions being asked and the attempt to find answers are still valuable.

Here are their findings (and again, these may not really apply to the real world)

While each firm finds it optimal to advertise its brand in an attempt to “grow” its base of loyal customers, in equilibrium, branding (1) reduces firm profits, (2) increases prices paid by loyals and shoppers, and (3) adversely affects gatekeepers operating price comparison sites. Branding also tightens the range of prices and reduces the value of the price information provided by a comparison site.

Their research shows that brand advertising allows a firm to have higher prices, since loyal consumers aren't as price sensitive, but their conclusions are that profits are less - which I don't understand, unless the cost of creating the brand is very high. The Ars Technica post shows that other research continues to confirm that viewing brand ads creates a positive impression, which is one step towards converting shoppers into loyal customers.

This got me thinking about the various forms of advertising. The second paper distinguishes 'brand advertising' from 'informational advertising', which I agree is a useful distinction. I think of it in terms of how actionable the advertising is - how delayed is the payoff. For branding, the payoff is very indirect but could be profitable (or not, depending on which research you subscribe to) due to higher prices or repeat business or cutting out the competition through causing the customer to not engage in comparison shopping. For ad listings that show a very specific product or category, which are often a gateway to a purchasing decision, the payoff is fairly direct. Some even feel that the advertising cost model may evolve into 'cost per action' and go beyond 'cost per click'. However, cost per click is currently much easier to gather metrics in a two-way trusted fashion than measuring the final transaction in a two-way trusted fashion.

Another example of a very actionable advertisement is the Amazon or EBay offer listings - these are immediately purchasable, and through syndication via Associates are widely used as advertisements. I don't know of any company that does placement optimization of Amazon Associate links (raising placement for offer with better click through rates, better commission for the associate, etc), but I think Amazon has started doing some of that with Omakase links. One interesting thing to consider is that both Amazon and EBay are similar to price comparison sites due to the large number of offers for a single authoritative item (EBay doesn't have very good item authority, but people work around that issue by using manual searches). Getting top placement on an Amazon offer listing page, or in the 'buy box' on the details page, doesn't use an auction bidding process the way Google or Yahoo paid search listings do. The offer listing position is based on the offering price and estimated shipping costs.

Online Ad industry consolidation

So, what's up with all the acquisitions of online ad networks within the past 30 days?

• Google purchased DoubleClick for $3.1B on 4/13/2007 (on revenues of $150M).
• Yahoo purchased the remaining interest in RightMedia for $680M on 4/29/2007 (on revenues of $70M) They had purchased a 20% stake back in Oct 2006.
• Microsoft acquired European mobile ad network ScreenTonic on 5/3/2007.
• Microsoft acquired aQuantive for $6.1B on 5/18/2007 (on revenue of $442M).
• AOL acquires major interest in Adtech AG on 5/16/2007.
• AOL acquires mobile ad network Third Screen Media on 5/17/2007.
• WPP Group acquired 24/7 RealMedia for $650M on 5/17/2007.

A few billion here, a few billion there, pretty soon you're talking real money.

What's happening is different for each player, but the overall trend is the same - expanding beyond paid listings into creative branding. Paid search was $6.7B last year and brand advertising was $3.3B. This interest in brand advertising may be a reaction to the expectation that television - which is mostly branding style ads - is moving online.

In Google's case, they are buying a company that has been successful with creative ads, essentially banner ads. Banner ads are the most common choice for branding rather than being used for actionable ad listings. They also inherit distribution agreements for AOL and MySpace, and more distribution capacity helps draw advertisers into Google to bid for placement.

Microsoft hasn't done well in any online ad segment - listings or branding - and with their acquisition they will be more involved with holistic ad campaigns and deep in the creative arena of advertising. The includes the ad agencies that do the actual construction of creative ads and create very innovative branding experiences like custom website which are blurring the lines between interaction and advertising. There may be some future tie-in with 'rich internet applications' and Silverlight. I may be wrong, but I don't see aQuantive providing additional distribution capacity - 'inventory' as the ad industry calls it. They of course claim it 'extends their platform'. Everything's a platform to Microsoft. Maybe they should simply try providing value instead.

The hope is that the contextual and behavioral profiling that is done for ad listings will be applied to better target brand advertisements. A requirement for this to work is for the ad network to know a lot about their audience - something that a single site cannot accomplish. Effective audience profiling is orthogonal to Web sites - orthogonal to the Web's organization. However, by looking at the architecture and technologies of the Web you can see the areas where this multi-site capability can exist :

• clients such as rich internet applications, browsers and browser extensions like toolbars
• intermediaries such as the proxies that ISPs like Comcast operate
• compound resource structure of current Web documents. Since each resource can be retrieved from different domains, information can leak between domains.

Look for control or partnerships in these areas in the future.

Command Links

Uh oh. I see a mess coming up...
From Jakob Nielsen's Alertbox post on
Command Links

"Windows Vista introduced a new GUI widget for commands: the command link. Once something is in the system that people use on a daily basis, it becomes a de facto standard. Because they'll encounter them frequently in Vista, users will come to know and expect command links."

From what I gather, Vista 'command links' appear to be glorified buttons for native applications, not 'underlined text' links on Web pages.

However, Jakob continues with the following regarding web page links:

To reduce confusion, link text should explicitly state that it leads to an action and not just to a new page. It's not enough to communicate this info in the surrounding text; users often scan Web pages for the areas they can act on. Thus, you should assume that most users will only read the link text. In fact, users often read only the link text's first few words, so it's important to start with a word (typically a verb) that indicates the action that results if they click the link.

It appears Jakob is also implicitly approving the use of a link on a web page as an 'action'. From my reading, I think this is seriously wrong. I believe web page 'action links' need to satisfy the following two requirements

• visually distinct from normal 'safe' links
  
• syntactically distinct from normal 'safe' links
  

The second point is important because of the large number of automated agents that traverse the Web through hyperlinks. Adding unsafe links into the web will cause confusion among both people and software agents.

It’s my Vineyard

I found this blog about a couple that have purchased a vineyard in France - what a dream!

It’s my Vineyard

"Sitting in glorious sunshine on the terrace of The Restaurant du Pont having a delicious lunch we are reflecting that it is almost two years ago that we fell in love and bought Maison des Bulliats and its vines in Regnie, Beaujolais, one of the most idylic spots on earth."

This reminds me of the book "The Olive Season" which is about an impetuous British actress that purchases a run down villa and Olive 'garden' in the South of France. We found that book in the apartment we rented while we stayed in France two summers ago. Interesting book - although the scattered personality and writing of the author can get exasperating - and I'm looking forward to following this French vineyard blog.

XML is in the House

I ran across this page - Legislative Documents in XML at the United States House of Representatives: "The purpose of this website is to provide information about the ongoing work of the U.S. House of Representatives in relation to the eXtensible Markup Language" - and thought the host name was very cool - xml.house.gov.
This page has pointers to DTDs and schemas for the governmental processes involved with making and amending laws. So the next time we need to form a more perfect union, we've got this going for us.

And that page led to this page of a summary of floor proceedings of the US House of Representatives. It reads like a blog, only with less detail and there is no RSS feed.

Apollo, Silverlight, blah blah blah

Looks like Hugh W is one of the few questioning the value of Silverlight and RIA for the Web.

Flash, applets, Silverlight, Javascript -- the more you use them, the suckier your web apps are at exploring the web information space. I don't think it has to be this way, but it takes a design discipline few seem to have. These programming models are from the 80s. They have web APIs, but they're not web oriented. Programs end up as little desktop applications, not web apps. I don't see Silverlight changing that. It is good to have super expressive widgets -- hear hear. But if you're not pushing a bunch of hypertext down to my browser, you're not helping me explore the space.

I agree with his sentiment - you might say that RIA is to user interfaces as RPC is to messaging interfaces : more is not better. There probably will be a few years of smooth looking but hard to use (and harder to re-use) applications, while we wait for people to re-learn the basics of usability. I can only hope folks read Nielsen's UseIt column.

Mathematica 6

It looks like Wolfram Research has release a huge new release of Mathematica. I have only dabbled in Mathematica but I could spend all day playing and learning with it. It's hard to believe Mathematica came out almost twenty years ago!

Something they've added recently - and apparently improved on - is server based computing and visualization. Imagine what could be done by putting this together with something like Amazon's Elastic Compute Cloud (EC2).

See the Wolfram Blog for more details.

REST - it's inevitable

A couple weeks ago I was having lunch with a friend at Amazon - who coincidentally used to work at MS with the data access team and is a brilliant architect and engineer - and we talked about REST and how he was helping use REST concepts in refactoring some core back-end services. I made the observation that I was never stressed about how long it has taken for REST to achieve common industry understanding and acceptance - I just said "It's inevitable". I didn't realize just how short it would be for inevitable to show up.

It looks like REST has taken Redmond by storm. First I read that Mark Baker did some consulting with Microsoft (I missed the chance to have dinner with him when he was in town due to email snafu - major bummer). Then I read Dare Obasanjo's post that says "REST is totally sweeping Microsoft."

We passed the tipping point quite a while back, but it's still good to see pragmatic architectural sensibilities take root finally.

Sparkly things factory

Oh, this quote from Performancing.com is beautiful!

"Snap's preview anywhere gizmo is ruining the reading experience for millions of people. Its intrusive, obstructive and unuseful in almost every respect and use case. The fact that so many big blogs are using it, big well respected blogs, does not mean that it's useful, it just means that they, like most bloggers, have all the self restraint of a magpie in a sparkly things factory."

Amazon.com Widgets and hypertext

It looks like Typepad and Amazon are collaborating to help bloggers link to Amazon products. They have three widgets outlined, and one of them - the quick linker widget - uses custom attributes on an HTML anchor tag to make it easier to reference a set of products. The 'old school' would just use a URI, but those are hard to construct, hard to type and the wizards slow people down, blah blah blah. This not-quite-micro-formats approach is more understandable and more forgiving for hand-crafted markup. They define a new 'type' attribute for the anchor with the value "amzn". Then there are several more attributes like 'search' or 'category' or you can create a direct link via an 'asin' attribute. I assume some snippet of javascript would scan the page after it was loaded and construct the URI on the fly and set the href attribute on these anchors.
This is a creative solution to the "how do you construct a URI" problem, but it does leave spiders out in the cold, breaking the hyperlinking which defines the Web.

However, there is a simple approach they could use which would make a fully declarative and locally described document work and continue to allow auto-discover of hyperlinks to work - add a 'meta' tag to the head of the document with the URI template that corresponds to the 'type' attribute. I think the URI template proposal may need to do a bit of work related to optional or conditional patterns, or the URI template could stay static and the type="amzn" could change to be type="amzn-direct-link" or some other more qualified value.