Showing posts with label api programming. Show all posts
Showing posts with label api programming. Show all posts

Monday, December 27, 2010

A case study of cleaner composition of APIs with the Reader monad

In my earlier post on composable domain models, I wrote about the following DSL that captures the enrichment of a security trade by computing the applicable tax/fees and then the net cash value of the trade. It uses chained composition of scalaz functors .. In this post we are going to improve upon the compositionality, introduce a new computation structure and make our APIs leaner with respect to type signatures ..

scala> trd1 ° forTrade ° taxFees ° enrichWith ° netAmount  
res0: Option[scala.math.BigDecimal] = Some(3307.5000)


Here are the building blocks for the above .. the individual functions and the type definitions for each of them ..

forTrade: Trade => (Trade, Option[List[TaxFeeId]])
taxFees: (Trade, Option[List[TaxFeeId]]) => (Trade, List[(TaxFeeId, BigDecimal)])
enrichWith: (Trade, List[(TaxFeeId, BigDecimal)]) => RichTrade
netAmount: RichTrade => Option[BigDecimal]


and here's the chaining in action with wiring made explicit ..


Note how we explicitly wire the types up so as to make the entire computation composable. Composability is a worthwhile quality to have for your abstractions. However in order for your functions to compose, the types for input and output for each of them must match. In the above example, we need to have forTrade spit out a Trade object along with the list of tax/fee id, in order for it to compose with taxFees.

For an API to be usable, the secret sauce is to make it lean. Never impose any additional burden on to your API's interface that smells of incidental complexity to the user. This is exactly what we are doing in the above composition. Note we are carrying around the Trade argument pipelining it through each of the above functions. In our use case the Trade is a read-only state and needs to be shared amongst all functions to read the information from the object.

Enter the Reader Monad

Refactor the above into the Reader monad. A Reader is meant to be used as an environment (it's also known as the Environment monad) for all the participating components of the computation. What we need to do for this is to set up a monadic structure for our computation. Here are the modified function signatures .. I have changed some of the names for better adaptability with the domain, but you get the idea ..

val forTrade: Trade => Option[List[TaxFeeId]] = //..
val taxFeeCalculate: Trade => List[TaxFeeId] => List[(TaxFeeId, BigDecimal)] = //..
val enrichTradeWith: Trade => List[(TaxFeeId, BigDecimal)] => BigDecimal = //..


Every function takes the Trade but we no longer have to do an explicit chaining by emitting the Trade also as an output. This is where a monad shines. A monad gives you a shared interface to many libraries where you don't need to implement sequencing explicitly within your DSELs.

And here's our DSEL that runs through the sequence of enriching a trade while using the passed in trade as an environment .. (thanks @runarorama for the help with the Reader in scalaz)

val enrich = for {
  taxFeeIds      <- forTrade        // get the tax/fee ids for a trade
  taxFeeValues   <- taxFeeCalculate // calculate tax fee values
  netAmount      <- enrichTradeWith // enrich trade with net amount
}
yield((taxFeeIds ° taxFeeValues) ° netAmount)


This is a comprehension in Scala which is like the do notation of Haskell. Desugar it as an exercise and explore how flatMap does the sequencing.

Here's what the type of enrich looks like ..

scala> enrich
res1: (net.debasishg.domain.trade.dsl.TradeModel.Trade) => Option[BigDecimal] = <function1>


enrich is monadic in nature and follows the usual structure of a monad that sequences its operations through bind to give it an imperative look and feel. If any of the above sub-computations fail, the whole computation fails. But show it to a person who knows the domain of security trading - the steps in enrich nicely models the ubiquitous language.

I have the entire DSL in my github repo. You can get the use of enrich here in the test case ..

Monday, April 20, 2009

Towards Combinator based API design

I was listening to the presentation by Alex Payne, the Twitter API lead, that he delivered at Stanford very recently. A nice presentation that sums up the philosophies and practices that they followed in the evolution of APIs for Twitter. In case you are interested in API design, Josh Bloch also has a great video up as part of a Javapolis interview that discusses in details all the nuances of designing good and robust APIs.

These days, we are getting more and more used to interesting programming languages, which, apart from being powerful themselves, also mostly happen to share the wonderful ecosystem of common runtimes. As Ola Bini noted sometime back on his blog, it is a time when we can design the various architectural layers of our application in different languages, depending on the mix of robustness, type-safety and expressiveness that each of them demands. Client facing APIs can be focused more towards expressiveness, being more humane in nature, a pleasure to use, while at the same time offering modest error handling and recovery abilities. But whatever the layer, APIs need to be consistent, both in signature and in return values.

One of the very important aspects of API design is the level of abstraction that it should offer to the user. The ideal level comes out only after a series of exploratory evolutions, refactorings and user implementations, and can often lead to loss of backward compatibility within the existing user community.

One of the very powerful ways of API design that many of today's languages offer is the use of combinators. I have blogged on uses of combinator in concatenative languages like Joy - it is truly a great experience as a user to use such compositional API s as part of your application design. Here is one from my earlier post on Joy combinators. It finds the arithmetic mean of a list of numbers ..

dup  0  [+]  fold  swap  size  /


The API is beautiful in the sense that it evolves the intention ground up and makes use of smaller combinators in building up the whole. This is beautiful composition.

In one of the comments to my earlier post, James Iry mentioned "I remain unconvinced that concatenative languages are really buying much over applicative languages, as interesting as they are". Since then I have been dabbling a bit with Haskell, a pure functional language that does many things right with the notion of static typing, offering powerful point free capabilities along with rich combinator composition ..

A simple pointfree sum ..

sum = foldr (+) 0


and a more complex map fusion ..

foldr f e . map g == foldr (. g) e


The main point is to seek the beauty of API design in expressiveness of the contract through effective composition of smaller combinators. The biggest advantage of combinators is that they offer composability i.e. the value of a bigger abstraction is given by combining the values of its sub-abstractions. And the power of composability comes from the world of higher order functions and their ability to combine them in your programming language, just as you would do the same in mathematics.

Object orientation is not so blessed in this respect. Composition in OOP is mostly confined to designing fluent interfaces that make expressive APIs but can be made useful only in a limited context and of course, without the purity that functional abstractions espouse. The Builder design pattern is possibly the most famous compositional construct in object oriented languages, and often lead to sleak APIs. Here is a great example from Google Collections MapMaker API ..

ConcurrentMap<Key, Graph> graphs = 
  new MapMaker()
    .concurrencyLevel(32)
    .softKeys()
    .weakValues()
    .expiration(30, TimeUnit.MINUTES)
    .makeComputingMap(
      new Function<Key, Graph>() {
        public Graph apply(Key key) {
          return createExpensiveGraph(key);
        }
      }
    );



But Java, being a language that is not known to offer the best of functional features, it is often quite clumsy to compose abstractions in a fluent way that can offer consistent, rich and robust APIs that match the elegance of functional combinators.

Scala is not a particularly rich language for pointfree programming. But Scala offers great library support for combinators. Of course the secret sauce to all these is the rich support of functional programming that Scala offers. Parser combinators that come with Scala standard library help design external DSL s with enough ease and convenience. Quite some time back I had blogged on designing a combinator based DSL for trading systems using Scala parser combinators.

The main power of combinators come from the fact that they are compositional, and it is the presence of non composable features that make combinators hard in some languages. And one of them is shared mutable state. Paul Johnson had it absolutely right when he said "Mutable state is actually another form of manual memory management: every time you over-write a value you are making a decision that the old value is now garbage, regardless of what other part of the program might have been using it". Languages like Erlang enforces confinement of mutable state within individual processes, Scala encourages the same through programming practices, while Haskell, Clojure etc. offer managed environments for manipulating shared state. Hence we have composability in these languages that encourage combinator based API design.

Combinator based API design is nothing new. It has been quite a common practice in the worlds of Haskell and other functional languages for quite some time. Simon Peyton Jones described his experience in ICFP 2000 in applying combinator based API design while implementing a financial system for derivative trading. It was one of those trend setter applications in the sense that "the ability to define new combinators, and use them just as if they were built in, is quite routine for functional programmers, but not for financial engineers".

Tuesday, July 24, 2007

Why API Design Matters

So, why are there so many bad APIs around? The prime reason is that, for every way to design an API correctly, there are usually dozens of ways to design it incorrectly.

Good API design is hard, while bad APIs are easy. Michi Henning's API Design Matters in May/June issue of ACM Queue is a comprehensive guide to designing good APIs.

He dissects the .NET socket Select() function in C# and points out the deficiencies which many of our day-to-day usable frameworks have. The article also discusses the interplay of abstraction hierarchies with good api design.
Much of software development is about creating abstractions, and APIs are the visible interfaces to these abstractions. .. The lower in the abstraction hierarchy an API defect occurs, the more serious are the consequences.

And finally, the most important aspect of API design is to manage the expectations of the closure of users who will be exercising them. And once you have published your API, it becomes immutable. Can't agree more ..

A very good read ..