Modular Abstractions in Scala with Cakes and Path Dependent Types

I have been trying out various options of implementing the Cake pattern in Scala, considered to be one of the many ways of doing dependency injection without using any additional framework. There are other (more functional) ways of doing the same thing, one of which I blogged about before and also talked about in a NY Scala meetup. But I digress ..

Call it DI or not, the Cake pattern is one of the helpful techniques to implement modular abstractions in Scala. You weave your abstract components (aka traits), layering on the dependencies and commit to implementations only at the end of the world. I was trying to come up with an implementation that does not use self type annotations. It's not that I think self type annotations are kludgy or anything but I don't find them used elsewhere much besides the Cake pattern. And of course mutually recursive self annotations are a code smell that makes your system anti-modular.

In the following implementation I use path dependent types, which have become a regular feature in Scala 2.10. Incidentally it was there since long back under the blessings of an experimental feature, but has come out in public only in 2.10. The consequence is that instead of self type annotations or inheritance I will be configuring my dependencies using composition.

Let me start with some basic abstractions of a very simple domain model. The core component that I will build is a service that reports the portfolio of clients as a balance. The example has been simplified for illustration purposes - the actual real life model has a much more complex implementation.

A Portfolio is a collection of Balances. A Balance is a position of an Account in a specific Currency as on a particular Date. Expressing this in simple terms, we have the following traits ..

// currency
sealed trait Currency
case object USD extends Currency
case object EUR extends Currency
case object AUD extends Currency

//account
case class Account(no: String, name: String, openedOn: Date, status: String)

trait BalanceComponent {
  type Balance

  def balance(amount: Double, currency: Currency, asOf: Date): Balance
  def inBaseCurrency(b: Balance): Balance
}

The interesting point to note is that the actual type of Balance has been abstracted in BalanceComponent, since various services may choose to use various representations of a Balance. And this is one of the layers of the Cake that we will mix finally ..

Just a note for the uninitiated, a base currency is typically considered the domestic currency or accounting currency. For accounting purposes, a firm may use the base currency to represent all profits and losses. So we may have some service or component that would like to have the balances reported in base currency.

trait Portfolio {
  val bal: BalanceComponent
  import bal._

  def currentPortfolio(account: Account): List[Balance]
} 

Portfolio uses the abstract BalanceComponent and does not commit to any specific implementation. And the Balance in the return type of the method currentPortfolio is actually a path dependent type, made to look nice through the object import syntax.

Now let's have some standalone implementations of the above components .. we are still not there yet to mix the cake ..

// report balance as a TUPLE3 - simple
trait SimpleBalanceComponent extends BalanceComponent {
  type Balance = (Double, Currency, Date)

  override def balance(amount: Double, currency: Currency, asOf: Date) = 
    (amount, currency, asOf)
  override def inBaseCurrency(b: Balance) = 
    ((b._1) * baseCurrencyFactor.get(b._2).get, baseCurrency, b._3)
}

// report balance as an ADT
trait CustomBalanceComponent extends BalanceComponent {
  type Balance = BalanceRep

  // balance representation
  case class BalanceRep(amount: Double, currency: Currency, asOf: Date)

  override def balance(amount: Double, currency: Currency, asOf: Date) = 
    BalanceRep(amount, currency, asOf)
  override def inBaseCurrency(b: Balance) = 
    BalanceRep((b.amount) * baseCurrencyFactor.get(b.currency).get, baseCurrency, b.asOf)
}

And a sample implementation of ClientPortfolio that adds logic without yet commiting to any concrete type for the BalanceComponent.

trait ClientPortfolio extends Portfolio {
  val bal: BalanceComponent
  import bal._

  override def currentPortfolio(account: Account) = {
    //.. actual impl will fetch from database
    List(
      balance(1000, EUR, Calendar.getInstance.getTime),
      balance(1500, AUD, Calendar.getInstance.getTime)
    )
  }
}

Similar to ClientPortfolio, we can have multiple implementations of Portfolio reporting that reports balances in various forms. So our cake has started taking shape. We have the Portfolio component and the BalanceComponent already weaved in without any implementation. Let's add yet another layer to the mix, maybe for fun - a decorator for the Portfolio.

We add Auditing as a component which can decorate *any* Portfolio component and report the balance of an account in base currency. Note that Auditing needs to abstract implementations of BalanceComponent as well as Portfolio since the idea is to decorate any Portfolio component using any of the underlying BalanceComponent implementations.

Many cake implementations use self type annotations (or inheritance) for this. I will be using composition and path dependent types.

trait Auditing extends Portfolio {
  val semantics: Portfolio
  val bal: semantics.bal.type
  import bal._

  override def currentPortfolio(account: Account) = {
    semantics.currentPortfolio(account) map inBaseCurrency
  }
}

Note how the Auditing component uses the same Balance implementation as the underlying decorated Portfolio component, enforced through path dependent types.

And we have reached the end of the world without yet committing to any implementation of our components .. But now let's do that and get a concrete service instantiated ..

object SimpleBalanceComponent extends SimpleBalanceComponent
object CustomBalanceComponent extends CustomBalanceComponent

object ClientPortfolioAuditService1 extends Auditing {
  val semantics = new ClientPortfolio { val bal = SimpleBalanceComponent }
  val bal: semantics.bal.type = semantics.bal
}

object ClientPortfolioAuditService2 extends Auditing {
  val semantics = new ClientPortfolio { val bal = CustomBalanceComponent }
  val bal: semantics.bal.type = semantics.bal
}

Try out in your Repl and see how the two services behave the same way abstracting away all implementations of components from the user ..

scala> ClientPortfolioAuditService1.currentPortfolio(Account("100", "dg", java.util.Calendar.getInstance.getTime, "a"))
res0: List[(Double, com.redis.cake.Currency, java.util.Date)] = List((1300.0,USD,Thu Jan 31 12:58:35 IST 2013), (1800.0,USD,Thu Jan 31 12:58:35 IST 2013))

scala> ClientPortfolioAuditService2.currentPortfolio(Account("100", "dg", java.util.Calendar.getInstance.getTime, "a"))
res1: List[com.redis.cake.ClientPortfolioAuditService2.bal.Balance] = List(BalanceRep(1300.0,USD,Thu Jan 31 12:58:46 IST 2013), BalanceRep(1800.0,USD,Thu Jan 31 12:58:46 IST 2013))

The technique discussed above is inspired from the paper Polymoprhic Embedding of DSLs. I have been using this technique for quite some time and I have discussed a somewhat similar implementation in my book DSLs In Action while discussing internal DSL design in Scala.

And in case you are interested in the full code, I have uploaded it on my Github.

Pushing the envelope on OO and functional with Scala

There has been quite a few inflamatory discussions in the twitterverse and the blogosphere recently regarding some of the ultra-functional voices dominating the Scala ecosystem. It's true that Scala is an object functional language. And it's also true that you can stretch Scala in both ways to get enough mileage out of it. Talking about the functional programming folks picking up Scala more and more, even Martin Odersky said this in his comment on one of such inflamatory posts ..

This is a good thing, IMO, because it pushes the envelope of what you can do with the language

Being a multi-paradigm language, I find Scala a wonderful mix. It has a very clean object model powered by its type system that helps programmers design scalable component abstractions. Scala also has a rich support for functional programming, which, though not as clean as Haskell, goes along very well complementing its OO capabilities. And from this perspective Scala offers an equal opportunity to developers coming from OO or FP paradigms.

In this post I will discuss one issue that can be solved elegantly in Scala using both of its paradigms. The OO version of the solution uses the elegance of mixins, abstract vals and the Cake pattern. While the functional version uses currying and partial applications.

Interface & Implementation

One of the recommended practices that we as software developers follow while designing domain models is to separate out the interface from an implementation. I am calling them by the terms interface and implementation. Replace them with the terminologies from your favorite language - contract, protocol, class, type .. etc. etc. But you know what I mean to say - distinguish between the nature of coupling of the generic and the specific parts of your abstraction.

Many people call this the Dependency Injection, where the actual implementation dependency is injected into your abstraction during runtime, resulting in reduced coupling. BTW I am not talking about DI frameworks, which may seem bolted on both in languages with a sane type system and those which don't have a static type system.

The OO way in Scala

Let's first consider how we can do dependency injection in Scala using the power of its object system. It has been covered in gory details by Jonas Boner in one of his Real World Scala blog posts. The example that I give here follows the same design that he used ..

Consider two abstractions that we use in our domain model for getting data out to the external world ..

// a repository for interacting with the underlying data store
trait TradeRepository {
  def fetch(refNo: String): Trade
  def write(t: Trade): Unit
}

// a domain service
trait TradeService {
  def fetchTrade(refNo: String): Trade
  def writeTrade(trade: Trade): Unit
}


And now the component for each of the above abstractions that contain the default implementations ..

trait TradeRepositoryComponent {
  val tradeRepo: TradeRepository

  class TradeRepositoryImpl extends TradeRepository {
    def fetch(refNo: String): Trade = //..
    def write(t: Trade): Unit = //..
  }
}

trait TradeServiceComponent{ this: TradeRepositoryComponent => // self type annotation that indicates the dependency
  val tradeService: TradeService

  class TradeServiceImpl extends TradeService {
    def fetchTrade(refNo: String) = tradeRepo.fetch(refNo)
    def writeTrade(trade: Trade) = tradeRepo.write(trade)
  }
}


Note how the self-type annotation is used in TradeServiceComponent to indicate a dependency on TradeRepositoryComponent. But still we are talking in terms of traits, without committing our final assembly to any specific object implementations. The abstract vals tradeRepo and tradeService have still not been materialized in terms of any concrete implementations. Thus we delay coupling to any implementation till the time we absolutely need them. And that is when we make the final assembly ..

// we are wiring up the components
object TradeServiceAssembly extends TradeRepositoryComponent with TradeServiceComponent {
  val tradeRepo = new TradeRepositoryImpl // impl
  val tradeService = new TradeServiceImpl // impl
}


Now we have the final object which encapsulates all implementation details and which we can use thus ..

// usage
import TradeServiceAssembly._
val t = tradeService.fetchTrade("r-123")
tradeService.writeTrade(t)


If we want to use different implementations (e.g. mocks for testing), we can either create another assembly module like the above and supply different implementation clasess to the abstract vals, or we can also directly mix in traits while we instantiate our assembly object ..

val assembly = new TradeRepositoryComponent with TradeServiceComponent {
  val tradeRepo = new TradeRepositoryMock
  val tradeService = new TradeServiceMock
}

import assembly._
val t = tradeService.fetchTrade("r-123")
tradeService.writeTrade(t)


So that was the implementation of DI using *only* the typesystem of Scala. All dependencies are indicated through self-type annotations and realized through concrete implementations specified in abstract vals right down to the object that you create as the final assembly. Shows the power of Scala's object model implemented over its type system.

The FP way in Scala

Now let's try to look at the same idiom through the functional lens of Scala.

We will still have the repository abstractions, but we implement the service contracts directly as functions.

trait TradeRepository {
  def fetch(refNo: String): Trade
  def update(trade: Trade): Trade
  def write(trade: Trade): Unit
}

// service functions
trait TradeService {
  val fetchTrade: TradeRepository => String => Trade = {repo => refNo => repo.fetch(refNo)}
  val updateTrade: TradeRepository => Trade => Trade = {repo => trade => //..
  val writeTrade: TradeRepository => Trade => Unit = {repo => trade => repo.write(trade)}
}


Now let's say we would like to work with a Redis based implementation of our TradeRepository. So somewhere we need to indicate the actual TradeRepository implementation class that the service functions need to use. We can define partial applications of each of the above functions for Redis based repository and put them in a separate module ..

object TradeServiceRedisContext extends TradeService {
  val fetchTrade_c = fetchTrade(new TradeRepositoryRedis)
  val updateTrade_c = updateTrade(new TradeRepositoryRedis)
  val writeTrade_c = writeTrade(new TradeRepositoryRedis)
}


So fetchTrade_c is now a function that has the type (String) => Trade - we have successfully abstracted the TradeRepository implementation class knowledge through currying of the first parameter.

These modules are somewhat like Spring ApplicationContext that can be swapped in and out and replaced with alternate implementations for other kinds of underlying storage. As with the OO implementation, you can plug in a mock implementation for testing.

We can now continue to use the curried versions of the service functions completely oblivious of the fact that a Redis based TradeRepository implementation has been sucked into it ..

import TradeServiceRedisContext._
val t = fetchTrade_c("ref-123")
writeTrade_c(t)


One of the potential advantages that you get with functional abstractions is the power of composability, which is, much better than what you get with objects. FP defines many abstraction models that composes in the mathematical sense of the term. If you can design your domain abstractions in compliance with these structures, then you can also get your models to compose as beautifully.

Instead of currying individual functions as above, we can curry a composed function ..

val withTrade = for {
  t <- fetchTrade
  u <- updateTrade
} 
yield(t map u)


withTrade is now a function of type (TradeRepository) => (String) => Trade. In order to make this work, you will need to have scalaz which defines higher order abstractions to make operations like bind (flatMap) available to a much larger class of abstractions than those provided by the Scala standard library. In our case we are using the function application as a monad. We can now inject the Redis based implementation directly into this composition ..

val withTradeRedis = withTrade(new TradeRepositoryRedis)


I find both variants quite elegant to implement in Scala. I use the version that goes better with my overall application design. If it's primarily an OO model, I go with the Cake based serving. When I am doing FP and scalaz, I use the functional approach. One advantage that I find with the functional approach is increased composability since free standing functions only need to agree on types to compose. While with objects, you need to cross over another layer of indirection which may not be that easy in some cases.

Dependency Injection as Function Currying

Dependency Injection is one of the techniques that I use regularly when I am programming in Java. It's a nice way of making an application decoupled from concrete implementations and localize object creation logic within specific bootstrapping modules. Be it in the form of Spring XML or Guice Modules, the idea is to keep it configurable so that specific components of your application can choose to work with specific implementations of an abstraction.

It so happens that these days possibly I have started looking at things a bit differently. I have been programming more in Scala and Clojure and being exposed to many of the functional paradigms that they encourage and espouse, it has stated manifesting in the way I think of programming. In this post I will look into dependency injection on a different note. At the end of it may be we will see that this is yet another instance of a pattern melding into the chores of a powerful language's idiomatic use.

In one of my projects I have a class whose constructor has some of its parameters injected and the others manually provided by the application. Guice has a nice extension that does this for you - AssistedInject. It writes the boilerplate stuff by generating an implementation of the factory. You just need to annotate the implementation class' constructor and the fields that aren't known to the injector. Here's an example from the Guice page ..

public class RealPayment implements Payment {
  @Inject
  public RealPayment(
        CreditService creditService,  // injected
        AuthService authService,  // injected
        @Assisted Date startDate, // caller to provide
        @Assisted Money amount);  // aller to provide
  }
  ...
}


Then in the Guice module we bind a Provider<Factory> ..

bind(PaymentFactory.class).toProvider(
  FactoryProvider.newFactory(
    PaymentFactory.class, RealPayment.class));


The FactoryProvider maps the create() method's parameters to the corresponding @Assisted parameters in the implementation class' constructor. For the other constructor arguments, it asks the regular Injector to provide values.

So the basic issue that AssistedInject solves is to finalize (close) some of the parameters at the module level to be provided by the injector, while keeping the abstraction open for the rest to be provided by the caller.

On a functional note this sounds a lot like currying .. The best rationale for currying is to allow for partial application of functions, which does the same thing as above in offering a flexible means of keeping parts of your abstraction open for later pluggability.

Consider the above abstraction modeled as a case class in Scala ..

trait CreditService
trait AuthService

case class RealPayment(creditService: CreditService,
                       authService: AuthService,
                       startDate: Date,
                       amount: Int)


One of the features of a Scala case class is that it generates a companion object automatically along with an apply method that enables you to invoke the class constructor as a function object ..

val rp = RealPayment( //..


is in fact a syntactic sugar for RealPayment.apply( //.. that gets called implicitly. But you know all that .. right ?

Now for a particular module , say I would like to finalize on PayPal as the CreditService implementation, so that the users don't have to pass this parameter repeatedly - just like the injector of your favorite dependency injection provider. I can do this as follows in a functional way and pass on a partially applied function to all users of the module ..

scala> case class PayPal(provider: String) extends CreditService
defined class PayPal

scala> val paypalPayment = RealPayment(PayPal("bar"), _: AuthService, _: Date, _: Int)
paypalPayment: (AuthService, java.util.Date, Int) => RealPayment = <function>



Note how the Scala interpreter now treats paypalPayment as a function from (AuthService, java.util.Date, Int) => RealPayment. The underscore acts as the placeholder that helps Scala create a new function object with only those parameters. In our case the new functional takes only three parameters for whom we used the placeholder syntax. From your application point of view what it means is that we have closed the abstraction partially by finalizing the provider for the CreditService implementation and left the rest of it open. Isn't this precisely what the Guice injector was doing above injecting some of the objects at module startup ?

Within the module I can now invoke paypalPayment with only the 3 parameters that are still open ..

scala> case class DefaultAuth(provider: String) extends AuthService
defined class DefaultAuth

scala> paypalPayment(DefaultAuth("foo"), java.util.Calendar.getInstance.getTime, 10000)
res0: RealPayment = RealPayment(PayPal(foo),DefaultAuth(foo),Sun Feb 28 15:22:01 IST 2010,10000)



Now suppose for some modules I would like to close the abstraction for the AuthService as well in addition to freezing PayPal as the CreditService. One alternative will be to define another abstraction as paypalPayment through partial application of RealPayment where we close both the parameters. A better option will be to reuse the paypalPayment abstraction and use explicit function currying. Like ..

scala> val paypalPaymentCurried = Function.curried(paypalPayment)
paypalPaymentCurried: (AuthService) => (java.util.Date) => (Int) => RealPayment = <function>



and closing it partially using the DefaultAuth implementation ..

scala> val paypalPaymentWithDefaultAuth = paypalPaymentCurried(DefaultAuth("foo"))
paypalPaymentWithDefaultAuth: (java.util.Date) => (Int) => RealPayment = <function>



The rest of the module can now treat this as an abstraction that uses PayPal for CreditService and DefaultAuth for AuthService. Like Guice we can have hierarchies of modules that injects these settings and publishes a more specialized abstraction to downstream clients.

Scala - To DI or not to DI

People have been discussing about dependency injection frameworks in languages that offer powerful abstraction techniques for construction and composition of objects. Scala mailing list points towards various approaches of implementing DI capabilities using the abstraction mechanisms over types and values and great composition techniques of traits. Gilad Bracha talks about his language, Newspeak, organized around namespaces, modules and lexically scoped nestable classes. Newspeak offers powerful class instantiation and module composition features that alleviate the necessity of writing explicit static factories or magical instantiation of objects through DI techniques.

I have also been thinking about the relevance of Dependency Injection in Scala and taking cue from the ideas discussed in the Scala community, tried out some techniques in a Scala application.

One of the features that a DI framework offers is the ability to decouple explicit concrete dependencies from the abstraction of the component. This is what Martin Odersky calls the service oriented software component model, where the actual component uses the services of other cooperating components without being statically dependent on their implementations. This composition of services is typically done using a DSL like module that sets up the wiring between components by injecting all relevant dependencies into the component graph.

Do I need a separate DI framework in Scala ?

With powerful mechanisms of object construction (and built-in factory method apply() with companion objects), abstraction over types and values and flexible composition using traits, Scala offers more power than Java in decoupling of concrete implementations from the abstract services. Every component can have separate configuration units that will inject the concrete type and data members that wire up non-intrusively to deliver the runtime machinery for the particular service.

Here is an example Scala class for computing the salary sheet of employees ..

abstract class SalaryCalculationEngine {
    trait Context {
        val calculator: Calculator;
        val calendar: Calendar;
    }

    protected val ctx: Context
    type E <: Employee

    def calculate(dailyRate: Double): Double = {
        ctx.calculator.calculate(dailyRate, ctx.calendar.noOfDays)
    }

    def payroll(employees: List[E]) = {
        employees.map(_.getDailyRate).foreach(s => println(calculate(s)))
    }
}


where Calculator is defined as a trait that mixes in ..

trait Calculator {
    def calculate(basic: Double): Double
}


Notice the use of the bounded abstract type and the abstract data members as configurable points of the abstraction. A concrete implementation of the above class will inject these abstract members to complete the runtime machinery. The configurable abstract data members form the context of the abstraction and has been encapsulated into another trait. This will be mixed in with a concrete implementation as part of the configuration module. Before going into the usage of the abstraction SalaryCalculationEngine, we need to configure the abstract types and data members. Let us define a module for Class1 Employees that will supply the exact concrete configuration parameters ..

trait Class1SalaryConfig {
    val calculator = new DefaultCalculator
    val calendar = new DefaultCalendar

    class DefaultCalculator extends Calculator {
        def calculate(basic: Double, noOfDays: Int): Double = basic * noOfDays
    }

    class DefaultCalendar extends Calendar {
        def noOfDays: Int = 30
    }
}


and use this configuration to instantiate the abstraction for generating salary sheet ..

val emps = List[Employee](..

val sal = new SalaryCalculationEngine {
    type E = Employee
    protected val ctx = new Class1SalaryConfig with Context
}
sal.payroll(emps)


Note how the concrete configuration (Class1SalaryConfig) mixes-in with the Context defined in the abstraction to inject the dependencies.

We can easily swap out the current implementation by mixing in with another configuration - the MockConfiguration ..

trait MockSalaryConfig {
    type T = MockCalendar
    val calculator = new MockCalculator
    val calendar = new MockCalendar

    class MockCalculator extends Calculator {
        def calculate(basic: Double, noOfDays: Int): Double = 0
    }

    class MockCalendar extends Calendar {
        def noOfDays: Int = 10
    }
}


and the application ..

val mock = new SalaryCalculationEngine {
    type E = Employee
    protected val ctx = new MockSalaryConfig with Context
}
mock.payroll(emps)


Extensibility ..

Traits make the above scheme very extensible. If we add another dependency in the Context, then we just need to provide a configuration for it in the implementation of the config trait. All sites of usage do not need to change since the Context mixes in dynamically with the configuration.

Powerful abstraction techniques of the Scala language help us achieve easy composability of services enable swapping in and out of alternative implementations in a fairly non-intrusive manner - one of the main features thet DI frameworks offer. The configurations can be easily reused at a much more coarse level of granularity and can be kept decoupled from the main flow of the application. From this point of view, these can act similar to the Modules of Guice, as they serve as the repository for binding information.

However, standard dependency injection frameworks shine in a couple of other ways which the above abstraction techniques fail to achieve :

• DI frameworks offer a container of their own that abstracts the object creation and injection services in a manner completely decoupled from the main business logic of the application. To the application code, the developer gets a bunch of unit testable classes with all dependencies injected transparently through declarative configuration based on DSLs. Language based techniques are not that non-intrusive and often are found to tangle with the main logic of the application.


• DI frameworks like Guice and Spring have a separate lifecycle of their own and perform lots of work upfront during application initialization. This is called the bootstrapping process, when all published dependencies are analysed and recursively injected starting from the root class. Hence runtime performance is greatly enhanced, since we have the injector as well as all bindings completely set up. In the case with Scala, every class asks for the configurations - hence it is more like a Service Locator pattern.


• Modules in Guice provide a nice way to decouple compile time independent components and act as the central repository for all published dependencies that need to be injected. The Spring application context does the same thing during startup. You create a dependency one time and use it in many places - hence development scalability improves. The configuration specification in Scala is at a finer level of granularity and is strictly not centrally managed. I would love to have a DSL that allows me to manage all configurations declaratively in a centralized repository.


• Another great feature that DI frameworks provide is integration with Web components and frameworks that allow objects to be created with specialized scopes, e.g. an object per http request, or an object per user session. These facilities come out of the box and provide great productivity boost to the developers.


• Interceptors and AOP are another area where the DI frameworks shine. The following snippet from Guice manual applies a transcation interceptor to all methods annotated with @Transaction ..

binder.bindInterceptor(
    any(),                              // Match classes.
    annotatedWith(Transactional.class), // Match methods.
    new TransactionInterceptor()        // The interceptor.
);


Despite all powerful abstraction techniques present in the Scala language, I think a separate dependency injection framework has a lot to offer. Most importantly it addresses the object construction, injection and interception of lifecycle services as a completely separate concern from the application logic, leading to more modular software construction.