Property based testing for domain models

One of the challenges that we face building a non trivial domain model is to write proper tests that verify the domain rules that the model implements. The domain rules can be quite complex, may have a number of edge cases which the developer himself may fail to take care of. When you use an implementation language that supports a decent type system, many of the rules and invariants can be encoded statically within the type system itself. This makes it impossible for the programmer to write any code that violates those constraints. But you can only encode some of the domain rules as part of your static type based constraints - you need to complement them with a testing procedure that validates the semantic behavior of the model.

If you are doing testing manually, you are doing it wrong. And if you use xUnit based testing procedures, there's enough scope for you to grow up towards better frameworks that offer true automation not only with respect to executing your tests, but generating the data as well.

Frameworks like QuickCheck in Haskell or ScalaCheck in Scala allows you to write property specifications that the model should satisfy and then generate data to guide evaluation of test executions for correctness as well as completeness. Here's what Bryan O'Sullivan et al mentions when discussing property based testing in their book Real World Haskell ..

Property-based testing encourages a high level approach to testing in the form of abstract invariants functions should satisfy universally, with the actual test data generated for the programmer by the testing library. In this way code can be hammered with thousands of tests that would be infeasible to write by hand, often uncovering subtle corner cases that wouldn't be found otherwise.

I have been doing lots of domain modeling on securities trading system. The model is a quite complex one and like the most of us, I started with xUnit based testing to verify some of the invariants and constraints that the system needs to honor. But very quickly I found that properties offer a more succinct way to abstract the constraints and invariants of my model. Hence using a tool like ScalaCheck makes it much simpler once I give it a proper data generator as input. Consider the simple property in the following example ..

A trade needs to be enriched in its lifecycle with tax/fee values and other attributes. We can then compute its net value which should be a positive numeric quantity.

property("Enrichment of a trade should result in netvalue > 0") {
  forAll((a: Trade) =>
    enrichTrade(a).netAmount.get should be > (BigDecimal(0)))
}

Here I don't need to construct concrete data by hand (in fact this is what makes xUnit based frameworks a sophisticated manual testing system - it only automates the execution part of your testing). Instead what I supply is a trade generator which gives ScalaCheck enough information based on which it can generate loads of trades. A typical simplified version of the trade generator is the following ..

implicit lazy val arbTrade: Arbitrary[Trade] =
  Arbitrary {
    for {
      a <- Gen.oneOf("acc-01", "acc-02", "acc-03", "acc-04")
      i <- Gen.oneOf("ins-01", "ins-02", "ins-03", "ins-04")
      r <- Gen.oneOf("r-001", "r-002", "r-003")
      m <- arbitrary[Market]
      u <- Gen.oneOf(BigDecimal(1.5), BigDecimal(2), BigDecimal(10))
      q <- Gen.oneOf(BigDecimal(100), BigDecimal(200), BigDecimal(300))
    } yield Trade(a, i, r, m, u, q)
  }

I can make more sophisticated properties and ask the system to check whether the model satisfies the property or not ..

property("Enrichment should mean netValue equals principal + taxes") {
  forAll((a: Trade) => {
    val et = enrichTrade(a)
    et.netAmount should equal (et.taxFees.map(_.foldLeft(principal(et))((a, b) => a + b._2)))
  })
}

This is a direct encoding of the business rule, which the domain model needs to satisfy. And using properties we can encode the verification in a more declarative fashion, and that too without bothering about how to generate concrete data for all the test cases.

Crosscutting Property Verification

When we talk about properties of the model, we can go all the way and write specifications for properties at all levels of granularity. It can be a local property limited to one module or it can be a system wide property, an invariant that holds good across multiple components. What I want to say is that property based testing can be very effective in system testing (or integration testing) as well.

In our system, we have Client Orders coming in that get executed in the stock exchanges resulting in broker side executions, which then get allocated to client accounts resulting in client trades. A succinct implementation of this entire flow can be the following ..

def tradeGeneration(market: Market, broker: Account, clientAccounts: List[Account]) =
  // client orders           executed at market by broker        & allocated to client accounts
  kleisli(clientOrders) >=> kleisli(execute(market)(broker)) >=> kleisli(allocate(clientAccounts))

We can use property based specification to test this entire lifecycle using ScalaCheck. Let's check for one of the invariants in this entire process -

Invariant: "The total quantity of order will be equal to the total quantity traded"

and here's the specification of the property in ScalaCheck ..

property("Client trade allocation in the trade pipeline should maintain quantity invariant") {
  forAll { (clientOrders: List[ClientOrder], args: (Market, Account, List[Account])) =>
    whenever (clientOrders.size > 0 && args._2.size > 0 && args._3.size > 0) {
      val trades = tradeGeneration(args._1, args._2, args._3)(clientOrders)
      trades.size should be > 0
      (trades.sequence[({type λ[a]=Validation[NonEmptyList[String],a]})#λ, Trade]) match {
        case Success(l) => {
          val tradeQuantity = l.map(_.quantity).sum
          val orderQuantity = fromClientOrders(clientOrders).map(_.items).flatten.map(_.qty).sum
          tradeQuantity should equal(orderQuantity)
        }
        case _ => fail("should get a list of size > 0")
      }
    }
  }
}

The properties are the domain rules and these can be prepared by the domain experts. I am a firm believer in collaborative involvement of domain experts in building the model. I have advocated the use of domain specific languages as a thin linguistic abstraction on top of a domain model that should be built iteratively with the domain experts. Property based testing is the third piece that completes the solution framework of developing complete domain models. Property based testing strengthens this view of increased involvement of the domain people in building the software. At the end of the day, leave everything to the respective experts - the domain people specifies properties and invariants, the developer implements the specification and the underlying test framework does the heavy lifting of generating enough data and automate the execution process.

It's the familiarity model!

James Iry recently blogged on code density and tries to find out the meaning of "dense code". As an example he took upon the topic of regular expressions, which despite being dense are not frowned upon in coding and hardly get replaced for making the code fragment more readable.

So the question is what makes code dense so that it's not acceptable to programmers and they complain about it's incomprehensibility ?

Later in the blog post James himself identifies unfamiliarity as one of the culprits. People don't complain about regexes since they are familiar with them, but will surely complain of something else which they are not familiar with.

Almost during the same time Ola Bini blogged about expressiveness in programming language syntax. He mentions that a well designed syntax should help programmers *read* the code easily. But he also questions about the target programmers ..

who is this person reading ? It makes a huge difference if we’re trying to design something that should be easy to read for a novice or we’re trying to design a syntax that makes it easier for an expert to understand what’s going on.

Once again we get into this territory of familiarity and mental model. An expressive piece of code becomes readable only to a person who is familiar with the underlying model. In my programming career I have come across this dichotomy a number of times where programmers complain of something being too dense the moment it crosses the threshold of his familiarity level. I have seen developers taking every pain to understand the nuances of a Spring XML configuration. Or who have spent zillions of hours mastering the whole bunch of performance tuning Hibernate with stuffs like query cache configuration. Believe me it's not simple with tonnes of corner cases to take care of and even today I am not sure if it can be achieved in a deterministic way for all kinds of data models. But these same developers complain when they are faced with maintaining code that needs a basic understanding of functional programming, set theory or algebraic data types. I think it's purely because these form outside the limits of their familiarity model.

For a programmer who is not familiar with higher order functions, combinators like map, fold or filter will look too dense. So when you say map (+1) [1..5], the code fragment looks much less comprehensible to him than his familar variant of using an imperative mutated-indexed for-loop. To the unfamiliar the functional variant appears dense, to the expert it becomes succinct.

One of the challenges that I face today is to make programmers believe that learning new stuff will only help them think better. It's not mandatory that they will need all of these tools as part of their day job. But broadening your mental model can only help your thought process to leverage a wider playground. Maybe in our part of the world big companies give no incentive to transform yourself from a billable offshore resource to a thinking programmer. But you really need to transcend the limits of your familiarity model in order to appreciate code which experts certify as succinct.

A Case for Orthogonality in Design

In Learning, using and designing command paradigms, John M Carroll introduces the notion of lexical congruence. When you design a language, one of the things that you do is lexicalization of the domain. If we extrapolate the concept to software designs in general, we go through the same process with our domain model. We identify artifacts or lexemes and choose to name them appropriately so that the names are congruent with the semantics of the domain. This notion of lexical congruence is the essence of having a good mnemonics for your domain language. I found the reference to Carroll's work in Testing the principle of orthogonality in language design, which discusses the same issue of organizing your language around an optimal set of orthogonal semantic concepts. This blog post tries to relate the same concepts of orthogonality in designing domain models using the power that the newer languages of today offers.

The complexity of a modeling language depends on the number of lexemes, their congruence with the domain concepts being modeled and the number of ways you can combine them to form higher order lexemes. The more decoupled each of these lower level lexemes are, the easier they are to compose. When you have overlapping concepts being modeled as part of lexemes, the mixing is not easy. You need to squeeze in some special boundary conditions as part of composition logic. Making your concepts independent yet composable makes your design orthogonal.

The first time I came across the concept of orthogonality in design and consciously appreciated the power of unification that it brings on to your model, is through Andrei Alexandrescu's idea of policy based design that he evangelized in his book Modern C++ Design and in the making of the Loki library. You have orthogonal policies that are themselves reusable as independent abstractions. And at the same time you can use the language infrastructure to combine them when composing your higher order model. Consider this C++ example from Andrei's book ..

template
<
  class T,
  template <class> class CheckingPolicy,
  template <class> class ThreadingModel
>
class SmartPtr;


CheckingPolicy enforces constraints that need to be satisfied by the pointee object. The ThreadingModel abstraction defines the concurrency semantics. These two concerns are not related in any way between themselves. But you can use the power of C++ templates to plug in appropriate behaviors of these concerns when composing your own custom type of SmartPtr ..

template SmartPtr<Widget, NoChecking, SingleThreaded>
  WidgetPtr;


This is orthogonal design where you have a minimal set of lexemes to model otherwise unrelated concerns. And use the power of C++ templates to evolve a larger abstraction by composing them together. The policies themselves are independent and can be applied to construct arbitrary families of abstraction.

The crux of the idea is that you have m concepts that you can use with n types. There's no static relationship between the concepts and the types - that's what makes orthogonality an extensible concept. Consider Haskell typeclasses ..

class Eq a where 
  (==) :: a -> a -> Bool


The above typeclass defines the concept of equality. It's parameterized on the type and defines the constraint that the type as to define an equality operator in order to qualify itself as an instance of the Eq typeclass. The actual type is left open which gives the typeclass an unbounded extensibility.

For integers, we can do

instance Eq Integer where 
  x == y =  x `integerEq` y


For floats we can have ..

instance Eq Float where 
  x == y =  x `floatEq` y


We can define Eq even for any custom data type, even recursive types like Tree ..

instance (Eq a) => Eq (Tree a) where 
  Leaf a         == Leaf b          =  a == b
  (Branch l1 r1) == (Branch l2 r2)  =  (l1==l2) && (r1==r2)
  _              == _               =  False


Haskell typeclasses, like C++ templates help implement orthogonality in abstractions through a form of parametric polymorphism. Programming languages offer facilities to promote orthogonal modeling of abstractions. Of course the power varies depending on the power of abstraction that the language itself offers.

Let's consider a real world scenario. We have an abstraction named Address and modeled as a case class in Scala ..

case class Address(no: Int, street: String, 
                   city: String, state: String, zip: String)


There can be many contexts in which you would like to use the Address abstraction. Consider printing of labels for shipping that needs your address to be printed in some specific label format, as per the following trait ..

trait LabelMaker {
  def toLabel: String
}


Note that printing addresses in the form of labels is not one of the primary concerns of your Address abstraction. Hence it makes no sense to model it as one of the methods of the class. It's only required that in some situations we may need to use the Address to print itself in the form of a label as per the specification mandated by LabelMaker.

One other concern is sorting. You may need to have your addresses sorted based on zip code before submitting them to your Printer module for shpping. Sorting may be required in combination with label printing or as well as on its own - these two are orthogonal concerns that should never have any dependence amongst themselves within your abstraction.

Depending on your use case, you can decide to compose your Address abstraction as

case class Address(houseNo: Int, street: String, 
  city: String, state: String, zip: String)
  extends Ordered[Address] with LabelMaker {
  //..
}


which makes your Address abstraction statically coupled with the other two.

Or you may like to make the composition based on individual objects which would keep the base abstraction independent of any static coupling.

val a = new Address(..) with LabelMaker {
  override def toLabel = {
    //..
  }
}


As an alternative you can also choose to implement implicit conversions from Address using Scala views ..

object Address {
  implicit def AddressToLabelMaker(addr: Address) = new LabelMaker {
    def toLabel =
      "%d-%s, %s, %s-%s".format(
        addr.houseNo, addr.street, addr.city, addr.state, addr.zip)
  }
}


Whatever be the implementation, take note of the fact that we are not polluting the basic Address abstraction with the concerns that are orthogonal to it. Our model, which is the design language treats orthogonal concerns as separate lexemes and encourages ways to compose them non invasively by the user.

Configurable Decorators using AOP

Sometime back I had posted on the increasing misuse (or overuse) of aspects by today's developers. In many of those cases, plain old decorators would have just been fine. However, very recently, I used a combination of decorators and aspects to add another degree of configurability to a domain model.

Decorators, by design, should be lightweight objects which can be attached to the skin of other domain objects to add to the roles and responsibilities of the latter. They are added and/or removed dynamically from original objects - hence we say that decorators offer an added level of flexibility over static inheritance. Consider the following aggregate, which is modeled as a Composite :

Here is the base abstraction for the Composite, which models the Salary components of an employee :

public abstract class Salary {
  public Salary add(final Salary component) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public Salary remove(final Salary component) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public Salary getChild(final int i) {
    throw new UnsupportedOperationException("not supported at this level");
  }

  public abstract double calculate();
}


and here is the composite ..

public class SalaryComposite extends Salary {
  private List<Salary> components = new ArrayList<Salary>();

  @Override
  public Salary add(Salary component) {
    components.add(component);
    return this;
  }

  @Override
  public double calculate() {
    double total = 0;
    for(Salary salary : components) {
      total += salary.calculate();
    }
    return total;
  }

  @Override
  public Salary getChild(int i) {
    return components.get(i);
  }

  @Override
  public Salary remove(Salary component) {
    components.remove(component);
    return this;
  }
}


and a couple of leaf level components ..

public class BasicPay extends Salary {
  private double basicPay;

  public BasicPay(double basicPay) {
    this.basicPay = basicPay;
  }

  @Override
  public double calculate() {
    return basicPay;
  }
}

public class HouseRentAllowance extends Salary {
  private double amount;

  public HouseRentAllowance(double amount) {
    this.amount = amount;
  }

  @Override
  public double calculate() {
    return amount;
  }
}


Decorators and Composites - a Marriage

Decorators work nicely with Composites, and I have some decorators to allow users add behaviors to the Composite elements dynamically. In order to have decorators for the Composite, I need to have the Decorator share the base class with the Composite.

public abstract class SalaryDecorator extends Salary {
  private Salary component;

  public SalaryDecorator(final Salary component) {
    this.component = component;
  }

  @Override
  public double calculate() {
    return component.calculate();
  }
}


and a couple of concrete decorators ..

a senior citizen adjustment which can be applied to some of the components of the salary ..

public class SeniorCitizenAdjustment extends SalaryDecorator {
  protected double adjustmentFactor;
  public SeniorCitizenAdjustment(final Salary component, double adjustmentFactor) {
    super(component);
    this.adjustmentFactor = adjustmentFactor;
  }

  @Override
  public double calculate() {
    //.. calculation of absolute amount based on adjustmentFactor
    //.. complex logic
    adjustment = ...
    return super.calculate() + adjustment;
  }
}


and a city compensatory allowance which varies based on the city where the employee leaves ..

public class CityCompensatoryAdjustment extends SalaryDecorator {
  private double adjustmentFactor;

  public CityCompensatoryAdjustment(final Salary component, double adjustmentFactor) {
    super(component);
    this.adjustmentFactor = adjustmentFactor;
  }

  @Override
  public double calculate() {
    //.. calculation of absolute amount based on adjustmentFactor
    //.. complex logic
    adjustment = ...
    return super.calculate() + adjustment;
  }
}


Now my clients can make use of the Composite and decorate them using the decorators designed to compose the individual salary components ..

//..
Salary s = new SalaryComposite();
s.add(new SeniorCitizenAdjustment(new BasicPay(1000), 100))
 .add(new CityCompensatoryAdjustment(new HouseRentAllowance(300), 150));
//..


In the above example, the use of decorators provide the flexibility that the particular design pattern promises and allows instance level customization of individual components of the composite.

Decorating the Decorators

How do you handle fine grained variations within decorators ? In our case many of the decorators had fine grained variations within themselves, which calls for either inheritance hierarchies between them or some other way to decorate the decorators (super-decorators ?). The problem with static inheritance hierarchies is well-known. They have to be defined during compile time and the codebase for each deployment need to have the explosion of all possible variations modeled as subclasses. Looks like the problem that decorators are supposed to solve and that which we have already solved for the Composite Salary aggregate. Now we have the same problem for the decorators themselves.

We rejected the idea of static inheritance hierarchies and decided to model the variations as yet another level of decorators. We did not want to make the original decorators too complex and focused on making small composable, additive abstractions that can be tagged on transparently onto the domain objects and decorators alike.

e.g. in an implementation we found that the CityCompensatoryAdjustment has another fine grained variation. When CityCompensatoryAdjustment is applied, we have the additional variation that states that if the employee's residence is in one of a group of premium cities, then he is eligible for an additional premium allowance on top of the normal CityCompensatoryAdjustment. We rejected the idea of changing CityCompensatoryAdjustment to incorporate the new variations for 2 reasons :

1. Trying to incorporate too much logic into decorators will make them complicated, which defeats the basic design motivation for the decorators


2. These variations change across deployments - hence it will be a bloat trying to incorporate everything into a single class

Modeling the new requirement as another decorator (PremiumCityAdjustment), we have the following usage of the above snippet ..

//..
Salary s = new SalaryComposite();
s.add(new SeniorCitizenAdjustment(new BasicPay(1000), 100))
 .add(new PremiumCityAdjustment(
          new CityCompensatoryAdjustment(
              new HouseRentAllowance(300), 150),
          adjustmentFactor));
//..


But the Variations are Deployment specific!

We cannot change the main codebase with the above fine-grained variations, since they are deployment specific. We need to find out ways to externalize the invocation of these decorators. We found out that these finer variations are policies that need to change the behavior of the original decorators whenever they are applied. In the above example, every invocation of CityCompensatoryAdjustment needs to be decorated with PremiumCityAdjustment.

public class PremiumCityAdjustment extends SalaryDecorator {
  public PremiumCityAdjustment(final Salary component) {
    super(component);
  }

  @Override
  public double calculate() {
    //.. calculation of adjustment amount
    adjustment = ..
    return super.calculate() + adjustment;
  }
}


Aspects again !

We decided to implement the finer grained variations as decorators which would be applied through aspects to the original decorators. This is the base aspect for all such SuperDecorators ..

public abstract aspect SalarySuperDecoratorBase<T extends SalaryDecorator> {

  pointcut decoratorCalculate(T s) : target(s)
        && execution(double T.calculate());

  abstract pointcut myAdvice();
}


and here is the implementation for adding PremiumCityAdjustment as a decorator for CityCompensatoryAdjustment ..

public aspect SuperDecoratorCityCompensatoryAdjustment
    extends SalarySuperDecoratorBase<CityCompensatoryAdjustment> {

  pointcut myAdvice(): adviceexecution()
    && within(SuperDecoratorCityCompensatoryAdjustment);

  Object around(CityCompensatoryAdjustment s) : decoratorCalculate(s) && !cflow(myAdvice()) {
    return new PremiumCityAdjustment(s).calculate();
  }
}


This way we had a set of core decorators for the main domain model and another set of deployment specific decorators which decorated the core ones through aspect weaving. By adopting this strategy we were able to keep the decorators lightweight, avoided deep inheritance hierarchies and managed to keep customizable code base completely separate from the core one.

Can you differentiate between an Abstraction and a Lie ?

In the June 2007 issue of Communications of the ACM, David Parnas differentiates between an abstraction and a lie. He mentions that abstractions that embody unrealistic assumptions represent nothing at all. He learnt the correct definition of abstraction from the late E. W. Dijkstra :

An abstraction is one thing that represents several real things equally well.

And he concludes ..

Finding the simplest model that is not a lie is the key to better software design.

So true, and how lucidly expressed! Go read the full article here. [Membership required for full text].

How do you decide when to use Aspects ?

Any new technology comes with its own powerbag and it depends upon us programmers to make meaningful and responsible use of the power. New age programming languages are riding their power continuum - it is upto us to stay a blub programmer, be an average coder or try to be the hacker. But, whatever, you decide to do, make your decision an informed one - be responsible to use the power in the proper place and perspective.

Aspect oriented programming is a new buzzword that has been popularized more recently through its integration with the Spring framework. The Spring guys have done a great job in bringing a difficult technology to the masses through its usual style of declarative programming. Spring AOP takes a lot of pain out of you by offering a greatly simplified programming model to have method interceptions baked in your codebase.

But making a technology look simpler has its obvious consequences of being misused. There appears to be lots of cases with programmers where they are using aspects, when good old simple Java design patterns, make a more appropriate cut. In the last couple of months, I found Spring AOP's method interception being used in many instances (including this one in an InfoQ article) when good old decorators could have solved the problem. The basic problem which the developer was trying to solve was to wrap some command with pre- and post- advices. The AOP based solution would make sense only if the same strategy needs to be repeated in multiple places and invocations of the command, which would otherwise have resulted in lots of boilerplates littering the codebase. Otherwise, a command and a bunch of decorators can provide a scalable solution to this ..

// the generic command interface
public interface ICommand {
void execute(final Object object);
}

// and a decorator interface for decorating the command
public abstract class Decorator implements ICommand {

// the command to decorate
private ICommand decorated;

public Decorator(final ICommand decorated) {
this.decorated = decorated;
}

public final void execute(Object object) {
pre();
decorated.execute(object);
post();
}

protected final ICommand getDecorated() {
return decorated;
}

// the pre-hook
protected abstract void pre();

// the post-hook
protected abstract void post();
}

// my custom command class
public class FileCommand implements ICommand {
//.. custom command
}

// my first custom decorator
public class MyDecorator_1 extends Decorator {

public MyDecorator_1(final ICommand command) {
super(command);
}

@Override
protected void post() {
//.. custom post hook
}

@Override
protected void pre() {
//.. custom pre hook
}
}

// another custom decorator
public class MyDecorator_2 extends Decorator {

public MyDecorator_2(final ICommand command) {
super(command);
}

@Override
protected void post() {
//.. custom post hook
}

@Override
protected void pre() {
//.. custom pre hook
}
}
// stack up the decorators
new MyDecorator_2(
new MyDecorator_1(
new FileCommand(...))).execute(..);


Use Aspects to address crosscutting concerns only

I use aspects as a last resort, when all options fail to address the separation of concerns that I am looking for in my code organization. Aspects help avoid the code tangle by identifying joinpoints through pointcuts and helping define advices that will be applied to the joinpoints. But I use them only when all traditional Java tools and techniques fail to localize my solution. Aspects bring in a separate machinery, the heavy lifting of bytecode instrumentation. Spring AOP is, however, pure Java, but based on dynamic proxies, which have their own limitations in method interceptions and performance penalties (however small) of creating proxies on every call. Spring AOP has less magic than pure AOP - but use the technology only if it is the right choice for the problem at hand. At least, recently, I find many instances of this wonderful technology being misused as a result of sheer over-engineering. Often, when we see nails, everything starts to look like a hammer. Keep your solution simple, ensure that it solves the immediate problem at hand and gives you adequate options for extensibility.

Domain Driven Design - What's in an Exception Name ?

Colin Jack does not feel that exception names should be part of the domain language. In one of my blog posts, he comments :

I also don't see the exceptions are part of the domain language, instead I see the associated rules as being part of the domain language. So for me the rule name should become part of the domain language but the exceptions do not have to be, its the rules that matter to me when discussing things with the domain experts.

Colin, I have to disagree with you. Here is an example from a real life project (as per your request in my blog comments) and a real discussion session with one of the domain experts.

Here is a snippet from a Capital Market Solutions codebase. The codebase is replete with domain specific terms, as it should be in a domain model implementation, starters (and unstarters) of the capital market domain are requested to look up relevant information for background of the terminologies .. apologies :-(

Class PositionManager is the domain abstraction for managing the Position of an account. Note that Position is the domain term which indicates the balance of the account.

class PositionManager {
  //..
  //..
  void updatePosition(Instrument ins, Date tradeDate,
    BigDecimal amount, TradeType tt, ..) {
    Position currentPosition = getPosition(acc, ins, tradeDate);
    currentPosition.doUpdate(amount, tt);
  }
  //..
}

Now have a look at the method Position#doUpdate invoked above. Here the update is done through a business rule modeled as a Strategy design pattern. The rule can have multiple implementations, hence modeled as an interface. Again we use domain driven principles here to isolate the execution of a process from the rules on which it operates. Note the checked exception which the method catches - ShortPositionException, we will come back to the name in just a moment. Keep reading ..

Here is the definition :

class Position {
  private PositionUpdationStrategy strategy;

  //..
  //..
  void doUpdate(BigDecimal amount, TradeType tt) {
    try {
      strategy.doUpdate(amount, tt);
    } catch (ShortPositionException sbex) {
      //.. handle
    }
  }
  //..
}

The PositionUpdationStrategy is the actual business rule for updating the position of an account and I show the default implementation of the rule.

interface PositionUpdationStrategy {
  void doUpdate(..) throws ShortPositionException;
}

The rule has been modeled purely using the domain language (aka the Ubiquitous Language) and is fairly intuitive to read even by a non-programmer domain guy. Just wear the hat of a domain guy and see if the following codebase sounds domainish.

If the trade is a buy-trade, add to the existing position. If the update results in a short-balance, throw an exception ShortPositionException.

I have highlighted the domain terms above to indicate that the rule implementation is a direct translation of what the domain guy speaks. Colin also agrees to the rule being part of the domain language. Now what about the exception which the rule throws ? We could have named it InsufficientBalanceException and in fact, we did it exactly that way only. Then I was going through a session with one of the domain experts, when he was trying to interpret the correctness of the business rule implementation looking at the codebase. He noted that the short-balance check should ideally raise an exception which indicates that it is related to a short-balance check - once again a part of the domain language. InsufficientBalanceException is too generic a term and should be narrowed to a more fitting domain terminology for the exception here. And thus came the ShortPositionException name, which is part of the Ubiquitous Language. Here is the snippet ..

class DefaultPositionUpdationStrategy
  implements PositionUpdationStrategy {
  //..
  //..

  void doUpdate(BigDecimal amount, TradeType tt)
    throws ShortPositionException {
    if (tt.isBuyTrade()) {
      // add to position
    } else if (isShortBalance(currentPosition, amount)) {
      throw new ShortPositionException(..);
    }
    // subtract from position
  }
}

Look at the interface for the rule, how the checked exception makes the contract much more domain friendly.

Design Tip: Localize your Object Creation Logic

Always try to localize the logic of object creation. This has been one of the big lessons that the Factory design pattern teaches us. This is also a teaching of the Separation of Concerns, which every well-designed software should honor. Don't let your object creation logic get spitted into the processing code. Always program by the interface returned from the factory contract - this way the processing logic remains independent of individual concrete subclasses. All this we knew from the GOF patterns, right ?

Factory with a Strategy

Another benefit of localizing your object creation code is to have the flexibility of plugging in various creation strategies transparently without impacting your clients. In a real life Java application, we were having some performance problems, which could be traced down to creation of a large number of some specific objects in memory. While the objects were not too heavy, the sheer numbers were bringing down the performance curve. Luckily we had the creation logic behind a factory abstraction in Java.

public class BorderObjectFactory {

  //..
  public BorderObject createBorderObject(int type,
    double thickness, Color color, BorderCategory category) {
    //..
  }
  //..
}

And all clients were using the factory for creating BorderObject instances. After a careful review of the creation pattern for BorderObjects, we concluded that we need to implement instance sharing through the Flyweight pattern. This will imply a change in the creation logic, which would have impacted all clients had they not been behind the factory firewall. Here goes the flyweight ..

public class BorderObjectFactory {

  //.. needs to be a singleton

  // pool of flyweights
  private Map<String, BorderObject> pool =
    new HashMap<String, BorderObject>();

  // don't instantiate me directly
  private BorderObjectFactory() {}

  public BorderObject createBorderObject(int type,
    double thickness, Color color, BorderCategory category) {

    //..
    // make the hash key
    String key = new StringBuilder()
        .append(type)
        .append(thickness)
        .append(color.toString())
        .append(category.toString()).toString();

    BorderObject bo = pool.get(key);
    // if it finds from pool, return it
    if (bo != null) {
      return bo;
    }

    // first time entry so create
    bo = new BorderObject(type, thickness, color, category);
    // cache for later use
    pool.put(key, bo);
    return bo;
  }
}

In Ruby, Factories are smell

Ruby has open classes - every class is an object, which can be changed during runtime. If you want to change the creation logic of an object, just open it up and plug in the new logic at runtime. Here is the similar looking Border class in Ruby ..

class Border

  attr_reader :width, :height

  def initialize(wid, ht)
    @width, @height = wid, ht
  end
  # other logic

end

In case you want to change the creation logic through an implementation of the Flyweight based pooling, just define a separate Flyweight module :

module Flyweight
  def Flyweight.included(klass)
    klass.instance_eval %{
      @_pool = {}

      alias :orig_new :new

      def new(*key)
        p "invoking new on " + self.to_s + " args = " + key.to_s
        (@_pool ||= {})[key] ||= orig_new(*key)
      end
    }
  end
end

and mix it in the Border class ..

class Border
  include Flyweight

  # rest similar
  #

end

and clients enjoy the benefit of sharing instances transparently without any change of code. And the best part is they continue to use the same Border.new(..) call to create new Border objects.

b1 = Border.new(2, 10)
b2 = Border.new(4, 20)
b3 = Border.new(4, 20) ## instance sharing with b2
b4 = Border.new(2, 10) ## instance sharing with b1

Depending on the language in which you are designing, the factory implementation may vary in shape and size. In Ruby you don't need to have separate factory abstractions, the factory design pattern is melded into the power of the language. But keeping the object creation logic localized always gives you the extra yard with respect to your design scalability.

Making Classes Unit-Testable

I had been working on the code review of one of our Java projects, when the following snippet struck me as a definite smell in one of the POJOs :

class TradeValueCalculator {
  // ..
  // ..

  public BigDecimal calculateTradeValue(final Trade trade, ..) {
    // ..
    BigDecimal tax = TradeUtils.calculateTax(trade);
    BigDecimal commission = TradeUtils.calculateCommission(trade);
    // .. other business logic to compute net value
  }
  // ..
  // ..
}

What is the problem with the above two innocuous looking Java lines of code ? The answer is very simple - Unit Testability of the POJO class TradeValueCalculator ! Yes, this post is about unit testability and some tips that we can follow to design classes that can be easily unit tested. I encountered many of these problems while doing code review of a live Java project in recent times.

Avoid Statics

When it comes to testability, statics are definitely not your friends. In the above code snippet, the class TradeValueCalculator depends on the implementation of the static methods like TradeUtils.calculateTax(..) and TradeUtils.calculateCommission(..). Any change in these static methods can lead to failures of unit tests of class TradeValueCalculator. Hence statics introduce unwanted coupling between classes, thereby violating the principle of easy unit-testability of POJOs. Avoid them, if you can, and use standard design idioms like composition-with-dependency-injection instead. And while using composition with service components, make sure they are powered by interfaces. Interfaces provide the right level of abstraction for multiple implementations and are much easier to mock while testing. Let us refactor the above snippet to compose using service components for calculating tax and commission :

class TradeValueCalculator {

  // .. to be dependency injected
  private ITaxCalculator taxCalculator;
  private ICommissionCalculator commissionCalculator;

  // ..
  // ..

  public BigDecimal calculateTradeValue(final Trade trade, ..) {
    // ..
    BigDecimal tax = taxCalculator.calculateTax(trade);
    BigDecimal commission = commissionCalculator.calculateCommission(trade);
    // .. other business logic to compute net value
  }
  // ..
  // ..
}

interface ITaxCalculator {
  BigDecimal calculateTax(..);
}

interface ICommissionCalculator {
  BigDecimal calculateCommission(..);
}

We can then have concrete instances of these service contracts and inject them into the POJO TradeValueCalculator :

class DefaultTaxCalculator implements ITaxCalculator {
  // ..
}

class DefaultCommissionCalculator implements ICommissionCalculator {
  // ..
}

Using standard IoC containers like Guice or Spring, we can inject concrete implementations into our POJO non-invasively through configuration code. In Guice we can define Modules that bind interfaces to concrete implementations and use Java 5 annotation to inject those bindings in appropriate places.


// define module to configure bindings
class TradeModule extends AbstractModule {

  @Override
  protected void configure() {
  bind(ITaxCalculator .class)
      .to(DefaultTaxCalculator .class)
      .in(Scopes.SINGLETON);

    bind(ICommissionCalculator .class)
      .to(DefaultCommissionCalculator .class)
      .in(Scopes.SINGLETON);
  }
}

and then inject ..

class TradeValueCalculator {

  // ..
  @Inject private ITaxCalculator taxCalculator;
  @Inject private ICommissionCalculator commissionCalculator;

  // ..
  // ..
}

How does this improve testability of our class TradeValueCalculator ?

Just replace the defined Module by another one for unit testing :

// define module to configure bindings
class TestTradeModule extends AbstractModule {

  @Override
  protected void configure() {
    bind(ITaxCalculator .class)
      .to(MockTaxCalculator .class)
      .in(Scopes.SINGLETON);

    bind(ICommissionCalculator .class)
      .to(MockCommissionCalculator .class)
      .in(Scopes.SINGLETON);
  }
}

What we have done just now is mocked out the service interfaces for tax and commission calculation. And that too without a single line of code being changed in the actual class! TradeValueCalculator can now be unit-tested without having any dependency on other classes.

Extreme Encapsulation

I have come across many abuses of FluentInterfaces, where developers use chained method invocations involving multiple classes. Take this example from this Mock Objects paper, which discusses this same problem :

dog.getBody().getTail().wag();

The problem here is that the main class Dog is indirectly coupled with multiple classes, thereby violating the Law of Demeter and making it totally unsuitable for unit testing. The situation is typically called "The Train Wreck" and has been discussed extensively in the said paper. The takeway from this situation is to minimize coupling with neighbouring classes - couple only with the class directly associated with you. Think in terms of abstracting the behavior *only* with respect to the class with which you collaborate directly - leave implementation of the rest of the behavior to the latter.

Privates also need to be Unit-Tested

There is a school of thought which espouses the policy that *only* public api s need to be unit-tested. This is not true - I firmly believe that all your methods and behaviors need unit testing. Strive to achieve the maximum coverage of unit testing in your classes. Roy Osherove thinks that we may have to bend some of the rules of pure OOD to make our design implementations more testable e.g. by exposing or replacing private instances of objects using interfaces, injection patterns, public setters etc. Or by discouraging default sealing of classes allowing overriding in unit tests. Or by allowing singletons to be replaced in tests to break dependencies. I think, I agree to many of these policies.

Fortunately Java provides a useful access specifier that comes in handy here - the package private scope of access. Instead of making your implementation members *private*, make them *package private* and implement unit test classes in the same package. Doing this, you do not expose the private parts to the public, while allowing access to all unit test classes. Crazy Bob has more details on this. Another useful trick to this may be usage of AOP. As part of unit test classes, you can introduce additional getters through AOP to access the implementation artifacts of your class. This can be done through inter-type declarations, and the test classes can access all private data at gay abandon.

Look out for Instantiations

There are many cases where the class that is being unit tested needs to create / instantiate objects of the collaborating class. e.g.

class TradeController {

  // ..
  // ..

  public void doTrade(TradeDTO dto, ..) {
    Trade trade = new Trade(dto);
    // .. logic for trade
  }
  // ..
}

Increase the testability of the class TradeController by separating out all creation into appropriate factory methods. These methods can then be overridden in test cases to inject creation of Mock objects.

class TradeController {
  TradeDTO dto;

  // ..
  // ..

  public void doTrade() {
    Trade trade = createTrade(dto);
    // .. logic for trade
  }
  // ..

  protected Trade createTrade(TradeDTO dto) {
    return new Trade(dto);
  }
}

and create MockTrade in test cases ..

class TradeControllerTest extends TestCase {

  // ..

  public void testTradeController(..) {
    TradeController tc = new TradeController() {
      protected Trade createTrade(TradeDTO dto) {
        return new MockTrade(dto);
      }
    }
    tc.doTrade();
  }
}

The Factory Method pattern proves quite helpful in such circumstances. However, there are some design patterns like The Abstract Factory, which can potentially introduce unwanted coupling between classes, thereby making them difficult to unit-test. Most of the design patterns in GOF are built on composition - try implementing them using Interfaces in Java, so that they can be easily mocked out. Another difficult pattern is the Singleton - I usually employ the IoC container to manage and unit-test classes that collaborate with Singletons. Apart from static methods, which I have already mentioned above, static members are also problematic cases for unit testing. In many applicatiosn they are used for caching (e.g. ORMs) - hence an obvious problem child for unit testing.