Customizable Business Rules using scripting on the Java Platform

Executing Javascript within the JVM has opened up a new avenue to add to your architectural paradigms. In today's enterprise applications lots of code (validations etc.) needs to be run both for the client and the server. Often we find repetitions in the form of Javascript for the client side code and Java for the server side. With server side Javascript being thrown up as an option in the JVM, the same client side code can be executed within the server using the ScriptEngine of Mustang. Rails people will suggest writing DSLs, which can be converted to Javascript and executed within the JVM.

This post is about using the new architectural paradigm in a project which we executed recently for a publishing house. It was a medium sized project where lots of stuff used to be run in VBScript within ASP.NET. The hapless users wanted to migrate to the JVM and sync up the application with the rest of their MIS running on big Java platforms. No additional functionality was needed to be developed. The VBScript code generates lots of SQL using complex business logic, which ran in the .NET server platforms.

Problem #1 : We do not know what the VBScript code does!

This was the first shocker that we received when visiting our client. The developers who had written the code have left - does this ring any bell? The usual problem of migration - and none of the existing MIS managers where prepared to explain us the flow of the business logic as they exist today.

Option #1 : Go dig it out ..

Struggle through the reams of VBScript code (no code comments - we are dynamic!) and enlighten yourself with the existing business process flow. The only thing that we realized was that all those VBScripts were being used to prepare tons of SQL queries that were later executed in the server. And that the logic code had the usual share of complicated if-then-else that sets up the SQL statements.

Problem #2 : We want externalization of business rules that exists today

The users today have the flexibility of manually tweaking SQL queries and minor business logic in the customary VBScript style of operations. And since some of the business rules change quite frequently, they need to have the same flexibility in the new system as well.

Wow! Rhino!

After shuddering in the thought of cost overrun that Option #1 would incur, we decided to look for an alternative. And there it was - the new scripting engine available for executing Javascript within the JVM. Without going into the details of the business logic, we took the route of converting the VBScript code to Javascript and then spitting out the result to the Rhino interpreter. This will also solve Problem #2 as well, since the same business logic will be available to the users today, only on a different scripting platform. The conversion process was made easier by googling some tools crafted by one generous soul. We wrote a script handler ourselves that we could invoke and execute all scripts from within the main Java application ..

boolean loadScriptFile(String fileName) {
  String jsFile = fileName + "." + scriptBean.getScriptExtension();
  InputStream is;
  Reader reader;

  try {
    is = this.getClass().getResourceAsStream(
      scriptBean.getScriptingSourcePath() + jsFile);
    reader = new InputStreamReader(is);
    engine.eval(reader);
    //..
    //..
  } catch (ScriptException e) {
    //.. handle
  }
  //..
  //..
}


The script handler had an initialization code that set the environment up for all executions to take place in the later stages .. we had developed a utility library which also needed to be bootstrapped ..

private void init() {
  ScriptEngineManager engineMgr =
      new ScriptEngineManager();
  ScriptEngine engine =
      engineMgr.getEngineByName(scriptBean.getScriptingEngine());

  try {
    // execute utility library
    InputStream is = this.getClass().getResourceAsStream(
      scriptBean.getScriptingSourcePath() + "defaultScripts.js");
    engine.eval(new InputStreamReader(is));

    // set up values in the state of the ScriptEngine
    engine.put(SCRIPT_RESULT_IDENTIFIER, engineResultList);
    engine.put(SCRIPT_MAP_RESULT_IDENTIFIER, engineResultMap);

  } catch (ScriptException e) {
    //.. handle
  }
}


In the above example snippet, the put() method allows assigning any variable used in the scripting environment from within Java code. Another alternative will be to use Bindings. Suppose we have a list of names in a java variable listOfNames, which we need to provide to a script to form an SQL statement ..

engine.put("namesKey", listOfNames);


and the script to be loaded contains the following ..

function performancePubReg() {
  //.. form the sql
  sqlStr = "select ...
  sqlStr = sqlStr + "where master.name in " + makeString(namesKey);
  //..
}


In course of the application, wherever we needed to execute queries, these sql queries were fetched by load-eval of Javascripts using the script handler implementation.

A Useful Extension to the Architectural Toolbox

In general, this approach proved to be a useful paradigm for small to medium sized migration projects. In an ideal scenario, these customizable business rules which generate SQL queries could be coded up as a DSL for the business users and then appropriately converted to any scripting platform. And the best part is that you need not always dig into the details of legacy code in situations like ours.

Domain Driven Design : Handling Alternate Business Scenarios

In one of my earlier posts, I had discussed about domain driven exceptions and how alternate business flows can be modeled using checked exceptions of Java. Recently there has been a thread going on in domain-driven-design discussion forum on a similar subject on the rationale of throwing in case of violations of business rules.

Business Rule Violation - violates what ?

Typically when a business rule is violated, the domain has a set of alternative rules, which kicks in to handle the situation. It's not that we have a fatal exception that needs to fall flat on the user's face. When I have an account.debit(amount) failing the balance check, there is an alternate path that the domain prescribes and which needs to be modeled as part of the implementation. So, instead of violation, I prefer to use the term alternate business scenario, since it's always choosing between one of the many pathways through which the model flows in course of the domain lifecycle. One of them is the most commonly executed path, while the others may be less frequently traversed, but very much a part of the domain logic.

In the last post, I had talked about Java's checked exceptions as a possible means to handle what I call alternate business scenarios. Many people has, since then, suggested the option of using the Specification pattern of domain driven design to handle alternate execution paths in the domain model. In this post, let us see how much we can improve using the Specification pattern both with respect to encapsulating the business constraints and also handling alternate sequence of execution on violation of those constraints.

Using the Specification pattern to handle alternate business scenarios

I take the example from my previous post, modified slightly to reflect the current understanding.

The class Position is the abstraction for an account's balance or position for particular instruments. The example is from the financial services domain where accounts are managed by the firm and updated with information from the trading and settlement systems.

class Position {
  private Account account;
  private Instrument instrument;
  private double balance;
  //..
  private PositionUpdationStrategy strategy;
  //..

  void doUpdate(..) {
    strategy.doUpdate(..);
  }
}


The method Position#doUpdate(..) updates the Position with the new information, which gets reflected in the state of the object. And we have a Specification class which validates if a Position is short - this class encapsulates the domain knowledge for determining a short position for an account.

class ShortPositionSpecification
    extends PositionSpecification {
  boolean isSatisfiedBy(Position pos) {
    //..
    //.. short position logic
    //return true if short position
  }
}


Now, here is a service class which uses the specification to determine short position and act accordingly.

class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    Position current = getPosition(..);
    current.doUpdate(..);

    //.. invoke the specification
    if (new ShortPositionSpecification().isSatisfiedBy(current)) {
      // .. alternate business path
      //.. do what ?
      return null;
    }
    return current;
  }
  //..
  //..

  Position getPosition(..) {
    //..
  }
}


Now, we have a couple of issues here :-

• What do we return from PositionManager#updatePosition(..) if the position is short and we need to execute the alternate business logic ? Null ? Sounds yucky - explicit null checks need to be done by clients of PositionManager, which looks smelly.


• One option may be to have a flag in Position which indicates short-position, and have it set when the Specification indicates short position. Once again the clients of PositionManager need to do an if-then-else check to determine the short position of the account and handle the alternate business logic. Yuck! Still smelly ..

Let us put our Position model on some more steroids. The domain language says that in case of existing delinquent positions, the method PositionManager#updatePosition(..) should bail out early without doing any updates. This adds yet another PositionSpecification to our module ..

class DelinquentPositionSpecification
    extends PositionSpecification {
  boolean isSatisfiedBy(Position pos) {
    //..
    //.. delinquent position logic
    //return true if delinquent position
  }
}


and add yet another check in PositionManager#updatePosition(..) :

class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    current = getPosition(..);

    if (new DelinquentPositionSpecification().isSatisfiedBy(current)) {
      //.. alternate business path
      //.. do what ?
      return null;
    }

    current.doUpdate(..);
    if (new ShortPositionSpecification().isSatisfiedBy(current)) {
      // .. alternate business path
      //.. do what ?
      return null;
    }
    return current;
  }
  //..
  //..

  Position getPosition(..) {
    //..
  }
}


Umm .. Things start getting a bit messy and procedural. In both the cases, we return null, since we do not have any typed abstraction indicating the alternate path of execution. Instead of nulls, we can think of implementing the Null Object pattern, NullPosition. While this will make things slightly better, still the client code has to rely on checking flags to determine the exact alternate pathway to take. The Specification pattern encapsulates the constraints (as it should be), but it does not provide any abstraction for identifying the alternate flow of control. The caller still has to process null returns and differentiate them based on some flags (which may be multiple). Here is what the client of PositionManager can do ..

Position pos = PositionManager.getInstance().updatePosition(..);
if (pos == null) {
  if (pos.isDelinquent()) {
    //.. handle delinquent position
  }
  if (pos.isShort()) {
    //.. handle short position
  }
  //.. any other ?
}
//..
//..


This is definitely not the OO way of handling things! Also, all the procedural if-then-else code violate the principle of Making Implicit Concepts Explicit, as specified by Eric Evans in his DDD book. The alternate business flow for short position or delinquent position is buried within the stream of if-then-else statements and flag manipulations. None of the abstractions publish any explicit contract that indicates the alternate pathway which our execution can take. Ideally we would like to have our apis publish all alternative paths explicitly, so that the api user can figure out all of them before programming-to-the-api set.

Can Polymorphism help ?

I can sense several fans of polymorphism raising their heads with the idea of returning polymorphic variants of Position from the updatePosition(..) method. This will never scale and we are bound to have an explosion of subclasses with this approach. Let us look elsewhere ..

Flamebait : The love and hate of Checked Exceptions

In my last post, I had indicated how checked exceptions in Java help us provide typed abstractions publishing the alternate execution paths as part of the contract. And, as we saw above, Specifications allow us to encapsulate the business rule itself, whose violation leads to the execution of those alternate paths. Combine the two, and we can have a better typed model for both the cause and the execution of the alternate flows.

Move the invocation of the specifications to the source instead of checking them as post-conditions in PositionManager#updatePosition(..) ..

class PositionManager {
  //..
  //..

  //.. explicit publication
  Position getPosition(..)
      throws DelinquentPositionException {
    //.. find the current position
    current = ..
    if (new DelinquentPositionSpecification().isSatisfiedBy(current)) {
      throw new DelinquentPositionException(..);
    }

  }
}


and short-position handling moves to the strategy ..

class DefaultPositionUpdationStrategy
    implements PositionUpdationStrategy {

  void doUpdate(Position position)
    throws ShortPositionException {
    //.. check for short position
    if (new ShortPositionSpecification().isSatisfiedBy(position)) {
      throw new ShortPositionException(..);
    }
    //..
  }
}


Now PositionManager#updatePosition(..) becomes cleaner with lots of specification checking moving inside the guts of respective classes ..

class PositionManager {
  //..
  //..
  Position updatePosition(..) {
    try {

      //.. normal flow
      current = getPosition(..);
      current.doUpdate(..);
      return current;

    } catch (ShortPositionException spex) {
      //.. short position handling
    } catch (DelinquentPositionException dpex) {
      //.. delinquent position handling
    }
  }
  //..
  //..
}


Does this look better than the procedural if-then-else code above ? Checked exceptions are typed contracts that the caller needs to honor and handle. When I am designing typed apis, I consider them to be my friend. There is a community out there divided on the benefits of checked exceptions - recently we have seen lots of flamebaits once again on this very topic. And there are enough examples to showcase on either side of the argument. I find checked exceptions very useful while handling alternate business logic paths in my application. There are many situations where checked exceptions look like getting into your way, resulting in too much try-catch-finally in your codebase. But in handling alternate business scenarios, checked exceptions serve the purpose of advertising explicit business rule violations to the caller. They are part of the api and appear in the Javadoc - the user of your api gets to be aware of all alternate business scenarios that the domain model handles. And this makes the implicit concept explicit.

Can you differentiate between an Abstraction and a Lie ?

In the June 2007 issue of Communications of the ACM, David Parnas differentiates between an abstraction and a lie. He mentions that abstractions that embody unrealistic assumptions represent nothing at all. He learnt the correct definition of abstraction from the late E. W. Dijkstra :

An abstraction is one thing that represents several real things equally well.

And he concludes ..

Finding the simplest model that is not a lie is the key to better software design.

So true, and how lucidly expressed! Go read the full article here. [Membership required for full text].

In Defense of Static Typing

Just thought it may be relevant to bring this thread up in context of the debate going on in the Artima forum between the camps of static and dynamic programming languages. The languages under discussion have changed, the thought leaders have changed, however, the topics debated have remained the same .. the verbosity of the static type system, benefits (if any) of type inferencing, power of dynamic typing and the defense of the straw man argument.

Here are some snippets :-

From Matthias Felleisen (of The Little Schemer and the rest of the series fame)

Static types are necessary
(1) to establish basic properties of software and
(2) to establish absolute yet cheap abstraction boundaries between modules.

and more of some of the basic values of static typing ..

With types, you can say something like this at a module boundary:

there is a type t, here are some instances of type t, i even give you functions to create t's, and work with t's, but i won't tell you what they are implemented in.

Now you or your partners have no way of corrupting these values. And you know what, they are actually integers, and if you had known, you could have used all kinds of numeric hacks to compute values. But fortunately you didn't, because I, the module writer, can now change t to a hash table and do all kinds of nice things with it.

What's the cost for that? Zero. Yeap, in a statically typed world, there is no run-time cost.

and of course there is Paul Graham on the opposite camp .. regarding static typing ..

It's like the way the military does things. Making everyone fill out forms to do anything certainly prevents mistakes. But it also inhibits innovation terribly.

I'm not saying the military is mistaken here. There are places where restrictions are good-- particularly when you don't trust individual people not to screw up. So requiring type declarations or other assertions (e.g. all the boilerplate that seems to be required in Java) could be a good idea in some big company where the programmers are guys who walked in off the street after Learning Java in 21 Days. But at the other end of the continuum, there is also a role for languages that put as few constraints on the programmer as possible.

Enjoy the complete thread here !

I think I would like to have some militarism in contracts when I am executing a big-bang enterprise project. As Cedric has rightly mentioned,

nothing beats a dynamic language to write a prototype, but if you know that the code will have to be maintained and that it will be in use for several years, the initial cost of using a statically typed language is an investment that is very quickly amortized...

Types are elements promoting Design By Contract at the lowest level and I would not want to lose the enforcements that they bring to my higher level abstractions. And finally, the refactoring support that I get from statically typed languages is way too precious to lose when I am working on a large codebase.

How do you decide when to use Aspects ?

Any new technology comes with its own powerbag and it depends upon us programmers to make meaningful and responsible use of the power. New age programming languages are riding their power continuum - it is upto us to stay a blub programmer, be an average coder or try to be the hacker. But, whatever, you decide to do, make your decision an informed one - be responsible to use the power in the proper place and perspective.

Aspect oriented programming is a new buzzword that has been popularized more recently through its integration with the Spring framework. The Spring guys have done a great job in bringing a difficult technology to the masses through its usual style of declarative programming. Spring AOP takes a lot of pain out of you by offering a greatly simplified programming model to have method interceptions baked in your codebase.

But making a technology look simpler has its obvious consequences of being misused. There appears to be lots of cases with programmers where they are using aspects, when good old simple Java design patterns, make a more appropriate cut. In the last couple of months, I found Spring AOP's method interception being used in many instances (including this one in an InfoQ article) when good old decorators could have solved the problem. The basic problem which the developer was trying to solve was to wrap some command with pre- and post- advices. The AOP based solution would make sense only if the same strategy needs to be repeated in multiple places and invocations of the command, which would otherwise have resulted in lots of boilerplates littering the codebase. Otherwise, a command and a bunch of decorators can provide a scalable solution to this ..

// the generic command interface
public interface ICommand {
void execute(final Object object);
}

// and a decorator interface for decorating the command
public abstract class Decorator implements ICommand {

// the command to decorate
private ICommand decorated;

public Decorator(final ICommand decorated) {
this.decorated = decorated;
}

public final void execute(Object object) {
pre();
decorated.execute(object);
post();
}

protected final ICommand getDecorated() {
return decorated;
}

// the pre-hook
protected abstract void pre();

// the post-hook
protected abstract void post();
}

// my custom command class
public class FileCommand implements ICommand {
//.. custom command
}

// my first custom decorator
public class MyDecorator_1 extends Decorator {

public MyDecorator_1(final ICommand command) {
super(command);
}

@Override
protected void post() {
//.. custom post hook
}

@Override
protected void pre() {
//.. custom pre hook
}
}

// another custom decorator
public class MyDecorator_2 extends Decorator {

public MyDecorator_2(final ICommand command) {
super(command);
}

@Override
protected void post() {
//.. custom post hook
}

@Override
protected void pre() {
//.. custom pre hook
}
}
// stack up the decorators
new MyDecorator_2(
new MyDecorator_1(
new FileCommand(...))).execute(..);


Use Aspects to address crosscutting concerns only

I use aspects as a last resort, when all options fail to address the separation of concerns that I am looking for in my code organization. Aspects help avoid the code tangle by identifying joinpoints through pointcuts and helping define advices that will be applied to the joinpoints. But I use them only when all traditional Java tools and techniques fail to localize my solution. Aspects bring in a separate machinery, the heavy lifting of bytecode instrumentation. Spring AOP is, however, pure Java, but based on dynamic proxies, which have their own limitations in method interceptions and performance penalties (however small) of creating proxies on every call. Spring AOP has less magic than pure AOP - but use the technology only if it is the right choice for the problem at hand. At least, recently, I find many instances of this wonderful technology being misused as a result of sheer over-engineering. Often, when we see nails, everything starts to look like a hammer. Keep your solution simple, ensure that it solves the immediate problem at hand and gives you adequate options for extensibility.

Domain Driven Design - What's in an Exception Name ?

Colin Jack does not feel that exception names should be part of the domain language. In one of my blog posts, he comments :

I also don't see the exceptions are part of the domain language, instead I see the associated rules as being part of the domain language. So for me the rule name should become part of the domain language but the exceptions do not have to be, its the rules that matter to me when discussing things with the domain experts.

Colin, I have to disagree with you. Here is an example from a real life project (as per your request in my blog comments) and a real discussion session with one of the domain experts.

Here is a snippet from a Capital Market Solutions codebase. The codebase is replete with domain specific terms, as it should be in a domain model implementation, starters (and unstarters) of the capital market domain are requested to look up relevant information for background of the terminologies .. apologies :-(

Class PositionManager is the domain abstraction for managing the Position of an account. Note that Position is the domain term which indicates the balance of the account.

class PositionManager {
  //..
  //..
  void updatePosition(Instrument ins, Date tradeDate,
    BigDecimal amount, TradeType tt, ..) {
    Position currentPosition = getPosition(acc, ins, tradeDate);
    currentPosition.doUpdate(amount, tt);
  }
  //..
}

Now have a look at the method Position#doUpdate invoked above. Here the update is done through a business rule modeled as a Strategy design pattern. The rule can have multiple implementations, hence modeled as an interface. Again we use domain driven principles here to isolate the execution of a process from the rules on which it operates. Note the checked exception which the method catches - ShortPositionException, we will come back to the name in just a moment. Keep reading ..

Here is the definition :

class Position {
  private PositionUpdationStrategy strategy;

  //..
  //..
  void doUpdate(BigDecimal amount, TradeType tt) {
    try {
      strategy.doUpdate(amount, tt);
    } catch (ShortPositionException sbex) {
      //.. handle
    }
  }
  //..
}

The PositionUpdationStrategy is the actual business rule for updating the position of an account and I show the default implementation of the rule.

interface PositionUpdationStrategy {
  void doUpdate(..) throws ShortPositionException;
}

The rule has been modeled purely using the domain language (aka the Ubiquitous Language) and is fairly intuitive to read even by a non-programmer domain guy. Just wear the hat of a domain guy and see if the following codebase sounds domainish.

If the trade is a buy-trade, add to the existing position. If the update results in a short-balance, throw an exception ShortPositionException.

I have highlighted the domain terms above to indicate that the rule implementation is a direct translation of what the domain guy speaks. Colin also agrees to the rule being part of the domain language. Now what about the exception which the rule throws ? We could have named it InsufficientBalanceException and in fact, we did it exactly that way only. Then I was going through a session with one of the domain experts, when he was trying to interpret the correctness of the business rule implementation looking at the codebase. He noted that the short-balance check should ideally raise an exception which indicates that it is related to a short-balance check - once again a part of the domain language. InsufficientBalanceException is too generic a term and should be narrowed to a more fitting domain terminology for the exception here. And thus came the ShortPositionException name, which is part of the Ubiquitous Language. Here is the snippet ..

class DefaultPositionUpdationStrategy
  implements PositionUpdationStrategy {
  //..
  //..

  void doUpdate(BigDecimal amount, TradeType tt)
    throws ShortPositionException {
    if (tt.isBuyTrade()) {
      // add to position
    } else if (isShortBalance(currentPosition, amount)) {
      throw new ShortPositionException(..);
    }
    // subtract from position
  }
}

Look at the interface for the rule, how the checked exception makes the contract much more domain friendly.

How do you define a Programmer ?

Stu writes :

I want to live in a world where language preference does not define a programmer. We'll write code in Ruby, or Scheme, or Scala, or Erlang. And we can all live in harmony on a JVM anywhere.

Wishful thinking! Add to this list .. Groovy, Scala and of course Java. I am sure this calls for a big change in the dynamics which control the programmer hiring and project staffing ecosystem today. And spare a thought for the team who will maintain all these codes many years after they have been written, refactored and refactored mercilessly.

Where are we headed to ?

Why Thinking in Ruby Matters

I was going through a great article by Jim Weirich where he discusses why it is important to think in Ruby, when you are programming in Ruby. He goes on to describe an implementation of the Adapter design pattern in Ruby and compares the implementation with a similar (but much scaled down) version in Java. Ruby gives you the power to express more in less words - conciseness of syntax, along with great metaprogramming abilities leads to the expressivity that Ruby offers.

Consider the Strategy design pattern, which allows encapsulation of an algorithm and allows users to vary them independent of the context. There is nothing object-oriented about a Strategy design pattern. It so happens that the GOF book describes it in the context of an object-oriented language, C++. Stated succinctly, the design pattern espouses yet another best practice of software design, separation of concerns, by decoupling the implementation of an algorithm from the context.

In Java, the Strategy design pattern is implemented through the composition of a polymorphic hierarchy. The strategy is composed into the context through an interface, which can support multiple implementations transparently. Here is an example from a Payroll application, where the algorithm for calculation of a salary is encapsulated into a strategy :

class SalaryComputation {
  // injected through dependency injection
  private SalaryComputationStrategy strategy;
  //..
  //..

  public BigDecimal compute(..) {
    //..
    //..
    return strategy.compute(..);
  }
  //..
}

Here SalaryComputationStrategy is an interface which can have multiple implementations. Typically in a Java project, the implementations are injected transparently through a DI container like Spring or Guice.

interface SalaryComputationStrategy {
  BigDecimal compute(..);
}

class DefaultSalaryComputationStrategy
  implements SalaryComputationStrategy {
    //.. implementation
}

class SpecialSalaryComputationStrategy
  implements SalaryComputationStrategy {
    //.. implementation
}

One of the immediate consequences of the above implementation in Java is the number of classes and the inheritance hierarchies involved in the implementation. Looks like an accidental complexity in implementation, but this is quite a normal way of dealing with a nail when all you have is a hammer.

How do you encapsulate an algorithm and ensure flexible swapping of implementations in a language like Ruby that offers powerful syntax extensibility ? Remember, Ruby is also object oriented, and the above Java implementation can be translated with almost no effort using equivalent Ruby syntaxes. And we have a Strategy implementation in Ruby. Can we do better ?

Abstraction Abstraction!

Abstraction is the key - when you program in a language, always choose the best form of abstraction that suits the problem you are modeling. In a strategy design pattern, all you are encapsulating is an algorithm, which is, by nature a functional artifact. The very fact that Java or C++ does not support higher order functions (anonymous inner classes and function pointers are for the destitutes) had forced us to use encapsulating objects as holders of algorithms. And that led to the plethora of class hierarchies in the Java implementation. Ruby supports higher order functions in the form of blocks and coroutine support in the form of yields - keep reading and hope for a better level of abstraction support.

class SalaryComputation
  def compute
    ## fetch basic, allowances
    @basic = ..
    @allowance = ..

    ## fixed logic goes here

    ## coroutine call for the specific algorithm
    yield(@basic, @allowance)
  end
  ##
end

and the algorithm can be injected inline by the client through a Ruby block :

SalaryComputation.new.compute {
  |b, a|
  tax = (b + a) * 0.2
  b + a - tax
}

and we do not have to define a single extra class for the strategy. The strategy is nicely embedded inline at the caller's site. Another client asking to use a different algorithm can supply a different block at her call site.

What if I want to use the same default algorithm for different clients, yet keeping the flexibility of plugging in multiple implementations ? DRY it up within a Ruby Module and use the power of Ruby mixins :

module DefaultStrategy
  def do_compute(basic, allowances)
    tax = basic * 0.3
    basic + allowances - tax
  end
end

class SalaryComputation
  include DefaultStrategy ## mixin

  def initialize basic
    @basic = basic
    @allowances = basic * 0.5
  end

  def compute
    do_compute(@basic, @hra)
  end
end

And for some places where I would like to inject a special strategy, define another Module for the same and extend the SalaryComputation class during runtime :

module SpecialStrategy
  def do_compute(basic, allowances)
    tax = basic * 0.2
    basic + allowances - tax
  end
end

s = SalaryComputation.new(100)
s.extend SpecialStrategy
puts s.compute

Handling Fine Grained Variations

Suppose a part of the algorithm is fixed, while the computation of the tax only varies. The typical way to handle this in Java will be through the Template Method pattern. When designing the same in Ruby, the mechanism melds nicely into the language syntax :

def compute
  @basic + @allowances - yield(@basic)
end

and the identity of the pattern disappears within the language. Ruby supports fine grained variations within algorithms through powerful language syntax, which can only be done using extra classes in Java.

And finally ..

Keep an eye on the Ruby open classes feature. You can rip open any class and make changes either at the class level or at a single instance level. This feature offers the most coarse grained way to implement strategies in Ruby.

## open up an existing class
class SalaryComputation

  ## alias for old method
  ## you may need it also
  alias :compute_old :compute

  def compute
    ## new strategy
  end
end

So, how many ways does Ruby offer to implement your Strategy Design Pattern ?

Design Tip: Localize your Object Creation Logic

Always try to localize the logic of object creation. This has been one of the big lessons that the Factory design pattern teaches us. This is also a teaching of the Separation of Concerns, which every well-designed software should honor. Don't let your object creation logic get spitted into the processing code. Always program by the interface returned from the factory contract - this way the processing logic remains independent of individual concrete subclasses. All this we knew from the GOF patterns, right ?

Factory with a Strategy

Another benefit of localizing your object creation code is to have the flexibility of plugging in various creation strategies transparently without impacting your clients. In a real life Java application, we were having some performance problems, which could be traced down to creation of a large number of some specific objects in memory. While the objects were not too heavy, the sheer numbers were bringing down the performance curve. Luckily we had the creation logic behind a factory abstraction in Java.

public class BorderObjectFactory {

  //..
  public BorderObject createBorderObject(int type,
    double thickness, Color color, BorderCategory category) {
    //..
  }
  //..
}

And all clients were using the factory for creating BorderObject instances. After a careful review of the creation pattern for BorderObjects, we concluded that we need to implement instance sharing through the Flyweight pattern. This will imply a change in the creation logic, which would have impacted all clients had they not been behind the factory firewall. Here goes the flyweight ..

public class BorderObjectFactory {

  //.. needs to be a singleton

  // pool of flyweights
  private Map<String, BorderObject> pool =
    new HashMap<String, BorderObject>();

  // don't instantiate me directly
  private BorderObjectFactory() {}

  public BorderObject createBorderObject(int type,
    double thickness, Color color, BorderCategory category) {

    //..
    // make the hash key
    String key = new StringBuilder()
        .append(type)
        .append(thickness)
        .append(color.toString())
        .append(category.toString()).toString();

    BorderObject bo = pool.get(key);
    // if it finds from pool, return it
    if (bo != null) {
      return bo;
    }

    // first time entry so create
    bo = new BorderObject(type, thickness, color, category);
    // cache for later use
    pool.put(key, bo);
    return bo;
  }
}

In Ruby, Factories are smell

Ruby has open classes - every class is an object, which can be changed during runtime. If you want to change the creation logic of an object, just open it up and plug in the new logic at runtime. Here is the similar looking Border class in Ruby ..

class Border

  attr_reader :width, :height

  def initialize(wid, ht)
    @width, @height = wid, ht
  end
  # other logic

end

In case you want to change the creation logic through an implementation of the Flyweight based pooling, just define a separate Flyweight module :

module Flyweight
  def Flyweight.included(klass)
    klass.instance_eval %{
      @_pool = {}

      alias :orig_new :new

      def new(*key)
        p "invoking new on " + self.to_s + " args = " + key.to_s
        (@_pool ||= {})[key] ||= orig_new(*key)
      end
    }
  end
end

and mix it in the Border class ..

class Border
  include Flyweight

  # rest similar
  #

end

and clients enjoy the benefit of sharing instances transparently without any change of code. And the best part is they continue to use the same Border.new(..) call to create new Border objects.

b1 = Border.new(2, 10)
b2 = Border.new(4, 20)
b3 = Border.new(4, 20) ## instance sharing with b2
b4 = Border.new(2, 10) ## instance sharing with b1

Depending on the language in which you are designing, the factory implementation may vary in shape and size. In Ruby you don't need to have separate factory abstractions, the factory design pattern is melded into the power of the language. But keeping the object creation logic localized always gives you the extra yard with respect to your design scalability.

Parameterizing Test Methods - Revisited

In my last post on refactoring and unit tests, Steve Freeman noted in one of his blog comments ..

I can't see the point in the data provider. Why don't you just call the method three times with different arguments?

In the process of refactoring, I had used TestNG's parameterized test method feature to decouple test data from test methods. While invoking a test method multiple times with different parameter gives me the same functional result, parameterized test methods provide great scalability and a neat separation between the test logic and test data suite. Complex business methods may involve lots of complicated calculations and conditionals - a unit test of the method needs to ensure all conditionals are exercised to ensure full coverage. This may result in lots of data combinations, which if handled directly through separate invocations of the same test methods may result in real explosion of test cases.

The feature of parameterizing test methods in TestNG through DataProviders provide a great way to DRY up the test methods. I can have the test data combination either in testNG.xml or in separate factory methods with specific signatures that provide huge extensibility. The test method can then have parameters and an annotation that specify the data source. Take the following example from the earlier post :

public class AccruedInterestCalculatorTest extends MockControl {
  //..
  @DataProvider(name = "test1")
  public Object[][] createAccruedInterestCalculationData() {
    return new Object[][] {
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(1000), BigDecimal.valueOf(30)},
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(60)},
      {30, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(90)},
    };
  }

  //..

  // the test method now accepts parameters
  //..
  @Test(dataProvider = "test1")
  public void normalAccruedInterest(int days,
    BigDecimal rate, BigDecimal principal, BigDecimal interestGold) {
    new CalculatorScaffold()
      .on(principal)
      .forDays(days)
      .at(rate)
      .calculate()
      .andCheck(interestGold);
  }
  //..
}

The calculation of the accrued interest is an extremely complicated business method which needs to consider lots of alternate routes of computation and contextual data. A typical data set for testing the calculation effectively will result in repeating the test method invocation lots of times. Instead of that, the parameterization technique allows me to declare all the varying components as parameters of the test method. And the data source comes from the method createAccruedInterestCalculationData(), suitably annotated with a data source name.

Making the test method parameterized has the additional side-effect of decoupling the data set from the logic - the entire data set can be populated by non-programmers as well, typically domain guys who can enrich the test coverage by simply putting in additional data points in the factory method that generates the data. We have actually used this technique in a real life project where domain experts took part in enriching parameterized unit test cases through data fulfillment in data provider methods.

There's actually more to it. The data provider method can be used to fetch data from other complicated data sources as well e.g. database, XML etc. and with logic to generate data also. All these can be encapsulated in the DataProvider methods and transparently fed to the test methods. The logic of test data generation is completely encapsulated outside the test method. I found this feature of TestNG a real cracker ..

Refactoring Unit Test Methods to speak the Domain

I do TDD, but I do not start writing unit tests before writing production code. It's not that I didn't try - I gave it an honest attempt for quite some time, before I gave up. Somehow the approach of writing tests first does not seem intuitive to me. At least I need to figure out the collaborators and the initial subset of public contracts before my TestNG plugin kicks in. Since then, it's all a game of collaborative pingpong between my production code and test code. Having said that, I honestly believe that an exhaustive unit test suite is no less important than the code itself. I take every pain to ensure that my unit tests are well organized, properly refactored and follow all principles of good programming that I espouse while writing the actual production code.

Refactoring is one practice that I preach, teach and encourage vigorously to my teammates. Same for unit tests - if I strive to make my production code speak the language of the domain, so should be my unit tests. This post is about a similar refactoring exercise to make unit tests speak the DSL. At the end of this effort of vigorous iterations and cycles of refactoring, we could achieve a well engineered unit test suite, which can probably be enhanced by the domain guys with minimum of programming knowledge.

The Class Under Test

The following is the fragment of a simplified version of an AccruedInterestCalculator, a domain class, which calculates the interest accrued for a client over a period of time. For brevity, I have a very simplified version of the class with only very simple domain logic. The idea is to develop a fluent unit test suite for this class that speaks the domain language through an iterative process of refactoring. I will be using the artillery consisting of the state of the art unit testing framework (TestNG), a dynamic mocking framework (EasyMock) and the most powerful weapon in programming - merciless refactoring.

public class AccruedInterestCalculator {
  //..
  //..
  private IAccruedDaysCalculator accruedDaysCalculator;
  private IInterestRateCalculator interestRateCalculator;

  public final BigDecimal calculateAccruedInterest(final BigDecimal principal)
    throws NoInterestAccruedException {
    int days = accruedDaysCalculator.calculateAccruedDays();
    if (days == 0) {
      throw new NoInterestAccruedException("Zero accrual days for principal " + principal);
    }
    BigDecimal rate = interestRateCalculator.calculateRate();
    BigDecimal years = BigDecimal.valueOf(days).divide(BigDecimal.valueOf(365), 2, RoundingMode.UP);
    return principal.multiply(years).multiply(rate);
  }

  AccruedInterestCalculator setAccruedDaysCalculator(IAccruedDaysCalculator accruedDaysCalculator) {
    this.accruedDaysCalculator = accruedDaysCalculator;
    return this;
  }

  AccruedInterestCalculator setInterestRateCalculator(IInterestRateCalculator interestRateCalculator) {
    this.interestRateCalculator = interestRateCalculator;
    return this;
  }
}

and the two collaborators ..

public interface IAccruedDaysCalculator {
  int calculateAccruedDays();
}

public interface IInterestRateCalculator {
  BigDecimal calculateRate();
}

Mocks are useful, but Noisy ..

I am a big fan of using mocks for unit testing - EasyMock is my choice of dynamic mocking framework. Mocks provide the most seamless way of handling collaborators while writing unit tests for a class. However, often, I find mocks introducing lots of boilerplate codes which need to be repeated for every test method that I write ..

public class AccruedInterestCalculatorTest {
  protected IMocksControl mockControl;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
  }

  @Test
  public void normalAccruedInterestCalculation() {
    IAccruedDaysCalculator acalc = mockControl.createMock(IAccruedDaysCalculator.class);
    IInterestRateCalculator icalc = mockControl.createMock(IInterestRateCalculator.class);

    expect(acalc.calculateAccruedDays()).andReturn(2000);
    expect(icalc.calculateRate()).andReturn(BigDecimal.valueOf(0.5));

    mockControl.replay();

    BigDecimal interest =
      new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(BigDecimal.valueOf(2000.00));

    mockControl.verify();
  }
}

Refactoring! Refactoring!

Have a look at the above test method - the mock framework setup and controls pollute the actual business logic that I would like to test. Surely, not a very domain friendly approach. As Howard has pointed to, in one of his NFJS writings, we can abstract away the mock stuff into separate methods within the test class, or still better in a separate MockControl class altogether. This makes the actual test class lighter and free of some of the noise. Here is the snapshot after one round of refactoring ..

// mock controls delegated to class MockControl
public class AccruedInterestCalculatorTest extends MockControl {

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
  }

  @Test
  public void normalAccruedInterestCalculation() {
    IAccruedDaysCalculator acalc = newMock(IAccruedDaysCalculator.class);
    IInterestRateCalculator icalc = newMock(IInterestRateCalculator.class);

    expect(acalc.calculateAccruedDays()).andReturn(2000);
    expect(icalc.calculateRate()).andReturn(BigDecimal.valueOf(0.5));

    replay();

    BigDecimal interest =
      new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(BigDecimal.valueOf(2000.00));

    verify();
  }
}

and the new MockControl class ..

public abstract class MockControl {
  protected IMocksControl mockControl;

  protected final <T> T newMock(Class<T> mockClass) {
    return mockControl.createMock(mockClass);
  }

  protected final void replay() {
    mockControl.replay();
  }

  protected final void verify() {
    mockControl.verify();
  }
}

The test class AccruedInterestCalculatorTest is now cleaner and the test method is lighter in baggage from the guts of the mock calls. But still it is not sufficiently close to speaking the domain language. One of the litmus tests which we often do to check the domain friendliness of unit tests is to ask a domain guy to explain the unit test methods (annotated with @Test) and, if possible, to enhance them. This will definitely be a smell to them, with the remaining litterings of mock creation and training still around. The domain guy in this case, nods a big NO, Eclipse kicks in, and we start the next level of refactoring.

One thing strikes me - each test method is a composition of the following steps :

1. create mocks


2. setup mocks


3. replay


4. do the actual stuff


5. verify

And refactoring provides me the ideal way to scaffold all of these behind fluent interfaces for the contract that we plan to test. How about a scaffolding class that encapsulates these steps ? I keep the scaffold as a private inner class and try to localize all the mockeries in one place. And the scaffold can always expose fluent interfaces to be used by the test methods. Here is what we have after a couple of more rounds of iterative refactoring ..

public class AccruedInterestCalculatorTest extends MockControl {
  private IAccruedDaysCalculator acalc;
  private IInterestRateCalculator icalc;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
    acalc = newMock(IAccruedDaysCalculator.class);
    icalc = newMock(IInterestRateCalculator.class);
  }

  // the scaffold class encapsulating all mock methods
  private class CalculatorScaffold {
    private BigDecimal principal;
    private int days;
    private BigDecimal rate;
    private BigDecimal interest;

    CalculatorScaffold on(BigDecimal principal) {
      this.principal = principal;
      return this;
    }

    CalculatorScaffold forDays(int days) {
      this.days = days;
      return this;
    }

    CalculatorScaffold at(BigDecimal rate) {
      this.rate = rate;
      return this;
    }

    CalculatorScaffold calculate() {
      expect(acalc.calculateAccruedDays()).andReturn(days);
      expect(icalc.calculateRate()).andReturn(rate);

      replay();

      interest =
        new AccruedInterestCalculator()
        .setAccruedDaysCalculator(acalc)
        .setInterestRateCalculator(icalc)
        .calculateAccruedInterest(principal);

      verify();
      return this;
    }

    void andCheck(BigDecimal interestGold) {
      assert interest.compareTo(interestGold) == 0;
    }
  }

  //..
  //.. the actual test methods

  @Test
  public void normalAccruedInterest() {
    new CalculatorScaffold()
      .on(BigDecimal.valueOf(1000.00))
      .forDays(days)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(30.0));
  }

  @Test
  public void normalAccruedInterest() {
      new CalculatorScaffold()
      .on(BigDecimal.valueOf(2000.00))
      .forDays(days)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(60.0));
  }

  //..
  //.. other methods
}

This class has test methods which now look more closer to the domain, with all mock controls refactored away to the scaffold class. Do you think the domain guys will be able to add more methods to add to the richness of coverage ? The accrued interest calculation is actually quite complicated with more collaborators and more domain logic than what I have painted here. Hence it is quite natural that we need to enrich the test cases with more possibilities and test coverage. Instead of making separate methods which work on various combinations of days, rate and principal (and other factors in reality), why not DRY them up with parameterized tests ?

Parameterized Tests in TestNG

TestNG provides a great feature for providing parameters in test methods using the DataProvider annotation. DRY up your test methods with parameters .. here is what it looks like in our case ..

public class AccruedInterestCalculatorTest extends MockControl {
  private IAccruedDaysCalculator acalc;
  private IInterestRateCalculator icalc;

  @BeforeMethod
  public final void setup() {
    mockControl = EasyMock.createControl();
    acalc = newMock(IAccruedDaysCalculator.class);
    icalc = newMock(IInterestRateCalculator.class);
  }

  @DataProvider(name = "test1")
  public Object[][] createAccruedInterestCalculationData() {
    return new Object[][] {
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(1000), BigDecimal.valueOf(30)},
      {20, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(60)},
      {30, BigDecimal.valueOf(0.5), BigDecimal.valueOf(2000), BigDecimal.valueOf(90)},
    };
  }

  private class CalculatorScaffold {
    //..
    //.. same as above
  }

  // the test method now accepts parameters
  //..
  @Test(dataProvider = "test1")
  public void normalAccruedInterest(int days,
    BigDecimal rate, BigDecimal principal, BigDecimal interestGold) {
    new CalculatorScaffold()
      .on(principal)
      .forDays(days)
      .at(rate)
      .calculate()
      .andCheck(interestGold);
  }

  @Test(expectedExceptions = {NoInterestAccruedException.class})
  public void zeroAccruedDays() {
    new CalculatorScaffold()
      .on(BigDecimal.valueOf(2000))
      .forDays(0)
      .at(BigDecimal.valueOf(0.5))
      .calculate()
      .andCheck(BigDecimal.valueOf(90));
  }
}

The test methods look DRY, speak the domain language and do no longer have to administer the mock controls. All test conditions and results are now completely decoupled from the test methods and fed to them through the annotated provider methods. Now the domain guy is happy!

XML - Not for Human Consumption

I have never been a big fan of using XML as a language for human interaction. I always felt that XML is too noisy for human comprehension and never a pleasing sight for your eyes. In an earlier post I had pitched in with the idea of using Lisp s-expressions as executable XML instead of the plethora of angle brackets polluting your comprehension power. XML is meant for machine interpretation, the hierarchical processing of XML documents gives you the raw power when you are programming for the machine. But, after all, Programs must be written for people to read, and only incidentally for machines to execute (from SICP).

With the emergence of the modern stream of scripting languages, XML is definitely taking a backseat in handling issues like software builds and configuration management. People are no longer amused with the XML hell of Maven and have been desperately looking for alternatives. Buildr, a drop-in replacement for Maven uses Ruby (inspired by Rake), SCons uses Python scripts, Groovy has also been used in the configuration space for Jini-based ComputeCycles project (via Artima). XML has also started looking like just yet another option for configuring your beans in the Spring community. We are just spoilt with other options of configuration - Spring JavaConfig uses Java configuration, Springy does it with JRuby, Groovy SpringBuilder does it with Groovy.

Configuration and builds of systems are nontrivial activities and need a mix of both declarative and procedural programming. Using XML as the brute force approach towards this has resulted in complex hierarchical structures, too complex for human comprehension. A perfect example in hand is Maven which attempts to do too many things with XML. The result is extremely complex XML hell much to the anguish of developers and build managers. A big build system configured using Maven becomes a maintenance nightmare in no time.

People are getting increasingly used to the comforts of DSLs and configurations of large systems have never been too trivial an artifact. Hence it is logical that we would like to have something which provides a cool humane interface. And XML is definitely not one of them ..

Unit Tests - Another Great Tool for Refactoring

In my last post on unit testing, I had talked about some techniques on how to make your classes unit test friendly. Unit testing is the bedrock of writing good software - never compromise on writing unit tests for the next class that you design. Unit testing is about testing classes in isolation - get them out of the natural environment and take them for a ride. That way unit tests give you a feedback about the correctness of the behavior of the *unit* as a whole. The other aspect of unit tests which I find extremely useful while designing classes is the support for refactoring. Unit tests are great vehicles for refactoring object collaborations - thus there is a 2-way feedback loop between class design and its unit test design. And once you have cycled through rounds of iteration in this feedback loop, you can expect to have stabilized your abstractions and collaborations.

Here's an example ..

class Payroll {
  // ..
  // ..

  void calculateSalary(Employee emp) {
    BigDecimal charge = emp.getSalary();
    // calculate total salary including taxes
  }
}

When I look at the possible unit test for the class Payroll, the first thing that strikes me is the argument Employee in calculateSalary(). The first attempt tries to instantiate an Employee and invoke the method calculateSalary() from the test case :

class PayrollTest extends TestCase {
  // ..
  // ..

  public void testSalaryCalculation() {
    Employee emp = new Employee(..);
    // ..
  }
}

A Mockery of sorts ..

Instantiating an Employee object brings into action lots of other machineries like the database layer, the ORM layer etc., which we already know should be best kept away from the unit test harness. Think Mocks ..

When you need to mock a concrete class, take a second look, if necessary take a third look as well. Mock roles, not objects and roles are best specified through interfaces. In the above example, we are really exploiting the fact that an employee is salaried and has a basic salary component for being employed in a company. In this method calculateSalary(), we are not concerned with the other details of the Employee class, which may be used in other context within the software.

Extract Interface

Now I have a new interface Salaried as :

interface Salaried {
  BigDecimal getBasicSalary();
}

and ..

public class Employee implements Salaried {
  // ..
}

The method calculateSalary() now takes an interface Salaried instead of the concrete class Employee - and the mock becomes idiomatic in the test case (I am using EasyMock as the mocking framework) :

class Payroll {
  // ..
  // ..

  public void calculateSalary(Salaried emp) {
    // ..
  }
}

class PayrollTest extends TestCase {
  // ..
  // ..

  public void testSalaryCalculation() {
    IBillable mock = createMock(Salaried .class);
    expect(mock.getBasicSalary()).andReturn(BigDecimal.valueOf(7000));
    replay(mock);
    // ..
  }
}

Thus unit testing helped me use the Extract Interface principle and improve the collaboration between my classes. Notice how the class Payroll now collaborates with a more generic role Salaried instead of a concrete class Employee. Tomorrow if the company decides to employ contractors with different salary computation, we will have a separate abstraction :

public class Contractor implements Salaried {
  // ..
  // ..
  BigDecimal getBasicSalary() {
    // .. different implementation than Employee
  }
  // ..
}

but the class Payroll remains unaffected, since it now collaborates with Salaried and NOT any concrete implementation.