Why I don't like ActiveRecord for Domain Model Persistence

When it comes to a rich domain modeling, I am not a big fan of the ActiveRecord model. The biggest problem that it entails is invasiveness - the persistence model invades into my domain model. And this was also my first reaction to the Lift-CouchDB integration module which was released recently.

The moment I say ..

class Person extends CouchRecord[Person] {
  //..
}


my domain model becomes tied to the persistence concerns.

When I started scouchdb, my very first thought was to make it non-invasive. The Scala objects must be pure and must remain pure and completely oblivious of the underlying persistence model. In the age of polyglot persistence there is every possibility that you may need to persist a domain model across multiple storage engines. I may be using a JPA backed relational store as the enterprise online database, which gets synchronized with some offline processing that comes from a CouchDB backend. I need the domain model persistence in both my Oracle and my CouchDB engines. The ActiveRecord pattern is difficult to scale to such requirements.

Here's what I implemented in scouchdb ..

// Scala abstraction : pure
case class ItemPrice(store: String, item: String, price: Number)

// specification of the db server running
val couch = Couch("127.0.0.1")
val item_db = Db("item_db")

// create the database
couch(item_db create)

// create the Scala object : a pure domain object
val s = ItemPrice("Best Buy", "mac book pro", 3000)

// create a document for the database with an id
val doc = Doc(item_db, "best_buy")

// add
couch(doc add s)

// query by id to get the id and revision of the document
val id_rev = couch(item_db by_id "best_buy")

// query by id to get back the object
// returns a tuple3 of (id, rev, object)
val sh = couch(item_db by_id("best_buy", classOf[ItemPrice]))

// got back the original object
sh._3.item should equal(s.item)
sh._3.price should equal(s.price)


It's a full cycle session of interaction with the CouchDB persistence engine without any intrusion into the domain abstraction. I am free to use ItemPrice domain abstraction for a relational storage as well.

CouchDB offers a model of persistence where the objects that we store should be close to the granularity of domain abstractions. I should be able to store the entire Aggregate Root of my model components directly as JSON. ActiveRecord model offers a lower level of abstraction and makes you think more in terms of persistence of the individual entities. The thought process is so relational that you ultimately end up with a relational model both in terms of persistence and domain. With CouchDB you need to think in terms of documents and views and NOT in terms of relations and tables. I blogged on this same subject some time back.

The philosophy that I adopted in scouchdb was to decouple the domain entities from the persistence layer. You hand over a pure Scala object to the driver, it will extract a JSON model from it and write it to CouchDB. I use sjson for this serialization. sjson works totally based on reflection and can transparently serialize and deserialize Scala objects that you hand over to it. From this point of view, the three aspects of managing domain abstractions, JSON serialization and persistence into CouchDB are totally orthogonal. I think this is difficult to get with an ActiveRecord based model.

scouchdb Views now interoperable with Scala Objects

In one of the mail exchanges that I had with Dick Wall before the scouchdb demonstration at JavaOne ScriptBowl, Dick asked me the following ..

"Can I return an actual car object instead of a string description? It would be killer if I can actually show some real car sale item objects coming back from the database instead of the string description."

Yes, Dick, you can, now. scouchdb now offers APIs for returning Scala objects directly from couchdb views. Here's an example with Dick's CarSaleItem object model ..

// CarSaleItem class
@BeanInfo
case class CarSaleItem(make : String, model : String, 
  price : BigDecimal, condition : String, color : String) {

  def this(make : String, model : String, 
    price : Int, condition : String, color : String) =
    this(make, model, BigDecimal.int2bigDecimal(price), condition, color)

  private [db] def this() = this(null, null, 0, null, null)

  override def toString = "A " + condition + " " + color + " " + 
    make + " " + model + " for $" + price
}


The following map function returns the car make as the key and the car price as the value ..

// map function
val redCarsPrice =
  """(doc: dispatch.json.JsValue) => {
        val (id, rev, car) = couch.json.JsBean.toBean(doc, 
          classOf[couch.db.CarSaleItem]);
        if (car.color.contains("Red")) List(List(car.make, car.price)) else Nil
  }"""


This is exciting. The following map function returns the car make as the key and the car object as the value ..

// map function
val redCars =
  """(doc: dispatch.json.JsValue) => {
        val (id, rev, car) = couch.json.JsBean.toBean(doc, 
          classOf[couch.db.CarSaleItem]);
        if (car.color.contains("Red")) List(List(car.make, car)) else Nil
  }"""


And now some regular view setup code that registers the views in the CouchDB design document.

// view definitions
val redCarsView = new View(redCars, null)
val redCarsPriceView = new View(redCarsPrice, null)

// handling design document stuff
val cv = DesignDocument("car_views", null, Map[String, View]())
cv.language = "scala"

val rcv = 
  DesignDocument(cv._id, null, 
    Map("red_cars" -> redCarsView, "red_cars_price" -> redCarsPriceView))
rcv.language = "scala"
couch(Doc(carDb, rcv._id) add rcv)


The following query returns JSON corresponding to the car objects being returned from the view ..

val ls1 = couch(carDb view(
  Views builder("car_views/red_cars") build))


On the client side, we can do a simple map over the collection that converts the returned collection into a collection of the specific class objects .. Here we have a collection of CarSaleItem objects ..

import dispatch.json.Js._;
val objs =
  ls1.map { car =>
    val x = Symbol("value") ? obj
    val x(x_) = car
    JsBean.toBean(x_, classOf[CarSaleItem])._3
  }
objs.size should equal(3)
objs.map(_.make).sort((e1, e2) => (e1 compareTo e2) < 0) 
  should equal(List("BMW", "Geo", "Honda"))


But it gets better than this .. we can now have direct Scala objects being fetched from the view query directly through scouchdb API ..

// ls1 is now a list of CarSaleItem objects
val ls1 = couch(carDb view(
  Views builder("car_views/red_cars") build, classOf[CarSaleItem]))
ls1.map(_.make).sort((e1, e2) => (e1 compareTo e2) < 0) 
  should equal(List("BMW", "Geo", "Honda"))


Note the class being passed as an additional parameter in the view API. Similar stuff is also being supported for views having reduce functions. This makes scouchdb more seamless for interoperability between JSON storage layer and object based application layer.

Have a look at the project home page and the associated test case for details ..

scouchdb Scala View Server gets "reduce"

scouchdb View Server gets reduce. After a fairly long hiatus, I finally got some time to do some hacking on scouchdb over the weekend. And this is what came out of a brief stint on Saturday evening ..

map was already supported in version 0.3. You could define map functions in Scala as ..

val mapfn = """(doc: dispatch.json.JsValue) => {
  val it = couch.json.JsBean.toBean(doc, classOf[couch.json.TestBeans.Item_1])._3;
  for (st <- it.prices)
    yield(List(it.item, st._2))
}"""


Now you can do reduce too ..

val redfn = """(key: List[(String, String)], values: List[dispatch.json.JsNumber], rereduce: Boolean) => {
  values.foldLeft(BigDecimal(0.00))
    ((s, f) => s + (f match { case dispatch.json.JsNumber(n) => n }))
}"""


attach the map and reduce functions to a view ..

val view = new View(mapfn, redfn)


and finally fetch using the view query ..

val ls1 =
  couch(test view(
    Views.builder("big/big_lunch")
         .build))
ls1.size should equal(1)


reduce, by default returns only one row through a computation on the result set returned by map. The above query does not use grouping and returns 1 row as the result. You can also use view results grouping and return rows grouped by keys ..

val ls1 =
  couch(test view(
    Views.builder("big/big_lunch")
         .options(optionBuilder group(true) build) // with grouping
         .build))
ls1.size should equal(3)


For a more detailed discussion and examples have a look at the project home page documentation or browse through the test script ScalaViewServerSpec.

The current trunk is 0.3.1. The previous version has been tagged as 0.3 and available in tags folder.

Next up ..

• JPA like collections of objects directly from scouchdb views


• more capable reduce options (rereduce, collations etc.)


• replication


• advanced exception management with new dbDispatch

.. and lots of other features ..

Stay tuned!