Groovy as a Service Layer: The Aftermath

A while back I posted about why you would want to use Groovy for your service layer within a service oriented architecture. I spent the last few weeks doing just that with Axis2, and now I can look back with some perspective and confirm some of the original ideas.

My service layer moves incoming requests from untyped primitives (Strings and XML blocks) into typed Java objects consumable by a business facade, and it moves typed Java objects produced by the facade back into untyped primitives (Plain Old XML) consumable by a client. I chose Groovy based on how easy it is to produce and consume XML data within the language.

Groovy as a service
Groovy has been a joy to work with and really exceeded my expectations. The XmlSlurper class took a while learn, but using it to read XML or Axis2 OMElement objects was super easy after the first few times doing it. And generating XML with MarkupBuilder or a custom OMElementBuilder is incredibly trivial compared to the Java equivalent. By the way, writing my own OMElementBuilder turned out to be easy. Extend BuilderSupport and implement four methods. Simply put, my service classes contain much less code than if I'd written them in Java.

My original post suggested that Groovy was a better language because it was dynamically typed, thus supporting untyped data (XML) better. This isn't quite true. It is a better choice because of the XML support, which some statically typed languages also share. Scala has good support for writing and reading XML. It's hard to say anything about type systems without being proven wrong, isn't it?

Asix2 tooling
wsdl2Java is a great tool you point at a WSDL and generate reams of framework code and a single "MyService.java" pojo that will be your service class. At this point, I would take MyService.java and rename it MyService.groovy and away you go! Nothing about using Groovy has made any of the Axis2 tools break. Axis2 is instantiating your service class using reflection, so as long as you produce a class file with the correct public method, then you're good to go. Scala services should work just fine too, as well as JRuby (I'm guessing). HOWEVER! The default "MessageReceiver" in Axis2 invokes your service method using reflection. This means your sweet metaprogramming invokeMethod tricks will not work. That took me a while to figure out.

That being said, I turned all that SOAP cruft off and just used the existing raw XML message receivers. Eventually I threw that out and wrote own. Just another case of Not Invented Here? Possibly. But it was the only way I could get my MessageReceivers created out of Spring (a post for another day, sigh).

REST vs. SOAP
Axis2 support for REST was underwhelming. Under the covers, the REST request is converted into a SOAP envelope for you. So your "MessageReceiver" always sees a SOAP message, even when you're using REST. Also, only POST and GET work. After looking breifly at Grails url mapping and REST support, I would encourage someone to use Grails instead of Axis2. We don't always have the authority to make these decisions though. I know I don't.

Mixing Groovy and Java classes
I never understood the Groovy joint compiler until now... My project has Groovy classes that reference Java classes and Java classes that reference Groovy classes. The joint compiler runs through the Groovy code and generates Java stubs for all your Groovy classes. Then it compiles the Java classes against the stubs so that Java has something to link to. Lastly, it compiles the Groovy code into real classes and replaces the stubs. Nifty, although sometimes you'll have MyClass.groovy and receive a compiler error referencing MyClass.java. That'll confuse you the first time!

I also felt Groovy's optional static type system really helped the Groovy classes integrate into the existing Java codebase. My goal (and constraint) was to write Groovy based service classes and leave the rest in Java. As my Groovy classes grew, and code started being refactored into reused components, I started tightening down the compile time contract of the shared code to make it more usable. My first step was to declare return types on methods. This made the calling code a little easier to work with. On some classes I eventually stripped out the Groovy specific libraries and converted them into pure Java classes (.java extension and all). It was great that I could have the thought, "Oh, I really should have written this in Java" and then actually convert the thing to Java within a few minutes.Although, by and large, my more common thought was, "I wish I could just do this in Groovy" rather than vice versa. But as I said, we don't always have the authority to make these decisions. However, a few of my classes can't be converted to Java because of the reliance on metaprogramming. If you're cautious, you might avoid putting yourself in this situation. I say go for it.

Lastly, I work in an IntelliJ IDEA shop. Having the great Groovy plugin for IDEA helped immensely. This would have all be a lot harder if you were switching IDEs or editors based on language. Being able to step through code from Java to Groovy and back again, as well as navigating the whole project as one unit, made the whole process work a lot better.


Big thanks to everyone who left a comment or contributed on the discussion on the Groovy.mn mailing list. You helped a lot!

Why Groovy as a service layer

I've spent the last week working on getting an Axis2 and Groovy based web service up and running. Getting Groovy to work as an Axis2 service class was incredibly simple... the Groovy scripts compile to class files and then it's just dropping the Groovy jar in the classpath.

So the implementation is finished. Now comes the hard part: convincing my co-workers in a Java shop that Groovy is the right choice for a service layer. Everyone knows it's my pet technology, and so far my demos have resulted in a lot of eye rolling. But darn it, I think it's the right choice, and I'm using this post to explain why.

The design has the service layer interacting with the Java-based business logic facades. My intent was to have the service layer's concern be strictly focused. It moves incoming requests from untyped primitives (Strings and XML blocks) into typed Java objects consumable by the facade, and it moves typed Java objects produced by the facade back into untyped primitives (Plain Old XML) consumable by the client. (I'm pretty sure this is not proprietary information at this point, but don't ask for more details).

Groovy's Java integration makes it well-suited to do this. My service class just instantiates the Java objects it needs almost exactly like it would in Java. As a result, the impedance mismatch between a Java and Groovy service class is very low. Context switching languages can be tough, but this is one of the easier cases.

But that explanation answers the question, "Which dynamic language?" rather than the question, "Should we even use a dynamic language?", which is the real decision.

In my career I've spent an inordinate amount of time at type system boundaries. Most my time has been spent in C++ and Java, statically typed languages enforced with a compiler. And at the same time, most of my career has been spent working with untyped data: primitive types coming from the database, byte streams coming from sockets, character data coming from files. I've spent a lot of time writing code like this:

String name = resultSet.getValue("name")
  
char* name = resultSet[4]

I'm not saying my projects are littered with code like this. I'm saying the projects contain code like this, and this type of code is where I've spent too much time. It's error prone because the compiler can't check it for you. It's hard to test because you usually need an integration test to really check it. And it's hard to read because the offset 4 and the entire getValue() call are mostly just noise in the signal.

The statically typed languages I've used simply don't support untyped data very well. This is why a lot of the frameworks exist. Hibernate makes working with untyped databases easier. XPath and Castor make consuming untyped XML easier. But using a dynamic language like Groovy makes working with untyped data natural.

For instance, consuming XML is trivial with XMLSlurper:

def records = new XmlSlurper().parseText(CAR_RECORDS)
def carRecords = records.car
def allNodes = carRecords.depthFirst().collect{ it }

The signal to noise ratio here is much higher than the Java equivalent. There is no getValue() or getChilden() cruft to obscure the intent. In my experience, this is easier to read, a little easier to write once you're over the learning curve, and about the same to debug. Before the IDEA/Eclipse Groovy plugins it was harder to debug, but now I believe it is about the same.

Producing XML is trivial too, using the built-in MarkupBuilder:

def writer = new StringWriter()
new MarkupBuilder(writer).records() {
  car(make:'Holden') {
    country('Australia')
  }
  car(make:'Peel') {
    country('Isle of Man')
  }
}

I find this easy to read because, even in more complicated examples, the structure of the builder calls mirror the structure of the XML produced. This is also very easy to write. So easy that I've yet had to debug it. Overall, I've been really satisfied with the ability of Groovy to produce XML without additional libraries.

Back to my service layer... remember it's sole purpose is to move data back and forth between Java types and XML primitives. When choosing a language for this layer, we have to ask, "How well does it support untyped data? (Moving data in and out of XML)" The Groovy language has Java beat here. We also have to ask, "How well does it support typed data? (Moving data in and out of Java classes)" Groovy probably has Java beat here again, with its support for builders and identity blocks. Once you concerns are isolated to a single layer, it seems that Groovy is the clear winner for implementation language.

The decision to use Groovy has not yet been made... so what do you think? What are your reasons to use Groovy?

The Frictionless Architecture (and other lessons from the punchcard era)

Tom Evans gave an inspired talk at OTUG last week with the (possibly) uninspiring name of Business Agility Principles and Architecture: Lessons from the Punch Card Era. I'm gun-shy about attending things with agile in the title, but this turned out to be a very practical session for software developers. The two points I took away from it were around his musings on designing a frictionless architecture and designing the failure path first.

The frictionless architecture idea came from his days in the telecom industry, in which punch cards were used to transport customer data. Requesting a new line to your office? It gets encoded on a human-readable punchcard and sent via inter-office mail to the correct department. Requesting residential service? This punchcard is delivered the same way and goes to a different department. The point is that the transport layer doesn't care at all about what is being transferred. Inter-office mail will deliver anything as long as it is in a brown envelope. Did you forget to fill out your billing address? Well, the punchcard is still delivered. Did you fill out the request in Kanji? Inter-office mail doesn't care. It will wind up in someone's inbox, ready to be processed. Compare this with my current b2b web services project. When a customer sends me a request and has forgotten to include required fields (the address perhaps), then that request errors out on dtd validation and hopefully the information is written to a log file by my web server. If you send a request in a multi-byte language like Kanji, and accidentally screw up the encodings, my web service is probably going to puke too. Hey, it's your fault for sending me "garbage" data, right? The point is that a customer is trying to contact me, and I'm just falling over and not fulfilling the request simply because they didn't know enough to fill out the form correctly. In the punchcard era, that request gets delivered properly. As long as the punchcard machines get oiled (the frictionless part), then every single transaction will be delivered. And that request is human readable, which leads to the next point: designing the failure path first.

The failure path for the punchcards was built into the system. A human received the punchcard. If that request was missing the address, then the telecom employee could just look at your telephone number on the request and call you to get the missing information. If the request was written in Kanji, then they can go find an interpretor to create the translated version. I have an apostrophe in my name (D'Arcy), and I can't tell you how many times websites and web services simply don't work because of that. I don't get a callback, or a human trying to fix it; I get an error message if I'm lucky. Punchcards had a great failure path. The business could almost always fulfill a request even if their customer totally screwed things up. So what's this about the failure path first stuff? Well, by default the business was able to fulfill automated transactions that failed. If they rolled out an entirely new service that was not supported by IT, they could still have humans fulfill all of those orders by hand. Supporting a new service by default required hiring temp workers to type in orders by hand. If my business wants to roll out a new service, there is no way for me to harness manual labor at all. Unless there is a new service and a dtd/schema (and maybe a wsdl, and and endpoint, etc. etc) I won't be able to handle any new services! On one of Tom's projects, his company actually rolled out new services, and those orders arrived on punch cards or faxes, and IT finally caught up with an automated SmallTalk system later when the daily order volume started exceeding 100 orders a day. It would be a huge value if, by default, all unrecognized incoming transactions were put into a queue in their entirety, where a fleet of temp workers could slowly process them by hand.

And business agility? The telecom could roll out new services without any IT support. Not even an iteration 1 to get the skeleton together. Talk about delivering value early; the architecture could deliver the value even before the first sprint started. Brilliant.

This slides aren't posted yet (c'mon Tom, email them over), but Tom did say he was preparing a longer published work later this year. Definitely looking forward to that. Lastly, if you're in the Twin Cities please come out to the May meeting to hear Ted Neward speak about Scala. Attendance and the speakers have been good this year... come join the party.