ChitChat with the Groovy Compiler

Ever wished to be able to influence the compiling process in Java?

To do this you usually don't influence the compiler directly, you use AOP tools with strange syntax rules and extra files for configuration or you are working on the bytecode level and sue the instrumentation API. But these technics have their limits. To be able to instrument a class you need a ready compiled, the verifier passing class. What if you can't produce such a class?

But ok, let us go some steps back. Programming by convention is in all mouths, so let us look at adding a method depending on the class name. I guess people would use a inter-type declaration in AspectJ to add methods to a certain class. The problem here is, we don't know the class before. We react to the name of the class and only if the name fits, we add the method.

Another thing is adding imports. There is no point in adding imports in a class file, all class names in a class file are already resolved against the imports. Adding another import would be pointless. But if you want to let people write a small peace of code where they can forget about importing every class they use, then we are talking about adding something to what the compiler gets as input. The most easy solution would be to use some kind of macro system. Expand the macros and give the result in the compiler. But then, what is about the line numbering? Exceptions will show useless information about the position in the source file, compiler errors will point to invalid places. That's a bit bad.

Compiler are normally working on an AST, an abstract syntax tree. Such a tree represents the source as a tree, each node is a part logic created by the source. For example, the expression "foo.bar()" is in Groovy a method call expression with foo as the object the call is made on, bar the name of the method and an empty number of arguments. Such a method call expression is a node in the AST. I am afraid that explaining all the expression types and statements in Groovy would be too much for this small article. I suggest to look into the packages org.codehaus.groovy.ast.expr, org.codehaus.groovy.ast.stmt and org.codehaus.groovy.ast. Of course having around 35 expressions and 19 statements doesn't make this task very nice, but it is needed. Groovy uses the visitor pattern to visit the classes. That makes it a bit problematic for people how are not used to that. but it really is nice to have that.

Each phase of the compilation process after the AST is created consists of a visitor travaling along the AST replacing or modifying nodes. Btw, the Groovy AST is for example used in Eclipse for the outline.

Now imagine you could change the AST by yourself. Rename methods, add code, add methods... yes, right, we wanted to add a method if the class name fullfills a certain convention.

Now the Groovy compiler offers here some hooks you can use to change what the compiler will produce. What I will explain now counts only for the case that a file is compiled using the GroovyClassLoader, for example because a script is found in the classpath or because we use the GroovyShell to execute a script. The basic process to creat a compilation unit, that will hold the AST and other information, add a source to the compilation unit and then start the compilation. By subclassing GroovyClassLoader you can overwrite createCompilationUnit(CompilerConfiguration, CodeSource), the very first step. The CompilationUnit is really our compiler. It holds all the logic. So if you return a subclass of CompilationUnit here, you would return a custom compiler. But I don't want to go this far here.

I talked about hooks, but subclassing CompilationUnit would be a bit big possibly. I talked also about the phases a compiler goes through and that's where CompilationUnit offers hooks. The addPhaseOperation methods do allow you to give the compiler additional logic for a phase of your choice. In Groovy these phses are:

• initialization: open files setting up the environment and such
• parsing: use the grammar to produce tree of tokens repersenting the source
  
• conversion: make a real AST out of these token trees
• semantic analysis: check for all things the grammar can't check for, resolv classes and other things
• canonicalization: complete the AST
• instruction selection: choose isntruction set, for example java5 or pre java5
• class generation: create the binaries in memory
• output: write the binaries to the file system
• finalization: cleanup
  

not all these phases are really filled, the canonicalization step is currently empty, but that is a implementation detail. If we want to operate on the the AST then we need one, thus the conversion phase is the most early phase to do that. You have to decide if your operation is about changing a certain source file or a certain class. Each file may contain multiple classes, so this is not equal. For example if we want to add an method, then it makes not much sense to work on the source (SourceUnit) but on the ClassNodes instead.

In our example we would call the addPhaseOperation(PrimaryClassNodeOperation, int) method. You may wonder about the name primary class node operation. A class node represents a class, but in the compilation process there are two kinds of classes: precompiled classes and classes we want to compile. These primary class node are classes we want to compile.

If you know the AST then the next steps are quite easy. For example:

class MyOperation extends PrimaryClassNodeOperation {
        public void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
               classNode.addMethod("foo", OpCodes.PUBLIC, null, null, null,code);
        }
}

This would add a method named foo to every class we compile, foo takes no arguments, is public and has the return type Object. The contents of the method are stored in "code". I haven't specified that here to keep the article not totally complicated. Ok, again let us think back to the orignal task of adding a method if the class name fullfills a certain convention. For example if a class is XyzFoo, then add a method xyz, if the class is AbcFoo, then abc. Ok, that is a bit crazy, but well, just for the demonstration of the flexibility:

import org.codehaus.groovy.ast.ClassNode;
import org.codehaus.groovy.ast.stmt.BlockStatement;
import org.codehaus.groovy.classgen.GeneratorContext;
import org.codehaus.groovy.control.CompilationFailedException;
import org.codehaus.groovy.control.CompilationUnit;
import org.codehaus.groovy.control.CompilerConfiguration;
import org.codehaus.groovy.control.Phases;
import org.codehaus.groovy.control.SourceUnit;
import org.codehaus.groovy.control.CompilationUnit.PrimaryClassNodeOperation;
import org.objectweb.asm.Opcodes;

import groovy.lang.GroovyClassLoader;

class MyGroovy extends GroovyClassLoader {
    protected CompilationUnit createCompilationUnit(CompilerConfiguration config, CodeSource source) {
        CompilationUnit cu = super.createCompilationUnit(config, source);
        cu.addPhaseOperation(new PrimaryClassNodeOperation() {
            public void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
                String name = classNode.getName();
                if (name.endsWith("Foo") && name.length()>3) {
                    name = name.substring(0,name.length()-3);
                    BlockStatement code = new BlockStatement();
                    classNode.addMethod(name,Opcodes.ACC_PUBLIC,null,null,null,code);
                }
            }
        },Phases.CONVERSION);
        return cu;
    }
}

Grails does use this mecahnism to add the toString and hashcode methods as well as the id field in domain classes.

Another easy example is adding a import. But since we don't add an import to a specific class, but to all classes in a file, we are operation on the SourceUnit (representation of the source file) so we use a SourceUnitOperation.

        cu.addPhaseOperation(new SourceUnitOperation() {
            public void call(SourceUnit source) throws CompilationFailedException {
                source.getAST().addImport("User",ClassHelper.make("my.company.UserDAO"));
            }
        },Phases.CONVERSION);

here we add an import for UserDAO and make the class known as User inside the file we compile. That is comparable to "import my.company.UserDAO as User" in Groovy.

As you may see this is much more than aspect oriented programming could do. You can place arbitary logic inside the compiler and modify the ASTs. If you want to do more complex operations or more local ones, then you might have to really go into the AST. I suggest to use the LabelVerifier as entry point for this. A little debugging helps you to understand how the AST looks like and how the visting works. I plan to write a big documentation about these things, but this will be on the Groovy wiki then. The purpose of this article is just to give you a insight in the internals of the compilation and how it can be influenced from the outside. Without knowing the AST there is not really much you can do. It is not that the AST is very complicated or bad documented, but learning about over 60 classes isn't done in a short form. The biggest exmaple of AST traversion is AsmClassGenerator, the part in the compiler creating the binaries. But it is a big class, not too good to learn.

My plans for the future are to add an mechansim that allows to write some text that gets mixed into the classes the compiler produces. In the import example above you would maybe write something like

cu.mixinPerSource("import my.company.UserDAO as User")

. Or in the method example:

cu.mixinPerClass {"def ${it[0..-3]}(){}"}

. But it looks like this will ahve to wait for post 1.0

You could also use the AST to transform any DSL containing binary operations such as "&" and "<" into a builder or a method based version. You could transform it into a database request or whatever. There are many possible usages of this. And in fact something like that is already used in GroovySQL - but well hidden ;)

So might I ask again: Ever wished to be able to influence the compiling process in Java?

Concurrent Loop Excetion - Comparing Java and Groovy

Neal showed in his blog a new example of how closures in java could look like, in real world usage.

If I would program this in Groovy (which runs on the JVM btw), I would maybe write this:

def getAttendees() {
  return getResponses()
         .findAll{it.mayAttend()}.collectAsync(threadPool) {it.attendee}
}

Right, that's nearly a one-liner. Of course that code is not fully equal to Neal's program, because mayAttend() is executed synchronous, and Groovy doesn't have collectAsync. But we could write such a method and use it in a category. Maybe the method would look like:

def collectAsync(threadPool, Collection col, Closure action) {
  def result = []
  def tasks = col.collect {
    try{
      synchronized (result) {
        result << action.call(it)
      } catch (Throwable ex) {
        LOGGER.log(Level.SEVERE, "Uncaught Exception", ex);
      }
  }
  threadPool.invokeAll(tasks)
  if (!result) return null
  return result
}

For the non Groovy people some comments.. Groovy has an dynamic typing system, native literals for lists (see result), operator overloading (<<), closures and of course runs on the jvm with very good integration. Anyway, I am wrapping the original closure in another closure for the logging. This removes some code from the usage. If we where funny, we could use an additonal closure for the logging. "it" is a shortcut for the current value given to the closure. "it.attendee" is equal to "it.getAttendee()", Groovy does have logic for beans.

So how does it work? I use findAll to filter out all EventResponses that don't attend. findAll returns a new list containing only the important responses. collectAsync creates a list of closures, each closure representing a task, and gives that to my threadpool as suggested in the lower part of the article - of course it wasn't with closures there. Anyway, during execution each task calls the action closure and writes in "result". The nice thing is, that this function could be a library function used somewhere. There is no reference to something special besides the logger and threadPool. In fact the code is short only if the method is a library function. It is no use to tell people how well closures might look, if I compare library functions and their usage with raw code. The Groovy version is short because of the lack of typing we are using and much other factors. If I would have to add types then it would become much larger. For example even if there where a findAll method working on a collection and able to take such a closure, I still would have to define the list element type... possibly findAll<EventResponse>{it.mayAttend()} Then, "it" is not typed. Ok then it might look like: findAll{EventResponse it -> it.mayAttend()}. the Groovy syntax for defining parameters is a little different form the closure proposal, don't care to much about that. Next is collectAsync.. collectAsync<Principal>(threadPool){EventResponse it -> it.attendee} That makes:

getResponses().findAll{
  EventResponse it -> it.mayAttend()
}.collectAsync<Principal>(threadPool){
  EventResponse it -> it.attendee
}

Could still be seen as a one-liner, but it is a bit too big for my taste. Of course collectAsync would have to be changed too:

  Collection<R> collectAsync(threadPool, Collection<T> col,
                                Closure<T,T> action) {
  List<R> result = []
  List<closure<Void>> tasks = col.collect<> {
    try{
      synchronized (result) {
        result << action.call(it)
      }
    } catch (Throwable ex) {
     LOGGER.log(Level.SEVERE, "Uncaught Exception", ex)
    }
  }   
  threadPool.invokeAll(tasks)   
  if (!result) return null
  return result 
}

And this version is just a wish... I mean [] must be replaced by new ArrayList<R>(), !result with isEmpty(). Then we still are not in java, because we need to redefine our generic closure type to Closure1 for the closure taking one argument and Closure0 for closure taking no arguments. Next downside is that findAll and collectAsnyc might not be defined on Collection to keep compatibility (that is no problem in Groovy, as Groovy does have a special mechanism to handle this). So findAll might be a static import, and collectAsync is possibly defined in whatever threadPool is. Then the code may look like:

return threadPool.collectAsync<Principal>(
  findAll(getResponses()) {
    EventResponse it -> it.mayAttend()
  }), {
    EventResponse it -> it.attendee
  })

which is much worse to read due to the new ordering...

List<EventResponse> filtered = findAll(getResponses()) { 
  EventResponse it -> it.mayAttend()
}
return threadPool.collectAsync<Principal>(filtered){
  EventResponse it -> it.attendee
}

looks much better again.. Of course we are not using the syntax suggested in the closure proposal.. then it would look like:

List<EventResponse> filtered = findAll(getResponses(),
          EventResponse(EventResponse it) {it.mayAttend()});
return threadPool.collectAsync<Principal>(filtered, 
          Principal(EventResponse it) {it.getAttendee()})

And a version using inner classes would like:

List<EventResponse> filtered = findAll(getResponses(),
   new Closure1<EventResponse,EventResponse) {
     EventResponse call(Eventresponse it) {it.mayAttend()}
   });
return threadPool.collectAsync<Principal>(filtered, 
  new Closure1<Principal,EventResponse) {
    Principal call(EventResponse){it.getAttendee()}
  });

So we really gain something from the closure proposal - if compared to the current usage of inner classes, but then if I compare that to the very short Groovy version... Java could be shorter here if more type inference was used. It is not the shortness of the Groovy version I like so much, it is the clear code.

But another thing should be made clear with this: the programming style in a language like Groovy is different from the style normally used in Java. I mean compare my endresult with the version without closures at the end of the article. I think I would prefere the other version. Even if I "beautify" the code by using proper identation and such things, I am not gaining much. Strange, isn't it? Java prevents us from defining usefull library functions because of its syntax bloat. Even though there is neraly no logic left in the method itself and all is done by library functions, I have to write so much code for the little compiler... annoying would I say.

I really hope they decide to add more type inheritance. Of course I know that type inference is no easy task for the compiler builder, but it should be doable. Java has way to go here I guess.

Annotations in Groovy

Annotations are thought for providing MetaData.

Such MetaData can be used by the compiler or other processing tools by using the source, the bytecode or at runtime. Annotations are not thought to have an semantic effect.... I always have to tell me this... Somehow I really have a problem with that. Anyway, Groovy doesn't have them. And Groovy won't get Annoatations in 1.0. But as soon as 1.0 is out we want to add them. Tools working with annoatations will then most likely work for Groovy too.

The new JUnit is an example of why Groovy should have annotations. It is not sure yet what set of default annotations Groovy will support, but people will be able to write annotations in Groovy. How much of the enhancements though Groovy can be used in annotations is open, we need to look at what the bytecode allows and if we can go around it... and if we want to do that.

And there is Hibernate... avoiding to persist the metaClass field is sometimes very important. We had several problems with serialization of that property, because the MetaClass is not serializeable and so the default serialization might fail. Using bean serialization might make things even worse. And Hibernate...

@Entity
class Customer implements Serializable {

    @Transient
    MetaClass metaClass
    
    @Id
    Long id

    String firstName
    String lastName
    Date birthday

    @Transient
    Integer age

    @Embedded
    private Address homeAddress

    @OneToMany(cascade=CascadeType.ALL)
    @JoinColumn(name="CUSTOMER_ID")
    Set orders
}

I "stole" that example form the Hibernate page and added the transient property for metaClass. Of course you don't have to add the getter and setter methods, as they are already generated by Groovy. Just add buisness code and be lucky ;)

And of course there are more tools using annoatations and I am sure there will be much more of them in the future. I am very curious if other dynamic languages on the JVM will go that way to. But as most scripting languages on the jvm do not generate compatible class objects it might be out of scope for many of them.

Groovy on Speed - a "fast mode" for Groovy

fast mode? What's that?

The Problem:
Did you ever try to meassure the performance of groovy? It is a quite difficult task. Because of the high integration of Groovy and Java, you are contantly calling Java and Groovy. For example think of some time consuming operations in a Java library. The Groovy part would probably call the library and then return the result. The overhead through Groovy is minimal.

Calling a Groovy method doing some calculations on the other hand might result in bad performance. The solution is quite simple. Identify these parts and write them in Java, by delegating to a Java library or subclass the Groovy class in Java and overwrite the methods in Java.

Of course this is a possibility. But it means you have to use Java for these parts. It means you can't use Groovy syntax for the operations. That sure is no problem for people seeing Groovy as frontend... somewhere... occasionally used, but no essential part. It does indeed become a problem for groovy evangelists.

What exactly is the Problem?
What Goovy makes slower than Java is it's ability to add/intercept methods at runtime and it's slightly different type conversion model. A method invocation in Groovy consists usually of several normal method calls, where the arguments are stored in a array, the classes of the arguments must be retrieved, a key is generated out of them, a hashmap is used to lookup the method and if that fails, then we have to test the available methods for compatible methods, select one of the methods based on the runtime type, create a key for the hasmap and then in the end, do a reflection like call on the method.

Benchmarks usually use the same typs of arguments, so the hasmap kicks in and allows a "fast" method selection. Only counting the method calls, you get around 4 calls before you are able to look in the Hashmap. Then you create the key, do the lookup in the map... given all that I think it is quite a wonder Groovy has such a good performance!

How to make it even faster?
The way to make something faster is either to find an algorithm with a better runtime complexity, or to reduce the constants. If we would not have to create the Object[], we might be able to reduce the costs very much.. I made some meassurments and it seems that invoking a method taking 3 arguments is 3 times slower than a method taking no arguments. That's because we not only have to create the Object[] for the 3 arguments, but we also have three objects which types we need to get, the key becomes longer, and as the key is a String there is a coherence of the length of the string and the number of arguments.

Java does avoid that. Java does select the method at compile time, so there is no Object[], no type extraction and such. Of course the VM checks the types of the parameters, but in case of Groovy the VM does have to do that more than once. The end of the call in Groovy is the same as in Java, but there has much action taken place before. So if we were able to select a method at compile time in groovy, we could avoid that problem completly.

What does it mean?
It means Groovy would be as fast as Java! But that is no general solution. Dynamic typing doesn't allow you to choose the methods at runtime. And the MetaClass facilities won't help here too. so you are changing the semantics of a method call when chanign the typing from dynamic to static.

So what is the Fast Mode then?
The idea is not do dynamic typing every where. In fact there was already a proposal going into that direction. It was even in the codebase, but it was a global switch. I think that is the wrong solution. My solution is to let the programer decide where he wants to have that. For simplicity I will talk about an annotation here, but there was no decision on what to use. Anyway, I would use this annotation for a class, for a method, for a block of code or a single statement. Each so marked part would loose dynamic typing and letting the compiler use static typing to choose the method. This means that the MetaClass is ignored. It does not mean, that we loose any of the addons to the jdk we made. It will still be possible to use "each" or native syntax for BigDecimals, lists and maps. It doesn't mean that you get back checked exceptions or such. I am not sure yet if the compiler should forbid the mixed usage of static typed and dynamic typed arguments, or if it simply should fall back to the normal invocation mode.. Anyway, so instead of asking the MetaClass to tranform the type and make the call the compiler, we will add code to make the type transformation and the method call based on the static types. We loose multimethods then, and we loose the ability to catch unknown methods and properties! So we loose one of the most appealing feature of Groovy. And given the fact, that a builder depends on them we loose them then too. But as you can limit the usage of the fast mode very much it is no problem at all.

Of course this adds much to the complexity of the language.

What about ivokedynamic?
invokedynamic is the great hope of all dynamic languages on the JVM. If it helps us to not to create the Object[] and if it helps us to avoid to create the hasmap key and if it helps us to avoid the type tranformations, then it might be a much better solution than my fast mode. But even if it does all this, it won't be in Java6. It is Java7 at best. And it is not yet defined. The work for this is still in the early stages and while I hope the result will be very good, it is no solution for now.

The Goal
The idea of the "fast mode" is to provide a way to give the maximum performance at certain places. Then you can do calculations in Groovy. I am no fan of using one language for all uasages, but on the other hand I think that such a mode would be very good for Groovy. And it would tell all those benchmark writters out there not to meassure a language with calculations. That is a very stupid thing to do I would say.

The implementation is surely not done in a few minutes and this "fast mode" will not go into Groovy 1.0. But 2.0 or even 1.1 may have that.

Other ideas?
There were other ideas, yes. For exmaple not to simply make the call through the Metaclass, but to let the compiler guess a method and check with the MetaClass if the guess is right. Sadly that doesn't help, because the method selection has still to be done and is the cost intensive factor here. Writing a hotspot engine for Groovy might be a solution. Java is making HotSpot open, so it is theoretically an option. And possible a much better option than anything else. But there are many unclear details.... none has the knowlefge to write something like that, noone knows the API, how is the HotSpot engine added to the normal Java SDK?

We always tried to avoid the need of a special Groovy JVM, even it is jsut the JVM with additional bytecodes. if the HotSpot engine can't be added to any normal JDK then I don't see why this should be any different.

Reality!
I think the space for doing optimizations is big. For example in the "fast mode" we probably don't need to always create a closure. Instead we can inline the closure handling method and the closure itself. But I think only a concrete inplementation of all the possibiilites would be a fair way to tell if something is better. But even if the normal method invocation in Groovy gets a boost fomr a better HotSpot engine, or new bytecode, it only means that the "fast mode" is no longer needed. I would always recommend to use the normal Groovy method invocation and identify the cost intensive parts. Neither will the "fast mode" solve all problems nor will it free you from the usage of profilers.

Remote Closures

a better RMI?

The discussion about closure support in Java is going on. Well, Groovy does have Closures and I like them much. But I asked myself a question: "What happens when I want to execute a closure on a remote VM?"

I answered myself, that that possibly would be a problem. If we use the normal serialization mechanisms, then we have bad luck, a Closures is not Serializable atm. Well, ok, that could be changed, but does it make sense? Usually a closure is a bit like a annonymous inner class. That means in case of RMI you don't have a stub as RMI would require. So maybe the real question is about how to make Groovy's closures Serializeable?

I had some nasty thoughts about somehow serializing the AST into the class, but then I recognized, that the original source might be good enough for that. And it doesn't bloat the class files which would have a bad effect on classloading time.

So I thought, maybe give them custom read and write methods for serializetion, where we transfer the source as well as the enviroment the closure closes around. This could do the job. But then the MetaClass needs to be serializeable as well.

No, I think if we are able to get the source for the closure, then why not sending this to the remote machine? We would put the source and state information of the closure in a wrapper object and when deserializing we reproduce our closure. We use ObjectStreams that know what to do with the closures and we somehow have to foist our custom streams upon RMI.

Or we don't use RMI at all. If we use the ObjectStreams directly we don't have any problems. I may have time in the future ot take a look at spring and it ability to use different remoting services. But thinking of how RMI is done, I always have the feeling that it is completly surplus. How would I implement Remote Method invocation? I need some kind of cahnnel, transfering the method message, that is name of the method, some kind of id for the object the call is made on and a serialized version of the parameters. On the other end, I need to use the id to lookup the goal object, I need to deserialize the arguments and then just make the call using a MetaClass. It is easy!!

I think I will try an example implementation the next days and keep you informed about the result. The nice thing is, we can use that from Java too, thx to the good integration of Groovy in Java!

Getting a Groovy S.O.D.A

For all who don't know S.O.D.A., it is a query language for object databases. SODA is used by the successful db4o project - a object oriented database for Java and .NET

Imagine an object as graph. The object itself is a node, every field is a connection to another node, another object. You build up a query by defining an entry point into that graph and setting constraints on the connections to other nodes. I borrow an class from their tutorial, I hope that will be no problem

class Pilot {
  String name
  int points
}

The above defines a class named Pilot with two properties, in Groovy this means a java like class with getters and setters for name and point. Now look at a query:

Query query=db.query();
query.constrain(Pilot.class);
query.descend("name").constrain("Michael Schumacher");

With this I build a query pattern using Pilot as entry point, looking at the connection "name" and constraining that to Objects equals to "Michael Schumacher". So giving this query a database will write out all Object of class Pilot, with the value "Michael Schumacher" stored in "name"

Pretty neat, isn't it? It is typesafe and such... but a bit long... Well Groovy supports operators, why not try something else:

Query query=db.query();
query.constrain(Pilot.class);
query.name == "Michael Schumacher"

we save the descend and the constraint method. But it gets much better when it comes to connecting query parts for logical operations:

Query query=db.query();
query.constrain(Pilot.class);
Constraint constr=query.descend("name")
    .constrain("Michael Schumacher");
query.descend("points")
    .constrain(new Integer(99)).and(constr);

This query asks for all Michael Schumacher with 99 points. I think we can do this much better in Groovy:

Query query=db.query();
query.constrain(Pilot.class);
query.name == "Michael Schumacher" && query.points == 99

hail operators! This is clear and simple, we don't need the temporary variable any longer and saved some lines of code. And of course operations as greater, smaller, equal, not, and, or are all possible.

Is it possible to improve that even more?

Query query=db.query();
query.constrain(Pilot.class);
with (query) {
  name == "Michael Schumacher" && points == 99
}

Maybe a matter of taste if that version is better, but imagine complex queries. I gues you will be lucky not always having to write "query" everywhere.

Another important part of SODA are evaluations. When we use evaluations, then every object our query had as result, is tested against an evaluation instance we provide, and tells the databse to include it in the result or not.

class NameLengthEvaluation implements Evaluation {
  public void evaluate(Candidate candidate) {
    Pilot p = (Pilot) candidate.getObject()
    candidate.include( p.getName().length() < 5 )
  }
}
query="db.query();"

This would get us all Pilots with a name of a length of less than 5. A small helper class will give us very much power for groovy:

class ClosureEvaluation implements Evaluation {
  ClosureEvaluation(c) {this.closure = c}
  def closure
  public void evaluate(Candidate candidate) {
    candidate.include(closure(candidate.object))
  }
}

using this class we can use a closure as evaluation:

Query query=db.query()
query.constrain(Pilot.class)
query.constrain (new ClosureEvaluation() {
  it.name.length()<5
})

overloading the constrain method we can even shorten this:

Query query=db.query()
query.constrain(Pilot.class)
query.constrain { it.name.length() < 5 }

Now it is very short, isn't it? We can also use a variant of the closure with two parameters to allow access to the object container, but given the fact, that we use a closure we can simply reuse already declared variables outside the closure.

Conclusion:
Yes, db4o provides a solution in Java too. I mean a shorter form of SODA queries named native queries. Basically the bytecode of the class is loaded, analyzed and transformed in a set of SODA queries and evaluations. Given these short forms in Groovy I am not sure I "need" native Queries.

Non Local Transfers in Groovy: A 90% Solution

The discussion about supporting break/continue and return in Closures is a old one.

I was always looking for a 100% solution that just works. Too sad I still haven't found one. But ok, let us look at my new proposal, which is inspired by a blog entry from John Rose about the ongoing discussion of closure support in Java. I think my proposal should cover 99% of all cases.

First let me introduce the term "appended closure". With that I mean closures appended to a method call. An example would be

list.each {println it}

or

if (atHome) {doHomework()}

Ok, the last could have been a closure, bit is none ;)

My solution now covers only these appended-closures. Closures passed around as variable are not covered. These appended closures are most looking like the normal control structures we have, as for example the while loop.

Now what happens when declaring a while loop? The compiler will make us a label marking the entry point of the loop, a label marking the exit point of the loop, and some gotos for break/continue. in Groovy Closures are inner classes, so they can't jump to a label using the bytecode. So we must transfer that exit state form the closure to the calling point and then do our goto there. There was much discussion about how to do that. We discussed return values, which won't work as we must exit the loop method which may be void. We discussed xceptions, which would work, but we where unsure if we really catch them all at the right places and having to rewrite all closure handling code out there is not nice too. We discussed additional fields, with the problem of using the same closure multiple times.

If you only look at appended closures, then some of the problems go away. We get a defined point where to catch the Exception, look at the fields or whatever. But we need the Exception, because we need to exit the method calling the closure. We need it in the case we want to do a break.

And if we limit break/continue to just this type of closures, we have no real problems. We can even do a labled break/continue and not jump only outside the loop method, but outside the surrounding loop too. So what would it look like?

def list = [1,10,100,10,1]
list.each {
it (it>10) break
println it
}

The intention is to print the values 1 and 10 and then stop processing. My suggestion now is to tranform this code into:

def list = [1,10,100,10,1]
int id = createLabelID()
Closure c = {
  it (it>10) throw new ClosureBreakException(id)
  println it
}
try {
list.each(c)
} catch (ClosureBreakException ce) {
if (ce.id != id) rethrow ce
}

Well, yes, looks like much code. But the compiler does the work for us. What about continue?

def list = [1,10,100,10,1]
list.each {
it (it>10) continue
println it
}

Printing 1,10,10,1. It would be transformed into something like

def list = [1,10,100,10,1]
list.each {
  it (it>10) return
  println it
}

ehm.. yes.. easy ;) All code in the method calling the closure is always executed then. Compared with a for-loop this is the part where the increment happens an the comparision is done. If we want to jump to a different point, like a surroung for loop, we need exceptions too, because we break the inner loop then

outer: while (foo) {
list.each {
      it (it>10) continue outer
      println it
}
}

is basically the same as

while (foo) {
list.each {
       it (it>10) {doContinue = true; break}
       println it
}
if (doContinue) continue
}

But as we use exceptions here, we don't need that "doContinue"

int id = createLabelID()
Closure c = {
  it (it>10) throw new ClosureBreakException(id)
  println it
}
while (foo) {
try {
  list.each(c)
} catch (ClosureBreakException ce) {
  if (ce.id != id) rethrow ce
  continue
}
}

What about combinations of continue and break?
To enable this I suggest the ussage of more than one id

while (foo) {
list.each {
  if (it<0)>10) break outer
}
}

is transformed to

int idBreak = createLabelID()
int idContinue = createLabelID()
Closure c = {
it (it<0)>10) throw new ClosureBreakException(idContinue)
}
outer: while (foo) {
try {
  list.each(c)
} catch (ClosureBreakException ce) {
  if (ce.id == idBreak ) break outer
  if (ce.id == idContinue) continue outer
  rethrow ce
}
}

The implementation is really straight forward.

Speed Issues?
We can tell the people that using break/continue might slow down.

Danger of doing break/continue on the wrong loop?
I think we eliminated that problem with our createID function, which produces a new unique id.

Why is it a 90% only?
What I can't do is:

def c = {if(it>10) break; println it}
list.each(c)

and expect the break to work. That is because we have no defined point where we can catch the exceptions. A normal continue can work.

What is the return value in case of continue?
The same as by break, nothing. People will have to know this. We can't have both, a return value and an exception, if the exception does not the job of transporting the return value. In case of a "collect" I expect to get at last a pratial list. To enable this, we must catch the exception and then store the return value od the loop method in there. The changes to the code I showed above is trivial. The changes to the loop method are trivial too, but must be done. The case of a continue is a bit more difficult. And to say the truth I don't know of a solution here. That is because

... I want to avoid people having to rewrite their closures handling code.
Respecting continue would mean to catch an exception in the closure handling method. I don't think that is nice. If I don't do it in case of a "break" my code still works, but might not return correct data. If I don't do it in case of a "continue" my code doesn't behave right and doesn't return correct data. And while supporting break is not needed in every method, supporting continue would be.

What about "return"?
We could support it the same way we support "break". No real problem. In fact that is the most easy version since we don't have to handle incomplete data.

My Suggestion:
Don't allow "continue" in closures. It might look strange, but it has semantic problems. People can always simulate it by returning from the closure with a special value or such and respecting that value in a loop. But avoid new users getting in trouble here and confronting them with more black magic as we already do, I suggest to replace "continue" with a "closure return". The advantage is, that it returns a value and thus is no problem in aspects of incomplete date. for Java ^ is suggested as additional return, I haven't thought about a name or pattern yet. break/return can be implemented as described above for appended closures. Other closures won't have break/return, the compiler would forbid them there. A bit of a problem is the new return. Becasue using the current return in a closure in current Groovy means to return from the closure if it is in a closure and return from the method if it is not in a closure. When seeing a closure as method this is ok, but we don't want to see them as methods. So "return" would change its semantics compared to older groovy versions.

All in all this suggestion here would allow people to use break and return in closures. It would allow to sue them in appended closures, the most used form of the closure when doing loops. A "closure return", or I should better say "block return" would replace the old continue statement. If people really wish to omit a single step, they have to write their own loop.

The advantages is that I don't have to write additional closure handling code for break/return. Only if I want to avoid the incomplete data in the case I return something. Next advantage is that I don't have to identify the closure somehow in the loop method, I don't care if the current closure caused the problem or something deeper inside. I have a defined point where I can catch the exception and don't have to be afraid that it might interfere with other loop methods.

The other possibility I see is to forbid them all ;)

Groovy and Scala Multithread Actors

Parallel exection of Methods.

In my last blog entry I used a MetaClass to change the method execution modell. Some days ago a paper surfaced in the german Java newsgroup about parallel execution of methods in Scala. I found it quite interesting, but maybe there is a way to have that for Groovy too?

I think it is possible using a custom MetaClass again. Again we would collect the method calls, but this time we would use multiple threads to execute the methods. Using a pool for that could be a good choice. Anyway, I don't want to go too much into the details this time, but if you look at my last blog entry and exchange the method execution part with something like:

   stack << method 
  while (stack) {
    def call = stack.pop()
    ThreadPool.run(delegate,call)
  } 
}

And then make asynchronous access to the stack to put the frames on, then it would work there too. A special method might be needed to synchronize the method calls again. Something like "metaClass.waitForMethodCalls()". Again we could use a method to switch between parallel and normal execution, just like we have done with recursive and iterative execution before. And of course we have the same disadvantages as the last time. Maybe I will have somewhen a bright idea how to solve the return value problem in a much nicer way than using a global variable. I will let you know it then.

And of course I wouldn't propose such a solution for working on a Cell processor, where we simply use as many Threads for the method calls as cells are available... but maybe, with a bit more research, this could be used to make an efficient execution system without the need to explicitly handle Threads. Yes... well... there is the synchronization problem. Not only to wait for methods to end their calls, but also to synchronize access on variables.

As always when walking on the edge, you should know what you do.

Iterative Method Execution Modell in Groovy?

I wonder if that is possible... what? uh, sure, you don't know what I mean. Ok, let us see...

def foo(){
 bar()
}

def bar(){
 foo()
}

You may know this example from the tail recursion article a few days ago. But I want to extend that example now and to this:

def foo(){
 bar()
 baz()
}

def bar(){
 foo()
 baz()
}

def baz(){}

So every method makes an recursion step and then calls the empty baz method. Congratulations, we succesfully eliminated a tail recursion. While executing these methods may never end in a endless loop, as we need to keep a trace to remember we have to call the baz methods, the available recursion depth isn't as deep as I wish it would be. The last time we work around the problem using Closures... Maybe this would work today again? But what to put in the closure? The calls? No that's no solution. Maybe a special version, that means "make a recursive call and give me what you intend to execute after". So our foo might transform into

def foo(){
 recursiveCall (this.&bar) {baz()}
}

we would put the Closure on our own Stack, leaving this method and then execute bar first.

def recursiveCall(Closure method, Closure tail){
 stack << tail  
 stack << method
}

And foo would not be called normally, but this way:

def recursiveCallEntry(Closure method) {
 stack << method  
 while (stack) {
   stack.pop().call()  
 }
} 
recursiveCallEntry(this.&foo) 

what about tree like structures?

def foo() {
 bar1()
 baz1()
 bar2()
 baz2()
}

could be a tree iteration or something. This would be converted into:

def foo() {
 recursiveCall(this.&bar1){baz1()}
 recursiveCall(this.&bar2){baz2()}
}

And does it still work as it should? After executing these methods we have a stack like this one:

bar2,baz2,bar1,baz1

With bar2 on top. That's bad, bar1 needs to be executed first. So we need to reorder that.. but not in recursiveCall. You may see that recursiveCall, is doing something like a collection step. Collecting all method calls that we need to do. So if we collect all the calls the recursiveCall methods do in a list or something, we can simply revert the list and then put the elements in the correct order on the stack. As our recursiveCallEntry method does have the real control over the custom stack, we can use that method.

def recursiveCallEntry(Closure method) {
 stack << method
 while (stack) {
   frame = []
   stack.pop().call()
   stack.addAll(frame.reverse())  
 }
} 

def recursiveCall(Closure method, Closure tail){   
 frame << method   
 frame << tail
}

As you see we no longer revert the order in recursiveCall, since we do this explicitly with frame.revert in recursiveCallEntry. Using this system we can bypass the normal stack, build our own stack and get a much better depth. But "recursiveCall(this.&bar1){baz1()}" looks really ugly. Oh.. do we need that? why distinguish between the real recursive call and the non-recusrive calls? No, I don't think so.

def foo() {
 recursiveCall this.&bar1
 recursiveCall this.&baz1
 recursiveCall this.&bar2
 recursiveCall this.&baz2
}

That is better, but still not nice. We could use Strings for the method name, but we then loose the object we want to execute the method closure on. Oh.. and parameters? In the end the code would look like

recursiveCall (this, "baz2", null)

to call the baz2 method. what most people probably don't know is that such a line already exists in the bytecode, because each normal method call is done by using a method of ScriptBytecodeAdapter. So if had a flag method or something we could use that to execute our methods and keep the readability. Oh no, again a crazy idea is born.

def foo() {
 switchToIterativeCallSystem()
 bar1()
 baz1()
 bar2()
 baz2()
 switchToRecursiveCallSystem()
}

but wait... do we really have to do that on the ScriptBytecodeAdapter? We can do that with the MetaClass too! We simply need to replace the MetaClass for "this" with our own crazy version...

class IterativeCallerMetaClass extends  DelegatingMetaClass {
 def stack =[]
 def frame
 boolean iterativeCallSystem=false

 public IterativeCallerMetaClass (final MetaClass delegate) {
   super(delegate);
 }

 def invokeMethod(Object object, String methodName, Object[] arguments) {
   if (iterativeCallSystem){
     frame << [object,methodName,arguments]  
   } else {    
     delegate.invokeMethod(object, methodName, arguments);  
   }  
   return null
 } 

 def recursiveCallEntry(Closure method) {   
   stack << method  
   while (stack) {
     frame = []
     def call = stack.pop()
     delegate.invokeMethod(call[0], call[1], call[2]);
     stack.addAll(frame.reverse())   
   }  
 }
} 

And the complete calling code would maybe llok like this:

def foo() {
  metaClass.iterativeCallSystem = true
  bar1()
  baz1()
  bar2()
  baz2()
  metaClass.iterativeCallSystem = false
}

this.setMetaClass ( new IterativeCallerMetaClass (this.getMetaClass()) )

Disadvantages? Yes, sure. This completly ignores the return value. And sadly I don't see a way to fizzle it in. I mean a call foo(foo(1)) is really something like "tmp = foo(1); foo(tmp)". That means I need the value of "tmp" while I am still in the collection phase, where it isn't available, since the call isn't done then. Next disadvantage is, that that MetaClass only works on one object. Any foo.toString(), where foo!=this, isn't collected. Which directly leads to the next problem, logic operations are executed before the values are available.

To work around that we can put all that code in Closure and add these closures to the custom MetaClass stack, but that gives again problems with local variables and such. If you have a tree with an unknown number of children and you want to do an depth first search on the children, you would normally do something like that:

def search(node,constraint) {
 if (constraint(node)) return node
 node.children.each{search(it,constraint)}
}

But when you want to use our iterative system...

def search(node,constraint) {
 metaClass.add {
   if (!found && !constraint(node)){
    node.children.each {search(it,constraint)}
  } else {
    found=node
  }
 }
}

That's an ugly level of indirection... I used the global variable found to indicate if we need to search deeper, this avoids the return value. Then the closure will be put on the stack and executed when the data is available, collecting the search calls and later execute each of them. This can only work if all actions other than pure method calls on this are done in closures.

Conclusion:
This let's me think that such an solution should be much more easy in Ruby. But as I don't know enough about Ruby I don't know if there is such a MetaClass system as we have in Groovy. But being able to simply put the method body on the stack instead of a closure would be nice. Anyway, it works. We can also remove the "metaClass.iterativeCallSystem=" lines with the convention that the system is always active. Of course this works with a builder just as fine as with a normal class, you only have to do the job of our MetaClass in the builder then.

You see, many tricks can be done with the MetaClass, even a total different method execution model. Ok, not toally different, as the VM still does the invocation and still keeps a trace, but we successfully shortend that stack trace. Oh, and of course the above works for tail recursion just as well. You can say we included them here already. Of course the solution developed into a completly different direction than my previous post about tail recursion.

And maybe a word about the execution speed. Of course you add an additional level of indirection for method calls, our delegating MetaClass. So instead of doing one mthod call we do much more of them, for example when testing the stack if it is empty, when getting the top frame and more. But if your focus is on avoiding a stack overflow it works just fine.

Tail Recursion in Groovy?

Is it possible?

Tail recursion can be very useful when traversing big lists or trees using recursive methods. The JVM doesn't support tail recursion - maybe there is a way to simlulate them. But first, what is a tail recursion?

A tail recursion is, well I thik the wikipedia explanation is quite good ;) For me the important thing here is to reduce the Stack pollution and not to get an exception from the JVM, because the trace got too big. So a transformation of one kind or another is needed to avoid that.

I know 3 kinds of techniques to accomlish this:

1. use the compiler to transform the recursion into a loop
2. let the vm recognize the recursion and eliminate it
3. mark tail recursion specially and react different then normal

To number 1 I must say, that we could do this in Groovy - partially. That would only work for self recursive functions and it requires the ability to ask the method execution part if that method call is a call to the current method or not. For Java, the last thing is out of question, there is no such query mechanism at runtime to do that. And there is no need, as all methods are selected at compile time. So Java could have this kind of tail recursion. In Groovy we can't be sure if the method we want to call is really being executed. And we may not be sure that the parameters we give really result in the desired method to be executed. Of course we could implement such a query system in Groovy and then we would have the answer. But that would give us only a partial solution as I mentioned already.

Then number 2. Well as long as the Java people don't decide that Java should have that, there is no way for us to get that done in Groovy. Groovy is not a VM in the VM, it is a runtime which does influence the method calls, yes, but it has no real control over the stack.

So what about number 3? Is there something we could do to get tail recursion? Before I get deep in the details imagine the stack that is created for

def foo(){
 bar()
}
def bar(){
 foo()
}

foo()

We will create a stack element for foo, then, when calling bar, we will create a new stack element for bar. After this one again for foo, and this would go on until the JVM complains about the stack becoming too big. What if we would now not directly cal the method, but return from the current stack frame and then do the call? Using Closures we can simulate that:

def foo(){
 {bar()}
}
def bar(){
 {foo()}
}
def cl = foo()
while (cl!=null) {
 cl = cl()
}

With the result, that the function foo is called and returns with the closure. After this, the closure is called, which does call bar. Then bar returns with the closure and the stack size becomes 0 again. Then the closure is again called. This continues till someone forcefully terminates the program, since we crated an endless loop. Of course it is not very nice to have to do that this way. An autmatism would be nice.

So let us say there are keywords like... trreturn and trcall, well not nice, but it is just for illustration. This signals the runtime to not to simply return after the call was made, but to examine the result and react to this. You need to know, that a normal method call foo() is in fact this: ScriptBytecodeAdapter.invokeMethod("foo"). So instead of using the normal invoke we use the our special invoke method invokeMethodTR.

def invokeMethodTR (String name) {
def c = invokeMethod(name)
while (c instanceof TailCall) {
 c = invokeMethod(c.name)
}
return c
}

and foo would be for the compiler:

def foo(){
 return new TailCall(name:"bar")
}

a normal user would see this:

def foo(){
 trreturn bar()
}

But the initial call to foo must be done using trcall, so the complete program would look like:

def foo(){
 trretrun bar()
}
def bar(){
trreturn foo()
}
trcall foo()

This way to do this has actually two problems:

1. Calling such a method from Java would give a TailCall isntance.
2. a void method can't return a TailCall instance

The most easy way to go around that would be not to avoid it. Instead we could use it as marker for the method and remove the trreturn:

TailCall foo(){
 bar()
}
TailCall bar(){
 foo()
}
trcall foo()

The TailCall instance may not only transport the method name and arguments, but also a return value if needed.

Conclusion:
All in all at last one additional keyword is needed to get it working? Maybe using a closure here not even that is needed. We could just say tailcall{foo()} and leave the loop part of the implementation to that method. The compiler is still needed, because someone does have to create the TailCall instances for the return. But the special invokeTR method isn't needed any longer. So any changes to the compiler are doable in no time. Regarding to my question at the start, if it is possible.. well, yes! It's not even very difficult!

Beyond Groovy 1.0: Groovy goes Lisp

I was always a fan of LISP, but not of the huge amount of (). Regardless the syntax, LISP gets it power through exactly that, the syntax. The late Binding of expressions and the simplicity of the language in syntax and semantic allows to extend the language itself very easily. And that again allows you to write DSLs embedded in your normal program. And how to get something like that to groovy? A list looks in groovy like that:

[1,2,3]

but that doesn't contain a operation. Adding an operation as normal Element would require a closure - I don't think that is very nice. Anyway, if you could remove the "," we would have a much nicer list:

[1 2 3]

Now let us assume that is something like a quoted list in LISP. How to get an operation inside? If we could omit the comma in general, then we could use a normal method call

add 1 2 3

which is the same as

add(1, 2, 3)

and if we need late binding, we can use closures

{add 1 2 3}

now, what is a simple program in LISP? Maybe

(defun add (x y)
(+ x y))
(add 1 2)

and what would it look like in Groovy with the optional comma?

{defun add {x y}
{x+y}}
{add 1 2}

Of course the curly braces are not that nice here, but needed. Anyway, it looks really a bit like the Lisp example, but would it work? Remember, defun would be a method call and "add" a parameter to that method. That means "add" is evaluated before defun is. So a evaluation system is needed here, that allows to handle such cases. So instead of evalutating directly to a value, we simply return a place holder. Let us name that place holder symbol here. So in fact the defun method would get a symbol and two closures. Ok, now let us define that every method call itself is first evaluted using symbols, so

{defun add {x y}
{x+y}}

would return a list with two symbols (defun and add) and two closures. Let us call the function that does this job preeval. In Groovy terms, preeval is a builder or special MetaClass, that intercepts all calls to methods and properties and does return a list of symbols and closures instead. Then a secound stage evaluation process would start and use that list to execute functions. Let us assume "defun" is delegated to some kind of method, then this method might look like:

def defun(Symbol name, Closure parameter, Closure body) {
evaluationSystem. register (
  name,
  preeval(arguments),
  body
)
}

Again preeval would return a list of symbols, this time they wouldn't be mapped to a method, they would be used to define a scope for the body closure. And when, the the body closure asks for x and y, the evalutation system would know, that these two are in fact the arguments. After this method is available, we can define car and cdr using defun:

{defun car {list}
{list[0]}
}
{defun cdr {list}
{list[0..-1]}
}

Of course having a generic name for car and cdr would also allow the "special" functions caaar and cadar and what else. This is enough to build a powerful LISP like system. I would have to look how macros are realized in LISP, but theoretically these basics here should be enough to allow them too. And then we could do code like this:

def result = lispBuilder.eval
{defun groovy { a b}
{a.name.length() + b.name.length()*10}
}
{groovy is lisp}

assert result == 42

Anyway, that is just a crazy idea, but possible. And all this only by making the comma optional. Of course it would work without that too, but {groovy is,lisp} doesn't read as nice as the version without comma. It would also allow us to become more like Smalltalk, but I will show that in another blog entry. I just wanted to show you how it could be made possible to get something with a sligthly feeling on LISP in Groovy. I won't say that the above fragments are really LISP, but I think by developing this idea to its end it would look much like LISP. but it doesn't have to be LISP, it could be also Scheme and then we could compare Kawa and Groovy *lol*

Conclusion:
There are limitations. I mean names can't have all shapes without being put into "" and requiering a qualificator like here: this."strange variable name". And of course you won't be able to stop Groovy from doing early binding for local variables - I am not sure that is really a disadvantage. I wouldn't expect LIPS people to really migrate from LISP to Groovy just because of this blog entry here. But maybe some LISP people out there, which are forced to program on the JVM might want to consider using Groovy in the future ;)

Getting Groovy 1.0 out is currently more important than having crazy ideas about making Groovy into a overcomplicated LISP. But I must say, I really like this idea.

EDIT:
there is a small error in the artice, closures need to be concatenated by } {, there can't be a newline in between. But maybe this is another thing to change?

Show me what you have! Featuring Groovy, Swing and db4o

Once again I had to write one of these stupid and boring GUIs. You know these things asking you stupid questions and showing irrelevant Data. Of course I do all in Groovy, no need for a IDE provided gui generation tool. So I used the SwingBuilder to make the GUI.

Really, I understand why people are asking for documentation. The 2 examples here are giving good examples, but basically you have to understand that components are registered and each property can be used using the map syntax. Embedding one component in another works using Closures. but there are things the Swing builder currently can't do. For example registering a MouseListener at a component using Closures as the functions executed by the MouseListener. Or a WindowsAdapter with Closures for certain action, like closing the window. So you still have to create a big bunch of useless classes. That's not nice. Ok, they are not that useless, because you can reuse them later.

We (the Groovy Team) had the idea to convert a closure to a class implementing a certain interface with just one method by the MetaClass or using the as-syntax. Just only that this won't help here, a listener normally has more than one method. So for now, I decided to make the helper classes, but maybe we should simply add an method to the Swingbuilder allowing to do such things. Would be possible and not that difficult I guess.

What puzzled me a bit was, that I have to do things like:

scrollpane {
 viewport {
   list()
 }
}

Now I know a viewport is inside the ScrollPane, but that I have to do that explicitly isn't very nice. In fact I would like to have a default property and then be abel to do that:

list (scrollpane:true)

or at last

scrollpane {
 list()
}

Anyway, it doesn't bother me too much.

So I wrote some glue classes like this:

class ClosureWindowClosingAdapter extends WindowAdapter {
   def closure
   void windowClosing(WindowEvent e) {
       closure(e)
   }
}

not much logic there, but it allows me to do things like that:

gui.addWindowListener (
 new ClosureWindowClosingAdapter(closure:{System.exit(0)})
)

After I have done all these little things I got a nice small gui application. Just a few lines long itself, some minor helper classes and yet no data.
Yes, that's the point where db4o comes to play with us. I had many lists basically showing all available instances of one type. And I never understood why I should make a custom model here to keep multiple lists of the same objects, maybe concat these lists so they can notify each other about changes and such.

I decided I want to use a central data holder this time, the objects are extracted from there using query-by-example and, no, the data holder is not a relational database with hibernate in the foreground. I wanted to have something easy, that allows me to change the class structure and doesn't requires as much overhead as hibernate. So I used one of my helper classes and wired some db4o parts in:

class ClosureQueryModel extends AbstractListModel implements ComboBoxModel{
    def cl
     def getElementAt(int i){cl()[i]}
     int getSize(){cl(query).size()}
    def selectedItem
    def addElement(foo){/* ignore */}
    def remove(foo){/* ignore */}
}

combo.setModel(new ClosureQueryModel(cl:closure))

nice isn't it? Again a predefined class would make life more easy here, but well, it isn't that this helper is a big one ;) and yes, I am ignoring removal and additon of elements, since I wanted to make that completely through the database. cl(query)[i] means I need a list or array or something like that as result. Now in db40 4.3 this would have been a problem, since a query on the database does return an instance of ObjectSet, which is no such list or array. But they changed that, and in 5.2, which means I can directly use the result of the query.

For all people here not knowing how such an query looks like:

def objectSet = db.get(Foo)

yes, that's all. Foo is the class of the objects I want to have and the above query allows me to get all instances from the database. So I would create a ClosureQueryModel isntance like that:

new ClosureQueryModel(cl:{db.get(Foo)})

So now my items would be displayed... but... there is no initial data. What is missing is a functionality to add remove and update items. Well, maybe I got here a bit over the edge, because I got this crazy idea of using a mapping to show the editable fields... but on the other hand, any bean editor is allowing comparable to that. Only that I don't use explicit beans and only that I don't use these extra files for beans describing the bean. Anyway, I made a map like this one:

[name:"field",view:"Feld",collection:{it.availableFields}]

to set a property/field named field, with the display name Feld and it is chooseable from a collection, that is returned by a closure {it.availableFields}. so basically I can choose here between predefined values returned by a property or whatver I use in the closure. then I thought that a simple collection is possibly not enough. What about numbers? so I added a "create" to the map, taking a closure that produces the object:

[name:"workload",view:"Auslastung in %",create:{new BigDecimal(it)}]

create and collection would be exclusive then, but I added no test for that.

I thought, ok, the let us have a popup menu, for the crud operations and put all that logic together in a method.

addPopup(
  component:jlist,
  producer:WorkloadEntry.class,
  maping:[
    [name:"workload",view:"Auslastung in %",create:{new BigDecimal(it)}] ,
    [name:"field",view:"Feld",collection:{it.availableFields}]
  ],
  newAction:{db.set(it)},
  updateAction:{db.set(it)},
  deleteAction:{db.delete(it)})
}

producer is the class use to create new instances from, mapping contains the fields/properties set for the new class and how they are displayed, and the *Actions are he database methods for the crud operations. Pretty easy, but the mapping tends to be big for man fields. I should think of something different here.

I don't want to show all of the code of that method here. The method itself is 100 lines long, as much as the rest of the program. A sure sign, that it is too complicated. But well, the basic parts:

I need to display the views:

  args.maping.each { item ->
    label(item.view+":",constraints:gbc1)
      if (item.collection != null) {
        def combo = comboBox(constraints:gbc1)
        combo.actionPerformed = {query.setProperty(item.name, combo.selectedItem)}
        combo.setModel(new ClosureQueryModel(query,item.collection))
      } else if (item.model){
        def combo = comboBox(constraints:gbc1)
        combo.actionPerformed = {query.setProperty(item.name, combo.selectedItem)}
        combo.setModel(new ChoiceModel(item.model))
      } else {
        selection << textField(constraints:gbc1)
      }
   }

saving it would look like:

  args.maping.eachWithIndex { values,index ->
    def sel = getViewComponent(index)
    if (sel instanceof JComboBox){
      obj.setProperty(values.name, sel.selectedItem)
    } else {
      def val = sel.text 
      if(values.create!=null) val = values.create.call(val)
      obj.setProperty(values.name, val)
    }
  }
  [...]
  if (args.newAction!=null) args.newAction.call(obj)
  args.component.model.addElement(obj)
  args.component.model.fireContentsChanged(editWindow, args.component.selectedIndex, args.component.selectedIndex)

You can see the create closure here, called if set and if a JTextField was used to display it. Maybe giving the component instead of the text only would be also a improvement and would allow other components. But this was more of a case study, not a general purpose solution.

Even without the usage of the database, such a component should be thought through. I mean you need something like that quite often.

Conclusion:
Groovy's Swing part currently consists only of the SwingBuilder. That helps a lot creating guis, but when it comes to models and listeners, the SwingBuilder lacks a huge amount of them. Models for spinner and tables are there, but not more. I think, developing Swing with such components can improve your productivity when it comes to make a prototype. You can alaways decide to change the class model and you can always change the gui itself. You could even think about a text, that you enter when a button is pressed and then that text is executed as script. Of course the combination with an plugin such as the eclipse groovy plugin would make things even more nice. But even without such self hosting features, it makes much more fun than plain Java, even if a WYSIWYG editor is involved. If I had more time for Groovy I would no go on and improve the Swing abilities of Groovy big time.. but maybe there is someone else who wants to do that. Groovy is open source and people doing such things are always welcome.

I hope I won't have to say that last part too often. Of course I would like to do all the ideas I have by myself, but I can only spend a few hours every week on Groovy and that is not quite enough to get issues fixed and to add new ideas.

a new blog - oh my, why that again?

Hi there

I am about to start my new blog here. In the future I want to let you people know about java and groovy related thoughts, problems and stories - maybe other stuff too.