Java Closures Prototype Feature-Complete

I'm pleased to announce that the Java Closures prototype now supports all of the features of its specification!

The complete source code, released under GPLv2, is in the project's openjdk repository. A binary build, suitable for use with an existing JDK6, is at http://www.javac.info/closures.tar.gz. Other related documents are on the website http://www.javac.info/

Although there is room for performance tuning, the prototype supports the full Closures (v0.5) specification. Based on your feedback, there are some changes in the prototype suitable for a future update of the specification:

• Renamed Unreachable to Nothing

We adopt the name used by Scala to represent the same concept.

• Removed support for the type null

We used null as a placeholder for an exception type when none can be thrown. The type Nothing now serves that purpose; null is no longer supported as the name of a type.

• Overhauled restricted versus unrestricted

In the specification, an interface is considered restricted if it extends a marker interface. Unfortunately, the specification only provides a syntax for function type interfaces that are unrestricted. We modified the syntax so that a function type written using the => token designates a restricted function type, while one written using the newly introduced ==> token represents an unrestricted function type. This allows programmers to easily write APIs that restrict (or don't restrict) the operations of closure expressions passed as parameters.

• Refined restrictions

We modified the distinction between restricted and unrestricted closures. As before, it is not legal to convert an unrestricted closure to a restricted interface type, nor is it legal to break, continue, or return from inside a restricted closure to a target outside the closure. However, a restricted closure is allowed to refer to a non-final local variable from an enclosing scope. In this case a warning is given unless one of the following conditions holds:

1. The variable is not the target of any assignment, or
2. The variable is annotated @Shared

It is possible to suppress the warning by annotating some enclosing construct @SuppressWarnings("shared").

• Relaxed the closure conversion

In response to user feedback, we've relaxed the relationship between a closure parameter's type and the target interface's parameter type. Rather than requiring them to be of the same type, they are now allowed to be related by an assignment conversion, including boxing or unboxing.

• for-qualified method declarations

The for keyword on a method declaration, meant to introduce a control abstraction method that works like a loop, is now treated syntactically like a modifier rather than appearing immediately before the method name. This helps make the declaration site more similar to the use site.

• Added support for method references

We added extensive support for treating a reference to a method as a closure using a newly introduced token #. The syntax is borrowed from the FCM proposal. The semantics are as follows:

A method reference written as

Primary # Identifier ( TypeList )

where the Primary designates an expression (as opposed to a type) is treated the same as a closure

{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...) }

or

{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...); }

Where tmp is a temporary value that holds the computed value of the primary expression. The former translation is used when the resolved method has a non-void return type, while the latter is used when the resolved method has a void return type.

If the primary resolves to a type, then this is translated to

{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...) }

or

{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...); }

when the resolved method is static, or

{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...) }

or

{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...); }

when the resolved method is an instance method.

In addition, optional explicit type arguments, between angle brackets, may be placed immediately after the # token. These are used directly in the translated method invocation to resolve the method to be invoked.

• Implemented a classfile format for the for qualifier

We've impleemnted a class file representation of the for qualifier to support separate compilation.

Closures: Control Abstraction, Method References, Puzzler Solution

The Java Closures prototype now supports control abstraction and implements restricted closures and function types. The syntax has changed slightly. Also, as hinted in the draft JSR proposal, there is now support for eta abstraction, which is called method reference in Stephen Colebourne's FCM proposal. We haven't updated the specification, so this will serve as a brief tutorial on the changes until we do. I don't know if this will be the syntax we will end up with, but it will do for now. Finally, we look at solutions to the closure puzzler in my previous post.

Control Abstraction

The first thing you'll notice when using the new prototype is that the compiler gives a warning when a closure uses a local variable from an enclosing scope:

Example.java:4: warning: [shared] captured variable i not annotated @Shared
        Runnable r = { => System.out.println(i); };
                                             ^

There are a few ways to make this warning go away:

• declare the variable final; or
• annotate the variable @Shared; or
• make sure the variable is not the target of any assignment expression; or
• put @SuppressWarnings("shared") on an enclosing method or class; or
• use an unrestricted closure, by using the ==> token instead of the => token (when possible).

The => token builds a restricted closure that triggers this warning. Restricted closures also do not allow a break or continue statement to a target outside the closure, nor a return statement from the enclosing method. You will rarely want to write an unrestricted closure; many (but not all) of the things you need to do with an unrestricted closure can be expressed more clearly with a control invocation statement instead.

You're not allowed to assign an unrestricted closure to a restricted interface. A number of existing JDK interfaces, such as java.lang.Runnable, have been modified to be restricted.

Error: cannot assign an unrestricted closure to a restricted interface type
        Runnable r = { ==> System.out.println(i); };
                       ^

In the less common case that you're writing a method intended to be used as a control API, you can write a function type with the (new) ==> token to designate an unrestricted function (interface) type. Let's do that to write a method, with, that will automatically close a stream for us. The idea is to be able to replace this code

FileInputStream input = new FileInputStream(fileName);
try {
    // use input
} finally {
    try {
        input.close();
    } catch (IOException ex) {
        logger.log(Level.SEVERE, ex.getMessage(), ex);
    }
}

with this

with (FileInputStream input : new FileInputStream(fileName)) {
    // use input
}

which is an invocation of the following method

public static void with(FileInputStream t, {FileInputStream==>void} block) {
    try {
        block.invoke(t);
    } finally {
        try {
            t.close();
        } catch (IOException ex) {
            logger.log(Level.SEVERE, ex.getMessage(), ex);
        }
    }
}

This is among the simplest control APIs, but it has some limitations:

• It works with the type FileInputStream, but not any other Closeable types
• It does not support exception transparency
• It does not support completion transparency

Completing the API by repairing these defects is left as an exercise to the reader. A solution will be discussed in my JavaOne talk Closures Cookbook.

Method References

A natural companion to closures is a way to refer to an existing method instead of writing a closure that accepts the same arguments and just invokes the method. This is sometimes known as eta abstraction or method references. We expect closures in their final form to include support for this convenient feature, which is why it is called out in the draft JSR proposal. The latest version of the prototype supports this, with a syntax based on javadoc conventions. Here are a few examples:

{ int => Integer } integerValue = Integer#valueOf(int);
{ Integer => String } integerString = Integer#toString();
{ int, int => int } min = Math#min(int, int);
{ String => void } println = System.out#println(String);
{ => String } three = new Integer(3)#toString();
{ Collection<String> => String } max = Collections#max(Collection<String>);
{ => Collection<String> } makeEmpty = Collections#<String>emptySet();
Runnable printEmptyLine = System.out#println();

Writing code as a method is sometimes more convenient than writing it as a closure:

void doTask() {
    // a complex task to be done in the background
}


Executor ex = ...;
ex.execute(this#doTask());

Puzzler Solution

A couple of weeks ago we looked at a Java puzzler involving closures, and a number of people discussed the underlying issue. My favorite is David's post "Color-flavor locking breaks chiral symmetry". Lessons include not exposing public fields (accessors are better) and being careful to avoid cyclic initialization dependencies.

The enum language feature provides support for one solution to the puzzle: specialize each instance of the enums.

import java.util.*;

enum Color {
    BROWN {
        public Flavor flavor() {
            return Flavor.CHOCOLATE;
        }
    },
    RED {
        public Flavor flavor() {
            return Flavor.STRAWBERRY;
        }
    },
    WHITE {
        public Flavor flavor() {
            return Flavor.VANILLA;
        }
    };
    abstract Flavor flavor();
}

enum Flavor {
    CHOCOLATE {
        public Color color() {
            return Color.BROWN;
        }
    },
    STRAWBERRY {
        public Color color() {
            return Color.RED;
        }
    },
    VANILLA {
        public Color color() {
            return Color.WHITE;
        }
    };
    abstract Color color();

}

class Neapolitan {

    static <T,U> List<U> map(List<T> list, {T=>U} transform) {
        List<U> result = new ArrayList<U>(list.size());
        for (T t : list) {
            result.add(transform.invoke(t));
        }
        return result;
    }

    public static void main(String[] args) {
        List<Color> colors = map(Arrays.asList(Flavor.values()), { Flavor f => f.color() });
        System.out.println(colors.equals(Arrays.asList(Color.values())));

        List<Flavor> flavors = map(Arrays.asList(Color.values()), { Color c => c.flavor() });
        System.out.println(flavors.equals(Arrays.asList(Flavor.values())));
    }
}

Another elegant solution, due to 5er_levart, uses closures:

enum Color {
    BROWN({=>Flavor.CHOCOLATE}),
    RED({=>Flavor.STRAWBERRY}),
    WHITE({=>Flavor.VANILLA});

    private final {=>Flavor} flavor;

    public Flavor flavor() { return flavor.invoke(); }

    Color({=>Flavor} flavor) {
        this.flavor = flavor;
    }
}

enum Flavor {
    CHOCOLATE({=>Color.BROWN}),
    STRAWBERRY({=>Color.RED}),
    VANILLA({=>Color.WHITE});

    private final {=>Color} color;

    public Color color() { return color.invoke(); }

    Flavor({=>Color} color) {
        this.color = color;
    }
}

In both solutions the idea is to compute the value lazily, a key technique to break dependency cycles.