Changes in this revision:

• The throws clause of a function type is now placed between the curly braces.
• The description of function types has been rewritten to emphasize more clearly that function types are interface types rather than a separate extension to the type system.
• The closure conversion can now convert a closure that has no result expression to an interface type whose function returns java.lang.Void. This change helps support completion transparency. A completion transparent method is one written such that the compiler can infer that an invocation completes normally iff the closure passed to it completes normally.
• Some examples have been modified to be completion transparent.
• null is now a subtype of Unreachable, and a number of small related changes. Thanks to Rémi Forax for pointing out the issues.

Closures for the Java Programming Language (v0.4)

Gilad Bracha, Neal Gafter, James Gosling, Peter von der Ahé

Modern programming languages provide a mixture of primitives for composing programs. Most notably, Ruby, Scala, Smalltalk, and Simula have direct language support for delayed-execution blocks of code, called closures. Closures provide a natural way to express some kinds of abstractions that are currently quite awkward to express in Java. For programming in the small, closures allow one to abstract an algorithm over a piece of code; that is, they allow one to more easily extract the common parts of two almost-identical pieces of code. For programming in the large, closures support APIs that express an algorithm abstracted over some computational aspect of the algorithm. There are two alternative specifications: the nominal version includes closure literals but requires programmer-provided interfaces for their types while the functional version includes all of the features of the nominal version but also includes a syntax for function types, which are system-defined interface types. This document contains both the nominal and the functional versions of the specification.

Closure Literals

We introduce a syntactic form for constructing a closure value:

Primary:

Closure

Closure:

{ FormalParameterDecls_opt => BlockStatements_opt Expression_opt }

A closure is converted to some object type at compile-time by a closure conversion. In the nominal version of the specification, it is a compile-time error if a closure appears in a context where it is not subject to a closure conversion. In the functional version of the specification, if a closure is not subject to a closure conversion it is converted to the corresponding function type of the closure, which is the function type with: identical argument types; a return type that is the type of the final expression, if one exists, or java.lang.Unreachable if the closure body cannot complete normally, or void otherwise; and a throws type list corresponding to the checked exception types that can be thrown from the body of the closure. The conversion, in either case, occurs entirely at compile-time.

Example: the following closure takes two integers and yields their sum:

{int x, int y => x+y}

A closure captures a block of code - the block statements and the expression - parameterized by the closure's formal parameters. All free lexical bindings - that is, lexical bindings not defined within the closure - are bound at the time of evaluation of the closure expression to their meaning in the lexical context in which the closure expression appears. Free lexical bindings include references to variables from enclosing scopes, and the meaning of this, break, continue, and return. Evaluating the closure expression does not cause the statements or expression to be evaluated, but packages them up at runtime with a representation of the lexical context to be invoked later.

Rationale: One purpose for closures is to allow a programmer to refactor common code into a shared utility, with the difference between the use sites being abstracted into a closure. The code to be abstracted sometimes contains a break, continue, or return statement. This need not be an obstacle to the transformation. One implication of the specification is that a break or continue statement appearing within a closure may transfer to a matching statement enclosing the closure. A return statement always returns from the nearest enclosing method or constructor. A closure may outlive the target of control transfers appearing within it.

At runtime, if a break statement is executed that would transfer control out of a statement that is no longer executing, or is executing in another thread, the VM throws a new unchecked exception, UnmatchedNonlocalTransfer. Similarly, an UnmatchedNonlocalTransfer is thrown when a continue statement attempts to complete a loop iteration that is not executing in the current thread. Finally, an UnmatchedNonlocalTransfer is thrown when a return statement is executed if the method invocation to which the return statement would transfer control is not on the stack of the current thread.

Function Types

Closure Conversion

A closure may be assigned to a variable or parameter of any compatible interface type by the closure conversion:

There is a closure conversion from a closure to every interface type that has a single method m such that the closure is compatible with m. A closure is compatible with a method m iff all of the following hold:

• Either
  • The closure has no final expression and the method m has return type void or java.lang.Void; or
  • The closure has a final expression and there is an assignment conversion from its type to the return type of m; or
  • The body of the closure cannot complete normally.
• m has the same number of arguments as the closure.
• For each corresponding argument position in the argument lists, both argument types are the same.
• Every exception type that can be thrown by the body of the closure is a subtype of some exception type in the throws clause of m.

The closure conversion supports converting a closure that yields no result to an interface whose method returns java.lang.Void. In this case a null value is returned from the function. This is necessary to write APIs that support completion transparency: that is, in which an invocation of the API can return normally if and only if the closure that is passed to it can complete normally.

If the target of the closure conversion extends the marker interface java.lang.RestrictedClosure then

• it is a compile-time error if the closure being converted contains a break, continue or return statement whose target is outside the closure; and
• it is a compile-time error if the closure being converted refers to a non-final local variable declared in an enclosing scope.

The closure conversion to a "restricted" interface only allows closures that obey the current restrictions on anonymous instances. The motivation for this is to help catch inadvertent use of non-local control flow in situations where it would be inappropriate. Examples would be when the closure is passed to another thread to run asynchronously, or stored in a data structure to be invoked later. We expect to retrofit a number of existing JDK classes with this marker interface.

Example: We can write a closure that adds two to its argument like this:

interface IntFunction {
    int invoke(int i);
}
IntFunction plus2 = {int x => x+2};

Alternatively, using function types:

{int=>int} plus2 = {int x => x+2};

We can use the existing Executor framework to run a closure in the background:

void sayHello(java.util.concurrent.Executor ex) {
    ex.execute({=> System.out.println("hello"); });
}

Exception type parameters

To support exception transparency, we add a new type of generic formal type argument to receive a set of thrown types. [This deserves a more formal treatment] What follows is an example of how the proposal would be used to write an exception-transparent version of a method that locks and then unlocks a java.util.concurrent.Lock, before and after a user-provided block of code. In the nominal version of the proposal:

interface Block<T,throws E> {
    T invoke() throws E;
}
public static <T,throws E extends Exception>
T withLock(Lock lock, Block<T,E> block) throws E {
    lock.lock();
    try {
        return block.invoke();
    } finally {
        lock.unlock();
    }
}

Or using the functional version of the proposal:

public static <T,throws E extends Exception>
T withLock(Lock lock, {=>T throws E} block) throws E {
    lock.lock();
    try {
        return block.invoke();
    } finally {
        lock.unlock();
    }
}

This can be used as follows:

withLock(lock, {=>
    System.out.println("hello");
});

This uses a newly introduced form of generic type parameter. The type parameter E is declared to be used in throws clauses. If the extends clause is omitted, a type parameter declared with the throws keyword is considered to extend Exception (instead of Object, which is the default for other type parameters). This type parameter accepts multiple exception types. For example, if you invoke it with a block that can throw IOException or NumberFormatException the type parameter E would be given as IOException|NumberFormatException. In those rare cases that you want to use explicit type parameters with multiple thrown types, the keyword throws is required in the invocation, like this:

Locks.<throws IOException|NumberFormatException>withLock(lock, {=>
    System.out.println("hello");
});

You can think of this kind of type parameter accepting disjunction, "or" types. That is to allow it to match the exception signature of a closure that throws any number of different checked exceptions. If the block throws no exceptions, the type argument would be the type null (see below).

Type parameters declared this way can be used only in a throws clause or as a type argument to a generic method, interface, or class whose type argument was declared this way too.

We introduce a meaning for the keyword null as a type name; it designates the type of the expression null. A class literal for the type of null is null.class. These are necessary to allow reflection, type inference, and closure literals to work for functions that result in the value null or that throw no checked exceptions. We also add the non-instantiable class java.lang.Null as a placeholder, and its static member field TYPE as a synonym for null.class.

Definite assignment

The body of a closure may not assign to a final variable declared outside the closure.

A closure expression does not affect the DA/DU status of any free variables it names.

Free variables referenced inside a closure receive their initial DA value from the DA status of the variable at the point where the closure expression appears.

The type Unreachable

We add the non-instantiable type java.lang.Unreachable. Values of this type appear where statements or expressions cannot complete normally. This is necessary to enable writing APIs that provide transparency for closures that do not return normally. Unreachable is a subtype of every obect type. The null type is a subtype of Unreachable. At runtime, a NullPointerException is thrown when a value of type Unreachable would appear. This occurs when a programmer converts null to Unreachable.

Example: The following illustrates a closure being assigned to a variable of the correct type.

interface NullaryFunction<T, throws E> {
    T invoke() throws E;
}
NullaryFunction<Unreachable,null> thrower = {=> throw new AssertionError(); };

Reachable statements

The initial statement of a closure is reachable.

An expression statement in which the expression is of type Unreachable cannot complete normally.

Control invocation syntax

A new invocation statement syntax is added to make closures convenient for control abstraction:

ControlInvocationStatement:

Primary ( Formals : ExpressionList_opt ) Statement
Primary ( ExpressionList_opt ) Statement

This syntax is a shorthand for the following statement:

Primary ( ExpressionList, { Formals => Statement } );

This syntax makes some kinds of closure-receiving APIs more convenient to use to compose statements.

Note: There is some question of the correct order in the rewriting. On the one hand, the closure seems most natural in the last position when not using the abbreviated syntax. On the other hand, that doesn't work well with varargs methods. Which is best remains an open issue.

We could use the shorthand to write our previous example this way

withLock(lock) {
    System.out.println("hello");
}

Rationale: This is not an expression form for a very good reason: it looks like a statement, and we expect it to be used most commonly as a statement for the purpose of writing APIs that abstract patterns of control. If it were an expression form, an invocation like this would require a trailing semicolon after the close curly brace of a controlled block. Forgetting the semicolon would probably be a common source of error. The syntax is convenient for both synchronous (e.g. see withLock) and asynchronous (e.g. see Executor.execute) use cases.

Another example of its use would be a an API that closes a java.io.Closeable after a user-supplied block of code, discarding any exception from Closeable.close:

class OneArgBlock<R, T, throws E> {
    R invoke(T t) throws E;
}
<R, T extends java.io.Closeable, throws E>
R with(OneArgBlock<R, ? super T,E> block, T t) throws E {
    try {
        return block.invoke();
    } finally {
        try { t.close(); } catch (IOException ex) {}
    }
} 

Or using the functional version:

<R, T extends java.io.Closeable, throws E>
R with({T=>R throws E} block, T t) throws E {
    try {
        return block.invoke();
    } finally {
        try { t.close(); } catch (IOException ex) {}
    }
} 

We could use the shorthand with this API to close a number of streams at the end of a block of code:

with(FileReader in : makeReader()) with(FileWriter out : makeWriter()) {
    // code using in and out
}

Type inference

The rules for type inference need to be augmented to accomodate the inference of exception type parameters. Similarly, the subtype relationships used by the closure conversion need to be reflected as well.

Future Directions

Specify the precise mapping from a function type to the system-provided generic interface.

Specify the unmatched nonlocal transfer exception in more detail, including support for resuming them across threads.

Elided exception type parameters enable existing interfaces to be retrofitted for exception transparency.

Allow volatile on locals.

Abstractions for loops that work with break, continue?

Acknowledgments

The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal:

Lars Bak, Cedric Beust, Joshua Bloch, Martin Buchholz, Danny Coward, Erik Ernst, Rémi Forax, Christian Plesner Hansen, Kohsuke Kawaguchi, Doug Lea,"crazy" Bob Lee, Martin Odersky, Tim Peierls, John Rose, Ken Russell, Mads Torgersen, Jan Vitek, and Dave Yost.