This revision of the closures proposal describes both the nominal and the functional versions of the proposed language specification, with a brief description of the differences between the two. A new syntax is introduced for both function types and closure literals. Function types are now defined as language-provided interface types, which simplifies much of the specification.

This revision also contains paragraphs in a smaller font, like this one, that briefly discuss the rationale for notable design decisions.

Closures for the Java Programming Language (v0.3)

Gilad Bracha, Neal Gafter, James Gosling, Peter von der Ahé

Modern programming languages provide a mixture of primitives for composing programs. Most notably, Ruby, Scala, Smalltalk, and Simula have direct language support for delayed-execution blocks of code, called closures. Closures provide a natural way to express some kinds of abstractions that are currently quite awkward to express in Java. For programming in the small, closures allow one to abstract an algorithm over a piece of code; that is, they allow one to more easily extract the common parts of two almost-identical pieces of code. For programming in the large, closures support APIs that express an algorithm abstracted over some computational aspect of the algorithm. There are two alternative specifications: the nominal version includes closure literals but requires programmer-provided interfaces for their types while the functional version includes all of the features of the nominal version but also includes a syntax and semantics for function types, which act as special kinds of interfaces. This document contains both the nominal and the functional versions of the specification.

Closure Literals

We introduce a syntactic form for constructing a closure value:

Primary:

Closure

Closure:

{ FormalParameterDecls_opt => BlockStatements_opt Expression_opt }

A closure is converted to some object type at compile-time by a closure conversion. In the nominal version of the specification, it is a compile-time error if a closure appears in a context where it is not subject to a closure conversion. In the functional version of the specification, if a closure is not subject to a closure conversion it is converted to the corresponding function type of the closure, which is the function type with: identical argument types; a return type that is the type of the final expression, if one exists, or java.lang.Unreachable if the closure body cannot complete normally, or void otherwise; and a throws type list corresponding to the checked exception types that can be thrown from the body of the closure. The conversion, in either case, occurs entirely at compile-time.

Example: the following closure takes two integers and yields their sum: {int x, int y => x+y}

A closure captures a block of code - the block statements and the expression - parameterized by the closure's formal parameters. All free lexical bindings - that is, lexical bindings not defined within the closure - are bound at the time of evaluation of the closure expression to their meaning in the lexical context in which the closure expression appears. Free lexical bindings include references to variables from enclosing scopes, and the meaning of this, break, continue, and return. Evaluating the closure expression does not cause the statements or expression to be evaluated, but packages them up at runtime with a representation of the lexical context to be invoked later.

Rationale: One purpose for closures is to allow a programmer to refactor common code into a shared utility, with the difference between the use sites being abstracted into a closure. The code to be abstracted sometimes contains a break, continue, or return statement. This need not be an obstacle to the transformation. One implication of the specification is that a break or continue statement appearing within a closure may transfer to any matching enclosing statement. A return statement always returns from the nearest enclosing method or constructor. But a closure may outlive the target of control transfers appearing within it.

At runtime, if a break statement is executed that would transfer control out of a statement that is no longer executing, or is executing in another thread, the VM throws a new unchecked exception, UnmatchedNonlocalTransfer. Similarly, an UnmatchedNonlocalTransfer is thrown when a continue statement attempts to complete a loop iteration that is not executing in the current thread. Finally, an UnmatchedNonlocalTransfer is thrown when a return statement is executed if the method invocation to which the return statement would transfer control is not on the stack of the current thread.

Function Types

We introduce a syntactic form for a function type:

Type:

FunctionType

FunctionType:

{ Types_opt => ResultType } FunctionThrows_opt

Types:

Type
Type , Types

ResultType:

void
Type

FunctionThrows:

throws ThrowsTypeList

ThrowsTypeList:

QualifiedIdentifier
QualifiedIdentifier | ThrowsTypeList

Note: the existing syntax for the throws clause in a method declaration uses a comma to separate elements of the ThrowsTypeList. For backward compatibility we continue to allow commas to separate these elements in methods and constructors, but in function types we require the use of the '|' (vertical-bar) character as a separator to resolve a true ambiguity that would arise when a function type is used in a type list. It is sometimes useful to think of the thrown type as a disjunctive type as well, and this syntax is suggestive of that.

Informally, a function type describes the set of closures that accept a given list of argument types, result in a value of the given type, and may throw the indicated checked exception types.

Example: the following assigns to the local variable plus a function that computes the sum of its two int arguments:

{int,int=>int} plus = { int x, int y => x+y };
  

{int,String=>Number}throws IOException xyzzy;

interface Closure1<R,A2,throws E> { // system-generated
    R invoke(int x1, A2 x2) throws E;
}
Closure1<? extends Number,? super String,null> xyzzy;

Closure Conversion

A closure may be assigned to a variable or parameter of any compatible interface type by the closure conversion:

There is a closure conversion from a closure to every interface type that has a single method m such that the closure is compatible with m. A closure is compatible with a method m iff all of the following hold:

• Either
  • The closure has no final expression and the method m has return type void; or
  • The closure has a final expression and there is an assignment conversion from its type to the return type of m; or
  • The body of the closure cannot complete normally.
• m has the same number of arguments as the closure.
• For each corresponding argument position in the argument lists, both argument types are the same.
• Every exception type that can be thrown by the body of the closure is a subtype of some exception type in the throws clause of m.

If the target of the closure conversion extends the marker interface java.lang.RestrictedClosure then

• it is a compile-time error if the closure being converted contains a break, continue or return statement whose execution would result in a transfer of control outside the closure; and
• it is a compile-time error if the closure being converted refers to a non-final local variable declared in an enclosing scope.

The closure conversion to a "restricted" interface type applies only to closures that obey the same restrictions that apply to local and anonymous classes. The motivation for this is to help catch inadvertent use of non-local control flow in situations where it would be inappropriate. Examples would be when the closure is passed to another thread to run asynchronously, or stored in a data structure to be invoked at a later time when the method invocation in which the closure originated no longer exists. We expect to retrofit a number of existing JDK classes with this marker interface.

Example: We can write a closure that adds two to its argument like this:

interface IntFunction {
    int invoke(int i);
}
IntFunction plus2 = {int x => x+2};

Alternatively, using function types we have

{int=>int} plus2 = {int x => x+2}

We can use the existing Executor framework to run a closure in the background:

void sayHello(java.util.concurrent.Executor ex) {
    ex.execute({=> System.out.println("hello"); });
}

Exception type parameters

To support exception transparency, we add a new type of generic formal type argument to receive a set of thrown types. [This deserves a more formal treatment] What follows is an example of how the proposal would be used to write an exception-transparent version of a method that locks and then unlocks a java.util.concurrent.Lock, before and after a user-provided block of code. In the nominal version of the proposal:

interface Block<throws E> {
    void invoke() throws E;
}
public static <throws E extends Exception>
void withLock(Lock lock, Block<E> block) throws E {
    lock.lock();
    try {
        block.invoke();
    } finally {
        lock.unlock();
    }
}

Or using the functional version of the proposal:

public static <throws E extends Exception>
void withLock(Lock lock, {=>void} throws E block) throws E {
    lock.lock();
    try {
        block.invoke();
    } finally {
        lock.unlock();
    }
}

This can be used as follows:

withLock(lock, {=>
    System.out.println("hello");
});

This uses a newly introduced form of generic type parameter. The type parameter E is declared to be used in throws clauses. If the extends clause is omitted, a type parameter declared with the throws keyword is considered to extend Exception (instead of Object, which is the default for other type parameters). This type parameter accepts multiple exception types. For example, if you invoke it with a block that can throw IOException or NumberFormatException the type parameter E would be IOException|NumberFormatException. In those rare cases that you want to use explicit type parameters with multiple thrown types, the keyword throws is required in the invocation, like this:

Locks.<throws IOException|NumberFormatException>withLock(lock, {=>
    System.out.println("hello");
});

You can think of this kind of type parameter accepting disjunction, "or" types. That is to allow it to match the exception signature of a closure that throws any number of different checked exceptions. If the block throws no exceptions, the type parameter would be the type null (see below).

Type parameters declared this way can be used only in a throws clause or as a type argument to a generic method, interface, or class whose type argument was declared this way too.

We introduce a meaning for the keyword null as a type name; it designates the type of the expression null. A class literal for the type of null is null.class. These are necessary to allow reflection, type inference, and closure literals to work for functions that result in the value null. We also add the non-instantiable class java.lang.Null as a placeholder, and its static member field TYPE as a synonym for null.class.

Definite assignment

The body of a closure may not assign to a final variable declared outside the closure.

A closure expression does not affect the DA/DU status of any free variables it names.

Free variables referenced inside a closure receive their initial DA value from the DA status of the variable at the point where the closure expression appears.

The type Unreachable

We add the non-instantiable type java.lang.Unreachable, corresponding to the standard type-theoretic bottom. Values of this type appear only at points in the program that are formally unreachable. This is necessary to allow transparency for closures that do not return normally. Unreachable is a subtype of every type (even primitive types). No other type is a subtype of Unreachable. It is a compile-time error to convert null to Unreachable. It is an error to cast to the type Unreachable.

Example: The following illustrates a closure being assigned to a variable of the correct type.

interface NullaryFunction<T, throws E> {
    T invoke() throws E;
}
NullaryFunction<Unreachable,null> thrower = {=> throw new AssertionError(); };

Reachable statements

An expression statement in which the expression is of type Unreachable cannot complete normally.

The initial statement of a closure is reachable.

Control invocation syntax

A new invocation statement syntax is added to make closures convenient for control abstraction:

ControlInvocationStatement

Primary ( Formals : ExpressionList_opt ) Statement
Primary ( ExpressionList_opt ) Statement

This syntax is a shorthand for the following statement:

Primary ( ExpressionList, { Formals => Statement } );

This syntax makes some kinds of closure-receiving APIs more convenient to use to compose statements.

Note: There is some question of the correct order in the rewriting. On the one hand, the closure seems most natural in the last position when not using the abbreviated syntax. On the other hand, that doesn't work well with varargs methods. Which is best remains an open issue.

We could use the shorthand to write our previous example this way

withLock(lock) {
    System.out.println("hello");
}

Rationale: This is not an expression form for a very good reason: it looks like a statement, and we expect it to be used most commonly as a statement for the purpose of writing APIs that abstract patterns of control. If it were an expression form, an invocation like this would require a trailing semicolon after the close curly brace of a controlled block. Forgetting the semicolon would probably be a common source of error. The syntax is convenient for both synchronous (e.g. see withLock) and asynchronous (e.g. see Executor.execute) use cases.

Another example of its use would be a an API that closes a java.io.Closeable after a user-supplied block of code:

class OneArgBlock<T, throws E> {
    void invoke(T t) throws E;
}
<T extends java.io.Closeable, throws E>
void closeAtEnd(OneArgBlock<? super T,E> block, T t) throws E {
    try {
        block.invoke();
    } finally {
        try { t.close(); } catch (IOException ex) {}
    }
} 

Or using the functional version:

<T extends java.io.Closeable, throws E>
void closeAtEnd({T=>void}throws E block, T t) throws E {
    try {
        block.invoke();
    } finally {
        try { t.close(); } catch (IOException ex) {}
    }
} 

We could use the shorthand with this API to close a number of streams at the end of a block of code:

closeAtEnd(FileReader in : makeReader()) closeAtEnd(FileWriter out : makeWriter()) {
    // code using in and out
}

Type inference

The rules for type inference need to be augmented to accomodate the inference of exception type parameters. Similarly, the new subtype relationships introduced by function types need to be reflected as well.

Future Directions

Specify the unmatched nonlocal transfer exception in more detail, including support for resuming them across threads.

Elided exception type parameters enable existing interfaces to be retrofitted for exception transparency.

Allow volatile on locals.

Abstractions for loops that work with break, continue?

Acknowledgments

The authors would like to thank the following people whose discussions and insight have helped us craft, refine, and improve this proposal:

Lars Bak, Cedric Beust, Joshua Bloch, Martin Buchholz, Danny Coward, Erik Ernst, Christian Plesner Hansen, Kohsuke Kawaguchi, Doug Lea,"crazy" Bob Lee, Martin Odersky, Tim Peierls, John Rose, Ken Russell, Mads Torgersen, Jan Vitek, and Dave Yost.