Java Closures Prototype Feature-Complete
I'm pleased to announce that the Java Closures prototype now supports all of the features of its specification!
The complete source code, released under GPLv2, is in the project's openjdk repository. A binary build, suitable for use with an existing JDK6, is at http://www.javac.info/closures.tar.gz. Other related documents are on the website http://www.javac.info/
Although there is room for performance tuning, the prototype supports the full Closures (v0.5) specification. Based on your feedback, there are some changes in the prototype suitable for a future update of the specification:
- Renamed
UnreachabletoNothing - Removed support for the type
null - Overhauled restricted versus unrestricted
- Refined restrictions
- The variable is not the target of any assignment, or
- The variable is annotated @Shared
- Relaxed the closure conversion
for-qualified method declarations- Added support for method references
- Implemented a classfile format for the
forqualifier
We adopt the name used by Scala to represent the same concept.
We usednullas a placeholder for an exception type when none can be thrown. The typeNothingnow serves that purpose;nullis no longer supported as the name of a type.
In the specification, an interface is considered restricted if it extends a marker interface. Unfortunately, the specification only provides a syntax for function type interfaces that are unrestricted. We modified the syntax so that a function type written using the=>token designates a restricted function type, while one written using the newly introduced==>token represents an unrestricted function type. This allows programmers to easily write APIs that restrict (or don't restrict) the operations of closure expressions passed as parameters.
We modified the distinction between restricted and unrestricted closures. As before, it is not legal to convert an unrestricted closure to a restricted interface type, nor is it legal tobreak,continue, orreturnfrom inside a restricted closure to a target outside the closure. However, a restricted closure is allowed to refer to a non-final local variable from an enclosing scope. In this case a warning is given unless one of the following conditions holds:
It is possible to suppress the warning by annotating some enclosing construct
@SuppressWarnings("shared").
In response to user feedback, we've relaxed the relationship between a closure parameter's type and the target interface's parameter type. Rather than requiring them to be of the same type, they are now allowed to be related by an assignment conversion, including boxing or unboxing.
The for keyword on a method declaration, meant to
introduce a control abstraction method that works like a loop, is now
treated syntactically like a modifier rather than appearing
immediately before the method name. This helps make the declaration
site more similar to the use site.
We added extensive support for treating a reference to a method as a closure using a newly introduced token
#. The syntax is borrowed from the FCM proposal. The semantics are as follows:A method reference written as
Primary # Identifier ( TypeList )where the Primary designates an expression (as opposed to a type) is treated the same as a closure
or{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...) }{ Type x0, Type x1 ... => tmp.Identifier(x0, x1 ...); }Where tmp is a temporary value that holds the computed value of the primary expression. The former translation is used when the resolved method has a non-void return type, while the latter is used when the resolved method has a void return type.
If the primary resolves to a type, then this is translated to
or{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...) }{ Type x0, Type x1 ... => Primary.Identifier(x0, x1 ...); }when the resolved method is static, or
or{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...) }{ Primary x, Type x0, Type x1 ... => x.Identifier(x0, x1 ...); }when the resolved method is an instance method.
In addition, optional explicit type arguments, between angle brackets, may be placed immediately after the
#token. These are used directly in the translated method invocation to resolve the method to be invoked.
We've impleemnted a class file representation of the for
qualifier to support separate compilation.
42 comments:
does it mean that closure will make it in jdk 7? god i hope so!
Cool, thanks for all the hard work!
Hopefully it'll make it into the JLS.
With kind regards
Ben
Thanks for the update. Almost 5 months since the last post on closures. For many people your blog is the only way to know whether BGGA closures are still alive and moving forward. I suppose you have little time for blogging, but your posts are very much appreciated. I enjoy reading them.
Congratulations on the milestone, Neal. As the other posters, I too am hopeful that Java 7 will feature closures.
Congrats, Neal. This is a huge milestone!
FTW!
There are a few features of the method reference syntax in FCM that weren't mentioned in your post. Are these included in the prototype?
1) Can the # token can be used to get the Method object for a particular method?
Method toString = Object#toString();
2) Can constructors be referenced using the # syntax?
{=>JButton} buttonFactory = JButton#();
3) Are both bound and unbound method references supported?
// Unbound method reference
List list = ...
Object o = ...
{List,Object=>boolean} listContains = List#contains(Object);
// Here the usage is similar to Method.invoke()
boolean contained = listContains.invoke(list, o);
// Bound method reference
List list = ...
Object o = ...
{Object=>boolean} listContains = list#contains(Object);
boolean contained = listContains.invoke(o);
Thanks
@Matt: (1) The closures prototype does not add any language features in support of reflection. (2) There is no special support for constructors. (3) The behavior of method references when the qualifying primary is a type versus expression is described explicitly in the blog post.
Well done Neal!
Now that it is possible to return from inside an unrestricted closure to a target outside the closure, I can code my database example in an even more elegant way ... and elegance is what programming is all about (not power of expression: all programming languages are Turing complete). Your closures proposal surely contribute to the elegance of Java.
I'm going to update the database example of my blog asap.
I sincerely hope that the Java standardization people will open their eyes and include your proposal.
Luc
@Neal:
(2) Is it too late to add this?
A constructor reference could be written as:
Primary # ( TypeList )
or maybe:
Primary # new ( TypeList )
and would be treated the same as the closure:
{ Type x0, Type x1 ... => new Primary(x0, x1 ...) }
(3) After careful scrutiny I see the answer to my question is yes.
Neal,
who determine whether Closure will be included in Java7 ? Can we vote to include it ?
I understand Closure is a feature originally intended for Java but due to lack of time, it is dropped
http://blogs.sun.com/jag/entry/closures
So hope that you can bring back what Java should have long ago.
@Matt: If your question is whether or not it's too late to add something to the Java language, I've been wondering that myself. The political landscape is a minefield.
If you mean the prototype, I don't think I'll be doing that. I don't mean to suggest it's a bad idea, but it is time for me to wrap up the work and leave the remainder to a JSR expert group, if the community decides to form one.
@GeekyCoder: If Sun Microsystems decides to resurrect the Java SE line of products after the current two-year hiatus and plan a next version, Danny Coward at Sun Microsystems is the presumed spec lead for the next version.
Thank you so much for all the work. I really hope this makes it into JDK 7.0 and politics don't make everyone live without closures for another five years.
Neal,
Great Job. I too think that now is the time to add closures - I hope that we won't have to wait until java 8.
Awesome work......
I really like that it works on JDK 1.6 now....
If they don't make it into 1.7 it would be nice to keep supporting them as an extension until 1.8 or whenever an official closures support is landed.
I'm done living without closures......
Kevin
It's looks so inconsistent with other java syntax.
Brilliant idea, disgusting realization.
I'm sad.
Thank you so much for all the hard work. This is a big contribution regardless of what the final outcome is.
Hi
I have been trying to implement a method that takes a closure as the argument and runs it synchronously in a DIFFERENT thread. It should also provide completion transparency and throw the same exceptions that the closure throws.
However, I have not beeen able to do it. The method signature should be like: 'public <T, throws E> T runSynchronously({=>T throws E} c) throws E
Could you show me an example of how to implement this?
Thank you so much!
Xu ShiChang
xushichangdesmond@gmail.com
@許: I've sent you the code privately. I plan to blog (or blog a pointer to) an implementation of a concurrent loop shortly.
I'll post the question that bothers me :). I just cant find any material to clear this thing to me. Why we need to retrofit some (all?) of JDK's interfaces with extends RestrictedFunction when the use cases are just opposite. So in detail. All existing regular SAM interfaces were designed with semantic in mind that share in common with restricted closures semantics in parts. In particular APIs with SAM interfaces as parameters does not expects those implementation to do thing that allowed to unrestricted closures such as throwing transfer Throwables for 'return', 'break' and 'continue' statemts. So really natural thing would be to treat all regular SAM interfaces as RestrictedFunctions (convertible from restricted closures only) by default, and add special marker interface for the opposite purpose: define interface as convertible from unrestricted closures. Just like: 'interace IterationHandler extends UnrestrictedFunction {...}'. Or Better: '@UnrestrictedFunction interface IterationHandler {...}'. Such solution does not require any change to existing interfaces, only those new that will want to allow use of unrestricted semantic and will prevent unexpected unrestricted closure conversion to interfaces that was not retrofitted to extend RestrictedFunction since their publishing prior to closure support in language. So Why SAM interfaces does not defaults to RestrictedFunction behavior?
@eugenelucash: The problem is that subtyping, and marker interfaces, just don't work that way. If the base class is restricted, and you extend it while mixing in an unrestricted marker interface, then any instance of the derived interface would be unrestricted, and yet would be a subtype of the base (restricted) interface. This is the kind of loophole in the restrictions that one can drive a truck through. If we want the restrictions to be consistency enforced by a compiler, then the unrestricted interface types must be supertypes of the restricted ones.
I would like a syntax C/C++ centric:
int (int,int) f = {int (int x,int y) { return x+y; }};
instead of:
{int,int =>int} f = {int x,int y => x + y};
Bjarne Stroustrup would be happier :)
I don't like either very much the type inference of the inline code.
But I know this has been discussed to death :)
Anyway I can believe with this new reified version of C/C++ pointer to functions in Java.
Good work.
I have a use case here that I do not seem to be able to directly implement using a closure:
I have a 3rd party method called M that I wish to call and I have to pass in an argument that implements the ILoveYou interface. The ILoveYou interface contains only a single method: 'boolean isItLove() throws ThirdPartyException'. The issue here is that I wanted to call the method M by passing it a closure, and my implementation of the isItLove() method will unconditionally throw the ThirdPartyException. However I cannot seem to do this directly because the compiler will think that my closure is a {=> void throws ThirdPartyException} while it expects a {=> boolean throws ThirdPartyException}.
So I end up having to either fall back to inner class, or write another method that returns boolean and throws the exception and call that method in my closure.
This is how I thought I would do it at first:
M({=> throw new ThirdPartyException("I am already in love with Neil Gafter");});
But this refuses to compile, and I had to do it this way:
M({=> X()});
....
private boolean X() throws ThirdPartyException {
throw new ThirdPartyException("I am already in love with Neil Gafter");
}
Great job! I'm really hoping this makes it into Java 7. It's really time that Java is enhanced as a language (considering how easy and readable some things can be done in all those new Java Platform languages, not to mention C#). All those frameworks and metaframeworks don't help, we need a language that offers more "out of the box".
So I hope politics won't stop this to be implemented as soon as possible (Java 7). For Java 8 there is still enough to do (multiline strings anyone? - string concatenation in Java is needed for SQL, Regexp and XML, so for all those things one is currently doing with Java, and it's a pain to write and a pain to read).
to: 許
Your closure: {=> throw new ThirdPartyException("...");}
is actualy of type: {=> Nothing throws ThirdPartyException}
which can be converted to: {=> boolean throws ThirdPartyException}
or your "ILoveYou" interface. I tried this with the latest prototype and it works.
Yuck.
I think the type syntax should be closer to other Java syntax. Maybe use the one from the other proposal or the one suggested by jmarranz:
int (int,int) f = {int x, int y => x + y}
The for syntax seems hackish, as the parameter order in the definition is inconsistant with the use. I would just leave the parameters inside the brackets, like other closures:
for each(map) { String key, Object value ==>
System.out.println(key + " = " + value);
}
@Denis et al: This looks a lot like the syntax in http://www.javac.info/closures-v01.html which we found hard to read in practice, especially in the presence of a throws clause.
The closures-v01 proposal lacks brackets, that is why it is hard to read. This is the FCM syntax with exceptions:
#(int(String) throws IOException)
Great work, Neil!
Hope that your closures make it into java 1.7. Lemme know if you need me to beat anyone up who's giving you flak.
Andrew Binstock claims closures won't make it into Java 7:
The one really handy feature we will not get in Java 7 is closures. Because of their great utility, closures would have been an important addition. And they would have addressed one of the key objections to Java, namely its verbosity. But they will have to await a future release. (If you need closures in the meantime, consider another JVM language, such as Groovy.)
What's his source of knowledge? Is he right?
I am a bit confused on the need for the ==> syntax. For example I would assume the two calls, below ( from your example in the talk ) equivalent
Yet this will only compile using ==> syntax. Yet the runtime behaviour for the former call(non control abstraction) is identical and compliles and works with => syntax.
Can you clarify?
Thanks
Liam
withLock(new Object(), { => System.out.println("doSomething"); } );
Object o = new Object();
withLock(o) {
System.out.println("doSomething");
} syntax.
@Liam: the two different syntaxes serve a couple of purposes, but they are mainly to help reduce accidental errors rather than enabling some functionality. In the specific case you're referring to, the idea of the distinction is to reduce the likelihood that a method is used as control abstraction unless specifically intended by the author to be used that way.
Why unrestricted closures are always void?
This makes impossible to implement functional-style control abstractions like generators:
Iterable[Integer] squares = yield(int i : someIterable()) { i * i }
int[] sqa = for collect(int i : array) { i * i }
It looks very inconsistent to be able to use control abstraction with "void" code and to be unable to use it with "code with result". It places major limitation to usage of control abstractions. Is this really necessary?
@Free Programmer: unrestricted closures may have any return type.
I think that closures in Java would be a fantastic addition to the language but I have a few suggestions that I believe would greatly enhance their usability.
1) Is there any way that we could have type inference on the argument types and return type of a closure like the way in which C# works with its "lambdas"?
If the compiler can infer the types then a closure could just be written along the lines of:
{ x, y => ... }
Where the type cannot be inferred correctly then a compiler error would alert the programmer who would then have to put in the appropriate types. In many (most?) situations the compiler could infer the types and thus allow the shortened version to be used.
2) If there are no arguments to the closure can we dispense with the '=>' at the start?
3) If a closure is the only argument to a function can we dispense with the parentheses? E.g.
list.each({ String s => ... })
could be written:
list.each{ String s => ... }
which is much easier on the eye.
Maybe this already exists in your proposal? Or do you have to do this?:
list.each(){ String s => ... }
4) We really need to extend the collections API to add methods for:
- applying a closure to every element and collecting the results
- running a closure over every element where there is no result
- being able to sort based on the result of a closure
- and many more
For example:
// Output lines sorted by word count
list.collect{ s => s.replaceAll("_", " ") }
.sort{ s => s.split("\\s").length }
.unique()
.each{ s => System.out.println(s); }
I don't know why there is a reluctance to add stuff to the APIs. How does adding new methods break backwards compatibility?
@James Crawford: these are all excellent ideas, which any JSR devoted to this subject should investigate.
One can't add methods to an existing interface because old implementations of the interface would no longer compile (they don't provide an implementation of the new method). Language features like mixins or extension methods allow one to accomplish similar things without breaking compatibility.
@James, regarding point 3:
I think the syntax should be:
myList.each(String s) {
//do something with each s
}
This syntax is more consistent with the already proposed similar syntax:
ListUtils.forEach(String s : myList) {
//do something with each s
}
Neal,
You are of course completely correct about the problems with extending the existing APIs. I hadn't considered that there might be other code apart from the libraries themselves that implemented these interfaces.
We need some way to extends existing APIs without breaking old code. Groovy has a way of allowing static methods to be invoked as though they were instance methods. Something like that might provide us a way forward.
Cheers,
James.
@James Crawford: Yes, extension methods are what you're referring to, and my proposal to add something like that to Java has not received much popular support. See, for example, http://www.javapolis.com/confluence/display/JP07/Whiteboard+results+-+Language+change
The current apparent design process for the Java language (popular voting on a feature-by-feature basis) is broken. That is among the worst ways to plan the evolution of a programming language. Something like C#'s LINQ, widely regarded by its users an a success, could never happen in this environment. Extension methods are one of the handful of language features that make LINQ work.
Neal,
I couldn't agree more. Language design by committee is bad enough but when the committee is effectively the entire user base then I don't think there is much hope for any significant change to occur.
I was recently staggered to see how far C# has come over the last few years and it pains me to think that at this rate Java will never be able to catch up.
I guess I will just have to do more and more stuff in Groovy from now on...
James.
Post a Comment