Synchronous Messages

December 1, 2011 is the first JDK8 Warnings Cleanup Day (WCD).

This is an OpenJDK community effort to stamp out warnings in the JDK build. You mean there are warnings when you build the JDK? There certainly are, but they’re fairly well hidden in the log files. In many cases you need to add build options to javac such as -Xlint:all to see them. If you do this, you’ll see that there are over 10,000 warnings emitted by javac during the course of the build. How did they all get there? There are many reasons, but there are two main ones.

First, javac has historically not issued very many warnings. A certain number of warnings were mandated by the Java Language Specification, but beyond that there were few. In JDK 5, javac was enhanced to provide additional warnings for “about legal, but suspect and often problematic, program constructs.” [link]

In addition, the Java Programming Language has evolved, rendering older code obsolescent. This also occurred in JDK 5 with the addition of generics. Older code that hadn’t been updated to use generics suddenly compiled with many “raw type” and “unchecked” warning messages. There’s still a lot of this old code in the JDK. In fact, these two warning messages account for the majority of javac warning messages that occur in builds of the JDK.

In the Summer of 2011, Sasha Boulgakov, a summer intern, did some analysis about build warnings and how to reduce them. He discovered that there were over 13,500 warnings issued during the build of the jdk repository alone, that is, not including corba, hotspot, jaxp, jaxws, and langtools repositories. (A notable point is that the langtools team has been quite diligent about warnings, having kept their build at zero warnings for quite some time. Kudos to the langtools team and especially Jon Gibbons for their efforts.) Although Sasha had only a short summer with us, he managed to knock off around 2,800 warnings from the jdk repository’s build. Further work by Oracle engineer Kurchi Hazra has brought this down further to around 10,000 which is where we stand today.

Why are warnings important? Most of them are probably noise and don’t represent real problems. Since the compiler is doing us the favor of pointing out “suspect and often problematic” code, a certain portion of these warnings indicate actual bugs. However, this message is drowned out in all the noise. If we could get rid of all the noise, we’d be able to see the real bugs more clearly.

That brings us to Warnings Cleanup Day.

This is a collaborative effort among the OpenJDK community members (inside and outside of Oracle) to clean up the build warnings. For this first event, we’re focusing on warnings from javac emitted during the build of the jdk repository. (There are also warnings emitted during native code compilation, but that’s a totally different subject.) We don’t have any specific numeric goals, but we’re going to try to knock off as many as we can. You can see the initial announcement here. The main WCD page is here, and the division of labor and status updates will be posted here and updated during the day. Email discussion will occur on [email protected] and some of us will be hanging out on the #openjdk IRC as well.

Let’s clean up those warnings!

I’m featured in this week’s episode (#54) of the Java Spotlight podcast. I spend some time talking with Roger Brinkley about a few features of Java 7′s Project Coin and my “coinification” effort, that is, applying Coin features to the JDK 7 code base.

It was really difficult talking about some of the more obscure Coin features, particularly the “more precise rethrow” feature, without having the code in front of me. It’s probably even more difficult to listen to me talk about it. If you want to look at some code, take a look at my JavaOne slide deck. Compare slides 15 and 17 to see the difference that more precise rethrow makes between Java 6 and 7.

For much of the podcast I talk about the URLJarFile scenario, which is the same one that I covered in my JavaOne presentation. In that presentation I edited the code on stage in NetBeans; unfortunately the video still isn’t available. You can see the edits incrementally in the backup slides of my slide deck, starting at slide 30.

Enjoy!

JavaOne 2011 was last week in San Francisco. I gave an updated version of my “Coin In Action” talk. It seemed to be pretty well received; thanks to all who attended. Slide sets have started appearing on the JavaOne 2011 content catalog but for some reason the slides from my presentation haven’t been posted yet. I’m posting them here for download. Note that a significant portion of the talk was a demo in NetBeans. Some slides that follow the general outline of the demo are at the end of the slide deck in the section entitled “Backup Slides.”

I gave the talk in the big keynote room, which had the full video setup. A bunch of the JavaOne talks were recorded and have started to appear at parleys.com. My talk hasn’t been posted yet, though. I think the video editing might take a while. From what I understand, a few additional talks per week will be posted to parleys.com, so it might take a while to post mine (if it gets posted at all). I’ll post again here when the video for my talk is posted. Or, let me know if you see it before I do!

I’ve also posted some photos from JavaOne on my Flickr stream.

Here’s a followup comment on some issues raised with the code I showed during the talk. The question was about the “original code” I showed during the presentation:

JarFile retrieve(URL url) throws IOException {
    InputStream in = url.openStream();
    OutputStream out = null;
    File tmpFile = null;
    try {
        tmpFile = File.createTempFile("jar_cache", null);
        out = new FileOutputStream(tmpFile);
        int read = 0;
        byte[] buf = new byte[BUF_SIZE];
        while ((read = in.read(buf)) != -1) {
            out.write(buf, 0, read);
        }
        out.close();
        out = null;
        return new JarFile(tmpFile);
    } catch (IOException e) {
        if (tmpFile != null) {
            tmpFile.delete();
        }
        throw e;
    } finally {
        if (in != null) {
            in.close();
        }
        if (out != null) {
            out.close();
        }
    }
}

During the Q&A section of the talk I had a question from the audience regarding closing the out variable and then setting it to null. I wasn’t entirely sure what the questioner was asking, but I said that setting it to null was a technique for communicating between the try-block and the finally-block. In the sample code, this doesn’t do much other than avoid calling close() a second time. This usually isn’t a problem, as closing a second time usually isn’t harmful. But in more complicated cases you might need to do cleanup differently depending upon whether or not the operation succeeded.

After the formal Q&A portion of the talk a few audience members gathered up near the stage to ask followup questions. A second questioner said that his interpretation of the first questioner’s question was, since the finally-block is always going to close out, why bother closing it at all in the try-block? In response (as I was busy packing up my stuff) I mumbled something about this being a simplified example for  talk, and the actual original code might not have been amenable to such a simplification.

Now that I have had a chance to think about it a bit further, the correct answer is that the try-block must close out before it creates the JarFile instance, in order to ensure that all bytes written to out are flushed properly to tmpFile. If out were not closed, the JarFile creation might see partially written contents in the temporary file, leading to an error. The out stream would eventually be closed by the finally-block, avoiding a leak, but not necessarily avoiding an error creating the JarFile.

Of course, the whole point of the talk is that these details are exposed to programmers in Java 6 and earlier. By using Java 7′s File.copy() utility and the try-with-resources statement, these details are abstracted away, resulting in cleaner and shorter code.

Java 7 includes a couple new features that change exception handling. Specifically, these features are try-with-resources and multi-catch with more precise rethrow. These are the first changes to the exception facility since chained exceptions were added in 1.4. Exception chaining was an important improvement, but it didn’t really change anything fundamental about the way you had to write exception handling code. Indeed, exception handling code is pretty much unchanged since the beginning of Java.

Java 7 changes that.

Consider this common exception handing task. In a two-step operation, if an exception is thrown in the second step, you need to clean up anything left from the first step.

For example, suppose you want to create and return a Foo, which in turns needs a Bar to have been created first. If creating the Foo fails for some reason, you want to make sure that the Bar is cleaned up. But if creating the Foo succeeds, you don’t clean up the Bar. You could start off with some code that looks like this:

Foo createFoo() throws FooException {
    Bar bar = new Bar();
    try {
        return new Foo(bar);
    } catch (FooException fe) {
        bar.cleanup();
        throw fe;
    }
}

(This assumes that the Foo constructor has been declared with throws FooException, a checked exception.)

The problem here is that if creation of the Foo (or something else within the try block) throws something other than FooException. The bar.cleanup() code gets skipped, and you’re screwed. In some sense what you want to do is to catch and rethrow a Throwable, but you can’t do this because you’d then be forced to change the declaration of createFoo to have throws Throwable which you really don’t want to do either.

A more effective approach is to use a finally clause for this, since it executes your code and then lets the exception propagate unchanged. Unfortunately, the finally clause also executes even if no exception is thrown, so you have to add some conditionals to make sure cleanup doesn’t occur in the successful case. This usually involves creating a local, initializing it to null, and then checking for null in the finally clause. Your code ends up looking something like this:

Foo createFoo() throws FooException {
    Foo foo = null;
    Bar bar = new Bar();
    try {
        foo = new Foo(bar);
    } finally {
        if (foo == null) bar.cleanup();
    }
    return foo;
}

The reasoning is as follows. The foo local variable is initialized to null and is assigned at the very end of the try block. If it’s null at the time the finally block executes, an exception must have been thrown from somewhere in the try block, and so the cleanup code needs to get run. This is a bit ugly, but it works, and I see it often enough to recognize it as an idiom.

Turns out that in Java 7 we don’t need to write this anymore. Instead, we can do this:

Foo createFoo() throws FooException {
    Bar bar = new Bar();
    try {
        return new Foo(bar);
    } catch (Throwable t) {
        bar.cleanup();
        throw t;
    }
}

That is, we can catch and rethrow Throwable, just like we wanted to originally, but we don’t have to change the method declaration to throws Throwable. How can this work?

This is the “more precise rethrow” portion of the multi-catch with more precise rethrow feature, part of Java 7′s Project Coin. This feature is often overlooked, since it’s combined with multi-catch. Most treatments of this feature focus on multi-catch and barely talk about “more precise rethrow.” But as we can see from the code above, this feature can be used independently of multi-catch, and quite effectively as well.

To understand what’s going on here, we need to revisit some fundamentals of checked exceptions.

All Java code must conform to the “catch or specify” requirement. Any checked exceptions that can be thrown by a piece of code must either be caught by a try-catch statement that contains the code, or the checked exceptions must be specified in the throws clause of the method that contains the code. In Java 7, fundamentally this is still true, but the change is that the compiler now does additional analysis of what exceptions can be thrown by a throw statement inside a catch clause.

Prior to Java 7, the analysis was as follows. What exceptions can the “throw t” statement throw? The t variable is declared as type Throwable. Therefore, “throw t” can throw Throwable. This must be caught or specified, which in turn requires the method declaration to be changed to throws Throwable.

In Java 7, the analysis has been made deeper. Instead of just looking at the declared type of t, the compiler looks at where t came from. First, it makes sure that t points to the same object as it did at the beginning of the catch clause. (That is, t must be “effectively final”.) If so, the compiler then looks at the checked exceptions that can be thrown from the try block. In this case, we’ve called the Foo constructor, which can throw FooException. That’s the only checked exception that can arise from the try block, therefore the only checked exception type that t could have in the catch block is FooException. (It could potentially be an unchecked exception, but we don’t care about that, since unchecked exception types don’t participate in the catch-or-specify rule.) Now we get to the “throw t” statement; since we know t is either unchecked or is a FooException, we need to make sure that the enclosing method declares “throws FooException.” Indeed it does, and we’re done.

Now, if this chain of reasoning doesn’t hold up — for example, t is assigned to something else in the catch block — we revert to the old analysis, and t is treated as its declared type, just as before. Fortunately, most exception catching code doesn’t assign to the exception variable, so the new analysis will usually apply. Note that the exception itself can be modified, for example with the addSuppressed or initCause methods, and the new analysis will still apply. You just can’t assign to t and still expect this to work.

Anyway, where are we after all that?

It used to be that “catch Throwable” was a no-no. It usually meant you were going to so something stupid like swallow the exception. You couldn’t rethrow the throwable, since you’d be forced to redeclare your method with “throws Throwable” and nobody wants to do that. Now, in Java 7, it makes perfect sense to catch Throwable, do some cleanup, and then rethrow, without garbaging up the throws clause of your method declaration.

Will this make your exception handling code better? Is this a new Java exception handling idiom?

* * *

I’ll be talking about this more at my OSCON/Java talk next week in Portland.

I’ve written an article on the new Java SE 7 try-with-resources feature for the JavaTech Journal. This is a free PDF download. (You have to register with the site in order to get the download link.) This issue has several articles about Java SE 7, which will be released in the summer.

The article covers exception handling in Java SE 6 and earlier and what made it tedious and error-prone, the new try-with-resources feature and what it does, and the multiple resources variant and how it solves exception handling issues with wrapped resources.

This article was informed by the “Coinification” work I had done in JDK 7. I had previously converted some JDK 7 code to use diamond, and I had blogged about it here and here. I had also converted some code to use try-with-resources and the more precise rethrow features of Project Coin, but I haven’t blogged about it, mainly because most of my writing effort was focused on the JTJ article. Now that it’s been published, I’ll be writing more here about these features, and I’ll cover some of the finer points that didn’t make it into the article. Meanwhile, download and read the latest JTJ issue. You can also download preview binaries, sources, and documentation for JDK7 here.

Enjoy!

In my earlier post on Diamond, I described four places where the diamond operator can be used:

1. field or local variable initializer (61%)
2. right-hand side of assignment statement (33%)
3. method argument (4%)
4. return statement (2%)

(The percentages refer to the frequency of occurrence in code found by the automated “diamond finder.”)

Now, just because you can use diamond in some code doesn’t mean that you should. So, when should you use it?

An observation before we begin. If you have some code that provides generic type arguments, and you use diamond, the compiler will almost always infer the type arguments that you would have put there anyway. Using diamond will only rarely change the semantics of your program. The decision about whether or not to use diamond is entirely driven by things like style, readability, and clarity.

Let’s step through the different use sites for diamond and see how it affects these aspects of the code.

1. Field or Local Variable Initializer

An example of this usage is as follows. Without diamond, the code looks like this:

private static Map<ClassLoader, Map<List<String>, Object>> loaderToCache
    = new WeakHashMap<ClassLoader, Map<List<String>, Object>>();

And with diamond, it looks like this:

private static Map<ClassLoader, Map<List<String>, Object>> loaderToCache
    = new WeakHashMap<>();

If you want to know what the compiler will fill into the diamond for you, it’s right there in the declaration on the left-hand side. The code without diamond is redundant, and the code with diamond is much shorter and loses no information compared to before. This is pretty clearly a win. Since this is the most common usage (over 60%) even using diamond only in this situation helps quite a bit.

2. Right-hand Side of Assignment Statement

Now consider this code:

perms = new ArrayList<>();

What type arguments are inferred here? Well, you have to hunt down the declaration of the perms variable. But, no big deal. If you want to use the perms variable, such as to call a method on it, you have to know what type it is. And if you know that it’s a List or a Collection and you want to add something to it, you have to know what the type argument is. So not having the type arguments in the constructor doesn’t really hurt anything. Plus, if you want to see the type, you can usually just ask your IDE.

(On NetBeans 7, the gesture that works for me is a bit odd. First, hold down Control and click on some whitespace in your code — not on a symbol, since that will navigate to its declaration. Then, still holding Control, hover over the constructor that uses diamond. The full declaration of the constructor being called, including type arguments, should be displayed in a tooltip. Of course, you can always ask to navigate to the declaration of the variable.)

This is a point on which reasonable people can disagree. My diamond conversion changeset for the core libraries included diamond in assignment statements. The primary reviewer was Joe Darcy, who as Coin project lead is of course quite familiar with diamond. Subsequent diamond conversion changesets were for the security libraries, and the security team was uncomfortable using diamond in assignment statements. Given that they’re the ones who have to maintain this code in the long term, I decided it was best to conform to their wishes and not do diamond conversion for assignment statements in their area.

Personally I’m fine with using diamond in assignment statements, but your opinion might differ.

3. Method Argument

It’s possible to use diamond in the argument for a method call. This is fairly rare, occurring in only 4% of the cases examined. There are some complications with type inference, so it cannot be used in certain cases. But it’s hard to tell which cases these are. Consider for example the Collections.synchronizedSet method. Its declaration is as follows:

<T> Set<T> synchronizedSet(Set<T> s)

That is, it’s a generic method that has a type parameter T; it takes an argument of type Set of T and returns a value that’s also Set of T. It’s possible, though rare, to specify the generic type argument explicitly. Here’s how it’s done:

  Set<Number> set0 = Collections.<Number>synchronizedSet(new HashSet<Number>());

It’s not necessary to specify the generic type argument explicitly in this case. The following code works just as well (and is much more common):

Set<Number> set1 = Collections.synchronizedSet(new HashSet<Number>());

Where does the generic type argument come from? In this case the compiler infers it from the type argument from the constructor, and the result of the inference is that T is Number. Now let’s try to use diamond:

Set<Number> set2 = Collections.synchronizedSet(new HashSet<>());

Unfortunately, this does not work. Why not? In the previous case, the compiler used the type argument Number from the HashSet constructor call to determine the value of the type argument T. When diamond is used, the compiler needs first to fill in the diamond before it can establish the value of T. But in order to fill in the diamond, the compiler needs to know what type is expected as the argument for synchronizedSet(), and this type includes the type parameter T. So we have a circular dependency. The compiler can’t infer the right type, so the compilation fails. The compiler doesn’t flag the circularity as an error, though. What it does is start off by attempting to use Object in the diamond and then proceeding from there. The actual error message is as follows:

error: incompatible types
Set<Number> set2 = Collections.synchronizedSet(new HashSet<>());
    required: Set<Number>
    found:    Set<Object>

This is a bit odd, but that’s where the Set<Object> comes from. But note that the following statement does work:

Set<Object> set3 = Collections.synchronizedSet(new HashSet<>());

Now this is weird. The compiler attempts to fill in the diamond with Object, and this ends up working, so the statement compiles successfully.

The compiler doesn’t always start with Object. It will use additional type information if it’s available, such as from other method arguments. Consider this example:

Set<Number> set4 = Collections.synchronizedSet(new HashSet<>(set1));

This also works. The difference here is that we’re passing set1 as an argument to the HashSet constructor. This supplies enough information to the compiler about the type that should go into the diamond, and this in turn supplies the information to establish the value of the T type parameter of the synchronizedSet() method.

What’s the point of all this? The type inference that goes on works pretty well for variable initializers and assignments, but it can break down in some obscure ways for method arguments, especially for generic methods. This can make the code obscure, too. In the set4 example above, what type is inferred for the diamond? One might think that it has to be Number, but that’s not always the case. It really depends on the type of set1, which is declared elsewhere. We know that set1 is HashSet<Number>, whose type argument Number happens to match the type argument on the left-hand side. It looks like this type is inferred from the left-hand side, but in fact it isn’t. The following example illustrates this.

  Set<? extends Number> set6 = Collections.synchronizedSet(new HashSet<>(set5));

What gets inferred for the diamond? We can’t tell, other than that it must be a Number or a subclass of Number. It really depends on the type of set5. It might be Number or it might be AtomicLong. Indeed, since I haven’t provided the declaration for set5, it goes to show that we can’t tell how the diamond gets filled in simply by looking at this statement. We have to go look elsewhere.

As you can see, there’s considerable subtlety surrounding the use of diamond in method arguments, and there are some obscure interactions with the compiler’s type inference algorithm. It might be quite difficult for a programmer to figure out what type gets filled into the diamond. In some cases diamond can’t be used, and it results in a compiler error that might be puzzling.

For these reasons we’ve concluded that it’s probably not a good style to use diamond within method arguments even when it would be legal to do so. In doing my diamond conversion over the core libraries, I made sure to avoid using diamond in method arguments. In fact I had put some diamonds in method arguments in early on, and when we decided to avoid using diamond in these cases, I went back in and pulled them out.

4. Return Statement

Finally, it’s possible to use diamond in the return statement. In this case the diamond is inferred from the return type of the method. This is similar to an assignment statement, since the type of the expression in the return statement must be assignment compatible to the return type of the method. The method’s return type might be declared fairly far away from the return statement, but this isn’t necessarily any worse than assigning to an object’s field, which might also be declared far away. For short methods, like the following,

List<Foo> makeFooList() {
    int capacity = ...; // compute initial capacity
    return new ArrayList<>(capacity);
}

it’s pretty much a no-brainer to use diamond. But, overall, the opportunity to use diamond in return statements is quite rare, so it’s probably not worth worrying too much about whether or not to use diamond for these cases.

Summary

The first two cases — field or variable initializers, and assignment statements — together comprise well over 90% of the opportunities for using diamond. I’d recommend using diamond in those cases. Even if you’re not comfortable using diamond in assignment statements, you can still get a lot of benefit using diamond just in initializers. Diamond in return statements are rare enough not to worry about very much, though it’s probably reasonable to use diamond in the obvious cases. Finally, it’s probably a good idea to stay away from diamond in method arguments. When it’s legal to use, it might be hard to see how the diamond is filled in, and when it’s not legal, it might be hard to figure out why.

My colleague Joe Darcy has recently posted the Early Draft Review for JSR 334, Project Coin. (Joe is the spec lead for this JSR.) Let’s see what the EDR has to say about the diamond operator. First, there are some changes to the grammar that allow an empty type argument list. Not too surprising. Then, it continues:

Let K be the class type being instantiated. To create an instance of K, i, a constructor of C is chosen at compile-time by the following rules, where C is chosen as follows:

• If the type denoted by K has an empty type argument list (diamond operator) and K is a classtype of the kind G<X1, X2… Xn>, then C is a type constructed from K, where all constructors c1(), c2() … cn() are obtained from corresponding constructors in G by prepending additional type-arguments T1, T2 … Tn, where the declared bounds of T1, T2 … Tn are obtained from the declared bounds of X1, X2 … Xn, where all occurrences of X1, X2… Xn are replaced by T1, T2 … Tn. Furthermore, all occurrences of types X1, X2 … Xn in the obtained constructor signatures are replaced by T1, T2 … Tn.
• otherwise C == K

Uhhm, I’m not entirely sure, but I think this means that the compiler just does the right thing.

I should mention that, just above this, the draft states:

Note to readers: the description of diamond semantics below will be reworked in future drafts of the specification.

Whew! I know that sometimes specifications are opaque — and I’ve written some pretty opaque specifications in my day — but this one really takes the cake. Fortunately, this is only a draft, so this can get fixed before the final version of the specification.

But what does this mean, really? Instead of trying to explain the draft specification, I’ll instead pursue a more intuitive approach. Basically, here is what happens, as I understand things. When you use the diamond operator, the compiler will infer from context a set of type arguments that make the expression work. This is a only little less abstract, so let’s look at some examples.

List<Number> list = new ArrayList<>();

What type argument does the compiler infer for the ArrayList? The only argument that will work is Number, so that’s what will get used. No other type will work in the diamond, not even a subclass of Number, such as Integer, since the resulting type ArrayList<Integer> would have no type relationship with List<Number>. (For further explanation, see Angelika Langer’s Generics FAQ #102, or Josh Bloch’s Effective Java, Second Edition, Item 25.)

In this case the context is pretty clear. In what other contexts can one use the diamond operator?

As I mentioned in my previous post, what I’ve done is to run a Jackpot-based diamond converter over a bunch of source files in the JDK. This applies the diamond operator everywhere it can possibly be used. It identified several hundred potential use sites for diamond. These cases broke down as follows:

1. field or local variable initializer (61%)
2. right-hand side of assignment statement (33%)
3. method argument (4%)
4. return statement (2%)

There were also a tiny number of obscure edge cases that I’ll ignore for now. Let’s look at each of the four major cases in turn.

1. Field or Local Variable Initializer

This case is by far the most common. This is what’s shown in my one-line example above. It’s also probably the most common usage cited in toy examples given in talks or articles about Coin and the diamond operator. At least, in Joe’s talks at JavaOne and Devoxx last year, the one-liner examples he showed were all field or local variable initializers. Here’s an example of an actual change I pushed recently, to src/share/classes/java/lang/reflect/Proxy.java. The original code was:

private static Map<ClassLoader, Map<List<String>, Object>> loaderToCache
    = new WeakHashMap<ClassLoader, Map<List<String>, Object>>();

and the replacement code is:

private static Map<ClassLoader, Map<List<String>, Object>> loaderToCache
    = new WeakHashMap<>();

Now even though this is a pretty complex generic type, figuring out what gets inferred inside the diamond is pretty easy. It’s right there in the declaration on the left-hand side. Having the type arguments repeated on the right-hand side is noisy and redundant.

2. Right-hand Side of Assignment Statement

Consider the following statement:

perms = new ArrayList<>();

What does this do? Well, you have to look for the declaration of the “perms” variable in order to figure it out, but that’s not too hard. In decent code it should be obvious anyway. In this case (src/share/classes/java/io/FilePermission.java) the declaration is

List<Permission> perms;

so this case is also pretty straightforward. It’s true that you can’t tell from the assignment statement by itself what the instantiated type will be. But in order to use the perms variable, you have to know what type it is anyway (or you have to ask your IDE) so this isn’t too much of a burden.

3. Method Argument

Here’s an example from src/share/classes/com/sun/java/util/jar/pack/BandStructure.java, around line 1700:

List<List<Attribute.Layout>> attrDefs = ...;
...
attrDefs.set(i, new ArrayList<>(...));

What gets filled into the diamond here? This is where it gets a bit tricky. Looking at the snippets above, it’s pretty obvious that the type argument has to be Attribute.Layout. And indeed it is. But the compiler has to go through a couple different steps to get here.

1. The “set” method of attrDefs takes parameters (int, E).
2. The attrDefs variable is declared List<E> where E is List<Attribute.Layout>.
3. ArrayList<> needs to match List<Attribute.Layout>; therefore, what goes in the diamond is Attribute.Layout.

This isn’t exactly the analysis the compiler goes through, but you can see this case has many more moving parts than the others. This is actually a pretty simple case of a method argument as well: we didn’t have to do method overload resolution. If the set() method had been overloaded, the compiler would use the types of the arguments to figure out which method to use. But with diamond we’re trying to infer the argument type based on the type the method accepts… from what I understand this isn’t actually a circularity, but it greatly complicates the compiler’s analysis.

4. Return Statement

Here’s an example from src/share/classes/java/lang/ClassLoader.java:

public Enumeration<URL> getResources(String name) throws IOException {
    ...
    return new CompoundEnumeration<>(tmp);
}

This isn’t too different from the assignment statement. The type required is determined based on the return type of the method, and in this case it’s URL.

Summary

The four cases described here account for virtually all of the cases where the diamond operator can be applied in real code. Of these four cases, one case — the initializer case — is the most common in real code and also the most common in toy examples and in slideware. This is a good validation of the designs that have been presented over the past couple years. How many times have we seen toy examples used to justify features that turn to be hardly useful in practice?

The two most common cases (initializers and assignment statements) comprise over 90% of actual uses found of the diamond operator. Furthermore, their type inference is straightforward and they’re fairly easy for programmers to understand. Other cases may involve much more complex type inference, and are potentially harder to understand (hm, maybe there’s a correlation there), but they also occur much less frequently.

Word has probably leaked around the InterTubes by this point, and it is in fact true, that I am now working on the JDK.

My new job (alluded to in this blog post from Joe Darcy) is to run through the JDK libraries and upgrade the code to use new language features of Project Coin (features listed here, JSR here). We’re concentrating on four of the Coin features, namely

• diamond operator
• strings in switch
• multi-catch
• try-with-resources

I’m working on diamond first. I’ve actually been working on this for a few weeks, but a lot of this time has been spent getting up to speed, pulling over and building OpenJDK (most of this work is being done in the open), understanding testing requirements, development procedures, and so forth. So it wasn’t until earlier this week that I pushed my first changeset into OpenJDK. This changeset was a pass over the java.io, java.lang, java.sql, and java.util (exclusive of java.util.concurrent) packages to upgrade them to use the diamond operator. This consisted of a couple hundred conversions spread over 71 files. After hitting a few other areas of the JDK with diamond, I’ll move on to other Coin features.

First, what is this “diamond” operator?

If you’ve ever written any generic Java code, you’ll understand that it can get quite verbose. The “diamond” operator is a new feature in JDK 7 that allows the programmer to omit the type arguments in the construction of a generic object. The compiler will infer the proper type argument. Actually, the diamond “operator” isn’t an operator at all, it’s an empty type argument list. But it looks like a diamond. For example, instead of

List<String> myList = new ArrayList<String>();

one can simply write

List<String> myList = new ArrayList<>();

Big deal, so the code gets a bit shorter. Who cares? Well, sometimes generic types can get pretty complicated. Consider the following:

Hashtable<String,Hashtable<String,Vector<String>>> table =
    new Hashtable<String,Hashtable<String,Vector<String>>> ();

This can be shortened to:

Hashtable<String,Hashtable<String,Vector<String>>> table = new Hashtable<>();

This isn’t some crazy hypothetical example; this is from real code! The conversion hasn’t been integrated yet, but you can see the original code in src/share/classes/sun/security/krb5/Config.java.

This raises the question of how I found sites where diamond conversion could be done. Fortunately, there’s some pretty good tooling that does the actual finding and fixing almost automatically. I’m using Jackpot, a syntax-level code pattern matcher and transformer. Somebody has already written a Jackpot-based diamond converter, so all I have to do is run it. I’ll dig up and post more information about Jackpot in the future. Basically, though, it’s the same technology that underlies some of the hinting and refactoring features in NetBeans 7.

Actually, making the code changes is the easy part. The hard part is in “all the rest of the stuff.” That is: breaking down the JDK libraries into suitable chunks, filing bugs, sending webrevs around for review, testing, pushing changes, updating bugs, and keeping track of several changesets all at different stages of the process. It’s not really that hard, but it is a fair amount of work, much more than the code changes themselves. Isn’t that always the case?

Although the diamond conversion seems simple, it belies a surprising amount of complexity. I’ll cover some of those issues in future posts. Meanwhile, enjoy the JDK7 library code as it becomes more fresh and minty!