Type inferencing with Lambdas and chained Comparators

[Simon Roberts]

Hi all, I'm pondering the example below. It seems that when I chain Comparators using the 'thenComparing" type constructs, the type inferencing of the compiler dies. The example works perfectly, but notice that I have to cast all those 's' lambda parameters to Student. I'm hoping to understand: a) did I screw something up, or b) why doesn't the thenComparing thing propagate the type info in a way that the compiler knows what it's doing?

import java.util.Arrays;
import java.util.Comparator;

public class Test {
    static class Student {
        String firstName;
        String lastName;
        int age;

        public Student(String firstName, String lastName, int age) {
            this.firstName = firstName; this.lastName = lastName; this.age = age;
        }
        @Override public String toString() {
            return "Student{" + "firstName=" + firstName + ", lastName=" + lastName + ", age=" + age + '}';
        }
    }

    public static void main(String[] args) {
        Arrays.asList(
            new Student("Alan", "Jones", 14), new Student("Alan", "Ancil", 14), new Student("Alan", "Jones", 14),
            new Student("Toby", "Ancil", 14), new Student("Fred", "Jones", 14), new Student("Alan", "Ancil", 15),
            new Student("Alan", "Jones", 14), new Student("Alan", "Ancil", 16), new Student("Alan", "Jones", 14),
            new Student("alan", "Jones", 14), new Student("Bert", "Jones", 14)
        ).stream()
        .sorted(Comparator
            .comparingInt(s -> ((Student) s).age)
            .thenComparing(s -> ((Student) s).lastName)
            .thenComparing(
                Comparator.comparing(s -> ((Student) s).firstName, 
                (s1, s2) -> s1.compareToIgnoreCase(s2)).reversed()
            )
        ).forEach(System.out::println);
    }
}

[Jeanne Boyarsky]

I think that because if the first one is a cast, Java is using Object as the generic type:

If I add getters and use a method reference, thenComparing doesn't need the cast:

Comparator.comparingInt(Student::getAge).thenComparing(s -> s.lastName)

Of course with that approach, you could use a method reference for last name too making this a moot point.

[Simon Roberts]

Interesting. It's not the cast, that's there to avoid the problem, can't remove any of them in the things that I've tried, if there's a chain.

However, you're right that using method references does fix it.

For me, I can't use an accessor method in a lambda, even when the equivalent method reference works.

So, I guess the question becomes, why does it break with fields or regular lambdas; the type of the lambda should be the same as the return type of the method reference, shouldn't it? And/or the question is how come it works with references, but not the other way round.

Curious, thanks for the input Jeanne!

[Mike. J. Thompson]

If you look at the signature of the comparingInt() method it is:

public static <T> Comparator<T> comparingInt(ToIntFunction<? super T> keyExtractor)

So the only information the compiler gets from the type signature of the method is that keyExtractor will be a ToIntFunction<Object>, because it would be valid to pass a ToIntFunction<Object>.

And when you look at the lambda you were passing in for the implementation of keyExtractor (minus the casts):

s -> s.age

There is no type information in that expression at all. So when the compiler puts the two things together it sees that the method you're calling could accept a ToIntFunction<Object>, and the actual implementation you have passed it gives no hint to what the type actually is, so it has to assume that s is an Object. Object does not have an age field, so it can't let you use that lambda.

Jeanne's method references work because the method reference has a definite type that the compiler can use to infer the type parameters, so it knows what type to use when implementing the ToIntFunction.

Exactly the same thing applies to the comparing methods you call, although they have extra type parameters.

So, to call your code without the casts you need to be explicit about which types you're using in your lambda. You do that with generic methods in exactly the same way you do with generic classes, by supplying specific types.

So your code becomes:

public static void main(final String[] args)
    {
        Arrays.asList(new Student("Alan", "Jones", 14),
                new Student("Alan", "Ancil", 14),
                new Student("Alan", "Jones", 14),
                new Student("Toby", "Ancil", 14),
                new Student("Fred", "Jones", 14),
                new Student("Alan", "Ancil", 15),
                new Student("Alan", "Jones", 14),
                new Student("Alan", "Ancil", 16),
                new Student("Alan", "Jones", 14),
                new Student("alan", "Jones", 14),
                new Student("Bert", "Jones", 14))
                .stream()
                .sorted(Comparator
                        .<Student> comparingInt(s -> s.age)
                        .thenComparing(s -> s.lastName)
                        .thenComparing(
                                Comparator.<Student, String> comparing(
                                        s -> s.firstName,
                                        (s1, s2) -> s1.compareToIgnoreCase(s2))
                                        .reversed()))
                .forEach(System.out::println);
    }

[Rob Spoor]

Simon Roberts wrote:

            .comparingInt(s -> ((Student) s).age)

Don't do it like that. Even if type inference would work 100%, the compiler would either see that as an IntFunction<Object>, or as an IntFunction<Student> with an unnecessary cast.

Instead, if you ever need to make a lambda type safe, specify the argument type as follows (note that you need to put parentheses around Student s):

.comparingInt((Student s) -> s.age)

That makes it an IntFunction<Student> without any ambiguity.

[Rob Spoor]

Mike. J. Thompson wrote:

                                Comparator.<Student, String> comparing(

Don't you mean Comparator.<String>, without Student?

Nope, I forgot that Comparator.comparing takes two generic types.

Note that you can still use String.CASE_INSENSITIVE instead as the case insensitive String comparator.

[Simon Roberts]

Ah, such good input, thanks all.

Rob, your observation is most embarrassing. I tried it like that first (I'm fully aware that that's the "normal" way to help the compiler out). However, for some reason--probably some stupid typo--it didn't "take" and I started thrashing around. Somehow the bizarre botch job I ended up with both compiled and ran, which is actually pretty frightening in itself!

I'm still a little puzzled why the method references worked, since they would have carried more _output_ type information, but not more input type information, wouldn't they?

[Rob Spoor]

A non-static-method reference has input type information based on the class the method is in. Any method xXX of class Student will have a first generic type of Student when used as method reference Student:xXX. For example, Student::getAge is an IntFunction<Student> (or a Function<Student, Integer>), and Student::getFirstName is a Function<Student, String>.

A static-method reference has input type information based on its actual arguments.

[Paul Clapham]

You might want to look at issue # 233 of Heinz Kabutz's newsletter, in particular "Heinz's Lambda Reduction Principle" which looks to me like it has some relevance to this question. And if you aren't following Heinz's newsletter maybe you should be.

[Simon Roberts]

OK, I'll check out the newsletter, in the meantime, I've clarified my thoughts question rather more if anyone's interested (none of the answers so far actually address the issue below, they're just about syntax, which while useful isn't the root of what's bothering me).

This works:

.sorted(Comparator.comparingInt(s -> s.age))    // WORKS 

This doesn't:

            Comparator.comparingInt(s -> s.age)    // BROKEN, BOTH LINES!
            .thenComparing(s->s.lastName)

And, in this second case, the first lambda fails (they both do, but it actually breaks the first, previously working expression), where it used to work, and, even NetBeans indicates that it's lost the thread, notice that it now claims it's looking for a Comparator<? super Object> rather than Comparator<? super Student> when using autocomplete

By adding anything that's more explicitly typed as the first lambda

            Comparator.comparingInt((Student s) -> s.age)    // WORKS 
            .thenComparing(s -> s.lastName)    // WORKS 

Or, just as effectively:

            Comparator.comparingInt(Student::getAge)    // WORKS 
            .thenComparing(s -> s.lastName)    // WORKS 

Or, again:

            Comparator.<Student>comparingInt(s -> s.age)    // WORKS 
            .thenComparing(s -> s.lastName)    // WORKS 

And, it seems that in these situations, we've now successfully and permanently "typed" the Comparator instance, because I can no chain the raw forms. That is, this works too:

            Comparator.comparingInt((Student s) -> s.age)    // WORKS 
            .thenComparing(s -> s.lastName)    // WORKS 
            .thenComparing(s -> s.firstName)    // WORKS 

Given Stream<Student>, sort takes a Comparator<? super Student> and that seems to persuade it that the argument is a Comparator<Student> in the very first lambda, but then it loses it and reverts to Comparator<Object>. When we kick it externally into believing that it's a Comparator<Student> the first time, it's willing to propagate that information, but despite being willing to make the guess the first time, it's not willing to propagate the information??

So, I'm trying to work out what's afoot here, I guess my question is this: Did the inferencing logic suddenly change when I added the andThen expression in the first failed example? Why was it willing to infer Comparator<Student> in the first case, but not the second? Can anyone trace the inferencer's logic for me?

Or, is it just that the complexity of the expression grew to a point that the inferencer isn't capable of handling it in practical terms (this would make me happy, I'd stop feeling stupid, and practical complexity limits are fine with me in real software!)

Anyway, I'll go look at that newsletter, but if anyone has easy answers to that, now that the question's become more specific, I'm listening

Thanks again all.

[Jeanne Boyarsky]

Simon Roberts wrote:I'm still a little puzzled why the method references worked, since they would have carried more _output_ type information, but not more input type information, wouldn't they?

I don't follow. Student::getLastName says to take a Student object and return a String. Information on both the input and output side.

[Winston Gutkowski]

Paul Clapham wrote:You might want to look at issue # 233 of Heinz Kabutz's newsletter, in particular "Heinz's Lambda Reduction Principle" which looks to me like it has some relevance to this question. And if you aren't following Heinz's newsletter maybe you should be.

And so should I. Bookmarked, and have a cow. Thanks.

Winston

[Simon Roberts]

Jeanne Boyarsky wrote:I don't follow. Student::getLastName says to take a Student object and return a String. Information on both the input and output side.

Well, hmm, yes, it does. But it feels like it says how to accept it more than what it is, gach, clearly I'm only very slowly understanding what it is I'm not happy with. It's not, I think, that these approaches that work do anything unexpected, I think it's actually "how the heck does it ever work in the version without the input type hint". That is, why does this:

Comparator.comparingInt(s -> s.age)

work, and why does it not insist that <? super Student> cannot be assumed to be any more specific than Object. And then, back to square one, since it works once, why doesn't it work again?

OK. I'm going to go and read the newsletter thing and see what light that might cast. I fear I'm just rambling (and almost certainly in circles!)

[Simon Roberts]

Ah well, the newsletter is great, though I don't believe it's relevant to my conceptual difficulty here. I'll go do some general reading, perhaps in JLS and see if I can't get my head on straight.

Thanks again all for your input, it's all been useful.