Generics and return-type substitutability

[Sarah Rising]

Hi All,

Generics and overriding/overloading can be confusing at times. For this particular problem, I can't understand why, according to the rules of return-type substitutability, a subtype method's return type doesn't satisfy the definition. Here's the definition of return-type substitutable from the Java Language Specification (3.0), p. 254:

A method declaration d1 with return type R1 is return-type-substitutable for another method d2 with return type R2, if and only if the following conditions hold:
- If R1 is a primitive type, then R2 is identical to R1.
- If R1 is a reference type then: R1 is either a subtype of R2 or R1 can be converted to a subtype of R2 by unchecked conversion (§5.1.9), or R1 = erasure of R2. If R1 is void then R2 is void

Here's the shell of the code that motivates my question:

public class SuperClass<T> {
    ...
    public T get() { ... }
  }
} 

class SubClass<N extends Number> extends SuperClass<N> {
  public Number get() { ... }
  }
}

- SubClass is inheriting the get() method from Superclass<N extends Number>, so it should be inheriting: T get() (where T = N extends Number)
- Return type of SubClass.get() is R1 = Number.
- SuperClass.get()'s return type is R2 = T = (N extends Number) and it's erasure is Number (I think).
- This would seem to me to satisfy the condition that "R1 = the erasure of R2"
- But the compiler says that's not the case.

Would anyone know why?

(And how do you get formatting on this board other than code?)

Thanks in advance.

[Matthew Brown]

Hi. Welcome to The Ranch!

Remember that when you override a method and want to change the return type, you can only change it to something more specific. Number is not more specific than N extends Number, it's less.

For a concrete example, consider this implementation:

class SubClass<N extends Number> extends SuperClass<N> {  
    public Number get() {
        return new Double(10.0);
    }  
}  

If Number was allowed as a return type, that would be a perfectly valid implementation. But what happens when I do this?

SuperClass<Integer> obj = new SubClass<Integer>();
Integer i = obj.get();

Also, should be valid. But in that last line, obj.get() is returning a Double object. So we've lost the type safety. So the compiler can't allow it.

I haven't confirmed it, but I suspect class SubClass<N super Number> will allow Number as a return type.


Edit: See UbbCode for details about what formatting is available.

[Sarah Rising]

Thanks much for the reply, Matthew.

I understand the intention of the rules on subtype override return types. And I agree with the compiler not allowing the override in that particular example. It's what one would intuit to be the case. But it doesn't seem to follow the rules determined by the definition of return-type-substitutable. Somewhere I seem to be wrong on the following items:

- R1 = return type of get() in SubClass = Number
- R2 = return type of get() in SuperClass = T = (N extends Number)
- the erasure of R2 = Number
- Hence R1 = erasure of R2 and so R1 should be return-type-substitutable and SubClass.get() should be a valid override of SuperClass.get().

But it's not, and according to the definition of return-type-substitutable, I don't see why? Somewhere in those (4) bullet items, I'm interpreting something incorrectly, but I don't know which one it is. My guess is the 2nd or 3rd item.

Any other ideas?

Thanks, again.

[jishnu dasgupta]

Matthew Brown wrote:Hi. Welcome to The Ranch!

Remember that when you override a method and want to change the return type, you can only change it to something more specific. Number is not more specific than N extends Number, it's less.

For a concrete example, consider this implementation:

class SubClass<N extends Number> extends SuperClass<N> {  
    public Number get() {
        return new Double(10.0);
    }  
}  

If Number was allowed as a return type, that would be a perfectly valid implementation. But what happens when I do this?

SuperClass<Integer> obj = new SubClass<Integer>();
Integer i = obj.get();

Also, should be valid. But in that last line, obj.get() is returning a Double object. So we've lost the type safety. So the compiler can't allow it.

I haven't confirmed it, but I suspect class SubClass<N super Number> will allow Number as a return type.


Edit: See UbbCode for details about what formatting is available.

i have a small question. If i define as SuperClass<Integer> obj = new SubClass<Integer>();

Isnt the overriden method in Subclass breaking the rules of overridding?

However i think that < N super Number> should work fine!!! although i have not checked it!!

[Sarah Rising]

The overriding rules aren't based on a particular parameterization. They're based on the declarations of the methods.
They're a bit surprisingly simple. You can find them here: Overriding by an Instance Method. Complications come in when various compiler errors are stipulated in the ensuing text of that subsection. One of those concerns the throws clause, another concerns inheriting multiple methods with the same erasure. And another concerns return types, acceptable ones were generalized when generics was introduced. That's the definition I'm having trouble with with this (seemingly simple) little example.

[dennis deems]

SubClass extends SuperClass<N>. So get() must return N, not Number.

[Sarah Rising]

Dennis Deems wrote:SubClass extends SuperClass<N>. So get() must return N, not Number.

Thanks for the reply, Dennis. My question isn't really how to fix either of the get() methods return types to work, but WHY the return types as they currently are don't satisfy the definition of return-type substitutability. My application of the definition tells me they should, but I'm clearly getting it wrong, and I just don't know why. Do you have any insight as to why the current methods don't satisfy the definition as they are currently written?

Thanks, again.

[Matthew Brown]

Having thought a bit more, I think you may be right. Maybe someone else can point out what I've missed, but I think that part of the spec is a little misleading.

It does appear to apply to some cases - specifically if you define a method in the superclass with a return type List<T>, then you can override it with a return type List - List being the erasure of List<T>. But I think you're right that Number is the erasure of N extends Number.

Which is why I preferred to work it out from first principles .

[Sarah Rising]

I've found some more interesting (or mysterious or maddening) things concerning this seemingly simple problem that hopefully someone can use to figure out the answer to the original answer.

Consider these few additions to the original example:

public class SuperClass<T> {
    public void  add(T dummy) { ... }  // Overridden by SubClass.add()
    public T     get() { ... } // Not overridden by SubClass.get()
    public <U extends Number> U get2() { ... } // Overridden by SubClass.get2()
  }
} 

class SubClass<N extends Number> extends SuperClass<N> {
    public void   add(Number dummy)  { ... } // Override of SuperClass.add()
    public Number get() { ... }  // Not an override of SuperClass.get()
    public Number get2() { ... }  // Override of SuperClass.get2()
  }
}

Note:
- As has already been discussed, according to the compiler, SubClass.get() is not a valid override of SuperClass.get() and that is because the return type of Sub Class.get() is not return-type substitutable for the return type of SuperClass.get(). The confusion, for me, is why that is, beings it seems the return type of Super Class.get() is (N extends Number) whose erasure is Number.

- SubClass.get2() is a valid override of Super Class.get2(). The only difference between SubClass.get() and SubClass.get2() is the name. But in SuperClass, get2() has been made a generic method with a type variable "U" with an upper bound of Number. So the compiler knows Super Class.get2()'s return type is bounded from above by Number. Therefore, in the terminology of the definition of return-type-substitutability R2 = N extends Number, its erasure is Number and, since the compiler recognizes SubClass.get2() a valid override of Super Class.get(), it is clearly saying the return types of the get2() methods are return-type-substitutable. Yet it didn't do that for the (2) get() methods. Note that the type parameter for SuperClass.get2() could also just be "U" without the "extends Number" and it would still be validly overriden by SubClass.get2().

- SubClass.add() is a valid override of SuperClass.add(). Hence the method signature of SubClass.add() is a sub-signature of SuperClass.add(). This is clearly because the method signature of SubClass.add() = add(Number) is the same as the erasure of SuperClass.add(). And that would clearly (I think) indicate that the compiler used all the information about N (i.e., N extends Number) when it determined the class SuperClass<N> that SubClass is extending. That is, it considered that the inherited method was "add(N extends Number)" and whose erasure is add(Number) which is the same as SubClass.add(). (And clearly the return types are return-type substitutable.)

This would suggest to me that the compiler, for whatever reason, uses all the information about the type variable N when determining method signature (and their erasures), but doesn't use that information when considering return-type substitutability.

Is that correct? If so, why? Thoughts anyone?

[Ikpefua Jacob-Obinyan]

Hello Sarah,
This is my personal point of view, observe VERY CAREFULLY:
Superclass:

public T get() { ... }

This method accepts T and anything that Is-a T. (covariant returns, in accordance with the java specification).

When you you say:
Subclass:

class SubClass<N extends Number> extends SuperClass<N> { }

The superclass's method now looks like This:

public N get() { ... }

This method accepts N and anything that Is-a N. (covariant returns, in accordance with the java specification).

Your subclass's method returns Number and 'Number' IS-NOT-A 'N' so the compiler complains!.
I hope this helps.

[Henry Wong]

Sarah Rising wrote:As has already been discussed, according to the compiler, SubClass.get() is not a valid override of SuperClass.get() and that is because the return type of Sub Class.get() is not return-type substitutable for the return type of SuperClass.get(). The confusion, for me, is why that is, beings it seems the return type of Super Class.get() is (N extends Number) whose erasure is Number.

The type after erasure is Object -- and the reason for erasure is that generics doesn't exist in the class files. That is the only reason that it is erased. It doesn't mean that the type is an Object during compile time though. So.... everything you said (with the exception of the erasure part) is correct. All type information is kept during compilation, just pretend that erasure doesn't exist when looking at this.

Henry

[Henry Wong]

Sarah Rising wrote:
- SubClass.get2() is a valid override of Super Class.get2(). The only difference between SubClass.get() and SubClass.get2() is the name. But in SuperClass, get2() has been made a generic method with a type variable "U" with an upper bound of Number. So the compiler knows Super Class.get2()'s return type is bounded from above by Number. Therefore, in the terminology of the definition of return-type-substitutability R2 = N extends Number, its erasure is Number and, since the compiler recognizes SubClass.get2() a valid override of Super Class.get(), it is clearly saying the return types of the get2() methods are return-type-substitutable. Yet it didn't do that for the (2) get() methods. Note that the type parameter for SuperClass.get2() could also just be "U" without the "extends Number" and it would still be validly overriden by SubClass.get2().

Actually, I don't think that this is a valid override either.... however, it looks like you only get off with a warning error, in this case. Not completely sure why.

Henry

[Matthew Brown]

Henry Wong wrote:Actually, I don't think that this is a valid override either.... however, it looks like you only get off with a warning error, in this case. Not completely sure why.

I think it's because you're overriding a generic method, but making it non-generic. So the rules for mixing generic/non-generic code kick in: it accepts it with a warning, and you get an exception at run-time if the cast doesn't work.

Edit: however, if you do this....

class SubClass<N extends Number> extends SuperClass<N> {
    public <U extends Integer> U get2() {
        return null; 
    }  // Override of SuperClass.get2()
}

This appears to be valid, but will not override the base class version! They appear to be considered separate methods. Slightly weird. Sure it all makes sense somehow, though!

[Sarah Rising]

Ikpefua Jacob-Obinyan wrote:Hello Sarah,
This is my personal point of view, observe VERY CAREFULLY:
Superclass:

public T get() { ... }

This method accepts T and anything that Is-a T. (covariant returns, in accordance with the java specification).

When you you say:
Subclass:

class SubClass<N extends Number> extends SuperClass<N> { }

The superclass's method now looks like This:

public N get() { ... }


This method accepts N and anything that Is-a N. (covariant returns, in accordance with the java specification).

Your subclass's method returns Number and 'Number' IS-NOT-A 'N' so the compiler complains!.
I hope this helps.

Thanks for your reply Ikpefua,

What you are saying is what I was getting at with the bolded statements in my last post: that the compiler uses all the information about the type variable N when determining method signature (and their erasures), but doesn't use that information when considering return-type substitutability.

Here's the code again:

public class SuperClass<T> {  
     public void  add(T dummy) { ... }  // Overridden by SubClass.add()  
     public T     get() { ... } // Not overridden by SubClass.get()  
     public <U extends Number> U get2() { ... } // Overridden by SubClass.get2()  
   }  
 }   
   
 class SubClass<N extends Number> extends SuperClass<N> {  
     public void   add(Number dummy)  { ... } // Override of SuperClass.add()  
     public Number get() { ... }  // Not an override of SuperClass.get()  
     public Number get2() { ... }  // Override of SuperClass.get2()  
   }  
 }  

Note that SubClass is parameterized with (N extends Number) and extends Superclass<N>. If Superclass<N>.get() looks like "N get()" then why does SuperClass<N>.add() look like "void add( N extends Number)" (which it would have to do so that SubClass.add() would be a subsignature of SuperClass.add() and thereby be an override of SuperClass.add() )?

That is, why does the compiler only consider "T" to be "N" for the return type of SuperClass.get() but the "T" of SuperClass.add() is taken to be "N extends Number". That's my confusion. If I can get that answered, I think there's a good chance all this might make sense?

Thanks again.

[Sarah Rising]

Henry Wong wrote:

Sarah Rising wrote:As has already been discussed, according to the compiler, SubClass.get() is not a valid override of SuperClass.get() and that is because the return type of Sub Class.get() is not return-type substitutable for the return type of SuperClass.get(). The confusion, for me, is why that is, beings it seems the return type of Super Class.get() is (N extends Number) whose erasure is Number.

The type after erasure is Object -- and the reason for erasure is that generics doesn't exist in the class files. That is the only reason that it is erased. It doesn't mean that the type is an Object during compile time though. So.... everything you said (with the exception of the erasure part) is correct. All type information is kept during compilation, just pretend that erasure doesn't exist when looking at this.

Henry

Thanks for your reply, Henry.

I understand about erasure and the reasons for it.

I understand the erasure is Object for the return type of SuperClass.get(). But why? It could only be that if the compiler didn't use the information about N having an upper bound of Number. (See the post directly above for that to be fleshed out in more detail?) And if the compiler did that, why did it use the upper bound information when it was considering the possible override of SuperClass.add() by SubClass.add()?

Thanks again.

[Henry Wong]

Sarah Rising wrote:
Note that SubClass is parameterized with (N extends Number) and extends Superclass<N>. If Superclass<N>.get() looks like "N get()" then why does SuperClass<N>.add() look like "void add( N extends Number)" (which it would have to do so that SubClass.add() would be a subsignature of SuperClass.add() and thereby be an override of SuperClass.add() )?

Actually, this is not true. You are extending SuperClass<N>, so in both cases, T is a template for N. And since N is declared as extending Number, the N in both cases, is considered as Number or a subclass of Number.

Sarah Rising wrote:
That is, why does the compiler only consider "T" to be "N" for the return type of SuperClass.get() but the "T" of SuperClass.add() is taken to be "N extends Number". That's my confusion. If I can get that answered, I think there's a good chance all this might make sense?

So, the question is why does it work for parameters, but not for the return? Take a look at the Liskov Substitution principle.

http://en.wikipedia.org/wiki/Liskov_substitution_principle

The SubClass IS-A SuperClass, so the SubClass must be able to be used everywhere the SuperClass was used. For the parameter, the SubClass takes a Number, which for N which extends Number (that the SuperClass takes) IS-A Number. For the return type, the SuperClass returns an N that extends Number, but the SubClass only returns a Number that can't be used everywhere that the SuperClass may be used.

In other words, every parameter that was passed to the SuperClass can be handled by the SubClass; however, the SubClass returns a value that can't be used in all places that the SuperClass return value can be used.

Henry

[Sarah Rising]

Henry Wong wrote:

Sarah Rising wrote:
- SubClass.get2() is a valid override of Super Class.get2(). The only difference between SubClass.get() and SubClass.get2() is the name. But in SuperClass, get2() has been made a generic method with a type variable "U" with an upper bound of Number. So the compiler knows Super Class.get2()'s return type is bounded from above by Number. Therefore, in the terminology of the definition of return-type-substitutability R2 = N extends Number, its erasure is Number and, since the compiler recognizes SubClass.get2() a valid override of Super Class.get(), it is clearly saying the return types of the get2() methods are return-type-substitutable. Yet it didn't do that for the (2) get() methods. Note that the type parameter for SuperClass.get2() could also just be "U" without the "extends Number" and it would still be validly overriden by SubClass.get2().

Actually, I don't think that this is a valid override either.... however, it looks like you only get off with a warning error, in this case. Not completely sure why.

Henry

Henry,

I believe it is a valid override, at least as defined in the JLS. (http://java.sun.com/docs/books/jls/third_edition/html/classes.html#8.4.8.1) The definition only concerns itself with whether the subclass method signature is a subsignature of the superclass method signature, and a few other items that the situation here clear satisfies. And clearly the get()'s in SubClass and SuperClass satisfy the definition. Return types only come into the picture later on in the JLS when the spec is defining various compiler errors. I suppose semantic hairs could be split, but... Anyway, if I'm unconvinced on whether my own calculations of whether one method should override another one, I just check with the @Override annotation. In this case, it tells me, it's an override. You're right, though, there is an unchecked warning. Still, I take that as an indication that the compiler is passing it as a valid override.

Regards

[Henry Wong]

Sarah Rising wrote:
I believe it is a valid override, at least as defined in the JLS. (http://java.sun.com/docs/books/jls/third_edition/html/classes.html#8.4.8.1) The definition only concerns itself with whether the subclass method signature is a subsignature of the superclass method signature, and a few other items that the situation here clear satisfies. And clearly the get()'s in SubClass and SuperClass satisfy the definition. Return types only come into the picture later on in the JLS when the spec is defining various compiler errors. I suppose semantic hairs could be split, but... Anyway, if I'm unconvinced on whether my own calculations of whether one method should override another one, I just check with the @Override annotation. In this case, it tells me, it's an override. You're right, though, there is an unchecked warning. Still, I take that as an indication that the compiler is passing it as a valid override.

Regards

I stand corrected. In the context of the JLS, I should have chosen my words better.

However, in the context of this topic, your question is not valid. When the compiler complains like that (unchecked warning), it means that it is mixing generics with non-generics, which means that for most intents and purposes, it should be treated as not using generics (because it can't be guaranteed to be type safe). Just take a quick example, let "U" be Integer, and you will see that the compiler would have been clearly wrong -- and is definitely not using generics in this regard.

Henry

[Sarah Rising]

Henry,
Thanks for your replies. I'm pretty sure I'm mostly solid on the items in your last several posts before I even got confused about this little problem of mine. I don't have any more time to look at it right now but will return either late tonight or tomorrow. One quick question about something in your first post that I might have missed: why did you say the erasure of the return type was Object if the compiler is assuming T is some subtype of Number?

Thanks again,
Sarah