Yes, you should. If you don't you are breaking the contract for the hashCode() and equals() methods as specific in the java.lang.Object class. HashMaps and HashSets make use of that contract, but other code might as well. There's no way you can guarantee that your class will never be used in a way that makes use of the contract, so you should follow it. Especially since it's not particularly difficult (in fact, modern IDEs will do it for you!).
By not doing, you are in effect "lying" about the specification of your class, as far as other developers are concerned.