The main reason why String made immutable was security.
Look at this example: We have a file open method with login check. We pass a String to this method to process authentication which is necessary before the call will be passed to OS. If String was mutable it was possible somehow to modify its content after the authentication check before OS gets request from program then it is possible to request any file. So if you have a right to open text file in user directory but then on the fly when somehow you manage to change the file name you can request to open "passwd" file or any other. Then a file can be modified and it will be possible to login directly to OS.
I am not sure about it but basically these are value Objects and you would use them as values not as reference. I hope you will get much better answers here.
You should consider whether any class can be made immutable. Look at the design of java.awt.Point. A mutable class with public fields. A disaster to try to synchronise. Immutability carries all sorts of advantages, for example: