Don't know. These blocks are annoying; they usually appear on shells or command prompt windows, and represent characters which are outwith the capabilities of the OS to print. Since Swing supports the whole of Unicode it oughtn't to print blocks. Suggest as a first step:-
Add a print button to your GUI,
Add an ActionListener to it,
In that ActionListener call a method which scans the text in the TextField,
In that method, break the text into chars (toCharArray()),
Print the individual characters, but cast to an int first. See if you can print them in hex.
Another way to do it is to try a substring missing out the last character in the String. I tried programming an app to mimic your problem and ran it with J1.4.2:-This does not have any boxes or squares in. So it probably isn't a bug in J1.4