You could probably implement something with the String.indexOf and String.substring methods.
Going back to your original code, String.replace replaces all instances of the searched for String, so you don't need to put it in a loop. You do however have to assign the value returned by the replace method to a variable.
In JDK 1.4, you can't use that version of replace(). You can however use replaceAll(). Despite the very poorly chosen names of these methods, the difference between them is not that one replaces "all" and the other doesn't. The difference is that one uses regular expressions and the other doesn't. If you use replaceAll(), you may need to learn about regular expressions to understand some details of its behavior. However to replace a single ',this will probably not e an issue. Give it a try.