StringTokenizer

[Jinny Morris]

Does anyone know why StringTokenizer is no longer "preferred"? I have looked at the Split method, but can see no way to retrieve which one of a list of possible separators was used. StringTokenizer lets me retrieve the separators as tokens. I hope that StringTokenizer isn't going to be deprecated ...

Thanks!

[Chris Corbyn]

split() takes a regular expression. If you don't want to loose any tokens when you split() then use a zero-width assertion, or a series of them.

Split at all word boundaries...

String[] tokens = someString.split("\\b");

There are loads of ways to tokenize the strng in a lossless fashion

If you're trying to do anything more than a basic split however then there's a pretty good chance you really do have to tokenize the string manually.