Win a copy of Design for the Mind this week in the Design forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

StringTokenizer Bug?

 
Ryan Bailey
Ranch Hand
Posts: 134
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok simple short and sweet.
I'm writting an application that used the StringTokenizer to parse thru data. I am delimiting it by comma (,).

Example Data:
"a,b,c,,e"
When I iterate thru the String I would expect the StringTokenizer to process this result:
"a" "b" "c" "" "e"
Instead I'm getting:
"a" "b" "c" "e"

Any idea why the StringTokenizer isn't tokenizing empty Strings??? I looked in the javadoc and it says nothing about ignoring empty Strings. I guess I could write my own tokenizer or do something to the data before tokenizing to place N/A where data doesn't exist. But don't want to go to the trouble if it's handled somehow in the StringTokenizer class.

Regards,
Ryan
 
Shailesh Chandra
Ranch Hand
Posts: 1082
Java Oracle Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Ryan Bailey:
Any idea why the StringTokenizer isn't tokenizing empty Strings???

Yes this a limitation of StringTokenizer or bug if you say.I have read somewhere that StringTokenizer is a legacy class and sun discourage use of it, but hasn't deprecated this so far

Originally posted by Ryan Bailey:
I guess I could write my own tokenizer or do something to the data before tokenizing


You need not to write your own tokenizer if you are using jdk1.4 as in JDK 1.4 String class has a split method.
also split method consider empty tokens in a string.

Shailesh
[ April 30, 2005: Message edited by: Shailesh Chandra ]
 
Ryan Bailey
Ranch Hand
Posts: 134
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
LOL... So much for my SuperStringTokenizer class

Thanks for the reply!
Ryan
 
David Harkness
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
By default, StringTokenizer groups one or more consecutive delimiter characters together to separate tokens. However, it has an alternate method that will return both the tokens and each single delimeter character, also as a token.
If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens.
While not as simple to use as String.split(","), it would allow you to determine when to create empty tokens. If you have to run in pre-1.4 JVMs, it's a reasonable route. Of course, if comma is your only delimiter character, you could write your own split method in a few lines of code.
[ April 30, 2005: Message edited by: David Harkness ]
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic