Win a copy of Re-engineering Legacy Software this week in the Refactoring forum
or Docker in Action in the Cloud/Virtualization forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

How to use the API to read StringTokenizer?

 
Ls chin
Ranch Hand
Posts: 99
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi all,

I was browsing through 'The Java Developers Almanac' and found this StringTokenizer API which I don't understand. I hope someone can help me, please? Thank you!



1) Where can we find the StringTokenizer class? Does it come with the JDK?
2) Why is it called 'parser'? Can we rename it to something else?
3) What is processWord, is it like a method?

Thank you for your replies.
 
aditee sharma
Ranch Hand
Posts: 182
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1) Where can we find the StringTokenizer class? Does it come with the JDK?
Yes.java.util.StringTokenizer

2) Why is it called 'parser'? Can we rename it to something else?
Yes. How about "tweety" ?

3) What is processWord, is it like a method?
just a arbitrary name to illustrate that you can pass the String token obtained from StringTokenizer to any method you like.

I guess you would like to explore more in Sun docs.
 
Henry Wong
author
Marshal
Pie
Posts: 20904
76
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1) Where can we find the StringTokenizer class? Does it come with the JDK?

http://java.sun.com/j2se/1.5.0/docs/api/java/util/StringTokenizer.html

2) Why is it called 'parser'? Can we rename it to something else?

From the code that you shown, it looks like a variable, so I guess you can name it to anything that you like.

3) What is processWord, is it like a method?

That is not part of the String tokenizer. It looks to be a method that has been defined somewhere else in your book.

Henry
 
Campbell Ritchie
Sheriff
Posts: 48448
56
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
. . . and when you did find StringTokenizer, did you find the bit telling you not to use it? It says to use String#split and the Regular Expressions API instead.

[edit]Fix broken link[/edit]
[ September 25, 2008: Message edited by: Campbell Ritchie ]
 
Ls chin
Ranch Hand
Posts: 99
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Aditee, Henry, Campbell,
Thanks for the explanations! I actually know a bit about StringTokenizer, but just wanted to confirm my understanding. Thank you!

Originally posted by Campbell Ritchie:
StringTokenizer, did you find the bit telling you not to use it? It says to use String#split and the URL=http://java.sun.com/docs/books/tutorial/essential/regex/index.html]Regular Expressions API[/URL] instead.

Campbell, I think the first link is broken. Hmm, the don't use StringTokenizer bit? I didn't see it mentioned anywhere. Did I miss it?

I haven't read the regex part yet. It sounds more complicated than the tokenizer.
 
Henry Wong
author
Marshal
Pie
Posts: 20904
76
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell, I think the first link is broken. Hmm, the don't use StringTokenizer bit? I didn't see it mentioned anywhere. Did I miss it?


The actual quote from the JavaDoc reads as....

StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.


Henry
 
Gamini Sirisena
Ranch Hand
Posts: 378
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
But take a look here.

It seems StringTokenizer is considerably faster than String.split for sufficiently large number of tokens.

What do you think guys?
 
Campbell Ritchie
Sheriff
Posts: 48448
56
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by LS chin:
Campbell, I think the first link is broken. Hmm, the don't use StringTokenizer bit? I didn't see it mentioned anywhere. Did I miss it?

I haven't read the regex part yet. It sounds more complicated than the tokenizer.


The link was broken; thank you for pointing it out. There was a duplication of the http:// bit which I have corrected. But the delimiter for StringTokenizer behaves a bit like a regular expression anyway.
 
Ove Lindström
Ranch Hand
Posts: 326
Android Firefox Browser Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by Gamini Sirisena:
But take a look here.

It seems StringTokenizer is considerably faster than String.split for sufficiently large number of tokens.

What do you think guys?


Well, it is true, but when you start parsing strings that are so large that the difference in performance matters, then you should look into other ways of parsing them than using split or StringTokenizer.

My opinion is that if you find yourself stuck in parsing huge amounts of text, redesign it so that it uses XML or direct data source access.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic