Two Laptop Bag
The moose likes I/O and Streams and the fly likes StreamTokenizer help Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "StreamTokenizer help " Watch "StreamTokenizer help " New topic

StreamTokenizer help

fred fredricks

Joined: Mar 04, 2003
Posts: 16
hi, does any1 know any good tutorial or examples of StreamTokenizer?? or maybe some1 can help me out a little?
i need to count words from a txt file..
i am really stupid and dont understand what it says in my txt book!
Maulin Vasavada
Ranch Hand

Joined: Nov 04, 2001
Posts: 1873
hi mike,
here is a sample code i wrote...

the input file i used is,]
243525 100 324200 2342 maulin
Ignoring WORD:maulin:
Ignoring WORD:ew22:
Ignoring WORD:fw:
Ignoring WORD:sdfsfsa:

i would try to explain the flow of events here...
- once we created the StreamTokenizerTest object and provided a file name to the argument of the method tokenizeFile() we are in the tokenizeFile() method...
- on Line-1 we create a Input stream to read the file
- on Line-2 we create a StreamTokenizer object that can tokenize the input file by reading it.
to do this we passed Reader type (which is FileReader) object to its constructor
- Line-3 says that -"run in the loop reading the file in tokens separated by white spaces untill you hit End Of File EOF". The StreamTokenizer.TT_EOF is a variable to indicate EOF value when the tokenizer reaches EOF while reading the file...
- Line-4 says that if the current token is "Number" then process it. "ttype" is a field in StreamToeknizer object to obtain the token type read by the tokenizer. the variable "nval" in the StreamToeknizer is to obtain the "number" value if the token was "number" indicated by "ttype".
- Line-5 is similary for WORD tokens (which are not numbers) and Line-6 is to identify the "new line" tokens in the file..
on Line-7 we close the FileReader as StreamTokenizer has reached the EOF and no more reading of the file is needed...
for more look at,
1. applet at this location
2. More explanation about the StreamTokenizer
3. Yet another tutorial
i bet u can find thousands on google...
hope this helps..
Maulin Vasavada
Ranch Hand

Joined: Nov 04, 2001
Posts: 1873
the second link i meant was,
this not the one i gave in above post...
this is better..
I agree. Here's the link:
subject: StreamTokenizer help
jQuery in Action, 3rd edition