This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Java in General and the fly likes StreamTokenizer Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "StreamTokenizer" Watch "StreamTokenizer" New topic


vinodh venkataraman

Joined: Oct 02, 2005
Posts: 3
I'm using StreamTokenizer to read contents from a text file. I want to retain the spaces in the text file, but streamtokenizer removes the spaces and appends all the characters in the file. I tried using ordinaryChar() method, but that didn't really help either. Any suggestions would really help...

Reader read = new BufferedReader(new InputStreamReader(myFile));//read the contents using buffered reader
StreamTokenizer in = new StreamTokenizer(read); //read the tokens
in.ordinaryChar(' ');

Layne Lund
Ranch Hand

Joined: Dec 06, 2001
Posts: 3061
I don't think StreamTokenizer returns the delimiters (i.e. the spaces) to you. Why do you need them anyways? If you can explain your reasoning here, we might be able to help you find an alternate solution. Perhaps StringTokenizer will help, but I'm unsure that this is what you really need.


Java API Documentation
The Java Tutorial
Norm Radder
Ranch Hand

Joined: Aug 10, 2005
Posts: 685
StreamTokenizer can be tailored:

stzr.ordinaryChar(' '); // Return spaces

will return the space character which you handle as below in the switch stmt:
case ' ':
vinodh venkataraman

Joined: Oct 02, 2005
Posts: 3
My program has to choose random sentences based on a randomly generated line number from a text file. I thought using StreamTokenizer will help me use the lineno() function for selecting a random line number.
Anyways, I'm gonna try the case ' ' option and see how it works...
It is sorta covered in the JavaRanch Style Guide.
subject: StreamTokenizer
Similar Threads
Parsing huge CSV file/ Time to load in the memory
Stream Tokenizer - Special case
StreamTokenizer usage
why i can't compare between streams ??