aspose file tools*
The moose likes I/O and Streams and the fly likes StreamTokenizer and hard returns Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "StreamTokenizer and hard returns" Watch "StreamTokenizer and hard returns" New topic
Author

StreamTokenizer and hard returns

Tom Griffith
Ranch Hand

Joined: Aug 06, 2004
Posts: 272
Hello. If somebody has a minute, I am trying to read the input from a semicolon delimited text file with the following format...

The Dog;The Cat;The Mouse
The Motorcycle;The Car;The Truck
The Square;The Circle;The Oval

I eventually want to convert each line to a row in a db table.

What I'm having trouble with is reading the hard return in the input stream. The crux of the code is as follows...

FileReader rd = new FileReader("relocated_input.txt");
StreamTokenizer st = new StreamTokenizer(rd);
st.quoteChar(';');
st.wordChars(0,' ');
st.eolIsSignificant(true);

// Parse the file
while (StreamTokenizer.TT_EOF != st.nextToken()) {
String word = st.sval;
System.out.println(word);
}
rd.close();

this is the output...

The Dog
The Cat
The Mouse
The Motorcycle
The Car
The Truck
The Square
The Circle
The Oval


Other than that, it seems I am able to get a null value for the hard returns if I have st.eolIsSIgnificant(true) and I ~don't~ designate the space (' ') as a word constituent. But I have to do that because the tokens might, or in this case, they all do, consist of two words and a space in between. So what I have is successful parsing but I can't get a handle on where the hard returns are located in the stream so I can seperate rows in the db. Thank you very much for reading. Any help is appreciated...I'll keep messing with it.
[ March 22, 2005: Message edited by: Tom Griffith ]
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8927
    
    9

I'd use a BufferedReader to read the file with readLine() so each record comes in as an individual string, then parse each line with a StringTokenizer (or better yet, String.split()).
[ March 22, 2005: Message edited by: Joe Ess ]

"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Tom Griffith
Ranch Hand

Joined: Aug 06, 2004
Posts: 272
I was trying just that but it seems like readLine() appears to be giving me every other line or something...

BufferedReader rd = new BufferedReader(new FileReader("relocated_input.txt"));

while (rd.readLine() != null) {

String current_line = rd.readLine();

InputStream in = new StringBufferInputStream(current_line);

StreamTokenizer st = new StreamTokenizer(in);

st.quoteChar(';');
st.wordChars(0,' ');
st.eolIsSignificant(false);

// Parse the line

while (StreamTokenizer.TT_EOF != st.nextToken()) {

String word = st.sval;

System.out.println(word);

}

// END WHILE
}

yields...

The Motorcycle
The Car
The Truck

Yeah, I was jsut reading somewhere that StringTOkenizer is being phased out and the documentation suggested using Split on a StringBuffer. But even so, something is weird with my looping on the nextLine() method. Thank you very much for reading my question...
[ March 22, 2005: Message edited by: Tom Griffith ]
Tom Griffith
Ranch Hand

Joined: Aug 06, 2004
Posts: 272
Making this change to the current_line declaration and assignment has the loop working ok...

BufferedReader rd = new BufferedReader(new FileReader("relocated_input.txt"));

String current_line;

while ( (current_line = rd.readLine()) != null) {


as opposed to this which was giving me unpredictable results (every other line, etc)...


while ( rd.readLine() != null) {

String current_line = rd.readLine();

Obviously creating a new String object within the loop as opposed to overwriting the reference to the String object was messing with it. Again, thank you very much for your help.
Carol Enderlin
drifter
Ranch Hand

Joined: Oct 10, 2000
Posts: 1364
Your while statement was throwing every other line away.

while ( rd.readLine() != null) { <<--- readLine, but don't capture it

String current_line = rd.readLine();
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: StreamTokenizer and hard returns