StreamTokenizer 'word' tokens

[Daniel Walker]

Hi there. I am using a StreamTokenizer to parse a data file, containing multiple entries which look like this:

@article{walker1980,
  author="Daniel Walker",
  title="BibtexingAndStuff",
  year="1980"
}

Things are working OK, until I get to any entry which has it's value spread over multiple lines. The StreamTokenizer will tokenize from a quote character (") to either another quote character or an end of line - and this is the problem. I have tried removing any newline characters from the source document in a preprocessing stage, but this did not work too well. Any ideas?

[Dale DeMott]

Are you reading in 1 line at a time? What are you removing from your string?? is it \n or \r??
-Dale

[Jim Yingst]

Try setting eolIsSignificant(false).