Nulls in StringTokenizer

[Nilesh Srivastava]

Hi ,

I have a tab separated file and I have to port the data to DB through StringTokenizer.Every thing is worrking fine, except whenever there is a "null", String Tokenizer simply ignores it.

For example,

If my string is : - TERMINOLOGY<TAB>GOOGLE (<TAB> signifies tab)
then I will get two tokens as TERMINOLOGY and GOOGLE.

But if I have TERMINOLOGY<TAB><TAB>GOOGLE
then I should get three tokens instead.
TERMINOLOGY,null(or space) and GOOGLE.
But StringTokenizer returns only two.TERMINOLOGY and GOOGLE.

I have an alternate way to do this through character matching then I will have to match each and every character in my TAB separated file, which will be too time taking.

Would be grateful if anyone helps me.

[Horatio Westock]

Try,

String tabseperated = "TOKEN1\t\tTOKEN2";
String[] tokens = tabseperated.split("\t");

Basically StringTokenizer is old and rubbish, and still there soas not to break legacy code. You should use regular expressions instead, where possible.

You can write your own StringTokenizer using regular expressions, use one of the many free implementations out there, or if you are using 1.5, you could look at java.util.Scanner.

Hope this helps
[ March 09, 2005: Message edited by: Horatio Westock ]

[Paul Sturrock]

You could use the StringTokenizer constructor which also returns the delimiters as tokens:

StringTokenizer(String str, String delim, boolean returnDelims)

Then you could cehck eack token and handle it if it is not a delimiter.

Alternatively, as the StringTokeniser JavaDocs suggest, you could use the split method of the String class, which will include the "null" value in the array it returns.

[ramprasad madathil]

But StringTokenizer returns only two.TERMINOLOGY and GOOGLE.

That's what the api promises and that's how it will work. You should write your own logic if you want it different. Probably String.split(String regex) method may help, check it out.

ram.