*
The moose likes Java in General and the fly likes StringTokenizer problem Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "StringTokenizer problem" Watch "StringTokenizer problem" New topic
Author

StringTokenizer problem

Jacky Luk
Ranch Hand

Joined: Aug 02, 2012
Posts: 590

While this input is taken from Excel, in the field called Qualification and Address, there may be some spaces in between, so the tokenizer recognizes the contents of those fields as
2 tokens. How do I avoid that from happening?
Thanks
Jack
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 37936
    
  22
Why are you using a tokenizer in the first place? Have you read its documentation?
One possibility: Export the spreadsheet file as a CSV and get yourself a CSV parsing program.
Jacky Luk
Ranch Hand

Joined: Aug 02, 2012
Posts: 590
Campbell Ritchie wrote:Why are you using a tokenizer in the first place? Have you read its documentation?
One possibility: Export the spreadsheet file as a CSV and get yourself a CSV parsing program.


still trying

Although this recognizes spaces in between, but this doesn't recognize completely empty fields.
Is there improvement that I can make?
Thanks
Jack
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 37936
    
  22
Exporting data from spreadsheets etc is very flaky'; there is tight coupling between the structure of the spreadsheet and the output. You would have to make sure there are no empty fields. Are you really using tab-null as the delimiter?
Why don’t you use a CSV as your output?
Jacky Luk
Ranch Hand

Joined: Aug 02, 2012
Posts: 590
Campbell Ritchie wrote:Exporting data from spreadsheets etc is very flaky'; there is tight coupling between the structure of the spreadsheet and the output. You would have to make sure there are no empty fields. Are you really using tab-null as the delimiter?
Why don’t you use a CSV as your output?


Dear Campbell,
Rather than exporting into a csv file for further processing, I am actually importing the clipboard data into the java application and break down the input into various fields, just for convenience. I thought you already knew, but in case you could misunderstand my intent.
Thanks
Jack
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 37936
    
  22
Yes, it was obvious you were taking data from the clipboard. Not that I have seen the text transfer class before. I was wondering whether that is the bext method, because it is so easy to turn spreadsheets into CSVs and vice versa. Don’t know what to do about your blanks. Sorry.
Jacky Luk
Ranch Hand

Joined: Aug 02, 2012
Posts: 590
Campbell Ritchie wrote:Yes, it was obvious you were taking data from the clipboard. Not that I have seen the text transfer class before. I was wondering whether that is the bext method, because it is so easy to turn spreadsheets into CSVs and vice versa. Don’t know what to do about your blanks. Sorry.


I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Thanks
Jack
Joanne Neal
Rancher

Joined: Aug 05, 2005
Posts: 3419
    
  12
Jacky Luk wrote:I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Which parser are you using ? Any properly designed CSV parser will handle that (assuming you want it split into three tokesn - 7, 10 and 10, High Street). Take a look at the suggested ones in the Excel section of AccessingFileFormats

Joanne
Jacky Luk
Ranch Hand

Joined: Aug 02, 2012
Posts: 590
Joanne Neal wrote:
Jacky Luk wrote:I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Which parser are you using ? Any properly designed CSV parser will handle that (assuming you want it split into three tokesn - 7, 10 and 10, High Street). Take a look at the suggested ones in the Excel section of AccessingFileFormats


Hello Joanne,
Actually I possess a parser of my own which looks like the following:



Joanne Neal
Rancher

Joined: Aug 05, 2005
Posts: 3419
    
  12
Jacky Luk wrote:Actually I possess a parser of my own

But that's not a CSV parser. The CSV specification is not as simple as it first appears and, as you have found, you certainly need something that does more than just split a string whenever it finds a comma.
Winston Gutkowski
Bartender

Joined: Mar 17, 2011
Posts: 7503
    
  18

Joanne Neal wrote:Take a look at ... AccessingFileFormats

Thanks, Joanne. Fabulous page! And kudos to whoever wrote it. Bookmarked.

Winston

Isn't it funny how there's always time and money enough to do it WRONG?
Articles by Winston can be found here
 
jQuery in Action, 2nd edition
 
subject: StringTokenizer problem
 
Similar Threads
Splitting a string to smaller strings.
please help with tokenizing
breaking up a file that has a delimiter
Please help.... Read data in between
A looping question