• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

StringTokenizer problem

 
Jacky Luk
Ranch Hand
Posts: 634
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

While this input is taken from Excel, in the field called Qualification and Address, there may be some spaces in between, so the tokenizer recognizes the contents of those fields as
2 tokens. How do I avoid that from happening?
Thanks
Jack
 
Campbell Ritchie
Sheriff
Pie
Posts: 49367
62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why are you using a tokenizer in the first place? Have you read its documentation?
One possibility: Export the spreadsheet file as a CSV and get yourself a CSV parsing program.
 
Jacky Luk
Ranch Hand
Posts: 634
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:Why are you using a tokenizer in the first place? Have you read its documentation?
One possibility: Export the spreadsheet file as a CSV and get yourself a CSV parsing program.


still trying

Although this recognizes spaces in between, but this doesn't recognize completely empty fields.
Is there improvement that I can make?
Thanks
Jack
 
Campbell Ritchie
Sheriff
Pie
Posts: 49367
62
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Exporting data from spreadsheets etc is very flaky'; there is tight coupling between the structure of the spreadsheet and the output. You would have to make sure there are no empty fields. Are you really using tab-null as the delimiter?
Why don’t you use a CSV as your output?
 
Jacky Luk
Ranch Hand
Posts: 634
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:Exporting data from spreadsheets etc is very flaky'; there is tight coupling between the structure of the spreadsheet and the output. You would have to make sure there are no empty fields. Are you really using tab-null as the delimiter?
Why don’t you use a CSV as your output?


Dear Campbell,
Rather than exporting into a csv file for further processing, I am actually importing the clipboard data into the java application and break down the input into various fields, just for convenience. I thought you already knew, but in case you could misunderstand my intent.
Thanks
Jack
 
Campbell Ritchie
Sheriff
Pie
Posts: 49367
62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, it was obvious you were taking data from the clipboard. Not that I have seen the text transfer class before. I was wondering whether that is the bext method, because it is so easy to turn spreadsheets into CSVs and vice versa. Don’t know what to do about your blanks. Sorry.
 
Jacky Luk
Ranch Hand
Posts: 634
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Campbell Ritchie wrote:Yes, it was obvious you were taking data from the clipboard. Not that I have seen the text transfer class before. I was wondering whether that is the bext method, because it is so easy to turn spreadsheets into CSVs and vice versa. Don’t know what to do about your blanks. Sorry.


I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Thanks
Jack
 
Joanne Neal
Rancher
Posts: 3742
16
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jacky Luk wrote:I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Which parser are you using ? Any properly designed CSV parser will handle that (assuming you want it split into three tokesn - 7, 10 and 10, High Street). Take a look at the suggested ones in the Excel section of AccessingFileFormats
 
Jacky Luk
Ranch Hand
Posts: 634
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Joanne Neal wrote:
Jacky Luk wrote:I've converted everything now. But just one thing
If I have a set of data like this, the parser will get confused.

Which parser are you using ? Any properly designed CSV parser will handle that (assuming you want it split into three tokesn - 7, 10 and 10, High Street). Take a look at the suggested ones in the Excel section of AccessingFileFormats


Hello Joanne,
Actually I possess a parser of my own which looks like the following:



 
Joanne Neal
Rancher
Posts: 3742
16
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jacky Luk wrote:Actually I possess a parser of my own

But that's not a CSV parser. The CSV specification is not as simple as it first appears and, as you have found, you certainly need something that does more than just split a string whenever it finds a comma.
 
Winston Gutkowski
Bartender
Pie
Posts: 10422
63
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Joanne Neal wrote:Take a look at ... AccessingFileFormats

Thanks, Joanne. Fabulous page! And kudos to whoever wrote it. Bookmarked.

Winston
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic