StringTokenizer

[Shannon Needham]

Help!! I've been out this program for awhile and can not get it to not count tab.
For example, if I imput 'k k k', with tabs between each k, it will count it as a one 3 letter word, instead of three 1 letter words. Space works correctly, but tab does not. I know tab is one of the default delimiters, but it's not working. I'm at a loss.

tokenizer = new StringTokenizer(lineOfText);
while (tokenizer.hasMoreTokens())
{
currentWord = tokenizer.nextToken();
characterCount = currentWord.length();
// Increment the counter corresponding to the length of this word.
switch (characterCount)
{
case 1 :
length1Count++;
break;
case 2 :
length2Count++;
break;
case 3 :
length3Count++;
break;
case 4 :
length4Count++;
break;
case 5 :
length5Count++;
break;
case 6 :
length6Count++;
break;
case 7 :
length7Count++;
break;
case 8 :
length8Count++;
break;
case 9 :
length9Count++;
break;
default:
length10Count++;
}

[Dirk Schreckmann]

Welcome to JavaRanch, Shannon!
This works as expected for me. Are you sure that your input String has tab characters in it? What does yourInputString.indexOf("\t") return?

import java.util.StringTokenizer;
 
public class Foo
{
  public static void main( String[] args )
  {
    String word = "c\tc\tc";
    StringTokenizer st = new StringTokenizer(word);
    while (st.hasMoreTokens()) {
      System.out.println(st.nextToken());
    }
  }
}

output
c
c
c
[ March 25, 2003: Message edited by: Dirk Schreckmann ]

[Shannon Needham]

Here's what the program should do:
1. Read in a series of lines of text from the user.
The user will signal that he is finished by typing in the word “DONE” (in uppercase) after the last line to be processed.
2. Keep track of the count of all words of length 1, 2, and so on
up through 9 characters, as well as the count of words 10 or more
characters long. Assume that words are delimited by a space, new line or tab.
3. Display these counts in a well labeled fashion.

When I tab any text w/tabs, they are not recoginized as delimiters. Hope I'm making sense.

[Layne Lund]

Whenver you have similar variable names that you only differ with a number, you can usually replace them all with an array. This is an easy way to your code significantly.
In this case, you should first declare an array wherever it is appropriate for your program, either as a class member or as a local variable:

int[] lengthCount = new int[10];

Be sure to initalize each element of the array to 0. If you declare lengthCount as a class member, the initialization is in the constructor. If lengthCount is local, then the intialization is local to the same method as well. Finally, replace your switch statement with this single line:

lengthCount[characterCount]++;

HTH
Layne

[Shannon Needham]

Thanks for the info. Will that fix the stringtokenizer problem that I'm having? The program works except when I enter a tab between two words, then it counts it as one word. For example, if I typed "yes no" with a tab between the words instead of a space, it would count it as a 5 letter word vs. one 3 letter word and one 2 letter word. Hope I'm making myself clear. So other than this tab issue, my program works great. But I just know the professor will test w/a tab since it was in the assignment to assume that words are delimited by a space, tab, or new line. Tab is the only part not working. Hope that explains my problem better.

[Dirk Schreckmann]

Originally posted by Dirk Schreckmann:
Are you sure that your input String has tab characters in it? What does yourInputString.indexOf("\t") return?

[Shannon Needham]

Dirk, I don't have an input String statement. The program reads in the text from user input. And when I test the program, I do hit "tab" key on keyboard. I'm probably banging my head over nothing b/c if I can't figure it out, most of the class probably can't. But darn it, I want it to work right.
System.out.print ("Enter text, type DONE (on new line) to quit: ");
lineOfText = Keyboard.readString();
I hope this beginner hasn't driven you crazy. Thanks for your help.

[Layne Lund]

You can't drive us somewhere we already are Well, I guess I can only speak for myself *mwhahahaha*
Also, my above suggestion won't fix the problem you've encountered with StringTokenizer, but it WILL reduce the amount of code you have. You will need to tweak it a little to keep track of words 10 and larger, but I'll leave that as the proverbial exercise for the reader.
Now on to your actual question, finally! I think the others are trying to point out that you may have pushed the Tab key when entering the text, but lineOfText may not actually contain a '\t' char. You should use the previous suggestion to see if it does or not:

int index = lineOfText.indexOf("\t");
System.println("index=" + index);

If index is -1 then your String doesn't even contain the '\t' char and the problem doesn't even concern StringTokenizer. If it is anything else (greater than 0), we'll have to look elsewhere for the solution.
HTH
Layne

[Jason Menard]

It works for me with tabs. Here's a hint that might help you out:

BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
String line = br.readLine();

Also, the above poster had a good point concerning using an array to keep track of your word count. For example, lets say I had the following:
int[] count = new int[11];
If my word had a length >= 0 but <= 10, then I could do count[length]++, and if my word had a length > 10, then I would do count[10]++.
HTH
[ March 25, 2003: Message edited by: Jason Menard ]

[Shannon Needham]

Thanks for the line of code to check what was happening in my program. Yes, it does give me
-1. So does that mean, the "tab" key does not actually register as \t in java. It sounds to me that tab can not be a delim in this program since the user types in the text. It will be interesting to see if professor tests with a tab and realizes the problem.

[Layne Lund]

Actually, I suspect the problem is with the Keyboard.readString() method. The tab key SHOULD generate a '\t' character as you originally assumed. Did you or your teacher write the readString() method? It isn't in the Java API, afaik. It seems there is a previously undiscovered bug in this method.

[Shannon Needham]

The keyboard class came with the textbook. It's written by Lewis & Loftus. That makes sense to me that would be where the problem lies. It will be interesting to see if the professor is aware of it. I'm learning more on this forum than in class. I don't feel so bad about missing class today. Now if I could just get as interested in my techinical writing class.

[Layne Lund]

Well, don't make a habit of skipping your Java class. However, feel free to skip technical writing whenever you want (or even if you don't want to skip, do it anyway).