Win a copy of Re-engineering Legacy Software this week in the Refactoring forum
or Docker in Action in the Cloud/Virtualization forum!

# StringTokenizer confusion

Sally Curtis
Greenhorn
Posts: 26
hi, i am writing a quadratics program. I want to use the string tokenizer to separate the 'problem' into three sections. for ex. -3x^2 + 2x -1....i want to have -3x^2 +2x and -1. How would i use each of the tokens, individualy? Can i do that?

Tim West
Ranch Hand
Posts: 539
Hi,
It seems to me that this problem is a little more difficult than it might first appear. Two issues:
(1) You want to "keep" the thing you're splitting with - you want to split at + or -, but the + or - is significant and is kept as part of the string to its right.
(2) The language you're describing could be made quite complex, depending on how much effort you want to put in.
I'm not an expert on parsing/language structure, but I think you'll want to make as many assumptions about the formation of the structure of the input strings as possible. (Eg, how many terms are you allowed? Can you have just 'x + 1'? Is the unknown variable always 'x'? Can you have multiple terms of the same order (eg 2x^2 + x + 2x + 1)? Is whitespace allowed? Do the highest-power terms always come first?
I think you'll have more luck with the split() method of String and regular expressions. If you're unfamiliar with regular expressions this could be a little difficult however.
That said, someone else out there could have a much more simple solution.
Cheers,

--Tim

Mike Gershman
Ranch Hand
Posts: 1272
You can try something like this:
new StringTokenizer(myString, "+-", true):
This would yield 6 tokens:
-
3x^2
+
2x
-
1
You could combine strings to get what you asked.
To solve the problem with more help from Java, take a look at StreamTokenizer. It will break up the expression completely into labels and operators, treat multi-character numbers and labels as single tokens, tell you what type each token is, etc. On the other hands, your HW may require using StringTokenizer, and StreamTokenizer is more complicated to learn.

Ben Wood
Ranch Hand
Posts: 342
I might consider splitting the whole thing into an array of characters and working from there. But then again I don't know how to use regular expressions so that might be the better way

Gabriel White
Ranch Hand
Posts: 233
I don't think you can even use a stream tokenizer because it only yields at TT_EOF, TT_EOL, TT_NUMBER, and TT_WORD. This would not include the exponent that you are trying to include. The best way to solve this equation is to use a stack, but I didn't think you just wanted to solve it. Lemme work on this a little more and see what I can come up with on the stream tokenizer.

Mike Gershman
Ranch Hand
Posts: 1272
A StreamTokenizer will return special characters, which it calls "ordinary characters", as separate tokens whose ttype is their character value. This will work for ^
To be sure + and - are treated this way and not part of a number, include these calls:
st.ordinaryChar('-');
st.ordinaryChar('+');