Parsing

[Chris Cairns]

Hi Folks,
I'm hoping that someone could help me out with the following problem. I'm trying to parse the following text.

           DIST   FUEL  TIME    ROUTE   CRZ       TOW      ZFW   KGS
TRIP-SAEZ   670    3940 01.31    9313   M74      47030    40560
RES                 310 00.09
ALTN-SUMU   134    1160 00.24                   ......   ......
HOLD               1060 00.30
AFU                   0 00.00  MAX POSS LOAD   TOF .. .. ..
MFR                6470 02.34  ZFW     48300   POB .../...
XFU                   0 00.00  TIME    01.31   ATD  ..:../..:..
TOF                6470 02.34  BRNOFF   4430   TIME       01.31
TXY                 100                        ETA        ..:..
TOTAL              6570                        ATA  ..:.. ..:..

As you can see, there is a sort of column and row heading. For example, RES/FUEL means reserved fuel. I'm a little clueless as how to approach this. I just started this job a few days ago (right out of college) and the company has absolutely no documentation on similar solutions to help me out. If I took the time to try to figure out the other developer's solution, it would take me forever because it's too complex at this point for me.
One idea I had is to create a multidimensional array. That way, for example, when I try to extract the value for attribute reserved fuel, I could just use indices. Another issue is dealing with the heading inside the txt document, which MAX POSS LOAD.
I don't know. Any help would truly be appreciated.
[ April 01, 2003: Message edited by: Jim Yingst ]

[Chris Cairns]

As you can see, the text didn't properly format when I pasted it.

[Chris Cairns]

Actually, it did come out all right. Just make sure your text size is lowered.

[Jim Yingst]

I added [code] tags to preserve the indentation.
First, you can break this into separate lines by reading it with a BufferedReader, which has a readLine() method. For each line then, you need to separate the different fields. It looks like you can identify fields just by counting characters - e.g. the DIST field seems to start in the 12th or 13th column, and end in the 15th. You can extract this with String's substring() method. (Read the API for this carefully.) The you can convert the data to a non-String format using methods like trim() and Integer.parseInt() - E.g.
String line = buffReader.readLine();
...
String distStr = line.substring(11, 15);
int dist = Integer.parseInt(distStr.trim());
This approach should work if the column numbers are consistent throughout your input file. If they're not, well, you'll have to study the file structure more to look for consistent patterns which you can use.

[Jason Menard]

Let's begin by defining the problem set? What are the different fields each line contains and the possible permutations thereof. Are there any field delimiters? It looks to me like it's just spacing things out, but it's still something to think about. Can we guarantee that each field occupies a certain range of character positions in all cases? For example, can we guarantee that the first field will always be found in the first nine characters?

[Chris Cairns]

I like your approach Jason. To answer your first question, I (nor does anyone else at my company) actually know what the possible permutations are. We're using this particular textfile as a example of a possible text that would be sent to us from an external data provider. When we begin to test the classes "live", then will have to address the code to any permuations. (A shitty approach if you ask me, but not much I can do.) The delimiters, as Jim suggested, are the spaces in between. Yes, at this point, have to assume each field specifies an exact range.
[ April 01, 2003: Message edited by: Chris Cairns ]
[ April 01, 2003: Message edited by: Chris Cairns ]

[Chris Cairns]

Jim,
I think your approach would be good, but I need to treat the textfile as a String. For example, here is a larger portion of the textfile that's being parsed.

RFP INPUT MESSAGE DATE TIME REF 051845
START OF RFP REF : LQUZ4  -  RG 8922 01 SCEL SAEZ
RG009313
SUBJ RG8922/VQZ-B2
BASED ON 55 PAX AND CARGO.
ETD 2225     ETA 0025
CAPTAIN:FRANCISCO BRAGA 618496
FLT DISPATCHER:WELLINGTON 798546
                         FLIGHT RELEASE
RG 8922 05  SCEL SAEZ  B737-300    PP-VQZ   DEG  3.7PCt REF:LQUZ4
           DIST   FUEL  TIME    ROUTE   CRZ       TOW      ZFW   KGS
TRIP-SAEZ   670    3940 01.31    9313   M74      47030    40560
RES                 310 00.09
ALTN-SUMU   134    1160 00.24                   ......   ......
HOLD               1060 00.30
AFU                   0 00.00  MAX POSS LOAD   TOF .. .. ..
MFR                6470 02.34  ZFW     48300   POB .../...
XFU                   0 00.00  TIME    01.31   ATD  ..:../..:..
TOF                6470 02.34  BRNOFF   4430   TIME       01.31
TXY                 100                        ETA        ..:..
TOTAL              6570                        ATA  ..:.. ..:..

ALT-1 SUMU FL 270 DIST 134NM TIME 00.24  WC P009 FUEL 1160 MFA1  2220
ALT-2 SBPA FL 370 DIST 484NM TIME 01.11  WC P014 FUEL 2780 MFA2  3840
FUEL FOR THIS ALTERNATE IS FOR INFORMATION ONLY
ALT-3 SBFL FL 370 DIST 683NM TIME 01.38  WC P014 FUEL 3700 MFA3  4760
FUEL FOR THIS ALTERNATE IS FOR INFORMATION ONLY

TAKEOFF PERFORMANCE BASED ON
QNH 1013     OAT 30C      RWY 17           FLAP 05      WIND  CLM
A - MAX WEIGHT FOR TAKEOFF
    (LOWEST OF RWY, CLB, PAVEMENT, STRUCTUAL)        59600
B - MAXIMUM RAMP WEIGHT              61462
      TAXI OUT FUEL        MINUS       100           61362
C - MAXIMUM ZERO FUEL WEIGHT         48308
      TAKEOFF FUEL         PLUS       6470           54778
D - MAXIMUM LANDING WEIGHT           51710
      TRIP FUEL            PLUS       3940           55650
E - CFP LIMITED TAKEOFF WEIGHT(ZFW)
      (FUEL TANKAGE)       PLUS

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
DESIT 007 172     CL    21 004 368 365 001 00.01 000130 006340
S33 32.2 W070 46.6                                                  /
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
X-PT  032 172     CL    26 014 368 369 006 00.07 000580 005890
S34 04.2 W070 44.2                                                  /
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
SUPRA 013 062     CL    27 033 368 398 002 00.09 000750 005720
S33 59.2 W070 30.1                                                  /
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ALBAL 036 105     CL    27 052 368 418 005 00.14 001210 005260
S34 11.0 W069 49.0                                                  
WX OBS TIME 1200Z 02/05/03 NAM 0609 AVG W/C P040 AVG/TMP M49 RC LQUZ4

Ignore most of that. Point I'm trying to get at is I'm breaking the text into blocks. Then parsing the blocks. So the block is actually a String, so I have to operate on that. The block I have pasted in my previous post is the one I have to work on.
[ April 01, 2003: Message edited by: Chris Cairns ]
[ April 01, 2003: Message edited by: Chris Cairns ]

[Jason Menard]

Just so I'm sure I understand you, are you saying you have that block as one String? So it would roughly be the equivalent of this?

String text =
        "           DIST   FUEL  TIME    ROUTE   CRZ       TOW      ZFW   KGS\n" +
        "TRIP-SAEZ   670    3940 01.31    9313   M74      47030    40560\n" +
        "RES                 310 00.09\n" +
        "ALTN-SUMU   134    1160 00.24                   ......   ......\n" +
        "HOLD               1060 00.30\n" +
        "AFU                   0 00.00  MAX POSS LOAD   TOF .. .. ..\n" +
        "MFR                6470 02.34  ZFW     48300   POB .../...\n" +
        "XFU                   0 00.00  TIME    01.31   ATD  ..:../..:..\n" +
        "TOF                6470 02.34  BRNOFF   4430   TIME       01.31\n" +
        "TXY                 100                        ETA        ..:..\n" +
        "TOTAL              6570                        ATA  ..:.. ..:..\n";

[ April 01, 2003: Message edited by: Jason Menard ]

[Chris Cairns]

Exactly, Jason.

[Chris Cairns]

I'm thinking that way one to do this is to assume that the fields hold a fixed position. However, let's just say that the distance for 670 for TRIP-SAEZ and DIST is 5,000. I could grab a substring with an index position a few line spaces before, then just trim it. That way, if one of the fields do change, it would be compensated for.

[Jason Menard]

I'm thinking regexes. See if this code points you in the right direction. I might have a couple of different patterns to handle different permutations if it gets too messy with one regex, but hopefully you get the idea.

package regextest;
import java.util.regex.*;
public class RegexTest {
    static String text =
        "           DIST   FUEL  TIME    ROUTE   CRZ       TOW      ZFW   KGS\n" +
        "TRIP-SAEZ   670    3940 01.31    9313   M74      47030    40560\n" +
        "RES                 310 00.09\n" +
        "ALTN-SUMU   134    1160 00.24                   ......   ......\n" +
        "HOLD               1060 00.30\n" +
        "AFU                   0 00.00  MAX POSS LOAD   TOF .. .. ..\n" +
        "MFR                6470 02.34  ZFW     48300   POB .../...\n" +
        "XFU                   0 00.00  TIME    01.31   ATD  ..:../..:..\n" +
        "TOF                6470 02.34  BRNOFF   4430   TIME       01.31\n" +
        "TXY                 100                        ETA        ..:..\n" +
        "TOTAL              6570                        ATA  ..:.. ..:..\n";
    static String recordPattern1 =
        "^" +
        "([A-Z-]{1,9})         # capture first field to $1       \n" +
        "\\s{2,8}              # 2 - 8 spaces                    \n" +
        "(\\d{1,4})?           # capture optional distance to $2 \n" +
        "\\s{3,11}             # 3 - 11 spaces                   \n" +
        "(\\d{1,5})            # capture fuel to $3              \n" +
        ".*";
    public RegexTest() {
    }
    public static void main(String[] args) {
        Pattern pattern = Pattern.compile(recordPattern1, Pattern.COMMENTS);
        String[] records = text.split("\n");
        for (int i = 0; i < records.length; i++) {
            Matcher matcher = pattern.matcher(records[i]);
            if (matcher.matches()) {
                System.out.println(matcher.group(1) + " | " +
                                   matcher.group(2) + " | " +
                                   matcher.group(3));
            }
        }
    }
}

I only worked up to the first three groups, but hopefully it will get you started.

[Jason Menard]

Let me break the regex down a little better in case you are unfamiliar with them:
1: ^
The beginning of the String...
2: ([A-Z-]{1,9})
...followed by 1-9 characters which may each be either capital A through Z, or the '-' character. Capture this sequence to group 1.
3: \\s{2,8}
Followed by 2 - 8 whitespace characters...
4: (\\d{1,4})?
...followed by zero or one occurences of a 1-4 digit sequence, which is captured to group 2.
5: \\s{3,11}
Followed by 3 - 11 whitespace characters...
6: (\\d{1,5})
followed by a 1-5 digit sequence which is captured to group 3.
7: .*
Followed by zero or more occurences of any other character.
HTH
[ April 01, 2003: Message edited by: Jason Menard ]

[Jim Yingst]

The fact that all your input is in one big string doesn't prevent you from using a BufferedReader. You can construct it like this:

    BufferedReader reader = new BufferedReader(new StringReader(inputStr));

You could also use regular expressions for this (as Jason is doing) - if you're familiar with them and/or have time to learn, they're extremely powerful and flexible. But I'm pretty confident you can do this with BufferedReader and StringReader too, if you can understand the format properly.
From your longer file example, it seems as if the biggest problem is not figuring out how to parse the individual lines that have the data you want, but rather, how do you parse just those lines, ignoring the other stuff (which I assume for now that you don't need)? Based on what you've said so far, I might suggest: read lines until you find one that says

           DIST   FUEL  TIME    ROUTE   CRZ       TOW      ZFW   KGS

That indicates the start of data, as far as you're concerned. Now read each subsequent line and try to parse it. If it's null or blank, that indicates the end of the table of useful data, so you can stop reading.
Of course, if the other parts of the file also contain useful data that you need to understand, but in a different format, then you will have to study the format more to decide how to approach it.
Good luck...
[ April 01, 2003: Message edited by: Jim Yingst ]

[Jason Menard]

Even if you are using regexes, my preference would be to do as Jim suggested and read in each line at a time.

[Chris Cairns]

Okay, thanks you guys. I really appreciate it. I have a lot to learn!

[William Barnes]

I say do it in perl!