Extracting information from a HDF5 file and placing it in a file for querying

[Tom Storm]

Hi there,

I have a number of HDF5 files and would like to extract information from them and then dump it into a format that I can then query. I was wondering if a CSV file would be suitable. Would I be able to select rows 1 to 10000 and analyse variable number x from each row in this format?

Questions
1) What is the best way in Java to extract information from a HDF5 file and dump it into another format.
2) Which format would allow me to easily select variables from a number of rows.

I found the HDF5 website very unhelpful for Java and the code I am currently using to work with the HDF5 files is extremely inefficient.

Thanks to all who reply,

TS

[Paul Clapham]

I see that HDF5 has two different Java interfaces. I don't know anything about either of them but I would start by investigating them.

As for what data format to use so that you can query it, I would suggest a database. SQL is designed to be queried.

[Tom Storm]

I had a look at SQL but received a number of errors when using the code I have to extract the information from the HDF file. However I have found CSV files and have another little issue. I receive an arrayoutofbounds exception with the current code. What am I doing incorrectly?

import com.opencsv.CSVReader;

import java.io.*;
import java.util.LinkedList;
import java.util.List;

class Storing{

    String number1;
    String number2;


    public Storing(String number1, String number2){
        this.number1 = number1;
        this.number2 = number2;
    }

}

public class ReadCSV {

    String tailID;
    String arrivalDelay;
    List<Storing> list = new LinkedList<Storing>();

    public static void main(String[] args) {

        ReadCSV obj = new ReadCSV();
        obj.run();

    }

    public void run() {

        try {
            CSVReader reader = new CSVReader(new FileReader("F:\\Downloads\\combined.csv"));
            String [] nextLine;

            while ((nextLine = reader.readNext()) != null) {
                tailID = nextLine[9];
                arrivalDelay = nextLine[32];
                list.add(new Storing(tailID,arrivalDelay));
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }


        System.out.println("Done");
    }

}

Thanks for the earlier response.

TS

[Richard Tookey]

Tom Storm wrote:I had a look at SQL but received a number of errors when using the code I have to extract the information from the HDF file. However I have found CSV files and have another little issue. I receive an arrayoutofbounds exception with the current code. What am I doing incorrectly?

I'm with Paul on this and believe you should be using JDBC/SQL.

There are two obvious places in your code (nextLine[9] and nextLine[32]) that could result in an ArrayOutOfBounds exception and you will be able to tell if you print the stack trace associated with the exception.

[Tom Storm]

I managed to fix the exception by looping through files in a directory rather than combining them into a single cvs.

Thank you for your replies.

TS