aspose file tools*
The moose likes JDBC and the fly likes What is the best way to store List<List<String>> in database? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "What is the best way to store List<List<String>> in database?" Watch "What is the best way to store List<List<String>> in database?" New topic
Author

What is the best way to store List<List<String>> in database?

Gaurav Kr. Arora
Ranch Hand

Joined: Feb 20, 2011
Posts: 37
Hi,

Requirement: To store List<List<String>> in database. Here String represents the value fetched from excel file and List<String> represents a row containing cells in excel.
List<List<String>> represents all the records from a excel file.
There can be lacs of records in the excel data file.

Now, the data of this file has to be persisted into database without performance overhead. I have stored data of the file in List<List<String>> object. Table structure in which data needs to be stored is same as that of excel data file.
We can do a batch update for insert queries but since data size is more, I don't find it efficient.

If we can do this using some other technique like JPA or Hibernate or directly saving the file in database table with some kind of mapping between file columns and table columns, please let me know.

Thanks in advance.
chris webster
Bartender

Joined: Mar 01, 2009
Posts: 1712
    
  14

The "best" way? Depends on your data and your database.

With Oracle, the fastest and most flexible way to do this with significant volumes of data is probably by loading the data directly from a CSV file into the database using an external table.

The general approach would be:

  • set up an Oracle directory definition in the database (one-off task)
  • copy your CSV file to the physical directory corresponding to the Oracle directory definition
  • create your external table
  • copy the data from the external table to a "real" table

  • Once you've set this up, it will be way faster than any row/cell-based process you can come up with in Java or Hibernate (e.g. I loaded 2.5 million rows of 60 columns on a slow machine in about 30 seconds the other day). The external table will end up with the same basic structure as your CSV file (the CSV header row will appear in the first row of the table if you don't exclude it from the loading process), but it's usually better to copy the data into a "real" table because it's more flexible e.g. you can add more data later, and it allows more performance optimisation e.g. use of indexes etc.

    But if you're not using Oracle, or you can't use a CSV file, you'll need some advice from a Java expert instead.

    Good luck!


    No more Blub for me, thank you, Vicar.
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: What is the best way to store List<List<String>> in database?