This week's book giveaway is in the Jobs Discussion forum.
We're giving away four copies of Java Interview Guide and have Anthony DePalma on-line!
See this thread for details.
The moose likes JDBC and Relational Databases and the fly likes Data Migration Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Java Interview Guide this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Databases » JDBC and Relational Databases
Bookmark "Data Migration" Watch "Data Migration" New topic

Data Migration

vivek makode

Joined: Oct 01, 2001
Posts: 27
I have a example.csv file (comma separated values) containing millions of records generated from the SAP database.
I want to read this file and insert the same records into Firebird database.
If I use my current jsp based application to read and insert records then after few thousands of records the application terminates with ServletException.
If I use stand alone java program to read and insert the same, it terminates with outofmemory error.
What is the professional and best way to do this? Should I read the file part by part and insert accordinly?
Thanking you in advance

Jeanne Boyarsky
author & internet detective

Joined: May 26, 2003
Posts: 33132

Yes you should read the file in batches and updated a bunch of records at a time. If you commit after each batch, you shouldn't have any problems because it won't be held in memory.

[OCA 8 book] [Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Other Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, TOGAF part 1 and part 2
I agree. Here's the link:
subject: Data Migration
jQuery in Action, 3rd edition