aspose file tools*
The moose likes JDBC and the fly likes Data Migration Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "Data Migration" Watch "Data Migration" New topic

Data Migration

vivek makode

Joined: Oct 01, 2001
Posts: 27
I have a example.csv file (comma separated values) containing millions of records generated from the SAP database.
I want to read this file and insert the same records into Firebird database.
If I use my current jsp based application to read and insert records then after few thousands of records the application terminates with ServletException.
If I use stand alone java program to read and insert the same, it terminates with outofmemory error.
What is the professional and best way to do this? Should I read the file part by part and insert accordinly?
Thanking you in advance

Jeanne Boyarsky
author & internet detective

Joined: May 26, 2003
Posts: 31079

Yes you should read the file in batches and updated a bunch of records at a time. If you commit after each batch, you shouldn't have any problems because it won't be held in memory.

[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
I agree. Here's the link:
subject: Data Migration