Get the tools you need to learn Java skills fast!
Video tutorials, eBooks, hands-on lab exercises, sample code.
Get started
The moose likes JDBC and Relational Databases and the fly likes Data Migration Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of The Software Craftsman this week in the Agile forum!
JavaRanch » Java Forums » Databases » JDBC and Relational Databases
Bookmark "Data Migration" Watch "Data Migration" New topic

Data Migration

vivek makode

Joined: Oct 01, 2001
Posts: 27
I have a example.csv file (comma separated values) containing millions of records generated from the SAP database.
I want to read this file and insert the same records into Firebird database.
If I use my current jsp based application to read and insert records then after few thousands of records the application terminates with ServletException.
If I use stand alone java program to read and insert the same, it terminates with outofmemory error.
What is the professional and best way to do this? Should I read the file part by part and insert accordinly?
Thanking you in advance

Jeanne Boyarsky
author & internet detective

Joined: May 26, 2003
Posts: 31680

Yes you should read the file in batches and updated a bunch of records at a time. If you commit after each batch, you shouldn't have any problems because it won't be held in memory.

[OCA 8 book] [Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Have you tried LearnNowOnline?
subject: Data Migration