• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Data Migration

 
vivek makode
Greenhorn
Posts: 27
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
I have a example.csv file (comma separated values) containing millions of records generated from the SAP database.
I want to read this file and insert the same records into Firebird database.
If I use my current jsp based application to read and insert records then after few thousands of records the application terminates with ServletException.
If I use stand alone java program to read and insert the same, it terminates with outofmemory error.
What is the professional and best way to do this? Should I read the file part by part and insert accordinly?
Thanking you in advance

 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 34073
335
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Vivek,
Yes you should read the file in batches and updated a bunch of records at a time. If you commit after each batch, you shouldn't have any problems because it won't be held in memory.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic