Win a copy of Re-engineering Legacy Software this week in the Refactoring forum
or Docker in Action in the Cloud/Virtualization forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Huge database access

 
Arjun Shastry
Ranch Hand
Posts: 1898
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi
We are developing an integration software which basically integrates between one software(which collects alot of data(from various computer nodes in a network) and puts/updates in a database.Our part is to retrieve this data and put it into another Assetmaangement software through API.There is no problem when dataset is small i.e. say 25000 total records.What algorithms or techniques are used while accessing tables containing millions of records?If we use the same,then it will be a probelm with JVM heap size,which occurs many times.We can increase the heap size,but want to know any other approach?
Regards
Capablanca
 
SJ Adnams
Ranch Hand
Posts: 925
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
outsource to india?
 
Joe Ess
Bartender
Pie
Posts: 9259
10
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And your Indian oursourcers will probably use a paging technique to get a block of records, process it, then retrieve the next block.
 
Arjun Shastry
Ranch Hand
Posts: 1898
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well,we don't depend on people and client who are 5000 miles away and conducting video conferences.[This will be a part of project for Indian Railway]
[ December 09, 2003: Message edited by: Capablanca Kepler ]
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic