aspose file tools*
The moose likes JDBC and the fly likes Huge database access Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "Huge database access" Watch "Huge database access" New topic
Author

Huge database access

Arjun Shastry
Ranch Hand

Joined: Mar 13, 2003
Posts: 1874
Hi
We are developing an integration software which basically integrates between one software(which collects alot of data(from various computer nodes in a network) and puts/updates in a database.Our part is to retrieve this data and put it into another Assetmaangement software through API.There is no problem when dataset is small i.e. say 25000 total records.What algorithms or techniques are used while accessing tables containing millions of records?If we use the same,then it will be a probelm with JVM heap size,which occurs many times.We can increase the heap size,but want to know any other approach?
Regards
Capablanca


MH
SJ Adnams
Ranch Hand

Joined: Sep 28, 2001
Posts: 925
outsource to india?
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8997
    
    9

And your Indian oursourcers will probably use a paging technique to get a block of records, process it, then retrieve the next block.


[How To Ask Questions On JavaRanch]
Arjun Shastry
Ranch Hand

Joined: Mar 13, 2003
Posts: 1874
Well,we don't depend on people and client who are 5000 miles away and conducting video conferences.[This will be a part of project for Indian Railway]
[ December 09, 2003: Message edited by: Capablanca Kepler ]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Huge database access