wood burning stoves 2.0*
The moose likes JDBC and the fly likes Huge database access Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "Huge database access" Watch "Huge database access" New topic
Author

Huge database access

Arjun Shastry
Ranch Hand

Joined: Mar 13, 2003
Posts: 1874
Hi
We are developing an integration software which basically integrates between one software(which collects alot of data(from various computer nodes in a network) and puts/updates in a database.Our part is to retrieve this data and put it into another Assetmaangement software through API.There is no problem when dataset is small i.e. say 25000 total records.What algorithms or techniques are used while accessing tables containing millions of records?If we use the same,then it will be a probelm with JVM heap size,which occurs many times.We can increase the heap size,but want to know any other approach?
Regards
Capablanca


MH
SJ Adnams
Ranch Hand

Joined: Sep 28, 2001
Posts: 925
outsource to india?
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8836
    
    7

And your Indian oursourcers will probably use a paging technique to get a block of records, process it, then retrieve the next block.


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Arjun Shastry
Ranch Hand

Joined: Mar 13, 2003
Posts: 1874
Well,we don't depend on people and client who are 5000 miles away and conducting video conferences.[This will be a part of project for Indian Railway]
[ December 09, 2003: Message edited by: Capablanca Kepler ]
 
Don't get me started about those stupid light bulbs.
 
subject: Huge database access
 
Similar Threads
garbage collection heap?
How jmap is implemented?
what is the default size of java heap
Heap Size
Heap Size