File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Web Component Certification (SCWCD/OCPJWCD) and the fly likes how to handle 10,000 records from database Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Java Interview Guide this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Certification » Web Component Certification (SCWCD/OCPJWCD)
Bookmark "how to handle 10,000 records from database" Watch "how to handle 10,000 records from database" New topic

how to handle 10,000 records from database

Katerina Karen

Joined: Oct 06, 2003
Posts: 6
in my web application I write a query that fetches 10,000 records from a remote do i handele it in my web application for optimal performance
Andres Gonzalez
Ranch Hand

Joined: Nov 27, 2001
Posts: 1561
There's a pattern called page-by-page iterator. It describes a solution to a problem like this one.
have a look at this link.

I'm not going to be a Rock Star. I'm going to be a LEGEND! --Freddie Mercury
Steve Agarwal
Ranch Hand

Joined: Feb 02, 2003
Posts: 51
I knew only about View list handler pattern of CORE J2EE.

SCJP1.4,SCWCD, SCBCD, SCEA part 1<br />"Its feels good to know the stuff in detail."
Jim Doyle
Ranch Hand

Joined: Jul 18, 2003
Posts: 36

It depends. You really need to understand more about databases and
database connectivity before you can answer questions about relative
Things you need to understand:
1. How is the result set delivered to JVM. In some cases, all 10,000
rows will be transmitted to the JVM. In other cases, blocks of perhaps
4K or 8K at are time are fetched as you iterate through the result set.
This has an impact on local memory utilization.
2. How is the statement prepared and at what isolation level. If
you are reading from the database at any level other than Uncommitted
Read, you are holding down some locks on the database. If you hold the
result set open too long, you may be affecting the performance of other
users of the database as they will busy wait on your row locks, or,
timeout because they reached the waiting limit to acquire row and table
locks. If you are reading from the database as Repeatable Read or Read
Stability, your front end application could adversely affect database
3. Use a scrolling cursor through the result set... Generally, folks
that display large amounts of data like this create alot of heap fragmentation from all of the string assembly that goes on... i.e. all
those TD and TR tags. You can design (or reuse) a class that is artfully
designed to construct HTML table elements from a result set cursor in a
way that is more memory friendly than simply string concatenation. A
certain app server I once used employed this technique - also known as
"scatter-gather" buffers from the Unix OS world, nothing is new under
the sun.
-- Jim (SCJP2, IBM DB2 Associate, Unemployed)
Michael Ernest
High Plains Drifter

Joined: Oct 25, 2000
Posts: 7292

This topic is closed until the topic starter changes his/her display name to match our naming policy.

Make visible what, without you, might perhaps never have been seen.
- Robert Bresson
I agree. Here's the link:
subject: how to handle 10,000 records from database
It's not a secret anymore!