File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Web Component Certification (SCWCD/OCPJWCD) and the fly likes how to handle 10,000 records from database Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Certification » Web Component Certification (SCWCD/OCPJWCD)
Bookmark "how to handle 10,000 records from database" Watch "how to handle 10,000 records from database" New topic
Author

how to handle 10,000 records from database

Katerina Karen
Greenhorn

Joined: Oct 06, 2003
Posts: 6
in my web application I write a query that fetches 10,000 records from a remote database.how do i handele it in my web application for optimal performance
Andres Gonzalez
Ranch Hand

Joined: Nov 27, 2001
Posts: 1561
There's a pattern called page-by-page iterator. It describes a solution to a problem like this one.
have a look at this link.


I'm not going to be a Rock Star. I'm going to be a LEGEND! --Freddie Mercury
Steve Agarwal
Ranch Hand

Joined: Feb 02, 2003
Posts: 51
I knew only about View list handler pattern of CORE J2EE.
Steve


SCJP1.4,SCWCD, SCBCD, SCEA part 1<br />"Its feels good to know the stuff in detail."
Jim Doyle
Ranch Hand

Joined: Jul 18, 2003
Posts: 36

It depends. You really need to understand more about databases and
database connectivity before you can answer questions about relative
performance.
Things you need to understand:
1. How is the result set delivered to JVM. In some cases, all 10,000
rows will be transmitted to the JVM. In other cases, blocks of perhaps
4K or 8K at are time are fetched as you iterate through the result set.
This has an impact on local memory utilization.
2. How is the statement prepared and at what isolation level. If
you are reading from the database at any level other than Uncommitted
Read, you are holding down some locks on the database. If you hold the
result set open too long, you may be affecting the performance of other
users of the database as they will busy wait on your row locks, or,
timeout because they reached the waiting limit to acquire row and table
locks. If you are reading from the database as Repeatable Read or Read
Stability, your front end application could adversely affect database
concurrency.
3. Use a scrolling cursor through the result set... Generally, folks
that display large amounts of data like this create alot of heap fragmentation from all of the string assembly that goes on... i.e. all
those TD and TR tags. You can design (or reuse) a class that is artfully
designed to construct HTML table elements from a result set cursor in a
way that is more memory friendly than simply string concatenation. A
certain app server I once used employed this technique - also known as
"scatter-gather" buffers from the Unix OS world, nothing is new under
the sun.
-- Jim (SCJP2, IBM DB2 Associate, Unemployed)
Michael Ernest
High Plains Drifter
Sheriff

Joined: Oct 25, 2000
Posts: 7292

This topic is closed until the topic starter changes his/her display name to match our naming policy.


Make visible what, without you, might perhaps never have been seen.
- Robert Bresson
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: how to handle 10,000 records from database
 
Similar Threads
com.sybase.jdbc2.jdbc.SybSQLException: Too many parameters -- symbol table overflow.
Search Engine
Load testing through a single gateway
landlords not welcome
speed up data saving