| Author |
how to handle 10,000 records from database
|
Katerina Karen
Greenhorn
Joined: Oct 06, 2003
Posts: 6
|
|
|
in my web application I write a query that fetches 10,000 records from a remote database.how do i handele it in my web application for optimal performance
|
 |
Andres Gonzalez
Ranch Hand
Joined: Nov 27, 2001
Posts: 1561
|
|
There's a pattern called page-by-page iterator. It describes a solution to a problem like this one. have a look at this link.
|
I'm not going to be a Rock Star. I'm going to be a LEGEND! --Freddie Mercury
|
 |
Steve Agarwal
Ranch Hand
Joined: Feb 02, 2003
Posts: 51
|
|
I knew only about View list handler pattern of CORE J2EE. Steve
|
SCJP1.4,SCWCD, SCBCD, SCEA part 1<br />"Its feels good to know the stuff in detail."
|
 |
Jim Doyle
Ranch Hand
Joined: Jul 18, 2003
Posts: 36
|
|
It depends. You really need to understand more about databases and database connectivity before you can answer questions about relative performance. Things you need to understand: 1. How is the result set delivered to JVM. In some cases, all 10,000 rows will be transmitted to the JVM. In other cases, blocks of perhaps 4K or 8K at are time are fetched as you iterate through the result set. This has an impact on local memory utilization. 2. How is the statement prepared and at what isolation level. If you are reading from the database at any level other than Uncommitted Read, you are holding down some locks on the database. If you hold the result set open too long, you may be affecting the performance of other users of the database as they will busy wait on your row locks, or, timeout because they reached the waiting limit to acquire row and table locks. If you are reading from the database as Repeatable Read or Read Stability, your front end application could adversely affect database concurrency. 3. Use a scrolling cursor through the result set... Generally, folks that display large amounts of data like this create alot of heap fragmentation from all of the string assembly that goes on... i.e. all those TD and TR tags. You can design (or reuse) a class that is artfully designed to construct HTML table elements from a result set cursor in a way that is more memory friendly than simply string concatenation. A certain app server I once used employed this technique - also known as "scatter-gather" buffers from the Unix OS world, nothing is new under the sun. -- Jim (SCJP2, IBM DB2 Associate, Unemployed)
|
 |
Michael Ernest
High Plains Drifter
Sheriff
Joined: Oct 25, 2000
Posts: 7292
|
|
|
This topic is closed until the topic starter changes his/her display name to match our naming policy.
|
Make visible what, without you, might perhaps never have been seen.
- Robert Bresson
|
 |
 |
|
|
subject: how to handle 10,000 records from database
|
|
|