The moose likes JDBC and Relational Databases and the fly likes Bulk Data Processing Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC and Relational Databases
Bookmark "Bulk Data Processing" Watch "Bulk Data Processing" New topic

Bulk Data Processing

Prateek Jauhari

Joined: Mar 07, 2011
Posts: 26

I am trying to process bulk amount of data....
I am using IBM DB2.

I am reading a list of Item from an Excel using POI interface and for each Item there is a particular set of queries that has to be fired.
The problem is that the if I am running the same set of query for each item then result processing is taking to much of time
(i.e. if the items increases to 50 or above) and ultimately causing the browser to display Request time out error.

Is there to any other way to implement the same with procedure or something ?

Thanks & Regards
Jeanne Boyarsky
author & internet detective

Joined: May 26, 2003
Posts: 33124

Do you have to process one at a time? If not, you can batch the queries so there are less server round trips (and less work for the database.)

[OCA 8 book] [Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Other Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, TOGAF part 1 and part 2
Prateek Jauhari

Joined: Mar 07, 2011
Posts: 26

Like I have a List
Such as
and so on
now after reading 1001 we fire a set of queries,
and then for 1002 the same set of queries,
and then for 1003..
and so on...
Mayank Pande

Joined: Apr 03, 2013
Posts: 1
The reason I raised this here because I'm using hibernate and wanted to check if there is a better option in Spring to do so.

Data Entry
I agree. Here's the link:
subject: Bulk Data Processing
It's not a secret anymore!