aspose file tools*
The moose likes XML and Related Technologies and the fly likes Suggested design ideas Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "Suggested design ideas" Watch "Suggested design ideas" New topic
Author

Suggested design ideas

Suhel Sayyed
Greenhorn

Joined: Nov 03, 2011
Posts: 8
Hi,

I need to make an enhancement in my existing Java application. Its a core java based application, which has two buttons "local" and "central" for search. In the existing application when the user click on "central", a request is sent to mainframe for data. Mainframe returns an XML response which is parsed and local database is populated.

The problem with the existing application is that if mainframe returns many records say 4000-5000 application might die due to out of memory exceptions. What I need to do is to fix this issue by reading response from mainframe in batches, instead of getting at one go.

Can any one please suggest what could be the best optimal design for this?
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18907
    
    8

I don't usually think of XML documents as having "records", but I guess your XML is just an XML version of a database table? Anyway 5000 "records" doesn't seem like much at all to me. I would suggest you don't try to optimize the process until you know what there is to optimize.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12825
    
    5
1. how are you parsing the XML returned from the mainframe? As a DOM or using SAX?
2. What is this "local database" you are populating?
3. What kind of lookup do you need from the "local database"

Bill
Suhel Sayyed
Greenhorn

Joined: Nov 03, 2011
Posts: 8
Hi Bill and Paul,

Thanks a ton for you inputs.

My application uses DOM parser. Scenario is like, MF will send me an XML request which shall contain say some 'n' number of records. My code will parse these records and populate local database (MS SQL 2005). Suppose if there are 5000 records, MF will send them in batches of 1000 each. After processing one batch I have to send a request to MF to send the next batch and so on.

Problem with this is, entire process should be seamless to the user and has to be done in one transaction. I cannot use pagination in this feature since that would not serve the purpose for my business.

Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18907
    
    8

Well, I don't see a problem with any of that. Sure, go ahead.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Suggested design ideas