Win a copy of Rust Web Development this week in the Other Languages forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Ron McLeod
  • Liutauras Vilda
  • Jeanne Boyarsky
Sheriffs:
  • Junilu Lacar
  • Rob Spoor
  • Paul Clapham
Saloon Keepers:
  • Tim Holloway
  • Tim Moores
  • Jesse Silverman
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Piet Souris
  • Frits Walraven

Suggested design ideas

 
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I need to make an enhancement in my existing Java application. Its a core java based application, which has two buttons "local" and "central" for search. In the existing application when the user click on "central", a request is sent to mainframe for data. Mainframe returns an XML response which is parsed and local database is populated.

The problem with the existing application is that if mainframe returns many records say 4000-5000 application might die due to out of memory exceptions. What I need to do is to fix this issue by reading response from mainframe in batches, instead of getting at one go.

Can any one please suggest what could be the best optimal design for this?
 
Sheriff
Posts: 26948
83
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't usually think of XML documents as having "records", but I guess your XML is just an XML version of a database table? Anyway 5000 "records" doesn't seem like much at all to me. I would suggest you don't try to optimize the process until you know what there is to optimize.
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
1. how are you parsing the XML returned from the mainframe? As a DOM or using SAX?
2. What is this "local database" you are populating?
3. What kind of lookup do you need from the "local database"

Bill
 
Suhel Sayyed
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Bill and Paul,

Thanks a ton for you inputs.

My application uses DOM parser. Scenario is like, MF will send me an XML request which shall contain say some 'n' number of records. My code will parse these records and populate local database (MS SQL 2005). Suppose if there are 5000 records, MF will send them in batches of 1000 each. After processing one batch I have to send a request to MF to send the next batch and so on.

Problem with this is, entire process should be seamless to the user and has to be done in one transaction. I cannot use pagination in this feature since that would not serve the purpose for my business.

 
Paul Clapham
Sheriff
Posts: 26948
83
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, I don't see a problem with any of that. Sure, go ahead.
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic