• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Object Stream for large volume of data

 
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Friends,

I have to write a service between DB and other applications.

This service is need to read the database and provide the stream of objects.
There will be millions of records in the DB.

I thought of below given approach.
I need to convert the records into Objects stream (Is is possible to do soo??!).
This stream will be send acros other applications which uses this object for further processing (like converting into flat file etc).

Please suggest me. How to impplement it.

Thanks in advance,
Bala
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I would not even attempt to do this with ObjectStream - instead I would attempt to represent each data item in some compact text format, for example XML. There are plenty of tools for converting between XML and Java objects.

Advantages:
1. You can test with text files that can be edited, passed around easily, saved as test cases with documentation as XML comments.
2. On the client side you can use well understood SAX or STaX parsing of XML
3. Clients don't have to have the same version of Java -OR- even written in Java, ever so flexible.

You may find my survey article on "pipeline" style processing of data to be helpful. Here is the link for part 2 of that article.

If you went with ObjectStream you would have to create a new connection for each object (or possibly collection of objects) and close it to finish the transmission correctly. With a text stream formatted as XML things are much more flexible.

Bill
 
Bala Thiruppathy
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks a lot for you genuine answer
reply
    Bookmark Topic Watch Topic
  • New Topic