Object Stream for large volume of data

Hi Friends,

I have to write a service between DB and other applications.

This service is need to read the database and provide the stream of objects.
There will be millions of records in the DB.

I thought of below given approach.
I need to convert the records into Objects stream (Is is possible to do soo??!).
This stream will be send acros other applications which uses this object for further processing (like converting into flat file etc).

Please suggest me. How to impplement it.

Thanks in advance,
Bala

I would not even attempt to do this with ObjectStream - instead I would attempt to represent each data item in some compact text format, for example XML. There are plenty of tools for converting between XML and Java objects.

Advantages:
1. You can test with text files that can be edited, passed around easily, saved as test cases with documentation as XML comments.
2. On the client side you can use well understood SAX or STaX parsing of XML
3. Clients don't have to have the same version of Java -OR- even written in Java, ever so flexible.

You may find my survey article on "pipeline" style processing of data to be helpful. Here is the link for part 2 of that article.

If you went with ObjectStream you would have to create a new connection for each object (or possibly collection of objects) and close it to finish the transmission correctly. With a text stream formatted as XML things are much more flexible.

Bill

Thanks a lot for you genuine answer