wood burning stoves 2.0*
The moose likes Threads and Synchronization and the fly likes Inter thread communication - Using PipedStreams Vs  ConcurrentQueus Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "Inter thread communication - Using PipedStreams Vs  ConcurrentQueus" Watch "Inter thread communication - Using PipedStreams Vs  ConcurrentQueus" New topic
Author

Inter thread communication - Using PipedStreams Vs ConcurrentQueus

om ganesh
Greenhorn

Joined: Dec 06, 2006
Posts: 18
Hi I a Java Newbi,

am playing around with Threads for some time.

Right now I am trying to come with a small app where in there will be multiple Producers generating JavaObjects and a single consumer reading these Objects.

example.
2 Producers generating objects and pushing them into a Pipe or Queue
1 Consumer reading the produced object from this Pipe or Queue.

For this which would be good. Should I go for PipedStreams or a Concurrent Queue??

I need my java app to have a smaller memory foot print & run faster.

which of the two would be good ? please give me some explainations or some pointers ( advantages & disadvantages)

Thanks in advance,
Omganesh
Steve Luke
Bartender

Joined: Jan 28, 2003
Posts: 4167
    
  21

There are a lot of differences between the two.

For starters, what kind of data do you want to transport? Complex objects will be easier to transport between threads using a Queue, rather than a Pipe. With the Pipes you would have to serialize the objects or transform them in some way to get them through the Stream, then reverse the procedure on the receiving side. With Queues you just put the object in the Queue and pull it out on the consumer.

Another: A PipedInputStream (consumer) can only connect to a single PipedOutputStream (producer), and it wouldn't be safe to share the PipedOutputStream with multiple threads without further synchronization. So on the consumer you would need 2 PipedInputStreams, one connecting to each producer's PipedOutputStream. In your situation of 2 producers -> 1 consumer, a thread safe Queue implementation will be easier. Both producers and the consumer can use the same Queue. But that may not be exactly what you want.

A Queue may be easier for the general purpose but if you have pre-written code optimized around stream communication, then perhaps the Pipe connection would be easier to implement - specifically if the data is all primitives or Strings (where PipedReader/Writer could be used).

Which will be faster is hard to predict. If you can get both working safely the only way to make the speed comparison would be to test. My guess is that the performance factor will be not be worth comparing though. Either using Streams to communicate or using Queues to collate data will be the better design decision.


Steve
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18531
    
  40

There is also another issue with piped streams -- apparently, the stream keeps track of the last reader and last writer thread. If you write to a pipe and the last reader thread is terminated, you will get a broken pipe exception. And you will continue to get this exception til another reader reads from the pipe.

Its actually a bit silly, but something to look out for, nonetheless.

Henry


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
om ganesh
Greenhorn

Joined: Dec 06, 2006
Posts: 18
Thanks Henry & Steve for the response..

Steve's question
what kind of data do you want to transport?
I am reading Objects from a data source and then passing them onto the Queue / Pipe.

I would now try out the ConcurrentLinkedQueue & see how it goes.


was playing around ... with the PipedStream..

Faced the Scenario which Henry had mentioned .

2 Producer threads are using the same PipedOutputStream reference but the pipe is broken if any one of the Producer terminates before the other

So the Producer thread which finished first would have to be running until the Producer 2 thread too has finished.

One more question.

If i go ahead with the PipedOutputStream, can data over 1MB be stored in it until it is read by the Consumer ?
om ganesh
Greenhorn

Joined: Dec 06, 2006
Posts: 18
Just tried with ConcurrentLinkedQueue,

the sample program ran smooth.. without much ...

here is the code .


 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Inter thread communication - Using PipedStreams Vs ConcurrentQueus
 
Similar Threads
JMS request/ Response Model using MDB.
producer-multiple consumers
PipedStreams fail for FTP upload a XML file
synchronous blocking queue n producer m consumers
queue issue