wood burning stoves 2.0*
The moose likes Threads and Synchronization and the fly likes Pipe-and-Filter Architecture for txt file. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "Pipe-and-Filter Architecture for txt file." Watch "Pipe-and-Filter Architecture for txt file." New topic
Author

Pipe-and-Filter Architecture for txt file.

Sultan Azizi
Greenhorn

Joined: Sep 24, 2012
Posts: 3
Hi everyone,

I hope you are all fine and live happy and safely. This is the first time to post here in the forum, and I feel that there is a nice and variety developers community here. I really appreciate and willing to share with you some of my experience and knowledge. By the way, my English is second language, so if you misunderstand me please let me know.

Currently, I am working in designing and implementing a Filter-and-Pipe architecture for Word document. I stuck with implementing the pipe class. My classes looks like;

Filter class
Pipe class

Class A extends Filter implements Runnable
Class B extends Filter implements Runnable
Class C extends Filter implements Runnable
Class D extends Filter implements Runnable

Class A: read a txt file line by line and SEND it to Class B.
Class B: read the line coming from A, processing it and SEND it to Class C.
Class C: read the line coming from B, processing it and SEND it to Class D.
Class D: read the line coming from C, processing it and print it out.

each of these classes should run in its own thread. Each Filter should wait if there is no data ready. once it ready, should the sender notify the receiver.

I have implement the classes and stuck with the pipe. I don't know how to send the processed line to the next Filter.

Here are what I have done;
The main class create all objects;

When I run the project, I notice that the Class D run and stop before it gets all data to print it out. This is because of the class D check if the queue is empty. If True, it will stop processing and stop the thread.

For example, the txt file has 10 line. Class D will stop after line 4 because the Class C takes time to process line 3 and can not pass data to the queue for class D.

I hope you understand the issue. I need your help if you have any idea.
Anayonkar Shivalkar
Bartender

Joined: Dec 08, 2010
Posts: 1502
    
    5

Hi Sultan Azizi,

Welcome to CodeRanch!

Its good to see that you've used code tags in your very first post.

Now, coming to your issue, you are almost there - you've detected the root cause (D processes data faster than it gets data by C).

So, now, all you have to do is - think about exit point for class D. Currently it is 'if queue is empty'.

Hint: how about 'if all data from file is read'?

On another note, if those operations performed by A,B,C and D are serial, why are you creating 4 threads? I guess it can be done in single class/thread where it will read a line from file, and perform all processes on it serially.

I hope this helps.


Regards,
Anayonkar Shivalkar (SCJP, SCWCD, OCMJD, OCEEJBD)
Sultan Azizi
Greenhorn

Joined: Sep 24, 2012
Posts: 3
Thank you Anayonkar Shivalkar for your fast reply.

Yes, (D processes data faster than it gets data by C)

Hint: how about 'if all data from file is read'?

I do believe it works this way but the requirements is to break the process into different filters to clean up the data from the txt file.

On another note, if those operations performed by A,B,C and D are serial, why are you creating 4 threads? I guess it can be done in single class/thread where it will read a line from file, and perform all processes on it serially.

Yes, the operations performed are serial. The reason I want it to work in 4 thread is to measure the performance of each process in order to come up with a new architecture that has the best and has the low processing time.
Sultan Azizi
Greenhorn

Joined: Sep 24, 2012
Posts: 3
Nobody willing to reply to my post.
Steve Luke
Bartender

Joined: Jan 28, 2003
Posts: 4167
    
  21

Sultan Azizi wrote:... I notice that the Class D run and stop before it gets all data to print it out. This is because of the class D check if the queue is empty. If True, it will stop processing and stop the thread.

Then it would appear that an empty Queue is not the correct signal to use for ending the task. So like Anayonkar said, you will have to come up with a different signal. One approach is the 'Poison Pill'. The Poison Pill is a specific Object used to signal the task is done. When Class A is done reading the file, it puts this Object into the Queue and shuts down. Then when Class B takes something from its incoming Pipe, it checks if it is the Poison Pill. If it is, don't process it, just pass it into the Outgoing Pipe and end the thread. Class C gets it, checks to see if it is the Poison Pill, if it is, puts it into C's Outgoing Pipe and shuts down, and finally, Class D gets the Object it tests against the Poison Pill and if it is shuts down.

Example:


Also, since you know that one thread may have to wait for another thread you would be better off using a BlockingQueue implementation. Without it you will have to do some internal blocking of your own, which may waste processor time (if you just keep checking for the next value), efficiency (if you add an arbitrary sleep), or complexity (if you add locks and conditions).


Steve
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Pipe-and-Filter Architecture for txt file.
 
Similar Threads
How to add in button and paint in applet?
Thread question
Exeptions with Threads
why thread priority is not working here
How to interrupt or stop the threads