GeeCON Prague 2014*
The moose likes Swing / AWT / SWT and the fly likes Regular updates to table from network messages Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Java » Swing / AWT / SWT
Bookmark "Regular updates to table from network messages" Watch "Regular updates to table from network messages" New topic
Author

Regular updates to table from network messages

Mark Dary
Greenhorn

Joined: Jan 16, 2009
Posts: 29
Hi all

I hope you don't mind, but it's best if I first describe my situation as quickly as I can.

I have a custom message API that manages sending and receiving custom Msg objects through a socket. Some messages are sent as requests from the client (Java) to the server (C), some are sent automatically from the server to the client; most msgs are sent asynchronously, so the client could receive msgs in any order.

Some of the messages represent a block, for example a block of 1000 M1 type messages. The end of each block is marked by a message (e.g. M1) with an empty payload to mark the end of this block (e.g. of M1 messages); however the block of 1000 M1 messages could be interspersed with other message types, e.g. M2, M3, etc.

So far my client handles this well by using a reader class that calls a message handler for each message type; however there is now a new requirement that I need help with as follows.

The new requirement is to receive an additional two blocks of messages, for example M1 messages and B1 messages, and to receive them at regular (configurable) intervals, e.g. every 5 seconds.

I could receive for example 1000 M1 messages, followed by 684 B1 messages, the B1 block will always follow the M1 block but both blocks could be interspersed with other messages (e.g. M2). Neither block is ordered in any way.

The B1 messages are a 1:m map to the M1 messages and the next two blocks could be different sizes, 1200 M1 and 748 B1 messages (for example).

After receiving the two blocks, parts of the M1 and B1 messages need to be joined (on key fields) and converted into rows to be published in a GUI table, other derived calculations (from M1 and B1 fields) must be published in additional columns.

Note I am not sending or receiving database data (although the messages do represent this and cannot be changed), so I cannot send SQL statements or receive row data, only Msg objects.

Could anyone offer advice on how to begin with this, I am not a Java expert so any help is greatly appreciated.

Thanks.

Mark.
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

Hi Mark and welcome to Javaranch,

OK, I'm going to read the requirements back to you to make sure I know what you're after . My understanding is that you can have something like this:

[M1 M3 M1 M1 M2 M2........... M1 M2 M1*]
[B1 B1 M2 M3 B1 B1 ............ B1 B1*]

Where:

[] denotes a message block
* denotes the empty payload end of block character

So your 'Message reader' can be enhanced (you'll probably want a Manager class at this point) to:

1.) Read the M1 block and store that in memory
2.) Read the B1 block and immediately marry it up with the previous M1 block that you have in memory
3.) Do appropriate processing

Is that basically what you're after? Or are you after more low level detail?

The scheduling thing you can do with an Open Source lib called Quartz


Cheers, Martijn - Blog,
Twitter, PCGen, Ikasan, My The Well-Grounded Java Developer book!,
My start-up.
Mark Dary
Greenhorn

Joined: Jan 16, 2009
Posts: 29
First of all Martijn, thank you very much for replying.

I am an experienced engineer/programmer and know some parts of Java quite well, but others I know very little about, I think I have learnt a lot about Java GUIs and Java multithreading lately but some (basic!) parts are still very weak

In short, yes what you said is correct. I am working on a set of GUIs (similar in some respects but very different in others) that are part of a very large application suite. In reality there are several other complicated (concurrent) processing steps in each GUI, but for the current GUI, what you say above is correct.

At the moment I am thinking of using the Swing Application Framework with NetBeans for the current and other GUIs (the previous one was much simpler and hand-coded), but I may not be allowed (corporate!) to use any external libraries

Below is more information if you have the time – and/or motivation to read …

The ‘main’ components are:

a client writer class
a client reader class
an API writer class
an API reader class
an API writer queue (used by the client writer and API writer)
an API reader queue (used by the API reader and client reader)

Each of the reader/writer classes implement Runnable and run (continuously) in their own thread. There are also a couple of other threads that are used for different reasons. The two main queues are LinkedBlockingQueues.

The client takes requests from GUI components, converts them into messages and puts them onto the writer queue using the client writer thread, this thread (client writer) also has to use a SynchronousQueue between the GUI and the writer queue because of other constraints.

The API writer thread takes msgs off the writer queue, extracts the raw bytes and sends them to the server.

Meanwhile, the API reader thread continuously reads raw bytes from the socket (a header has the length), and then creates a new Msg from them which it puts it on the reader queue.

The client reader thread takes msgs off the reader queue and calls the appropriate message handler. In the simple GUI, the message handler simply updates a JTextArea from the message payload.

The GUIs have to be very responsive to change or cancel requests or to perform other (simpler derived calculations) because some of the (other) message blocks may contain hundreds of thousands of messages.

Now for the new requirements, the other GUIs have more complicated (logic) processing and much more complicated GUI components. The main view on the current GUI is a table view, this is what the original post was for. Eventually, the table view (possibly with a certain amount of historical data) must also be displayed as an interactive graph – I’m not even going to think about that one yet!

Did you read this far, wow, thank you again Martijn.

Best regards Mark.
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

OK,

It's certainly an interesting problem you have there! I'll try to split up my response into areas I think you want addressed (let me know if I missed the mark), I'll apologise in advance if I'm babbling on about stuff you already know, in fact I suspect you'll be far more of an expert than I am in most of this.

GUI:
-----

What are the current GUIs written in? I'm going to assume they are written in Swing. Other GUI technologies would probably be just as competent in dealing with the display of data etc as long as it's decoupled from the processing layer.

Disclaimer #1 - I'm not a Swing GUI expert by any means but an open source project I'm on uses a very complex multi-threaded/concurrent GUI and it works just fine in terms of speed. Now that's not too say there wasn't pain in getting the GUI to that point . I'm especially unaware if Swing is very good at say using multiple threads to update multiple rows in a table, that sort of thing.

So I think your choice of building the GUI in Swing on Netbeans is fine. Further down the line you can ask Swing Q's directly in our Swing forum where there are lots of gurus to assist (not me ;p). One thing I'll add straight away is that:

* Yes using Swing App Framework is a good idea.
* Using the very latest version of the JDK/JRE is a good idea. That is if you can build on and have your clients run on JDK/JRE 1.6.0_10+ you're going to get a much nicer experience (radical changes were made in 1.6.0_10 to make desktop Java a bit nicer and faster again).
* Decouple the MessageHandler(s) from the GUI if you haven't done so already.

The whole concurrent messaging model thing:
-----------------------------------------------------

Aside #1 - I'd normally prescribe a healthy dose of JMS (which would deal with some of your queuing and concurrency issues), but typically that involves introducing an application server and I don't think you want to go down that path .

Right now onto the heart of the matter! I think what you've described sounds just fine up to:


The client reader thread takes msgs off the reader queue and calls the appropriate message handler. In the simple GUI, the message handler simply updates a JTextArea from the message payload.


As you say with the new complex GUI you can no longer have the simple MessageHandler model. I can only suggest something like (same as before really):

* Have the M1 MessageHandler simply store the M1 block (storage type should be swappable) but no longer call the GUI to update.
* Have the B1 MessageHandler retrieve the M1 block hand off to another class to do the merge with M1 processing and then eventually take the result of that and update the GUI (again I'm not sure how efficiently you can update various Swing components).

Naturally the above is simplistic and you'll have all sorts of Manager classes, an interface or two and maybe a storage solution . Oh and you'll need to deal with two M1 blocks arriving before a B1 block, that sort of thing!

The important bit?:
---------------------

I _think_ an important concept you might have to introduce is a Admin or Thread Manager interface on some of this stuff as you want to be able to cancel threads at the drop of the hat (the GUIs have to be very responsive), obviously the stop threads would have to be given the higher priority.

I'm not sure if you 'stop' a unit of work whether you lose it all or whether you've got everything wrapped in transactions....

The graph:
-------------

I think there are some 3rd party APIs for that sort of thing, JFreeChart is one to look into..

Well I hope that helped in some small way, it certainly sounds like you'll be busy!
Mark Dary
Greenhorn

Joined: Jan 16, 2009
Posts: 29
in fact I suspect you'll be far more of an expert than I am in most of this

I doubt it! Some parts of Java I know reasonably well, other (even basic parts) I've never used.

I'm going to assume they are written in Swing.

They don't exist yet, but I am writing them in Swing.

I'm especially unaware if Swing is very good at say using multiple threads to update multiple rows in a table, that sort of thing.

Yes and no, some parts of Swing are thread safe, most is not. The biggest problem (I believe) has always been synchronizing GUI updates from long running background processes.

Further down the line you can ask Swing Q's directly in our Swing forum

That sounds like a good idea as I think my main issue is how to effectively use a JTable, do you know if this thread can be forwarded to them?

Using the very latest version of the JDK/JRE is a good idea

The current standard is 1.5, however by the time I (hopefully!) release, the corporate standard will be 1.6.

Decouple the MessageHandler(s) from the GUI if you haven't done so already

I can't really do this, the message handlers exist purely to update the GUI.

I designed it so that the client reader and client writer can call a process method of the message handler (base class or extended) to do what ever processing is required for each message; and to then update the GUI with any results. If you want, I can give you more details but it would take a few lines.

I'd normally prescribe a healthy dose of JMS

I would have to agree, however it is out of the question in my current (corporate!) environment.

As you say with the new complex GUI you can no longer have the simple MessageHandler model. I can only suggest something like (same as before really)

As I noted above, the message handler model does well so far. I already handle other message types that come in hundreds or thousands of messages in a block, all message processing is handled by the default (base class) process method, or overriden methods in subclasses.

I think by posting and replying to your points it is obvious what my problem is, I don't know how to use TJables (in general), and specifically for my particular situation, although your suggestion has certainly given me a few ideas to play with.

I _think_ an important concept you might have to introduce is a Admin or Thread Manager interface

I already designed and implemented a thread manager, however I now realise it may be better to use an Executor (or the background management methods in the Swing Application Framework) - oh well live and learn

I think there are some 3rd party APIs for that sort of thing, JFreeChart is one to look into..

That's great, if you (or anyone else) can offer more suggestions for charting it would be greatly appreciated, this part is totally new to new.

Thanks again Martijn.

Mark.
Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

I'm going to move this to the Swing forum so you can hopefully get those specific Q's answered there
Mark Dary
Greenhorn

Joined: Jan 16, 2009
Posts: 29
Thanks for all your help Martijn.

Mark.
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42047
    
  64
JFreeChart is very capable for static charts, but if there will be "live" updates to the chart data, I think JChart2D would be a better choice. JFreeChart doesn't handle updates.


Ping & DNS - my free Android networking tools app
Mark Dary
Greenhorn

Joined: Jan 16, 2009
Posts: 29
Thanks Ulf, the chart has to be updated regularly so JChart2D sounds better.

Could anyone offer help with my JTable problem, maybe with a pseudo code exampe?

Here is a summary, I receive a variable number of messages in a block, followed by another variable number of messages in a second block.

The second block is a one to many mapping to the first block. I need to join the blocks on a key field and then display in a JTable.

The sequence then repeats (with various caveats described above) at regular intervals.

The first block of messages define the number of rows but this can change at each interval, the JTable should only show the rows it receives in each interval.

Thanks.

Mark.


Martijn Verburg
author
Bartender

Joined: Jun 24, 2003
Posts: 3274
    
    5

Not a code sample, but a article talking about a similar problem?
 
GeeCON Prague 2014
 
subject: Regular updates to table from network messages