permaculture playing cards*
The moose likes OO, Patterns, UML and Refactoring and the fly likes Searching for a Design Pattern Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » OO, Patterns, UML and Refactoring
Bookmark "Searching for a Design Pattern" Watch "Searching for a Design Pattern" New topic
Author

Searching for a Design Pattern

don cline
Ranch Hand

Joined: Oct 10, 2003
Posts: 35
I am searching for the correct design pattern for the associated problem.

I have a system that processes, lets say purchase orders. This would then have the normal associated objects for items, purchase orders, customers, etc. etc. The system can have hundreds of thousands to millions of purchase orders.

I need to display information about the purchase orders, for time periods such as the last week, last month, how many P.O. there were, the average size of the P.O., the maximum size of the P.O. and other statistics I will not bore you with.

I am not looking for a solution to this (I can think of many), but the correct design pattern to use. Basically a design pattern that allows the collection of statistics for a very large number of objects, where it is not practical to load all those objects to gain the statistics, but the statistics change with time.

So, anyone to suggest the correct design pattern?
Janeice DelVecchio
Saloon Keeper

Joined: Sep 14, 2009
Posts: 1660
    
  11

Uhmm.... the Model View Controller? Seems like a classic situation where you have data and the view needs to get computed with some functionality..... each portion of the mvc is a simple pattern in itself, like the model is an observer of the controller, and the view is a composite pattern, the controller implements the strategy pattern....

If it's a web app, then the MVC2.


When you do things right, people won't be sure you've done anything at all.
David Newton
Author
Rancher

Joined: Sep 29, 2008
Posts: 12617

I guess I don't really understand the question. What makes you think there's a "design pattern" for this? Isn't it just keeping some statistics, dropping old stats, loading new ones?
Dawn Charangat
Ranch Hand

Joined: Apr 26, 2007
Posts: 249
Well, at the web front, I do agree with Janeice on the applicability of MVC. In the middle tier, ofcourse, since you will not be able to load all the objects together, you can shuttle a "business delegate" to get your data to and fro. In the business layer, however, I am not sure if there is one single design pattern which can satisfy your requirement of continuous data integration and manipulation - You can obviously think of composite ones though, but not sure you'll be lucky enough to find one just addressing this situation.
Jelle Klap
Bartender

Joined: Mar 10, 2008
Posts: 1760
    
    7

When you say you need to display this information, do you mean the system itself should provide a reporting mechanism to display those kinds of statistics on demand?
If not, and what you are actually interested in periodically gathering statistical information used to generate management reports , then you may want to look at data warehousing.
Especially if you're dealing with large amounts of data for which retrieval and analysis could hamper the system's performance when done on-the-fly in a production environment.

don cline
Ranch Hand

Joined: Oct 10, 2003
Posts: 35
Of course we would use the MVC for the web display. This is not really a reporting problem, but statistical data is used interactively on a large number of objects. Although P.O.s are not the real problem, (I'm not allowed to describe the system), but they describe the problem well.

I need to know how many P.O.s we had in the last month, what was the average size, what was the max size, etc. I do not want to load a million objects to get the answer.

My solution is to use a Relational store and get the answer through SQL, but is there an object oriented design pattern for this other than reverting back to SQL.
David Newton
Author
Rancher

Joined: Sep 29, 2008
Posts: 12617

I still don't know what you're looking for. There's no design pattern for "cumulative statistics", and if the data's in a DB, I don't see how you're going to avoid SQL, unless you accumulate the data before it's relational.
Ryan McGuire
Ranch Hand

Joined: Feb 18, 2005
Posts: 1006
    
    3
David Newton wrote:I still don't know what you're looking for. There's no design pattern for "cumulative statistics", and if the data's in a DB, I don't see how you're going to avoid SQL, unless you accumulate the data before it's relational.


In fact, why try to avoid SQL? It would be most efficient to calculate the statistic in a SQL stored procedure. If your database is on a separate server, doing the calculations in a stored proc reduces the amount of data that's being put "on the wire". There is no need to transfer all that data for a million P.O.s over your network if all you need is a max, and average, etc. SQL is designed to do exactly those calculations -- you might as well use it.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16012
    
  19

I'm not a big fan of stored-procedure SQL, as I've seen some really horrible things done with it, but there are certain things that really don't do anyone any good in a client-side process. This is one of them.

Personally, I'd set up a statistics table and a scheduled ("cron") task on the DBMS server. When the schedule indicates, run a backend stored procedure that gathers up the latest stats and adds an entry to the statistics table. You can then access this table for reporting purposes and not only have the benefits of running at regular intervals without having to worry about what's happening on the webserver. but also have the data pre-digested so that interactive processes can retrieve it faster. Since they won't have to calculate it on-demand.


Customer surveys are for companies who didn't pay proper attention to begin with.
Jimmy Clark
Ranch Hand

Joined: Apr 16, 2008
Posts: 2187
Poorly written SQL code written by an individual with poor SQL skills is never good for any application. However, elegantly written SQL code integrated via stored procedures is very good.

Creating dependencies on "hard-coded" CRON scripts typically is never a good solution, in my experience.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16012
    
  19

Frank Bennett wrote:Poorly written SQL code written by an individual with poor SQL skills is never good for any application. However, elegantly written SQL code integrated via stored procedures is very good.

Creating dependencies on "hard-coded" CRON scripts typically is never a good solution, in my experience.


Writing half an app in SQL and half in Java is a maintenance nightmare. Plus, the SQL parts rarely end up under source code control.

I used "cron" in the generic sense. A lot of DBMS's have built--in schedulers, which I recommend, since that puts them under the DBA's master "control panel". Very few shops don't have at least something that has to run on a regular schedule. Backups, I hope, at an absolute minumum.
David Newton
Author
Rancher

Joined: Sep 29, 2008
Posts: 12617

Frank Bennett wrote:Poorly written SQL code written by an individual with poor SQL skills is never good for any application. However, elegantly written SQL code integrated via stored procedures is very good.

We'll skip the stored procs v. no stored procs debate--I've consistently run in to issues with business logic wrapped up in stored procs making duplication mandatory, and keeping the Java (etc) and SQL code in sync is a PITA.
Creating dependencies on "hard-coded" CRON scripts typically is never a good solution, in my experience.

That, however, doesn't make any sense to me.
Jimmy Clark
Ranch Hand

Joined: Apr 16, 2008
Posts: 2187
Data processing logic can be efficiently coded using SQL and invoked by Java-based business logic code via stored procedures. However, to do this effectively and efficiently it takes certain skill levels. The separation of data processing logic and business logic is paramount to getting it "done right."

Advanced SQL programming and advanced object-oriented Java programming are different skill sets. When two individuals with the proper level of both skills get together, then things will run smoothly. Or, when an individual has both skill sets him/herself, and can either execute or guide development teams in the right direction, things will run smotthly. It is the absence of either one or both of these abilities that makes things problematic. It isn't the technology, it is the people.

Individuals consistently put the blame on technologies, rather than deficiencies in their abilities and/or knowledge.

It is easier for me to say, "stored procedures are crappy", than "I don't know how to write efficient stored procedures." lol
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16012
    
  19

I know how to write stored procedures. In about 10 different languages for 4 different DBMS's. There are other reasons for using them judiciously than because "they're crappy". It's going to cost a customer of mine a considerable sum of money because the source of the program I'm working on originally had most of its business logic written in T-SQL and the new servers aren't going to be running in a Windows environment. Ever tried pricing SQL Server for Linux?

It has already cost them a lot of money because when they shipped the source code half of it wasn't in the source code. It was in the database. I had to get them to unload a copy of the database and ship that to me, as they weren't exactly SQL Server experts who could pare it down to essentials first. One of the first things I did was extract all the stored procedures and put them in a "sql" directory in the project (next to the schema definitions) so that they'd be under source control and hence, if I in turn shipped out a copy of the project, the source code would actually be capable of reproducing the project from scratch.

And that is nothing to laugh about.
Jimmy Clark
Ranch Hand

Joined: Apr 16, 2008
Posts: 2187
Transact-SQL stored procedures are cool!
David Newton
Author
Rancher

Joined: Sep 29, 2008
Posts: 12617

Frank Bennett wrote:When two individuals with the proper level of both skills get together, then things will run smoothly.

But that simply isn't always the case, because there are other factors involved than just having two skilled developers.
It is easier for me to say, "stored procedures are crappy", than "I don't know how to write efficient stored procedures."

I'll assume you're not casting aspersions on my skills, here, but it certainly sounds like it.

The bottom line is that having distributed technology stacks will, by definition, always be more complicated than keeping it in a single location--no matter how skilled (or not) the developers are. And in my experience, splitting up the responsibilities has almost always led to fairly major issues throughout the life of projects. Occasionally? I've used stored procs to great effect--on relatively limited occasions.

YMMV--but making a blanket statements, neglecting other mitigating factors, is deceptive.
Ryan McGuire
Ranch Hand

Joined: Feb 18, 2005
Posts: 1006
    
    3
Tim Holloway wrote:I know how to write stored procedures. In about 10 different languages for 4 different DBMS's. There are other reasons for using them judiciously than because "they're crappy". It's going to cost a customer of mine a considerable sum of money because the source of the program I'm working on originally had most of its business logic written in T-SQL and the new servers aren't going to be running in a Windows environment. Ever tried pricing SQL Server for Linux?

It has already cost them a lot of money because when they shipped the source code half of it wasn't in the source code. It was in the database. I had to get them to unload a copy of the database and ship that to me, as they weren't exactly SQL Server experts who could pare it down to essentials first. One of the first things I did was extract all the stored procedures and put them in a "sql" directory in the project (next to the schema definitions) so that they'd be under source control and hence, if I in turn shipped out a copy of the project, the source code would actually be capable of reproducing the project from scratch.

And that is nothing to laugh about.


Yes, but...

The fact that the project did not have the source for the stored procedures in source code control is not an issue with the underlying technology -- it's a project management issue. Yes, I know... it would be great if all the development tools, including SQL Server Management Studio (or whatever was used to develop the stored procs to begin with) integrated seamlessly with the SCC. Since it doesn't, it was the responsibility of the developer(s) to take those steps manually and of the project manager to make sure they did.

At the very least, they could have zipped up a DB backup and included that in SCC. That would have included all the stored procs, views, initial data, table definitions, and at least an idea of the required permission settings.

Oh well, you work with what you have.

David Newton
Author
Rancher

Joined: Sep 29, 2008
Posts: 12617

We're getting off-track; if we want to debate the merits, or lack thereof, of stored procs, SCCSs, etc. let's do it in their respective forums.
Ranganathan Kaliyur Mannar
Bartender

Joined: Oct 16, 2003
Posts: 1076
    
  10

I get a feeling that Composite or Decorator will help you...


Ranga.
SCJP 1.4, OCMJEA/SCEA 5.0.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Searching for a Design Pattern