• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Design considerations

 
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello!
I need som thoughts on some design choises, regarding a project I am to undertake.
I'm going to build a web site and I want to use all open source technology and tools to develop it. What sort of Http server should I go for and what sort of Database system? I need it all to be stable and reliable.
The project will not be very complex, but I need to write a content publisher for the site so i need some serverside programming. I therefore also need a webserver that can handle jsp and servlets.
From what I have found out: To use appache http server, with tomcat as jsp and servlet engine, and
the mySQL database, it would be all open source and perhaps a good solution.
But I do not have to much experience with any of these products, do you have any pros and cons, any general thoughts? The choise of database is what I am most uncertain about.
Thanks
 
Ranch Hand
Posts: 61
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
taking all softwares as open source is a good idea , but in certain cases we have to comprimise, like in database part, mysql wont have as many features that oracle have for further productivity.
i suggest u to consider the future issues and choose the open sources. no doubt apache and tomcat is good combination, but apache and weblogic kind of thing is also a feasible solutlion.
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What do you need the database for?
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for reply
I undestand from you that you are not to exitet about using mySql in a procuction environment.
Why not? What will i miss from this database, I will not build a heavy database, but again it must be stable and reliable, so therfore I also see why you mention it. Anyone with comments on this, are there other stable RDBM out there, open source that I have missed.
Also you suggest WebLogic, this is the commercial application server from BEA, right? I know that it is good, but it might be overkill for my project as I am not planning to build any EJB for example and can do with a servlet and jsp server for now. Of course I will reconsider this for future growth, so thanks again for your comment.
My idea of this is by using j2ee technology, I should anyway be able to move the solution to a different "platform" later.... isn't that what java is all abput
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sorry, I saw that, <<what is the database for>>, to late.
I will use the database for mainly storing some news and technical articles, a time schedule for events, also I will probably be using the database for storing some information about staff at the organization ect.
The main use will be the schedule and articles, which are updated with only 1-10 new articles daily and perhaps new entries in the schedule 3 times a week with only 1-20 events. Not a huge amount of data. But the data must be stored in database for historically use later.
I also am thinking about storing references to images used on the site in the database, is this a good idea, or could I also store the images in the database?
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I suspect a relational database might be overkill for your application - some XML files might totally suffice, for example. You could even use an XML marshalling framework like Castor for this (www.castor.org).
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the www.castor.org link.
It seems realy interesting,but I am not so familiar with xml, so I would be happy if anyone familiar with the castor product could explain briefly low level about how this works.
Is the concept that you use xml files as the "database" and with the castor product you work with this data as if it was a relational database? or have I got it completely wrong? Sorry for not having done a deeper resarch before this message. To many options to litle time -> but that's whats forums all about?!!!
Hopefully more people can benifit of this discussion...
 
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmm, from your description I would really think twice about using a rlational database. Flat files should work wonders for you. For things like name properties files and java.util.Properties are fine, and for articles - nothign beats a flat text file.
Using XML to store static data is silly, imho. You now have to add the full overhead of an XML parser and the programming infrastructure around it to read what is simply text. XML is great if it you need highly structured data that is generated by your app and read by an app, but if it is not highly structured, is mostly flat text... go with flat text.
If you NEED to maintain relational integrity, have the data accessed by other applications, or support a LOT of data that is used heavily then an RDBMS may be the solution, but try using simple object serialization and flat file reads/writes first - it is a lot less complex, a lot more portable, and saves you from having to have a database behind your app.
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
All your replies got me thinking. So I continue this thread with more info to think of, hope you will continue to discuss this with me!!
Sounds like flat files will do great for my purpose, because of the rather small amount of data.
Though, I want the data that I store(the articles and time schedule) to be accessible for years, i.e it should be possible to search for articles and old events in the old time schedule.
Events in the time schedule will have references to old articles.
The time schedule I'am talking about is for holding a list of events which will be displayed on the web site. The list will show monthly events, and the user should be able to view events scheduled one year in advance, and also search for old events on date and title, etc.
The site will, a fair guess, be accessed by about 1000 to 3000 user's a day.
Sorry for holding back these details until now, but at first I didn't expect this great feedback and only wanted some open source thoughts, now this has leaded my to thinking of the technical application design, and You are great Help-for a junior consultant
Should I still be using flat files, or should I consider organizing my data in a RDBM ?
What is the performance and security issues with working with flat files? To do search in flat files after a couple of years with articles sounds inefficient to me......
Please comment, and correct me!
 
"The Hood"
Posts: 8521
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Roger,
Please change your name to be compliant with JavaRanch's naming policy.
Your displayed name should be 2 separate names with more than 1 letter each. We really would prefer that you use your REAL name.
You can change your name: here.
Thanks,
Cindy
 
Greenhorn
Posts: 23
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Have you looked at any of the existing open source content management systems? The hard work will have already been done for you. All you have to do is tweak the appearance of the page templates, and plug in your content.
I'm an OpenACS user. It has packages tailor-made for the needs you mention. It's not Java, but a TCL system that's actually much easier. If you're set on a Java solution, you might try OpenCMS, MMbase, or Redhat's CCM, which is a Java re-implementation of ACS/OpenACS. Zope is another possibility (though not Java either).
If you need new capabilities not offered by the standard packages, you can write them yourself, and offer them back to the community. In doing so, you'll learn from the more experienced programmers -- probably more than if you had gone your own way.
Finally, as far as flat file websites go, they can become a nightmare to manage beyond a couple of hundred pages. Maintaining links, etc., gets really out of hand. An database backed site is really the way to go.
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also thanks for those tips, I will certainly consider these projects, after my first quick glance, openCms seems like the most interesting.
-I am going to use Java
Though it might again be a bit overkill, I saw the requirements for the hardware and to have more than 10 simultaneus user you need a lot of power...
I am also sure I will learn a lot from using this product, but I think building something from scratch will also be challinging, i.e I will meet a lot of "problems" to consider database caching, multiple simultaneus user etc..
This discussion has given me a lot to think of, and I am still open for suggestions.
Does anyone know about other good java open source content managment systems??
 
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Roger,
I have to agree with your first thought which is to use Java and MySQL. I use this combo often and it sounds like a great fit for your project.
Plan your database tables well and you won't have to worry about it in the future. Searching for specific articles will be easy and backing up your data, editing, deleting, adding too will be simple and straightforward.
-Pat
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Pat!
Your comment back up my first thought, since you obvioulsy have some experience with these technologies, do you have any good source for how I should build a simple database app for the web.
I am most concerned with getting a good performance, i.e how do i handle caching of the data from the database in my java objects, how much data can i cache, what to cache etc.
I would also be happy to get some comment on how to build the GUI for the publisher roles for the application. I know now that more people are interessting in this thread so keep up the good discussion!!
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also I would be happy to get some input about performance issues doing a relatively small web-site with java, tomcat and mySql... This is planned to run on a average server with 1Gb RAM 1Mhz 40Gb disk.
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, 1Mhz might in fact be a little bit slow...
In the case you in fact meant 1GHz, I wouldn't really worry about performance, caching etc. to much.
 
Brian McCallister
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you are running a 1ghz machine with a gig of ram... unless you write some really really REALLY ugly code I cannot imagine you having a performance hit with 10 simultaneous users.
Some thoughts on architecture. If this is your first real Java web appand you are doing it to learn, do NOT use the various frameworks like STruts, Velocity, or Tapestry. Implement a minimal framework yourself. I highly suggest an action->controller->redirect based setup like Struts uses. Basically you send any page request that has to do any thinking to a servlet that looks at the request and forwards it to controller object based on the action specified in the URL, the controller gets the Request and Response objects and is responsible for handling it. Typically it will do processing, fill in relevent values on a Helper object which is then added to the request and forwarded to a JSP to make a pretty web page showing data pulled form the Helper. Sun's blue prints specify this as an example, and Struts uses it (ignore all the forms stuff in Struts until you get this concept down, but reading about it will help you there). Keep any application logic in the Controller or in objects it references (ideally in a domain model behind the controller if the app gets big, but for a small app these can be pushed together).
This will get you going for your ten simultaneous users. When you want to scale up, and if you dtermein you NEED a relational database... go use one. It is highly worth your time to write your own Object-Relational mapping system using a Persistence Gateway pattern (have a class for each object you need to persist that knows how to create, load, update, and delete that class) and use that class as a factory to get instances of your persistable classes. After you have done this look into O-R mappers like Hibernate, OJB (my favorite), Castor, or whatnot. When you make your O-R persistence layer you will probably want to investigate caching. Make sure to look at the Jakarta Commons COllections LRUMap class when you do this - it will save you much time and effort.
One thing you will run into while doing this is a cranky database because of lots of connections are being held. If you use POstgres it'll get cranky at about 33 conenctions unless you tweak its default settings (it defaults to a very low number of simultaneous connections) and you may want to investgate connection pooling. Writing a basically functional one isn't difficult, but grabbing the implementations from either JBoss or Jakarta Commons is easy too.
Don't worry about distributed objects (ie, EJB's or RMI) until you profile your system and see that it is efficient and is still dieing under the load. That is a different story, but remember Martin Fowler's first two rules of distributed objetcs: 1) Don't distribute your objects. 2) See rule number 1. It is usually cheaper and easier to buy a 4 cpu RISC server from HP than to pay a developer enough to make a distributed application - and if the 4 cpu machine doesn't handle the load, and the load is from you app, not the relational database then you are writing some cool stuff.
-Brian
 
Brian McCallister
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
followup - the reason I say it is worth writing your own O-R mapping layer is because it is worth it for you to learn what is going on, and undertsand what the various O-R mapping tools out there. They mostly rock, and will save you a ton of effort - but unless you understand what they are doing for you, and have messed around with making a similar system, beware using them blindly.
-Brian
 
Lorus Ghut
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Great, thanks for the informativ comment!
I have started to make myself some sketches, and I think I will follow your suggestion Brian, your ideas confirmed many of my thoughts so fare, and I think I will learn a lot from it.
Even though I am doing this to learn, I will strive for doing this project as good as possible, becuse it certainly will be used...
This application would possible have more than 10 users, my guess is 1000 - 3000 visitors a day, but that is mainly people that will view data, only a couple(2-4) administrators will add, update and modify the data on this application. So I am thinking if manage to do some caching of data, I shouldn't worry to much about the db connections, right?!
Of course I'm not afraid to learn about connection pooling as well
 
Brian McCallister
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
cacheing is always good, but fankly, with 1000-3000 hits per day I very much doubt you will *need* to cache much - particularly as you will be able to run the database locally sinc eyou have a pretty powerful machine (still have to go through sockets, but effectively no latency and much better bandwidth). I suggest making it work first, then make it work fast. Leave yourself open to caching options, iow - don't embed JDBC in your view code - get data through object calls, not database access, but the object calls may very well do database calls every time to get things rolling, but you can change that much easier later.
Good luck!
-Brian
 
Ranch Hand
Posts: 898
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Roger With:
My idea of this is by using j2ee technology, I should anyway be able to move the solution to a different "platform" later.... isn't that what java is all abput


IMHO, it is idea of Java but not of J2EE's practice and J2EE proprietary technologies. Especially if you intend avoiding EJB. Correct me.
 
Ranch Hand
Posts: 388
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi
i think if you use jdbc3.0 compliant driver (there are some for mySQL) then you have connection caching built in.
k
 
reply
    Bookmark Topic Watch Topic
  • New Topic