File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
Win a copy of Clojure in Action this week in the Clojure forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Any Insight on how JavaRanch scales

 
m lathe
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Guys,
I was just wondering if the JavaRanch team could provide some insight to how JForum is deployed to scale on this site.
You appear to be the largest implementation of JForum so some pointers would be appreciated.

The gist from what i understand on the JForum site is that it doesn't scale horizontally (ie you can't distribute one instance across several boxes)
http://jforum.net/posts/list/4898.page
http://jforum.net/posts/list/2553.page

Rafael basically says that it scales on one box and the best we could is run a cold backup connected to the common DB. Nor does it look like offloading reads to a replicated DB are allowed.

One solution that is posed is that in theory connecting several instances with a common cache (ie ehcache with multicast messages to advertise dirty caches) should work. However this doesn't appear to be supported.

Any information would be helpful.
Thanks
--Matthias
 
paul wheaton
Trailboss
Pie
Posts: 21183
Firefox Browser IntelliJ IDE Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Others will be able to provide more accurate data, but I think we are currently using about 5% of our CPU and we are serving about 6.5 million page views per month. And this is with caching turned off.

I've been thinking that if there was a need to run on more than one box, it would be wise to divide the forums: some forums would be on one box and the rest on a second box. But I don't see that happening.
 
m lathe
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the reply.

So the application runs on one box (presumably commodity hardware) and the DB on another box (presumably MySQL on commodity hardware)?

Do you run a cold/warm standby?

We have also considered running parallel instances, where some boards are hosted on one silo of boxes (ie app and DB) and others are hosted on another silo. This is in effect just a "poor man's" sharding. Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?

Thanks again
--Matthias
 
paul wheaton
Trailboss
Pie
Posts: 21183
Firefox Browser IntelliJ IDE Java
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Who are you making forums for that you would need something bigger than what we have here?
 
m lathe
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's for a customer website for a company. You're probably correct that the load will not be as large as what is found here. However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site and also how to deal with crashes without having to start up a cold backup.

I guess the lesson here is that even the big installs of JForum only deploy one instance. No one seems to attempt to cluster several instances with a cache.

Thanks
 
Ernest Friedman-Hill
author and iconoclast
Marshal
Pie
Posts: 24204
34
Chrome Eclipse IDE Mac OS X
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We do have a completely separate install that we use as a test server, so after developers test locally, we deploy to the test server and bang on that. But I'll agree that JForum isn't really designed for five-nines operation.

 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 33694
316
Eclipse IDE Java VI Editor
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
m lathe wrote: Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?

I would imagine you would still have one database and just multiple app servers.

m lathe wrote:However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site

Who tests changes in production?

m lathe wrote: also how to deal with crashes without having to start up a cold backup.

The software doesn't crash. The only time we are down is for deployments. Which in a real company would be scheduled. If you mean the box crashing, you can have the software on another box and route the web server to it. Of course, starting up another instance of Tomcat doesn't take long.
 
m lathe
Greenhorn
Posts: 4
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Jeanne, your comments are apt. A suitably fast DB (worst case Oracle RAC) would probably allow allow you to run multiple app servers. That being said there might be some issues with how JForum writes to the DB that count on only one app running. One example might be to queue writes in the app layer and periodically flush them to the DB (ie view counts). I'm not sure if it does this, but it might.

Jeanne Boyarsky wrote:
Who tests changes in production?


Everyone should test in production, but you are right it shouldn't be the first time you test the change. If you are trying to get 100% uptime you really don't want to release a product to your customer until it was verified in production. So while you are doing functional testing in the test environments, you don't want to just slap a war into production and walk away. Ideally you have two nodes, you take one away from the customer, roll your change, verify, add it back and do the same to the second node.

Thanks everyone.

Please holler if you have any pointers for how to cluster JForum.
 
Jeanne Boyarsky
author & internet detective
Marshal
Posts: 33694
316
Eclipse IDE Java VI Editor
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You are correct that JForum caches things in a way that prevents clustering. I meant have an app server upp that nobody can access just in case something happens to the first one.

I don't see a forum as being so mission critical though. I work for a bank and we are still allowed X hours of downtime a year.
 
Ulf Dittmer
Rancher
Pie
Posts: 42966
73
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
JForum supports several cache implementations, though, one of them being EhCache, which is billed as a "distributed cache". So you should be able to configure it to handle caches in multiple JVMs: http://ehcache.org/documentation/distributed_caching.html

That is probably easier to set up -and, IMO, architecturally cleaner- than having separate servers handle separate forums (which by itself wouldn't solve all caching issues).
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic