aspose file tools*
The moose likes Ranch Office and the fly likes Any Insight on how JavaRanch scales Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » This Site » Ranch Office
Bookmark "Any Insight on how JavaRanch scales" Watch "Any Insight on how JavaRanch scales" New topic
Author

Any Insight on how JavaRanch scales

m lathe
Greenhorn

Joined: Nov 30, 2009
Posts: 4
Hi Guys,
I was just wondering if the JavaRanch team could provide some insight to how JForum is deployed to scale on this site.
You appear to be the largest implementation of JForum so some pointers would be appreciated.

The gist from what i understand on the JForum site is that it doesn't scale horizontally (ie you can't distribute one instance across several boxes)
http://jforum.net/posts/list/4898.page
http://jforum.net/posts/list/2553.page

Rafael basically says that it scales on one box and the best we could is run a cold backup connected to the common DB. Nor does it look like offloading reads to a replicated DB are allowed.

One solution that is posed is that in theory connecting several instances with a common cache (ie ehcache with multicast messages to advertise dirty caches) should work. However this doesn't appear to be supported.

Any information would be helpful.
Thanks
--Matthias
paul wheaton
Trailboss

Joined: Dec 14, 1998
Posts: 20639
    ∞

Others will be able to provide more accurate data, but I think we are currently using about 5% of our CPU and we are serving about 6.5 million page views per month. And this is with caching turned off.

I've been thinking that if there was a need to run on more than one box, it would be wise to divide the forums: some forums would be on one box and the rest on a second box. But I don't see that happening.

permaculture Wood Burning Stoves 2.0 - 4-DVD set
m lathe
Greenhorn

Joined: Nov 30, 2009
Posts: 4
Thanks for the reply.

So the application runs on one box (presumably commodity hardware) and the DB on another box (presumably MySQL on commodity hardware)?

Do you run a cold/warm standby?

We have also considered running parallel instances, where some boards are hosted on one silo of boxes (ie app and DB) and others are hosted on another silo. This is in effect just a "poor man's" sharding. Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?

Thanks again
--Matthias
paul wheaton
Trailboss

Joined: Dec 14, 1998
Posts: 20639
    ∞

Who are you making forums for that you would need something bigger than what we have here?
m lathe
Greenhorn

Joined: Nov 30, 2009
Posts: 4
It's for a customer website for a company. You're probably correct that the load will not be as large as what is found here. However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site and also how to deal with crashes without having to start up a cold backup.

I guess the lesson here is that even the big installs of JForum only deploy one instance. No one seems to attempt to cluster several instances with a cache.

Thanks
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24187
    
  34

We do have a completely separate install that we use as a test server, so after developers test locally, we deploy to the test server and bang on that. But I'll agree that JForum isn't really designed for five-nines operation.


[Jess in Action][AskingGoodQuestions]
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 30767
    
156

m lathe wrote: Did you consider what would happen for shared resources (notably post counts etc?) would you just sync this in the background somehow?

I would imagine you would still have one database and just multiple app servers.

m lathe wrote:However I think it's more of a question of how do you build in redundancy so you can roll and test changes without bringing down the site

Who tests changes in production?

m lathe wrote: also how to deal with crashes without having to start up a cold backup.

The software doesn't crash. The only time we are down is for deployments. Which in a real company would be scheduled. If you mean the box crashing, you can have the software on another box and route the web server to it. Of course, starting up another instance of Tomcat doesn't take long.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
m lathe
Greenhorn

Joined: Nov 30, 2009
Posts: 4
Jeanne, your comments are apt. A suitably fast DB (worst case Oracle RAC) would probably allow allow you to run multiple app servers. That being said there might be some issues with how JForum writes to the DB that count on only one app running. One example might be to queue writes in the app layer and periodically flush them to the DB (ie view counts). I'm not sure if it does this, but it might.

Jeanne Boyarsky wrote:
Who tests changes in production?


Everyone should test in production, but you are right it shouldn't be the first time you test the change. If you are trying to get 100% uptime you really don't want to release a product to your customer until it was verified in production. So while you are doing functional testing in the test environments, you don't want to just slap a war into production and walk away. Ideally you have two nodes, you take one away from the customer, roll your change, verify, add it back and do the same to the second node.

Thanks everyone.

Please holler if you have any pointers for how to cluster JForum.
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 30767
    
156

You are correct that JForum caches things in a way that prevents clustering. I meant have an app server upp that nobody can access just in case something happens to the first one.

I don't see a forum as being so mission critical though. I work for a bank and we are still allowed X hours of downtime a year.
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42289
    
  64
JForum supports several cache implementations, though, one of them being EhCache, which is billed as a "distributed cache". So you should be able to configure it to handle caches in multiple JVMs: http://ehcache.org/documentation/distributed_caching.html

That is probably easier to set up -and, IMO, architecturally cleaner- than having separate servers handle separate forums (which by itself wouldn't solve all caching issues).


Ping & DNS - my free Android networking tools app
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Any Insight on how JavaRanch scales