• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

High Traffic website

 
Ranch Hand
Posts: 34
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi all,

we are expecting a project which has high number of concurrent requests. The application will have 5 million registered users and approximate number of users online at a time is around 100,000. The business logic is not complex. But handling the traffic is our problem. Can any one let me know the strategies in implementing such a web site.

Thanks,
Dinakar.K
 
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Bring money.

For that many concurrent users, you'll need load balancers, clustered servers, redundant resources and almost certainly some very powerful database servers. You'll need infrastructure such as climate-controlled computer rooms (and it's a good idea to have several geographically separate sites), UPS's and power conditioners. You'll need good Internet connections and you'll need tools that can monitor for system degradation and outages. You'll need a good Business Continuity plan and someone to administer it. You'll need trained operations personnel. You'll need security staff - both for physical security and data security.

This is the era of Cloud Computing and outsourcing, so you have the option of renting out a lot of these things from third parties. However, if you do, bear in mind that you'll then be hostage to your outsourcers for critical resources. If they decide you're not interesting enough or if another customer outbids you for service levels, you could suffer. Outsource solution providers will never have the financial or emotional stake in the product that your own company will. Nor will they have the overall expertise in the product, hopefully. Otherwise they could always decide that you're completely redundant and may end up becoming your biggest competitor.
 
Dinakar Kas
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim,

By clustered servers, do you mean clustered application servers or clustered database servers or both? Can you explain what do you mean by redundant resources? and can you give me names of "very powerful database servers"?

Also, the number of users that I have mentioned will not be using the site from day one. It's only that the web site should scale to those many users as time goes on.

Can you please give me suggestions with the technology selection?

Thanks,
Dinakar.K
 
Bartender
Posts: 10336
Hibernate Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'd second Tim's first suggestion.

With regards to clustered servers, given the very large volumes you are talking about I'd say both. The usual big players are the ones most people look at, e.g. Oracle RAC on pretty meaty hardware.
 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Dinakar Kas wrote:Thanks Tim,

By clustered servers, do you mean clustered application servers or clustered database servers or both? Can you explain what do you mean by redundant resources? and can you give me names of "very powerful database servers"?

Also, the number of users that I have mentioned will not be using the site from day one. It's only that the web site should scale to those many users as time goes on.

Can you please give me suggestions with the technology selection?

Thanks,
Dinakar.K



Both. Redundant resources means more than one rack, preferably in more than one location. A hotsite is also a good options in case of failure.

As far as "very powerful" Database servers go, IBM will quite happily sell you a set of z systems. Oracle has a hardware line, which has undoubtedly further enhanced by their acquisition of Sun. I believe HP has a set of boxes designed to host databases. The usual suspects, in other words.

Blade servers are popular for web hosting.

But do budget for electricity. Some racks can pull 24 kW or more, and that's as much as a whole neighbourhood of houses typically uses.

More specific I can't be. It depends on a number of factors, and it's not exactly something I can do for free.
 
Bartender
Posts: 2700
IntelliJ IDE Opera
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I would also get (a lot of) people involved that know what they're talking about, who have done this before.
 
Dinakar Kas
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Paul, Tim, and Wouter,

I would also get (a lot of) people involved that know what they're talking about, who have done this before.



That is what I am trying to do.

More specific I can't be. It depends on a number of factors, and it's not exactly something I can do for free.





Just one question more,

Can I start the project using traditional Java Technologies such as a web framework, EJB/JPA combination, and Lucene for some of the searches in the documents. Will they be able to scale as I add more hardware?

Really appreciate the help.

Thanks,
Dinakar.K

 
Tim Holloway
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
As a platform, Java often gets criticized because it's expensive to develop for. Unlike the trendy scripting-based platforms so much in vogue at the moment, you can't just "Git 'R Dun!", you have to engage in a certain amount of planning and setup.

However, the payback comes in security, reliability and performance. Scalability has always been a major consideration in the design of Enterprise Java technologies.
 
Dinakar Kas
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Tim,

Thanks a lot. I think I got some answers and I will do some research before we zero-in on some thing.

Wonderful experience talking to you all.

Dinakar.K
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic