Win a copy of 97 Things Every Java Programmer Should Know this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Jeanne Boyarsky
  • Junilu Lacar
  • Henry Wong
Sheriffs:
  • Ron McLeod
  • Devaka Cooray
  • Tim Cooke
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Frits Walraven
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • salvin francis
  • fred rosenberger

Storm Applied: Deploying Storm on popular infrastructure services

 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Sean, Matthew and Peter!

As an independent developer with average sysadmin skills, one problem I've faced while exploring big data products is deploying them cost effectively at modest scale on infra services like AWS and Linode.
Often, the steps to do this the right way are not there, and discovering them becomes a matter of (costly) trial and error.

Does your book cover *deploying* Storm at scale on such services? What are your opinions on its ease of deployment?
 
Author
Posts: 14
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We don't cover any specific service such as AWS etc, we do however go through how to deploy Storm.

Deploying Storm is relatively easy. We strongly advise that you use a tool like Puppet to allow for recreateable deploys.
From there, you can easily deploy new nodes to the Cluster and what not.

My #1 piece of advise if you are setting up a cluster is to pay close attention to Zookeeper. Its a vital part of a Storm cluster.
Make sure that you are running Zookeeper machines with fast disk as its IO intensive and a lack of stability with Zookeeper
will impact on your entire cluster.
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you for the reply and the ZK tip! It's good to know Storm is fairly easy to deploy. Planning to try it out.

Wishing you great success with your book promotions!
 
The knights of nee want a shrubbery. And a tiny ad:
ScroogeXHTML 9.0 - RTF to HTML5 and XHTML converter
https://coderanch.com/t/731949/ScroogeXHTML-RTF-HTML-XHTML-converter
    Bookmark Topic Watch Topic
  • New Topic