• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Hadoop work for Small database of about 1-2 GB

 
Ranch Hand
Posts: 42
1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Guys !
I am new In Hadoop. But I heard lot abut hadoop.
My Manager want is to use in a project. So I check Hadoop and I find very interesting.

But I want to know that Will Hadoop work for Small database of about 1-2 GB.

Right know we all are using MYSql Database.

Will hadoop work for us ?
Please Reply. Thanks in advance.
 
clojure forum advocate
Posts: 3479
Mac Objective C Clojure
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It will works but 1~2 GB is really a small amount of data.
Hadoop shines when the dataset it -really, really- big.
Check your requirements and write a prototype to see whether Hadoop is going to be helpful or not.
You might find that a hand-crafted solution is more vital (and less memory-stresser) than Hadoop.
 
Bartender
Posts: 2407
36
Scala Python Oracle Postgres Database Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You can certainly run Hadoop with small amounts of data on a single machine e.g. to explore how it works. But Hadoop and map-reduce in general are really designed for "big data" problems that are suitable for this kind of parallel batch processing across lots of machines e.g. bit ETL processes,or analysing huge amounts of data, etc. In your example, 2GB is not "Big Data" - 2GB will fit in RAM on most laptops these days, and you could store more data than that on your phone! - so as Hussein says, a normal database might be a better solution. Also, you need to think about what you want to do with your data, as map-reduce might not be particularly relevant for your needs if you are doing lots of OLTP or serving data to busy websites, for example.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic