File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Hadoop and the fly likes Practical usage of Hadoop Map Reduce,Pig Hive HBase in application. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Practical usage of Hadoop Map Reduce,Pig Hive HBase in application." Watch "Practical usage of Hadoop Map Reduce,Pig Hive HBase in application." New topic
Author

Practical usage of Hadoop Map Reduce,Pig Hive HBase in application.

Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 586
Hi am learning Hadoop.I read Hadoop Tutorial ,Map Reduce concepts,Hive Pig did small programs of each....Now I want to know how these hadoop technologies are used in real applications...I am confused.
Mark Spritzler
ranger
Sheriff

Joined: Feb 05, 2001
Posts: 17249
    
    6

Monica. Shiralkar wrote:Hi am learning Hadoop.I read Hadoop Tutorial ,Map Reduce concepts,Hive Pig did small programs of each....Now I want to know how these hadoop technologies are used in real applications...I am confused.


We will use Google as an example. Google uses Map Reduce when you do a search.

Think about how much data Google has of websites and indexes. When you search, you get results real quick. Because they break it apart to many small pieces running the same code over a smaller subset of data in their entire farm.

So Imagine 1000s of servers all with data on it. How would you query for the search string Java. The data could be a little on each of those 1000s of machines. So instead of running the query against each server and copy all the data over. something like Hadoop copies the code to all the machines have them run on each machine on subsets of data with multi cores, multi cpus and multi threads. Then gather data, reduce it to a smallest chunk, then pass the chunk to a combiner that then comes back to the user with results.

You couldn't do that with a simple database on a simple single machine. It is all about huge amounts of data.

Mark


Perfect World Programming, LLC - Two Laptop Bag - Tube Organizer
How to Ask Questions the Smart Way FAQ
Monica. Shiralkar
Ranch Hand

Joined: Jul 07, 2012
Posts: 586
Thanks Mark.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
 
subject: Practical usage of Hadoop Map Reduce,Pig Hive HBase in application.
 
Similar Threads
Hadoop: confusion between Hive tables and HBase tables..
What is in Hadoop map reduce which was not in Google Map Reduce?
Mapreduce using Java
How much Hadoop knowledge should one have before diving into Pig?
setting number of reducers in the map reduce jobs converted from PIG script