aspose file tools
The moose likes Hadoop and the fly likes Hadoop and multicore Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Hadoop and multicore" Watch "Hadoop and multicore" New topic

Hadoop and multicore

Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
is there a tuning parameters or configurations for Hadoop to utilize the underlying hardware capabilities (like multicore, processing threading etc ) or it is totally agnostic and abstracted from the underlying hardware?


Best Regards, Mohamed El-Refaey
Srinath Hperera

Joined: Mar 19, 2013
Posts: 3
You can control the number of tasks per task tracker via property and mapred.tasktracker.reduce.tasks.maximum . If your machines has more core, you can give a happy number. Recommendation is to give number between 1XCores and 2XCores.
Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
Thanks a lot Srinath!
Gartner says :Bigdata will be most advanced analytics products by 2015 !

Time to Become Big data architect by learning Hadoop(Developer, Administration,Analyst,QA),Cassandra,MongoDb,HBase,Datascience, Mahout, Splunk,R etc) from scratch to expert level
subject: Hadoop and multicore