File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Hadoop and the fly likes Hadoop and multicore Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Hadoop and multicore" Watch "Hadoop and multicore" New topic

Hadoop and multicore

Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
is there a tuning parameters or configurations for Hadoop to utilize the underlying hardware capabilities (like multicore, processing threading etc ) or it is totally agnostic and abstracted from the underlying hardware?


Best Regards, Mohamed El-Refaey
Srinath Hperera

Joined: Mar 19, 2013
Posts: 3
You can control the number of tasks per task tracker via property and mapred.tasktracker.reduce.tasks.maximum . If your machines has more core, you can give a happy number. Recommendation is to give number between 1XCores and 2XCores.
Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
Thanks a lot Srinath!
Gartner says :Bigdata will be most advanced analytics products by 2015 !

Time to Become Big data architect by learning Hadoop(Developer, Administration,Analyst,QA),Cassandra,MongoDb,HBase,Datascience, Mahout, Splunk,R etc) from scratch to expert level
subject: Hadoop and multicore