• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

How to execute a simple program in hadoop cluster 2.6

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a Linux,Unix and a windows machine having installed hadoop 2.6 in each of them.Soon I will configure them to be in cluster so that Linux will be master node and other one or both OS will be slave.

I have been unable to execute even simple word count program in standalone mode on Linux machine.

I wish to execute any simple program in cluster.

I would be grateful if someone provide me sample program and exact steps (or pointer to appropriate resources) to execute it successfully in my cluster.

My configuration :

Hadoop 2.6
Linux 14.04 (Master)
Mac OS X 10.x (One slave)
Windows 10 (another slave)
 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Anup Wadhekar wrote:I have been unable to execute even simple word count program in standalone mode on Linux machine.


Have you worked through the single node setup tutorial? What problems did you face?

I feel it's better to get the single node working and familiarizing with it before jumping to cluster, because there are many more things that can go wrong in cluster.
Install single node in a VirtualBox VM if you want to keep your physical machines sanitized for the eventual cluster setup.
 
Anup Wadhekar
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Karthik.. thanks for the reply.

I have successfully configured hadoop 2.6 in standalone mode and pseudo distributed mode on Linux,Mac OS X and windows 10 and I can see all required daemons using jps command.

configuration doesn't seem to be problem for me so far.

I tried once executing a sample word count program in standalone linux but couldn't execute anyhow.. don't remember the exact error/problem though.

I will again execute it and will post the exact problem.

thanks again.
 
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Anup,

Try the following simple wordcount program to test your environment -
1. Please start the hadoop services - start-dfs.sh
2. Please start the yarn services - start-yarn.sh
3. Create a input directory
$ hadoop fs -mkdir /user/hadoop/input
-- If it will give you error that directory not found-Create the parent directory first
4. Upload the input file
$ hadoop fs -put $HADOOP_INSTALL/LICENSE.txt /user/hadoop/input/License
5. Use the hadoop example jar available with hadoop installation package to run the code
$ hadoop jar $HADOOP_INSTALL/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount /user/hadoop/input/License /home/hadoop/Output
Note:- Define the input file and output directory based on your system/OS
reply
    Bookmark Topic Watch Topic
  • New Topic