aspose file tools*
The moose likes Hadoop and the fly likes Windows - Cygwin - Hadoop installation - issue with starting tasktracker Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Windows - Cygwin - Hadoop installation - issue with starting tasktracker" Watch "Windows - Cygwin - Hadoop installation - issue with starting tasktracker" New topic
Author

Windows - Cygwin - Hadoop installation - issue with starting tasktracker

Kranti Sri
Ranch Hand

Joined: Feb 19, 2013
Posts: 31

I am trying to make Hadoop 1.1.1 work based on the above configuration..
I got through most of the steps. When i run start-all command, jobtracker and namenode start fine.
I have to manually start datanode and it starts fine. (not sure, why I have to manually start though)

Right now i just see an issue with starting task tracker.
Below is the issue. Seems like permissions issue. not sure how to resolve. any help is greatly appreciated.

Thanks in advance.

2013-02-24 22:36:37,738 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException:
Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
Kranti Sri
Ranch Hand

Joined: Feb 19, 2013
Posts: 31

Ok, so its a permission issue.

What happened was Cygwin created a user on windows when i installed it. So, I started using that new user for Hadoop. Apparently, the new user created on windows is not an 'Admin' user. I changed the permission and was able to start it manually. However, I am still not sure why I have to start datanode and task tracker manually.
Kranti Sri
Ranch Hand

Joined: Feb 19, 2013
Posts: 31

spoke too soon .. still couldnt make task tracker running

Anyone ??

org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
Kranti Sri
Ranch Hand

Joined: Feb 19, 2013
Posts: 31

after three days of struggling with different versions..(hadoop 1.1.1 and 0.23.6 .)
I finally got it working with version 1.0.4.

below are links that came handy ..
http://itellity.wordpress.com/2009/12/06/setting-up-hadoop-on-windows-vista/
http://ben-tech.blogspot.com/2011/08/if-java-is-installed-in-cprogram.html
https://github.com/congainc/patch-hadoop_7682-1.0.x-win

Tips :
1.If you are installing Java for the first time, to make things simple do not install in "program files". use a directory with no spaces. if you already have java installed in program files, use sym link to java path and use that path as JAVA_HOME in hdfs-site.xml
2.If start-all.sh command did not initiate datanode and tasktracker, just start those two manually..
bin/hadoop-daemon.sh start tasktracker
bin/hadoop-daemon.sh start datanode
similarly you have to stop them manually..
3. make a folder logs within hadoop home directory and check the logs to make sure everything is OK.

hope it helps saving time for someone else..

i can sleep peacefully tonight..


 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Windows - Cygwin - Hadoop installation - issue with starting tasktracker