| Author |
Windows - Cygwin - Hadoop installation - issue with starting tasktracker
|
Kranti Sri
Greenhorn
Joined: Feb 19, 2013
Posts: 28
|
|
I am trying to make Hadoop 1.1.1 work based on the above configuration..
I got through most of the steps. When i run start-all command, jobtracker and namenode start fine.
I have to manually start datanode and it starts fine. (not sure, why I have to manually start though)
Right now i just see an issue with starting task tracker.
Below is the issue. Seems like permissions issue. not sure how to resolve. any help is greatly appreciated.
Thanks in advance.
2013-02-24 22:36:37,738 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException:
Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
|
 |
Kranti Sri
Greenhorn
Joined: Feb 19, 2013
Posts: 28
|
|
Ok, so its a permission issue.
What happened was Cygwin created a user on windows when i installed it. So, I started using that new user for Hadoop. Apparently, the new user created on windows is not an 'Admin' user. I changed the permission and was able to start it manually. However, I am still not sure why I have to start datanode and task tracker manually.
|
 |
Kranti Sri
Greenhorn
Joined: Feb 19, 2013
Posts: 28
|
|
spoke too soon .. still couldnt make task tracker running
Anyone ??
org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
|
 |
Kranti Sri
Greenhorn
Joined: Feb 19, 2013
Posts: 28
|
|
after three days of struggling with different versions..(hadoop 1.1.1 and 0.23.6 .)
I finally got it working with version 1.0.4.
below are links that came handy ..
http://itellity.wordpress.com/2009/12/06/setting-up-hadoop-on-windows-vista/
http://ben-tech.blogspot.com/2011/08/if-java-is-installed-in-cprogram.html
https://github.com/congainc/patch-hadoop_7682-1.0.x-win
Tips :
1.If you are installing Java for the first time, to make things simple do not install in "program files". use a directory with no spaces. if you already have java installed in program files, use sym link to java path and use that path as JAVA_HOME in hdfs-site.xml
2.If start-all.sh command did not initiate datanode and tasktracker, just start those two manually..
bin/hadoop-daemon.sh start tasktracker
bin/hadoop-daemon.sh start datanode
similarly you have to stop them manually..
3. make a folder logs within hadoop home directory and check the logs to make sure everything is OK.
hope it helps saving time for someone else..
i can sleep peacefully tonight..
|
 |
 |
|
|
subject: Windows - Cygwin - Hadoop installation - issue with starting tasktracker
|
|
|