• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Paul Clapham
  • Jeanne Boyarsky
  • Knute Snortum
Sheriffs:
  • Liutauras Vilda
  • Tim Cooke
  • Junilu Lacar
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Joe Ess
  • salvin francis
  • fred rosenberger

Windows - Cygwin - Hadoop installation - issue with starting tasktracker

 
Ranch Hand
Posts: 35
Hibernate Chrome Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am trying to make Hadoop 1.1.1 work based on the above configuration..
I got through most of the steps. When i run start-all command, jobtracker and namenode start fine.
I have to manually start datanode and it starts fine. (not sure, why I have to manually start though)

Right now i just see an issue with starting task tracker.
Below is the issue. Seems like permissions issue. not sure how to resolve. any help is greatly appreciated.

Thanks in advance.

2013-02-24 22:36:37,738 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException:
Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
 
Kranti Sri
Ranch Hand
Posts: 35
Hibernate Chrome Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, so its a permission issue.

What happened was Cygwin created a user on windows when i installed it. So, I started using that new user for Hadoop. Apparently, the new user created on windows is not an 'Admin' user. I changed the permission and was able to start it manually. However, I am still not sure why I have to start datanode and task tracker manually.
 
Kranti Sri
Ranch Hand
Posts: 35
Hibernate Chrome Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
spoke too soon .. still couldnt make task tracker running

Anyone ??

org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: \tmp\hadoop-cyg_server\mapred\local\taskTracker to 0755
at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:689)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:670)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:810)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1557)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3893)
 
Kranti Sri
Ranch Hand
Posts: 35
Hibernate Chrome Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
after three days of struggling with different versions..(hadoop 1.1.1 and 0.23.6 .)
I finally got it working with version 1.0.4.

below are links that came handy ..
http://itellity.wordpress.com/2009/12/06/setting-up-hadoop-on-windows-vista/
http://ben-tech.blogspot.com/2011/08/if-java-is-installed-in-cprogram.html
https://github.com/congainc/patch-hadoop_7682-1.0.x-win

Tips :
1.If you are installing Java for the first time, to make things simple do not install in "program files". use a directory with no spaces. if you already have java installed in program files, use sym link to java path and use that path as JAVA_HOME in hdfs-site.xml
2.If start-all.sh command did not initiate datanode and tasktracker, just start those two manually..
bin/hadoop-daemon.sh start tasktracker
bin/hadoop-daemon.sh start datanode
similarly you have to stop them manually..
3. make a folder logs within hadoop home directory and check the logs to make sure everything is OK.

hope it helps saving time for someone else..

i can sleep peacefully tonight..


 
Think of how dumb the average person is. Mathematically, half of them are EVEN DUMBER. Smart tiny ad:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!