• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

verify hadoop installation

 
danthuluri pavan kumar
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am very new to Hadoop. I am trying to run few sample prog. in Standalone (or local) mode. I have installed Cygwin in my windows machine. I am trying to run the command “hadoop version” to verify whether hadoop installed successfully or not. I have downloaded hadoop-2.2.0 version.



I have downloaded hadoop : C:\Users\Pavan\Downloads\

I tried to setup the PATH variable in Cygwin as below


$ echo $JAVA_HOME
C:\Program Files\Java\jdk1.6.0_32
$ export HADOOP_INSTALL=/cygdrive/c/Users/Pavan/Downloads/hadoop
$ export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin

I am trying to run the command “hadoop version” and end up with the following error. Please help me in resolving this error.

$ hadoop version
java.lang.NoClassDefFoundError: org/apache/hadoop/util/VersionInfo
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.VersionInfo
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.VersionInfo. Program will exit.
Exception in thread "main"
 
Marcos Serrano
Ranch Hand
Posts: 46
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
mm maybe your environment is not defined properly,

I share my environment defined in my cygwin user's .bashrc file [CYGWIN environment]

export JAVA_HOME=/cygdrive/c/Java/jdk1.6.0_23/
export JRE_HOME=/cygdrive/c/Java/jre6
export HADOOP_INSTALL=/home/saivulle/apps/hadoop-1.2.1
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$PATH

then from cygwin command window

$ hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /C:/cygwin/home/saivulle/apps/hadoop-1.2.1/hadoop-core-1.2.1.jar
 
chris webster
Bartender
Posts: 2407
32
Linux Oracle Postgres Database Python Scala
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you are a complete beginner, I would strongly recommend you try one of the pre-configured Hadoop virtual machines (VMs) available from various suppliers. Hadoop is really horrible to install and configure, especially if you want to try using some of the related tools like Pig, Hive, etc. There are lots of compatibility problems between different versions of the various libraries, and it's generally a huge PITA to get working the first time you try it.

If your main goal is to learn how to install/administer Hadoop, then fine, install it yourself. But if your initial aim is to find out how to use Hadoop to do interesting things with Big Data, make life easy on yourself by using a pre-configured installation.

Cloudera has a Quickstart VM and Hortonworks has its developer sandbox. Both of these are Linux-based VMs with all the main Hadoop-related tools pre-installed and configured correctly, and the VMs can be downloaded for free (the files are pretty big) and run inside the (free) VirtualBox virtualisation environment.

Hortonworks sandbox comes with a lot of built-in tutorials and gives you a browser-based interface to upload files to HDFS, try out Hadoop processing with Pig, Hive etc.

Cloudera's VM is used as the basis for the free Udacity course Introduction to Hadoop and MapReduce, which is aimed at complete beginners and will take you through the basics of what Hadoop does. Choose the "free courseware" option when you sign up for the course.
 
danthuluri pavan kumar
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Thanks all for you response. I recently found the below information and thought might be useful to other hadoop Enthusiasts.

As of the last release, Hadoop 2.2 supports Windows natively. Thus, old workarounds like Cygwin are now un-supported. Please refer to the new documentation mentioned below (and copied here) for latest instructions for Hadoop on Windows.

https://wiki.apache.org/hadoop/Hadoop2OnWindows

If you are running Hadoop 1, then please check out branch-1-win and use that for running Hadoop 1 natively on Windows.


Now I am exploring hadoop-1.2.1 on windows using (cygwin + eclipse europa) , can anyone guide/help me in finding the information on eclipse plugin ?
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic