aspose file tools*
The moose likes Hadoop and the fly likes verify hadoop installation Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "verify hadoop installation" Watch "verify hadoop installation" New topic
Author

verify hadoop installation

danthuluri pavan kumar
Greenhorn

Joined: Sep 25, 2008
Posts: 3
I am very new to Hadoop. I am trying to run few sample prog. in Standalone (or local) mode. I have installed Cygwin in my windows machine. I am trying to run the command “hadoop version” to verify whether hadoop installed successfully or not. I have downloaded hadoop-2.2.0 version.



I have downloaded hadoop : C:\Users\Pavan\Downloads\

I tried to setup the PATH variable in Cygwin as below


$ echo $JAVA_HOME
C:\Program Files\Java\jdk1.6.0_32
$ export HADOOP_INSTALL=/cygdrive/c/Users/Pavan/Downloads/hadoop
$ export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin

I am trying to run the command “hadoop version” and end up with the following error. Please help me in resolving this error.

$ hadoop version
java.lang.NoClassDefFoundError: org/apache/hadoop/util/VersionInfo
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.VersionInfo
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.VersionInfo. Program will exit.
Exception in thread "main"
Marcos Serrano
Ranch Hand

Joined: Jul 13, 2006
Posts: 46
mm maybe your environment is not defined properly,

I share my environment defined in my cygwin user's .bashrc file [CYGWIN environment]

export JAVA_HOME=/cygdrive/c/Java/jdk1.6.0_23/
export JRE_HOME=/cygdrive/c/Java/jre6
export HADOOP_INSTALL=/home/saivulle/apps/hadoop-1.2.1
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin:$PATH

then from cygwin command window

$ hadoop version
Hadoop 1.2.1
Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152
Compiled by mattf on Mon Jul 22 15:23:09 PDT 2013
From source with checksum 6923c86528809c4e7e6f493b6b413a9a
This command was run using /C:/cygwin/home/saivulle/apps/hadoop-1.2.1/hadoop-core-1.2.1.jar

toutes les jours
chris webster
Bartender

Joined: Mar 01, 2009
Posts: 1772
    
  14

If you are a complete beginner, I would strongly recommend you try one of the pre-configured Hadoop virtual machines (VMs) available from various suppliers. Hadoop is really horrible to install and configure, especially if you want to try using some of the related tools like Pig, Hive, etc. There are lots of compatibility problems between different versions of the various libraries, and it's generally a huge PITA to get working the first time you try it.

If your main goal is to learn how to install/administer Hadoop, then fine, install it yourself. But if your initial aim is to find out how to use Hadoop to do interesting things with Big Data, make life easy on yourself by using a pre-configured installation.

Cloudera has a Quickstart VM and Hortonworks has its developer sandbox. Both of these are Linux-based VMs with all the main Hadoop-related tools pre-installed and configured correctly, and the VMs can be downloaded for free (the files are pretty big) and run inside the (free) VirtualBox virtualisation environment.

Hortonworks sandbox comes with a lot of built-in tutorials and gives you a browser-based interface to upload files to HDFS, try out Hadoop processing with Pig, Hive etc.

Cloudera's VM is used as the basis for the free Udacity course Introduction to Hadoop and MapReduce, which is aimed at complete beginners and will take you through the basics of what Hadoop does. Choose the "free courseware" option when you sign up for the course.


No more Blub for me, thank you, Vicar.
danthuluri pavan kumar
Greenhorn

Joined: Sep 25, 2008
Posts: 3

Thanks all for you response. I recently found the below information and thought might be useful to other hadoop Enthusiasts.

As of the last release, Hadoop 2.2 supports Windows natively. Thus, old workarounds like Cygwin are now un-supported. Please refer to the new documentation mentioned below (and copied here) for latest instructions for Hadoop on Windows.

https://wiki.apache.org/hadoop/Hadoop2OnWindows

If you are running Hadoop 1, then please check out branch-1-win and use that for running Hadoop 1 natively on Windows.


Now I am exploring hadoop-1.2.1 on windows using (cygwin + eclipse europa) , can anyone guide/help me in finding the information on eclipse plugin ?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: verify hadoop installation