• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

want to learn hadoop

 
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hello everyone,
i am a fresher in IT having knowledge of core java no knowledge of linux and no knowledge of databases. but still i want to start learning hadoop. please suggest me some steps to learn hadoop.
secondly, my computer specification is 4gb ram,dual core with vmware 7.1.2 installed in it. is it possible to install hadoop for practice in my computer.

looking forward to your guidance.
thanks in advance..
 
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

akshay naidu wrote:hello everyone,
i am a fresher in IT having knowledge of core java no knowledge of linux and no knowledge of databases. but still i want to start learning hadoop. please suggest me some steps to learn hadoop.
secondly, my computer specification is 4gb ram,dual core with vmware 7.1.2 installed in it. is it possible to install hadoop for practice in my computer.

looking forward to your guidance.
thanks in advance..



with so low memory, for learn, you can try hadoop 1.3, better 8GB ram for basic work

hadoop 2.3 woud be a nightmare with 4GB, need 12GB ram with hbase, 16GB ram config is enough for some basic work.


down on last one in the list

take HDP 1.3 for vmware

http://hortonworks.com/products/hortonworks-sandbox/#archive


 
Ranch Hand
Posts: 1609
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think Cloudera VM recommends 64-bit CPU with 4 GB ram for 5.4.x but 8 GB should be reasonably smooth.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I downloaded HDP 1.3 for VMware.
now when I try to open it with VMware a message appears saying "failed to query for source information".
I am unable to find any discussion related to this problem anywhere.
please someone help me here.
thanks in advance.
 
Greenhorn
Posts: 4
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello Akshay

Its good that you have some prior knowledge of java. Hadoop is a framework which is based on java programming. It is intended to level up from single server to thousands of machines each offering local computation and storage. It supports the huge data set in a distributed computing environment. Hadoop is written in Java, so there is a need of Java installed on the machine and version should be 1.6 or later.

As you said that you have 4GB RAM and whatever the configuration you have that's enough to work on Hadoop. Hadoop is supported by Linux platform and its flavors. So you have to install a Linux OS for setting up Hadoop environment. If you have another operating system than Linux then in this case you can install virtual machine and have Linux inside the virtual machine

To learn Hadoop basic commands knowledge of UNIX, SQL scripting. Prior knowledge of Apache Hadoop is not required. Hadoop is a booming and on demand technology in the industries. Top Big Data Analytics companies like Google, Yahoo, Apple, eBay, Facebook and many others are hiring skilled professionals capable of handling Big Data. So its a good opportunity for you give perk in your career to learn hadoop from basics to advance level.

To learn Big Data Hadoop and its ecosystem and work on real-time applications of big data, a professional training from Industry Experts is considered a must-have.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hello ruchika, thanks for that explaination.
i have windows system, i am following the book hadoop definative guide and facing a lot lot of trouble, still struck with environment variable , path and other simple initial problems.
 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What problems?
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
for now i am trying to my first wordcount program and this is the error..


 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
These are java compilation problems, not Hadoop problems.
You should include hadoop-mapreduce-client JARs in classpath.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
and how do i do that?
please suggest
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Using -cp command line argument. Read https://coderanch.com/t/660138/Wiki/Set-Classpath and https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html

If you're not familiar with Java itself, trying to learn or deploy Hadoop is a non-starter. You should become comfortable with Java first.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
i am familiar with java.. i can set classpath but what i didnt understood was what are hadoop-mapreduce-client jars and where can i find it
thanks.
 
akshay naidu
Ranch Hand
Posts: 85
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
there are no values for classpath in my computer, what do i need to add here?
 
Karthik Shiraly
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Start by downloading and extracting hadoop 2.x archive and then do a file search in it for .jar files.
 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
It's really good that you want to learn Hadoop. It's a really good time to start your career in Big Data Hadoop field. Hadoop framework is written in Java. So, you have an added advantage because you know Java.
For learning Hadoop, start with the basics and then move ahead with the complex part. First learn the challenges of Big Data, why we should learn big data etc. then move ahead with Hadoop. Start with the Hadoop introduction then learn its 3 main parts i.e HDFS, MapReduce and Yarn. After that learn PIG, HIVE, HBase and try to write its queries. If you know SQL then you can easily write HIVE queries
HIVE + SQL = HiveQL
Hive use language called HiveQL (HQL), which is similar to SQL. HiveQL automatically translates SQL-like queries into MapReduce jobs.
Hope this helps
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic