aspose file tools*
The moose likes Hadoop and the fly likes how to use SequenceFileAsTextInputFormat in hadoop map? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Hadoop
Bookmark "how to use SequenceFileAsTextInputFormat in hadoop map?" Watch "how to use SequenceFileAsTextInputFormat in hadoop map?" New topic
Author

how to use SequenceFileAsTextInputFormat in hadoop map?

Joseph Hwang
Greenhorn

Joined: Aug 17, 2013
Posts: 16
Hello! I am novice in hadoop programming. I try to make some codes of hadoop map-reduce programming.
This is my text data format.

==================
Title : CAREER: Markov Chain Monte Carlo Methods
Type: Award
NSF Org : CCR
Latest
Amendment
Date : May 5, 2003
File : a0237834

Award Number: 0237834
Award Instr.: Continuing grant
Prgm Manager: Ding-Zhu Du
CCR DIV OF COMPUTER-COMMUNICATIONS RESEARCH
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date : August 1, 2003
Expires : May 31, 2008 (Estimated)
Expected
Total Amt. : $400000 (Estimated)
Investigator: Eric Vigoda vigoda@cs.uchicago.edu (Principal Investigator current)
Sponsor : University of Chicago
5801 South Ellis Avenue
Chicago, IL 606371404 773/702-8602

NSF Program : 2860 THEORY OF COMPUTING
Fld Applictn:
Program Ref : 1045,1187,9216,HPCC,


I don't know how to read the data from files with hadoop map function. This is my sample codes.

===== Mapper Class
public class MapperClass extends Mapper<Text, Text, Text, Text> {

private Text outputKey = new Text();
private Text outputValue = new Text();

public void map(Text key, Text value, Context context) throws IOException, InterruptedException{

// I don't know how to make codes
....

====== Driver Class
Configuration conf = new Configuration();
Job job = new Job(conf, "Hadoop Task");

FileInputFormat.addInputPath(job, new Path("/home/user01/input"));
FileOutputFormat.setOutputPath(job, new Path("/home/user01/output"));

job.setJarByClass(HadoopMapredDriver.class);
job.setMapperClass(MapperClass.class);
job.setReducerClass(ReducerClass.class);

job.setInputFormatClass(SequenceFileAsTextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);

job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);

I don't know how to set key and value text code with SequenceFileAsTextInputFormat,and how to
handle those data in hadoop map function, either.

Any advice or reference site? Your help will be appreciated!
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: how to use SequenceFileAsTextInputFormat in hadoop map?