| Author |
MongoDB map reduce
|
Karthik Shiraly
Ranch Hand
Joined: Apr 04, 2009
Posts: 364
|
|
I recently migrated a personal google maps based web app from a Mysql database to MongoDB.
I found the mapreduce APIs kind of difficult to use in java, because they expect javascript code as strings and munging javascript code with the right escaping for double quotes and so on was a pain.
Eventually, what I did was write the mapreduce code in js files, and later read them as strings and pass those strings.
Nevertheless, it still seemed rather inefficient to me to pass code from client side, especially if the MR code is complex or executed often. I'm not sure if this is how it's supposed to be done.
If we look at mapreduce functions as analogous to the stored procedures of RDBMSes, then would it not make for less latency to deploy them just once on the mongo server side and call them by name? Is that possible?
Also, from your book TOC, it seems like mapreduce capabilities are not covered. Is MR not seen as a priority for mondodb? Are any new MR features planned?
By the way, I for one have been impressed by mongo's spatial query capabilities, the fact that it leaves the choice of strong or eventual consistency in hands of developer, and also by the document oriented paradigm which certainly reduced the friction against adding new features. I hope full text search is also in the pipeline. Great product overall!
|
 |
Arun Radhakrishnan
Greenhorn
Joined: Jan 11, 2011
Posts: 7
|
|
@ Karthik
I don't think Map Reduce is tied to MongoDB in any manner. You could use any data-source with Map Reduce.
|
 |
Kyle Banker
author
Greenhorn
Joined: Nov 22, 2011
Posts: 16
|
|
Hi Karthik,
The MapReduce API is a bit difficult to use, I agree. But the biggest problem with MongoDB's MapReduce is that it's kind of slow for large data sets. MongoDB 2.1 (unstable) includes the new aggregation framework that will replace MapReduce for most purposes. It's a lot faster, if a bit less flexible, than the current MapReduce. You can read about it here:
http://www.mongodb.org/display/DOCS/Aggregation+Framework
The main reason I chose not to spend a lot of time in the book on MapReduce is precisely because this new framework will become much more relevant to most developers with the next release.
Regards,
Kyle
|
 |
Karthik Shiraly
Ranch Hand
Joined: Apr 04, 2009
Posts: 364
|
|
|
Thanks for your reply and the info on upcoming aggregation APIs - will try them out.
|
 |
 |
|
|
subject: MongoDB map reduce
|
|
|