This week's book giveaway is in the OO, Patterns, UML and Refactoring forum.
We're giving away four copies of Refactoring for Software Design Smells: Managing Technical Debt and have Girish Suryanarayana, Ganesh Samarthyam & Tushar Sharma on-line!
See this thread for details.
The moose likes Hadoop and the fly likes Real-time Hadoop Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

JavaRanch » Java Forums » Databases » Hadoop
Bookmark "Real-time Hadoop" Watch "Real-time Hadoop" New topic

Real-time Hadoop

Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
What about Hadoop in the context of real-time requirements for some systems like those exist in mobile networks?
and if it is not supported, what are the alternatives in particularly for the real-time case?

Mohamed El-Refaey

Best Regards, Mohamed El-Refaey
Garry Turkington

Joined: Apr 23, 2013
Posts: 15
So true realtime isn't a core Hadoop strength at this point. The processing model really doesn't lend itself to those sort of requirements.

Looking ahead though two interesting developments are Yarn and Tez. The former treats the Hadoop platform as a more generic processing framework and Tez (just starting) will be building processing models that differ significantly from what we are used to in MapReduce today. Those seem like areas of major innovation to me and I suspect we'll see a whole raft of different types of processing happening on Hadoop in the future.

Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
I see, but it looks that tez is trying to increase processing speed but not really a real-time solution ..
Thanks for referring to yarn and tez, wonderful framework

Carlos Morillo
Ranch Hand

Joined: Jun 06, 2009
Posts: 221

Real Time in a Hadoop cluster is one of the many things possible when you use MapR Enterprise Grade Distribution compared to Apache Hadoop or other distributions thanks to the many enhancements and features of MapR-FS, which is a read/write file system among others.

See this use case documented here

If you want to accomplish this using other Hadoop distributions at least you would need 2 clusters and a lot of more hardware, therefore way more expensive.



Mohamed El-Refaey
Ranch Hand

Joined: Dec 08, 2009
Posts: 119
Interesting use case also mentioned that it deals with real time sensors' data, in deployment such as IoT.
Thanks Carlos!
I’ve looked at a lot of different solutions, and in my humble opinion Aspose is the way to go. Here’s the link:
subject: Real-time Hadoop
It's not a secret anymore!