• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Copious Data - the killer app for Functional Programming

 
Bartender
Posts: 2407
36
Scala Python Oracle Postgres Database Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Dean Wampler (author of Programming Scala) gave a really interesting talk at the Lambda Jam conference on why Copious Data is the Killer App for Functional Programming. He looks at the factors that make FP a great tool for dealing with "Big Data", and shows examples of how existing FP-based tools such as Scalding or Cascalog already make it much easier to work with Hadoop's map-reduce framework, which is after all based on the FP "flatmap" and "reduce" operations. More provocatively - and entertainingly - he describes Hadoop as the "EJB of our times" and sees a need for better alternatives to allow scalable "big data" processing without all the ceremony and inflexibility of Hadoop/Java and the limitations of the current map-reduce model, and he presents FP as a better paradigm on which to base the next generation of "big data" processing.

Anyway, it's interesting stuff, and speaking as an ex-database application developer keen to move into FP and "big data", I find it very encouraging that smarter people than me are also spotting the ways in which FP and big data seem like a natural match for each other.
reply
    Bookmark Topic Watch Topic
  • New Topic