• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Spark In Action: Languages

 
Ranch Hand
Posts: 462
Scala jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I looked at Spark a long time ago and it seemed it was a part of a larger ecosystem, can you use it on its own or does it need a Hadoop cluster to run (or some other framework)?
 
Bartender
Posts: 361
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Does it is going to cover Spark integration with various other technologies(reading data from data lake, diverse data sets, diverse real time streams, Snowflake and other NoSQL DBs and Apache Kafka)?
Are the code examples correct and working fine OOTB?
 
Ranch Hand
Posts: 32
3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
@Will:

Spark can run on your laptop reading/writing data to flat-files locally, or it can run in cluster mode on various distributed platforms (HDFS, Hadoop, Kubernetes, Cassandra, ...), as well as being available as a service from major cloud providers like Google (Dataproc), Azure, Amazon.

So you can learn or experiment with Spark on your laptop, develop code there, then deploy the same code to run on a cluster in the cloud.

Databricks is the company behind Spark (as well as the proprietary Databricks data platform), so check out their website for tutorials and background info on Spark:

https://databricks.com/spark/about

There are also lots of interesting talks - about what you can do with Spark and how people are using it in the real world - available online from the various conferences e.g. https://databricks.com/sparkaisummit/north-america-2020
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic