This week's giveaway is in the Cloud/Virtualization forum.
We're giving away four copies of Secure Financial Transactions with Ansible, Terraform, and OpenSCAP and have Lucian Maly on-line!
See this thread for details.
Win a copy of Secure Financial Transactions with Ansible, Terraform, and OpenSCAP this week in the Cloud/Virtualization forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Rob Spoor
  • Henry Wong
  • Liutauras Vilda
Saloon Keepers:
  • Tim Moores
  • Carey Brown
  • Stephan van Hulst
  • Tim Holloway
  • Piet Souris
Bartenders:
  • Frits Walraven
  • Himai Minh
  • Jj Roberts

Spark In Action: Languages

 
Ranch Hand
Posts: 462
Scala jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
I looked at Spark a long time ago and it seemed it was a part of a larger ecosystem, can you use it on its own or does it need a Hadoop cluster to run (or some other framework)?
 
Bartender
Posts: 354
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Does it is going to cover Spark integration with various other technologies(reading data from data lake, diverse data sets, diverse real time streams, Snowflake and other NoSQL DBs and Apache Kafka)?
Are the code examples correct and working fine OOTB?
 
Ranch Hand
Posts: 32
3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
@Will:

Spark can run on your laptop reading/writing data to flat-files locally, or it can run in cluster mode on various distributed platforms (HDFS, Hadoop, Kubernetes, Cassandra, ...), as well as being available as a service from major cloud providers like Google (Dataproc), Azure, Amazon.

So you can learn or experiment with Spark on your laptop, develop code there, then deploy the same code to run on a cluster in the cloud.

Databricks is the company behind Spark (as well as the proprietary Databricks data platform), so check out their website for tutorials and background info on Spark:

https://databricks.com/spark/about

There are also lots of interesting talks - about what you can do with Spark and how people are using it in the real world - available online from the various conferences e.g. https://databricks.com/sparkaisummit/north-america-2020
 
That feels good. Thanks. Here's a tiny ad:
SKIP - a book about connecting industrious people with elderly land owners
https://coderanch.com/t/skip-book
reply
    Bookmark Topic Watch Topic
  • New Topic