Win a copy of Transfer Learning for Natural Language Processing (MEAP) this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Paul Clapham
  • Devaka Cooray
  • Bear Bibeault
Sheriffs:
  • Junilu Lacar
  • Knute Snortum
  • Liutauras Vilda
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Piet Souris
Bartenders:
  • salvin francis
  • Carey Brown
  • Frits Walraven

Apache Kafka support for millions of messages

 
Ranch Hand
Posts: 255
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I am using Apache Kafka to handle millions of messages per day. Now I need to form the environment, like

(i) How many Topics to be created?
(ii) How many partitions/replications to be created?
(iii) How Brokers to be created?
(iv) How many consumer instances in consumer group?

How we can go about it? Please clarify.

Thanks.
 
Rancher
Posts: 43015
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I don't understand the question; what do you mean by "form the environment"? It's not a commonly used phrase.
 
clojure forum advocate
Posts: 3479
Mac Objective C Clojure
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I think the question is how to configure their Apache Kafka installation. If so, then Apache Kafka documentation is the place to start with.
 
Rithanya Laxmi
Ranch Hand
Posts: 255
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Hussein, I went through the Apache Kafka document, I will be great how we can configure the below to handle millions of messages per day:-

(i) How many Topics to be created?
(ii) How many partitions/replications to be created?
(iii) How Brokers to be created?
(iv) How many consumer instances in consumer group?

How we can go about it? Please clarify.




 
Clowns were never meant to be THAT big! We must destroy it with this tiny ad:
the value of filler advertising in 2020
https://coderanch.com/t/730886/filler-advertising
    Bookmark Topic Watch Topic
  • New Topic