• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Knute Snortum
  • Paul Clapham
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Frits Walraven
Bartenders:
  • Ganesh Patekar
  • Tim Holloway
  • salvin francis

Kafka Streams in Action: Partitioning with Clustered Type  RSS feed

 
Ranch Hand
Posts: 77
2
Firefox Browser Java Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Mr. Bill Bejeck,

I have experimented with how to create topics with kafka-console-procer.sh and kafka-console-consumer.sh tools. It is intended to run with zookeeper tool.
For the Figure 1.15, I have created five (5) set of topics in which each represented with its own category of records. I used the command line output as show in below attachment file.


I used simple logic using Java by developing Producer and Consumer classes. So it can be used to create a topic and messages. Then it gets queued up into message queuing system.
What I would clarify here is about the “paritions”, whether it is 1, or more. If it is more number of partitions allocated in it, then messages are not read by consumer in sequential order, but it is read as random.

N-clustered servers are located and accessed with broker logic in a single computer.

Question is, “Do we need to use an offset of a partition in order to identify in N-clustered servers?

Thank you
screen-1-my-topics.png
[Thumbnail for screen-1-my-topics.png]
 
author
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Noorul,

If I understand your question, when using more than 1 partition in a topic, records are read in-order in a partition, but the order of records is not guaranteed across partitions.  

You don't need to worry about using offsets in a multi-node Kafka cluster, the offsets are only used by the consumer when re-connecting so it can pick up where it left off.

HTH,
Bill
 
Noorul Hameed
Ranch Hand
Posts: 77
2
Firefox Browser Java Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Mr. Bill Bejeck,

Thank you for your answer.


 
Those are the largest trousers in the world! Especially when next to this ad:
how do I do my own kindle-like thing - without amazon
https://coderanch.com/t/711421/engineering/kindle-amazon
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!