Win a copy of Event Streams in Action this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Knute Snortum
  • Rob Spoor
Saloon Keepers:
  • Tim Moores
  • Ron McLeod
  • Piet Souris
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

Kafka Streams in Action: Partitioning with Clustered Type

 
Ranch Hand
Posts: 77
2
Netbeans IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Mr. Bill Bejeck,

I have experimented with how to create topics with kafka-console-procer.sh and kafka-console-consumer.sh tools. It is intended to run with zookeeper tool.
For the Figure 1.15, I have created five (5) set of topics in which each represented with its own category of records. I used the command line output as show in below attachment file.


I used simple logic using Java by developing Producer and Consumer classes. So it can be used to create a topic and messages. Then it gets queued up into message queuing system.
What I would clarify here is about the “paritions”, whether it is 1, or more. If it is more number of partitions allocated in it, then messages are not read by consumer in sequential order, but it is read as random.

N-clustered servers are located and accessed with broker logic in a single computer.

Question is, “Do we need to use an offset of a partition in order to identify in N-clustered servers?

Thank you
screen-1-my-topics.png
[Thumbnail for screen-1-my-topics.png]
 
author
Posts: 18
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Noorul,

If I understand your question, when using more than 1 partition in a topic, records are read in-order in a partition, but the order of records is not guaranteed across partitions.  

You don't need to worry about using offsets in a multi-node Kafka cluster, the offsets are only used by the consumer when re-connecting so it can pick up where it left off.

HTH,
Bill
 
Noorul Hameed
Ranch Hand
Posts: 77
2
Netbeans IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dear Mr. Bill Bejeck,

Thank you for your answer.


 
Don't get me started about those stupid light bulbs.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!