• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
  • Campbell Ritchie
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Devaka Cooray
  • Paul Clapham
  • Tim Cooke
  • Knute Snortum
  • Bear Bibeault
Saloon Keepers:
  • Ron McLeod
  • Tim Moores
  • Stephan van Hulst
  • Piet Souris
  • Ganesh Patekar
  • Frits Walraven
  • Carey Brown
  • Tim Holloway

Unit Test a Sample Kafka Consumer and returned messages for a topic

Ranch Hand
Posts: 41
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, I am extremely new to Kafka and I have just written a Kafka Consumer following the example here: https://www.packtpub.com/books/content/writing-consumers
Kafka is running on a EC2 instance and I would like to test that my consumer is actually returning the messages from a topic as created by the Producer. I have been trying to understand how to Unit Test the Consumer app and came across this: https://ransilberman.wordpress.com/2013/07/19/how-to-unit-test-kafka/ that says that the Kafka API comes with an embedded Kafka server that can easily be used for unit tests. However, I am not sure I need to go down that road in order to test the outputted messages from a particular topic. Can you help? Is there an easy way to test the returned topic messages?

Thank you so much,



I am expecting my message in a JSON format, do I need a serializer to read them?
Posts: 13510
Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would try to stop thinking about the Kafka infrastructure altogether. The point of unit testing is to see behavior in isolation. If you're going to unit test, then isolate your class under test (CUT). Use the test to say something like this: "Ok, I have this CUT, and if it were actually hooked up to Kafka, this is how it would be called..."

In code, that would look like this:

Whatever code would go where line 6 is, that should not include actually setting up Kafka, etc. That code would just do things to make your testSubject "think" that it's getting real input from the real Kafka.

It's kind of the same thing I taught my son. If you have a program that needs the user to enter something, don't test by asking the user to enter something. Instead, write your test so that it pretends that it's using something that the user entered. That way, you take the User out of the test equation.
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!