This week's book giveaway is in the Reactive Progamming forum.
We're giving away four copies of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams and have Adam Davis on-line!
See this thread for details.
Win a copy of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams this week in the Reactive Progamming forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Junilu Lacar
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Knute Snortum
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Ganesh Patekar

Unit Test a Sample Kafka Consumer and returned messages for a topic

 
Ranch Hand
Posts: 41
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, I am extremely new to Kafka and I have just written a Kafka Consumer following the example here: https://www.packtpub.com/books/content/writing-consumers
Kafka is running on a EC2 instance and I would like to test that my consumer is actually returning the messages from a topic as created by the Producer. I have been trying to understand how to Unit Test the Consumer app and came across this: https://ransilberman.wordpress.com/2013/07/19/how-to-unit-test-kafka/ that says that the Kafka API comes with an embedded Kafka server that can easily be used for unit tests. However, I am not sure I need to go down that road in order to test the outputted messages from a particular topic. Can you help? Is there an easy way to test the returned topic messages?

Thank you so much,

I.

[EDIT]

I am expecting my message in a JSON format, do I need a serializer to read them?
 
Marshal
Posts: 14018
233
Mac Android IntelliJ IDE Eclipse IDE Spring Debian Java Ubuntu Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would try to stop thinking about the Kafka infrastructure altogether. The point of unit testing is to see behavior in isolation. If you're going to unit test, then isolate your class under test (CUT). Use the test to say something like this: "Ok, I have this CUT, and if it were actually hooked up to Kafka, this is how it would be called..."

In code, that would look like this:

Whatever code would go where line 6 is, that should not include actually setting up Kafka, etc. That code would just do things to make your testSubject "think" that it's getting real input from the real Kafka.

It's kind of the same thing I taught my son. If you have a program that needs the user to enter something, don't test by asking the user to enter something. Instead, write your test so that it pretends that it's using something that the user entered. That way, you take the User out of the test equation.
 
Don't sweat petty things, or pet sweaty things. But cuddle this tiny ad:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!