You need a chat server program which can persist conversations (like have a DB or pesistent store at the server side). Chat client(s) can connect to chat server and communicate with each other.
Google for chat server programs in java, you can find many.
I promise I will be the best, most loyal friend ever! All for this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop