You can start with the
Java Media Framework. That's what Java uses for managing audio and video in applications. Working with the JMF is not trivial. You may be better off working through the Java Tutorial's
Custom Networking trail and making a simple chat app and server, then once you've got some experience with networking, attempting your audio chat server.