I'm using JCAPTCH as my CAPTCHA engine. I dunno the drawbacks yet... But since I'm encountering other problems in our deployment I couldn't test yet if this problem would add to them. So how do I find our that x server is running? Thanks!
Look carefully at that page; it explains several things you can do. One of those is *not* to run an actual X server -- it's not going to be a convenient solution, and doesn't do much for your site security.
As far as finding out if an X server is running, type
ps -ef | grep X
and look for a process just named "X". But even if one is running, unless it's running as the same user that the server is running as, chances are it's set up with security options to prevent the server from connecting, anyway.
So your best option is to use the "-Djava.awt.headless=true Java command-line switch when starting your Java server.