Although you can develop any application in Clojure, I think developing UI applications is often centered around changing state of the windowing system and as such doesn't fit very well in the functional approach (and Clojure in particular). You can, but it doesn't necessarily mean you should.
At a recent Austin Clojure Meetup someone demonstrated JavaFX integration, showing building and updating a UI from the repl. I don't do enough desktop app work to compare it to Seesaw, but it was quite interesting.
That exactly reflects my sentiments - it's quite interesting and amusing to see how one could develop a decent UI application in Clojure (or any other functional language) as the process is more dynamic (REPL) as well as challenging where mutable state is wrapped by a thin layer atop a immutable/pure one. It'd be a highly recommended exercise for anyone who'd like to enter the functional programming scene - it's especially helpful to see someone who's good at FP doing it (I think I've seen a video on youtube or vimeo).
I haven’t done much desktop Java with Clojure, although the little I have done I found it much easier than using Java itself. There is a function in Clojure called doto that helps you chain functions together, making your code much easier to follow and cleaner than the GUI code you need to write in Java. Is it following the functional programming idiom, possibly not in a pure way, but I'd consider it a more attractive option that writing all that GUI code in Java. However, there were quite a few GUI applications developed in Scheme over the years, so it would be interesting to see if there are any good concepts functional GUI development to discover.
I would certainly recommend using Seesaw if you wanted to develop Java desktop applications, along with the Netbeans platform.