If your Web Start application can't use a properties file on the client machine, then chances are you can't use the Windows registry there either. And anyway you can't access the registry (easily) from Java anyway. However, you could try using the Preferences class in the java.util.prefs package. (That's an excellent alternative to properties files in any Java application.) To tell you the truth I'm not sure whether that will work either, but you should try it.
However what's this folder name you want to save? If your Web Start application can't access the file system on the client machine, then what folder is that?