Double-download of JNLP download=lazy jars

Hi,

I've got an app that has a resource jar that is almost 10 MiB. I'm being asked to increase it's payload up to 30 MiB and am concerned because sometimes the Java Web Start (JWS) fails at the current size. I've used Wireshark to see that sometimes it fails to download the entire file.

Since the resources aren't needed initially in the program, after some research I thought I would give the download=lazy attribute of the jar resource a try. I've got a simple Swing dialog JWS app that attempts to load org.postgresql.Driver via Class.forName in response to a button click. My main jar is indexed (META-INF/INDEX.LIST) and the lazy load looks great on initial launch.

First run only downloads jnlplazy.jnlp and JNLPLazyDownloader.jar

Invoking Class.forName for org.postgresql.Driver downloads lib/postgresql-8.3-603.jdbc3.jar

Additional attempts in that run result in no additional downloads.

Then I close the program, touch lib/postgresql-8.3-603.jdbc3.jar on the web server and launch the test app again. JWS hits the server for all three files with an If-Modified-Since header and gets the "updated" postgresql jar.

304 - jnlplazy.jnlp

304 - JNLPLazyDownloader.jar

200 439723 lib/postgresql-8.3-603.jdbc3.jar

Then I invoke Class.Forname and it downloads the postgresql jar again, this time without If-Modified-Since, and again the web server shows it sends it again.

a.b.c.d - - [08/Feb/2011:14:24:30 -0700] "GET /jnlplazy/jnlplazy.jnlp HTTP/1.1" 304 - "-" "JNLP/6.0 javaws/1.6.0_23 (b05) Java/1.6.0_23" "-"
a.b.c.d - - [08/Feb/2011:14:24:30 -0700] "GET /jnlplazy/JNLPLazyDownloader.jar HTTP/1.1" 304 - "-" "JNLP/6.0 javaws/1.6.0_23 (b05) Java/1.6.0_23" "-"
a.b.c.d - - [08/Feb/2011:14:24:30 -0700] "GET /jnlplazy/lib/postgresql-8.3-603.jdbc3.jar HTTP/1.1" 200 439723 "-" "JNLP/6.0 javaws/1.6.0_23 (b05) Java/1.6.0_23" "-"

a.b.c.d - - [08/Feb/2011:14:25:08 -0700] "GET /jnlplazy/lib/postgresql-8.3-603.jdbc3.jar HTTP/1.1" 200 439723 "-" "JNLP/6.0 javaws/1.6.0_23 (b05) Java/1.6.0_23" "-"

I verified with Wireshark that the bits are all being sent down twice. It's as if the "check for updates" download ignored the response data but remembered that there was a response that it needed to ask for later when that jar was referenced. I can reproduce this every time. I haven't even gotten to making sure a download progress is shown. At 400K downloading a couple times is an annoyance. At 30M it will be a pain.

Have others seen this? Is it a bug or am I just doing something wrong.

Maybe I'm tackling the problem in the wrong way anyway, especially if the JWS implementation of the Java Nework Launching Protocol (JNLP) DownloadService times out too quickly or is otherwise unreliable on large files and doesn't support restartable or incremental downloads just like the automagic download. After all the reading I've done I am still scratching my head on when to use the JNLP API vs having it auto-magically handled via the JNLP file entries.

Thanks for your thoughts,

I did some testing with the javax.jnlp classes to see what the API would tell me about the state of the files for the two runs, before and after updating the timestamp on the postgresql jar. I added a different button and call the following code on it's action handler:

try {
    BasicService bs = (BasicService)ServiceManager.lookup("javax.jnlp.BasicService");
    System.out.println("Codebase: "+bs.getCodeBase());
    DownloadService2 service = (DownloadService2) ServiceManager.lookup("javax.jnlp.DownloadService2");
    ResourceSpec spec = new ResourceSpec(bs.getCodeBase().toString() + ".*",
            null,service.ALL);
     // returns all cached resources that match the given ResourceSpec
    ResourceSpec results[] = service.getCachedResources(spec);
    System.out.println("Cached resources: " + results.length);
    for(ResourceSpec result:results) {
       System.out.println(result.getUrl()+"; "+result.getVersion()+"; "+result.getLastModified()
       +"; "+result.getType()+"; "+result.getSize());
    }
    // returns all resources for which an update is available
    results = service.getUpdateAvailableResources(spec);
    System.out.println("Updatable resources: " + results.length);
    for(ResourceSpec result:results) {
       System.out.println(result.getUrl()+"; "+result.getVersion()+"; "+result.getLastModified()
       +"; "+result.getType()+"; "+result.getSize());
    }
} catch (IOException ex) {
    Logger.getLogger(LazyDownloaderFrame.class.getName()).log(Level.SEVERE, null, ex);
} catch (UnavailableServiceException ex) {
    Logger.getLogger(LazyDownloaderFrame.class.getName()).log(Level.SEVERE, null, ex);
}

Here is the output of that code before touching the postgresql jar:

Codebase: http://mysite/jnlplazy/
Cached resources: 2
http://mysite/jnlplazy/jnlplazy.jnlp; null; 1297208281000; 1; 1386
http://mysite/jnlplazy/JNLPLazyDownloader.jar; null; 1297208295000; 4; 12513
Updatable resources: 0
Looking for org.postgresql.Driver
Found org.postgresql.Driver
Codebase: http://mysite/jnlplazy/
Cached resources: 3
http://mysite/jnlplazy/lib/postgresql-8.3-603.jdbc3.jar; null; 1297199364000; 4; 443706
http://mysite/jnlplazy/jnlplazy.jnlp; null; 1297208281000; 1; 1386
http://mysite/jnlplazy/JNLPLazyDownloader.jar; null; 1297208295000; 4; 12513
Updatable resources: 0

And again after:

Codebase: http://mysite/jnlplazy/
Cached resources: 2
http://mysite/jnlplazy/jnlplazy.jnlp; null; 1297208281000; 1; 1386
http://mysite/jnlplazy/JNLPLazyDownloader.jar; null; 1297208295000; 4; 12513
Updatable resources: 0
Looking for org.postgresql.Driver
Found org.postgresql.Driver
Codebase: http://mysite/jnlplazy/
Cached resources: 3
http://mysite/jnlplazy/lib/postgresql-8.3-603.jdbc3.jar; null; 1297208457000; 4; 443706
http://mysite/jnlplazy/jnlplazy.jnlp; null; 1297208281000; 1; 1386
http://mysite/jnlplazy/JNLPLazyDownloader.jar; null; 1297208295000; 4; 12513
Updatable resources: 0

The server logs show it being downloaded twice on the second run as I explained in my first post. Note how the postgresql jar isn't listed in the cache until I look it up.

When I run the software a third time without touching the timestamps, it does show in the first call of getCachedResources without forcing it to load.

I don't understand why I don't see the postgresql jar in the update available list at least once. I am just basing my tests off of the code at the JNLP API Examples page.

http://download.oracle.com/javase/6/docs/technotes/guides/javaws/developersguide/examples.html

Thanks again,

Hi Jacob,

Did you ever get to the bottom of this?, I am having the exact same issue.

We have an application jars total to 15mb, with an empty cache, the download size is 30mb, when the cache is populated, the download is always 15mb, (verified by wireshark)

I can see in the tcp trace I am getting 304's as expected, but then the app gets downloaded again (200's) since there is another request without the if-modified-since in the header.

Any help with this will be most greatfull.

Ian