Why Firefox is four times faster to download zip file then my own Java program?

[Boris Osipchuk]

I tried using regular IO or NIO but Firefox always beats it by being about 4 times faster. Is there something special they are using?
For example, I tried downloading eclipse distributions and needed about 40 minutes for like 360 MB and Firefox did in 10 minutes.
I will post the code, if requested, but hope this is something beyond my code.

[Rob Spoor]

Perhaps Firefox uses multiple connections to download from, then pastes the contents together on your PC.

[Ernest Friedman-Hill]

Let's see your loop for actually reading/ writing, and maybe something will jump out at us. I suspect some of that speed comes from using multiple buffers in rotation, and reading and writing from dedicated threads, but there are some very basic things -- using a large - enough buffer, for example -- that can make a difference.

[Boris Osipchuk]

Hi guys, I have tried to break it up into four threads, as Rob suggested, by I don't know multi-threading very well, so what I get is I can see that 4 files are being created but only one file is being populated with data. After that is done the application terminates and 3 other files don't contain any data.
Here's my downloader class.

  
package download;

import java.io.FileOutputStream;
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;

public class DownloaderNIO implements Runnable {

  protected String targetUrl;
  protected String localUrl;
  protected int number;
  protected int startPosition;
  protected int count;

  public DownloaderNIO(String targetUrl, String localUrl, int number, int startPosition, int count) {
	this.targetUrl = targetUrl;
	this.localUrl = localUrl;
	this.number = number;
	this.startPosition = startPosition;
	this.count = count;
  }

  @Override
  public void run() {
	downloadUsingNIO(targetUrl, localUrl + number, startPosition, count);
  }

  /**
   * Downloads a file using NIO.
   * 
   * @param address
   *          address from which to download file
   * @param localFilename
   *          address where to store file locally
   */
  public void downloadUsingNIO(String address, String localFilename, int startPosition, int count) {
	URL downloadUrl;
	try {
	  downloadUrl = new URL(address);
	  ReadableByteChannel rbc = Channels.newChannel(downloadUrl.openStream());
	  FileOutputStream fos = new FileOutputStream(localFilename);
	  fos.getChannel().transferFrom(rbc, startPosition, count);
	}
	catch (MalformedURLException e) {
	  // TODO Auto-generated catch block
	  e.printStackTrace();
	}
	catch (IOException e) {
	  // TODO Auto-generated catch block
	  e.printStackTrace();
	}
  }
}

and here's how I call it.

    
package download;

import java.util.Date;

public class DownloaderMain {

  /**
   * URL that contains Eclipse download.
   */
  public static final String ECLIPSE_TARGET_URL = "http://www.gtlib.gatech.edu/pub/eclipse/technology/epp/downloads/release/galileo/R/eclipse-modeling-galileo-incubation-win32.zip";

  /**
   * URL that contains Eclipse storage location on locally.
   */
  public static final String ECLIPSE_LOCAL_URL = "c:/eclipse-modeling-galileo-incubation-win32.zip";

  /**
   * @param args
   */
  public static void main(String[] args) {
	long beginTime = System.currentTimeMillis();
	System.out.println("Download  Started: " + new Date(beginTime));

	int startPosition = 0;
	int bytesToRead = 1 << 26;
	int downloadNumber = 0;

	new Thread(new DownloaderNIO(ECLIPSE_TARGET_URL, ECLIPSE_LOCAL_URL, ++downloadNumber, startPosition,
		bytesToRead)).start();
	startPosition = bytesToRead;

	new Thread(new DownloaderNIO(ECLIPSE_TARGET_URL, ECLIPSE_LOCAL_URL, ++downloadNumber, startPosition,
		bytesToRead)).start();
	startPosition = 2 * bytesToRead;

	new Thread(new DownloaderNIO(ECLIPSE_TARGET_URL, ECLIPSE_LOCAL_URL, ++downloadNumber, startPosition,
		bytesToRead)).start();
	startPosition = 3 * bytesToRead;

	new Thread(new DownloaderNIO(ECLIPSE_TARGET_URL, ECLIPSE_LOCAL_URL, ++downloadNumber, startPosition,
		bytesToRead)).start();
  }
}