• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

ObjectOutputStream faster than nio?

 
Ranch Hand
Posts: 529
C++ Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I thought that nio was faster than the standard io for Java. But today as I was writing a rather large int[] to disk, I discovered that using ObjectOutputStream was significantly faster. Twice as fast in fact. Can someone take a look at this code and please tell me if I did something wrong with the nio part? If I didn't do anything wrong, then can someone please explain why the OOS is so much faster?




Many thanks!
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The biggest performance problem I see in your code has nothing to do with IO or NIO - it's your for loop:

This is equivalent to

The iterator() method returns a custom Iterator<Integer> that creates an Integer object to wrap each and every int in your array - just so that the for loop can unwrap it with intValue() and discard the wrapper. The combination of an enhanced for loop with autoboxing has added some unseen overhead to each and every iteration of the loop - you're creating and discarding an object. And making a couple method calls, and a cast. Normally these are fairly minor, insignificant things. But the loop you've written is also doing something small and insignificant - it's putting four tiny bytes into a buffer. And it's doing this many, many times, because you've got a very large array. So the overhead of creating objects and making method calls is significant, compared to the time to move the four bytes. Anytime you've got a short loop repeated many times, efficiency may become noticable. Try replacing your code with a more traditional idiom:

On my machine, this makes the NIO code a bit more efficient than the ObjectOutputStream code. It's not a big difference in this case. Note that NIO can be much faster than IO, in some situations. It isn't always. I believe NIO is often fastest when you use ByteBuffers throughout your processing. (Or at least, as much as possible.) Here you're starting with a traditional int[] array, and then you're converting it to a ByteBuffer. That takes some extra time. If you can put your data into a ByteBuffer (or an IntBuffer wrapped around a ByteBuffer) in the first place, you can probably get this to go faster. I don't know if this is really possible for you - it depends on how and where you get the data for your int[] array. If you use NIO as early in the process as possible (including using buffers rather than arrays), you maximize your chances of benefitting from NIO.
[ January 20, 2005: Message edited by: Jim Yingst ]
 
Barry Andrews
Ranch Hand
Posts: 529
C++ Java Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Jim! I did not realize that about the for each loop. That is good to know.

I changed my loop to the traditional kind and I did see a slight improvement but it was not as significant as I had hoped. On my machine it took an average time of 1.27 secs to write 786432 ints to disk using the enhanced for and nio, and it took 1.17 secs using the traditional loop and nio. I am still getting better results with the OOS though. It only takes .75 secs.

You mention that it would be better to use the ByteBuffer in the first place so I don't have to copy the data over. Is that how you got nio to perform better than OOS? The int[] that I am getting comes from a 3rd party library (Eclipse), so I would have to modify a lot of code to get my data into a ByteBuffer. However, if I could provide a case and show it would be a nice performance enhancement maybe I could convince them to add a method to their API.



thanks for the help,

Barry
 
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also, you're making a ton of unnecessary calls by using a ByteBuffer to write an int[]. First, take a gander at the source for ByteBuffer (specifically HeapByteBuffer).putInt(int x). It uses a static helper class, Bits, to do the work. But first it has to calculate the offset (two method calls).

Bits makes a single delegating method call (big- or little-endian) which makes eight method calls: four to get the individual bytes of the int and four more to put those bytes into the buffer.

Total method calls, including call to putInt() itself: 13, though the calls to separate the int into bytes are static and could be inlined. That's just ugly. My first recommendation would have been to use an IntBuffer and skip all that mess, instead using System.arraycopy(), but ... you can only write ByteBuffers to a FileChannel! Many four-letter words spring to mind, but I'll be nice and leave it at "lame."

So my next suggestion would be to convert the int[] to an intermediary byte[], wrap it with a ByteBuffer, and write that to the FileChannel. After running some tests (see below), this turns out to be faster than the IntBuffer (by about 25%) but still slower than the OOS (more than twice as long).

[ Insert appropriate warning abuot micro-benchmarks here. ]

Here's the code I used to test.Here are tests I ran, two of each with 100 iterations writing 786432 pixels.After making the number of pixels configurable, I ran the tests again using 10240 pixels instead of 786432 with different results:The average is too small, but notice that the ByteBuffer method is now faster than the ObjectOutputStream, and the latter is more erratic. It's conceivable that using a smaller buffer and writing chunks of the int[] in a loop may prove to be the best method, but I'm too tired to take that up at the moment.
reply
    Bookmark Topic Watch Topic
  • New Topic