• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

reading primitive types from files

 
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
DataInputStream and RandomAccessFile have methods for reading primitive types. Unfortunately only bytes can by read into array. The other types must be read one by one.



I use similar code to read terrain elevations from large files. It is quite slow.

Is there any better way to do it?




 
Sheriff
Posts: 3063
12
Mac IntelliJ IDE Python VI Editor Java
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The way Java implemented I/O Streams allows you to mix their capabilities together, which can be a really powerful feature. In this case, you need a BufferedInputStream to improve performance. Something like:



That's a stream that reads from the file "myfile.dat" into a 4096 byte buffer, and then parses your data out of the buffer. You get the data interface while still minimizing slow disk I/O calls.

This technique wouldn't really work with a RandomAccessFile though since that lets you jump around the file and read or write.
 
Ondrej Cervinka
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you Greg,

I forgot to mention that after reading the whole file I should end up with array of doubles or DoubleBuffer. I guess your code will be much faster because minimizing I/O access. There is still overhead of copying from buffer (BufferedInputStream) and calling readDouble method really often. The fastest would be reading directly into an array of doubles or DoubleBuffer. This can be achieved by calling once read(byte[]) but only for byte array not double array.
 
Greg Charles
Sheriff
Posts: 3063
12
Mac IntelliJ IDE Python VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Interesting. I've never worked with the DoubleBuffer class, but following the API, you should be able to read in all your data into a byte array, then call ByteBuffer.wrap(byteArray). That gives you a ByteBuffer to work with, so you can call: myByteBuffer.asDoubleBuffer(). I don't know how the performance compares with using a buffered DataInputStream though.
 
Ondrej Cervinka
Greenhorn
Posts: 5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Interesting too. I will try to compare the performance.
 
Greg Charles
Sheriff
Posts: 3063
12
Mac IntelliJ IDE Python VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's great. Be sure to come back and post what you find out!
 
Oh. Hi guys! Look at this tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic