FileChannel and read method

[oli renard]

Hello,
this is my first post on this forum and I have a question regarding NIO. I am trying to read data from a file using a FileChannel from a RandomAccessFile. It does work, but I am wondering whether the code that I am using is correct or not. Can anyone please tell me whether this:

ByteBuffer bb = null;
RandomAccessFile raf = new RandomAccessFile(myFile, "r");
FileChannel fc = raf.getChannel();
bb = bb.allocate((int)fc.size());
[B]while(bb.hasRemaining()) {
fc.read(bb);
}[/B]
fc.close();
raf.close();

or this:

ByteBuffer bb = null;
RandomAccessFile raf = new RandomAccessFile(myFile, "r");
FileChannel fc = raf.getChannel();
bb = bb.allocate((int)(fc.size()*0.1));
[B]while(fc.read(bb) != -1) {
// Fill the buffer
}[/B]
fc.close();
raf.close();

is better and what the differences are and what the implications possibly are? I prefer the first, but I have never found an example of if and only ever found examples of the second. However, in the second case, if the whole file is not read, -1 (or EOF) will never be reached. As you can see, in the second case, I had to increase the size of the buffer so that it can accommodate the EOF marker.
Thank you very much

[Jim Yingst]

The first code sample looks good; this is probably what I would do. The only reason not to to this is if the file is very large, and you can't afford to keep all the data in memory at once. In which case you'd use a smaller buffer and process one buffer-full at a time. Which is what it looks like you're doing in the second code sample. But your comments don't make sense to me:
if the whole file is not read, -1 (or EOF) will never be reached.
You've made a loop. If the whole file is not read, you'll repeat the loop and read some more. Eventually the whole file will be read, won't it? Assuming that you're resetting the buffer in side the loop so it can be reused. Perhaps you should show what you mean by "// Fill the buffer".
I had to increase the size of the buffer so that it can accommodate the EOF marker.
Compared to the first code sample, you've multiplied the buffer size by 0.1. That makes it smaller, not larger.
Can you explain what you want to do with the contents of the file?

[oli renard]

Thank you very much, Jim. The code that I have put in my question is for the SCJD and I though it would be more appropriate to post it here. Just like you, I prefer the first example.
What I meant by this:

if the whole file is not read, -1 (or EOF) will never be reached.

is that if you allocate the size of the file to the buffer and then loop until the read method finds -1, then this will never happen because the EOF marker is in fact fc.size() + EOFMarkerLength. Your comments about a large also worry me a bit and make me want to reconsider the use of the first example. Would it be acceptable then to use this:

ByteBuffer bb = null;
RandomAccessFile raf = new RandomAccessFile(myFile, "r");
FileChannel fc = raf.getChannel();
bb = bb.allocate((int)(fc.size()+1));
while(fc.read(bb) != -1) {
// Fill the buffer
}
fc.close();
raf.close();

Effectively, what I want to do is read the whole file, check the metadata and then run a small sanity check on the data itself to make sure it is in a consistent state before proceeding.
Thank you

[Jim Yingst]

[oli]: What I meant by this:
    if the whole file is not read, -1 (or EOF) will never be reached.
is that if you allocate the size of the file to the buffer and then loop until the read method finds -1, then this will never happen because the EOF marker is in fact fc.size() + EOFMarkerLength.

But you were talking about the second code sample. In this case (a) the buffer size is one tenth of the file size, and (b) regardless of the file size, you're in a loop and you will eventually reach the -1 if you empty the buffer on each iteration, but (c) I can't tell if this will happen because your comment "// Fill the buffer" does not make any sense to me. The buffer is already full; if you want to use it again on the next iteration of the loop, it needs to be emptied. It's like you've got a big pile of dirt that you want to move somewhere, and you've filled up a wheelbarrow with dirt. You can't put any more dirt in the wheelbarrow unless you first do something with the dirt already in it.
Your comments about a large also worry me a bit and make me want to reconsider the use of the first example. Would it be acceptable then to use this:
No, I think the first version is preferable. That last byte will never have any value read into it; it will always be 0. You're effectively adding a zero to the end of the file. Hopefully you'll just ignore this in whatever subsequent processing you do, but there's also a chance it will cause confusion later when you or someone else forgets there's an extra zero. I don't see any benefit in appending the extra zero.
The original code sample if fine. There's no need to actually read the EOF if you're sure you've read the full file size worth of data. EOF is not data; it's a signal that there is no more data. If you already know how much data there should be (thanks to fc.size()) and you've read all that, then you don't really need to do one more read to see the -1.
Hope that helps...
[ February 10, 2004: Message edited by: Jim Yingst ]

[oli renard]

Thanks again Jim,
you are right, my second example is not correct and should have read this:

ByteBuffer bb = null;
RandomAccessFile raf = new RandomAccessFile(myFile, "r");
FileChannel fc = raf.getChannel();
bb = bb.allocate((int)(fc.size()*[B]1.1[/B]));
while(fc.read(bb) != -1) {
    // Fill the buffer
}
fc.close();
raf.close();

What I was trying to say is that if I use this code, then -1 is never read and I never come out of the while loop:

ByteBuffer bb = null;
RandomAccessFile raf = new RandomAccessFile(myFile, "r");
FileChannel fc = raf.getChannel();
bb = bb.allocate((int)(fc.size()));
while(fc.read(bb) != -1) {
    // Fill the buffer
}
fc.close();
raf.close();

Obviously, I need to use a while loop since I cannot be sure that the whole file will be read in one go, and the only reason I increment the size by 10% is to ensure that the -1 marker is read.
Thank you

[Jim Yingst]

OK. Then most of my commentary on your second code example is irrelevant, since I thought you were using a smaller buffer.
What I was trying to say is that if I use this code, then -1 is never read and I never come out of the while loop:
Correct. Once the buffer fills up, each read() returns 0, not -1, and it keeps getting called in an infinte loop. There are several ways to code this, and several different errors that you can run into if you don't do it just right. The solutions using (fc.size() + 1) and (fc.size() * 1.1) will work, but the first solution using hasRemaing() is the most elegant; I see no reason to change it.
Actually there's a tiny problem with the 1.1 solution. If you ever have a file with less than 10 bytes, (int)(fc.size()*1.1)) will round down too much. E.g. for 9 bytes (int) (9 * 1.1) = (int) (9.9) = 9, too small. better to just add 1, since that's all you needed. Well, better to keep using the hasRemaining() since you don't need the extra 0 byte at all really.
On a related note though, if you were reading a different type of channel, say a SocketChannel, then you might have no way of knowing in advance how many bytes there are. In that case, allocate a buffer with some arbitrary size, and plan for the possibility that it might be too big, or too small. You can't use hasRemaining() because you might reach EOF before the buffer is full. Instead you end up with something like

ReadableByteChannel channel = getChannelFromSomewhere();
ByteBuffer bb = ByteBuffer.allocate(8192);
while (channel.read(bb) != -1) {
    bb.flip();
    // do something with bb contents
    bb.clear();
}
channel.close();

[oli renard]

Jim,
thank you evr so much for your help. I apologise for not understanding everything at first, so thank you for bearing with me. By the way, why do you use this quote every time? I am baffled by the last word:

For every complex problem, there is a solution that is simple, neat, and wrong.

Do you mean that simple solutions are usually wrong?

[Jim Yingst]

By the way, why do you use this quote every time?
I saw it on the net somewhere and liked it, so it's currently set as my signature. I don't know who said it initially. I change signatures occasionally, but I guess I've had this one for a while now.
Do you mean that simple solutions are usually wrong?
No, that would be an exaggeration. There are many simple solutions that are also correct; those are the ones you want. But often there are also other "solutions" that are too simple. Especially if the problem is complex. I guess my signature doesn't send the best possible message for programmers. A better message might be: strive for simplicity - but test for correctness.
[ February 10, 2004: Message edited by: Jim Yingst ]