StandardOpenOption

[Jesse Silverman]

In the Sybex OCP Java Certified Programmer II 1Z0-819 book by Jeanne & Scott, the StandardOpenOptions are described thusly:

APPEND If the file is already open for write then append to the end.

TRUNCATE_EXISTING If file is already open for write then erase file and append to beginning.

I haven't used NIO.2 in Java yet, but from years of working with files that sounded quite wrong to me, so I checked the JavaDocs which sound different:

APPEND
public static final StandardOpenOption APPEND
If the file is opened for WRITE access then bytes will be written to the end of the file rather than the beginning.
If the file is opened for write access by other programs, then it is file system specific if writing to the end of the file is atomic.

TRUNCATE_EXISTING
public static final StandardOpenOption TRUNCATE_EXISTING
If the file already exists and it is opened for WRITE access, then its length is truncated to 0. This option is ignored if the file is opened only for READ access.

I will go the tried and true route of writing some sample code to figure out what's going on, but the neat descriptions in the Sybex book don't go with what I am expecting.
They sound like they apply only if the file is already open for write before we issue the command featuring this option as a parameter.
Alternatively, they just mean to say that the WRITE option must precede this in the option list for it to work.  If so, fine, it is hard to fit descriptions into a neat table/chart.  I don't think that is true either tho...
So the question is what do JB & SS mean by "already open"?

Looking at the JavaDocs, it sounds more like "is opened" means "is being opened in this operation".  It doesn't mean to say that we won't get an append behavior unless the file is already currently open for write at the time of executing the open command??  Or does it?  I see the second line seems to mean that if the file is already opened for write access by other programs, then we will append, but this may not be atomic depending on underlying file system.

TRUNCATE_EXISTING description -- does this mean to say that if the file already exists at the time we issue the open command, but nobody has it open at that moment, we will not TRUNCATE?

I am leaning already towards the description in Sybex is "Wrongish" or at least misleading, and the description of the enums in the Javadocs is a little unclear.

I cribbed this from usage examples in the JavaDocs:

If no options are present then this method works as if the CREATE, TRUNCATE_EXISTING, and WRITE options are present. In other words, it opens the file for writing, creating the file if it doesn't exist, or initially truncating an existing regular-file to a size of 0 if it exists.

Usage Examples:

    Path path = ...

    // truncate and overwrite an existing file, or create the file if
    // it doesn't initially exist
    OutputStream out = Files.newOutputStream(path);

    // append to an existing file, fail if the file does not exist
    out = Files.newOutputStream(path, APPEND);

    // append to an existing file, create file if it doesn't initially exist
    out = Files.newOutputStream(path, CREATE, APPEND);

    // always create new file, failing if it already exists
    out = Files.newOutputStream(path, CREATE_NEW);

Dang it, these descriptions from different sources sound contradictory with each other!
APPEND in Sybex doesn't tell us that the open fails if the file didn't already exist unless combined with CREATE, something we'd want to remember.
Neither does JavaDocs, for that matter.  The usage notes cited last seem to make that clear tho, at least for newOutputStream...maybe used elsewhere they are different??

Come to think of it, it is confusing what happens if the only option passed is CREATE:

Sybex:
Create a new file if it does not exist.

And if it does exist, what?  Presumably truncate it and continue on, but maybe say that?

JavaDocs:
public static final StandardOpenOption CREATE
Create a new file if it does not exist. This option is ignored if the CREATE_NEW option is also set. The check for the existence of the file and the creation of the file if it does not exist is atomic with respect to other file system operations.

Also unclear what happens if we issue CREATE, WRITE tho from my general experience with files in different systems I am guessing it would clobber/truncate...something should explain that more clearly.  These are all different behaviors.

I was rather frustrated, then I remembered this instructional video that made me stop fretting about exactly what the options meant:
https://www.youtube.com/watch?v=2f5MvVx8RM8
The options being explained are from Python, but seem to apply here.

[Jesse Silverman]

My confusion may go deeper than I realized, I decided on a long quiet dog walk.

When you try to open a resource that someone else already has open (in general), that can either fail, imply a close on whomever has it open, or you both have it open at that point.

Normally, I feel like when you open something for write that is already open for write, the most common thing is to fail.
Second-most common might be closing them, unexpectedly to them, now it is yours.
It is possible in some systems that you both are open for write, but I feel that is least common.

I don't know which case applies given StandardOpenOption usage as in this thread.
That's pretty basic, and I am not sure.

I believe the "Old NIO" with channels let you have multiple writers/output streams on a channel open simultaneously?
I am not sure and don't expect to be using them, I know they aren't on the 819 exam.
For the stuff that is, if a resource is already open for write, I was thinking a second or subsequent attempt to open them for write would fail, but in explanations of their meaning in terms of channels it seems otherwise.

Multiple opens for reads I think are commonly okay (in general) if nobody has that resource open for write, but I thought even there in many systems you can't open it for read unless the last write was closed...

I am still pretty sure an existing resource, not opened by anyone, would still get append/truncate behavior, so my complaint about Sybex being wrong/misleading would still apply.
Because I can't tell what the behavior is expected to be if someone has the resource open for write, I still feel the JavaDocs are un-specific or un-clear on that.
Maybe it is different when using channels than when not?

Anyway, I confess to being confused, despite thinking I know all the possible behaviors that might be the case from working with various kinds of file-related and stream-related resources in tons of different systems.  I am not sure how to get clear on this without writing a ton of programs to reverse engineer the behavior, or getting schooled by someone who knows these API's inside-out rather than just repeating the JavaDocs that left me confused so far.

Thanks, anyone.

[Jesse Silverman]

I looked around on the web, and while there is a lot of cool stuff about NIO and NIO.2 which are apparently huge topics in their entirety, I didn't find answers to the confusion above.

For example, this helpful page tells less than I got on my own:
https://careydevelopment.us/blog/java-nio-how-to-avoid-overwriting-an-existing-file-when-creating-a-new-file

This told me some cool new stuff I didn't know, but didn't give any deeper insight into the particular options that had descriptions which confused me as ambiguous:
https://www.baeldung.com/java-file-options

I saw some noisy stuff on StackOverflow which suggests the usage examples I saw tell the truth and both the Sybex and JavaDocs descriptions I was confused by could use some work.
In particular, the file certainly doesn't need to be already open for write, but also the weirdness about how if you want to append-if-exists and create if not, and don't fail either way you specify both CREATE and APPEND, but other whether other combinations might be necessary or invalid left somewhat under-documented.

[Campbell Ritchie]

I presume you have been through the Java™ Tutorials.

[Jesse Silverman]

Campbell Ritchie wrote:I presume you have been through the Java™ Tutorials.

I perceived them as being a little too centered on Channel stuff I was trying to avoid, but they do have the neatest summary of the confusing options of any place I've yet seen:

WRITE – Opens the file for write access.
APPEND – Appends the new data to the end of the file. This option is used with the WRITE or CREATE options.
TRUNCATE_EXISTING – Truncates the file to zero bytes. This option is used with the WRITE option.
CREATE_NEW – Creates a new file and throws an exception if the file already exists.
CREATE – Opens the file if it exists or creates a new file if it does not.

These brief descriptions avoid the various confusing weird implications of the definitions I had complained about.

[Stephan van Hulst]

I wanted to write a short and sweet summary for you, but I see that the latest descriptions you quotes hit the nail exactly on the head.

Note that if you use methods from the Files API, some of the options may already be implied.

For instance, if you call Files.newBufferedWriter() and you want to overwrite a file if it exists, I think you only have to specify CREATE and TRUNCATE_EXISTING, because WRITE is implied by the method itself.

[Jesse Silverman]

My confusion is at a minimum of any point since I saw the table in the Sybex book.

Perhaps this should be cross-posted in the Certifications forum or I should do a new Post with a link here because there is no way I can correlate what is written in the book with what I now believe to be how things work?

In particular, I find the following state of affairs not to be immediately obvious to someone who has opened and written ten million files on Mainframes, Unix machines and Windows boxes in Pascal, FORTRAN, C++, Perl, Python etc.:

   // append to an existing file, fail if the file does not exist
   out = Files.newOutputStream(path, APPEND);

   // append to an existing file, create file if it doesn't initially exist
   out = Files.newOutputStream(path, CREATE, APPEND);

The tutorial phrases it best of the bunch, but still needs these examples to understand (for me at least) what happens with the combinations of options illustrated above:
WRITE – Opens the file for write access.
APPEND – Appends the new data to the end of the file. This option is used with the WRITE or CREATE options.
TRUNCATE_EXISTING – Truncates the file to zero bytes. This option is used with the WRITE option.
CREATE_NEW – Creates a new file and throws an exception if the file already exists.
CREATE – Opens the file if it exists or creates a new file if it does not.


The Javadocs were more confusing than the tutorial (and more likely to be consulted!) it sounds like they only apply if the file has already been opened for write access, maybe, and doesn't help you realize that if you want "Create if file not found, append if it is" you need to explicitly specify BOTH CREATE and APPEND to work:
APPEND
public static final StandardOpenOption APPEND
If the file is opened for WRITE access then bytes will be written to the end of the file rather than the beginning.
If the file is opened for write access by other programs, then it is file system specific if writing to the end of the file is atomic.

TRUNCATE_EXISTING
public static final StandardOpenOption TRUNCATE_EXISTING
If the file already exists and it is opened for WRITE access, then its length is truncated to 0. This option is ignored if the file is opened only for READ access.

Also, "already exists and it is opened for WRITE access" sounds like the file is currently open for write when the command is issued.  I don't even know if that is legal, but it certainly is not required.
Secondly, if you use or default the WRITE option, APPEND will FAIL if the file doesn't exist.  Not sure what happens if someone decides to specify TRUNCATE_EXISTING as the sole option...

The original Sybex description (which sent me on the wild goose chase) is the most confusing of the lot:

APPEND If the file is already open for write then append to the end.

TRUNCATE_EXISTING If file is already open for write then erase file and append to beginning.

Presuming I now understand all the options, I don't see how we can square these brief descriptions with actual reality.  It most definitely sounds like they only apply to files already open for write before the command is issued, which ain't true.  They share the ambiguities about how to do "Append if exists, else create" and have it work but add their own confusion by saying "If the file is already open for write".

[Campbell Ritchie]

Jesse Silverman wrote:. . . cross-posted in the Certifications forum . . .

Don't do anything; I can add this thread to a secondary forum in a few seconds.

The tutorial phrases it best of the bunch, but still needs these examples . . .

Unfortunately the Java™ Tutorials are very laconic on some subjects; this is obviously one of them

The Javadocs were more confusing than the tutorial . . .

Yes, I see what you mean. I think APPEND says, “if,” when it should say, “whether,” too. Also, the documentation is definitive; if there are any discrepancies between that and the Java™ Tutorials, you must assume the Java™ Tutorials are in error.

[Jesse Silverman]

Perhaps ironically, the confusion stems from sloppy uses of tenses:

If the file already exists and it is opened for WRITE access,

Sounds ambiguous, was it already open for WRITE access before the command came, or just in the command providing the option just now?
If they just added the word 'being' to yield:

If the file already exists and it is being opened for WRITE access,

The Sybex book goes one further/worse, and states incorrectly that the file WAS already open for WRITE access before the command came along.  A restriction that leaves out the extremely common case of being the first/only open of an existing file passively sitting there in the file system.

These ambiguities wouldn't arise in several other languages that are more careful with tenses, because they have no way to write a valid sentence ambiguous with respect to when the different conditions came into being.

Regardless, the language in English results in sentences that don't tell you what the story is unless you already knew the answer before you read them.
I think that ambiguous sentence resulted in the Sybex wording picking up on the wrong meaning of the two.

[Jesse Silverman]

Cleaning up all my errata reports (those of others I missed and my own almost-reports)...should we be posting this to the OCPJP forum for the authors, if for nothing else, their Java 17 re-write?

It confused me a fair amount when I hit it and tho I learned a LOT, it did consume a lot of time I could have spent studying other stuff...

Cheers,
Jesse