*
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes Validating the Data File Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "Validating the Data File" Watch "Validating the Data File" New topic
Author

Validating the Data File

Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Before you attempt to read from the data file, to what extend did you guys go to validate the data file?

At a very basic level you could just check that the magic cookie is the correct value.

You could check other things like:

* The file length is at least the length of the schema.

* If you pad fields with empty spaces (rather than using the null terminated character) then you could check that the entire file is the length of the schema plus a multiple of an entire record length.


SCJP (1.4 | 5.0), OCJP (6.0), OCMJD
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2258
    
    3

Although I think this would be necessary in the real world, I think this isn't really necessary for this assignment... what I did was verify if the magic cookie value present in the file being read was the same as the magic cookie present in the file that was provided to me in the assignment (257, if I make no mistake). But some people passed and didn't even consider doing that because this isn't a requirement. So, if you don't do any validation, then it is ok. Just don't forget to mention your choice in your choices.txt file because (in my opinion) this is an important point of the assignment.


Cheers, Bob "John Lennon" Perillo
SCJP, SCWCD, SCJD, SCBCD - Daileon: A Tool for Enabling Domain Annotations
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5126
    
  12

I just verified the value of the magic cookie. Nothing more, nothing less.


SCJA, SCJP (1.4 | 5.0 | 6.0), SCJD
http://www.javaroe.be/
Oladeji Oluwasayo
Ranch Hand

Joined: Sep 10, 2010
Posts: 101

I didn't do any validation. The only thing is that my file chooser (in the gui) comes with a filter that allows only .db files. If an invalid database file is chosen, the display will be blank. If the file does not exist (maybe a path was just typed into the database location textfield, the file will be created. I didn't even think of this validation.


OCPJP 6, OCMJD 6
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Thanks guys. I think I will just validate the magic cookie and ensure that the file is at least as long as the schema. Then any other issues with the file when reading I will just throw runtime exceptions as I process the file - for example if there are not enough bytes for a record.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5126
    
  12

On the server I only checked the magic cookie. At the client I tested a few other things: only .db extension, length bigger than 0, writable, readable,...

When something went wrong on the server while reading the file, an exception is thrown (which is caught by the client and the user gets an appropriate message).
Sarah Archer
Greenhorn

Joined: Jul 25, 2010
Posts: 19
I took a different view on this.

I didn't check my magic cookie because the instructions did specify which values were valid. It seemed to be more a version number, in hex 0x101, and perhaps compatibility is maintained for minor version number increments - you don't know.

I did check though that the record length = sum of field lengths - sizeof (deleted flag). We have enough information from the specification to know this must hold. You don't have to do it because it is not in the spec, but it is a simple check that will catch a lot of errors including bugs in reading the data file or a corrupt schema.

Finally, I wrote the code in a similar manner to the way I would write production code - lots of logging etc.
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Sarah Archer wrote:I did check though that the record length = sum of field lengths - sizeof (deleted flag). We have enough information from the specification to know this must hold.

Did you check record lengths when validating the file? I did think about putting this in. But decided against it. I think there was a thread about this before that suggested you can't be guaranteed what the record length is for an individual record, all you can be guaranteed of is its max length, but you aren't guaranteed that a record will always be its max length.

My instructions say:
Data section. 
Repeat to end of file: 
1 byte "deleted" flag. 0 implies valid record, 1 implies deleted record 
Record containing fields in order specified in schema section, no separators between fields, each field fixed length at maximum specified in schema information

But they also say:
All text values, and all fields (which are text only), contain only 8 bit characters, null terminated if less than the maximum length for the field.

Which seems confusing to me. The first set of instructions sound like they are saying that each record will fill out its maximum allowed space. Whereas the second set of instructions sound like records may be stored where the fields are not filled out to the maximum allowed space - they can be shorter than the max allowed space and will in such cases be null terminated.

My data file is set up based on my understanding of the first set of instructions. All fields are padded out with empty spaces in my data file. If I assume this will always be the case then I can easily validate the file by checking that the number of bytes in the file:


Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5126
    
  12

Sean Keane wrote:I think there was a thread about this before that suggested you can't be guaranteed what the record length is for an individual record, all you can be guaranteed of is its max length, but you aren't guaranteed that a record will always be its max length.

My program expects a fixed record length. If a file is provided where fields are null-terminated and not the length described in the database schema, an error will occur (because database file can't be read) or if it could be read successfully (with a bit of luck ) complete nonsense will be shown to the user.
David Byron
Rancher

Joined: Jan 20, 2009
Posts: 171

I checked the Magic Cookie, read the schema from the file, and then validated the remainder of the file against the schema. Validation checked total file size, correctness of flag on each record, and record count.

The schema-reading and validation were just about the first methods I wrote, and I used them mainly to check whether I was corrupting the database file as I developed the app. (I worked against the file directly rather than via a cache.) When the time came to submit my project, I saw no reason not to leave them as they were.


SCJD 6, OCPJP7, Baroque Potion, G+
 
 
subject: Validating the Data File
 
Similar Threads
NX: RandomAccessFile / Singleton DataSchema
B & S: stuck on calculating offset for updateRecord()
URLyBird DB Null Terminated Values
Relationship among Data class, RAF instance(s), and clients
URLBirdy Collections Advice