File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes URLyBird DB Null Terminated Values Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Soft Skills this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "URLyBird DB Null Terminated Values" Watch "URLyBird DB Null Terminated Values" New topic
Author

URLyBird DB Null Terminated Values

Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
Hey Guys,

my URLyBird spec says that values inserted into the DB should be null terminated if they are shorter than the maximum length.

I have found that the data stored in the DB is not null terminated.

I plan on commenting on this in my documentation & that the previous system didn't implement the null termination of values but I am going to.

Is that what people who passed the test have done?

cheers
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Excuse my ignorance, but what does it to say a value is "null terminated"?

How do you tell if a value is null terminated? In the API for RandomAccessFile.html I don't see anything about null terminated.


SCJP (1.4 | 5.0), OCJP (6.0), OCMJD
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
Hey Sean,

Data File Format section: "null terminated if less that the maximum length..."

The RandomAccessFile doesn't provide null terminating operations.

The null terminating value is 0, placed at the end of values
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

In terms of the actual file contents it means that it ends with a byte representing the value zero?

I'm currently using the RandomAccessFile to handle my I\O with the database file. What would I need to use in order to be able to write back the null terminating value of 0?
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
Field values end with 0,

Use file.write(0);
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

the database given doesn't contain null terminated fields. all fields are padded with spaces to their maximum length and
this is the way i write too...


SCJP 5, SCWCD 5, SCBCD 5
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Cathal Mullan wrote:The RandomAccessFile doesn't provide null terminating operations.


If I'm understanding this correctly, null terminating operation simply means "write back the value zero". So couldn't I just use write from the class RandomAccessFile? According to it's JavaDoc it will write back a single byte value:

write

public void write(int b)
throws IOException

Writes the specified byte to this file. The write starts at the current file pointer.
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

anyway the only thing you actually change when running the program is the customer ID
and there's not null there (it's an 8 digit number, as per assignment)
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Jonathan Elkharrat wrote:anyway the only thing you actually change when running the program is the customer ID
and there's not null there (it's an 8 digit number, as per assignment)


Surely it depends on how you handle deleted records. How do you handle them?

Some people write back "empty" records in which case you would be changing more than the customer ID - you would be writing back empty strings.
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

just updating the deleted flag. (by the way, that's exactly what the hard disk do when you delete a file )
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
I set the deleted record to invalid just
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Jonathan Elkharrat wrote:just updating the deleted flag. (by the way, that's exactly what the hard disk do when you delete a file )


Ok, so for your approach it's true to say you aren't updating fields that may not fill their max length. But that's not true in general i.e. for people who are writing back "empty" records then they are updating fields that may not fill their max length?

Also, if you are over writing the records section of your data file with the contents of the cache then you are writing back out every single record (rather than updating the data in the file), so surely anyone adopting this approach would need to worry about null termination?

So the only people who don't need to worry about null termination are people who implement a solution that when writing back to the file they seek the location of each individual record in the file, and only update either the customer id or deleted flag and do not touch the fields that are strings?
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Cathal Mullan wrote:I set the deleted record to invalid just


Did you use a cache in your application to store the records and did you overwrite deleted records in the cache when creating new records in the application?
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

i think all people should worry, because even if you don't change anything else in your
program, you should fully implement the update method. i personally use the padding
method, and so did Roel, i think. i don't know about Roberto, though..
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
I didn't use a cache. The spec doesn't mention a cache so why bother?
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

it's easier for me to use with a cache, and this way i also don't have to deal with IOException.
but it's your choice, whatever you choose is OK..
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Yep, spec doesn't mention a cache. Just it makes the design cleaner. As Jonathan mentioned you don't have to deal with IOExceptions when carrying out operations on records if you use a map because all your operations are on objects in a Map.

I guess you are modifying the record directly in the file each time an operation is carried out on it in your application Cathal? So you simply move the file point to the location of the record in the file and update either the customer id or deleted flag depending on what operation was carried out in the application. So you wouldn't ever need to worry about null terminated values. Does this sound right?

But for people who are using a cache. The simplest approach is to find the start of the records in the data file and overwrite the entire records section with the contents of the cache. In which case you would need to worry about null terminated values?

Jonathan, how do you actually write the contents of your cache back to the file? Do you overwrite the entire records section in the file or do you simply update the customer id and deleted flag for each record? Or something different?
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
Yeah I update the contents of the file depending on the operation that is called. I have to worry about null terminated values because I am fully implementing the interface methods.

So you use a HashMap for the records & are writing them all to the file at once when you shutdown your DB?

If so what if the server crashed, you'd loose all the changes since it started?
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

Cathal Mullan wrote:Yeah I update the contents of the file depending on the operation that is called. I have to worry about null terminated values because I am fully implementing the interface methods.


Ah ok, very good. So then would it not be correct to say that everyone needs to worry about null terminated values regardless of whether they use a cache or update the data file directly for each operation? Because everyone must implement the interface fully - surely failing to implement the interface fully will result in failure?

Cathal Mullan wrote:
So you use a HashMap for the records & are writing them all to the file at once when you shutdown your DB?

If so what if the server crashed, you'd loose all the changes since it started?


Yep, I've a Map which I write to file on shutdown. If the server crashes then yes all changes since the server started will be lost. This is the approach has been taken by others and they have passed so it must be a valid approach. I believe they just documented in their decisions that information would be lost if the server crashed and that an enhancement could be implemented in the future to periodically write the contents of the cache back to the file.
Dennis Grimbergen
Ranch Hand

Joined: Nov 04, 2009
Posts: 144

Cathal Mullan wrote:
my URLyBird spec says that values inserted into the DB should be null terminated if they are shorter than the maximum length.

Does your spec really say that? In my URLyBird it only says that it contains 8 bit chars null terminated values. (which it doesn't, or I have no clue what null terminated means...)
Cathal Mullan wrote:
I have found that the data stored in the DB is not null terminated.

Right, it contains spaces (hex code 20).


SCJP, SCWCD, SCJD
Cathal Mullan
Ranch Hand

Joined: May 20, 2011
Posts: 118
My spec says:

"All text values, and all fields (which are text only) , contain only 8 bit characters, null terminated if less than the maximum length for the field"
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

A quick search shows this question has been asked a few times before.

The answer would appear to be: ignore the reference to null terminated values. Simply pad fields with white spaces and mention something in your choices.txt about choosing to ignore the reference the null terminated values. Simple
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2268
    
    3

Howdy, y'all.

Just to give you all my point of view, I also completed the field values with blank spaces and ignored the null terminator stuff. I also used a memory cache because this eases a lot the solution.


Cheers, Bob "John Lennon" Perillo
SCJP, SCWCD, SCJD, SCBCD - Daileon: A Tool for Enabling Domain Annotations
Sean Keane
Ranch Hand

Joined: Nov 03, 2010
Posts: 581

There was another viewpoint on one of the threads listed in the search I linked to in my previous post which seemed to hit the nail on the head.

The Data File Does Match the Schema...

Yes, the data file does not contain the null terminating characters - but that does not mean that the content does not match the schema. It does match the schema.

The data file contains spaces to fill out the entire field so there is no need to add the null terminating value i.e. there is nothing in the schema that says you cannot use empty spaces to pad out the entire length of the field, so you can do so and still match the schema.

Justifying Padding with Empty Spaces

When justifying this approach in your choices.txt the following excerpt from the Oracle instructions will be helpful:

The company's IT department has a data file that contains the essential information for the company, but because the data must continue to be manipulated for reports using another custom-written application, the new system must re-implement the database code from scratch without altering the data file format.


So if you choose to pad fields with extra empty spaces you can use the point highlighted in bold above as justification for inserting padded spaces i.e. reduce the risk of the other custom-written application being impacted by you removing the empty spaces and inserting the null terminated value.

Another Possible Solution

Another thread mentioned adding in the functionality to insert the null terminated value but always writing back fully padded fields. This would seem like the most correct approach - but probably not worth the effort.

I guess the idea is that your class that handles I/O with the data file would search for the null terminated value when reading the file and insert the null terminated value when writing if required. But the data you pass into the class that handles I/O would always contain fields that are padded out with empty spaces.

So in your application the code would operate as if you never had functionality to handle null terminated values - but if another application wanted to use the class that handles I/O to it could choose to pass in data that does not contain fields that are padded out with empty spaces and everything would still work correctly i.e. the null terminated value would be written to the file for fields that do not fill the max length of the field.

Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5596
    
  15

If I think back to my Applied Informatics education (almost 10 years ago, I'm getting old) I know a string in C was null-terminated. I guess this "null-terminated" thing is a left-over in the analysis (instructions) of a previous program (maybe written in C) and the main purpose of keeping this sentence in the instructions is nothing more than confusing you a bit (I think)

Because the given database file contains space-padded values, I padded values with spaces and just ignored this null-terminated thing (but of course documented this decision in choices.txt)


SCJA, SCJP (1.4 | 5.0 | 6.0), SCJD
http://www.javaroe.be/
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

Roel De Nijs wrote:If I think back to my Applied Informatics education (almost 10 years ago, I'm getting old) I know a string in C was null-terminated. I guess this "null-terminated" thing is a left-over in the analysis (instructions) of a previous program (maybe written in C) and the main purpose of keeping this sentence in the instructions is nothing more than confusing you a bit (I think)

Because the given database file contains space-padded values, I padded values with spaces and just ignored this null-terminated thing (but of course documented this decision in choices.txt)


try and guess the result of the following: (just to remind you '\0' = null)

Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5596
    
  15

Jonathan Elkharrat wrote:try and guess the result of the following: (just to remind you '\0' = null)

At this hour I don't want to think, so I just let Eclipse give me the result The output is Some ? String (? represents a character which can not be displayed) and if you want to copy-and-paste the complete string, only Some is pasted. The length of the String is 13, so it's a valid character
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

thats weird.
it should only print "Some"
"\0" is a null character that should terminate the string, as in C & C++.
(well, at least in my eclipse it did. maybe it didn't copy well? )
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

anyway, i think the reason for null terminating value is in case you overwrite a longer record with a small one.
if you don't pad the rest of the record, you must use a null character ('\u0000') so the printing will stop at the
end of the String. former C/C++ programmer should be familiar with that...
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5596
    
  15

Small test program



Compiled and ran without any IDE just old java and javac (version: 1.6.0_17) gives this output:
ABC DEF
Some String


Back to you, Jonathan
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

for me it print:
ABC
Some


my version:
java version "1.6.0_20"
on Linux (ubuntu maverick)

try with the '\u0000' instead of '\0' and tell me the result.
(so much for "write once run everywhere"...)
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

from the command line (not IDE) it print indeed:
ABCDEF
Some String.

sachin verma wrote:Don't use IDE to run these type of program.
Run on command line and see the output.
It is a bug in JCreator.I don't know about other IDE
[ May 07, 2008: Message edited by: sachin verma ]

http://www.coderanch.com/t/268680/java-programmer-SCJP/certification/happens-when-we-print-null
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5596
    
  15

Jonathan Elkharrat wrote:try with the '\u0000' instead of '\0' and tell me the result.

Exactly the same (I'm on a WinXP machine).

You should never use an IDE when you are playing around When doing tech review for the SCJP 6 Practice Exams one question had to be replaced with another one, because when using Eclipse it worked as expected, but when using javac and java you got an exception (which violated the java specs) and because the advice for this certification is "don't use IDE, but javac and java" that question had to be deleted
Jonathan Elkharrat
Ranch Hand

Joined: Dec 31, 2008
Posts: 170

thanks for the advice.
for the final test i'll use only the console, of course..

this behavior is still weird, though..
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: URLyBird DB Null Terminated Values