File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes Database file reader Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "Database file reader" Watch "Database file reader" New topic
Author

Database file reader

Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy y'all, what y'all doin'?!

Guys, here's a simple tool I created that reads the .db file that is sent along with the SCJD assignments we currently have. This will particularly be helpful to people in the beginning of the development, which is when we have lots of doubts, like, "where should I start from", or "I think my .db file is corrupted", etc.

In order to correctly work, it is likely that a few changes will be required, and they should be done according to the database description, which can be found in the instructions.html file. For instance, some databases have 2 bytes storing each record flag; in this case, it would be necessary to change the value of the RECORD_FLAG_BYTES field to 2, as well as the VALID field.

Note that this code only reads the database file, that's why FileInputStream is being used. Also, I'd really really appreciate if you guys could test it and let me know if you run into any issue.



Cheers, Bob "John Lennon" Perillo
SCJP, SCWCD, SCJD, SCBCD - Daileon: A Tool for Enabling Domain Annotations
Rodrigo R. Branas
Greenhorn

Joined: May 13, 2009
Posts: 9
Great tool!! It´s very useful for those who are performing the project and need to discover the size of each field and so on!!

Congratulations!!

I just did some ajusts like que size of some fields and everything becomes perfect!!


SCJA, SCJP, SCJD, SCWCD and SCBCD Certified.
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Fala Rodrigo!!!

I'm glad you liked it! That's the idea. Since there are a few .db formats, you end up having to make some adjustments in order to work correctly. But just changing the number of bytes of the class-level fields should be enough. Did you have to have to change something else?
Rodrigo R. Branas
Greenhorn

Joined: May 13, 2009
Posts: 9
I just changed the size of number of fields from 4 to 2 and the size of number of each field from 2 to 1.

Everything else worked perfectly...

Thanks
Yusuf Ziyad
Greenhorn

Joined: May 19, 2009
Posts: 4
Thanks a lot, I just got my assignment (URLyBird 1.3.1) and tried your program and it worked great, I just had to change DATABASE_LOCATION and comment out package suncertify.db;

Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy, Mr. Zew. Welcome to JavaRanch!

That's the spirit! It shouldn't be necessary to change many things to use this tool. That's the idea!
Also, please take a look at an important administrative private message I just sent you.
owolabi legz
Greenhorn

Joined: Feb 22, 2008
Posts: 10
Simply put, this tool is a life saver!

I was so confused about the database file I got from Sun for my SCJD assignment (URLyBird 1.1.2). I just downloaded the assignment a week ago and I have been trying unsuccessfully to make sense of the .db file I received. Until I ran across your tool, that is.

Please do what you have to do to make this tool popular on the ranch. I am sure other SCJD candidates will find it extremely useful.

Thank you.

SCJP 5.0, SCWCD 5.0
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy, Owolabi!

this tool is a life saver!


Ok, I like this description!

When we download the project, we usually have lots of doubts, and it usually takes sometime to understand everything that has to be done. The idea of this tool is to offer a "kick off" to people that is starting with the development. With it, you can have an understanding of how the database is organized, and then you can start writing your own code!
J Flew
Greenhorn

Joined: Sep 25, 2008
Posts: 6
Thanks Roberto

This tool was a great help.

Ulises Pulido
Ranch Hand

Joined: Jul 24, 2008
Posts: 81
This guy is great, follow his posts he has created many tools that will help you on your assignment. Thanks Roberto


SCJP 5.0, SCWCD 5.0, SCBCD 5.0, SCJD, SCEA in progress
www.ulisespulido.com
Grary Stimon
Ranch Hand

Joined: Sep 30, 2008
Posts: 42
I'm worried that I'm missing some simple approach to accessing B&S 2.1.1 .db file records...

Roberto's tool is the closest thing I've found to accessing the .db file provided with my B&S 2.1.1 materials. Somehow, though, I think it should be easier to skip right to the records and read data than Roberto's tool makes it appear. Maybe I feel this way because the Camerlengo, Monkhouse SCJD book's sample project has no code reckoning with RandomAccessFile file seeking.

Can anyone comment on how I should locate the first record in my B&S .db file, e.g., using ?

Thanks,

G
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy, Ulises!

Thanks for the words, partner! I feel really happy when I see a "passed" post here, and even more happy when I'm able to help with that!

Grary, my friend, this tool is just for you to have an idea of how the reading of the .db file can be done. It simply reads the .db file and displays its content on the console. I believe that, in your case, you'll need to have something a little bit more related to your particular implementation (such as getting the data of a given record, according to its number, etc). For this, you'll have to go through some thinking and implement your solution. For instance, in my implementation of the Data class, you can go directly to the location of a given record in the .db file:



Where the getFilePointerOffset method gets the offset of a particular record. For that, I have a formula that finds out what is the offset in the .db file of a given record (I use information, like, header length, record length, etc).

This tool is just for you to have an idea of how this can be done, so you can start thinking about your particular implementation.
vaibhav mishra
Ranch Hand

Joined: Jun 18, 2008
Posts: 168
great tool, I am jsut starting off with some help from Andrew's book, there's a lots of stuff , and information about database in the instructions file seem cryptic to me, but this tool helped, now I get a clear picture on what's in that db file


SCJP
Bob Wheeler
Ranch Hand

Joined: Apr 24, 2009
Posts: 317
Thanks Roberto for the tool. I had just to change it here, change it there and it worked. For instance I had to change the RECORD_LENGTH_BYTES to zero.

cheers
Bob
Ps. working on Early Birdy Stuff (real name: URLyBird)


SCJP 6 - SCJD - SCWCD 5 - SCBCD 5
JavaEnterpriseEditionFaq - TomcatFaq
Binu Johny
Greenhorn

Joined: Jul 28, 2009
Posts: 1

Why is byteArray[i] & 0x000000FF necessary ?


boom boom pow ?
Nicolas Zozol
Ranch Hand

Joined: Oct 13, 2009
Posts: 33
Thanks a lot. I'll just have to understand all the byte/int stuff !


Robusta Web Library : Open Source for REST & Java
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy, Nicolas! Welcome to JavaRanch!

Glad to know this tool was helpful!!!
Nicolas Zozol
Ranch Hand

Joined: Oct 13, 2009
Posts: 33
I used it like a booster. But I'm in love with Enums, so I'm changing quite everything :



in update() method :


But starting with your work is so helpful !
Naveen Narayanan
Ranch Hand

Joined: Sep 11, 2007
Posts: 114

Thanks Roberto for presenting this nicely readable.
It saved a lot of hard-coded numbers from my assignment.

Probably recordLength is a field missing from the db file I have.
Second 4 –byte from the file start is “4 byte numeric, offset to start of record zero”
And I feel its file pointer to data-section-start.

possibly recordLength is to be build from adding schema-description-section fieldLengths.

Am I right sir ?

SCJP 5.0, SCJD, SCWCD
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Howdy, champion!

Thanks Roberto for presenting this nicely readable.


You are very welcome, champion!

Now, I'm not sure if I got your point... could you please elaborate it a little bit better?
Which one is your assignment?
Naveen Narayanan
Ranch Hand

Joined: Sep 11, 2007
Posts: 114

Thanks roberto

line no. 99,100 of DBFileReader.java

First 4 byte magic-cookie , next 4 byte record-length , right ?
My assignment says a Data File Format

Start of file
4 byte numeric, magic cookie value identifies this as a data file
4 byte numeric, offset to start of record zero

So, is that "start of record zero" pointing to Data section. (offset into file equal to "offset to start of record zero" value)
I feel probably my db format doesn't have a recordLength value directly given


Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Interesting... I had never seen this description...

Well, I think you don't really have the record length info in your .db file. But that shouldn't be a problem. The only think is that you have to make sure that these 4 bytes related to the "offset to start of record zero" are considered when you read your database, so it is read correctly.

When you run the tool, are the records displayed correctly? Which one is your assignment?
Naveen Narayanan
Ranch Hand

Joined: Sep 11, 2007
Posts: 114

Yes Roberto,
Well, I think you don't really have the record length info in your .db file

correct, I feel its manageable by reading field by field.
When you run the tool, are the records displayed correctly? Which one is your assignment?

Exactly, I got a clean display of all the records when I run your DBFileReader , with only couple of changes like,

My assignment is URLyBird 1.2.2
Interestingly recordLength & "offset to start of record zero" are having same value by coincidence

Kedarnath Bhagwat
Greenhorn

Joined: Oct 27, 2009
Posts: 20
Hi Robert ,

I saw one post about using seek .
I don't see this in your tool .

Also I don't see FileInputStream having this method .

So will I need to use RandomAccessFile instead of FileInputStream .

Please help me !



kmbhagwat , SCJP 6
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5408
    
  13

Hi Kedarnath,

This tool is created for everybody to read the complete content of your database file, so an InputStream will be more than enough. For the real assignment, you will jump directly to the position of a record and then you will use the RandomAccessFile and its seek-method.

Kind regards,
Roel


SCJA, SCJP (1.4 | 5.0 | 6.0), SCJD
http://www.javaroe.be/
Kedarnath Bhagwat
Greenhorn

Joined: Oct 27, 2009
Posts: 20
Thanks Robert !
I am using your code .
Only change that I made is I am using RandomAccessFile .
I am able to fetch all the data .

I am getting the following error .
Exception in thread "main" java.lang.RuntimeException: The given file does not exist.
at scjd.DBFileReader.main(DBFileReader.java:184)
Caused by: java.io.FileNotFoundException: C:\db-1x3.db (Access is denied)

Robert, what changes I will need to make ?
I want to record the location for each record so that I will be able to use the seek command.
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5408
    
  13

Hi Kedarnath,

Make sure your database file exists in the specified DATABASE_LOCATION and is accessible and readible.

Kind regards,
Roel
Kedarnath Bhagwat
Greenhorn

Joined: Oct 27, 2009
Posts: 20
Thanks !
I can read the file with Parameter "r" .
The problem is when I pass "rw"

It says Access Denied .
Cristiano Sganzerla
Ranch Hand

Joined: Jan 14, 2006
Posts: 44
Naveen Narayanan wrote:Thanks roberto

line no. 99,100 of DBFileReader.java

First 4 byte magic-cookie , next 4 byte record-length , right ?
My assignment says a Data File Format

Start of file
4 byte numeric, magic cookie value identifies this as a data file
4 byte numeric, offset to start of record zero

So, is that "start of record zero" pointing to Data section. (offset into file equal to "offset to start of record zero" value)
I feel probably my db format doesn't have a recordLength value directly given




I didnt get the recordlenght. I got the offset too.

Offset means the number of bytes that need to be jumped, if you want to acess the first record.

Its redundant, because, if you put a variable that stores the number of bytes read until the first record, it should be the same as the content of these bytes.

the same as record lenght, that is the sum of the flag bytes with the sum of each field size.

both are redundant.

Roberto, i compared my tool with yours, and was almost the same. With few changes.

And, ahhh, i used the method



These shift operators, grrrrrrr

Long time ago since my first cert
I never used in my job and forgot almost everthing about this section





By the way

Great tool !!!





SCJP 1.4 - SCWCD 1.4
Michael Hilgert
Greenhorn

Joined: Apr 23, 2010
Posts: 17
Hi,
I'm now studying for SCJD and found this nice thread.
Can I create the database for myself (ASCII-File) or do I need a special file-format (database)?

Thanks,
Michael


You're welcome.
SCJP 6, SCWCD 5
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5408
    
  13

You need an assignment that you have to purchase and it will come with a database file which you must use. Different assignment versions will (or can) have different different database files.
Michael Hilgert
Greenhorn

Joined: Apr 23, 2010
Posts: 17
My question meant, that I want to try out standard code before getting the assignment.
Can I create my own file to test something or is it a special database-format?
Roel De Nijs
Bartender

Joined: Jul 19, 2004
Posts: 5408
    
  13

No special file format, just use a FileoutputStream or a RandomAccessFile to create your database file.
Michael Hilgert
Greenhorn

Joined: Apr 23, 2010
Posts: 17
ok, thank you.
Olu Shiyan
Ranch Hand

Joined: Jun 10, 2010
Posts: 57

@Roberto, your DBReader is awesome - talk about plug n play code. My assignment is URLyBird 1.1.1 and I didnt need to change the size contants you have in your code ; it just worked and greatly familiarized me with the db file. Thanks.

SCJP 6, OCMJD6
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

I'm very glad to hear that, Olu! Our goal here is to help people get the SCJD certification, and the idea of this tool is to give people a first step in the development. I'm glad it worked well, and remember that you can always count on us to solve your doubts!
Olu Shiyan
Ranch Hand

Joined: Jun 10, 2010
Posts: 57




Yup! This thread about the find() implementation in Data class and the filtering needed for the GUI has been very helpful as well.


Cheers.
Andy Gertjejansen
Greenhorn

Joined: Dec 30, 2010
Posts: 16

wonderful!!


A+ 2009, OCPJP 6, (Working on: OCMJD 6)
Roberto Perillo
Bartender

Joined: Dec 28, 2007
Posts: 2266
    
    3

Good to know you liked it, Andy! And welcome to JavaRanch!
Greg Funston
Ranch Hand

Joined: Feb 09, 2011
Posts: 81

Awesome tool Roberto. It has made my life easier already. I mistakenly put my file in the wrong place and fumbled around for a few minutes but I just changed the path and the package and voila, I had access. Now to evaluate the results and carry on. It would have taken me a while to develop my own reader so this is a great help.

Thanks,
Greg Funston SCJP
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Database file reader