• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Database file reader

 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy y'all, what y'all doin'?!

Guys, here's a simple tool I created that reads the .db file that is sent along with the SCJD assignments we currently have. This will particularly be helpful to people in the beginning of the development, which is when we have lots of doubts, like, "where should I start from", or "I think my .db file is corrupted", etc.

In order to correctly work, it is likely that a few changes will be required, and they should be done according to the database description, which can be found in the instructions.html file. For instance, some databases have 2 bytes storing each record flag; in this case, it would be necessary to change the value of the RECORD_FLAG_BYTES field to 2, as well as the VALID field.

Note that this code only reads the database file, that's why FileInputStream is being used. Also, I'd really really appreciate if you guys could test it and let me know if you run into any issue.

 
Rodrigo R. Branas
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Great tool!! It´s very useful for those who are performing the project and need to discover the size of each field and so on!!

Congratulations!!

I just did some ajusts like que size of some fields and everything becomes perfect!!
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Fala Rodrigo!!!

I'm glad you liked it! That's the idea. Since there are a few .db formats, you end up having to make some adjustments in order to work correctly. But just changing the number of bytes of the class-level fields should be enough. Did you have to have to change something else?
 
Rodrigo R. Branas
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I just changed the size of number of fields from 4 to 2 and the size of number of each field from 2 to 1.

Everything else worked perfectly...

Thanks
 
Yusuf Ziyad
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot, I just got my assignment (URLyBird 1.3.1) and tried your program and it worked great, I just had to change DATABASE_LOCATION and comment out package suncertify.db;

 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy, Mr. Zew. Welcome to JavaRanch!

That's the spirit! It shouldn't be necessary to change many things to use this tool. That's the idea!
Also, please take a look at an important administrative private message I just sent you.
 
owolabi legz
Greenhorn
Posts: 10
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Simply put, this tool is a life saver!

I was so confused about the database file I got from Sun for my SCJD assignment (URLyBird 1.1.2). I just downloaded the assignment a week ago and I have been trying unsuccessfully to make sense of the .db file I received. Until I ran across your tool, that is.

Please do what you have to do to make this tool popular on the ranch. I am sure other SCJD candidates will find it extremely useful.

Thank you.
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy, Owolabi!

this tool is a life saver!


Ok, I like this description!

When we download the project, we usually have lots of doubts, and it usually takes sometime to understand everything that has to be done. The idea of this tool is to offer a "kick off" to people that is starting with the development. With it, you can have an understanding of how the database is organized, and then you can start writing your own code!
 
J Flew
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Roberto

This tool was a great help.

 
Ulises Pulido
Ranch Hand
Posts: 81
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This guy is great, follow his posts he has created many tools that will help you on your assignment. Thanks Roberto
 
Grary Stimon
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm worried that I'm missing some simple approach to accessing B&S 2.1.1 .db file records...

Roberto's tool is the closest thing I've found to accessing the .db file provided with my B&S 2.1.1 materials. Somehow, though, I think it should be easier to skip right to the records and read data than Roberto's tool makes it appear. Maybe I feel this way because the Camerlengo, Monkhouse SCJD book's sample project has no code reckoning with RandomAccessFile file seeking.

Can anyone comment on how I should locate the first record in my B&S .db file, e.g., using ?

Thanks,

G
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy, Ulises!

Thanks for the words, partner! I feel really happy when I see a "passed" post here, and even more happy when I'm able to help with that!

Grary, my friend, this tool is just for you to have an idea of how the reading of the .db file can be done. It simply reads the .db file and displays its content on the console. I believe that, in your case, you'll need to have something a little bit more related to your particular implementation (such as getting the data of a given record, according to its number, etc). For this, you'll have to go through some thinking and implement your solution. For instance, in my implementation of the Data class, you can go directly to the location of a given record in the .db file:



Where the getFilePointerOffset method gets the offset of a particular record. For that, I have a formula that finds out what is the offset in the .db file of a given record (I use information, like, header length, record length, etc).

This tool is just for you to have an idea of how this can be done, so you can start thinking about your particular implementation.
 
vaibhav mishra
Ranch Hand
Posts: 168
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
great tool, I am jsut starting off with some help from Andrew's book, there's a lots of stuff , and information about database in the instructions file seem cryptic to me, but this tool helped, now I get a clear picture on what's in that db file
 
Bob Wheeler
Ranch Hand
Posts: 317
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Roberto for the tool. I had just to change it here, change it there and it worked. For instance I had to change the RECORD_LENGTH_BYTES to zero.

cheers
Bob
Ps. working on Early Birdy Stuff (real name: URLyBird)
 
Binu Johny
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Why is byteArray[i] & 0x000000FF necessary ?
 
Nicolas Zozol
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks a lot. I'll just have to understand all the byte/int stuff !
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy, Nicolas! Welcome to JavaRanch!

Glad to know this tool was helpful!!!
 
Nicolas Zozol
Ranch Hand
Posts: 33
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I used it like a booster. But I'm in love with Enums, so I'm changing quite everything :



in update() method :


But starting with your work is so helpful !
 
Naveen Narayanan
Ranch Hand
Posts: 114
Eclipse IDE Java Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Roberto for presenting this nicely readable.
It saved a lot of hard-coded numbers from my assignment.

Probably recordLength is a field missing from the db file I have.
Second 4 –byte from the file start is “4 byte numeric, offset to start of record zero”
And I feel its file pointer to data-section-start.

possibly recordLength is to be build from adding schema-description-section fieldLengths.

Am I right sir ?
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Howdy, champion!

Thanks Roberto for presenting this nicely readable.


You are very welcome, champion!

Now, I'm not sure if I got your point... could you please elaborate it a little bit better?
Which one is your assignment?
 
Naveen Narayanan
Ranch Hand
Posts: 114
Eclipse IDE Java Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks roberto

line no. 99,100 of DBFileReader.java

First 4 byte magic-cookie , next 4 byte record-length , right ?
My assignment says a Data File Format

Start of file
4 byte numeric, magic cookie value identifies this as a data file
4 byte numeric, offset to start of record zero

So, is that "start of record zero" pointing to Data section. (offset into file equal to "offset to start of record zero" value)
I feel probably my db format doesn't have a recordLength value directly given


 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting... I had never seen this description...

Well, I think you don't really have the record length info in your .db file. But that shouldn't be a problem. The only think is that you have to make sure that these 4 bytes related to the "offset to start of record zero" are considered when you read your database, so it is read correctly.

When you run the tool, are the records displayed correctly? Which one is your assignment?
 
Naveen Narayanan
Ranch Hand
Posts: 114
Eclipse IDE Java Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes Roberto,
Well, I think you don't really have the record length info in your .db file

correct, I feel its manageable by reading field by field.
When you run the tool, are the records displayed correctly? Which one is your assignment?

Exactly, I got a clean display of all the records when I run your DBFileReader , with only couple of changes like,

My assignment is URLyBird 1.2.2
Interestingly recordLength & "offset to start of record zero" are having same value by coincidence

 
Kedarnath Bhagwat
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Robert ,

I saw one post about using seek .
I don't see this in your tool .

Also I don't see FileInputStream having this method .

So will I need to use RandomAccessFile instead of FileInputStream .

Please help me !


 
Roel De Nijs
Sheriff
Posts: 9934
113
AngularJS Chrome Eclipse IDE Hibernate Java jQuery MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Kedarnath,

This tool is created for everybody to read the complete content of your database file, so an InputStream will be more than enough. For the real assignment, you will jump directly to the position of a record and then you will use the RandomAccessFile and its seek-method.

Kind regards,
Roel
 
Kedarnath Bhagwat
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Robert !
I am using your code .
Only change that I made is I am using RandomAccessFile .
I am able to fetch all the data .

I am getting the following error .
Exception in thread "main" java.lang.RuntimeException: The given file does not exist.
at scjd.DBFileReader.main(DBFileReader.java:184)
Caused by: java.io.FileNotFoundException: C:\db-1x3.db (Access is denied)

Robert, what changes I will need to make ?
I want to record the location for each record so that I will be able to use the seek command.
 
Roel De Nijs
Sheriff
Posts: 9934
113
AngularJS Chrome Eclipse IDE Hibernate Java jQuery MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Kedarnath,

Make sure your database file exists in the specified DATABASE_LOCATION and is accessible and readible.

Kind regards,
Roel
 
Kedarnath Bhagwat
Greenhorn
Posts: 20
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks !
I can read the file with Parameter "r" .
The problem is when I pass "rw"

It says Access Denied .
 
Cristiano Sganzerla
Ranch Hand
Posts: 44
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Naveen Narayanan wrote:Thanks roberto

line no. 99,100 of DBFileReader.java

First 4 byte magic-cookie , next 4 byte record-length , right ?
My assignment says a Data File Format

Start of file
4 byte numeric, magic cookie value identifies this as a data file
4 byte numeric, offset to start of record zero

So, is that "start of record zero" pointing to Data section. (offset into file equal to "offset to start of record zero" value)
I feel probably my db format doesn't have a recordLength value directly given




I didnt get the recordlenght. I got the offset too.

Offset means the number of bytes that need to be jumped, if you want to acess the first record.

Its redundant, because, if you put a variable that stores the number of bytes read until the first record, it should be the same as the content of these bytes.

the same as record lenght, that is the sum of the flag bytes with the sum of each field size.

both are redundant.

Roberto, i compared my tool with yours, and was almost the same. With few changes.

And, ahhh, i used the method



These shift operators, grrrrrrr

Long time ago since my first cert
I never used in my job and forgot almost everthing about this section





By the way

Great tool !!!




 
Michael Hilgert
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
I'm now studying for SCJD and found this nice thread.
Can I create the database for myself (ASCII-File) or do I need a special file-format (database)?

Thanks,
Michael
 
Roel De Nijs
Sheriff
Posts: 9934
113
AngularJS Chrome Eclipse IDE Hibernate Java jQuery MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You need an assignment that you have to purchase and it will come with a database file which you must use. Different assignment versions will (or can) have different different database files.
 
Michael Hilgert
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My question meant, that I want to try out standard code before getting the assignment.
Can I create my own file to test something or is it a special database-format?
 
Roel De Nijs
Sheriff
Posts: 9934
113
AngularJS Chrome Eclipse IDE Hibernate Java jQuery MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No special file format, just use a FileoutputStream or a RandomAccessFile to create your database file.
 
Michael Hilgert
Greenhorn
Posts: 17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
ok, thank you.
 
Olu Shiyan
Ranch Hand
Posts: 57
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
@Roberto, your DBReader is awesome - talk about plug n play code. My assignment is URLyBird 1.1.1 and I didnt need to change the size contants you have in your code ; it just worked and greatly familiarized me with the db file. Thanks.
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm very glad to hear that, Olu! Our goal here is to help people get the SCJD certification, and the idea of this tool is to give people a first step in the development. I'm glad it worked well, and remember that you can always count on us to solve your doubts!
 
Olu Shiyan
Ranch Hand
Posts: 57
Java Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator



Yup! This thread about the find() implementation in Data class and the filtering needed for the GUI has been very helpful as well.


Cheers.
 
Andy Gertjejansen
Greenhorn
Posts: 16
Eclipse IDE Java Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
wonderful!!
 
Roberto Perillo
Bartender
Posts: 2271
3
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Good to know you liked it, Andy! And welcome to JavaRanch!
 
Greg Funston
Ranch Hand
Posts: 81
Eclipse IDE Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Awesome tool Roberto. It has made my life easier already. I mistakenly put my file in the wrong place and fumbled around for a few minutes but I just changed the path and the package and voila, I had access. Now to evaluate the results and carry on. It would have taken me a while to develop my own reader so this is a great help.

Thanks,
Greg Funston SCJP
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic