*
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes Extending Data class or not Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "Extending Data class or not" Watch "Extending Data class or not" New topic
Author

Extending Data class or not

Joakim Eriksson
Greenhorn

Joined: Sep 13, 2004
Posts: 25
In the specs it says that you should write a class (Data.java) that implements a certain interface (DBMain.java).
I'm ok with that.
I interpret it like this:
They should be able to take my Data.java class and run it through their test programs that uses a DBMain interface.

Esentially this has some implications:
1. The database must have an "open" method to open it
2. The database must have a "close" method to close it
3. As there is no method available to acquire an instance, it
must be possible to "new" this object.
4. By the specs it must be thread-safe and keep track of who (which client)is holding a lock. I can see no easy way of doing this with multiple instances of this class. Essentially Data should be used as a singelton. However, it is difficult to enforce this pattern strictly because of 3. above.

Ok this is fine and can be done...
Essentially I ended up writing several extra public methods in the Data class, then I started thinking. Hmm, maybe it should work the other way around also. Ie, if the DBMain is supposed to be using "protected variations" I should also be able to pick any implementation of Data.java that implements the specs to the letter and plug it into my app.
And away all the extra public methods went...

This implied:
1. No other public methods outside DBMain and Object
2. The database must open itself before reading/writing records
3. The database must close itself before it is gc

This can also be done, but because 1, it is very difficult to perform
efficient searching without resorting to storing all data in memory or using an intermediary file.

Hmm, anyone else thinking along these lines?
Most ppl just argue that it is fine to extend the Data class or have another class directly access the database, but is it really?

What if a RDBMS was implemented on the other side of this interface by someone else, and they insisted on using the new database with our app. It would be very sad if that meant rewriting our app, just because some custom public methods in the data layer, wouldn't it?
Anton Golovin
Ranch Hand

Joined: Jul 02, 2004
Posts: 476
I think providing private methods in Data class for io and caching all records at startup is the way to go. There are not that many records and even with thousands of records (@ 160 bytes per record), the footprint of caching is a few megabytes - arguably nothing for a dedicated computer on which the company should run its server.

I think that keeping it very simple is the way to go. My suncertify.db package has only 5 classes, including 2 exceptions.
[ September 13, 2004: Message edited by: Anton Golovin ]

Anton Golovin (anton.golovin@gmail.com) SCJP, SCJD, SCBCD, SCWCD, OCEJWSD, SCEA/OCMJEA [JEE certs from Sun/Oracle]
Joakim Eriksson
Greenhorn

Joined: Sep 13, 2004
Posts: 25
Hi Anton!

I'm putting my records in a Map in my adapter class (which also is a facade to Data and only using the DBMain interface*).
If I put the Map in Data, I'll have to violate "protected variations (PV)" to get efficient searches by exposing extra public methods in Data or other classes directly accessing the Data.

*Hmm come to think of it... I'm already violating that pattern by providing a possibility to change the database file path (required by specs). It says nothing about changing filepath in the interface specs of DBMain. Thus it is by necessity implementaion specific for Data.
Hmm, at least apart from that external influence by the filepath setting method, I'm using PV anyway. That method is not necessary if a default database name is used and the file is placed in the same directory as the jar file.

Anyway thanks for the reply ANton. I'm also using data in memory now, and search times are much more tolerable.

Cheers,

/J
Joakim Eriksson
Greenhorn

Joined: Sep 13, 2004
Posts: 25
Hi Anton!

I'm putting my records in a Map in my adapter class (which also is a facade to Data and only using the DBMain interface*).
If I put the Map in Data, I'll have to violate "protected variations (PV)" to get efficient searches by exposing extra public methods in Data or other classes directly accessing the Data.

*Hmm come to think of it... I'm already violating that pattern by providing a possibility to change the database file path (required by specs). It says nothing about changing filepath in the interface specs of DBMain. Thus it is by necessity implementaion specific for Data.
Hmm, at least apart from that external influence by the filepath setting method, I'm using PV anyway. That method is not necessary if a default database name is used and the file is placed in the same directory as the jar file.

Anyway thanks for the reply ANton. I'm also using data in memory now, and search times are much more tolerable.

Cheers,

/J
peter wooster
Ranch Hand

Joined: Jun 13, 2004
Posts: 1033
Originally posted by Joakim Eriksson:
In the specs it says that you should write a class (Data.java) that implements a certain interface (DBMain.java).
I'm ok with that.
I interpret it like this:
They should be able to take my Data.java class and run it through their test programs that uses a DBMain interface.

Esentially this has some implications:
1. The database must have an "open" method to open it
2. The database must have a "close" method to close it
3. As there is no method available to acquire an instance, it
must be possible to "new" this object.
4. By the specs it must be thread-safe and keep track of who (which client)is holding a lock. I can see no easy way of doing this with multiple instances of this class. Essentially Data should be used as a singelton. However, it is difficult to enforce this pattern strictly because of 3. above.

Ok this is fine and can be done...
Essentially I ended up writing several extra public methods in the Data class, then I started thinking. Hmm, maybe it should work the other way around also. Ie, if the DBMain is supposed to be using "protected variations" I should also be able to pick any implementation of Data.java that implements the specs to the letter and plug it into my app.
And away all the extra public methods went...

This implied:
1. No other public methods outside DBMain and Object
2. The database must open itself before reading/writing records
3. The database must close itself before it is gc

This can also be done, but because 1, it is very difficult to perform
efficient searching without resorting to storing all data in memory or using an intermediary file.

Hmm, anyone else thinking along these lines?
Most ppl just argue that it is fine to extend the Data class or have another class directly access the database, but is it really?

What if a RDBMS was implemented on the other side of this interface by someone else, and they insisted on using the new database with our app. It would be very sad if that meant rewriting our app, just because some custom public methods in the data layer, wouldn't it?


This is another area where they are trying to make you think. What I've done is to make a Singleton DatabaseFile class that defines the low level file access for the database. My Data class implements the DBAccess interface and also provides close() and metadata() methods that are not part of that interface. If you don't need them then you don't have to call them, for example my old DataProxy class doesn't use them. The Data class uses finalize if you don't call close. Nothing you can do with Data actually closes the DatabaseFile object, there is a DatabaseManager that is responsible for that. You can use DatabaseManager to open the DatabaseFile, but any attempt to use Data will do it by lazy instanciation anyway.

Given this architecture, its possible to instanciate Data and get an open DatabaseFile, and use it with just the DBAccess interface. It is also possible to have multiple Data instances, one per client command and to get metadata and explicitly close the Data instance. It is also possible to change the File used by the DatabaseFile without impacting any existing Data instances. The DatabaseFile.open() method uses the properties singleton to determine its configuration, so it takes no parameters.

Some of my test programs imitate the Sun test, by only using DBAccess.

You don't need to worry about efficient searching, reread you instructions about performance attributes. Also most operating systems will cache a large amount of file data for you.

I obtain thread safety by synchronizing all low level file access on the RandomAccessFile instance held by the DatabaseFile object. For record locking I use a seperate LockManager, also a singleton, that keeps a Map of longs and locks, which it synchronizes on. The lock contains a long cookie and a Data reference. this way when a Data instance closes or finalizes all its locks get released. All locking assumes that a lock is just an arbitrary positive long value, with no requirement that it be a valid record. Once I obtain a lock, I can validate the number as a record number and release it if it is invalid, before throwing RecordNotFound. This avoids the problems of nested synchronized blocks.

I know this is a long and rambling response, but I hope it answers some of your questions, as you seem to be headed down the same road I've taken. BTW I'm on my 4th iteration, using a rich client and socket i/o. I started using the thin client, an approach that I now believe is incorrect.
Anton Golovin
Ranch Hand

Joined: Jul 02, 2004
Posts: 476
I chose the thin client path, and so far, it has worked out ok. The problem arises with such perks as multi-threading, but I have solved all of them so far. There is also inflexibility with thin clients, but I do not export datafile schema to the client at all. I don't know if that is good or bad or neither.
Joakim Eriksson
Greenhorn

Joined: Sep 13, 2004
Posts: 25
Thanks for the input Peter and Anton!

Anyways, here goes (we all like rambling):

I have a threadsafe dataclass that is not strictly a singelton (it can be newed), although if there are multiple instances of it, locking won't work properly atm, so for all purposes it should be used as a singelton.

My problem is this:

I use remote iterators (that is I send only the first say 25) results of a search. by pushing corner buttons under and above the vertical JTable scrollbar you load the next or previous 25 search (query) results etc.

That means that I either have to
store query results server side
or
use an index (TreeMap), sorted the way I want to present query results,
with pointers into this index. Essentially providing all possible
queries as submaps of this map.

As searching the data directly from disc (with say 10000) records is pretty slow, I figured I needed the data in memory.
Another option is to store data in new data files on disk.

As I somehow have run into another barrier I might consider this.
The new barrier beeing Memory Usage.
Doh! but each record is only ~190 bytes on file.
Yeah, but it turns out each row when represented as a Value Object in turn containing objects (Date, Money, Integer, String etc) for each column graps a lot more than 190 bytes.
Adding then that I use a LinkedHashMap to store the ValueObjects with Integer(recNo) as key, I reached a whopping 40 MB of memory usage for 10000 records! Adding 100000 records created an outofMemory exception, and my heart fell.
Damn, why can't it just simply work at once!
Reading up it seems that LinkedHashMap uses extra memory (depending partly on loadfactor), and that Objects generally comes with overhead. I don't know the exact workings of memory allocation procedures for an Object on the heap, but possibly the Objects allocated might not be very densly packed (a guess). I'll try out using arrays and primitives also, to measure memory consumption difference, but...
My dilemma is I want them to be objects, so that they can implement Comparable and give my very neat code for auto sorting (see below):



[Andrew: Replaced each tab with four spaces so that it fits on a single horizontal page better]
[ September 15, 2004: Message edited by: Andrew Monkhouse ]
Joakim Eriksson
Greenhorn

Joined: Sep 13, 2004
Posts: 25
Sorry for my terrible layout, interpunctation and spelling.

/J
 
 
subject: Extending Data class or not
 
Similar Threads
[URLyBirdHotel 1.3.3] Can I add more interfaces on Data.java?
question about Data class
NX:Contractor About RemoteException
B&S: find/search
DBMain interface