This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes Developer Certification (SCJD/OCMJD) and the fly likes NX: [UrlyBird 1.3.1]  how flexible with the data format should we be? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Certification » Developer Certification (SCJD/OCMJD)
Bookmark "NX: [UrlyBird 1.3.1]  how flexible with the data format should we be?" Watch "NX: [UrlyBird 1.3.1]  how flexible with the data format should we be?" New topic
Author

NX: [UrlyBird 1.3.1] how flexible with the data format should we be?

james render
Ranch Hand

Joined: May 08, 2003
Posts: 72
I've created a fileloader that discerns the fieldNames and sizes from the data file. I'm trying to avoid hard-coding as much as possible, but at some point you must make assumptions about the data string array you read in.
I'm converting the String array into a business object to by-pass the majority of hard-coding, and also plan on passing the field names straight to my table-model. My file reading and writing mechanism doesn't make any assumptions about the data taking all its settings (record length & field sizes) from the schema.
I just can't see how you could make it 100% flexible using this approach, at the end of the day my business object has to know that position n in the String[] its passed is a name/location/size etc.
You could make it a JavaBean and use reflection to set the values, i.e. field name [0] is name, so instantiate a setName(String s) method and call it. But this complexity would be completely by-passed if a new field were to be added, though it would cope with re-ordering.
If I by-passed the idea of this business object, I still need to know that position n is a date that needs to be checked against.
I think that it would be very difficult to write this with 100% flexibility on the data layout. Every system needs to know how the data is stored surely.
So what I've done is to keep all of knowledge about the data layout in my business object, that way should the file change, then yes unfortunately a recompilation would be needed, but it would be an easy piece of maintenance.

Just looking for feedback on this issue. What have others done on this and does my approach sound reasonable? If only we could serialize objects to and from file like Max could

thanks guys!!!
James


[SCJP][SCWCD][SCJD]
james render
Ranch Hand

Joined: May 08, 2003
Posts: 72
does this make any sense or is it my usual tired ranting and waffle??
okay time to watch some tennis... come on henman!!
Perry Board
Greenhorn

Joined: May 03, 2003
Posts: 29
I can relate to this. My decision was to provide an interface that the database or client could query for information about the schema. Since the field names and positions can be parsed from the data file header, you can return a list to the GUI so it knows which columns contain which fields. This allows it to be flexible in case the data file adds additional fields later.
I wrote a specific implementation of this interface for the UrlyBird schema, which implements the methods that the database or client might use to get information about the file format (number of fields, record length, etc). The database doesn't care what fields are in the schema, and the GUI can work with any data file that has its minimum set of fields ("city", "hotel", etc.)
[ June 26, 2003: Message edited by: Perry Board ]
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
So, does this business object class get used server-side, or is it just attached to the GUI client (as the model part of MVC)? I favor the latter if possible - I've been able to keep the networking classes completely ignorant of the nature of the data they're shipping around, other than that a record has an int record number, and its data are represented as an array of Strings. And there's a Metadata class (immutable, Serializable) representing the info read from the file header - a network client requests a copy of the MetaData from the server, and then looks to see which column is which, and what the allowed lengths should be. The Contractor requirements never really require the data to be treated as anything other than Strings (no Date interpretation for example), though I can imagine future enhancements that would change this. (E.g. searching for a pay rate range range should be done numerically, not alphabetically.) But this can still be done as part of the model adapter.
My goal is: if a customer said "hey, we added a column to the data file, changed the length of some fields, swapped the column order around, and changed one of the column names, what changes are necessary?" the ideal answer is "none". Well, changing column names is a problem, since that's how I identify which column is which. Three columns have specific roles: name, location, and cusotmer ID. If the Metadata doesn't have columns with the appropriate names, it's a fatal error - gotta adapt the client to look for a different column name. As for other columns though - the GUI tries to display the columns in a particular order if they're available, but if the're missing, they just appear as blank; the program can still functions. I considered displaying any new columns at the right side of the table, but decided not to as it's unrequired work and the new columns might represent data that users aren't supposed to see. But it would be easy to modify the GUI client to take advantage of the new fields.
Another way to look at it - what if a customer said "hey, we've got some other files using the same sort or format, but instead of airline flights and seats, they represent hotel rooms, or home improvement contractors - can you whip up a program to handle those?" How many classes would have to change? For me, only classes in the MVC really need to know what the different fields represent. The network and DB classes should be completely unaffected.
Hope that helps...


"I'm not back." - Bill Harding, Twister
james render
Ranch Hand

Joined: May 08, 2003
Posts: 72
Thanks guys..
I think that I've got a similar solution. I encapsulate the String[] in my business object as soon as it emerges from the Data class. Its really a model implementation, so it will have responsibility for data validation. Its really the go-between the client and the server, the server generates them and the client consumes them. I guess it would have been possible to put this logic into the controller on the client side. Just used to working with business objects and not swapping loads of raw data around.
Jim, I take on board your points about changing the system for another booking function. Hmmm maybe I'll get my business object to implement an interface, that way should another function be required, you would just create a new business object implementing the interface, and tinker with the DataAdapter to create these instead.
Just wondering how far to take this. My major problem is that I used to work as a re-usable component engineer, and always had to consider future use, as a result I usually completely over-engineer every solution.
must go and recite my mantra, 'clean and simple, clean and simple' deep breaths..
Jim Yingst
Wanderer
Sheriff

Joined: Jan 30, 2000
Posts: 18671
Just used to working with business objects and not swapping loads of raw data around.
This was my first inclination as well. And if they hadn't specified the DB interface they way they did, I probably would have done something like this. But the fact that they specified a generic interface implied to me that the company might want to attach other things to this interface that they hadn't told me about - and the fact that the file format had those header fields to specify column names and lengths implied that there might be other files for other applications using the same base format - so I tried to play the game on those terms.
Just wondering how far to take this.
Well, it's all subjective. I started designing from the DB interface out, and just avoided anything specific to "contractors" as long as seemed reasonable. If I'd come to a spot where that required some additional complexity to keep things generic, I would have switched over. But I was pleased with how many of the classes turned out not to require any contractor-specific assumptions.
A question - for the find() method (or whatever it's called in your interface) - is it specified in terms of Strings only? Where does the Date logic come in? It's quite possible that the Contractor requirements are easier to keep generic, as we don't really have anything that requires treatment as any data type other than String.
must go and recite my mantra, 'clean and simple, clean and simple' deep breaths..
Well, I'm probably not someone you should be talking to in that case. Well, I agree in principle that "clean and simple" is good - I just acknowledge that I too have a tendency to want to over-engineer something, so you should probably be suspicious of my advice for that reason.
james render
Ranch Hand

Joined: May 08, 2003
Posts: 72
thanks for your valuble comments Jim.
On UrlyBird 1.3.1 there is no additional complexity that requires handling anything other than strings on the find() method. However, there is some Date validation required, bookings can only be accepted within 48hrs of the room becoming available. I guess that this could still be dealt with by some GUI controller, but I thought that it would make more sense in a business object.
There is also the question of checking updates, it would be possible to do a check on an array position if (data[6].length > 0) type of thing, but this implies knowledge that position 6 is the record owner. Now I know its not entirely possible to check this without any data knowledge, but I have factored it all away into my business object so all data knowledge is nicely done in one place.
THen I read your comments about using the system for other purposes, using different business objects. This prompted me to factor the code again. Now my DatabaseAdapter has no specific knowledge about my concrete business class type (Vacancy) but rather deals with an interface (BusinessObject). When it comes to instantiating business objects it defers to a BusinessObjectFactory that currently produces vacancies. Updates and searches can work with this interface. THe theory being that should the system be used for another purpose, just introduce a new business object based on the interface, tweak the factory and hey-presto. I am quite happy with this solution.
BUT! have I gone overboard, now that I have introduced another layer on-top of my specific business object. I now have a business package with five classes in it. At the end of the day this is to help factor away a date and owner check! It is definately something that I wouldn't think twice about normally, but the old 'keep things simple for the junior programmer' factor is niggling away at me..
Maksim Golubkow
Greenhorn

Joined: Jun 04, 2003
Posts: 22
Hi James,
I'm also doing URLyBird but version 1.2.2.
I think we have to validate the parameter data in the methods updateRecord(...) and createRecord(...). If we don't do it, it could be possible for example to write "01-01-2003" in the field date and so it will break at least one requirement
..., but because the data must continue to be manipulated for reports using another custom-written application, the new system must reimplement the database code from scratch without altering the data file format.

Regards, Maksim
 
 
subject: NX: [UrlyBird 1.3.1] how flexible with the data format should we be?
 
Similar Threads
URLyBird: The Data File
Deleted Flag
URLyBird passed
Passed!
passed SCJD with 360/400