• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

NX: [UrlyBird 1.3.1] how flexible with the data format should we be?

 
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I've created a fileloader that discerns the fieldNames and sizes from the data file. I'm trying to avoid hard-coding as much as possible, but at some point you must make assumptions about the data string array you read in.
I'm converting the String array into a business object to by-pass the majority of hard-coding, and also plan on passing the field names straight to my table-model. My file reading and writing mechanism doesn't make any assumptions about the data taking all its settings (record length & field sizes) from the schema.
I just can't see how you could make it 100% flexible using this approach, at the end of the day my business object has to know that position n in the String[] its passed is a name/location/size etc.
You could make it a JavaBean and use reflection to set the values, i.e. field name [0] is name, so instantiate a setName(String s) method and call it. But this complexity would be completely by-passed if a new field were to be added, though it would cope with re-ordering.
If I by-passed the idea of this business object, I still need to know that position n is a date that needs to be checked against.
I think that it would be very difficult to write this with 100% flexibility on the data layout. Every system needs to know how the data is stored surely.
So what I've done is to keep all of knowledge about the data layout in my business object, that way should the file change, then yes unfortunately a recompilation would be needed, but it would be an easy piece of maintenance.

Just looking for feedback on this issue. What have others done on this and does my approach sound reasonable? If only we could serialize objects to and from file like Max could

thanks guys!!!
James
 
james render
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
does this make any sense or is it my usual tired ranting and waffle??
okay time to watch some tennis... come on henman!!
 
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I can relate to this. My decision was to provide an interface that the database or client could query for information about the schema. Since the field names and positions can be parsed from the data file header, you can return a list to the GUI so it knows which columns contain which fields. This allows it to be flexible in case the data file adds additional fields later.
I wrote a specific implementation of this interface for the UrlyBird schema, which implements the methods that the database or client might use to get information about the file format (number of fields, record length, etc). The database doesn't care what fields are in the schema, and the GUI can work with any data file that has its minimum set of fields ("city", "hotel", etc.)
[ June 26, 2003: Message edited by: Perry Board ]
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So, does this business object class get used server-side, or is it just attached to the GUI client (as the model part of MVC)? I favor the latter if possible - I've been able to keep the networking classes completely ignorant of the nature of the data they're shipping around, other than that a record has an int record number, and its data are represented as an array of Strings. And there's a Metadata class (immutable, Serializable) representing the info read from the file header - a network client requests a copy of the MetaData from the server, and then looks to see which column is which, and what the allowed lengths should be. The Contractor requirements never really require the data to be treated as anything other than Strings (no Date interpretation for example), though I can imagine future enhancements that would change this. (E.g. searching for a pay rate range range should be done numerically, not alphabetically.) But this can still be done as part of the model adapter.
My goal is: if a customer said "hey, we added a column to the data file, changed the length of some fields, swapped the column order around, and changed one of the column names, what changes are necessary?" the ideal answer is "none". Well, changing column names is a problem, since that's how I identify which column is which. Three columns have specific roles: name, location, and cusotmer ID. If the Metadata doesn't have columns with the appropriate names, it's a fatal error - gotta adapt the client to look for a different column name. As for other columns though - the GUI tries to display the columns in a particular order if they're available, but if the're missing, they just appear as blank; the program can still functions. I considered displaying any new columns at the right side of the table, but decided not to as it's unrequired work and the new columns might represent data that users aren't supposed to see. But it would be easy to modify the GUI client to take advantage of the new fields.
Another way to look at it - what if a customer said "hey, we've got some other files using the same sort or format, but instead of airline flights and seats, they represent hotel rooms, or home improvement contractors - can you whip up a program to handle those?" How many classes would have to change? For me, only classes in the MVC really need to know what the different fields represent. The network and DB classes should be completely unaffected.
Hope that helps...
 
james render
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks guys..
I think that I've got a similar solution. I encapsulate the String[] in my business object as soon as it emerges from the Data class. Its really a model implementation, so it will have responsibility for data validation. Its really the go-between the client and the server, the server generates them and the client consumes them. I guess it would have been possible to put this logic into the controller on the client side. Just used to working with business objects and not swapping loads of raw data around.
Jim, I take on board your points about changing the system for another booking function. Hmmm maybe I'll get my business object to implement an interface, that way should another function be required, you would just create a new business object implementing the interface, and tinker with the DataAdapter to create these instead.
Just wondering how far to take this. My major problem is that I used to work as a re-usable component engineer, and always had to consider future use, as a result I usually completely over-engineer every solution.
must go and recite my mantra, 'clean and simple, clean and simple' deep breaths..
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Just used to working with business objects and not swapping loads of raw data around.
This was my first inclination as well. And if they hadn't specified the DB interface they way they did, I probably would have done something like this. But the fact that they specified a generic interface implied to me that the company might want to attach other things to this interface that they hadn't told me about - and the fact that the file format had those header fields to specify column names and lengths implied that there might be other files for other applications using the same base format - so I tried to play the game on those terms.
Just wondering how far to take this.
Well, it's all subjective. I started designing from the DB interface out, and just avoided anything specific to "contractors" as long as seemed reasonable. If I'd come to a spot where that required some additional complexity to keep things generic, I would have switched over. But I was pleased with how many of the classes turned out not to require any contractor-specific assumptions.
A question - for the find() method (or whatever it's called in your interface) - is it specified in terms of Strings only? Where does the Date logic come in? It's quite possible that the Contractor requirements are easier to keep generic, as we don't really have anything that requires treatment as any data type other than String.
must go and recite my mantra, 'clean and simple, clean and simple' deep breaths..
Well, I'm probably not someone you should be talking to in that case. Well, I agree in principle that "clean and simple" is good - I just acknowledge that I too have a tendency to want to over-engineer something, so you should probably be suspicious of my advice for that reason.
 
james render
Ranch Hand
Posts: 72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks for your valuble comments Jim.
On UrlyBird 1.3.1 there is no additional complexity that requires handling anything other than strings on the find() method. However, there is some Date validation required, bookings can only be accepted within 48hrs of the room becoming available. I guess that this could still be dealt with by some GUI controller, but I thought that it would make more sense in a business object.
There is also the question of checking updates, it would be possible to do a check on an array position if (data[6].length > 0) type of thing, but this implies knowledge that position 6 is the record owner. Now I know its not entirely possible to check this without any data knowledge, but I have factored it all away into my business object so all data knowledge is nicely done in one place.
THen I read your comments about using the system for other purposes, using different business objects. This prompted me to factor the code again. Now my DatabaseAdapter has no specific knowledge about my concrete business class type (Vacancy) but rather deals with an interface (BusinessObject). When it comes to instantiating business objects it defers to a BusinessObjectFactory that currently produces vacancies. Updates and searches can work with this interface. THe theory being that should the system be used for another purpose, just introduce a new business object based on the interface, tweak the factory and hey-presto. I am quite happy with this solution.
BUT! have I gone overboard, now that I have introduced another layer on-top of my specific business object. I now have a business package with five classes in it. At the end of the day this is to help factor away a date and owner check! It is definately something that I wouldn't think twice about normally, but the old 'keep things simple for the junior programmer' factor is niggling away at me..
 
Greenhorn
Posts: 22
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi James,
I'm also doing URLyBird but version 1.2.2.
I think we have to validate the parameter data in the methods updateRecord(...) and createRecord(...). If we don't do it, it could be possible for example to write "01-01-2003" in the field date and so it will break at least one requirement

..., but because the data must continue to be manipulated for reports using another custom-written application, the new system must reimplement the database code from scratch without altering the data file format.


Regards, Maksim
 
reply
    Bookmark Topic Watch Topic
  • New Topic