NX: some questions concerning find-method

[Ulrich Heeger]

Hi everyone,
I have some questions concerning the findByCriteria-method. I have the URLYBird assignement.
1. Concerning following requirement:

// Returns an array of record numbers that match the specified
// criteria. Field n in the database file is described by
// criteria[n]. A null value in criteria[n] matches any field
// value. A non-null value in criteria[n] matches any field
// value that begins with criteria[n]. (For example, "Fred"
// matches "Fred" or "Freddy".)
public long[] findByCriteria(String[] criteria);

By implementation of this function, I haven't included the possibility to search for separated words within one field.
For example within the location field, we have "Bed & Breakfast", so if I search for "Br" I won't find it, but for "Be" or "B". Is that OK?
2. Concerning the validating of the String-Array in findByCriteria, delete, update, create.
I don't know if you guys have implemented a validating of the search criteria at the client side. But I think we have also to validate the search criteria at the server side. That's the same for the delete, update and create-method.(?)
My Example of the validating of the Date-String:

private boolean validateDate(String date){
boolean retVal = false;
String regDate ="^(19\\d\\d|20\\d\\d)\\/" +
 "(0[1-9]|1[0-2])\\/((0[1-9]|2\\d)|3[0-1])$";
Pattern p = Pattern.compile(regDate);
Matcher m = p.matcher(date);
if(m.find())
retVal = true;

return retVal;
} 

Because I have the validating code would be so long, I'm thinking about to create an own class, perhaps called DataValidation. Perhaps you can tell me about your own solutions.
It would be very nice if you could help me
Regards
Ulrich

[Bharat Ruparel]

Hello Ulrich,
Point-by-point response:
1. That is how I have it. You are right or we both are wrong.
2. Ulrich, the only validation that I do on the data on server is whether the record number is a valid record number, i.e., > 0 and <= total number of records; and check to see if it is not a deleted record.

I do have client side validation by using the JComboBox to choose from the drop-down list, this way the user is not allowed to enter nonsensical text.
In my opinion, you are going beyond the requirement by doing the server side validation such as you show in point number 2. I would like others to comment on it though.
Regards.
Bharat

[Arun Kumar]

I have the contractor assignment. But this is how i go about doing it.
1. Yes i believe thats how it should be and thats how my code works.
I have a name and location JTextfields (The assignment requires the CSR to search by name and/or location). On the client side i do not have any kind of validation to check what they enter in those text fields. Since the CSRs should be sensible enough to enter strings for name/location and not dates in those fields. To get a list of all the existing rescords, they just leave the name and the location fields empty and lick the search button.
2. On the server side, inside the findByCriteria method, i just check only for the delete flag, if deleted, proceed to the next record. (my recNo starts from 0). This method returns a long[] of all the matching records numbers.
Then have to call the readRecord(recNo) to read the corresponding records' data. Here i check for the delete flag of the recNo (if deleted throw RecordNotFoundException), then check the filechannel position for that recNo, if the position is greater than the filechannel.size() (throw RecNotFound) or if the filechannel.read(buf) cannot read the supposed length of the record (indicating corrupted record? throw RecNotFound) or if the read(buf) returns -1 (throw RecNotFound).
But these methods in my case are synchronized on the static instance of the RAF.
Arun

[Philippe Maquet]

Hi Ulrich,
1. OK
2. I'd like to say "good point", but how do you abstract the field type (none of the fields are typed in the schema) ? If you don't hardcode your test, you are OK too IMO even if, as Bharat, I think you go beyond the requirements.
The only tests I do (from all methods receiving a String[] record (or criteria) as parameter and besides the RecordNotFoundException check), concern the array length and the Strings lengths. But you know that I like simplicity.
Arun,

I have a name and location JTextfields

Did you think of the JComboBox solution ?
Regards,
Phil.

[Vlad Rabkin]

Hi,
Bharat:

In my opinion, you are going beyond the requirement by doing the server side validation such as you show in point number 2. I would like others to comment on it though.

Agreed.
Best,
Vlad

[Ulrich Heeger]

Hi everyone,
Thanks a lot for your help
Regards
Ulrich

[Arun Kumar]

Originally posted by Philippe Maquet:

Arun,

Did you think of the JComboBox solution ?
Regards,
Phil.

Phil,
I am just waiting to finish this stuff, and get it over with
And i dont have a SCWCD exam to take a break... (already got it)
After this i need to go on with SCBCD (and i know nuts about EJB). Unless what i have done will qualify me for an automatic failure, i think i will leave it as JTextFields
This is my requirement for the user interface

It must be composed exclusively with components from the Java Foundation Classes (Swing components).

It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user.

It must present search results in a JTable.

It must allow the user to book a selected record, updating the database file accordingly.

Thankyou for your suggestion though Phil
Arun

[Ulrich Heeger]

Hi Bharat,
you wrote:

I do have client side validation by using the JComboBox to choose from the drop-down list, this way the user is not allowed to enter nonsensical text.

I think I have misunderstood (like ever ) following sentence in the submission:

It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user.

Does it mean that we have only to consider the search combinations:
"location", "null", "null", "null", "null", "null", "null"
"null", "name", "null", "null", "null", "null", "null"
"location", "name", "null", "null", "null", "null", "null"
"null", "null", "null", "null", "null", "null", "null"
Because I've intepreted the requirement "It must allow the user to search the data for all records" that we would have a search field at the GUI, in which the user could type one criteria and it would search through all fields. But I think now that this sentence means we have to search for:
"null", "null", "null", "null", "null", "null", "null"
in which case all the records would be returned.
Is that right?
Thanks in advance
(I can remember once you wrote that you live in Boston, so I can wish you a good morning
Ulrich

[Andrew Monkhouse]

Hi Ulrich,
Quite apart from the issues of going outside specification, there is also the issue that your date check seems to allow invalid dates (such as the 31st September).
If you are going to check the date, then something like this might be more accurate:

    private boolean validateDate(String date) {
        boolean validDate = false;
        try {
            SimpleDateFormat df = new SimpleDateFormat("yyyy/MM/dd");
            df.setLenient(false);
            if (df.parse(date,new ParsePosition(0)) != null) {
                validDate = true;
            }
        } catch (Exception ignored) {
            // Ooops, eating an exception - don't do this in your submission    :wink:   
        }
        return validDate;
    }

Of course you could change the logic around so that validDate starts as being true, then anything that goes wrong will change it. Which would give the catch block something to do. And you probably don't want to be catching "Exception". But I have to leave something for you to do
Regards, Andrew
[ September 30, 2003: Message edited by: Andrew Monkhouse ]

[Ulrich Heeger]

Hi Andrew,
thank you for the new function
Before, I thought also to go in this direction, I was just a little bit afraid because the creation of the new object SimpleDateFormat serve only to check the corectness of the String date and has no further application, so I thought that the use of regex would be more direct
Regards
Ulrich

[Ulrich Heeger]

Hi Phil,
you wrote:

2. I'd like to say "good point", but how do you abstract the field type (none of the fields are typed in the schema) ? If you don't hardcode your test, you are OK too IMO even if, as Bharat, I think you go beyond the requirements.

What do you mean with "abstract the field type" and "hardcode your test"?
Thanks & Regards
Ulrich

[Ulrich Heeger]

Hi Arun, Bharat, Vlad, Phil, Andrew
I know I'm deviating from the topic of this thread, but I'm still a little bit unsure of my solution of RandomAccessFile.
Arun, you wrote:

But these methods in my case are synchronized on the static instance of the RAF.

I'm using a mutex to asure the thread safety and for certain functions, within this mutex, I will have a call to RAF. (Sorry, Bharat you have allready pointed me to test this implementation and I have done it, it works fine , but I don't have synchronized the access to RAF itself because I would have nested synchronization which increases the danger of dead-locking.
So my question, RAF uses another thread to access the db.-file or I'm fine with the implementation described above?
Thanks & Regards
Ulrich

[Philippe Maquet]

Hi Ulrich,

What do you mean with "abstract the field type" and "hardcode your test"?

Sorry to have been confused (but me speaking franzose ).
What I meant is this : "who" knows (is aware of), in the whole system (your application), that field "date" is a "date" field ? (Mmh it's still vague but quite involuntary ).
Did you add some "type" notion at MetaData level (something like myDateField.setType(MetaData.DATE)), in which case that field will get its data checked automatically (by Data, MetaData or even itself) because of the type you assigned to it ? Or did you hard-code the call to validateDate() for the fifth field because you know it's a date ? These were the questions, and the answers I expect are yes/no (I mean yes to the first one, and no to the second ).
In fact, I thought about extending MetaData in such a non db-persistent way (we cannot change the file format, but persistence can be achieved through application properties). We have a Date field, but also a Boolean one and a Currency one. It may be handy for the system to know those field types, including for the presentation tier (For all types, which is the best alignment by default (left/right/centered) ? How to present a boolean field : yes/no, true/false, check box ? etc.).
Best,
Phil.

[Philippe Maquet]

Hi Arun,

I am just waiting to finish this stuff, and get it over with
And i dont have a SCWCD exam to take a break... (already got it)

Nasty boy !
JTextFields are OK, but JComboBoxes probably will give a better user experience and they solve any criteria validation issue. In your instructions as in mine, the criteria values must match exactly. It means that entering "New Yor" instead of "New York" as location should return no record. With a JcomboBox, the user may just type "N", check that he got "New York", press enter ... and get the results he expects. Better, he can see which are the possible locations without performing any search ...
Mmh. Why did I write that JTextFields are OK ?!
Best,
Phil.

[Bharat Ruparel]

Hello Ulrich/Arun,
Please pay attention to Phil's last post:

JTextFields are OK, but JComboBoxes probably will give a better user experience and they solve any criteria validation issue. In your instructions as in mine, the criteria values must match exactly. It means that entering "New Yor" instead of "New York" as location should return no record. With a JcomboBox, the user may just type "N", check that he got "New York", press enter ... and get the results he expects. Better, he can see which are the possible locations without performing any search ...

I agree with him completely. Arun, this is not a big deal to implement. Plus the benifits that you get for doing this are substantial.
Ulrich, you wrote:

"location", "null", "null", "null", "null", "null", "null"
"null", "name", "null", "null", "null", "null", "null"
"location", "name", "null", "null", "null", "null", "null"
"null", "null", "null", "null", "null", "null", "null"
Because I've intepreted the requirement "It must allow the user to search the data for all records" that we would have a search field at the GUI, in which the user could type one criteria and it would search through all fields. But I think now that this sentence means we have to search for:
"null", "null", "null", "null", "null", "null", "null"
in which case all the records would be returned.
Is that right?

I agree with you provided that you take this into consideration:
1. Your server side find(String [] Criteria) method should be generic, i.e., it shouldn't hard-code the name and location fields.
2. I have hard-coded the name and location fields on the client-side. I think that you can use reflection mechanism to determine the search fields on the client side as well, but I am not doing that.
Regards.
Bharat

[Arun Kumar]

Phil,
Now i am confused .. i see the advantages of using JComboBox. One for the name, one for the location. Then have to have a 'hook' which has to update the JComboBox in case of new inserts.

Originally posted by Philippe Maquet:

In your instructions as in mine, the criteria values must match exactly. It means that entering "New Yor" instead of "New York" as location should return no record.

But the statement above is what caught me. I was thinking that if i enter "New" in my location field, it should fetch all the records where location starts with "New" - like "New York". Let me paste the interface .. and see if i am wrong.

// Returns an array of record numbers that match the specified
// criteria. Field n in the database file is described by
// criteria[n]. A null value in criteria[n] matches any field
// value. A non-null  value in criteria[n] matches any field
// value that begins with criteria[n]. (For example, "Fred"
// matches "Fred" or "Freddy".)
public long[] findByCriteria(String[] criteria);

Shouldnt the method return the record number which has "New York" as the location, if i search by entering "New" in my location?

[Arun Kumar]

Originally posted by Ulrich Heeger:

So my question, RAF uses another thread to access the db.-file or I'm fine with the implementation described above?

Hi Ulrich,
I am using multiple data instances and a single static RAF. So i cannot just synchronize the public methods. See Bharats reasons for it above. My public methods (other than lock/unlock which are synched on the WeakHashMap) are synched on this RAF. If my methods are not synched on RAF, then i will face a problem, where one of the methods might change the RAFs' position when another method is reading some bytes.
I believe in your case you have multiple data instances, single RAF handle and a single instance of a Datahelper class. Your public methods are not synched in your Data class, but the two methods in your datahelper class are synchronized (readFixedString and writeFixedString or something like it). In this case i dont see the need to synchronize on the RAF handle. Only one instance of the data class will have access to the datahelper class which will do the actual reading and writing, there by your RAF handle will be controlled by only that instance. So you will not have any dirty reads or dirty writes or dead locks.
In short what you have done should work and you dont have to sync on the RAF handle like me, as long as you use only one instance of DataHelper class and its methods are synchronized.
Arun

[Philippe Maquet]

Hi Arun,
I am writing my 396th post on this forum, often I ask myself why I write them, but this one at least should be worthy.

But the statement above is what caught me. I was thinking that if i enter "New" in my location field, it should fetch all the records where location starts with "New" - like "New York". Let me paste the interface .. and see if i am wrong.

That statement comes from your own post :

It must allow the user to search the data for all records, or for records where the name and/or location fields exactly match values specified by the user.

Must ... exactly.
"Exactly" means what it means, and for the "must" word, its definition (and consequences when not respected) are given in the instructions (if possible choose to read them again preferably in the morning, but in any case not just before going to sleep ).
The client implements an "exact-match-search", while your server-side DB-interface must implement a "wildcard-search". There is a trap, even a vicious one.
Fortunately, boths ways of searching are compatible, as an "exact-match" is just a sub-set of a "wildcard-match" (if "Fred" matches "Freddy", "Freddy" matches "Freddy" too).
IMO, it's an "automatic-failure-trap", too big to be just there "by chance", and I wouldn't play with it.

Now i am confused .. i see the advantages of using JComboBox. One for the name, one for the location. Then have to have a 'hook' which has to update the JComboBox in case of new inserts.

In case of new inserts ... updates and ... deletes. If you want, it's possible to implement some DataChangeListener server-side, sending (by callback) some DataChangeEvents to clients when they occur. But doing it, including the network callback support, does not worth while IMO. As Mark would say, keep it simple, some "Refresh" button in your search pane will do the job. Actually.
Best,
Phil.
[ September 30, 2003: Message edited by: Philippe Maquet ]

[Arun Kumar]

Hi Phil,
If only i can get my hands on those people who write this kind of specs
Thanks for pointing it out to me mate. Will implement JComboBox solution with a refresh button.

Arun

[Ulrich Heeger]

Hi Phil,
Thank you for your answer. We could continue to discuss in french, beside german it's my mother tongue (my mom is from Geneva and I grew up there, oui, je suis un p'tit suisse Do you live in France?
You Wrote:

Did you add some "type" notion at MetaData level (something like myDateField.setType(MetaData.DATE)), in which case that field will get its data checked automatically (by Data, MetaData or even itself) because of the type you assigned to it ? Or did you hard-code the call to validateDate() for the fifth field because you know it's a date ? These were the questions, and the answers I expect are yes/no (I mean yes to the first one, and no to the second

My programmation of the validating functionality isn't elaborated so far. I just wrote this validateDate()-method for the reason that I didn't want to post an assumption which I haven't tested before.(Because I made once before and I noticed that it isn't a good idea, leading to much confusion My main concern was the question if I should implement such a validating of each field or not.
Because you and the others said that it is beyond the requirements, I think I will pass on it (Should we document this decision?)
But you're right, it's not very elegant to hardcode it. If we wouldn' be reduced to this f.... interface, then it would be much easier to create an own Class Record with all setters and getters for the fields.
Greetings
Ulrich

[Ulrich Heeger]

Hi Arun, hi Bharat,
thanks for your help.
Concerning RAF and the Singleton DataSchema, I will create an own thread, because I see, that I have still more questions.
Greetings & Regards
Ulrich

[Philippe Maquet]

Hi Ulrich,

oui, je suis un p'tit suisse

... et moi un p'tit belge !(de Bruxelles)
Mmh, let's stop with french or Andrew won't be able to moderate us !
Are you still in Switzerland or in Germany like Vlad ?

Because you and the others said that it is beyond the requirements, I think I will pass on it (Should we document this decision?)
But you're right, it's not very elegant to hardcode it. If we wouldn' be reduced to this f.... interface, then it would be much easier to create an own Class Record with all setters and getters for the fields.

"Should we document this decision?". I don't know. Maybe just mention the lack of field types and the benefits they could bring. But it doesn't make much sense IMO to invest in extending MetaData with a fixed database file format we cannot change.
Best,
Phil.

[Ulrich Heeger]

Hi Phil,
Brussels, wow, that's cool, my brother and my sister are both living in Brussels and they try to convince me to move there too Sometimes I miss the frenchspeaking enviroment...
I live in Cologne, I didn't know that Vlad lives also in Germany

But it doesn't make much sense IMO to invest in extending MetaData with a fixed database file format we cannot change.

You're right, I think I won't invest more effort in this subject
Greetings
Ulrich

[Ulrich Heeger]

Hi Arun,

I am using multiple data instances and a single static RAF. So i cannot just synchronize the public methods. See Bharats reasons for it above. My public methods (other than lock/unlock which are synched on the WeakHashMap) are synched on this RAF. If my methods are not synched on RAF, then i will face a problem, where one of the methods might change the RAFs' position when another method is reading some bytes.
I believe in your case you have multiple data instances, single RAF handle and a single instance of a Datahelper class. Your public methods are not synched in your Data class, but the two methods in your datahelper class are synchronized (readFixedString and writeFixedString or something like it). In this case i dont see the need to synchronize on the RAF handle. Only one instance of the data class will have access to the datahelper class which will do the actual reading and writing, there by your RAF handle will be controlled by only that instance. So you will not have any dirty reads or dirty writes or dead locks.
In short what you have done should work and you dont have to sync on the RAF handle like me, as long as you use only one instance of DataHelper class and its methods are synchronized.

Thanks once more for your help, like I mentioned before, I willcreate a new thread, where I will described my issue more detailed with code samples, so it will clarify better my concerns (I hope so
Greetings
Ulrich

[Philippe Maquet]

We are all three neighbours : Vlad is in Frankfurt.

[Jim Yingst]

[Ulrich]: Because I've intepreted the requirement "It must allow the user to search the data for all records" that we would have a search field at the GUI, in which the user could type one criteria and it would search through all fields. But I think now that this sentence means we have to search for:
"null", "null", "null", "null", "null", "null", "null"
in which case all the records would be returned.
Is that right?
One point which does not seem to have been addressed here so far is your use of "null". The instructions refer to null, which is the null reference, rather than "null", which is a reference to a non-null String object four characters long. If you enter "null" as one of the criteria, your search should return only records in which the corresponding field begins with "null". If you enter null instead, your search will ignore that field, and return all fields which match the other criteria. So:

int[] recs = findByCriteria(new String[] {"null", "null", "null", "null", "null", "null"});

Matches no records (unless you happen to have one for which all fields begin with "null", which seems very unlikely).

int[] recs = findByCriteria(new String[] {null, null, null, null, null, null});

Matches all records.

[Ulrich Heeger]

Hi Jim,
of course, you're right, I wanted just clarify that these are arguments
sorry for confusion
Regards
Ulrich

[Ulrich Heeger]

Hi Arun,
I have one more question concerning your request:

if the filechannel.read(buf) cannot read the supposed length of the record (indicating corrupted record? throw RecNotFound)

Does it mean you catch IOException for throwing a RecNotFoundException?
Greetings
Ulrich

[Arun Kumar]

Originally posted by Ulrich Heeger:
Hi Arun,
I have one more question concerning your request:

Does it mean you catch IOException for throwing a RecNotFoundException?
Greetings
Ulrich

Hi Ulrich,
i use fc.position(calculated position value);
then i declare a ByteBuffer with a certain capacity, where the capacity is the length of a record.
then i use fc.read(ByteBuffer);
I throw record not found exception for 3 different cases, the IOException will be caught and thrown as a runtimeexception.
3 cases are -
1. When i move the filechannel pointer to a position beyond the size of the filechannel size, cos reading bytes from such a position will return -1.
2. when the record is deleted.
3. When i read from the filechannel into the ByteBuffer, and the number of bytes read is not equal to the capacity of the ByteBuffer.
Here, this case will not throw any exceptions. But when i try to read into a byte[] from the ByteBuffer, i might have BufferUnderFlow or BufferOverFlow runtime exceptions. I check for this case, even before it can hit the runtimeexception, and throw it as recordnotfound exception.
This is case is rare, and this can only happen when my calculation for the position is wrong and ends up somewhere in the middle of the last record, or there is a dirty write at the end of the file.
[ September 30, 2003: Message edited by: Arun Kumar ]

[Ulrich Heeger]

Hi Arun,
Thanks for your help
Regards
Ulrich