Data file change and cache

[Uros Mesaric]

Hi,

When implementing URLyBird, did anyone thought about data file change during application run.
For example. I read data from file, manage that data in memory and when data changes I write it back to file. But what if other - "legacy" app that they have, adds a record or changes file in some other way?

What about caching of data. Is it better to suck all the data in server part memory and manage data there or read data from file for every select that comes from client side. Former might cause memory problems when there is too many records in file. That might be resolved using paging, but that adds complexity...

I know, I have to solve all this questions, but it helps me to think about by boring you with my problems.

[Roberto Perillo]

But what if other - "legacy" app that they have, adds a record or changes file in some other way?

This is what I have in my instructions:

"You may assume that at any moment, at most one program is accessing the database file; therefore your locking system only needs to be concerned with multiple concurrent clients of your server"

I assumed that this "one program" was my application.

What about caching of data. Is it better to suck all the data in server part memory and manage data there or read data from file for every select that comes from client side. Former might cause memory problems when there is too many records in file. That might be resolved using paging, but that adds complexity...

Given the number of records, I assumed that the database would never grow to a point that keeping these records in memory would become a problem. And as far as I could understand from the context of the application, they would be exchanging the database file every day (with new records) so I just mentioned that the application must be shut down before they do that.

[Justin Rundle]

what if other - "legacy" app that they have, adds a record or changes file in some other way?
I recommend you read your instructions carefully, as I pretty sure the instructions states, �you may assume only 1 application will access the data file�.

What about caching of data.
This is a design decision, neither is right or wrong and each have their pros and cons, however you should decide and more importantly you must be able to justify your decision.

Former might cause memory problems when there is too many records in file
WeakHapMap and WeakReference are great references for this problem.

[Uros Mesaric]

Thank you for suggestions. I will remove my CRC checking of file consistency.
I will have to reread spec, I usually hurry into implementation details.

Probably will go with "in memory" DB. Although real DB does not work like that.