I need to build a service which is used for persistence of arbitrary data. Typically this may be structured data saved out from spreadsheets...we don't care what the data structure is, or even the format (may be json, xml, yaml etc). The service must be available over HTTP and have a WebDAV interface. The service must also send out update messages over JMS for persisted data (inserts and updates). Update messages must be consistent with the state of the database (i.e. the last update message sent must be consistent with the database in the database). We aim to complete small writes in under 100ms and reads in under 50ms.
Our current service is based on Slide, with an Oracle database, and is deployed in London only. The main problem currently is that the WAN link between Asia and London is awful (250ms round trip), so cross-region performance is not acceptable. All data from all regions must be available globally, as it is read/written from many regions.
Any recommendations on suitable architectures, open source components etc to achieve these requirements much appreciated. My current thinking is to have regional caches to improve read performance and force writes to be asynchronous on the client side, but I'd like to avoid forcing async writes if possible.
Jim: My current thinking is to have regional caches to improve read performance and force writes to be asynchronous on the client side, but I'd like to avoid forcing async writes if possible.
If a write is converted to async then how is the result of the write (success/failure) reported to the user? There will arise cases where the data becomes out of date by the time it is written to the server. This may lead to data corruption i.e. an older data over writes a newer data.
I am not so sure why the performance numbers you have given are inclusive of the network latency. I think the numbers must be for the server i.e. the persistence process and not the roundtrip time back to the client. Using a cache for reading is good and more professional but doing an async write is a bit dicey.
Why don't you use a distributed cache with write-behind technic. Fist level updates can be made directly into the cache itself. and the write behind technic to update to the database. this way sice you are wriing into the cache you should have a good latency.
Products like Tangosol and Gigaspaces will let you distribute geographically