• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Efficient way of handling data in memory

 
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi All,

The client request datas are stored in multiple files.
My requirement is to handle the data in efficient way to process the client request. For this I have loaded the data in memory using java object during the startup of the server process. But it can not be scaled for larger information in memory.

To make it efficient I thought of storing the infrequently requested java object in file by interface. So that whenever in need it loads the data to memory and serve the response.

I am not sure will this approach scale or not.

Any one please validate the above approach or prove me solution to do alternate approach.

Thanks
Shan
 
Bartender
Posts: 1638
IntelliJ IDE MySQL Database Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What you are doing is called caching and persistence is one of the aspects of a cache solution.

From what you have described, it looks like you have huge user data in memory. In order to avoid memory problems, you want to remove less frequently used data from memory and store in a persistence store.

If this is what is your problem statement then a persistence enabled caching solution is the correct way. EHCache is one of the freely available cache library, it may be worth using it.
 
Ranch Hand
Posts: 194
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am confused by your question. Maybe I can clear some things up for you.

The client request datas are stored in multiple files.




1. Persistence requires Serialization. Serialization converts an object to a byte stream using writeObject() and readObject() and stores the bytes in a file. This is efficient, in comparison to storing an object with your own algorithm. That would be ridiculous.


To make it efficient I thought of storing the infrequently requested java object in file by




2. I think I understand your dilemma. For example, you have a properties file that stores user names and passwords, and you also have a file that stores a less requested object, say an object that represents a pool of conference rooms. The properties file is accessed many times to check logging information, and you don't need to store object states. It's fine to store the data is characters because the schema is simple. The conference room pool has many rooms which have states. Like I said storing them with your own algorithm is ridiculous, hence why Java provides you with Serialization.
Obviously, it would the quickest way to get some data.

3. As far as caching is concerned, it only matters in the context of data accessing. If your reading large amounts of data from a file, whether its many times or even once, its best to use a BufferedInputStream. Chunks of data will be read from the file instead of character by character. I'd rather access the hard drive and read a 1KB chunk once, than read a 16 bit character 8192 times.

I hope this helps.
 
shan raj
Ranch Hand
Posts: 42
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks Nitesh Kant & Ryan Beckett.

Your understanding of my question is exactly correct. if there is no side effect I will use object serialization way for persisting the data in file.
In parallel I would check EHCache mechanism.

Between, if I use the Serializable to persist and retrieve the data like below


1. Will it take more time compared to plain text file reading or any other drawbacks?
2. How to use BufferedInputStream here or the ObjectOutputStream itself take of buffered reading.

With Regards
Shan
 
Marshal
Posts: 79177
377
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
A database connection? Would that help?
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic