Win a copy of Think Java: How to Think Like a Computer Scientist this week in the Java in General forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

super efficient non-rdbms data storage

 
Harry Slaughter
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Let's say i want to store rows of data. the primary key will be a timestamp and an int (short). the data will be an arbitrary length array of ints (short). the length of this array will actually probably be 64 and will be static.

these rows represent snapshots in time of the state of another object.

the application will take snapshots every N seconds and store that data. the user will then be able to view graphed out representations of that data over an arbitrary length of time.

so if the user is taking a snapshot every 1 second and wants to look at 1 month of data, we'd be talking 1*60*60*24*4, or 345000 data points. So we're talking fairly large number of rows. take snapshots every .1 seconds, etc...

what would be the most efficient method of storing this data in a non-volitile fashion that would allow for data lookups via a primary key?

i'm not familiar with what's available out there for java in terms of "flat" (ie non-relational) databases.

i'm new to java and my programming is pretty rusty to boot.

i'd appreciate any tips, articles, suggestions, etc..

thanks

harry
 
Stefan Wagner
Ranch Hand
Posts: 1923
Linux Postgres Database Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

to get easily rid of old files or store them in backups...
Opening a few hundret files for each query could be slower, than one real big file.
Depends much on the usage of the program...

btw.: what is 'super-efficient'?
Do you have a clear specification?
Time or size?

[ October 06, 2004: Message edited by: Stefan Wagner ]
[ October 06, 2004: Message edited by: Stefan Wagner ]
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic