File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Beginning Java and the fly likes super efficient non-rdbms data storage Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "super efficient non-rdbms data storage" Watch "super efficient non-rdbms data storage" New topic

super efficient non-rdbms data storage

Harry Slaughter

Joined: Aug 16, 2004
Posts: 3
Let's say i want to store rows of data. the primary key will be a timestamp and an int (short). the data will be an arbitrary length array of ints (short). the length of this array will actually probably be 64 and will be static.

these rows represent snapshots in time of the state of another object.

the application will take snapshots every N seconds and store that data. the user will then be able to view graphed out representations of that data over an arbitrary length of time.

so if the user is taking a snapshot every 1 second and wants to look at 1 month of data, we'd be talking 1*60*60*24*4, or 345000 data points. So we're talking fairly large number of rows. take snapshots every .1 seconds, etc...

what would be the most efficient method of storing this data in a non-volitile fashion that would allow for data lookups via a primary key?

i'm not familiar with what's available out there for java in terms of "flat" (ie non-relational) databases.

i'm new to java and my programming is pretty rusty to boot.

i'd appreciate any tips, articles, suggestions, etc..



Harry Slaughter<br />proud java n00b
Stefan Wagner
Ranch Hand

Joined: Jun 02, 2003
Posts: 1923

to get easily rid of old files or store them in backups...
Opening a few hundret files for each query could be slower, than one real big file.
Depends much on the usage of the program...

btw.: what is 'super-efficient'?
Do you have a clear specification?
Time or size?

[ October 06, 2004: Message edited by: Stefan Wagner ]
[ October 06, 2004: Message edited by: Stefan Wagner ]
I agree. Here's the link:
subject: super efficient non-rdbms data storage
It's not a secret anymore!