• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

I need to serialize a 2gb file

 
Ranch Hand
Posts: 63
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have a big file that is 2gb and might even exceed. Will there be ne probs if i serialize it. Can u suggest ne other ideas.
 
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What do you mean exactly by "serializing a file?" In Java, serialization is something you do to an object, and a file isn't an object. What do you hope to achieve?
 
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Ni!
 
Nirmal Mekala Kumar
Ranch Hand
Posts: 63
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
sorry for the ambiguity ... i meant tht, can jvm handle such huge objects in the memory. What i need is a Collection (a hashmap or hashtable) that maps a file code to its URL. this collection should store information of atleast 4 million documents.
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I appologise for my previous reply, I was just amusing myself.

I was trying to indicate that we prefer users to UseRealWords and try to avoid abbreviations like ne, ur, tht and so on. We don't charge by the letter!

Dave
 
David O'Meara
Rancher
Posts: 13459
Android Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you're just storing file codes and URLs you don't really need the file in memory, do you? This should make the memory requirements vastly smaller.
 
Ernest Friedman-Hill
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Sun's 32-bit JVMs can hold somewhat less than 2GB worth of data in memory. Further, because characters are held as 16-bit Unicode values, a disk file's size might actually double when read in as a Java String. So serialization issues aside, you may have problems just assembling this data structure in the first place, unless you have 64-bit hardware.

Consider putting the data into a relational database (making sure, of course, that it can handle that volume of data) and manipulating it from Java without loading it all in.
 
Nirmal Mekala Kumar
Ranch Hand
Posts: 63
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you guys. It helped a lot. I guess I was too ambitious to increase the performance and wanted to eliminate the database connection overhead.
...and David I will take care not to use abbreviations again. I guess you noticed it already in this post ;-)
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic