aspose file tools
The moose likes Java in General and the fly likes Load 20000 String Objects into Memory Big Moose Saloon
  Search | Java FAQ | Recent Topics
Register / Login
JavaRanch » Java Forums » Java » Java in General
Reply Bookmark "Load 20000 String Objects into Memory" Watch "Load 20000 String Objects into Memory" New topic
Author

Load 20000 String Objects into Memory

Ankittt Shah
Greenhorn

Joined: Aug 19, 2008
Posts: 6
Hi All,

How to load 20,000 String Objects into a HashSet and keep it those Keep that into memory so any class can find information from memory.

Those string objects will be like real long Strings.

ABCCORPORATIONABC
MYNAMEISEARLHIKKI
MYNAMEISABCDABCD

There are 20,000 strings like this always loaded into memory and i want to do a patter matching on them, for example a request comes in to find string 'NAME' in the memory, so it will be like a wild card match, i should find following two records in the above sample. [MYNAMEISEARLHIKKI, MYNAMEISABCDABCD]


What will be the memory implications and performace implications of storing 20,000 records which are 150 to 200 character long and doing a find pattern matching on them, can you give me a simple code to load strings into memory
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

what is 20000*200*16 bits?
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 32599
    
    4
Originally posted by David O'Meara:
what is 20000*200*16 bits?
A lot less than the capacity (usual heap size) of the JVM.
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19216

Actually not a lot less. It's 64,000,000 bytes, and the JVM has a default of 64MB (67,108,864 bytes). However, the JVM also needs memory for its default footprint (classes etc) and other objects, and will therefore quite easily exceed the 64MB. Therefore, you will need to specify a larger maximum heap size.


SCJP 1.4 - SCJP 6 - SCWCD 5
How To Ask Questions How To Answer Questions
Satish Chilukuri
Ranch Hand

Joined: Jun 23, 2005
Posts: 266
Originally posted by Rob Prime:
Actually not a lot less. It's 64,000,000 bytes, and the JVM has a default of 64MB (67,108,864 bytes).


Good old bits/bytes mix up . Its 64,000,000 bits. So it will take around 8MB.

Ankitt, why don't you write a sample program and see how it is performing?
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19216

Originally posted by Satish Chilukuri:
Good old bits/bytes mix up . Its 64,000,000 bits. So it will take around 8MB.



That was a pretty stupid mistake yeah.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12265
    
    1
When you actually write the code, you will be surprised at how compact the serialized object is and how fast Java can create a HashSet from the serialized form.

My phonetic lookup demonstration has one dataset with 14 thousand keys and almost 60 thousand words. The serialized collection is only 871kb.

Bill


Java Resources at www.wbrogden.com
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Load 20000 String Objects into Memory
 
Similar Threads
find method, regex and predef characters
Running Out of Memory
Diff between equals() and == operator
NX: URLYBird / my approach of the reading problem
String problem