Problem with ArrayList !!!!!

[Naren Chivukula]

Hi,

When I use ArrayList in my program, it is working fine for small no.of objects which I'm going to add. But, when I come to add many objects say
around 444444 (as my appication demands), I'm getting error "java.lang.OutofMemoryError". As I debugged my code, I could come to know that the problem is with ArrayList. Though I specified the initial size of the ArrayList by giving a large number or using the method "ensureCapacity(intValue)", things went in vain. Suggest to how to overcome with this problem kindly.

Regards,
Narendranath.

[Jimmy Die]

Hi,


This is a memory error. You've loaded so many objects into memory on your machine that, well... your out of memory. It is not a out of capicity of the ArrayList, infact ArrayList will just increase capicity automatically. Try scaling back your ensureCapicity(444444) until you find a size that you machine can handle(or maybe add more ram ) .

Good luck!

[karl koch]

Hi

why your application demands 444'444 Objects to be created ? you probably wont use them all at the same time ? so just create the objects you need (e.g. 20 items displayed in a list and if the user scrolls down you discard the ones not visible and create the new ones).


k

[Doug UnspecifiedLastName]

I've had this exact problem in situations where the vm's memory size had been artificially limited. I don't know what the default is, but you can set it manually with something like:
java -Xmx1024 MyProgram (in megabytes I think)

The other solution is something like what Karl was proposing- implement caching. If the objects are large, you'd create a small container class for them which would stay in the ArrayList, control access to your objects, and serialize/reconstitute the "object"s as necessary. It's a little more complicated than that of course, you need a caching algorithm, etc. But it might be better than the alternative of drastically changing your application to not require all 444444 objects to be accessible at once.

You might also be able to force the VM to use more physical memory than the machine has and rely on the OS's virtual memory management to swap stuff out... not sure how this might work though.

[ December 03, 2004: Message edited by: Doug UnspecifiedLastName ]
[ December 03, 2004: Message edited by: Doug UnspecifiedLastName ]

[Doug UnspecifiedLastName]

Another solution still: build a RandomAccessFile.

[Naren Chivukula]

It's great for these many responses. But my application is to display calendar (I'm using swings). For smaller dates ranges I'm getting pretty well. But say if u give date range about 1900 to 2050, application will hang. As I told u I came to know that the problem is with arrayList. Put some more light in this problem.

Thanks and Regards,
Narendranath

[fred rosenberger]

What we're trying to tell you is the problem is NOT with ArrayList.

Every time you create a new object, a little hunk of memory is used. create 3 objects, you use a little. create 300, you use a lot more.

you are creating about 1/2 a million objects. if each object requires 100 bytes of memory (which is not a whole lot), you are using 50,000,000 bytes of memory for your OBJECTS. you then have the OS, the JVM, any OTHER objects you may create, and whatever other programs you have running claiming memory.

if you had used an array, an array list, or any other collection, you'd run into this problem.

The only thing you can do is a) create fewer objects, b) write them to a file and access (and release) them as needed, or c) put more memory in your machine.

[Naren Chivukula]

Hello Fred,

I'm very much satisfied with your solution depicted in a easy understandable way. Can you tell how can we use Random Access Files to solve my problem?

Thanks in advance,
Narendranath

[Nik lammar]

Use some methods to load dynamically the records.
Use an integer for pointer...
Load records every , for example, 300 records.
Then proccess them.
Then store the size of arraylist on integer pointer (using array.size() method) and clear your arraylist. ( array.clear() .
Then proccess next 300 records starting from record 301. etc...

This is a simple method to share proccess DATA in your MEMORY, PROCCESSOR and HARD DISK.
Use full proccess power of you computer, memory power and HD size, if you cant load the hole recordset...

[fred rosenberger]

the specific solution would depend on the specific problem. But basically what we're saying is that instead of creating all the objects in memory, create them and write them to a file. You want to use a random access file so that if you need to get, say, the 148,276th record, you can jump straight to it instead of having to read the first 148,275 records.

I'm assuming you're not DISPLAYING all the records on the screen at once - mabye only 30 or so at a time? or maybe 365 at a time...

so it's possible you could read those in from the disk as needed, then clear them out/read in new ones when you need new info.

there is going to be a performance tradeoff here. Disk access is going to be MUCH slower than memory access, but that's the kind of balances we have to deal with all the time. But if it's a tradeoff between being a little slower and dying with an "Out of Memory" error, the choice seems obvious to me.

[Layne Lund]

Since a calendar is very predictable, you probably don't even need a random access file, assuming you are just displaying the days (or months) in some fashion. Say that your user requests the date range from 1900 - 2000. You should probably only display a certain number of days, months, or years at a time. So say you only show a month at a time. I assume you need to calculate the starting day of the month to display, right? Also, you can give the user a "next" and a "previous" button to navigate between months. This way at any given time your program will only store enough information for a single month, not all the information for all 1200 months possible.

Of course, if there is more to your calendar than just displaying the dates, such as scheduling appointments or something similar, you will probably still need the random access file.

HTH

Layne

[Jeroen Wenting]

yes, but you wouldn't need to store an entry for each date...
You'd only store entries for specific dates on which something is happening and while building the display for any month/week/whatever look in the file (consider it a database) for any entries in that period.

[Layne Lund]

Originally posted by Jeroen Wenting:
yes, but you wouldn't need to store an entry for each date...
You'd only store entries for specific dates on which something is happening and while building the display for any month/week/whatever look in the file (consider it a database) for any entries in that period.

I left out that little detail for the OP to figure out (Or at least that's my excuse!)