Now, add the specification that items is quite large in memory (~1 million items), the loop takes 5 minutes to execute, and after an item is used in the loop it is no longer needed. It seems to me a good modification (or not?) is to modify the loop to remove each item from the collection as it iterates in order to free up memory for other processes.
So two questions, is i.remove() safe to use at the end of each loop or will I likely get concurrency exceptions. I've learned not to use i.remove() often since it leads to runtime errors. Second, is removing the items really likely to free up much memory? [ July 02, 2008: Message edited by: Scott Selikoff ]
According to the Javadoc, i.remove() is safe as long as it is only called once for each call to i.next(). This of course assumes that your Iterator is implemented correctly and that it supports the remove method. Removing objects from the underlying collection in any other way has an undefined affect on the Iterator.
Assuming there are no other references to the objects then removing them will free up memory the next time the garbage collector runs. Of course, having the garbage collector run several times freeing up a few objects rather than once at the end freeing up all the objects may affect performance. If you only have a single thread running in your program, freeing up the memory whilst iterating may not be important. If there are multiple threads running, then freeing up memory as soon as you can may have a performance affect on the other threads. The garbage collection algorithm being used may also make a difference.
In other words - get a good profiler and experiment
Using iterator.remove(), you won't get ConcurrentModificationException - unless of course you've got another thread accessing the collection at the same time. If you've seen CME before in a similar situation, you may have been calling items.remove(item) rather than using the Iterator. Using the Iterator for removal is good. Removing directly from the Collection while iterating is bad. (At least, it's bad for common Collections like ArrayList and LinkedList.)
For freeing up memory, removing sounds like it could be a good idea. Assuming no other references are retained, the items should be eligible for GC right away.
Note also that if the individual objects are small, and the list itself is a significant part of the memory used, then ArrayList will probably continue to hold its million-member backing array in memory, even if the list has become much smaller. The references in the array will be null, so they won't prevent GC of the objects removed. But the backing array may still use up more memory than you'd like. You could use trimToSize() to avoid this - but don't do it too often. It would probably be rather slow to do that every time you remove, for example. If you use LinkedList rather than ArrayList, the list will automatically stay trim, with no extra effort.
Also note that from a performance perspective, removing from an ArrayList may be slow, O(N). Removing from a LinkedList would be much faster, O(1). If you must use an ArrayList, consider using a ListIterator and looping in reverse, so you're always removing from the end of the array - should be much faster.
If you're using some other Collection type, let us know. Or just measure the performance to see what works best. [ July 02, 2008: Message edited by: Mike Simmons ]
Originally posted by Scott Selikoff: Assuming a very large collection of possibly unknown type, how's this for a modification:
I'd suggest to try and time it.
The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Unless you're playing really close to the wire and need to free up resources ASAP, I think probably simply removing the entire collection will be more efficient. Or at least allow the disassembly of the collection to be done in the background by the garbage collector once the anchor object has been released.
An IDE is no substitute for an Intelligent Developer.
I’ve looked at a lot of different solutions, and in my humble opinion Aspose is the way to go. Here’s the link: http://aspose.com