File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Java in General and the fly likes Removing items in an iterator Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Removing items in an iterator" Watch "Removing items in an iterator" New topic
Author

Removing items in an iterator

Scott Selikoff
Saloon Keeper

Joined: Oct 23, 2005
Posts: 3697
    
    5

Consider the following code:


Now, add the specification that items is quite large in memory (~1 million items), the loop takes 5 minutes to execute, and after an item is used in the loop it is no longer needed. It seems to me a good modification (or not?) is to modify the loop to remove each item from the collection as it iterates in order to free up memory for other processes.

So two questions, is i.remove() safe to use at the end of each loop or will I likely get concurrency exceptions. I've learned not to use i.remove() often since it leads to runtime errors. Second, is removing the items really likely to free up much memory?
[ July 02, 2008: Message edited by: Scott Selikoff ]

My Blog: Down Home Country Coding with Scott Selikoff
Joanne Neal
Rancher

Joined: Aug 05, 2005
Posts: 3169
    
  10
According to the Javadoc, i.remove() is safe as long as it is only called once for each call to i.next(). This of course assumes that your Iterator is implemented correctly and that it supports the remove method.
Removing objects from the underlying collection in any other way has an undefined affect on the Iterator.

Assuming there are no other references to the objects then removing them will free up memory the next time the garbage collector runs. Of course, having the garbage collector run several times freeing up a few objects rather than once at the end freeing up all the objects may affect performance.
If you only have a single thread running in your program, freeing up the memory whilst iterating may not be important. If there are multiple threads running, then freeing up memory as soon as you can may have a performance affect on the other threads.
The garbage collection algorithm being used may also make a difference.

In other words - get a good profiler and experiment


Joanne
Mike Simmons
Ranch Hand

Joined: Mar 05, 2008
Posts: 2969
    
    9
Using iterator.remove(), you won't get ConcurrentModificationException - unless of course you've got another thread accessing the collection at the same time. If you've seen CME before in a similar situation, you may have been calling items.remove(item) rather than using the Iterator. Using the Iterator for removal is good. Removing directly from the Collection while iterating is bad. (At least, it's bad for common Collections like ArrayList and LinkedList.)

For freeing up memory, removing sounds like it could be a good idea. Assuming no other references are retained, the items should be eligible for GC right away.

Note also that if the individual objects are small, and the list itself is a significant part of the memory used, then ArrayList will probably continue to hold its million-member backing array in memory, even if the list has become much smaller. The references in the array will be null, so they won't prevent GC of the objects removed. But the backing array may still use up more memory than you'd like. You could use trimToSize() to avoid this - but don't do it too often. It would probably be rather slow to do that every time you remove, for example. If you use LinkedList rather than ArrayList, the list will automatically stay trim, with no extra effort.

Also note that from a performance perspective, removing from an ArrayList may be slow, O(N). Removing from a LinkedList would be much faster, O(1). If you must use an ArrayList, consider using a ListIterator and looping in reverse, so you're always removing from the end of the array - should be much faster.

If you're using some other Collection type, let us know. Or just measure the performance to see what works best.
[ July 02, 2008: Message edited by: Mike Simmons ]
Scott Selikoff
Saloon Keeper

Joined: Oct 23, 2005
Posts: 3697
    
    5

Actually I just checked and the underlying collection is an ArrayList, so perhaps I should cast it as such as remove from the end while reading elements in reverse order.
Scott Selikoff
Saloon Keeper

Joined: Oct 23, 2005
Posts: 3697
    
    5

Assuming a very large collection of possibly unknown type, how's this for a modification:
Mike Simmons
Ranch Hand

Joined: Mar 05, 2008
Posts: 2969
    
    9
That's probably good. There's a small risk that the initial copying to a LinkedList will use too much memory, or be too slow, but probably not. Things should be fast & memory-efficient after that.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Scott Selikoff:
Assuming a very large collection of possibly unknown type, how's this for a modification:


I'd suggest to try and time it.


The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15641
    
  15

Unless you're playing really close to the wire and need to free up resources ASAP, I think probably simply removing the entire collection will be more efficient. Or at least allow the disassembly of the collection to be done in the background by the garbage collector once the anchor object has been released.


Customer surveys are for companies who didn't pay proper attention to begin with.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Removing items in an iterator
 
Similar Threads
ConcurrentModificationException
syntax question
Priority queue question?
PriorityQueue issue
remove key from Map