• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Removing items in an iterator

 
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Consider the following code:


Now, add the specification that items is quite large in memory (~1 million items), the loop takes 5 minutes to execute, and after an item is used in the loop it is no longer needed. It seems to me a good modification (or not?) is to modify the loop to remove each item from the collection as it iterates in order to free up memory for other processes.

So two questions, is i.remove() safe to use at the end of each loop or will I likely get concurrency exceptions. I've learned not to use i.remove() often since it leads to runtime errors. Second, is removing the items really likely to free up much memory?
[ July 02, 2008: Message edited by: Scott Selikoff ]
 
Rancher
Posts: 3742
16
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
According to the Javadoc, i.remove() is safe as long as it is only called once for each call to i.next(). This of course assumes that your Iterator is implemented correctly and that it supports the remove method.
Removing objects from the underlying collection in any other way has an undefined affect on the Iterator.

Assuming there are no other references to the objects then removing them will free up memory the next time the garbage collector runs. Of course, having the garbage collector run several times freeing up a few objects rather than once at the end freeing up all the objects may affect performance.
If you only have a single thread running in your program, freeing up the memory whilst iterating may not be important. If there are multiple threads running, then freeing up memory as soon as you can may have a performance affect on the other threads.
The garbage collection algorithm being used may also make a difference.

In other words - get a good profiler and experiment
 
Master Rancher
Posts: 4806
72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Using iterator.remove(), you won't get ConcurrentModificationException - unless of course you've got another thread accessing the collection at the same time. If you've seen CME before in a similar situation, you may have been calling items.remove(item) rather than using the Iterator. Using the Iterator for removal is good. Removing directly from the Collection while iterating is bad. (At least, it's bad for common Collections like ArrayList and LinkedList.)

For freeing up memory, removing sounds like it could be a good idea. Assuming no other references are retained, the items should be eligible for GC right away.

Note also that if the individual objects are small, and the list itself is a significant part of the memory used, then ArrayList will probably continue to hold its million-member backing array in memory, even if the list has become much smaller. The references in the array will be null, so they won't prevent GC of the objects removed. But the backing array may still use up more memory than you'd like. You could use trimToSize() to avoid this - but don't do it too often. It would probably be rather slow to do that every time you remove, for example. If you use LinkedList rather than ArrayList, the list will automatically stay trim, with no extra effort.

Also note that from a performance perspective, removing from an ArrayList may be slow, O(N). Removing from a LinkedList would be much faster, O(1). If you must use an ArrayList, consider using a ListIterator and looping in reverse, so you're always removing from the end of the array - should be much faster.

If you're using some other Collection type, let us know. Or just measure the performance to see what works best.
[ July 02, 2008: Message edited by: Mike Simmons ]
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Actually I just checked and the underlying collection is an ArrayList, so perhaps I should cast it as such as remove from the end while reading elements in reverse order.
 
Scott Selikoff
author
Posts: 4335
39
jQuery Eclipse IDE Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Assuming a very large collection of possibly unknown type, how's this for a modification:
 
Mike Simmons
Master Rancher
Posts: 4806
72
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's probably good. There's a small risk that the initial copying to a LinkedList will use too much memory, or be too slow, but probably not. Things should be fast & memory-efficient after that.
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Scott Selikoff:
Assuming a very large collection of possibly unknown type, how's this for a modification:



I'd suggest to try and time it.
 
Saloon Keeper
Posts: 27762
196
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Unless you're playing really close to the wire and need to free up resources ASAP, I think probably simply removing the entire collection will be more efficient. Or at least allow the disassembly of the collection to be done in the background by the garbage collector once the anchor object has been released.
 
reply
    Bookmark Topic Watch Topic
  • New Topic