aspose file tools*
The moose likes Java in General and the fly likes How to remove duplicate from ArrayList? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "How to remove duplicate from ArrayList?" Watch "How to remove duplicate from ArrayList?" New topic
Author

How to remove duplicate from ArrayList?

jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Anyone can help me with remove duplicate from ARrayList?
I need to add item into this ArrayList , and remove all the duplicated records if it exists.

I have an POJO object call MyClass




Please help, i really need this to be resovled. Greatly appreciated with your help
Jesper de Jong
Java Cowboy
Saloon Keeper

Joined: Aug 16, 2005
Posts: 13884
    
  10

The way you show us here most likely does not work because you have not added hashCode() and equals() methods to your class MyClass. You must add these methods so that HashSet can properly detect when two MyClass objects are equal.

See the description of the methods hashCode() and equals() in the API documentation of class java.lang.Object for details on how you should implement those methods in MyClass.


Java Beginners FAQ - JavaRanch SCJP FAQ - The Java Tutorial - Java SE 7 API documentation
Scala Notes - My blog about Scala
Maneesh Godbole
Saloon Keeper

Joined: Jul 26, 2007
Posts: 9994
    
    7

need to add item into this ArrayList , and remove all the duplicated records if it exists.

Wouldn't it be simpler to check before you add?
for (int x=0; x< dataList.size(); x++){
MyClass myClass=new MyClass(useridvalue, usernamevalue);
if(!(dupList.contains(myClass))
duplist.add(myClass);
}

Hope this helps.
Best of luck


[How to ask questions] [Donate a pint, save a life!] [Onff-turn it on!]
jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Thank you for a quick reply:
This is what I added in Myclass



Again thank and greatly apprciated for your help
jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Thanks Maneesh Godbole , I did try your suggestion, but still not eliminate dups from the ArrayList

Any other suggestion, please help.
regards
Justin Chu
Ranch Hand

Joined: Apr 19, 2002
Posts: 209
    
    1
Use a.equals(b) instead of == sign. (check for null pointer first)
Tony Docherty
Bartender

Joined: Aug 07, 2007
Posts: 1950
    
  28
Originally posted by jay lai:
Thanks Maneesh Godbole , I did try your suggestion, but still not eliminate dups from the ArrayList.
The code posted by Maneesh doesn't remove duplicates it only checks to see if the ArrayList already contains a duplicate object or not and only adds the new object if there isn't already a duplicate object in the ArrayList. If you need to remove all duplicates and then add the new object you could do something like:

If the list is only ever filled in this loop then there will never be more than 1 duplicate so you don't need to the while loop just the the line

BTW I used the same for loop as shown in your code for clarity but it doesn't make sense. You are adding dataList.size() number of identical MyClass objects (they are identical because you are creating them all with the same initial values of id and name which your equals method uses to test for equality) so you'll only ever have 1 object in dupList.
jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Hi Thanks for your suggestion
But I did what you suggest, it not work either.
Thanks for the help
Tony Docherty
Bartender

Joined: Aug 07, 2007
Posts: 1950
    
  28
Originally posted by jay lai:
But I did what you suggest, it not work either.
In what way didn't it work?

EDIT: I've just looked at your equals method and there's definitely a problem there which maybe why it's not working. The line should be (assuming they are both Objects and not primitive types) and you need to check for null values as well as Chu Tan previously suggested.
[ August 07, 2007: Message edited by: Tony Docherty ]
jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Hi Again:



I did implement like this


In equals methods:


how about in public int hashcode() method? what do I have to do,
thanks very much
Nicholas Jordan
Ranch Hand

Joined: Sep 17, 2006
Posts: 1282
You should not have to remove them, unless that is a specific program constraint that one search for and remove duplicates.


There is more than one way to do this, what this code does is not add if the Object is there already. You can write update methods to update a record that already exists.
[ August 07, 2007: Message edited by: Nicholas Jordan ]

"The differential equations that describe dynamic interactions of power generators are similar to that of the gravitational interplay among celestial bodies, which is chaotic in nature."
jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
thank you so much for posted and responded.

One question: So I just instantiate this PrevenDuplicates and call add(), remove() accordingly?

jay lai
Ranch Hand

Joined: Apr 04, 2002
Posts: 180
Hi:
I kind of figure out, but now the method remove duplication but it remove everything and only left 1 record


For example
In my dupList Array: it stored
userid username
123 john
345 mike
345 mike
456 lisa

the method remove everything and keep only
userid username
123 john

it should keep
userid username
123 john
345 mike
456 lisa


What is wrong with this method?


Again always thanks for all the help and suggestion
Jesper de Jong
Java Cowboy
Saloon Keeper

Joined: Aug 16, 2005
Posts: 13884
    
  10

Originally posted by jay lai:
how about in public int hashcode() method? what do I have to do,
thanks very much

Did you look at the Javadoc documentation of the method hashCode() in class java.lang.Object? It explains in detail how you should implement this method.

Note, if you write an equals() method, then you are also required to write a hashCode() method. One of the requirements of hashCode() is that two objects are equal, the hashCode() method must return the same value when called on those two objects. If it doesn't, then containers like HashSet can get confused when you add objects of that type to them.
[ August 08, 2007: Message edited by: Jesper Young ]
Nicholas Jordan
Ranch Hand

Joined: Sep 17, 2006
Posts: 1282
[jay lai :]One question: So I just instantiate this PrevenDuplicates and call add(), remove() accordingly?

The way I wrote it, you do not need any of the ArryaList and other code you have in your post of posted Tuesday, August 07, 2007 8:03 PM but be dilligent in reading the code, and testing it many times. Look for things like what Chu Tan suggests. I just wrote 9,000 characters of code, just to discuss an idea with some of the ranchers. It is useful to write small, tight code - but any code must be thought through many times.

For example:

public int hashCode()
{
// Implement nothing here, unless there is some reason you want a hash code
}

What do you need the hash code for ? You need it because you have been reading about hash codes and you think you need it. Eventually you will see that I keyed the map on the string called user id. Well, userid's should be unique - this is where code reuse comes in, the Java Collections have worked on this problem so I just used their work. I just wrapped it in a class that included your comments and accessor methods.

Use it like this;

PreventDuplicate pd = new PreventDuplicate();

if(pd.add( PlainOldJavaObject pojo )).... code ....;//

PlainOldJavaObject pojo = pd.get(String userid);

if(pd.remove(String userid)).... code ....;//

if(pd.find(String userid)).... code ....;//


It wraps everything you need, you do not need any of the externals that you wrote. You do need the stuff that Jesper Young is talking about, it goes in the PlainOldJavaObject, and would be written thus:

Comparator states:
/* Compares its two arguments for order. Returns a negative integer,
* zero, or a positive integer as the first argument is less than, equal
* to, or greater than the second.
*/
and is written:

public int compare(Object o1, Object o2)
{
return o1.userid.compareTo(o2.userid);
}

OR

public int compare(PlainOldJavaObject o1, PlainOldJavaObject o2)
{
return o1.userid.compareTo(o2.userid);
}

depending on what the current release takes as it's idea of an object


The way I would write the hascode method is just to call Sring.hashCode on user id - this will more than suffice for any collection that needs a hash code to operate:

// this goes in the pojo
public int hashCode()
{
return userid.hashCode();//Returns a hash code for userid.
}
[ August 12, 2007: Message edited by: Nicholas Jordan ]
Rambo Rolling
Greenhorn

Joined: Jun 10, 2009
Posts: 1
Simple Way to do this .. I am sure this can help..

public ArrayList removeDups()
{
ArrayList list = new ArrayList();
list = addList();
for(int i = 0 ; i < list.size(); i++)
{
NqDlvrOutOfCompliance nqDlvrOutOfCompliance = (NqDlvrOutOfCompliance) list.get(i);
String waybill = nqDlvrOutOfCompliance.getWaybill();
for(int j = i+1; j < list.size(); j++)
{
NqDlvrOutOfCompliance nqDlvrOutOfCompliance1 = (NqDlvrOutOfCompliance) list.get(j);
if(waybill.trim().equalsIgnoreCase(nqDlvrOutOfCompliance1.getWaybill()))
list.remove(j);
}
}

return list;
}


public ArrayList addList()
{
ArrayList list = new ArrayList();
NqDlvrOutOfCompliance nqDlvrOutOfCompliance;
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("1234");
list.add(nqDlvrOutOfCompliance);
}
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("ABCD");
list.add(nqDlvrOutOfCompliance);
}
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("XYZ");
list.add(nqDlvrOutOfCompliance);
}
return list;
}




public class NqDlvrOutOfCompliance {

String waybill;

/**
* author xyz
* @return
*
*/
public String getWaybill() {
return waybill;
}

/**
* author xyz
* @param string
*
*/
public void setWaybill(String string) {
waybill = string;
}
}


Call the removeDups method from any class ... This method can be modified to accept the arraylist (in which the dups have to be removed) as the input.
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19556
    
  16

"Rambo",

Please read your private messages.

Also:
http://faq.javaranch.com/java/UseCodeTags
http://faq.javaranch.com/java/DontWakeTheZombies. This thread was almost 2 years old.


SCJP 1.4 - SCJP 6 - SCWCD 5 - OCEEJBD 6
How To Ask Questions How To Answer Questions
Gyanesh Kumar
Greenhorn

Joined: Aug 03, 2009
Posts: 1
package rnd.collections;

/*Gyanesh Kumar Pandey
* DuplicateArray
*
*
*/

public class DuplicateArray
{
public static final int NUM_ITEMS = 50;

// Return true if array a has duplicates; false otherwise
public static boolean duplicates( Object [ ] a )
{
for( int i = 0; i < a.length; i++ )
for( int j = i + 1; j < a.length; j++ )
if( a[ i ].equals( a[ j ] ) )
return true; // Duplicate found

return false; // No duplicates found
}





// Test the duplicates method
public static void main( String [ ] args )
{
Integer [ ] a = new Integer[ NUM_ITEMS ];
for( int i = 0; i < a.length; i++ )
a[ i ] = new Integer( i );

System.out.println( "Should be false: " + duplicates( a ) );

a[ NUM_ITEMS - 1 ] = a[ 0 ];

System.out.println( "Should be true: " + duplicates( a ) );
}

public void rearr(int aa[])
{
int l=aa.length;

for (int j =l-1;j<l/2;j--)
{



}

}

}
@Gyanesh
zahid zubair
Ranch Hand

Joined: Aug 29, 2009
Posts: 32
Very simple. Use

Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 36579
    
  16
Maybe simple, but it is likely the original poster is no longer reading this thread. Have a look at this.
Jevgeni Pogorelov
Greenhorn

Joined: Oct 22, 2009
Posts: 2
One possible solution in case we nead remove dublicates in List by one or more field condition.

public class SomeObject () {

private String field1;

private String field2;

public String getField1() {
return field1;
}

public void setField1(String field1) {
this.field1 = field1;
}

public String getField2() {
return field2;
}

public void setField2(String field2) {
this.field2 = field2;
}

public List<SomeObject> removeDublicated(List<SomeObject> someObjectsList) {

List<SomeObject> ret = new ArrayList<SomeObject>();

for (SomeObject aSomeObjectsList : someObjectsList) {

String field1 = aSomeObjectsList.getField1();
String field2 = aSomeObjectsList.getField2();

boolean exist = false;

for (SomeObject someObject : ret) {

if (field1.trim().equalsIgnoreCase(someObject.getField1())
&& field2.trim().equalsIgnoreCase(someObject.getField2())) {

exist = true;

}
}

if (!exist) {

ret.add(aSomeObjectsList);
}

}

return ret;
}

}

Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18155
    
    8

It is STILL likely the original poster is no longer reading this thread. Have a look at this.
Rob Spoor
Sheriff

Joined: Oct 27, 2005
Posts: 19556
    
  16

This?
Jevgeni Pogorelov
Greenhorn

Joined: Oct 22, 2009
Posts: 2
My answer was for all user who try to find answer in the internet and got link to this forum from search engines. Like me:-) I hope it helpful for all, not only for original poster.
prathap venkata naga yelugula
Ranch Hand

Joined: Aug 13, 2008
Posts: 243
Hi Friends,

ArrayList l=new ArrayList();
l.add("sasi");
l.add("sasi1");
l.add("sasi2");
l.add("sasi3");
l.add("sasi4");
l.add(0);
l.add(0);

l.add(null);
l.add(null);
l.add(null);
System.out.println(l);
HashSet hs=new HashSet(l);
l.clear();
l.add(hs);
System.out.println(l);



Cheers!
Prathap.
sudheer kiran
Ranch Hand

Joined: Jun 26, 2008
Posts: 237

oh lot of replies for this question.there are good answers.let me give a final explanation.
here you want to add objects to a collection and remove duplicates.
but how do java knows that two objects are equal.for that it uses "==" operator.
meaning for that operator is if both are referencing to same memory location.

Object's equal method uses this operator,String and wrapper classes implemented the equals n hash code methods.
so if you are comparing your own objects like MyClass objects then that class must implement equals and hashcode methods.sp just add these lines to your code in MyClass.java



now HashCannot nontain duplicates of Myclass Objects.

Sudheer
SCWCD, SCJP 5
xsunil kumar
Ranch Hand

Joined: Dec 14, 2009
Posts: 129
Why you not using Set. Set does not allow duplicates.
Harshana Dias
Ranch Hand

Joined: Jun 11, 2007
Posts: 327
see the problem here is not override the hashCode..so basically even the two object has same attributes (eg:123 john) they have different hash codes.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: How to remove duplicate from ArrayList?
 
Similar Threads
How to migrate from struts 1 to struts2?
JUnit testing void methods
unable to pass value of resultset to servlet to jsp
DisplayTag Probelm.
java.lang.IllegalArgumentException: Parameter UserName does not exist