File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
Win a copy of Clojure in Action this week in the Clojure forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

How to remove duplicate from ArrayList?

 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Anyone can help me with remove duplicate from ARrayList?
I need to add item into this ArrayList , and remove all the duplicated records if it exists.

I have an POJO object call MyClass




Please help, i really need this to be resovled. Greatly appreciated with your help
 
Jesper de Jong
Java Cowboy
Saloon Keeper
Pie
Posts: 15150
31
Android IntelliJ IDE Java Scala Spring
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The way you show us here most likely does not work because you have not added hashCode() and equals() methods to your class MyClass. You must add these methods so that HashSet can properly detect when two MyClass objects are equal.

See the description of the methods hashCode() and equals() in the API documentation of class java.lang.Object for details on how you should implement those methods in MyClass.
 
Maneesh Godbole
Saloon Keeper
Posts: 10971
11
Android Eclipse IDE Google Web Toolkit Java Mac Ubuntu
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
need to add item into this ArrayList , and remove all the duplicated records if it exists.

Wouldn't it be simpler to check before you add?
for (int x=0; x< dataList.size(); x++){
MyClass myClass=new MyClass(useridvalue, usernamevalue);
if(!(dupList.contains(myClass))
duplist.add(myClass);
}

Hope this helps.
Best of luck
 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you for a quick reply:
This is what I added in Myclass



Again thank and greatly apprciated for your help
 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Maneesh Godbole , I did try your suggestion, but still not eliminate dups from the ArrayList

Any other suggestion, please help.
regards
 
Justin Chu
Ranch Hand
Posts: 209
1
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Use a.equals(b) instead of == sign. (check for null pointer first)
 
Tony Docherty
Bartender
Pie
Posts: 2878
59
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by jay lai:
Thanks Maneesh Godbole , I did try your suggestion, but still not eliminate dups from the ArrayList.
The code posted by Maneesh doesn't remove duplicates it only checks to see if the ArrayList already contains a duplicate object or not and only adds the new object if there isn't already a duplicate object in the ArrayList. If you need to remove all duplicates and then add the new object you could do something like:

If the list is only ever filled in this loop then there will never be more than 1 duplicate so you don't need to the while loop just the the line

BTW I used the same for loop as shown in your code for clarity but it doesn't make sense. You are adding dataList.size() number of identical MyClass objects (they are identical because you are creating them all with the same initial values of id and name which your equals method uses to test for equality) so you'll only ever have 1 object in dupList.
 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Thanks for your suggestion
But I did what you suggest, it not work either.
Thanks for the help
 
Tony Docherty
Bartender
Pie
Posts: 2878
59
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by jay lai:
But I did what you suggest, it not work either.
In what way didn't it work?

EDIT: I've just looked at your equals method and there's definitely a problem there which maybe why it's not working. The line should be (assuming they are both Objects and not primitive types) and you need to check for null values as well as Chu Tan previously suggested.
[ August 07, 2007: Message edited by: Tony Docherty ]
 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Again:



I did implement like this


In equals methods:


how about in public int hashcode() method? what do I have to do,
thanks very much
 
Nicholas Jordan
Ranch Hand
Posts: 1282
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You should not have to remove them, unless that is a specific program constraint that one search for and remove duplicates.


There is more than one way to do this, what this code does is not add if the Object is there already. You can write update methods to update a record that already exists.
[ August 07, 2007: Message edited by: Nicholas Jordan ]
 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you so much for posted and responded.

One question: So I just instantiate this PrevenDuplicates and call add(), remove() accordingly?

 
jay lai
Ranch Hand
Posts: 180
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi:
I kind of figure out, but now the method remove duplication but it remove everything and only left 1 record


For example
In my dupList Array: it stored
userid username
123 john
345 mike
345 mike
456 lisa

the method remove everything and keep only
userid username
123 john

it should keep
userid username
123 john
345 mike
456 lisa


What is wrong with this method?


Again always thanks for all the help and suggestion
 
Jesper de Jong
Java Cowboy
Saloon Keeper
Pie
Posts: 15150
31
Android IntelliJ IDE Java Scala Spring
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Originally posted by jay lai:
how about in public int hashcode() method? what do I have to do,
thanks very much

Did you look at the Javadoc documentation of the method hashCode() in class java.lang.Object? It explains in detail how you should implement this method.

Note, if you write an equals() method, then you are also required to write a hashCode() method. One of the requirements of hashCode() is that two objects are equal, the hashCode() method must return the same value when called on those two objects. If it doesn't, then containers like HashSet can get confused when you add objects of that type to them.
[ August 08, 2007: Message edited by: Jesper Young ]
 
Nicholas Jordan
Ranch Hand
Posts: 1282
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
[jay lai :]One question: So I just instantiate this PrevenDuplicates and call add(), remove() accordingly?

The way I wrote it, you do not need any of the ArryaList and other code you have in your post of posted Tuesday, August 07, 2007 8:03 PM but be dilligent in reading the code, and testing it many times. Look for things like what Chu Tan suggests. I just wrote 9,000 characters of code, just to discuss an idea with some of the ranchers. It is useful to write small, tight code - but any code must be thought through many times.

For example:

public int hashCode()
{
// Implement nothing here, unless there is some reason you want a hash code
}

What do you need the hash code for ? You need it because you have been reading about hash codes and you think you need it. Eventually you will see that I keyed the map on the string called user id. Well, userid's should be unique - this is where code reuse comes in, the Java Collections have worked on this problem so I just used their work. I just wrapped it in a class that included your comments and accessor methods.

Use it like this;

PreventDuplicate pd = new PreventDuplicate();

if(pd.add( PlainOldJavaObject pojo )).... code ....;//

PlainOldJavaObject pojo = pd.get(String userid);

if(pd.remove(String userid)).... code ....;//

if(pd.find(String userid)).... code ....;//


It wraps everything you need, you do not need any of the externals that you wrote. You do need the stuff that Jesper Young is talking about, it goes in the PlainOldJavaObject, and would be written thus:

Comparator states:
/* Compares its two arguments for order. Returns a negative integer,
* zero, or a positive integer as the first argument is less than, equal
* to, or greater than the second.
*/
and is written:

public int compare(Object o1, Object o2)
{
return o1.userid.compareTo(o2.userid);
}

OR

public int compare(PlainOldJavaObject o1, PlainOldJavaObject o2)
{
return o1.userid.compareTo(o2.userid);
}

depending on what the current release takes as it's idea of an object


The way I would write the hascode method is just to call Sring.hashCode on user id - this will more than suffice for any collection that needs a hash code to operate:

// this goes in the pojo
public int hashCode()
{
return userid.hashCode();//Returns a hash code for userid.
}
[ August 12, 2007: Message edited by: Nicholas Jordan ]
 
Rambo Rolling
Greenhorn
Posts: 1
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Simple Way to do this .. I am sure this can help..

public ArrayList removeDups()
{
ArrayList list = new ArrayList();
list = addList();
for(int i = 0 ; i < list.size(); i++)
{
NqDlvrOutOfCompliance nqDlvrOutOfCompliance = (NqDlvrOutOfCompliance) list.get(i);
String waybill = nqDlvrOutOfCompliance.getWaybill();
for(int j = i+1; j < list.size(); j++)
{
NqDlvrOutOfCompliance nqDlvrOutOfCompliance1 = (NqDlvrOutOfCompliance) list.get(j);
if(waybill.trim().equalsIgnoreCase(nqDlvrOutOfCompliance1.getWaybill()))
list.remove(j);
}
}

return list;
}


public ArrayList addList()
{
ArrayList list = new ArrayList();
NqDlvrOutOfCompliance nqDlvrOutOfCompliance;
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("1234");
list.add(nqDlvrOutOfCompliance);
}
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("ABCD");
list.add(nqDlvrOutOfCompliance);
}
for (int i = 0 ; i < 3 ; i++)
{
nqDlvrOutOfCompliance = new NqDlvrOutOfCompliance();
nqDlvrOutOfCompliance.setWaybill("XYZ");
list.add(nqDlvrOutOfCompliance);
}
return list;
}




public class NqDlvrOutOfCompliance {

String waybill;

/**
* author xyz
* @return
*
*/
public String getWaybill() {
return waybill;
}

/**
* author xyz
* @param string
*
*/
public void setWaybill(String string) {
waybill = string;
}
}


Call the removeDups method from any class ... This method can be modified to accept the arraylist (in which the dups have to be removed) as the input.
 
Rob Spoor
Sheriff
Pie
Posts: 20372
44
Chrome Eclipse IDE Java Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
"Rambo",

Please read your private messages.

Also:
http://faq.javaranch.com/java/UseCodeTags
http://faq.javaranch.com/java/DontWakeTheZombies. This thread was almost 2 years old.
 
Gyanesh Kumar
Greenhorn
Posts: 1
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
package rnd.collections;

/*Gyanesh Kumar Pandey
* DuplicateArray
*
*
*/

public class DuplicateArray
{
public static final int NUM_ITEMS = 50;

// Return true if array a has duplicates; false otherwise
public static boolean duplicates( Object [ ] a )
{
for( int i = 0; i < a.length; i++ )
for( int j = i + 1; j < a.length; j++ )
if( a[ i ].equals( a[ j ] ) )
return true; // Duplicate found

return false; // No duplicates found
}





// Test the duplicates method
public static void main( String [ ] args )
{
Integer [ ] a = new Integer[ NUM_ITEMS ];
for( int i = 0; i < a.length; i++ )
a[ i ] = new Integer( i );

System.out.println( "Should be false: " + duplicates( a ) );

a[ NUM_ITEMS - 1 ] = a[ 0 ];

System.out.println( "Should be true: " + duplicates( a ) );
}

public void rearr(int aa[])
{
int l=aa.length;

for (int j =l-1;j<l/2;j--)
{



}

}

}
@Gyanesh
 
zahid zubair
Ranch Hand
Posts: 32
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Very simple. Use

 
Campbell Ritchie
Sheriff
Pie
Posts: 47243
52
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Maybe simple, but it is likely the original poster is no longer reading this thread. Have a look at this.
 
Jevgeni Pogorelov
Greenhorn
Posts: 2
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
One possible solution in case we nead remove dublicates in List by one or more field condition.

public class SomeObject () {

private String field1;

private String field2;

public String getField1() {
return field1;
}

public void setField1(String field1) {
this.field1 = field1;
}

public String getField2() {
return field2;
}

public void setField2(String field2) {
this.field2 = field2;
}

public List<SomeObject> removeDublicated(List<SomeObject> someObjectsList) {

List<SomeObject> ret = new ArrayList<SomeObject>();

for (SomeObject aSomeObjectsList : someObjectsList) {

String field1 = aSomeObjectsList.getField1();
String field2 = aSomeObjectsList.getField2();

boolean exist = false;

for (SomeObject someObject : ret) {

if (field1.trim().equalsIgnoreCase(someObject.getField1())
&& field2.trim().equalsIgnoreCase(someObject.getField2())) {

exist = true;

}
}

if (!exist) {

ret.add(aSomeObjectsList);
}

}

return ret;
}

}

 
Paul Clapham
Sheriff
Pie
Posts: 20177
25
MySQL Database
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It is STILL likely the original poster is no longer reading this thread. Have a look at this.
 
Rob Spoor
Sheriff
Pie
Posts: 20372
44
Chrome Eclipse IDE Java Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This?
 
Jevgeni Pogorelov
Greenhorn
Posts: 2
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My answer was for all user who try to find answer in the internet and got link to this forum from search engines. Like me:-) I hope it helpful for all, not only for original poster.
 
prathap venkata naga yelugula
Ranch Hand
Posts: 253
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Friends,

ArrayList l=new ArrayList();
l.add("sasi");
l.add("sasi1");
l.add("sasi2");
l.add("sasi3");
l.add("sasi4");
l.add(0);
l.add(0);

l.add(null);
l.add(null);
l.add(null);
System.out.println(l);
HashSet hs=new HashSet(l);
l.clear();
l.add(hs);
System.out.println(l);



Cheers!
Prathap.
 
sudheer kiran
Ranch Hand
Posts: 237
Firefox Browser MyEclipse IDE Tomcat Server
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
oh lot of replies for this question.there are good answers.let me give a final explanation.
here you want to add objects to a collection and remove duplicates.
but how do java knows that two objects are equal.for that it uses "==" operator.
meaning for that operator is if both are referencing to same memory location.

Object's equal method uses this operator,String and wrapper classes implemented the equals n hash code methods.
so if you are comparing your own objects like MyClass objects then that class must implement equals and hashcode methods.sp just add these lines to your code in MyClass.java



now HashCannot nontain duplicates of Myclass Objects.
 
xsunil kumar
Ranch Hand
Posts: 139
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Why you not using Set. Set does not allow duplicates.
 
Harshana Dias
Ranch Hand
Posts: 331
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
see the problem here is not override the hashCode..so basically even the two object has same attributes (eg:123 john) they have different hash codes.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic