Compare and Contrast Two ArrayList<Object>

[Nickolas Jennings]

Dear All,

I have two ArrayList<Object[]> and I want to be able to compare the differences. At the moment, I have a mediocre way of solving. In a perfect world I have two list next to each other and the differences listed in red or something like that.

Below is the code I'm using. This determines whether there are pinDifferences as a flag and a string. The string and flag are then passed to a jsp to print out. (This is in a service.) The problem I'm running into is if the pins are identical (1-17) but the second one has two options for pin #3. I'm printing out that pin 3-18 are different. Because I've sorted it. What I want to do is show that pin #3 doesn't exist on the other connector. This case could happen both ways and multiple times. Would there be a efficient way to compare two ArrayList<Object> and list out the similarities and and differences. Kind of like what gitHub does with merge conflicts.

List<PlatformPin> connectorPins = new ArrayList<PlatformPin>();
Map<Integer, Integer> pinDifferences = new HashMap<Integer, Integer>();
Map<Integer, String> pinDifferenceDetails =  new HashMap<Integer, String>();

connectorPins.addAll(connector.getPins());
if (connectorPins.size() > 1) {
Collections.sort(connectorPins, new PlatformPinComparator());
}

for (PlatformConnector seeker : similarConnectors) {
	List<PlatformPin> seekerPins = new ArrayList<PlatformPin>();

	String allDifferences = "";
	boolean pinsIdentical = true;
	seekerPins.addAll(seeker.getPins());
	if (seekerPins.size() > 0) {
		if (seekerPins.size() > 1) {
			Collections.sort(seekerPins, new PlatformPinComparator());
		}

	for (int i = 0; i < Math.min(connectorPins.size(), seekerPins.size()); i++) {
		if (!StringUtils.equalsIgnoreCase(StringUtils.trimToEmpty(
                   connectorPins.get(i).getName()),
                   StringUtils.trimToEmpty(seekerPins.get(i).getName()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + "Pin"
                     + connectorPins.get(i).getName() + " ";
		}
		if (!StringUtils.equalsIgnoreCase(
                StringUtils.trimToEmpty(connectorPins.get(i).getCircuit().getFunction()),
			StringUtils.trimToEmpty(seekerPins.get(i).getCircuit().getFunction()))) {
			pinsIdentical = false;
			allDifferences = allDifferences
                              + connectorPins.get(i).getName() + "-CircuitFunction ";
		}
		if (!StringUtils.equalsIgnoreCase(
                         StringUtils.trimToEmpty(connectorPins.get(i).getCircuit().getName()),
					StringUtils.trimToEmpty(seekerPins.get(i).getCircuit().getName()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + connectorPins.get(i).getName() + "-CircuitName ";
		}
		if (!StringUtils.equalsIgnoreCase(
                       StringUtils.trimToEmpty(connectorPins.get(i).getColor()),
                       StringUtils.trimToEmpty(seekerPins.get(i).getColor()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + connectorPins.get(i).getName() + "-Color ";
		}
		if (!StringUtils.equalsIgnoreCase(
                  StringUtils.trimToEmpty(connectorPins.get(i).getTracer()),
                  StringUtils.trimToEmpty(seekerPins.get(i).getTracer()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + connectorPins.get(i).getName() + "-Tracer ";
		}
		if (!StringUtils.equalsIgnoreCase(
                         StringUtils.trimToEmpty(
                                connectorPins.get(i).getGauge()),
                                StringUtils.trimToEmpty(seekerPins.get(i).getGauge()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + 
                                connectorPins.get(i).getName() + "-Gauge ";
		}
		if (!StringUtils.equalsIgnoreCase(
                       StringUtils.trimToEmpty(connectorPins.get(i).getOption()),
                       StringUtils.trimToEmpty(seekerPins.get(i).getOption()))) {
			pinsIdentical = false;
			allDifferences = allDifferences + 
                             connectorPins.get(i).getName() + "-Option ";
		}

pinDifferenceDetails.put(seeker.getId(), allDifferences);

	}
}
if (StringUtils.equalsIgnoreCase(
                seeker.getCavityCount(), connector.getCavityCount())
	&& StringUtils.equalsIgnoreCase(seeker.getColor(), connector.getColor())
	&& StringUtils.equalsIgnoreCase(seeker.getConnectorId(), connector.getConnectorId())
&& StringUtils.equalsIgnoreCase(seeker.getConnectorNo(), connector.getConnectorNo())
	&& (seeker.getConnectorType() != null &&
                 connector.getConnectorType() != null || seeker.getConnectorType() == null &&
                 connector.getConnectorType() == null
	&& seeker.getConnectorType().equals(connector.getConnectorType()))
	&& StringUtils.equalsIgnoreCase(
                  seeker.getDescription(), connector.getDescription())
	&& StringUtils.equalsIgnoreCase(
                  seeker.getDevice().getDeviceId(), connector.getDevice().getDeviceId())
	&& StringUtils.equalsIgnoreCase(
                 seeker.getDevice().getDeviceNo(), connector.getDevice().getDeviceNo())
	&& StringUtils.equalsIgnoreCase(
                 seeker.getEngineeringName(), connector.getEngineeringName())
	&& StringUtils.equalsIgnoreCase(
                 seeker.getHarnessFamily().getName(),
                 connector.getHarnessFamily().getName())
	&& StringUtils.equalsIgnoreCase(
                 seeker.getPartNumber(), connector.getPartNumber())
	&& (seeker.getGender() == null && 
                 connector.getGender() == null || 
                 seeker.getGender() != null && connector.getGender() != null
	&& seeker.getGender().equals(
                 connector.getGender())) && 
                 connectorPins.size() == seekerPins.size() && pinsIdentical) {
System.out.println("findIdentical:FoundIdentical");
uniqueConnectors.add(seeker);
} else {
differentConnectors.add(seeker);

// Print Individual differences, add flag to pinDifferences
String differences = "";
if (StringUtils.equalsIgnoreCase(
            seeker.getCavityCount(), connector.getCavityCount())) {
} else {
differences = differences + "Cavity Count ";
}
if (StringUtils.equalsIgnoreCase(seeker.getColor(), connector.getColor())) {
} else {
differences = differences + "Color ";
}
if (StringUtils.equalsIgnoreCase(
                seeker.getConnectorId(),
                connector.getConnectorId())) {
} else {
differences = differences + "ConnectorId ";
}
if (StringUtils.equalsIgnoreCase(
                seeker.getConnectorNo(), connector.getConnectorNo())) {
} else {
differences = differences + "ConnectorNo ";
}
if (seeker.getConnectorType() != null &&
                connector.getConnectorType() != null || 
                seeker.getConnectorType() == null && connector.getConnectorType() == null
&& seeker.getConnectorType().equals(connector.getConnectorType())) {
} else {
differences = differences + "ConnectorType ";
}
if (StringUtils.equalsIgnoreCase(
                seeker.getDescription(), connector.getDescription())) {
} else {
differences = differences + "Description ";
}
if (StringUtils.equalsIgnoreCase(
                seeker.getDevice().getDeviceId(),
                connector.getDevice().getDeviceId())) {
} else {
differences = differences + "DeviceId ";
}
if (StringUtils.equalsIgnoreCase(
                seeker.getDevice().getDeviceNo(),
                connector.getDevice().getDeviceNo())) {
} else {
differences = differences + "DeviceNo ";
}
if (StringUtils.equalsIgnoreCase(
               seeker.getEngineeringName(),
              connector.getEngineeringName())) {
} else {
differences = differences + "EngineeringName ";
}
if (StringUtils.equalsIgnoreCase(
               seeker.getHarnessFamily().getName(),
               connector.getHarnessFamily().getName())) {
} else {
differences = differences + "Name ";
}
if (StringUtils.equalsIgnoreCase(
               seeker.getPartNumber(), connector.getPartNumber())) {
} else {
differences = differences + "PartNumber ";
}
if (seeker.getGender() == null && connector.getGender() == null || 
              seeker.getGender() != null && connector.getGender() != null
&& seeker.getGender().equals(connector.getGender())) {
} else {
differences = differences + "Gender ";
}
if (StringUtils.equalsIgnoreCase(
               seeker.getEndViewFileName(), connector.getEndViewFileName())) {
} else {
differences = differences + "EndViewFileName ";
}
if (StringUtils.equalsIgnoreCase(
              seeker.getIsometricEndViewFileName(),
              connector.getIsometricEndViewFileName())) {
} else {
differences = differences + "IsometricEndViewFileName ";
}
if (connectorPins.size() == seekerPins.size() && pinsIdentical) {
pinDifferences.put(seeker.getId(), 0);
} else {
differences = differences + "PinDifferences ";
pinDifferences.put(seeker.getId(), 1);
}
System.out.println("findIdentical:Different{ " +
         seeker.getVehicleFamily().getYear()
	+ differences + ": " + allDifferences + "}");

/* Code below passes uniqueConnectors,
   differentConnectors,pinDifferences,
   pinDifferenceDetails to a controller then to the jsp */

So, the important part is from code lines 21-50. I compare every characteristic about the individual objects. I just build a string of the differences and tag it to the seekerId. This will work for 90% of the data, but it's a remedial way to do. I pass the pinDifferenceDetails to a jsp and print accordingly. Personally, I want to have the text "Pin Differences Exist" as a link to a separate page with a beautiful table that display similarities and differences. I hope I explained myself cleanly.

Thanks for your help!

[Mike Cox]

It's a bit confusing to me, so I can't answer for sure, but the loop starting on line 21 won't find all the differences if the lists are not the same size. That could be why it's not showing that pin #3 doesn't exist. It might be easier and less bug-prone in the long term to rethink the approach a little bit. There are 3 pieces of information you eventually want:

- Added objects (exist in List 1 but not in List 2).
- Removed objects (exist in List 2 but not in List 1).
- Changed objects (exist in both lists, but one or more of their fields have changed).

The first two are solved with a utility method that can compare two lists and return the difference of them. Maybe make a ListUtils class to hold it:

List<PlatformPin> addedPins = ListUtils.difference(pinsList1, pinsList2);
List<PlatformPin> removedPins = ListUtils.difference(pinsList2, pinsList1);

Then make copies of both lists that only contain the common objects:

List<PlatformPin> newList1 = new ArrayList<PlatformPin>(list1);
List<PlatformPin> newList2 = new ArrayList<PlatformPin>(list2);
newList1.removeAll(addedPins);
newList1.removeAll(removedPins);
newList2.removeAll(addedPins);
newList2.removeAll(removedPins);

At this point you should be able to compare the list contents directly, because they're now the same size and contain the same objects (who'se fields have possibly changed):

for (int i = 0; i < newList1.size(); i++) {
	PlatformPin pin1 = newList1.get(i);
	PlatformPin pin2 = newList2.get(i);

	// Compare pin1 and pin2 fields...
}

The difference and intersection methods are pretty simple:

	// return the objects in list2 that are not in list1.
	public static <T> List<T> difference(Collection<T> list1, Collection<T> list2) {
		
		if (list1.isEmpty()) {
			if (list2.isEmpty()) {
				return new ArrayList<T>();
			}
			return new ArrayList<T>(list2);
		}
		
		if (list2.isEmpty()) {
			return new ArrayList<T>();
		}
		
		List<T> difference = new ArrayList<T>();
		
		for (T object : list2) {
			if (list1.contains(object) == false) {
				difference.add(object);
			}
		}
		
		return difference;
		
	}

	// return the objects in list2 that are also in list1
	public static <T> List<T> intersection(Collection<T> list1, Collection<T> list2) {
		
		if (list1.isEmpty() || list2.isEmpty()) {
			return new ArrayList<T>();
		}
		
		List<T> intersection = new ArrayList<T>();
		
		for (T object : list1) {
			if (list2.contains(object) == true) {
				intersection.add(object);
			}
		}
		
		return intersection;
		
	}

This divides the problem into smaller and simpler parts. Hope that helps!

[Nickolas Jennings]

Mike Cox,

Your solution definitely helped me, but it has a problem. When you have this magical function that finds added/removed pins how does it not include 'changed' pins. Feel free to reference my mock table below: There is no way to check a modification vs an addition/removal (That I know even if I'm just eyeing the data versus java calculating it). It's impossible to check for changes since there is no unique key that can be compared across the two connectors. So is Connector1 Pin 2 a modification from Connector 2 or an addition? What about Connector 2 pin 6 F3, a modification or an addition?

Connector 1
Name Function
1 A1
2 B1
3 C1
3 C2
4 D1
4 D2
4 D3
5 E1
5 E2
6 F1
6 F2

Connector 2
Name Function
1 A1
2 B2
3 C1
4 D1
4 D3
5 E1
5 E3
5 E4
6 F1
6 F3
6 F4


My new approach will be very similar to yours, but mutated to fit appropriately.

Just to clarify I'll define PlatformPin

public class PlatformPin {
public string name;
public string function;
public string color;
public string tracer;
public string gauge;
public string option;

// a pin doesn't become unique when combining all characteristics (there may be duplicates, bad data)
}

Back to your
-Added pins (exist in List1, not in List2)
-Removed pins (exist in List2, not in List1)

Will be found using mySQL
Select * from pin p where not exists (select * from pin p2 where p.name=p2.name and p.function=p2.function and p.color=p2.color and p.tracer=p2.tracer and p.gauge=p2.gauge and p.option=p2.option);

This will return all rows that don't exist in the other connector. Applying this to both connectors, and removing from each.

List<PlatformPin> addedPins = pinNotExistOnOtherConnector(connector1, connector2);
List<PlatformPin> removedPins = pinNotExistOnOtherConnector(connector2, connector1);

// If both lists empty, connectors are pin identical

List<PlatformPin> newList1 = new ArrayList<PlatformPin>(connector1);  
newList1.removeAll(addedPins);  
// This is the intersection. Remove the elements in list1 that are not in list 2, (A intersect B = A - (A intersect !B))

Now I can have three parts. (Not all shown but implicitly found)
-A intersect !B
-A intersect B
-B intersect !A

I'd like to present that data in a user friendly way. I believe the best way is two have a "Two" Column Table. One column for one connector. If the pin is in both connectors, display copied data on both sides. If the pin is only on one connector that display only on one column. I also would like the data sorted by name regardless of the fact that it's in one column or the other. This would be sorting the union of these three groups.

[Campbell Ritchie]

Welcome to the Ranch both of you.

Whenever I see code that complicated, (and difficult to read: I had to break lots of long lines), I am liable to go into suspicious mode and not believe it will work at all.
I presume by connector you mean things to join cables together and similar. Can a connector ever change? Can it gain or lose pins ever? Do you have a Pin class and a Connector class? The difference detection ought to go in those classes, just as equals is implemented inside an object.

[Mike Cox]

Nickolas Jennings wrote:Mike Cox,

Your solution definitely helped me, but it has a problem. When you have this magical function that finds added/removed pins how does it not include 'changed' pins. Feel free to reference my mock table below: There is no way to check a modification vs an addition/removal (That I know even if I'm just eyeing the data versus java calculating it). It's impossible to check for changes since there is no unique key that can be compared across the two connectors.

The difference function's purpose is not to find the changed objects, it's only for finding the added/removed ones. The changed objects are found by creating the two new lists and then comparing their contents; that was the point of creating copies and removing the added/removed objects. Your specific solution may be a little bit different, because you have a parent/child relationship between connectors and pins. But what I described is, in simple terms, a correct way to compare two flat lists similar to what you need to do here.

[Winston Gutkowski]

Nickolas Jennings wrote:I have two ArrayList<Object[]> and I want to be able to compare the differences.
...
Below is the code I'm using. This determines whether there are pinDifferences as a flag and a string. The string and flag are then passed to a jsp to print out. (This is in a service.) The problem I'm running into is if the pins are identical (1-17) but the second one has two options for pin #3. I'm printing out that pin 3-18 are different. Because I've sorted it. What I want to do is show that pin #3 doesn't exist on the other connector....

OK, first thing: Your code seems very "targeted". This (diff, or Levenstein distence) is a generic problem (as Mike tried to show you with his <T>-based code), so forget all about "pins" and "connectors", unless they are specifically part of that problem.

Right now, you basically have a 2-D Object array, when what you almost certainly want (as Campbell said) is a List of some type (Connector?), each of which contains an array (or List) of Pins.
The business of "equality" is then up to those classes to implement, after which you can then start to deal with the "diff" side of things.

If I'm right, then two Pins should be "equal" if their IDs (or "numbers") are equal, and two Connectors should be equal if:
(a) They have the same number of Pins.
(b) The Pins in every "position" (probably an index to an internal array or List) are "equal".

If that's NOT the case is where the fun starts...but I suggest that the solution is still a generic one.

My suggestion (and I think it's similar to Mike's): Work on a single-level solution first - perhaps just using Pins; but you could just as easily use Strings for testing if you want (but deal with the Strings as a whole, NOT their individual characters).

Given two unsorted Lists of objects of a specific type, work out the minimum number of changes required to turn ListA into ListB.

And don't even think about anything else until you have THAT working. Once you do, your "diff" for two Lists of Connectors will probably involve two such implementations: one for the Connectors themselves, and one for the Pins they hold; and once you have a basic solution, implementing it for another type should be trivial.

PS: You may also find this page worth a read.

HIH

Winston

[Piet Souris]

Winston Gutkowski wrote:(....)
If I'm right, then two Pins should be "equal" if their IDs (or "numbers") are equal, and two Connectors should be equal if:
(a) They have the same number of Pins.
(b) The Pins in every "position" (probably an index to an internal array or List) are "equal".

Yes, that is what I assumed as well.

However, if you look at the <PlatformPin> class, and at the query that Nickolas
does, you see that two pins are only equal if all fields are equal,
and that Nickolas does not mention anything about connectors.

Mike's solution is what my initial thoughts were, although I would probably
have transformed the list of pins into a hashmap<pinid, Set<Option>).

But since a pin is defined by all its fields, this wouldn't hold anyway.
So my conclusion is that I do not understand the topic well enough to say anything
smart about it.

Greetz,
Piet

[Winston Gutkowski]

Piet Souris wrote:But since a pin is defined by all its fields, this wouldn't hold anyway.
So my conclusion is that I do not understand the topic well enough to say anything smart about it.

Agreed. However, the language Nickolas has used sounds very "diff"-like to me, so it still boils down to having a basic "equality" method. Once you have that, I suspect very strongly that it's simply a case of applying it in the context of a generic edit distance algorithm.

@Nickolas: It would be useful if you could give us a bit more background on this, because one thing that will make a HUGE difference is whether the lists involved can be sorted in order to work out the differences, or whether the results need to be given in terms of the Lists' original sequence.

Winston

[Nickolas Jennings]

Ok, I will give a bit of background to clarify.

The business problem is that I, as an application user, am making a note on a connector in a system. It would save time to apply this note to all identical connectors. I also want to see connectors that are named the same, but have characteristic differences. This is due to the fact that maybe there is bad data, like data input value of "Body_IP" vs. "Body-IP". So as a user I want to be able to see the bad data to go and fix it.

Now as a programmer, I'm in one method. As for the comments that the comparison should be done in the class of the PlatformConnector, I do not want to do this because I would like to follow suit of the project and keep comparisons in Service classes.


The PlatformConnector class, and the PlatformPin Class

public class PlatformConnector {
String color;
String connectorId;
String connectorNo;
String connectorType;
String description;
String device;
String engineeringName;
String gender;
String harnessFamily;
String id; // database id
String partNumber;
List<PlatformPin> pins = new ArrayList<PlatformPin>();
String optionLocation;
String cavityCount;
}

public class PlatformPin {
String circuitName;
String circuitFunction;
String color;
String tracer;
String gauge;
String id; // database id
String name; // like the pin #, can be letters. (Pin A,B,C...)
String option;
}

So that defines the classes that I'm working with. As for the method in service that I'm changing. I'll give a bird's eye view of the method

public class IdenticalConnectorsDTO DetermineIdenticals (PlatformConnector connector, List<PlatformConnector> similarConnectors) {
// connector is the one I'm looking at, and similarConnectors are connectors in the with the same 'connectorNo'. The cardinality of similarConnectors ranges from 0-100

List<PlatformConnector> differentConnectors = new ArrayList<PlatformConnector>();
List<PlatformConnector> identicalConnectors = new ArrayList<PlatformConnector>();

for (PlatformConnector seeker : similarConnectors) {
if(seeker.getPins().getSize() == seeker.getPins().getSize()){
// test all pin characteristics for seeker vs connector. Set boolean pinsIdentical true
// This is where I just have a bunch of if statements in my original post
} else { 
// set pinsIdentical boolean false 
}
}
if (test all seeker characteristics vs connector characteristics, include pinsIdentical boolean) {
identicalConnectors.add(seeker);
} else {
differentConnectors.add(seeker);
}
}
IdenticalConnectorsDTO sortedConnectors = new IdenticalConnectorsDTO(identicalConnectors, differentConnectors, pinDifferences);
return sortedConnectors;
// I can pass anything to the DTO. I would just make a new constructor method in there.
}

So now you may understand the generality of the problem. I'm comparing connectors to see if they're identical. Identical connectors are easy to present to the user. I just print out the Connector, and say it's identical. For the different connectors I print a difference table out. At the moment I'm only printing out connector level differences. The table is simple, the two connectors at the top, column for each. Then each row has a row header labeling the characteristic type (ie: Gender), and then next two columns display each connectors value for that characteristic. As for the pin row, I just combined the two columns and stated "Pin Differences Exist"

I want to be able to print out the differences between the two tables of pins. So to clarify, connectors and pins have database ids that make them unique but they are not displayed to user.


I have read your comments, and I can't find the ListUtils.difference method. I see the intersection,disjunction, add, subtract, and union. I was not aware of the ListUtils, and I'm really liking this package. I use the apache common libraries but have not completely familiarized myself with it yet. As for the diff function, it seems to work on the command line and not in java. The levenstein distence seems extremely complicated. I always dislike the way wikipedia presents a concept to a person new to it. They overgeneralize it to the point that confuses me.

Thanks you all for your input.

[sai rama krishna]

Here is good example on this topic

http://beginnersbook.com/2013/12/how-to-compare-two-arraylist-in-java/

[Aryan S Kapoor]

Here is a good example of comparing two arraylist

https://www.java8net.com/2020/03/compare-two-arraylist-in-java.html

[Campbell Ritchie]

Welcome to the Ranch

I am afraid that isn't a good tutorial; it contains a lot of unnecessary code, very little explanation, and there is a serious error in it. I shall let you find the error.