Get to text descriptions and check similarities

[Angus Ferguson]

Hi all,

I would like to get two texts(for example the descriptions of two countries), and check if they belong to the same continent. Two texts from Africa will contain references to Lions so the code can determine that both countries are similar and that maybe they belong to the same continent

Is there any specific aproach or API for implement that?

And also for example if I want to list the events which are scheduled for today. Is there any free feed from which I could get the data?

Regards, Isaac

[Campbell Ritchie]

Sounds like something for the grep tool.

Linux in a Nutshell by Ellen Siever, Stephen Figgins, Robert Love, and Arnold Robbins, page 182, wrote:grep
grep [options] pattern [files]
Search one or more files for lines that match a regular expression pattern. Regular expressions are described in Chapter 7. Exit status is 0 if any lines match, 1 if none match, and 2 for errors. See also egrep and fgrep

Of course, that probably won't work on a Windows® box.

Start by writing down an algorithm you intend to follow. Make it simple. Very simple.

[Ron McLeod]

Isaac Ferguson wrote:I would like to get two texts ...

Can you define what texts are?

[Angus Ferguson]

text one:

A computer is a general-purpose device that can be programmed to carry out a set of arithmetic or logical operations automatically.
 Since a sequence of operations can be readily changed, the computer can solve more than one kind of problem.

Conventionally, a computer consists of at least one processing element, typically a central processing unit (CPU), 
and some form of memory. The processing element carries out arithmetic and logic operations, and a sequencing
 and control unit can change the order of operations in response to stored information. Peripheral devices allow information 
to be retrieved from an external source, and the result of operations saved and retrieved.

I mean descriptions then check both and get how similar they are, in order to know if both topics, for example desk computers and laptops

[Angus Ferguson]

For example this code tell you if two texts are exactly equal

package utils;
import java.io.*;

class compare { 
    public static void main(String args[]) throws IOException {
        FileInputStream file1 = new InputStream(args[0]);
        FileInputStream file2 = new InputStream(args[1]);

        try {
            if(args.length != 2)
                throw (new RuntimeException("Usage : java compare <filetoread> <filetoread>"));
            while (true) {
                int a = file1.read();
                int b = file2.read();
                if (a==-1) { 
                    System.out.println("Both the files have same content"); 
                }
                else{
                    System.out.println("Contents are different");
                }
            }
        }
        catch (Exception e) {
            System.out.println("Error: " + e);
        }
    }
}

But I want to check the percentage of similarities. Any idea, please?

[Campbell Ritchie]

Don't mess around with Exceptions.

public static void main(String... args)
{
   if (args.length != 2)
   {
      System.err.printf("Two command-line arguments required for the two files to compare." +
         "%nUsage: java myPackage.TextDemo file1.txt file2.txt%n");
   }
   else
   {
      new TextComparer(args[0], args[1]).run();
   }
}

You cannot work out percentage similarities until you have an algorithm. What algorithm do you plan to use? Have you searched for such an algorithm?

[Campbell Ritchie]

That loop will not work. I shall let you work out what will happen. After which you will (I hope) never go anywhere near a read() method again.

[Angus Ferguson]

When I try to run it, it shows compilation errors:

 FileInputStream file1 = new InputStream(args[0]);
                  FileInputStream file2 = new InputStream(args[1]);

It says that abstract classes should be implemented, but I dont want that I just want it simple at the moment.

Any idea for change that code and make it works in the simplest way possible?

[Campbell Ritchie]

Don't use input streams for text files. Use file readers or Scanners.
But you will not manage to find similarities unless you have found an algorithm.

 Now is the time for all good men to come to the aid of the party.

Now is the time for all good men to come to the aid of the party.

Your technique will show 100% difference between those two pieces of text because one of them begins with a space.

[Angus Ferguson]

I am trying the next code:

public class Compare {

    public static void main(String[] args) {

        String[] myStringArray = new String[]{"Black", "Shirt", "50"};
        String[] myStringArray1 = new String[]{"White", "Shirt", "50"};
        boolean nicth = false;
        int nichtWords = 0;
        for (int i = 0; i < myStringArray.length; i++) {
            nicth = false;
            for (int j = 0; j < myStringArray1.length; j++) {
                if (myStringArray[i].equals(myStringArray1[j])) {
                    nicth = true;
                }
            }
            if (nicth) {
                nichtWords = nichtWords + 1;
            }
        }
        System.out.println("Result :" + nichtWords);

        int total = myStringArray.length;
        float percentage = 0;

        percentage = nichtWords * 100 / total;
        
        System.out.println("percentage"+percentage);

    }

}

The result shows that 2 words from the first text are contained in the second text. It makes a percentage of 66 % and it is rigth.The algorithm I made is too simple, I would like to find a more human understandable answer for example, in this case 2 of 3 words are equal but 66% sounds like they are plenty of differences when in reality they are nearly equal, of course if the texts would have 200 words each maybe it would make more sense.

Any idea of how to do it more "realistic"?