Readability Score

[Kelvin Okornoe]

This task is from JetBrains Academy/Hyperskill.
reference https://hyperskill.org/
Description
In this stage, you will program the Automated readability index. It was introduced in 1968 and a lot of research works rely on this.The index is calculated by the following formula:

score = 4.71 * {characters}/{words} + 0.5 * {words}/{sentences} - 21.43

You can look at different ages corresponding to the different scores by the table in this article.

Also, your program should read a file instead of typing a text manually. You should pass the filename through the command line arguments.

The program should output the score itself and an approximate age needed to comprehend the text.
Use rounding function to calculate the score as integer.

You should also print how many characters, words, and sentences the text has.

The number of characters is any visible symbol (so, in the real text it's everything except space, newline "\n" and tab "\t").

Notice, that the text can contain multiple lines, not just a single line like in the previous stages. You should analyze all the lines.

package readability;

import java.io.File;
import java.io.IOException;
import java.util.Scanner;

public class Main {

    static int numOfSentence = 0;
    static int numOfWords = 0;
    static int numOfCharacters = 0;

    public static void main(String[] args) throws IOException {
        // use args[0] to read in the file name
        //for ide testing
        //String path = "/moocs/jetbrains/Readability Score/in.txt";

        String path = args[0];
        File file = new File(path);
        Scanner scanner = new Scanner(file);

        //reads the file and calculates
        while (scanner.hasNextLine()) {
            String readLine = scanner.nextLine();
            String[] sentence = readLine.split("[!?.]");
            numOfSentence = numOfSentence + sentence.length;

            String[] words = readLine.split("\\s+");
            numOfWords = numOfWords + words.length;
            for (String word : words) {
                numOfCharacters = numOfCharacters + word.length();
            }
        }

        System.out.println("Words: " + numOfWords);
        System.out.println("Sentences: " + numOfSentence);
        System.out.println("Characters: " + numOfCharacters);
        System.out.print("The score is: ");
        System.out.format("%.2f", calculateReadableScore());
        System.out.println();
        System.out.println("This text should be understood by " + readableAge((int) Math.ceil(calculateReadableScore())) + " year olds.");
    }

    //calculates the score of the text.
    private static float calculateReadableScore() {
        final float constVal4_71 = 4.71f;
        final float constVal0_5 = 0.5f;
        final float constVal321_43 = 21.43f;
        float numCharDivideNumWords;
        float numWordsDivideNumSentence;
        numCharDivideNumWords = (float) numOfCharacters / numOfWords;
        numWordsDivideNumSentence = (float) numOfWords / numOfSentence;
        return (constVal4_71 * numCharDivideNumWords) + ((constVal0_5 * numWordsDivideNumSentence) - constVal321_43);

    }

    //This method determines which age range will be able to read the text.
    private static String readableAge(int score) {
        switch (score) {
            case 1:
                return "5-6";
            case 2:
                return "6-7";
            case 3:
                return "7-9";
            case 4:
                return "9-10";
            case 5:
                return "10-11";
            case 6:
                return "11-12";
            case 7:
                return "12-13";
            case 8:
                return "13-14";
            case 9:
                return "14-15";
            case 10:
                return "15-16";
            case 11:
                return "16-17";
            case 12:
                return "17-18";
            case 13:
                return "18-24";
            case 14:
                return "24+";
            default:
                return "Unknown";
        }
    }
}

Calling for review..

[Campbell Ritchie]

Expunge the word “float” from your vocabulary. [At least whilst programming.]
Why have you got so many instances of the keyword static?

Divide your task into smaller parts. You need to read from the file before you do anything else, so show us reading on its own. Don't mix reading and anything else. Do the other tasks, like counting words and sentences (which I think you are doing wrongly) later.

[salvin francis]

Why is your class called "Main" ? Reading the name of this class does not tell me what it does .
Also you have a lot of code in your main method <-- It's a link

[Kelvin Okornoe]

Campbell Ritchie wrote:Expunge the word “float” from your vocabulary. [At least whilst programming.]
Why have you got so many instances of the keyword static?

Divide your task into smaller parts. You need to read from the file before you do anything else, so show us reading on its own. Don't mix reading and anything else. Do the other tasks, like counting words and sentences (which I think you are doing wrongly) later.

I used float because double gives me wrong value/computation, I used double initially.

I will work on the other recommendations, thank you for your time

[Kelvin Okornoe]

salvin francis wrote:Why is your class called "Main" ? Reading the name of this class does not tell me what it does .
Also you have a lot of code in your main method <-- It's a link

I cloudn't change the name of the class because it was part of the skeleton code.

[Campbell Ritchie]

Kelvin Okornoe wrote:. . . double gives me wrong value/computation . . .

In which case a float will be worse. You are getting wrong values because you aren't counting or calculating correctly.

[Kelvin Okornoe]

Campbell Ritchie wrote:

Kelvin Okornoe wrote:. . . double gives me wrong value/computation . . .

In which case a float will be worse. You are getting wrong values because you aren't counting or calculating correctly.

wow!, I need to known what am probably doing wrong.

But my code did pass some test from JetBrains.

[Piet Souris]

What does this line in your opening post mean:
Notice, that the text can contain multiple lines, not just a single line like in the previous stages. You should analyze all the lines.

If the file is:

I was reading a
book. But when
I got sleepy, I
went to bed.

do we have two sentences?

[Kelvin Okornoe]

Piet Souris wrote:What does this line in your opening post mean:
Notice, that the text can contain multiple lines, not just a single line like in the previous stages. You should analyze all the lines.

If the file is:

I was reading a
book. But when
I got sleepy, I
went to bed.

do we have two sentences?

No, am getting 5 sentences...something is wrong.
Will try and fix it.
Thank you

[Campbell Ritchie]

Analyse the whole of the text, not separate lines.

[Liutauras Vilda]

Style point:
since some of the defined variable names don't tell the context what do they represent, just make the readability worse, I'd suggest to have them inlined in the formulae.

That would remove few lines of very hardly readable lines of code. I have in mind the following ones:

final float constVal4_71 = 4.71f;
final float constVal0_5 = 0.5f;
final float constVal321_43 = 21.43f;

Often when the code is cleaner in terms of semantics, mechanics (minimal noise with unnecessary stuff), exemplary formatting and indentation - problem is much easier to solve and with much less frustration.

Comments like this one:

//reads the file and calculates

It doesn't tell much, quite opposite actually, it makes you wonder where the actual file reading is and where the calculation starts. What is the direct relation between them so they aren't separated in different methods (ignore invalid syntax):

sentences = readLines(file)
numberOfSentences = count(sentences)

Once you decompose code into self explaining methods along with their clear names, you figure out that such type of comments become redundant, so you can remove them, and as a bonus you get less places to maintain when the code changes.

Keep code clean and tidy as you write, at least to a bigger portion of it so you don't get lost on what's going on.

Such code as it is now is not only hard to understand, it is also hard to discuss, because everthing is in one place, one method - luckily we have line numbers one can refer to.

[Piet Souris]

@Kelvin

I had a closer look at your code, and I think it is correct. Recognizing a sentence by ? !  and .  seems very reasonable.

[Campbell Ritchie]

Not if you divide individual lines. That is how your code block became five sentences rather than two.

[Liutauras Vilda]

Continuing on the points Piet and Campbell just mentioned, @Kelvin, do you think potentially would be easier if after reading the whole file you'd make everything as one line first?

Not necessarily saying it is the way to go, but certainly one of options. Make the input look in the way you thought it actually is until now.

[Campbell Ritchie]

I would probably join the whole thing into one line, but there are other ways to solve that problem.

[Kelvin Okornoe]

A total refactor and bug solved.

package readability;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;


public class Main {
    public static void main(String[] args) throws IOException {

        printAll();
    }

    private static void printAll() throws IOException {
        System.out.println("Words: " + numberOfWords());
        System.out.println("Sentences: " + numberOfSentences());
        System.out.println("Characters: " + numberOfCharacters());
        System.out.print("The score is: ");
        System.out.format("%.2f", calculateReadableScore());
        System.out.println();
        System.out.println("This text should be understood by " + readableAge((int) Math.ceil(calculateReadableScore())) + " year olds.");
    }

    //reads the file
    private static String readFile() throws IOException {
        String path = "/moocs/jetbrains/Readability Score/in2.txt";
        return new String(Files.readAllBytes(Paths.get(path)));
    }

    //calculates the number of sentences in the file
    private static int numberOfSentences() throws IOException {
        String[] s = readFile().replaceAll("\\s+", "").split("[!?.]");
        return s.length;
    }

    //calculates the number of words in the file
    private static int numberOfWords() throws IOException {
        String[] words = readFile().split("\\s+");
        return words.length;
    }

    //calculates the number characters in the file
    private static int numberOfCharacters() throws IOException {
        String words = readFile().replaceAll("\\s+", "");
        return words.toCharArray().length;
    }

    //calculates the score of the text.
    private static float calculateReadableScore() throws IOException {
        final float constVal4_71 = 4.71f;
        final float constVal0_5 = 0.5f;
        final float constVal321_43 = 21.43f;
        float numCharDivideNumWords;
        float numWordsDivideNumSentence;
        numCharDivideNumWords = (float) numberOfCharacters() / numberOfWords();
        numWordsDivideNumSentence = (float) numberOfWords() / numberOfSentences();
        return (constVal4_71 * numCharDivideNumWords) + ((constVal0_5 * numWordsDivideNumSentence) - constVal321_43);
    }

    //This method determines which age range will be able to read the text.
    private static String readableAge(int score) {
        switch (score) {
            case 1:
                return "5-6";
            case 2:
                return "6-7";
            case 3:
                return "7-9";
            case 4:
                return "9-10";
            case 5:
                return "10-11";
            case 6:
                return "11-12";
            case 7:
                return "12-13";
            case 8:
                return "13-14";
            case 9:
                return "14-15";
            case 10:
                return "15-16";
            case 11:
                return "16-17";
            case 12:
                return "17-18";
            case 13:
                return "18-24";
            case 14:
                return "24+";
            default:
                return "Unknown";
        }
    }
}

Thank for the assistance, looking forward to more review

[Campbell Ritchie]

Afraid I don't like the technique of getting rid of all the spaces. There must be a simpler way to count sentences than that. Much simpler.
Similarly I don't like that way of counting printing characters. The definition of line end characters given in your first post is incorrect, and won't work on a Windows® machine.

[Kelvin Okornoe]

Campbell Ritchie wrote:Afraid I don't like the technique of getting rid of all the spaces. There must be a simpler way to count sentences than that. Much simpler.
Similarly I don't like that way of counting printing characters. The definition of line end characters given in your first post is incorrect, and won't work on a Windows® machine.

Any ideas on how to count the sentences? Please.

[Liutauras Vilda]

@OP

Run through your sentences counting this nonsensical: "Done! A boy with toy. Really? Yes... Oh boy!"

I counted 5 sentences. How many you got?

[Liutauras Vilda]

Liutauras Vilda wrote:How many you got?

No need to answer

[Kelvin Okornoe]

Liutauras Vilda wrote:@OP

Run through your sentences counting this nonsensical: "Done! A boy with toy. Really? Yes... Oh boy!"

I counted 5 sentences. How many you got?

haha!, I got 7 from my code...It should be five.

The bug is probably coming from the regexp.

Thank you soo much, I will fix it.

[Liutauras Vilda]

Yes...

Some help. This is an offending part which screws up things for you. And that happens only when after ... there is some more text.

[Kelvin Okornoe]

Current Implementation. Bug fixed!

package readability;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;


public class Main {
    static String str = "";
    static String path = "/moocs/jetbrains/Readability Score/test2.txt";
    public static void main(String[] args) throws IOException {
        //str = readFile(args[0]); // read file name from console;
        str = readFile(path); // read file from a directory.
        printAll();
    }

    private static void printAll() {
        System.out.println("Words: " + numberOfWords());
        System.out.println("Sentences: " + numberOfSentences());
        System.out.println("Characters: " + numberOfCharacters());
        System.out.print("The score is: ");
        System.out.format("%.2f", calculateReadableScore());
        System.out.println();
        System.out.println("This text should be understood by " + readableAge((int) Math.ceil(calculateReadableScore())) + " year olds.");
    }

    //reads the file
    private static String readFile(String fileName) throws IOException {
        return new String(Files.readAllBytes(Paths.get(fileName)));
    }

    //calculates the number of sentences in the file
    private static int numberOfSentences()  {
        String[] s = str.replaceAll("\\s+", "").split("[!?.]+");
        return s.length;
    }

    //calculates the number of words in the file
    private static int numberOfWords(){
        String[] words = str.split("\\s+");
        return words.length;
    }

    //calculates the number characters in the file
    private static int numberOfCharacters()  {
        String words = str.replaceAll("\\s+", "");
        return words.toCharArray().length;
    }

    //calculates the score of the text.
    private static float calculateReadableScore() {
        final float constVal4_71 = 4.71f;
        final float constVal0_5 = 0.5f;
        final float constVal321_43 = 21.43f;
        float numCharDivideNumWords;
        float numWordsDivideNumSentence;
        numCharDivideNumWords = (float) numberOfCharacters() / numberOfWords();
        numWordsDivideNumSentence = (float) numberOfWords() / numberOfSentences();
        return (constVal4_71 * numCharDivideNumWords) + ((constVal0_5 * numWordsDivideNumSentence) - constVal321_43);
    }

    //This method determines which age range will be able to read the text.
    private static String readableAge(int score) {
        switch (score) {
            case 1:
                return "5-6";
            case 2:
                return "6-7";
            case 3:
                return "7-9";
            case 4:
                return "9-10";
            case 5:
                return "10-11";
            case 6:
                return "11-12";
            case 7:
                return "12-13";
            case 8:
                return "13-14";
            case 9:
                return "14-15";
            case 10:
                return "15-16";
            case 11:
                return "16-17";
            case 12:
                return "17-18";
            case 13:
                return "18-24";
            case 14:
                return "24+";
            default:
                return "Unknown";
        }
    }
}

[Liutauras Vilda]

Kelvin Okornoe wrote:Current Implementation

As Campbell asked, what do you achieve by removing all the spaces before counting sentences? Perhaps trim() would be enough?

[Liutauras Vilda]

private static int numberOfCharacters()  {
    String words = str.replaceAll("\\s+", "");
    return words.toCharArray().length;
}

This bit. String has length() method. So perhaps you don't need to convert to char array too.

[Kelvin Okornoe]

Liutauras Vilda wrote:

private static int numberOfCharacters()  {
    String words = str.replaceAll("\\s+", "");
    return words.toCharArray().length;
}

This bit. String has length() method. So perhaps you don't need to convert to char array too.

Yes!, That will be more efficient.
Thank you

[Kelvin Okornoe]

Liutauras Vilda wrote:

Kelvin Okornoe wrote:Current Implementation

As Campbell asked, what do you achieve by removing all the spaces before counting sentences? Perhaps trim() would be enough?

I am not getting the understanding. Please throw more light

[Kelvin Okornoe]

Am I allowed to reference the code from github? So I don't keep re-posting the code here.

[Liutauras Vilda]

1. coderanch is a forum! Cozy one.
2. coderanchisaforum!Cozyone.

When you split by ! you get array with elements:
1. [coderanch is a forum] [Cozy one.]
2. [coderanchisaforum] [Cozyone.]

So what do you achieve by removing all the spaces?

Only the first and last spaces might affect the result, so really trim() (removes leading and trailing spaces) might be the wise thing to do right after reading the file.

[Kelvin Okornoe]

Liutauras Vilda wrote:1. coderanch is a forum! Cozy one.
2. coderanchisaforum!Cozyone.

When you split by ! you get array with elements:
1. [coderanch is a forum] [Cozy one.]
2. [coderanchisaforum] [Cozyone.]

So what do you achieve by removing all the spaces?

Only the first and last spaces might affect the result, so really trim() (removes leading and trailing spaces) might be the wise thing to do right after reading the file.

Okay, I think I get the idea.

Thank you

[Liutauras Vilda]

Kelvin Okornoe wrote:Am I allowed to reference the code from github? So I don't keep re-posting the code here.

You are, just not all people will be interested to click on the link and go "somewhere". And to keep discussion and the tangible asset on the forum might be useful for future readers, while your Git repository technically can get demolished at any given point in time.

[Kelvin Okornoe]

Liutauras Vilda wrote:

Kelvin Okornoe wrote:Am I allowed to reference the code from github? So I don't keep re-posting the code here.

You are, just not all people will be interested to click on the link and go "somewhere". And to keep discussion and the tangible asset on the forum might be useful for future readers, while your Git repository technically can get demolished at any given point in time.

I understand. Thank you,

[Liutauras Vilda]

For the moment let's assume your calculations are correct.

Do you think would be better if you'd create a class similar reading to TextStatistics, to which you could pass potentially a text or file and be able to:

TextStatistics myFavouriteBook = new TextStatistics(text);
or

TextStatistics myFavouriteBook = TextStatistics.parse(text); // or similar

System.out.println("Words: " + myFavouriteBook.getNumOfWords());
System.out.println("Sentences: " + myFavouriteBook.getNumOfSentences());

[Kelvin Okornoe]

Current state of the code.

package readability;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;


public class Main {
    static String str = "";
    static String path = "/moocs/jetbrains/Readability Score/test1.txt";
    public static void main(String[] args) throws IOException {
        //str = readFile(args[0]); // read file name from console;
        str = readFile(path); // read file from a directory.
        printAll();
    }

    private static void printAll() {
        System.out.println("Words: " + numberOfWords());
        System.out.println("Sentences: " + numberOfSentences());
        System.out.println("Characters: " + numberOfCharacters());
        System.out.print("The score is: ");
        System.out.format("%.2f", calculateReadableScore());
        System.out.println();
        System.out.println("This text should be understood by " + readableAge((int) Math.ceil(calculateReadableScore())) + " year olds.");
    }

    //reads the file
    private static String readFile(String fileName) throws IOException {
        return new String(Files.readAllBytes(Paths.get(fileName)));
    }

    //calculates the number of sentences in the file
    private static int numberOfSentences()  {
        String[] s = str.trim().split("[!?.]+");
        return s.length;
    }

    //calculates the number of words in the file
    private static int numberOfWords(){
        String[] words = str.split("\\s+");
        return words.length;
    }

    //calculates the number characters in the file
    private static int numberOfCharacters()  {
        String words = str.replaceAll("\\s+", "");
        return words.length();
    }

    //calculates the score of the text.
    private static float calculateReadableScore() {
        final float constVal4_71 = 4.71f;
        final float constVal0_5 = 0.5f;
        final float constVal321_43 = 21.43f;
        float numCharDivideNumWords;
        float numWordsDivideNumSentence;
        numCharDivideNumWords = (float) numberOfCharacters() / numberOfWords();
        numWordsDivideNumSentence = (float) numberOfWords() / numberOfSentences();
        return (constVal4_71 * numCharDivideNumWords) + ((constVal0_5 * numWordsDivideNumSentence) - constVal321_43);
    }

    //This method determines which age range will be able to read the text.
    private static String readableAge(int score) {
        switch (score) {
            case 1:
                return "5-6";
            case 2:
                return "6-7";
            case 3:
                return "7-9";
            case 4:
                return "9-10";
            case 5:
                return "10-11";
            case 6:
                return "11-12";
            case 7:
                return "12-13";
            case 8:
                return "13-14";
            case 9:
                return "14-15";
            case 10:
                return "15-16";
            case 11:
                return "16-17";
            case 12:
                return "17-18";
            case 13:
                return "18-24";
            case 14:
                return "24+";
            default:
                return "Unknown";
        }
    }
}

[Kelvin Okornoe]

Liutauras Vilda wrote:For the moment let's assume your calculations are correct.

Do you think would be better if you'd create a class similar reading to TextStatistics, to which you could pass potentially a text or file and be able to:

TextStatistics myFavouriteBook = new TextStatistics(text);
or

TextStatistics myFavouriteBook = TextStatistics.parse(text); // or similar

System.out.println("Words: " + myFavouriteBook.getNumOfWords());
System.out.println("Sentences: " + myFavouriteBook.getNumOfSentences());

Yes! I will do that.
Thank you

[Campbell Ritchie]

Kelvin Okornoe wrote:Am I allowed to reference the code from github? So I don't keep re-posting the code here.

Afraid not. It would take too long to find the current version, and some people are reluctant to go to outside links.

[Kelvin Okornoe]

I am adding a method to count the number of syllables in a word.
This is my implementation so far.

//calculates number of syllables in a file

    /**
     * 1. Count the number of vowels in the word.
     * 2. Do not count double-vowels.
     *    (e.g "rain" had 2 vowels but is only 1 syllable)
     * 3. If the last letter in the word is 'e' do not count it as a vowel
     *    (e.g "side" is 1 syllable)
     * 4. If at the end it turns out that the word contains 0 vowels,
     *    then consider this word as 1-syllable.
     */
    private static void numberOfSyllables () throws IOException {
        String vowels = "aeiouy";
        BufferedWriter fileWriter = new BufferedWriter(new FileWriter("/moocs/jetbrains/Readability Score/debug_word_vowel_count.txt"));
        String[] words = str.split("\\s+");
        for (String word : words) {
            StringBuilder stringBuilder = new StringBuilder();
            String s = "";
            for (int i = 0; i < word.length(); i++) {
                int val = vowels.indexOf(word.charAt(i));
                if (val != -1) {
                    String cha = String.valueOf(word.charAt(i));
                    s =  stringBuilder.append(cha).toString();
                }
            }
            long count = s.chars().distinct().count();
            if (word.charAt(word.length() -1) == 'e') {
               count =  count -1;
            }

            fileWriter.write(word + "     " + count + "   " + s);
            fileWriter.newLine();
        }
        fileWriter.close();
    }

I need a review to make it better and more rigorous.
Thank you

[Peter Ream]

Campbell Ritchie wrote:Expunge the word “float” from your vocabulary. [At least whilst programming.]
Why have you got so many instances of the keyword static?

Divide your task into smaller parts. You need to read from the file before you do anything else, so show us reading on its own. Don't mix reading and anything else. Do the other tasks, like counting words and sentences (which I think you are doing wrongly) later.

I'm not sure this thread is still being watched. I'm an old (however 70 is the new 50) IBM assembly language programer from the early 80s learning java. I've been at it several years, but rate myself a beginner. I love these coderanch exercises? I look for them every month and learn so much. My question is about float. It was never addressed in this exercise. I'm interested in what is wrong here. Better solution?

[Stephan van Hulst]

For most applications that have a legitimate use case for floating point types (which is rare to begin with), double is the default to use. Most processors have native support for double precision floating points.

The keyword float should probably only be used when you have a valid use case for floating point types in the first place (again, not as common as people think) and you know that the target hardware doesn't have native support for double precision, or memory, networking or storage is extremely expensive.

An example of where float might be useful is when accessing sensors or controlling servomotors in small embedded devices.

[Paul Clapham]

And there are a few classes in the standard API which use float values in some way. The java.awt.Font class comes to mind and I expect there are other java.awt classes which are guilty of using float values.