Text Crawler

[Mary Brennan]

I'm kinda new to java and I'm currently trying to create a text file crawler. The program will request that file that user would like to start the crawler, a word the user wants to look for and the number of occurrences of the word the user wants to find. Each file will have one or more reference to another file that the program will find and add to a list for the program to search next. Once the program finds the number of occurrences requested the program should stop and output a message showing the total number of occurrences and the number of occurrences in each file. If the total number of occurrences doesn't equal the number requested the program should output the number found. I know how to do the output messages but I am having a slight problem getting the program to stop once the number of occurrences has been found. Can anyone help me with this? It would be greatly appreciated. I have placed the code I have written below:

import java.io.*;
import javax.swing.JOptionPane;
import java.util.*;

public class searchFile
{
	public static Vector<String> temp = new Vector<String>();
	public static Vector<String> occurance = new Vector<String>();

	public static void main(String [] args) throws IOException
	{
		String pattern = "[a-zA-Z0-9]{1,}.txt";
		String line;
		String userInput = JOptionPane.showInputDialog(null,"Enter filename to start","Text Crawler",1);
		File filename = new File(userInput);
		String lineFile = "";
		String wordToSearch = JOptionPane.showInputDialog(null,"Enter word to search","Text Cralwer",1);
		int occur = 0;
		int totalOccur = 0;
		String aFile = "";
		String num = "";
		String total ="";
		String numSearch = JOptionPane.showInputDialog(null,"Enter number of occurrences you would like","Text Crawler",1);
		int wordCount = 0;
		wordCount = Integer.parseInt(numSearch);

		if(filename.exists())
        {
			if(userInput.equals(""))
				JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt");
			else if(userInput.equals(" "))
				JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt"); 
	        else if(!userInput.matches(pattern))
	             JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt"); 
	        else if(userInput.matches(pattern))
	        {
		     try{
			        temp.add(userInput);
        		for(int j = 0; j < temp.size(); j++)
        		{
	        		aFile = temp.elementAt(j);
	        		System.out.println(aFile);
	        		File afileSearch = new File(aFile);
	        		FileReader aFileReader2 = new FileReader(aFile);
        			BufferedReader aBufferReader2 = new BufferedReader(aFileReader2);
        		while((line = aBufferReader2.readLine()) != null)
				{
					line = line.toLowerCase();
					char aChar = 'x';
	       			for (int i = 0; i <= line.length()-1; i++) 
	       			{
		       			aChar = line.charAt(i);
						lineFile += aChar;
            		}
            	
            		String words [] = lineFile.split(" ");
            		if (line == null)
                		break;
            		else {
	            		for(int k = 0; k <= words.length - 1;k++)
	            		{
		            		if(words[k].matches(pattern))
		            		{
		            			System.out.println(words[k]);
		            			if(temp.contains(words[k]))
		            				System.out.println("Word already in list");
		            			else
		            			{
                					temp.add(words[k]);	
            					}
        					}
        					else if(words[k].matches(wordToSearch) && occur <= wordCount)
        					{
	        					occur++;
        					}
            			}
            		}
    				lineFile = "";	
            	}
            	
            	if(occur <= wordCount)
            	{
            		System.out.println("word count has been found");
            		num = Integer.toString(occur);	
                	occurance.add(num);	
            	}
            	else
            		System.out.println("occurences exceeds word count");
                occur = 0;
            aBufferReader2.close();
        		}

        		Iterator iterator;
        		iterator = occurance.iterator();
				if(occurance.size() != 0)
				{
					while(iterator.hasNext())
					{
						total = (String) iterator.next();
						totalOccur += Integer.parseInt(total);	
					}
				}
        		System.out.println(temp);
        		System.out.println(occurance);
        		System.out.println(totalOccur);
        		
    		}
    		catch (Exception e) 
    		{
        	System.out.println("Error: " + e.getMessage());
    		}	
    		
    		}
		}
	}
}

[Steve Luke]

There are a number of things you could do:

1) You have a for loop and while loops. You can break out of them using a break; statement. i don't know if you would need more than one because it is hard to follow how deeply nested your code is (since you don't indent consistently).

2) Use a flag to indicate that the count is above the threshold. And add conditions to the while and for loops to stop when the flag is positive. For example, boolean thresholdReached = false; , then your for loop and while loops would be something like for (...; (Current Condition) && !thresholdReached; ...) and while ( (current condition) && !thresholdReached)

It might be easier if you organized your code with methods (at least). Maybe some Objects to keep track of things...

[Mary Brennan]

Hi Steve, thanks for the reply. I did indent properly however, when I pasted my code in here the indention I did was removed in some places. Not entirely sure why. I've re-entered it below and added the indentation I think was removed last time. I was thinking the same about methods which I think I will implement. I would just like to figure out what is going wrong here first though. I tried a number of other ways including adding conditions to for and while loops but nothing I did seemed to fix the issue.

import java.io.*;
import javax.swing.JOptionPane;
import java.util.*;

public class searchFile
{
	public static Vector<String> temp = new Vector<String>();
	public static Vector<String> occurance = new Vector<String>();

	public static void main(String [] args) throws IOException
	{
		String pattern = "[a-zA-Z0-9]{1,}.txt";
		String line;
		String userInput = JOptionPane.showInputDialog(null,"Enter filename to start","Text Crawler",1);
		File filename = new File(userInput);
		String lineFile = "";
		String wordToSearch = JOptionPane.showInputDialog(null,"Enter word to search","Text Cralwer",1);
		int occur = 0;
		int totalOccur = 0;
		String aFile = "";
		String num = "";
		String total ="";
		String numSearch = JOptionPane.showInputDialog(null,"Enter number of occurrences you would like","Text Crawler",1);
		int wordCount = 0;
		wordCount = Integer.parseInt(numSearch);

		if(filename.exists())
        {
		    if(userInput.equals(""))
				JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt");
			else if(userInput.equals(" "))
				JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt"); 
	        else if(!userInput.matches(pattern))
	             JOptionPane.showMessageDialog(null,"Please enter a valid filename eg. filename.txt"); 
	        else if(userInput.matches(pattern))
	        {
		    	try{
			        temp.add(userInput);
        			for(int j = 0; j < temp.size(); j++)
        			{
	        			aFile = temp.elementAt(j);
	        			System.out.println(aFile);
	        			File afileSearch = new File(aFile);
	        			FileReader aFileReader2 = new FileReader(aFile);
        				BufferedReader aBufferReader2 = new BufferedReader(aFileReader2);
        				while((line = aBufferReader2.readLine()) != null)
						{
							line = line.toLowerCase();
							char aChar = 'x';
	       					for (int i = 0; i <= line.length()-1; i++) 
	       					{
		       					aChar = line.charAt(i);
								lineFile += aChar;
            				}
            	
            				String words [] = lineFile.split(" ");
            				if (line == null)
                				break;
            				else 
            				{
	            				for(int k = 0; k <= words.length - 1;k++)
	            				{
		            				if(words[k].matches(pattern))
		            				{
		            					System.out.println(words[k]);
		            					if(temp.contains(words[k]))
		            						System.out.println("Word already in list");
		            					else
		            					{
                							temp.add(words[k]);	
            							}
        							}
        							else if(words[k].matches(wordToSearch) && occur <= wordCount)
        							{
	        							occur++;
        							}
            					}
            				}
    						lineFile = "";	
            			}
            	
            			if(occur <= wordCount)
            			{
            				System.out.println("word count has been found");
            				num = Integer.toString(occur);	
                			occurance.add(num);	
            			}
            			else
            				System.out.println("occurences exceeds word count");
                		occur = 0;
            			aBufferReader2.close();
        			}

        			Iterator iterator;
        			iterator = occurance.iterator();
					if(occurance.size() != 0)
					{
						while(iterator.hasNext())
						{
							total = (String) iterator.next();
							totalOccur += Integer.parseInt(total);	
						}
					}
        			System.out.println(temp);
        			System.out.println(occurance);
        			System.out.println(totalOccur);
        		
    			}
    			catch (Exception e) 
    			{
        			System.out.println("Error: " + e.getMessage());
    			}	
    		
    		}
		}
	}
}

[Suresh Sajja]

1. Your class name "searchFile" starts with lowercase letter. The first letter will be uppercase. Please read here for code conventions. CodeStyle
2. Your code is all cluttered in main method. If i have to write this, i would start with the following skeleton. If the code is separated as various methods, it has got more advantages: easy to read, easy to understand, less error-prone and many more..

public class SearchFile {

	public static void main(String[] args) {

	}

	private void readUserInput() {
		// read fileName, word, number of occurrences from user.
		// add appropriate validation for each input
	}

	private void searchFile(String fileName) {
		// Search file for a word
	}
}

[Mary Brennan]

Thanks Suresh. We were never taught to name our classes with a capital, that is now something I will continue to do from now on. I know how to write the methods and I will rewrite my code using methods.

[Campbell Ritchie]

There are conventions about how you code: you will find links in this recent post. It is important to conform to conventions, because other programmers will need to read your code.