using BufferedReader and looping

[vasu singla]

I am using BufferedReader to read through a csv file, there could be 100,00 rows or may be half million rows in the csv file. I have following questions:

1. is BufferedReader an efficient way to read the file and process the data
2. how do i stop at each 25,000 row --> do some processing, then come back again further from the BufferedReader. I am not able to mark the BufferedReader properly to read next 25,000 records after i have read first 25,000 records

flow of my program is as follows:

for(loop through n number of csv files)
{
in = new BufferedReader(new FileReader(fname));

read the 1st line of the csv file to retrieve the header info.

while (ln = in.readLine())!= null) // read through next 25,000 records
{
add all the records in an Array
break out of while as soon as i have read 25,000th record
}

do some processing with the data stored in array above

if there are more than 25,000 records go back to the next 25,000 //HAVING trouble with this implementation of going back and read through next 25,000
else go to next csv file in the loop

}

any help is appreciated
thanks

[Carl Trusiak]

Break the application up into logical methods for each step. An exmple using pseudo-code:

 public void processAll(List csvFiles){
   //loop through the list of files
     //call getReader()
     //call processReader()
     //call cloaseReader()
 }
 
 private BufferedReader getFile(String fileName){
   //open the file return A new BufferedReader for it
 }
 private void closeReader(BufferedReader reader){
   //close the open stream
 }
 
 private void processReader(BufferedReader reader){
    //call getList()
    //test to see if the returned List is empty
    //if empty end
    //call processRecords()
    //loop back to start of method, can be a recursive call
  }
 
  private List getList(BufferedReader reader){
    //create a new List
    //loop reading in each line counting The number of Lines
    //add the line to the List
    //if the number Lines read = stop count or there are no more lines to read return List
  }
 
  private void processRecords(List records){
    //whatever you need to do
  }

By breaking this down into seperate methods, it makes it easy to gain the finctionality you want.

I used pseudo-code to do this, stepping back and using some form of analysis to get through a problem helps. This anaylsis could also be done using a Sequence Diagram to determine calls in the system and identifying methods a class needs to perform a function.

[vasu singla]

thanks carl
will try to implement this