aspose file tools*
The moose likes Java in General and the fly likes duplicate lines in a text file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "duplicate lines in a text file" Watch "duplicate lines in a text file" New topic
Author

duplicate lines in a text file

ron poram
Greenhorn

Joined: Feb 12, 2011
Posts: 3
hello everyone,

got issue regarding reading duplicate lines in a text file...

im not trying to eliminate the duplicates lines, all i need is to put all the duplicates lines to another file...

for instance, in my file "data.txt" i have duplicates lines

AAAAA
BBBBB
BBBBB
CCCCC

i need to copy the [BBBBB] lines to another file...

how can i do that...

thanks...
Siddhesh Deodhar
Ranch Hand

Joined: Mar 05, 2009
Posts: 117
Welcome to forum.
If duplicate lines appearing in file are one below the other, than simplest way will be to read file line by line and compare the content of current line and previous line.


Good, Better, Best, Don't take rest until, Good becomes Better, and Better becomes Best.
Sidd : (SCJP 6 [90%] )
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24184
    
  34

And if theĆ½'re not, you could read them all in and sort them. Alternatively, you can store all the lines in a HashSet as you read them, checking the Set to see if each line has been read before.


[Jess in Action][AskingGoodQuestions]
Kowshik Nandagudi
Ranch Hand

Joined: Dec 09, 2010
Posts: 57
yes... .. you can put it to set

simple steps

1) Create a set

Set<String> lines = new HashSet<String();

2) read the line
put each of them to set

if(!lines add(line)){
// then its a duplicate..
//copy to another file
}
ron poram
Greenhorn

Joined: Feb 12, 2011
Posts: 3
should i use a for loop in the if-else statement to display the duplicate lines...

thanks!...

still newbie in java
ron poram
Greenhorn

Joined: Feb 12, 2011
Posts: 3
hello again,

i already figure it out how to do it...thanks for replying to this thread!
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: duplicate lines in a text file