wood burning stoves
The moose likes Java in General and the fly likes duplicate lines in a text file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of OCA Java SE 8 Programmer I Study Guide this week in the OCAJP 8 forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "duplicate lines in a text file" Watch "duplicate lines in a text file" New topic

duplicate lines in a text file

ron poram

Joined: Feb 12, 2011
Posts: 3
hello everyone,

got issue regarding reading duplicate lines in a text file...

im not trying to eliminate the duplicates lines, all i need is to put all the duplicates lines to another file...

for instance, in my file "data.txt" i have duplicates lines


i need to copy the [BBBBB] lines to another file...

how can i do that...

Siddhesh Deodhar
Ranch Hand

Joined: Mar 05, 2009
Posts: 118
Welcome to forum.
If duplicate lines appearing in file are one below the other, than simplest way will be to read file line by line and compare the content of current line and previous line.

Good, Better, Best, Don't take rest until, Good becomes Better, and Better becomes Best.
Sidd : (SCJP 6 [90%] )
Ernest Friedman-Hill
author and iconoclast

Joined: Jul 08, 2003
Posts: 24193

And if theĆ½'re not, you could read them all in and sort them. Alternatively, you can store all the lines in a HashSet as you read them, checking the Set to see if each line has been read before.

[Jess in Action][AskingGoodQuestions]
Kowshik Nandagudi
Ranch Hand

Joined: Dec 09, 2010
Posts: 57
yes... .. you can put it to set

simple steps

1) Create a set

Set<String> lines = new HashSet<String();

2) read the line
put each of them to set

if(!lines add(line)){
// then its a duplicate..
//copy to another file
ron poram

Joined: Feb 12, 2011
Posts: 3
should i use a for loop in the if-else statement to display the duplicate lines...


still newbie in java
ron poram

Joined: Feb 12, 2011
Posts: 3
hello again,

i already figure it out how to do it...thanks for replying to this thread!
I agree. Here's the link: http://aspose.com/file-tools
subject: duplicate lines in a text file
It's not a secret anymore!