wood burning stoves*
The moose likes Java in General and the fly likes duplicate lines in a text file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "duplicate lines in a text file" Watch "duplicate lines in a text file" New topic

duplicate lines in a text file

ron poram

Joined: Feb 12, 2011
Posts: 3
hello everyone,

got issue regarding reading duplicate lines in a text file...

im not trying to eliminate the duplicates lines, all i need is to put all the duplicates lines to another file...

for instance, in my file "data.txt" i have duplicates lines


i need to copy the [BBBBB] lines to another file...

how can i do that...

Siddhesh Deodhar
Ranch Hand

Joined: Mar 05, 2009
Posts: 117
Welcome to forum.
If duplicate lines appearing in file are one below the other, than simplest way will be to read file line by line and compare the content of current line and previous line.

Good, Better, Best, Don't take rest until, Good becomes Better, and Better becomes Best.
Sidd : (SCJP 6 [90%] )
Ernest Friedman-Hill
author and iconoclast

Joined: Jul 08, 2003
Posts: 24183

And if theĆ½'re not, you could read them all in and sort them. Alternatively, you can store all the lines in a HashSet as you read them, checking the Set to see if each line has been read before.

[Jess in Action][AskingGoodQuestions]
Kowshik Nandagudi
Ranch Hand

Joined: Dec 09, 2010
Posts: 57
yes... .. you can put it to set

simple steps

1) Create a set

Set<String> lines = new HashSet<String();

2) read the line
put each of them to set

if(!lines add(line)){
// then its a duplicate..
//copy to another file
ron poram

Joined: Feb 12, 2011
Posts: 3
should i use a for loop in the if-else statement to display the duplicate lines...


still newbie in java
ron poram

Joined: Feb 12, 2011
Posts: 3
hello again,

i already figure it out how to do it...thanks for replying to this thread!
It is sorta covered in the JavaRanch Style Guide.
subject: duplicate lines in a text file
Similar Threads
Configure custom properties of a Datasource in WebSphere 7 after creating it with Jython scripts
Why is this code killing almost everything, not just duplicates?
how to get plain text in file using print stream
How to store unique elements in List?
Is nested c:out possible?