• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

duplicate lines in a text file

 
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hello everyone,

got issue regarding reading duplicate lines in a text file...

im not trying to eliminate the duplicates lines, all i need is to put all the duplicates lines to another file...

for instance, in my file "data.txt" i have duplicates lines

AAAAA
BBBBB
BBBBB
CCCCC

i need to copy the [BBBBB] lines to another file...

how can i do that...

thanks...
 
Ranch Hand
Posts: 118
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Welcome to forum.
If duplicate lines appearing in file are one below the other, than simplest way will be to read file line by line and compare the content of current line and previous line.
 
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
And if theĆ½'re not, you could read them all in and sort them. Alternatively, you can store all the lines in a HashSet as you read them, checking the Set to see if each line has been read before.
 
Ranch Hand
Posts: 57
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
yes... .. you can put it to set

simple steps

1) Create a set

Set<String> lines = new HashSet<String();

2) read the line
put each of them to set

if(!lines add(line)){
// then its a duplicate..
//copy to another file
}
 
ron poram
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
should i use a for loop in the if-else statement to display the duplicate lines...

thanks!...

still newbie in java
 
ron poram
Greenhorn
Posts: 3
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hello again,

i already figure it out how to do it...thanks for replying to this thread!
 
reply
    Bookmark Topic Watch Topic
  • New Topic