posted 12 years ago
I'm not clear what the problem is here. You know how to read one file, so how is reading 5000 any different? Is it that reading so many files would take a really long time? There's no easy solution to that. You can use a profiler ... a tool that will show where your program runs slowly or uses too much memory. That might make the processing of each file a bit faster.
On the other hand, maybe you're asking about how to implement a duplicate check. That's a bit too vague for me to even guess at. You'd have to give us more information on what you mean for us to be able to help you there.