File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Java in General and the fly likes File Parsing - Implementation Feedback needed Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Java Interview Guide this week in the Jobs Discussion forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "File Parsing - Implementation Feedback needed" Watch "File Parsing - Implementation Feedback needed" New topic

File Parsing - Implementation Feedback needed

Saurabh Pillai
Ranch Hand

Joined: Sep 12, 2008
Posts: 524
We need to develop a functionality where user can upload a file, then file gets parsed and saved into database. Here is how I think it should be implemented and I need feedback on this.

We are going to implement it in asynchronous way.

  • User will hand over (upload) the file to server.
  • There would be a daemon process running at specific interval say half an hour. It would parse the file and save it in DB.

  • Now, let's consider that there would be 100 users and they upload the files. Now, if we implement FIFO algorithm , then last user would have starvation. To avoid that we are thinking about having multiple schedulers which will run at the same time and they will divide the files and parsing would be done in parallel.

    Is this OK implementation? The only critical question is how can we make sure two threads/schedulers do not parse the same file? i.e managing task allocation.

    Thank you.
    Paul Clapham

    Joined: Oct 14, 2005
    Posts: 19973

    How do you make sure that two threads don't parse the same file? You don't let those threads choose the files they are going to parse. Have a controller thread which selects the files from wherever they get uploaded to, and have it distribute those files to however many parser threads you choose to create.
    I agree. Here's the link:
    subject: File Parsing - Implementation Feedback needed
    It's not a secret anymore!