This is the problem am having. My form has a textarea which accepts comma separated records. The form submits to a servlet which takes these rows of records, makes an insert statement after substituting the appropriate keys and date transformations, and writes to the database.
If I submit 1000 records it works. But if I submit some 10,000 - mid way the browser says "could not open this page after waiting for 60 seconds". And only a part of the 10,000 records gets submitted.
Question : I want the browser to be more patient and wait till the servlet gets back. Is it done at the browser level or is a configuration in tomcat? I see a ConnectionTimeout setting in server.xml : Is it for the session timeout or for such browser time out problems?
Thanks a lot folks. [ November 05, 2004: Message edited by: venkatraman kandaswamy ]
did you try changing the ConnectionTimeout setting?
the docs for HTTP Connector had this to say:
The number of milliseconds this Connector will wait, after accepting a connection, for the request URI line to be presented. The default value is 60000 (i.e. 60 seconds).
Which makes me think about initating the session rather than how long the browser will wait, which I really don't think is up to the server.
If I were you, I'd look at optimizing my database methods.
The only time I've gotten such a slow response was when I mistakenly was creating, using once and closing a connection to the database for each record.
Joined: Jul 07, 2004
Thanks Ray for the reply. I tried ConnectionTimeout = 0. But it does not seem to solve the problem. Now am trying to process 500 records at a time, forward to a processing.jsp page which autosubmits to the servlet; servlet continues processing the next 500 records and so on - till all the records are processed. Right now its sitting in some infinite loop.
Joined: Jul 07, 2004
Well I finally got this working. Now I can process an unlimited number of records without running into session, browser time outs.
Heres how I did this:
Step 1 : Once submitted the input records are tokenized by line and the lines are stored in an ArrayList. This ArrayList is stored as a session variable.
Step 2 : I then forward this page to processing.jsp page. Which looks like this :
Step 3 : The processing.jsp submits after a 5 second delay - and then 500 records are processed, a variable called cursor is updated and stored in session so next time when processing.jsp submits - the records until cursor are skipped. The other alternative is to create a new arraylist without the processed records - so this cursor need not be maintained. For me cursor was the lazy easy solution !! Then the action forwards to processing.jsp and the process is repeated till all the lines are processed.
Here is how the action mapping looks like :
I hope this post will help anyone searching for such a web based bulk data processing.