File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes I/O and Streams and the fly likes File Handle Limit Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » I/O and Streams
Bookmark "File Handle Limit" Watch "File Handle Limit" New topic
Author

File Handle Limit

Scott Clark
Greenhorn

Joined: Jul 06, 2005
Posts: 4
I've run into what appears to be a 2KB limit on the number of concurrent open files.

Windows 2000 or 2003; JDK 1.4.2
The error occurs in a native method : java.io.FileInputStream.open()
The error does not occur in JDK 1.5

I'm assuming its something along the lines of : the JVM is using some old NT4 limit as a default. I need to find the setting and change it, no luck so far.

The real situation is more complex than the example code below, I am limitted to the OS and JDK version specified.

The code : FileLeak.java
Like the name says, it leaks handles, to demonstrate the limit.
The sleep at the end is just so I can use process viewer in Task Manager to verify the number of handles opened by the java process.

Any help is greatly appreciated, thanks.


[ July 06, 2005: Message edited by: Scott Clark ]
Joe Ess
Bartender

Joined: Oct 29, 2001
Posts: 8710
    
    6

As I understand it, the max number of open files is set by the operating system. If the problem is with a native method (i.e. java.io.FileInputStream.open()) then it's happening outside the VM. This number is a hard limit in Unix-like OS's but in Windows the limit depends on how much paging space exists and how big the files are, so it's a little harder to nail down.


"blabbing like a narcissistic fool with a superiority complex" ~ N.A.
[How To Ask Questions On JavaRanch]
Scott Clark
Greenhorn

Joined: Jul 06, 2005
Posts: 4
I wrote a C app to test the OS, calling the OpenFile() win API. It works fine, doesn't stop until the 50,000 cap I put in the code.
Also, JDK 1.5 works too.
So, yes there is a limit imposed by the OS, but not a 2K, that's something the 1.4 JVM is deciding to do.
Scott Clark
Greenhorn

Joined: Jul 06, 2005
Posts: 4
I did find an answer. I had to register to search Sun's bug logs, basically I'm screwed. Its a limit in the MS implementation of the C run time library.

Bug Log
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24166
    
  30

Why would you need to keep so many files open at once? Surely you can modify your design.


[Jess in Action][AskingGoodQuestions]
Scott Clark
Greenhorn

Joined: Jul 06, 2005
Posts: 4
The real problem started in Weblogic 8 when we went to SP4; on new hardware in a new data center. There is allot of network lag connecting to the new hardware, we think that may be a factor.
I don't think its the application code alone, because we don't have the same problem in our test environment or prod. Maybe something in the code that was not exposed until the lag was introduced, there is allot of I/O between some parts of the app and a back end repository, but mostly just DB access.

It only takes like 6 people hitting the site to cause the error, so it should be easy enough to profile and figure out the cause.

I was just trying to find a way to raise the limit as a fall back plan. If giving it 5000 was to eliminate the errors, then it would be easier to just up the limit for now and do the research as a side task. It would also be nice to have a higher threshold to monitor the leak, see how bad it gets, because I don't think 2048 is an insanely high number for an I/O oriented server process.
[ July 15, 2005: Message edited by: Scott Clark ]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: File Handle Limit
 
Similar Threads
getServletContext() : doubt
Copy file
java.lang.OutOfMemoryError
try without catch
Error printing out from a read in file.