The problem is solved in a way, but still not completely understood and is only tested on Gentoo :
It seems that JNI uses the stack size with a default size of 2M.
You can increase it by setting Xss : for example
java -Xss65536k
But there seems to be several issues related to this.
The stack size used by JNI is related to the stack size of your operating system, you can view your current stack size by calling ulimit -a
You can set your operating system stack size by calling something like ulimit -s 65536
You can test your current stack size by running my little test program.
The problem I had was that because the stack size is related to the operating system it seems that when Xss exceeds the stack size of the OS, the size of the used stack of JNI will automatically be limited to the minimum of 2Mb and your stack size.
Still the figure of 2Mb still keeps to be a little vague to me.
But by setting your stack size by using ulimit and setting Xss to the same value, you get the best of both worlds. But only in the situation that you spawn a new
thread and use that specific thread to execute your program, the main program still seems to be limited to the 2Mb limitation. So use the thread instead of the main program, and you will have an extended stack size. Strange isn't it?
Please also note however that when Xss exceeds the stack size of your operating system, even by one 1 byte. You will be punished and your stack will decrease to a max of 2Mb.
Currenlty I'm looking at how setrlimit could be used so that it isn't necessary to set Xss during execution of the JVM, but to set the stack size in a more dynamic way during the execution of the program.The program is use to trace the system calls is called strace, but maybe there is a better solution.
And ofcourse I didn't solve the problem on my own, otherwise I didn't have to post the question at all :-), it was a joined effort of myself and Peter Trevellick. Great help, thnx!
[ August 04, 2005: Message edited by: Arnold Reuser ]