Too many connections

[Ulf Dittmer]

It's hard to say where the slowdown is coming from without knowing what the code is doding for each record. Are objects being created for each record? Is some work done locally before hitting the DB? How much memory are you allocating to the JVM? What happens with each record after the thread pool limit is exhausted? How is the code even dealing with 80000 records if there are only 30 threads?

I tried a couple of runs with 5 threads and each time the results were not comparable to when i took 30 threads.

The point I was trying to make was not whether the sweet spot is at 5 or 10 or 30 threads; the point was that using 20000 will have a major negative impact on performance.

[mayank gupta]

Are objects being created for each record? Is some work done locally before hitting the DB?

Hardly any objects are being created. Most of it is math calculations inside the functions.
This is what happens:

 
ExecutorService threadExecutor = Executors.newFixedThreadPool( 30 );

while ( ((cdr=raf.readLine())!=null))  
{  
  threadExecutor.execute( new CreditCalculatorThread(cdr) ); // start task
}

If method execute is called and all the threads in the ExecutorService are being used, the Runnable will be placed in a queue and assigned to the first thread that completes its previous task.

How is the code even dealing with 80000 records if there are only 30 threads?

I dont know. Maybe thats why it takes so long.
But when i did the test with 10,000 records also my performance was bad:
Time elapsed = 1019.74 sec, CDR=10000, no of threads= 30 maxActive= 35

Even with 4000 records it wasnt a remarkable performance:
Time elapsed = 179.25sec, CDR=4000, no of threads= 30

And i did understand the point that you were trying to make about threads. I just wanted to check what would be optimal.
Increasing the thread size would only worsen the performance, what else could be the bottle neck?

[Ulf Dittmer]

mayank gupta wrote:Hardly any objects are being created.

I beg to differ:

 while ( ((cdr=raf.readLine())!=null)) {  
  threadExecutor.execute( new CreditCalculatorThread(cdr) ); // start task
}

This looks like 80000 CreditCalculatorThread objects are being created. Depending on how much memory these objects consume, this could be a very substantial amount of memory. Depending on how much memory is allocated to the JVM, it could be spending most of its time doing garbage collection.

That's the reason I kept saying from the beginning: Don't allocate that many threads. Rewrite your worker threads so that each can work on multiple records sequentially.