synchronizing and bottlenecks

[Ravi Kiran Va]

Hi ,

How to deal with parallel or simultaneous contacts to database in a big application

We are using Oracle 10g as our database .
At time there can be many onine users (even 100000 sometimes) who can do Transactions .

The flow of our Application is this way JSP ---> servlet ----> EJB ---> DataBase operations

Here i am having a query :

Here i am having a query in order to ensure safe transactions is synchronizing the Java Method (which makes Database operation) is the only solution ??
We are using JDBC Framework right now .

Will synchronizing this way will be not a bottle neck for the Application ??

Please correct me if i am wrong , and please let me know what is the best approach to handle this situation ??

Thanks , please share your ideas.

[Paul Sturrock]

At time there can be many onine users (even 100000 sometimes) who can do Transactions

100000 concurrent users?!

Here i am having a query in order to ensure safe transactions is synchronizing the Java Method (which makes Database operation) is the only solution ??

Synchronization of access to a database is an anti-pattern. Why do you believe you need to do this?

Will synchronizing this way will be not a bottle neck for the Application ??

Yes, it will be a bottleneck.

[Ravi Kiran Va]

Thanks Paul , yes it can be more than 100000 Users as ours is a banking Application .

Why do you believe you need to do this?

Then how to safely handle different users transactions (Inserting or Updating data at same time )??

[Paul Sturrock]

Wow. How many servers do you have supporting that?

Then how to safely handle different users transactions (Inserting or Updating data at same time )??

Not sure I follow. The purpose of transactions is to handle this sort of operation. What issue is it you are concerned about? Do you have a specific use case that is different from normal transactional operations?

[Ravi Kiran Va]

Wow. How many servers do you have supporting that?

two weblogic servers are clustered for this .

What issue is it you are concerned about? Do you have a specific use case that is different from normal transactional operations?

For example for every Online Transaction an Unique Id should be generated (For every request) . (This is used for our reference )

so for example if 1000 users hit the submit button at the same time , so 1000 requests will not be in a queue waiting to generate a unique Id ??

If this seems different from my Actual question :
My question can be treated as does synchronizing a java Method or a block , will give rise to bottlenecks in big application . then how to handle this ??

Please share your experience on this , thanks in advance .

[Paul Sturrock]

Two servers - each handling 50000 concurrent users? Are you sure you mean concurrent users?

For example for every Online Transaction an Unique Id should be generated (For every request) . (This is used for our reference )

Isn't an Oracle sequence the best way to handle this?

The fact you've got two servers accessing one database should show (one of) the follies of synchronizing at the application level. How can you use this sort of synchronization to guarantee safe access to data shared in the database?

[Tim Holloway]

You've got big problems.

If you intend to handle that many concurrent transactions - and especially with only 2 servers - you need to have people on staff who already know the answers to questions like this and have already done things like that.

I know that the watch word of modern day IT is "All You Have To Do Is...", but this is no more "All You Have To Do" related to JDBC than bandaging a cut finger is doing a liver transplant.

In any event, I'd beware of doing this kind of stuff with basic JDBC. When you scale that high, a good ORM with good caching products will get you a lot further in most cases and be a LOT less trouble and expense to maintain.

So, in brief, if this is a project with a budget of 27 lahks and a 6-month deadline and the staff is all fresh out of Uni, start updating the CV NOW and avoid the rush.

[Ravi Kiran Va]

Update the CV

Thanks for the concern , i can take care of myself .

So you mean to say that with JDBC there is no solution for this kind of problems , rather than baring the bottle necks ??

[Tim Holloway]

Ravi Kiran Pattu wrote:

Update the CV

Thanks for the concern , i can take care of myself .

So you mean to say that with JDBC there is no solution for this kind of problems , rather than baring the bottle necks ??

No. But JDBC is a lot like assembly language. Back in the early PC days, it was common for vendors to brag about how their apps were more efficient, because they were in assembler rather than in a high-level language. And, in theory, they were, since ultimately all application code can be/is reduced to machine language, and assembly is just a human-readable form of machine language.

What was overlooked was that this efficiency didn't scale. Assembly language is roughly 10 times more labor-intensive than most high-level languages to code and debug. It led to the use of inefficient algorithms (trust me, I'm still trying to forget all the bubble sorts!), and once optimized in a particular way, the cost of optimizing it in any other way was prohibitive.

In the mean time, compilers finally got to the point where they could product code that was as tight or tighter than all but the best hand-coders on a good day, and globally re-optimize every time something changed, no matter how trivial. It simply wasn't cost-effective to code anything but the simplest apps or low-level system services in assembly language, and especially when the target computer no longer cost more than the combined salaries of the people programming it.

In the case of JDBC, I've seen benchmarks indicating that a decent ORM can have twice the throughput on complex apps, because the ORM can optimize the SQL requests and manage cache. Add on a good management framework like Spring, and you can not only gain efficiency and productivity in the data access, but gain support over the "grunt work" so that transactions are managed and resources are released without the need to code (and debug!) in detail.

So yes, you can do raw JDBC, but the effort is likely to be very expensive, and the cost of maintenance even more so.

However, that was just a side observation. What worries me more about where you're working is that if you don't have expertise available, the transactions may not be properly managed, the load-balancing can't be done, and the performance will be sub-par. Not to mention the potential for data corruption. I spent several weeks optimizing one system, and we only had to to go through 3 million records a day and had basically all day to do it in. And it really helped that I had a really good DBA and an outstanding operations support staff at my disposal.

[Ravi Kiran Va]

Overall it was a good comparision of ORM tools and JDBC framework .