Has any one created a CMP entity for real with aCMR in a table having 10,000 records?

[Mallik Hiremath]

Hello All,
I have two EJB's with CMR (1 to Many) (Dept and Employee) and EMPLOYEE_TABLE has 10,000 records and each one belongs to same dept say dept 1. ( Note that these rows are existing already in the DB)
For these tables I created CMP EJBs and set the CMR relationship i.e set the Dept of Employee in the ejbPostCreate() method of EmployeeBean.
Now I tried to add one more employee to the DB using the CMP EJB Employee with FK relation to dept 1 !!! the process executes the ejbCreate Method but sits forever in the ejbPostCreate() method i.e it sits there for more than 1 minutes and 30 seconds and then the OC4J throws the following exception
------
com.evermind.server.rmi.OrionRemoteException: Transaction was rolled back: timed out
-----
To explain the cause for this please look at the other message posted by me that says "ejbActivate() and ejbPassivate() method calls"
Can some one let me know is there any workaround for this?
I am using Oracle OC4J 9.0.4 release on win 2000 and Oracle 9i Database.
Thank you
Mallik

[Mallik Hiremath]

Just refreshing the question!

[Mallik Hiremath]

Just refreshing the question.
The problem is that, OC4J thinks that I will use the other 10,000 Employee Beans for some other purpose and hence it starts calling ejbActivate() method on these 10,000 beans one bean for each db record!!!
While the setDepartment(dept1_Local)is being executed the CPU is at 100% full and memory consumption of the OC4J shoots up by 25 MB, this is just to add one employee!!! think about adding 10 employee to the DB.
This is on a machine having 512 MB or RAM and 1 GHz CPU.
Does every vendor implement the CMR relationship setting like this? Does any one tested the EJBs with CMR relations in a DB having large number of records ( say 10,000 to 1 million records ) ??? Does any one used EJBs in such an environment?
Thank you
-Mallik

[Mallik Hiremath]

Common guys some one let me know if you ever used CMP EJBs having CMRs in a Databse having large number of records i.e in the order of 10,000 records to 100,000 records or even 1 Million records? Or Is every one using CMP EJB just for fun or just for Toy Programs?
-Thank you
[ September 19, 2003: Message edited by: Mallik Hiremath ]

[Tim Holloway]

Ask me again in about 6 months and hopefully I'll be able to say so from personal experience. However, in the mean time, you might want to inquire in forums like the JOnAs forum at objectweb.org, the JBoss forum at jboss.org, and maybe even the commercial EJB support groups.
I can say this about overall EJB optimization, however. A fair amount of care has been put in most modern servers to avoid what it sounds like you're seeing. If you called up an EJB that had a CMR link to 10K children and attempted to randomly access them, you might be able to trigger such a scenario, but most systems are designed to provide a lazier fetch in order to spare resources while still giving the option of doing enough anticipatory reads to get better performance out of sequential accesses. Usually this kind of stuff is tunable so you can adjust it to your needs.
In the last year, I've been prone to use the Fast-Path Browse J2EE design pattern for read-only sequential access and EJBs where cacheable random access and/or data updates are needed. It's not as "pure" a solution as all EJB or all JDBC, but it works well for me, since it exploits the strengths of both and bypasses many of their weaknesses.
I've said it before, but it bears repeating. Any browser-oriented web application (i.e. not a web service) that indiscriminately attempts to bring up thousands of records at a time is asking for trouble. No one person can view that much info without going blind, and if you actually need a big pool like that because there are lots of users looking at different parts of the data, your hardware's not powerful enough for the task. So I recommend seeing if you can't formulate requests that give smaller result sets.
[ September 19, 2003: Message edited by: Tim Holloway ]

[Mallik Hiremath]

Thank you very much Tim,
At least I got some answer from you!!!
Does any one else has to say any thing about the EJBs in large systems and your experience with EJB (both CMP and BMP ) with and with our CMR relationships in large system?
Thank you
Mallik

[Mallik Hiremath]

It looks like no one wants to comment on the usage of CMP EJBs with CMRs on large Databases!!!
I think I have to discard the inbuilt CMR provided by the container and have to code the CMR for myself, this would give me the control of loading or not loading the childern entities depending on the needs.
Did any one code the CMRs for yourself? what are the pros and cons of coding the CMRs for ourselfs? let me know, thanx in advance!
--Mallik

[Lasse Koskela]

what are the pros and cons of coding the CMRs for ourselfs?

The pros and cons are probably the same as with any other build-buy decision. If you decide to build the functionality yourself, here are the trade-offs:
- you'll need more time to develop the application (more things to do)
- you'll probably not be able to achieve the same level of quality/robustness as a commercial product has (the vendor might have put millions of dollars into the development over several years)
+ you'll get exactly the functionality you need (you won't pay for stuff you don't need)
It's all up to you to decide which option seems better considering your situation.

[Juan Rolando Prieur-Reza]

Originally posted by Mallik Hiremath:
. . .EJBs in large systems and your experience with . . .CMR relationships in large system? . . .
--Mallik

Mallik,
I am interested in the problem you described because I'm facing the same issue. Lets keep in touch about it. Its important to the whole EBJ/CMR technology.
Regarding someone's comment, I don't think the occurrance of 10,000 entities suggests "indiscriminant" application of the CMR construct, nor is our use of CMR restricted to web apps with an inherently small search space.
Regarding "judging" trade-offs...
One thing I would look at is the timeout settings; also space utilization. Sorry I don't know anything about JBoss administration specifically. But it seems to me that with sufficient hardware resources (space and speed) that CMR should work, even with 10,000 entities. But I would want to be able to calculate the speed and space requirements needed before I waste time tweeking designs. So...

Perhaps a little Taguchi experiment would be appropriate: perform a series of trials on progressively larger sets of data and tabulate the response times; apply the usual design-of-experiements techniques.
As in Architecture, you want to know the performance characteristics of the "materials" you are working with so that you can calculate where the critical trade-off values are. Otherwise, we are just kind-of-guessing.
Armed with a performance curve for your particular application and environment, you can then determine what the safe timeout + margin_of_error setting should be (assuming it can be administratively set), or if a different architecture will be needed.
But this above all else, to thine own performance functions be true!
Good luck

[Mallik Hiremath]

Hello john,
Thank you for responding to my post!
Yes its really an issue for those who are trying to use EJB CMP-CMR technology. The EJB Container vendors would have put so much of thought on this before they could really code such a thing!!
But I really did not understand that why in the world the Container is trying to ejbActivate() all those child entities 1:M ( one to many relations) when the many side is not getting changed for any reason!!! Note: the many side may have 10's of 1000s or even million records in the DB. To finish such a transaction if the Container takes the time in the order of minutes its not acceptable!!! this means the EJB-CMP-CMR is not good for large systems unless otherwise they do the lazy loading and even lazy ejbActivating.
Has any one has any thoughts on this? Let us know your comments and your experiences with your EJB Containers
Thank you
Mallik

[Marcos Maia]

Hi,
I have worked in a project where we have db tables with 6.000.000 entries using ejb1.1(no entity relationship, using daos) and 2.0(using entity relationship, no daos) running under weblogic 6.1 and now 7.1, with oracle db and sun solaris on a sun fire 6800 machine and weblogic does not have this king of problem. Also have worked with cmp 2.0 using jboss and boraland app server with small db thought and it worked fine. Don't know why this is happening???

[Mallik Hiremath]

Hello Marcos,
Thank you for responding!!
If you dont' mind, couple questions for you
1. Are you guys using the EJB-CMP CMRs?
2. If your answer is yes for question 1, with in your latest implementation using Weblogic, have you found out that the Container calls ejbActivates() for all those child entities of CMR relationships?
3. Could you please try to print some verbose from your ejbActivate(), ejbPassivate(), ejbLoad() and ejbStore() methods of your CMP beans with CMRs and let us know your experience with the weblogic container?
Thank you in advance
Mallik

[Juan Rolando Prieur-Reza]

why in the world the Container is trying to ejbActivate() all those child entities 1:M ( one to many relations) when the many side is not getting changed for any reason[/QB]

Ah good! Some evidence that the CMR 1:M-large scenario DOES work under some (to be discovered) conditions.
Pardon me for adding only my speculations at this point as to those conditions.
1. The Isolation Level setting of the underlying database and that of the EJB should is a suspect. Clearly, up-front Activation might have been necessary to implement Serializable (even if YOU are not doing updates);
in contrast, something like Read-Uncommitted would easily allow implementation of lazy Activation. I can only imagine the in-between levels' effects . Even if there is no difference in the
EJB implementation directly, the orders of magnitude difference in performance caused by Isolation Level could possibly account for some of
your reported problems.
2. Just to eliminate one obvious confounding variable I suggest changing the Timeout on the database to none, or maximum. Assuming you can
use BMT, I think you can also setTransactionTimeout() on your BMT EJB with SessionSynchronization.
I may have some empirical results on my end in near future.
Cheers!