aspose file tools*
The moose likes Object Relational Mapping and the fly likes insert 10,00000 records using hibernate with struts2 Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » Object Relational Mapping
Bookmark "insert 10,00000 records using hibernate with struts2" Watch "insert 10,00000 records using hibernate with struts2" New topic
Author

insert 10,00000 records using hibernate with struts2

Prakash Aradwad
Greenhorn

Joined: Jan 17, 2012
Posts: 15
i need to be insert 10,00000 records in table using hibernate at single time
for high performance

i am trying but not still working
Akhilesh Trivedi
Ranch Hand

Joined: Jun 22, 2005
Posts: 1558
Prakash Aradwad wrote:i am trying but not still working


What did you try?
What did actually work or happen?


Keep Smiling Always — My life is smoother when running silent. -paul
[FAQs] [Certification Guides] [The Linux Documentation Project]
debraj mallick
Ranch Hand

Joined: Mar 08, 2011
Posts: 188

hi Prakash,

please give us the full stack trace, so that we could help.
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
If you want high performance, why are you using Hibernate? Have you looked into using ETL tools, or JDBC batch inserts?
Pramod Krishna Murthy
Ranch Hand

Joined: Jun 30, 2009
Posts: 37

Yes, I completely agree with the earlier post. even we did similar thing through batch updates. the performance is good and since it is plain java code, development exception handling, fall back are all easy to code


"Nothing is forever except Change"
Prakash Aradwad
Greenhorn

Joined: Jan 17, 2012
Posts: 15
thanks friend's

ya i replaced hibernate with batchinsert using jdbc it works

through hibernate it is not possible to insert large data quickly.

leonardo ginting
Ranch Hand

Joined: Jul 25, 2011
Posts: 48
Prakash Aradwad wrote:thanks friend's

ya i replaced hibernate with batchinsert using jdbc it works

through hibernate it is not possible to insert large data quickly.



hi prakashh by the way i have same issue with you but i couldnt figure it out why when i inserting 100000 rows it takes a long time?
how you can solve your problem?
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
A common reason is the presence of indexes on the table - every one of which will be updated for each insert. It's common practice to drop indexes before mass inserts, and re-add them later.

An ORM tool adds some overhead, and you normally don't need it for mass inserts. As I said before, that's the domain of ETL tools.
leonardo ginting
Ranch Hand

Joined: Jul 25, 2011
Posts: 48
hi Tim thanks for reply im new in this field anyway is there any other way i could speed up inserting mass rows using hibernate? i've been searching any tutorial for this and i still couldn figure it out by myself
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
Just what has been suggested already: Use JDBC instead of Hibernate. Remove indexes before the insert. Make sure you don't use a separate transaction for each insert.
Kunal Lakhani
Ranch Hand

Joined: Jun 05, 2010
Posts: 622
A common reason is the presence of indexes on the table - every one of which will be updated for each insert. It's common practice to drop indexes before mass inserts, and re-add them later.

I didn't get that? What index are you all talking about?

Thanks


kunal
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
DB indexes on the table you're inserting into
Kunal Lakhani
Ranch Hand

Joined: Jun 05, 2010
Posts: 622
Thanks Tim
leonardo ginting
Ranch Hand

Joined: Jul 25, 2011
Posts: 48
Tim Moores wrote:Remove indexes before the insert.


hi tim thanks again for replying anyway i'm sorry i didn't get that? how could i remove indexes before insert? would you please explain to me how could i do that?
and how about
don't use a separate transaction for each insert

prior i used 1 EntityManager for many insert transaction each table, does it mean what you said about don't use a separate transaction for each insert?
Tim Moores
Rancher

Joined: Sep 21, 2011
Posts: 2408
leonardo ginting wrote:how could i remove indexes before insert? would you please explain to me how could i do that?

I'm talking about the SQL "DROP INDEX" statement. Check your DB's documentation for the exact syntax.

does it mean what you said about don't use a separate transaction for each insert?

Not sure what an EntityManager does by default; as I've said repeatedly, you should not use JPA or Hibernate or any other ORM for mass inserts. But I'm sure the documentation of EntityManager tells you how to start a transaction for multiple inserts, instead of using one for every single one.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
 
subject: insert 10,00000 records using hibernate with struts2