Brajesh Choudhary

Greenhorn
+ Follow
since May 24, 2012
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
1
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Brajesh Choudhary

Hi All,

I am trying to generate reports in my project using spring batch. I have more than 8 million records in my database. Earlier I have set commit-interval as 1 but after reading some articles, I have set commit-interval as 10000 and page-size as 10000 but still, it's taking more than 44 hours to generate the report. In every itertaion, It is taking 3 to 4 minutes to get the records, processes those records and writing in my CSV file.

Please help me, friends. maybe I am doing something wrong.

Jobs-Context.xml

<job id="reportJob" xmlns="http://www.springframework.org/schema/batch">

<!-- master step, 10 threads (grid-size) -->
<step id="masterStep">
<partition step="slave" partitioner="rangePartitioner">
<handler grid-size="10" task-executor="taskExecutor" />
</partition>
<next on="*" to="combineStep"/>
</step>

<step id="combineStep">
<tasklet>
<chunk reader="multiResourceReader" writer="combineFlatFileItemWriter"
commit-interval="1" />
</tasklet>
<next on="*" to="deleteFiles"/>
</step>

<step id="deleteFiles">
<tasklet ref="debitfileDeletingTasklet" />
</step>
</job>

<!-- each thread will run this job, with different stepExecutionContext
values. -->
<step id="slave" xmlns="http://www.springframework.org/schema/batch">
 <tasklet>
<chunk reader="pagingItemReader" writer="flatFileItemWriter"
processor="itemProcessor" commit-interval="10000" />
</tasklet>
</step>

<bean id="rangePartitioner" class="com.test.RangePartitioner" />

<bean id="debittaskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
   <property name="corePoolSize" value="10" />
   <property name="maxPoolSize" value="10" />
   <property name="allowCoreThreadTimeOut" value="true" />
 </bean>


itemreader bean


<bean id="pagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="gemsDataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="gemsDataSource" />
<property name="selectClause" value="SELECT * " />
<property name="fromClause" value="***QUERY****/>
<property name="whereClause" value="where rn between :fromId and :toId" />
<property name="sortKey" value="rn" />
</bean>
</property>
<!-- Inject via the ExecutionContext in rangePartitioner -->
<property name="parameterValues">
<map>
<entry key="fromId" value="#{stepExecutionContext[fromId]}" />
<entry key="toId" value="#{stepExecutionContext[toId]}" />
</map>
</property>
<property name="pageSize" value="10000" />
<property name="rowMapper">
<bean class="com.hello.ItemRowMapper" />
</property>
</bean>

Please let me know if there is any issue with my code.
2 years ago
Hi All,

I am trying to generate reports in my project using spring batch. I have more than 4 million records in my database. Earlier I have set commit-interval as 1 but after reading your response, I have set commit-interval as 100 and page-size as 100 but still, it's taking more than 2 days to pull the reports. It is taking 3 to 4 minutes to get the records and process the records and pasting in my CSV file. I have set commit-interval as 1000 also but it took more than 20  minutes to get the records and process the records and pasting in my CSV file.

Please help me, friends. maybe I am doing something wrong.

Jobs-Context.xml

<step id="slave" xmlns="http://www.springframework.org/schema/batch">

<chunk reader="pagingItemReader" writer="flatFileItemWriter"
processor="itemProcessor" commit-interval="100" />
</tasklet>
</step>

itemreader bean


<bean id="pagingItemReader"
class="org.springframework.batch.item.database.JdbcPagingItemReader"
scope="step">
<property name="dataSource" ref="gemsDataSource" />
<property name="queryProvider">
<bean
class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="gemsDataSource" />
<property name="selectClause" value="SELECT * " />
<property name="fromClause" value="***QUERY****/>
<property name="whereClause" value="where rn between :fromId and :toId" />
<property name="sortKey" value="rn" />
</bean>
</property>
<!-- Inject via the ExecutionContext in rangePartitioner -->
<property name="parameterValues">
<map>
<entry key="fromId" value="#{stepExecutionContext[fromId]}" />
<entry key="toId" value="#{stepExecutionContext[toId]}" />
</map>
</property>
<property name="pageSize" value="100" />
<property name="rowMapper">
<bean class="com.hello.ItemRowMapper" />
</property>
</bean>

Please let me know if there is any issue with my code.

3 years ago
Hello Friends

I have been attempting to work with Drools and have mostly met with frustration.
My code involves a Bean, a class to read the DRL file but it gives an error given below the stack trace.


my DRL file


My Login Bean


Can you please provide me the correct DRL rule for my login page.I want a DRL file having a rule that username and Password must be 'a'.

Regards
Brajesh
10 years ago
Hello Friends

At one time when I add my CompanyInformationBean by using modal.addAttribute() there I am able to get my values as I print those values . Again when I am trying to get those values of my CompanyInformationBean in another request mapping I got only null values.How I am able to get those values as I have to print those values in my Jsp page.

Here is my stack trace



Here is my controller class


Here is my spring-business-context.xml


Regards
Brajesh
10 years ago
Hi Ken

Thanks for your quick reply.I have solved the previous problem but now a new problem comes.

Here is the stack trace





my spring-business-context.xml


my controller class


There is also a problem when I provide the hbm file or cfg file it gives error that file not found.As you see I have commented that code.
10 years ago
Hi leonardo

I am configuring hibernate in my spring project i hope you are able to configure.I have a question that you are using persistence.xml or not or there is any kind of entry in your web.xml file for your persistence.xml file.can you provide me your sample files which you use in your project just a basic one.
Hi Ken

I don't know how but yesterday code works today but i get a new problem today while integrating hibernate with my spring project.

Here is stack Trace


my spring-business-context.xml


my hibernate.cfg.xml file


my datasource.xml


This is a maven project so I here providing my pom.xml


10 years ago


Hi Ken

First I use *.html but after that I use *.do extension but it also not work for me as you see in my files.

10 years ago
Hi Friends

I'm getting this error

No mapping found for HTTP request with URI [/springmvc/login.do] in DispatcherServlet with name 'dispatcher'.

my web.xml file


------------------------------------------------------------------------------------------------------------------------------------------------------------------------
my dispatcher-servlet.xml

----------------------------------------------------------------------------------------------------------------------------------------
my spring-business-context.xml

----------------------------------------------------------------------------------------------------------------------------------------------------------
my Controller class

Kindly help me
Regards
10 years ago
Hi Everybody

I have asked this question in an interview that

how the server knows that which framework is currently working.There are many frameworks like spring,struts,struts2,wicket.Then how server will understand that which particular framework is working currently

.Any reply will be deeply appreciated.

Regards

[mg]please avoid unnecessary use of colors as it makes the post difficult to read; use quote tags instead
10 years ago