File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes JBoss/WildFly and the fly likes slow web page response times Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCA/OCP Java SE 7 Programmer I & II Study Guide this week in the OCPJP forum!
JavaRanch » Java Forums » Products » JBoss/WildFly
Bookmark "slow web page response times" Watch "slow web page response times" New topic
Author

slow web page response times

jon london
Greenhorn

Joined: Aug 26, 2010
Posts: 8
Hi We are using jboss-4.0.4.GA and are experiencing very slow web page times... on AIX 5.3.0.0, 2GB RAM, 1 processor (Processor Type: PowerPC_POWER5,Processor , clock Speed: 1656 MHz, CPU Type: 64-bit, Kernel Type: 64-bit)

We use quartz scheduler to run lots of scheduled jobs with a threadpool size of 20. We also use Spring, Hibernate and multiple web services within the Jboss container

I am using jconsole to view memory usage, which seems to peak (at 1280MB) every 20 minutes or so, then GC runs and the memory usage goes back down to around 256MB.

Sometimes just logging in takes around 30 seconds or longer (using BASIC JAAS)... This is even when memory and CPU usage of the JVM are very low.

Is there any way you can give priority to web sessions, so that they can use JVM resources first? It's frustrating to see that memory and CPU are available but its still taking so long to navigate our web application.

Would really appreciate any advice here.

This is our lscfg
====

INSTALLED RESOURCE LIST

The following resources are installed on the machine.
+/- = Added or deleted from Resource List.
* = Diagnostic support not available.

Model Architecture: chrp
Model Implementation: Multiple Processor, PCI bus

+ sys0 System Object
+ sysplanar0 System Planar
* vio0 Virtual I/O Bus
* ent1 U9119.590.518191E-V19-C3-T1 Virtual I/O Ethernet Adapter (l-lan)
* vscsi1 U9119.590.518191E-V19-C42-T1 Virtual SCSI Client Adapter
* hdisk8 U9119.590.518191E-V19-C42-T1-L860000000000 Virtual SCSI Disk Drive
* hdisk3 U9119.590.518191E-V19-C42-T1-L820000000000 Virtual SCSI Disk Drive
* hdisk2 U9119.590.518191E-V19-C42-T1-L810000000000 Virtual SCSI Disk Drive
* vscsi0 U9119.590.518191E-V19-C41-T1 Virtual SCSI Client Adapter
* hdisk11 U9119.590.518191E-V19-C41-T1-L890000000000 Virtual SCSI Disk Drive
* hdisk10 U9119.590.518191E-V19-C41-T1-L880000000000 Virtual SCSI Disk Drive
* hdisk9 U9119.590.518191E-V19-C41-T1-L870000000000 Virtual SCSI Disk Drive
* hdisk6 U9119.590.518191E-V19-C41-T1-L860000000000 Virtual SCSI Disk Drive
* hdisk1 U9119.590.518191E-V19-C41-T1-L820000000000 Virtual SCSI Disk Drive
* hdisk0 U9119.590.518191E-V19-C41-T1-L810000000000 Virtual SCSI Disk Drive
* ent0 U9119.590.518191E-V19-C2-T1 Virtual I/O Ethernet Adapter (l-lan)
* vsa0 U9119.590.518191E-V19-C0 LPAR Virtual Serial Adapter
* vty0 U9119.590.518191E-V19-C0-L0 Asynchronous Terminal
+ L2cache0 L2 Cache
+ mem0 Memory
+ proc0 Processor

and our JVM options:
=====================

/usr/java5/bin/java -Xms1280m -Xmx1280m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dprogram.name=run.sh -Djboss.bind.address=10.40.30.229 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8004 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -verbosegc -Djava.endorsed.dirs=/opt/middleware/products/jboss/jboss-4.0.4.GA/lib/endorsed -classpath /opt/middleware/products/jboss/jboss-4.0.4.GA/lib:/opt/middleware/products/jboss/jboss-4.0.4.GA/lib/endorsed:/opt/middleware/products/jboss/jboss-4.0.4.GA/server/all/lib:/usr/java5/lib:/opt/kpi/runtime/jboss/lib:/opt/middleware/products/jboss/jboss-4.0.4.GA/bin/run.jar:/usr/java5/lib/tools.jar org.jboss.Main -Djboss.home.dir=/opt/middleware/products/jboss/jboss-4.0.4.GA -Djboss.server.home.dir=/opt/kpi/runtime/jboss -Djboss.server.base.dir=/opt/kpi/runtime/jboss -Djboss.server.log.dir=/opt/kpi/runtime/jboss/log -Djboss.server.lib.url=file:/opt/kpi/runtime/jboss/lib -Djboss.lib.url=file:/opt/middleware/products/jboss/jboss-4.0.4.GA/lib -Djboss.server.data.dir=/opt/kpi/runtime/jboss/data -Djboss.server.temp.dir=/opt/kpi/runtime/jboss/tmp -Djboss.server.name=managed1 -Dprogram.name=KPI -Djboss.patch.url=file:/opt/middleware/products/jboss/jboss-4.0.4.GA/patch --configuration=/opt/kpi/runtime/jboss


Paul Sturrock
Bartender

Joined: Apr 14, 2004
Posts: 10336


Is there any way you can give priority to web sessions, so that they can use JVM resources first? It's frustrating to see that memory and CPU are available but its still taking so long to navigate our web application.

Have you determined if the bottle neck is memory or CPU bound? It could be IO - have you run a profiler against it?


JavaRanch FAQ HowToAskQuestionsOnJavaRanch
jon london
Greenhorn

Joined: Aug 26, 2010
Posts: 8
Is there a profiler that you would reccommend for AIX? How would I determine and resolve I/O issues?

CPU usage is generally low so I don't think the bottleneck is there. Memory is generally available, especially after GC so I don't think the problem is with memory.

Lester Burnham
Rancher

Joined: Oct 14, 2008
Posts: 1337
Memory is generally available, especially after GC so I don't think the problem is with memory.

Given the earlier statement
which seems to peak (at 1280MB) every 20 minutes or so

and
-Xms1280m -Xmx1280m

I'd say something is a bit odd memory-wise. There are apparently a lot of objects being created and junked in a short period of time; both creating and GC'ing 1 GB of objects takes time - is all that really necessary?
jon london
Greenhorn

Joined: Aug 26, 2010
Posts: 8
regards I/O

looking at topas (AIX) I can see that the top two hards disks are continously working very hard...



unfortunately I know very little about how hard disk utilisation works on AIX.

I have also noticed that our log4j logging is in debug mode and the logs are constantly being written.

Reading here, http://www.jboss.com/pdf/JB_JEAP4_3_PerformanceTuning_wp.pdf, it seems that verbose logging could be one of our issues, which ties in with Paul's comment on I/O issues.

Lester I have also thought that our memory usage cycles are odd... We trigger many jobs via quartz (1000's per day) and these jobs (Extract Transform Load) use alot of objects including RMI, and hibernate via Spring. So the creation of so many objects is not such a surprise, however I think we need to tweak our garbage collection config to stop the memory peaking out so often. Can anyone offer any GC tips for such an application?
jon london
Greenhorn

Joined: Aug 26, 2010
Posts: 8
Also, is it normal for PgSp to be always increasing and never reducing in size?

 
It is sorta covered in the JavaRanch Style Guide.
 
subject: slow web page response times