Monitoring is more than just performance. For example you can monitor errors (http errors, jdbc errors), and how users use your application. Below I created a list off the top of my head of things that one might consider monitoring. There may be more standard terms than the ones I came up with for them. If there isn't a standard list it is time our industry came up with one, as far too often we talk in a fuzzy way when we discuss monitoring.
Here is a list of things I like to monitor. Some of them are 'performance' related others are not. The hope is that by having a better handle on these things we quickly find and solve any problems our application might encounter.
- Performance Monitoring/Execution time (These are typically what people refer to when they discuss performance) - Execution time of Web Pages, High level business methods (transferFunds, purchaseItem etc), SQL/JDBC, IO (file reads, network transfers, messages), Low level java methods (this type of monitoring is typically done with a profiler)
- Error/Exception Monitoring - Java Exceptions, http errors, SQL/JDBC errors, IO errors, network timeouts, errors in log files
- Resource Monitoring - Free memory, free disk space, cpu utilization. This could also include items like the number of bytes sent over the network.
- User Usage Monitoring - Web Page hits, peak times, time spent on web site, who are the power users
- Security Monitoring - Invalid login attempts, attempted accesses on protected resources
- Scalability Monitoring - Scalability monitoring interacts with some of the other items in the list. For example do execution time and errors increase as users increase.
The above types of monitoring can be applied at different levels such as hardware host, software server (i.e. database server, web server etc. More than one of these can run on a host), cluster level, and application level to name a few. Another important concept is whether monitoring is active (alerts) or passive (reports). It is also important to distinguish between the monitoring that is done in the applications development, test and production environments. Each might have different needs.
I like to build perpetual performance testing in the system, so you catch performance problems early on. However, this takes discipline to not tune constantly just because you can make things faster.
JAMon provides ways to pretty easily do some of the things listed above. Web Page/SQL/JDBC/Log performance and error monitoring can be done without changing your code using Servlet filters, JDBC JAMon proxy driver, and log4j JAMon appender which can all be configured in minutes external from the code. There are many other tools out there that all have strengths and weaknesses depending on your particular needs and budget.
Also look at the Java Performance FAQ listed below.