Joel Sander

Ranch Hand
+ Follow
since Jan 09, 2009
Cows and Likes
Total received
In last 30 days
Total given
Total received
Received in last 30 days
Total given
Given in last 30 days
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Joel Sander

Bear Bibeault wrote:Mixing the assignment within the condition is a bad practice. You need the cast for the assignment. Refactor the code to assign before doing the check.

Let me see if I understand. Is this what you are suggesting:

7 years ago

Bear Bibeault wrote:

Joel Sander wrote:Interesting and informative. I'll look into filters. Thanks! Are you saying that using Java to create server content is completely antiquated? Does that mean that all modern servers use php?

No. I said using Java in a JSP is antiquated. Java itself, outside of a JSP, is alive and kicking.

Java code belongs in Java classes. It does not belong in a (modern) JSP.

Tomcat does sometimes swap the authentication tokens between sessions and no information or ideas about how to go about debugging this.

No, Tomcat doesn't. It must be something in your code.

I would start with cleaning it up as outlined earlier (removed new session check and extraneous cast). Then we can talk about how to further diagnose this.

Thanks for clarifying on the code belonging in Java classes. 98% of my code is in classes.
I can remove the session check; that will take about a week since this is a weekend/evenings project. I'll post back when I've finished that.

The cast I think you are talking about is this one: (UserToken)session.getValue("uToken"). It is required by Java's syntax. The pages won't compile without it, so I must be misunderstanding you somehow again.
7 years ago
Interesting and informative. I'll look into filters. Thanks! Are you saying that using Java to create server content is completely antiquated? Does that mean that all modern servers use php?


While interesting, I'm still facing the issue that Tomcat does sometimes swap the authentication tokens between sessions and no information or ideas about how to go about debugging this.
7 years ago

Bear Bibeault wrote:Here's a few questions:

  • Why are you bothering to test whether the session is new or not? Either the session contains the token, or it doesn't. Why the extraneous check?
  • Why isn't this in a filter rather than being cut/pasted into every page? (Or a prelude at miniumum?)
  • Why are you using obsolete Java scriptlets in a JSP? Is this 12-year-old legacy code? If not, then you're using obsolete practices and technology from over a dozen years ago.

  • Thanks for the fast reply! That is one of the great things about this site!

    * I don't remember why; I'm working with a system that was originally written several years ago. Seems like a reasonable "make sure nothing strange is happening" check.

    * This is going to be a dumb question, but what is a "filter"?

    * The code is primarily in server pages serving out text to javascript/html5 front pages. My understanding is that using JSP this way isn't a 12-year-old legacy style. If that is incorrect, it likely doesn't make sense to rewrite hundreds of pages to upgrade and why does Apache still make new versions of Tomcat?
    7 years ago

    William Brogden wrote:Looks like the userToken variable is shared by all requests executing that JSP. Why are you surprised?

    Possibly because looking at a JSP does not really give you an idea of the resulting actual servlet code - look at the .java file being created by your JSP for illumination.


    Hi Bill,

    Sorry, I don't understand what you wrote at all.

    Each session should have it's own userToken. I can't imagine the userToken doing anything wrong here itself. Unless Tomcat starts using the userToken stored in person A's session for person B, then I don't see how this can happen.
    7 years ago
    My server has a rare session mix that sounds something like this one:

    except that I observe it on rare occasion regardless of the number of users.

    From looking through my code, the only type of request.set is the following:

    I do store a user token on login and just about every page retrieves it to get user data and as to use to make calls on the underlying servers:

    Any suggestions would be appreciated.
    7 years ago
    Hung twice more today. Any suggestions at all?
    10 years ago
    I've new information from calling jstack during one of these "freezes". Any help would be greatly appreciated!

    Here is the relevant portion of a jstack from the Database Server:

    And here is the relevant portion from Server B:

    What are my options for handling this? "Hanging" is a real issue - I currently am forced to call the shutdown() method on the Database Server causing Server B's rmi request to fail at which point it restarts the Database Server and resubmits the same BasicStructure without a problem. Rmi timeout and retry, or an Exception being thrown would be acceptable behavior that could easily be handled.
    10 years ago

    William Brogden wrote:What is the CPU usage when the system freezes?


    Good question. I'm fairly sure that it is normal (I don't remember checking, but it is the kind of thing I'd be unlikely to miss). I'll make sure to check next time it freezes which may be a few days.

    To be clear, the entire system doesn't freeze, just Server B's update process. Call from the JSP Pages continue to go through without trouble.
    10 years ago
    I have a real puzzle with "freezing" that I've been struggling with for over a year now. This has been perhaps the best java forum that I've been a part of, so I'm hoping that you can help. First, I'll describe the layout (see attached flowchart), then the problem.

    The Layout
    JSP pages and Server B both make calls to a mysql database through some intermediaries. All calls to the database are brokered by the Database Server. To get Object O from the database, the Server B would make the following rmi call: databaseServer.getObjectO(); A JSP page makes a similar rmi call, but through an additional interface class, JSP Interface Class, which in turn relays the call to Database Server. For example:

    JSP page calls jspInterfaceClass.getObjectO(); causing the instance of JSP Interface Class to return the results of the call databaseServer.getObjectO();

    The Problem
    Occasionally, an update call from Server B randomly hangs as it waits for the update method to return. It is always the same updateMyObjects(Object[] myObjects) method that hangs. The update method doesn't always hang. It succeeds a few thousand times for every hang.

    Debug Info
    Here are the possibly suspects that I've considered:

    bad/invalid data/calls - rerunning the Server B's process in a way that produces identical results does not result in a hang.

    mysql deadlocks - I didn't find any mysql deadlocks despite checking the processlist during a hang.

    rmic failures - This doesn't seem likely since I'd naively thing that they should happen for all rmi calls equally.

    database server crashes - The first line of the updateMyObjects prints out to a log file. I've verified that this line does not print during a hang.

    restarting the database server - I put the call to the update method in a try block and put a long pause, reconnect, and recall the same update method in the catch block. Upon getting a hang, if I restart the Database Server the method runs without trouble.

    synchronization - I've tried making every method in the Database Server that communicates with relevant mysql table be synchronized. This didn't help the problem at all. Al methods in the JSP Interface Class are synchronized.

    Any ideas?
    10 years ago

    Martin Vajsar wrote:Just an idea: some connection pools cache PreparedStatement objects (for every connection in the pool) so that they can properly reuse them with the database and save parsing time. Maybe some other objects are cached for various reasons too. Is the leak proportional to the number of calls to your methods, or does it level off with growing number of calls (indicating the caches became saturated)?

    This situation is probably worsened by the fact that in the updateDBObjects method you create many different versions of prepared statement, by not using bind for one of the values (line 71 in the corresponding printout). If I'm right then binding this value should greatly reduce the leak, because the cache will recognize you're creating identical statements and will reuse the cached version.

    I haven't much experience with hunting memory leaks though, so please be forbearing if I'm way off the mark.

    Just wanted to follow up here. I spent several hours tonight testing this theory and it seems to hold water: making the suggested change did work(!!) and further I found in the API the ability to turn off caching for the pool. I undid the suggestion and tried running with disabled caching for the connection pool and that worked as well! Thank you so much!

    One small followup question: do you have any feel for if disabling the pool's caching adversely impacts performance?

    Martin Vajsar wrote:Hi Joel,

    I've learned about prepared statement caching from documentation to Oracle's connection pool I'm using.

    I assume memory profiler should help you to determine whether the statements are cached. Run the original version of your code with it, since it creates lots of distinct prepared statements and if they are indeed cached, they should be spotted easily.

    Your modified code is almost right, you just need to specify a value for the fourth variable using ps.setInt(4, dbObjects[i].id).

    You should also reconsider how you are measuring the memory usage. The amount of used memory you compute will steadily rise until GC collects objects that are no longer used, and this may not happen for a long time. That the amount of used memory steadily rises at the beginning of the execution is not necessarily a sign of memory leak. If you didn't get an OutOfMemory exception with the connection pool you're using now, you need to use a memory profiler to determine whether there really is a memory leak.

    Thanks again for the helpful advise. I'll try the batch prepared statements and report back.

    I haven't looked at memory profilers in years. Would you have any advise for which one might work best for profiling this stand alone server (if need be I can make external calls to the getall and updateall methods when desired).

    Paul Clapham wrote:So this isn't a resource leak, right? You don't run out of connections, as far as I can see. You're suggesting that memory is being leaked by that package. In which case, have you done any profiling to see what's going on?

    Edit: I see Martin has been posting while I have been goofing off. His theory sounds not bad, and a profile should be able to test it quite readily.

    I don't run out of connections. When I used my own homebrewed connection pool, I'd get the memory used:

    during each call to get a pooled connection and could see the memory usage rise (and occasionally fall, but the trend was always upward) until the server started having out of memory exceptions.
    Hi Martin,

    Thanks for the thoughts. Please excuse my ignorance, none of my experience chasing memory leaks in java extends to database work.

    Do you know of a way to test whether the connection pool is trying to cache prepared statement objects? Could you explain how caching would defeat/override the explicit ps.close()?

    I'm a bit confused by the issue with mentioned on line 71, since line 71 only appends to a StringBuffer. Are you suggesting that I modify the for loop to something more like this to see if the memory leak scales with # of prepared statements created:

    I've been trying to update my database server, which connects to a mysql database, to use connection pooling. Without pooling, it runs without any memory leak issues. When I add connection pooling it begins to memory leak. I've tried creating my own connection pool and am now using snaq's pool:

    I've reproduced the memory leak by repeated calling the getAll and updateAll methods listed below. As far as I can tell these methods close all connections properly which makes me wonder if a single pooled connection is storing info about all past uses until it runs out of memory so that it can do a "global" rollback.

    Here is the getAll method:

    The following memory leaks:

    Any help would be greatly appreciated. I'm really running out of ideas to solve this.