aspose file tools*
The moose likes Java in General and the fly likes Slew of null pointer exceptions. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Slew of null pointer exceptions." Watch "Slew of null pointer exceptions." New topic
Author

Slew of null pointer exceptions.

Nicholas Jordan
Ranch Hand

Joined: Sep 17, 2006
Posts: 1282
Been prototyping an Alice - looking glass class, no Phd in cs or anything like that.
Program intent is to look through files for unauthorized use of email in business environments, printing out:
badWord : lineNumber : fileName : the actual line in the file where found.
I got enough of the program working that I got word frequencies in a format I could code, but there are some issues buried deep in the vm that I cannot drill down to with:
  • J:\java\bin>
  • java -version
  • java version "1.2"
  • Classic VM (build JDK-1.2-V, native threads)
  • Run Log from whatever text files I could find laying around:
    I have a hook in the launcher where I can limit the number of open file handles, the run log continues as above for quite awhile, but I do not want to patch for winnie's low-carb consumer kernel / 20 max open file handles instead of Solaris 60, only to have the program intent fail in unpredictable ways when in the hands of the user.

    I tried placing try/catches all over the place - but the method is not effective. I have the sources here, I can do a build on JDK-5, probably will in a moment.

    Question: Is the principal area of failure most likely too many open file handles ?
    I tested once on this basis and got improved execution reliability. I went back this morning and fixed earlier issues ignored to get it to sputter, but want to ask in advanced fora whether limiting MAX_OPEN_FILE_HANDLES to reasonable value in a ternary decision in the load phase is likely to introduce an additional masked failure route (?)
    [ November 02, 2006: Message edited by: Nicholas Jordan ]

    "The differential equations that describe dynamic interactions of power generators are similar to that of the gravitational interplay among celestial bodies, which is chaotic in nature."
    pascal betz
    Ranch Hand

    Joined: Jun 19, 2001
    Posts: 547
    no, probably not because too many open fiel handles as this would produce (i quess) a IOException (or a subclass of this).

    NPE (NullPointerException) means that you are calling a method or acccessing a field on a null reference:




    Without seeeing the WordCount.readWords() method (the method where the NPE occurs) it is hard to tell/help.

    can you post some code ?

    pascal
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by pascal betz:
    (snip)...can you post some code ?

    The sources are 65,000 bytes at 73% compression under WinZip's Limpel-Ziev
    This is the loop in main() where I launch the instances, the blowouts are occuring all over the place, often Compiled Code) is all the stack trace gives.

    The stdout runs some 232 lines of these stack traces and from the general pattern of that file and how the program works, it looks clearly that MAX_OPEN_FILE_HANDLES (spare me, that's a manner of speaking for clarity) is failing because each of those starts needs an additional two file handles for building standard dictionaries alongside of word/freq dictionaries.

    The code I am really wanting to inspect tightly is:That's straight from the latest build two hours ago.
    It's the core loop - for first phase of program - and if you want to really look at something, dig into this and let me know what you think.
    [ December 03, 2006: Message edited by: Nicholas Jordan ]
    pascal betz
    Ranch Hand

    Joined: Jun 19, 2001
    Posts: 547
    some hints:

    - according to the stacktrace you showsed the NPE does not occur in the fetter() method but in the readWords() method. It just bubbles up and you catch it in the catch clause of the fetter() method. So can you show the code for the readWords() method ?
    - you should always close resources (streams, ...) in a finally clause
    - if you use import statements then you do not have to specify the fully qualified classname. this makes the code more readable.

    pascal
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by pascal betz:
    - according to the stacktrace you showsed the NPE does not occur in the fetter() method but in the readWords() method. It just bubbles up and you catch it in the catch clause of the fetter() method. So can you show the code for the readWords() method ?
    - you should always close resources (streams, ...) in a finally clause
    - if you use import statements then you do not have to specify the fully qualified classname. this makes the code more readable.


    - according to the stacktrace you showsed the NPE does not occur in the fetter() method but in the readWords() method.
    Yes - most definitely - bear with me a minute here untill I get back to the original motivation for the post...


    -It just bubbles up and you catch it in the catch clause of the fetter() method. - correct, I intended from the start to use the bubble up method to avoid what develops into the debug/release versioning problems. In reliability engineering, this is called destructive testing. I have 50,000 hours and 20 years of what amounts to building up skills in this arena. ONe of the team members is formally degreed in exactly this discipline.

    - you should always close resources (streams, ...) in a finally clause. I will implement this advice, directly througout my project ... the problem is that I am unsure how to go about this ..... you get into a cascading catch chain, closes can throw ? correct ? When I go reading File.java and streams/writers,... it always degrades into information overload trying to sift through the DBCS/UNICODE v byte/char issue which, as I am sure you are all too aware, can become a house of mirrors in a field of clowns trying to be effective at reliability engineering.

    - if you use import statements then you do not have to specify the fully qualified classname. this makes the code more readable.

    My C++ compiler issues a diagnostic on namespaces, I do not want to develop coding habituations that will haunt me when I cross-code other parts of the project in C++ ..... I did a rough thumbnail on my work since midnight, it clocks in at about 36,000 keystrokes, I have no patience for games.
    Here is where my skills break down:I cannot run the timing loops the other poster requested - they would be dubious until I get some reliability coded in.
    So here's where that problem develops for me, and is beyond my cs skills to find the cannonical construcor:


    remainder of code is same, this is my attempt to fix the core problem.
    //eof
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24168
        
      30

    Your "no wonder there was a slew of NPEs at line 559" comment makes no sense to me -- there's nothing in those few lines of code that demonstrates that there should be an NPE. If, for example, outStream or outString are null, then you'd see NPEs.

    WordCount.java:492: Incompatible type for constructor. Can't convert java.lang.Integer to java.util.SortedMap.
    = new java.util.TreeMap(new java.lang.Integer(super.toString().hashCode()));


    By default a TreeMap constructed with no argument will sort objects using the Comparable interface, which Integer supports; so, if I understand your intent, just use the no-argument constructor here.


    [Jess in Action][AskingGoodQuestions]
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by Ernest Friedman-Hill:
    Your "no wonder there was a slew of NPEs at line 559" comment makes no sense to me -- there's nothing in those few lines of code that demonstrates that there should be an NPE. If, for example, outStream or outString are null, then you'd see NPEs.

    .... just use the no-argument constructor here.


    Most definitely, thank you:

    Let me be somewhat verbose for a moment - for clarity - I'm overloading the machine, by intent. I get exceptions in complied code, so I don't know what happend. This morning I resorted to placing ++lineNumber; all through the code, then doing e.getMessage() + toString(lineNumber)

    No, my statement is non-substantiated by the code sample, it doesn't make any sense to me now either.

    This is why I posted to advanced, just because I (thought) traced the failure down, then saw the null assignment, I got ahead of myself .... BUT this is the core-engine - the TreeMap keys on an int, and how it does that may result in another question - but with this information I can drill it down quite a bit further.....

    Quite a bit. [emphasis]


    Placing boolean debug = true; and doing if(debug){print "lineNumber.toString()"} throughout the code results in the necssity to come back and do search replace to set debug false on about twenty files presents it's own set of challenges. This may be straightforward for masters, and I did some C++ coding using this approach....

    Since nothing in those few lines of code that demonstrates that there should be an NPE, I am back where I started. class WordData contains an int, from which I can construct an Integer object for the key thus:



    Should I examine for the presence of the key first ?

    This portion of the project has to be highly-reliable and fail only in known ways that can be caught by try/catch semanitics - that's why I put it up in advanced......

    I can rebuild on:


    But if there's nothing in those few lines of code that demonstrates that there should be an NPE, what would be your suggestions on how to drill this down ?

    I cannot do prototype testing on the jdk- 5 machine. Maybe Beta, maybe.

    I will most definitely encode your default constructor, and place some more if(debug)'s around and really try to bracket an area of interest, but your comments are telling.

    The code:

    Works - printing out dictionaries.
    pascal betz
    Ranch Hand

    Joined: Jun 19, 2001
    Posts: 547
    I will implement this advice, directly througout my project

    i'm always happy to help....

    check out the compiler arguments. what debug information settings do you have ? (the "-g" switch) see also here:
    http://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/javac.html
    and - for testing purpose - you could also disable the JIT compiler (does this still apply to java 1.5 ?):
    http://www.thunderguy.com/semicolon/2004/02/11/java-stack-trace-with-line-numbers/

    as for putting "if (debug) print something" statements i would avoid them and use a logging framework (either log4j or java.util.Logging or commons logging from apache...). With these frameworks you can print out logging information (to file/console/...) including class/method/line information.


    pascal
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by pascal betz:i'm always happy to help....
    I know the feeling, I especially enjoy helping beginners.
    check out the compiler arguments. what debug information settings do you have ? (the "-g" switch) see also here:
    I haven't used the -g switch because I don't know how to use it.
    You could also disable the JIT compiler
    (my javac is java 1.2 !)
    ... logging information (to file/console/...)

    That is - if I understand what you are saying - something I did myself.
    I decided to send you the sources for WordCount - but there is no email link, so here ya go...
    That'll be there for a few days, let me know by pm when you got it.
    http://docdubya.com/belvedere/pascal_betz/WordCount.java
    Bear in mind that this is version 0.2 - still very much in prototyping ... it is a work in progress: made the change to the constructor as suggested by Ernest Friedman-Hill, Philosophi´┐Ż´┐Ż Doctor, but have not changed the variable calls to OuterClass.this.variableName as suggested to me in beginner.

    Please bear with me on my coding style, I code such that I can read on the basis that the compiler strips most of the unused area in the pre-processor - I do not have to answer to anybody, so I code so that I can read it.

    My goal is to test this to destruction, again and again, therefore any suggestions on how to achieve reliability are welcome, even if we don't agree or they are wrong. I did not know that the jit had anything to do with my "(compiled code)" -- this is an example of something that will really help me.

    I am still of the opinion that too many open file handles is the principal area of failure, I did discover that my throttling harness in main slowed down the number of files opened, therefore can be used during prototyping as a stressor management tool.

    There are reliability & strength issues relating to proprietary windows kernel io routines that make testing for Solaris on a Win box dicey at best. Perhaps some Phd with field experience can happen across this conversation and provide much needed comment on the issue. I would prefer to use Alex Jones style to criticise, but that loses 97%of people.


    [ November 02, 2006: Message edited by: Nicholas Jordan ]
    [ November 02, 2006: Message edited by: Nicholas Jordan ]
    Ernest Friedman-Hill
    author and iconoclast
    Marshal

    Joined: Jul 08, 2003
    Posts: 24168
        
      30

    Originally posted by Nicholas Jordan:
    as suggested by


    Ernest.
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by Ernest Friedman-Hill:
    Ernest.

    Sorry, also I hit the wrong button a moment ago, can someone delete the
    spotted one blunder post or move it here ?
    Jim Yingst
    Wanderer
    Sheriff

    Joined: Jan 30, 2000
    Posts: 18671
    Generally I've found you need to get EFH liquored up a bit before he accepts cute nicknames.

    I wonder if we could get Dr. "Ernie" Friedman-Hill to write a book with a coauthor like... say... Bert Bates? The mind boggles at the possibilities.

    As for your accidental post, looks like someone else has cleaned it up already.


    "I'm not back." - Bill Harding, Twister
    Nicholas Jordan
    Ranch Hand

    Joined: Sep 17, 2006
    Posts: 1282
    Originally posted by Jim Yingst:
    Generally I've found you need to get EFH liquored up a bit before he accepts cute nicknames.

    I wonder if we could get Dr. "Ernie" Friedman-Hill to write a book with a coauthor like... say... Bert Bates? The mind boggles at the possibilities.


    I just love this type of challenge, when I walk into a new shop, the first thing I do is announce:

    "If anybody can say anything good or bad about me to me or behind my back and get me to care, I will buy you a soda-pop at lunchtime."

    It eliminates ALL backstabbing in about thirty or fourty secods....


    Who is Bert Bates ?
    [ November 02, 2006: Message edited by: Nicholas Jordan ]
    pascal betz
    Ranch Hand

    Joined: Jun 19, 2001
    Posts: 547
    Hey Nicholas


    i probably won't find the time to go trough your code....

    pascal
     
    I agree. Here's the link: http://aspose.com/file-tools
     
    subject: Slew of null pointer exceptions.
     
    Similar Threads
    java.util.Collections
    Linkage Error while starting JBoss in eclipse 3.4 [Issue not occuring when I start JBoss from /bin]
    Reader/Writer problems
    For Loop
    Im a bit confused