Removing entries from a collection

[Nicholas Jordan]

Program is starting to work; various collections are used to manage various, often large counts of, instance variables - when they are done I need to let them be reclaimed.

By testing and fiddling, I came up with the following design scenario which seems to be working:

Get key set for collection - such as tree map

Get an iterator for the key set

Walk the collection using the iterator

Reset it an walk it agin if at least one instance was still active on last pass

TYPICAL - loop logic is wrong,

/**
 * The Dictionaries that Grace Hopper needs have been implemented.
 * The name " Grace " is from an early worker
 * in computer science at Navy in D.C.
 * She had a clock that ran backwards, you were supposed to notice.
 */
public class Grace  extends java.lang.Thread implements java.lang.Comparable
{
........[snip]
........[snip]
boolean foundOne = false;
 
ekendrivel: 
do
{
    java.util.Set zarvendugel = this.huntersOfEvil.keySet();
    java.util.Iterator blosphevar = zarvendugel.iterator();
    if(blosphevar.hasNext())
    {
       do
       {
        java.lang.Integer k_9 = (java.lang.Integer) blosphevar.next();
        this.oneSearch_ = (InternationalHarvester)this.huntersOfEvil.get(k_9);
          if(this.oneSearch_.isDone())
          {
this line -> blosphevar.remove();// this is the line I am asking about
this line -> this.oneSearch_ = null;// this is the line I am asking about
          }
          else
          {
               foundOne= true;
          }
      }
      while(blosphevar.hasNext());
      if(zarvendugel.isEmpty())
      {
          this.done = true;
          break ekendrivel;
   }
}
while(foundOne);

I hope the absurd variable names are not overly distracting, I have grown weary and leary of trying to pick variable names that are satisfiying to discriminating coders, but may be non-unique in twenty files of several hundred lines typical.

[Jim Yingst]

So, what is the question?

It looks like you only need the keys in order to get the values. In that case, you might as well iterate through the values directly:

java.util.Iterator iter = this.huntersOfEvil.values().iterator();
if (iter.hasNext())
{
    do
    {
        InternationalHarvester ih = (InternationalHarvester) iter.next();
        if(ih.isDone())
        {
            iter.remove(); // this removes from the underlying map as well
        }
        ...
    }
    while (iter.hasNext();
    if (huntersOfEvil.isEmpty()
    {
        ...
    }
}

As for the this.oneSearch_ stuff, I really can't imagine what you might be trying to do with it in the first place, so I have no further comment; I left it out of my code above.

[NJ]: I hope the absurd variable names are not overly distracting, I have grown weary and leary of trying to pick variable names that are satisfiying to discriminating coders, but may be non-unique in twenty files of several hundred lines typical.

As a matter of fact, yes they're quite distracting. I can't remember what one random string of gibberish means from one line to the next. Are you intentionally trying to make your code difficult to read? If you're going to use long names, you might at least try making them somehow descriptive. In a language which is understood by the people you're soliciting help from. Most of these are local variables anyway, so the names don't need to be unique in the file, they just need to be unique in the block they're declared in. If that's difficult, you really need smaller blocks of code - this can usually be accomplished by refactoring one long method into several smaller methods.
[ November 12, 2006: Message edited by: Jim Yingst ]

[Ådne Brunborg]

Originally posted by Nicholas Jordan:
I hope the absurd variable names are not overly distracting, I have grown weary and leary of trying to pick variable names that are satisfiying to discriminating coders, but may be non-unique in twenty files of several hundred lines typical.

Variable names can, and should, be descriptive enough to make sense - they are in a way part of the documentation for the code. They make it alot easier to understand the code at first glance, and are really not that hard to come up with:

     java.util.Set zarvendugel = this.huntersOfEvil.keySet();
    java.util.Iterator blosphevar = zarvendugel.iterator();   

could just as easy be

    java.util.Iterator huntersOfEvilKeyIterator = this.huntersOfEvil.keySet().iterator();     

and anyone seeing this variable name will automatically think something like "ok, so here we iterate over the huntersOfEvil keys - that means the huntersOfEvil is a Map - and then ...", instead of seeing 'blosphevar', wondering what this is, having to look up the variable declaration, seeing that it is an iterator for the collection 'zarvendugel', which again is the keySet of huntersOfEvil.

Variable names does not need to be unique across classes or even methods, as I'm sure you know.

OK, now to the question:

    java.util.Set zarvendugel = this.huntersOfEvil.keySet();
    java.util.Iterator blosphevar = zarvendugel.iterator();
    if(blosphevar.hasNext())
    {
       do
       {
        java.lang.Integer k_9 = (java.lang.Integer) blosphevar.next();
        this.oneSearch_ = (InternationalHarvester)this.huntersOfEvil.get(k_9);
          if(this.oneSearch_.isDone())
          {
this line -> blosphevar.remove();// this is the line I am asking about
this line -> this.oneSearch_ = null;// this is the line I am asking about
          }
          else
          {
               foundOne= true;
          }
      }
      while(blosphevar.hasNext());

My understanding of what you want to do here: Iterate over the huntersOfEvil, and if an entry in this Map "isDone", remove it from the Map, else set "foundOne=true".

Personally, I always try not to operate on the iterator when I can help it. I'd remove it from the Map instead (I assume that is what you want to do?)

I wouldn't use any reference to "this.oneSearch_" - it seems to me that it is enough to check the entry in the huntersOfEvil Map, and if it is done, remove it from the Map.

Rewritten:

java.util.Iterator huntersOfEvilKeyIterator = this.huntersOfEvil.keySet().iterator();
while (huntersOfEvilKeyIterator.hasNext())
{
    Object key = huntersOfEvilKeyIterator.next();
    if((InternationalHarvester) this.huntersOfEvil.get(key).isDone())
    {
        this.huntersOfEvil.remove(key);
    } else {
        foundOne = true;
    }
}    

Note that I don't cast the key to Integer - the class of the key is irrelevant at this point, and doing it this way decouples it from the Map, and you can change the Map keys later without having to rewrite this code.
[ November 13, 2006: Message edited by: �dne Brunborg ]

[Nicholas Jordan]

Originally posted by Jim Yingst:

So, what is the question ?

iter.remove(); // this removes from the underlying map as well

Question is one of memory management: The program is intended to scale from both single-processor, consumer-grade os to mp and back effectively, fluently and safely. I have no way to manage nor detect the mental landscape of people who will use the program, and by experience, this is a profoundly failure-prone weakness. gc is an issue here, the issue that I am trying to dig into. I understand that gc is in the arena of system/JVM interaction, and may not yield to my efforts.

// this removes from the underlying map as well

Okay, but does this set the originally declared object instance variable to null ? I may accumulate 20,000 - 50,000 instances in an hour of operation ... or may be running on a consumer - grade kernel, with sparse instance counts.

It looks like you only need the keys in order to get the values.

Correct.

In that case, you might as well iterate through the values directly:

java.util.Iterator iter = this.huntersOfEvil.values().iterator();

This syntax was just comprehended by me a few weeks ago:

func().func().func()

With some practice, such as implementing your advice in code, I can probably get it built-in to fingertips over the next few months, but it is the memory-management issue which concerns me, for the reasons clarified above. To iterate through the values directly was one approach I gave careful thought to, it's just that using a map keyed to a random int seems a natural way to keep things managable, no matter how large or small the dataset. I wasted time at one point trying to get over a mental beginner's re-invent the wheel before I could see that for some purpose, an int keyed by String would be the only way to get some jobs done.

As for the this.oneSearch_ stuff, ..... I left it out of my code above.

Understood, expected. I have been warned.

As a matter of fact, yes they're quite distracting.

Noted. I have been warned.

I can't remember what one random string of gibberish means from one line to the next.

I trust the compiler to find execution paths, to do type-checking of some kind on those paths - I fixed 50, count-em, compiler diagnostics this morning cross coding Adaptive Resonance Theory from a C/C++ source file. {continued after next quote}

Are you intentionally trying to make your code difficult to read?

Most definitely the contrary ,....I constrained my earlier remarkst to Dr. Friedman-Hill about Carpal Tunnel . In fact I do bare-knuckle push-ups on hot-asphalt as one of sevral Carpal Tunnel management tools. In fact I long ago resorted to using Hotbits to obtain variable names because of feeling lost in a House of Mirrors. The code execution path was the only thing I focused on; relying on the compiler to track such effectively.

In my mind, it is information warfare. A morass of sluggish thinking, vs getting something done.

I really can't imagine what you might be trying to do with it in the first place.

Exploratory answer: Big Corp, who has recently been on the Business Section of NYT:

to quote my team-lead for the project: .....

I use (Big Corp) for my job, I know plenty about (authority verification).

I know they didn't have good security checks.

Now that they do, dealing with them is a (difficulty).

I also know they were bought out by (commercial co) and that they are even worse.

I also know that (commercial co) was founded by (specially trained ops) ....

I also know that they are one of the biggest influencers of Total Information Awareness,...

I know.

So, work multiple dictionaries against multiple email files, looking for un-authorized use, without triggering Razorbacks , which can grow as long as 8 to 9 feet and weigh in at almost half a ton ?... Working this safely and effectively over operating systems that were never intended to be robust, and given the known tradeoffs between passwords, access, reliability, effectiveness and time-cost,... we have a situation with which I am highly familiar and usually employ Quantum Chaotic thought models to frame the situation in my mind.

You are a degreed engineer, I am not trying to exaggerate , the correct answer to: <tt> So, what is the question ? </tt> . is lengthy,... I have better things to do with my time than engage in wombat with Ph.D's - at least without reason for no determined goal. Social Engineering, if disciplined, requires near absolute isolation from the test bed. To look into a commercial concern's email traffic for purposes of working dictionaries against email feeds requires a disciplined approach, for humanitarian reasons.

If you're going to use long names, you might at least try making them somehow descriptive.

It just leaves me reeling. Code path / execution path is something I guess I will just have to translate into meaningful variable names, out of respect for the skills of many of the people here.

In a language which is understood by the people you're soliciting help from.

I really wish I was up to the skill level of reading the JVM source code and writing a simple compiler. I have a copy of Kenneth C. Louden's Compiler Construction, Principles and Practice, PWS Publishing, 0-534-93972-4, ( I'm hung on 3.3 LEX AND YACC .... ) Along with having attempted to read the sources for the JVM not too long before this post in an attempt to answer some of these questions. Often, an answer may appear to work - but lacks robustness which I could correct with a few hundred lines of code.

Again, recognize I realize I am talking to Ph.D'S......several of my team members are degreed.

Most of these are local variables anyway, so the names don't need to be unique in the file, they just need to be unique in the block they're declared in.

I tried this recently, to thwart having to recode to meaningful variable names just to address gc questions to people I solicit help from. I got compiler diagnostics along the line of variable name already used in class or method - or some diagnostic that sounded like that.

If that's difficult, you really need smaller blocks of code - this can usually be accomplished by refactoring one long method into several smaller methods.

I tried this approach, it just moves the mirrors around for me.

Oh, assuming one or more instance variables managed in the Collection, be it a map or otherwise, were found to be still active, does the iterator have to be reset to rewalk ? I am assuming it does for obvious resons.
[ November 19, 2006: Message edited by: Nicholas Jordan ]

[Nicholas Jordan]

Originally posted by �dne Brunborg:

Variable names can, and should, be descriptive enough to make sense.
they are in a way part of the documentation for the code.

Noted, as corrected. That's what huntersOfEvil was chosen as a variable name for, because that's what that Object does.

It hunts evil, according to the definition of the System Administrator who may be using the program.

However the System Administrator defines bad-words, I cannot and do not want access. Consider the Larry Wall case. I will remain isolated at any cost.

They make it alot easier to understand the code at first glance, and are really not that hard to come up with:

could just as easy becode:

java.util.Iterator huntersOfEvilKeyIterator = this.huntersOfEvil.keySet().iterator();

and anyone seeing this variable name will automatically think something like "ok, so here we iterate over the huntersOfEvil keys - that means the huntersOfEvil is a Map - and then ..."

...(snip)

Variable names does not need to be unique across classes or even methods, as I'm sure you know.

Well, that's what I thought .... something along this line failed recently and I just blazed ahead with my established coding skills, as hammered, to get to this memory management issue.

I [em] do [/em] know the documentation makes clear the JVM does gc when IT thinks you need it,... not when you think the machine needs it. Recent tests I was able to get running demonstrate that things happen in a non-uniform manner - the way I am building the program - and that completions are not uniformly distributed in time. Okay, what this proves for me is that not all of the JVM runs exactly as my preset world-view assumes. Therefore, I must address this question of whether to set the original reference to null. BTW, I have heard this reference terminology many times - I cannot determine from reading code where I am creating references. Generally, my overall program design uses drill-down. Whatever we have in scope when some - any - point is reached, all variables will go out of scope as the call chain is collapsed by returns. The question here, narrowly, is that I fire instance variables in a loop, from a directory listing, and cannot leave the method until all of them have had a chance to complete. Also, I cannot anticipate how many or what size or how heavily loaded the machine will be.

OK, now to the question: code:

java.util.Set sifterKeys = this.huntersOfEvil.keySet();
java.util.Iterator sifterIterator = sifterKeys.iterator();    
if(sifterIterator.hasNext())
{
    // PSUEDO CODE
    {
       BabySit( Collection );//
       while(Collection.oneStillActive());
       

My understanding of what you want to do here: Iterate over the huntersOfEvil ....

Return from method if and when and only if and when *ALL* object instance variables have completed, which is reported by setting a var in the instance to something along the lines of boolean done = true;

Personally, I always try not to operate on the iterator when I can help it.

Explain, be verbose and hammer me like I really need it. The documentation for Collections generally state that for <K,V> V is only safely removed by calling .remove() on the Iterator ???....

This gets into the core of why I posted, please let the hammer down on me here - this intrigues me.

I'd remove it from the Map instead (I assume that is what you want to do?)

I want to babysit the Map/Set/TreeMap or whatever Collection until all instance variables have completed, then return();

I do not want to accumulate massively garbaged Objects laying all over RAM that will never go out of scope and be recognized as such by the JVM.

Some non-deterministic behaviour is expected as a reasonable concession to OS engineers.

I wouldn't use any reference to "this.sifter"

The general design that seems to be effective is to design one class to do the work, then call instantiations of that class from a resource manager class, Touche' for OO !

This design pattern has clearly emerged at my fingertips after only a quarter million keystrokes - the Krell Empire must be close at hand.....

it seems to me that it is enough to check the entry in the huntersOfEvil Map, and if it is done, remove it from the Map.

Explain. Make it work for three or three hundred - that's what the Map is for, flexibility of size.

Rewritten:

/*** psuedocode for linger ***/
void linger() {sleep(0);}// let's not get thrown out to threads, I've already done that research.
 
  java.util.Iterator huntersOfEvilKeyIterator = this.huntersOfEvil.keySet().iterator();// ? explain.
  /** I always do if(Iterator.hasNext()) before entering loop **/
  /** I am left to assume aggressive code path optomizing will deal with either form **/
  while (huntersOfEvilKeyIterator.hasNext())
  {
     Object key = huntersOfEvilKeyIterator.next();
     if((InternationalHarvester) this.huntersOfEvil.get(key).isDone())// syntax makes me shaky, no var - anonymous var ?
     {
        /** Earlier, there will be an instance variable that was put() in the Map **/
        this.huntersOfEvil.remove(key);// great, but where's the call to set it out of scope ? 
     } else { 
        linger();// just the os decide when, let it do it's job 
     }
  }

[QU0TE]Note that I don't cast the key to Integer - the class of the key is irrelevant at this point,
???... I cannot see the Forest because all the Trees are in the way. I am missing something here.

[QU0TE]and doing it this way decouples it from the Map,
I am dealing with 900# wild RazorBack feral Hogs, I will keep everything tightly coupled, if I need another Map, I will write one.

and you can change the Map keys later without having to rewrite this code.

I throw away more code than I keep.

*knock* *knock*- Who's there?- Cthul.- Cthul who?

I dont get it, P.S. I tried to fix quote tags after this last code, seems to be beyond my skills.
[ November 19, 2006: Message edited by: Nicholas Jordan ]

[Stan James]

Short answer: Get as close as you possibly can to Jim's example solution. I believe �dne's suggestion of removing objects from the map will take you right into "undefined" territory. See the doc for Map.keySet() and Map.values().

Okay, but does this set the originally declared object instance variable to null ? I may accumulate 20,000 - 50,000 instances in an hour of operation ... or may be running on a consumer - grade kernel, with sparse instance counts.

What is the "originally declared object instance variable"? (We'd usually just call it a variable or an object reference.) More importantly, where is it declared ... in a class, static or not, or in a method? These are the factors that determine its scope or life span. If the variable has already gone out of scope we have no worries. If it's still in scope you'll need to make sure it no longer references the object you want to remove. It would be fine if it is null or references some other object.

I strongly feel you need to back up and start some of this from a simpler point and build it up slowly. Your problems with unique variable names hint at some larger problems in structure. You really need to be comfortable with the recommendations to use local variables instead of instance variables and to break code into shorter methods. Those will be critical skills.

The good news is we love to help with stuff like that.

[Nicholas Jordan]

Originally posted by Stan James:
Short answer: Get as close as you possibly can to Jim's example solution. I believe �dne's suggestion of removing objects from the map will take you right into "undefined" territory. See the doc for Map.keySet() and Map.values().

I concur.

What is the "originally declared object instance variable"?

/** psuedo-code**/
while(--index > 0)
{
  map.put(new Integer i, new PreviouslyWrittenClass Sifter);
 
// Then we call .start() on Sifter and wait for it do it's work
// That's what the Map is for, to linger() while Sifters to the bidding of System Admin.

(We'd usually just call it a variable or an object reference.)

Explain,....

More importantly, where is it declared ... in a class, static or not, or in a method?

Generally, I am still skill building, but the design pattern of instance of class declared as varaible in a method, then used for what the class was written for.

These are the factors that determine its scope or life span.

That's what my question is, how I word that question may skew our approach to the quesstion.

If the variable has already gone out of scope we have no worries.

IOW - worst case, I can just return to put all out of scope and the JVM is well rigged to recognize that everything I coded is now out of scope and can rely on the o.s. & JVM not to think something I have no references to is useful any longer.

If it's still in scope

I either linger() or leave the method with a return. This design pattern has clearly emerged to a reliable pattern.

It would be fine if it is null or references some other object.

It ?, what it, I gotta lotta its all over the place, as you can tell from my variable naming morass

I strongly feel you need to back up and start some of this from a simpler point and build it up slowly.

I am sure you will be in populace company in this opinion. It is a matter of innate mechanical recognition of how machines work, how minds work, and a brutal attitude consequent to some of the Clowns I have to deal with. I fully recognize I will be corrected and corrected as you do here. I can take it.

Your problems with unique variable names hint at some larger problems in structure.

Correct, it is the structure of the program that I am working out. I will only be able to relax enough to focus on that design necessity when I can quit worrying about the bazillionth Clown phenomena.

You really need to be comfortable with the recommendations to use local variables instead of instance variables and to break code into shorter methods.

I must concede that the src.jar largely supports your contention. I think, from the wording of your advice that there are some non-trivial concepting issues we may explore productively. The class which contains the main() from which the program is run is approaching 1,000 lines: I cannot think of any reason to be embarrased about any of those 1,000 lines .... I absolutely ->** LOVE **<- being torn to shreds by hoards of Ph.D's, because if I can quit re-inventing the wheel long enough to learn something, it's like when the 20,000 # elephant trainer pushed me over with two fingers. Just to watch the mastery of it is an indelible memory to savor.

General program design:

|......
|......
|...... feeds.

|......
|......
|...... Artificial Ants


Work all Ants against all feeds.

First, I have to build dictionaries. I got that part working using the skills you criticise.

Those will be critical skills.

The good news is we love to help with stuff like that.

It's not my first day.

My next question will involve converting [][] in C/C++ files to Vectors in Java.
This challenge comes up cross coding A.I. into JAVA.
I would really rather take some advice there in lieu of my usual approach.
Which you seem to be able to recognize.

I have strong feelings about reliability testing, engineering and design.

[Jim Yingst]

/** psuedo-code**/
while(--index > 0)
{
  map.put(new Integer i, new PreviouslyWrittenClass Sifter);

All right, it looks like the only references to each sifter and Integer are the references held within the Map itself. Good. If you call remove() using the iterator of one of the collection views of the map (i.e. using keySet()iterator() or values().iterator() or entrySet().iterator(), then the remove() will remove all references to that entry (key and value) which were held by that map. Assuming that there aren't any other references which you've kept somewhere outside the map (and based on your code snippet above, there aren't) then both key and value will be eligible for garbage collections after you call remove() on a given element. At least, assuming you're using one of the standard well-tested Map implementations, like HashMap, TreeMap or various others in the JDK.

Succcessfully removing the references from the underlying Map is one of the documented properties that Maps are expected to have - e.g., look at TreeMap, at, for example, the values() method specifically. "The collection supports element removal, which removes the corresponding mapping from the map through the Iterator.remove, Collection.remove, removeAll, retainAll, and clear operations." The API for keySet() and entrySet() has similar statements. It's part of what those methods are designed to do - give you collections and iterators that you can use to (among other things) remove references from the underlying map. It's a pretty safe bet at this point (years after the classes were first introduced) that they will do this successfully.
[ November 19, 2006: Message edited by: Jim Yingst ]

[Jim Yingst]

[NJ]: Oh, assuming one or more instance variables managed in the Collection, be it a map or otherwise, were found to be still active, does the iterator have to be reset to rewalk ? I am assuming it does for obvious resons.

Not necessarily. Assuming you'd like to continue the iteration from where you were, you can - as long as you only modify the map using that same Iterator. That's why I'm advocating using the Iterator's remove() method rather than calling remove() on the Map - because most standard Map and Collection implementations will throw a ConcurrentModificationException if they detect modification of the Map or Collection via some other object while an Iterator is going over it. The idea is that only the code running the iterator itself is qualified to make modifications during iteration, and such code has access to the Iterator, so why not use it? Any code that tries to modify the Map/Collection without using the Iterator is assumed to be doing so from a separate thread, and the fact that the modification is happening before iteration is complete is taken to be a sign that the code hasn't been written in a thread-safe manner, so they try to force you to not do such a thing, by throwing a ConcurrentModificationException. And for better or worse, this can affect you even if you're not using multiple threads. If you're iterating through some Map or Collection, make changes to the Map/Collection only using the Iterator - unless and until you're done using the iterator.

Conversely though, as long as you do use the Iterator to make your changes (e.g. remove()), the classes are designed so that you can do this, and still continue to use the same Iterator to iterate through the other elements in the collection. Again, that's just how it's designed. You can read in the APIs where various classes like TreeMap talk about throwing ConcurrentModificationException to get the exact details.

Hope that helps...

[Stan James]

The class which contains the main() from which the program is run is approaching 1,000 lines: I cannot think of any reason to be embarrased about any of those 1,000 lines

That's fine, if it works don't be embarrased at all.

But a thousand lines of anything would make me stop and think about changing something. Not because it's "wrong" - it may be beautifully done and producing absolutely correct output - but because it will be relatively painful to live with over time. That's a fairly confident prediction because I've paid my dues on COBOL programs that were closer to 30,000 lines and worked my way down in size over a lot of years. Those giant monoliths produced the correct output and paid the bills, but having been there before, I wouldn't go back.

If you'd like to chat about ways to refactor - change the design without changing the inputs and outputs - scroll on down to the OO, UML, etc. forum and ask away.

[Nicholas Jordan]

Originally posted by Jim Yingst:
[JY]: continue to use the same Iterator to iterate through the other elements in the collection.

Reset to rewalk - not finish the iteration.....

All right, it looks like the only references to each sifter and Integer are the references held within the Map itself.

This is the intent I am trying to achieve, here, for the purpose of encapsulation of Thread Management. Locally, I want to do something without affecting World-State beyond getting the worker classes .start()'d and then do heap management on them to cycle through the same loop again and again before going on the the next phase of the program without being forced to call CreateProcess() in the platform underlying the JVM. This is for doing testing on consumer-grade platforms without access to a Solaris test bed.

[list]

keySet().iterator()

values().iterator()

entrySet().iterator()

do
{
    loopFlag = false;
    /**Unlike the new collection implementations, Hashtable is synchronized.*/
    feeds_ = threadJuggler.keySet();
    generalPurposeIterator = feeds_.iterator();
    synchronized(generalPurposeIterator)
    {
        if(generalPurposeIterator.hasNext())
        {
            do
            {
                java.lang.Integer i = 
                     (java.lang.Integer) generalPurposeIterator.next();
                Belvedere.WordCount dictionaryBuilder = 
                     (Belvedere.WordCount) threadJuggler.get(i);
                if(dictionaryBuilder != null)
                {
                    if(dictionaryBuilder.isFininshed())
                    {
                        generalPurposeIterator.remove();
                        ++vocabulariesParsed;
                        dictionaryBuilder = null;
                    }
                    else // found one still working.....
                    {
                        currentlyExecutingThread.sleep(0);//
                        loopFlag = true;// here's the hook.
                    }
                }
                else
                {
                    stdout.println("Null word count found in map");
                    throw new Belvedere.UserException(0x0d);
                }
            }
            while(generalPurposeIterator.hasNext());
        }// end if(generalPurposeIterator.hasNext())
    }// end synchronized(generalPurposeIterator)
}
while(loopFlag && !threadJuggler.isEmpty());

Comments ?

Good.

Alice.ZIP - 118,580 @ 83% compression

If you call remove() using the iterator of one of the collection views of the map (i.e. using keySet().iterator() or values().iterator() or entrySet().iterator(), then the remove() will remove all references to that entry (key and value) which were held by that map. Assuming that there aren't any other references which you've kept somewhere outside the map (and based on your code snippet above, there aren't) then both key and value will be eligible for garbage c re using one of the standard well-tested Map implementations, like HashMap, TreeMap or various others in the JDK.

/**
 * Removes a single instance of the specified element from this
 * collection, if it is present (optional operation).  More formally,
 * removes an element <tt>e</tt> such that <tt>(o==null ?  e==null :
 * o.equals(e))</tt>, if this collection contains one or more such
 * elements.  Returns true if this collection contained the specified
 * element (or equivalently, if this collection changed as a result of the
 * call).
 *
 * @param o element to be removed from this collection, if present.
 * @return <tt>true</tt> if this collection changed as a result of the
 *         call
 * 
 * @throws UnsupportedOperationException remove is not supported by this
 *         collection.
 */
boolean remove(Object o);

That says remove the element, it does not say remove the reference....

It's part of what those methods are designed to do - give an approach that is reasonably reliable after years of doing this operation successfully.

That's the only thing I look to when dealing with eight hundred pound Russian
Feral Hog. My real enemy is much more dangerous, and can only be handled by
Artificial Ants, see docs in cited Alice.zip link above.

[/list]
[ November 23, 2006: Message edited by: Nicholas Jordan ]

[Nicholas Jordan]

Originally posted by Stan James:
.... as given ....

See Alice dot zip - please provide your opinion.

There is more than one way to do it,... COBOL is one of those ways.
When I can express in java WHAT I am thinking, I may be able to identifiy common factors - Please provide the cannonical beginner question for UML which leads directly to the strength of UML....
[ November 23, 2006: Message edited by: Nicholas Jordan ]

[Jim Yingst]

Please use [ quote ] tags for when you're actually quoting something. Don't put your own original responses in [ quote ] tags; it just makes your response harder to follow. Additionally, many parts of your post are just sentence fragments or code fragments without context. I have no idea what you're trying to say. I really think you need to put some more effort into communicating here. This isn't a game to see of much you can obfuscate things. I'm ignoring stuff about Russian Feral Hogs and Artificial Ants as being useless to this conversation.

[NJ]: That says remove the element, it does not say remove the reference....

True. But how could they possibly remove an element without removing the reference? For better or worse, it's very common for people to speak somewhat loosely and interchangeably about objects and references - e.g. we may talk about an "object" when we really mean "reference to an object", because it's shorter - and because around 90% of the time it's either obvious or irrelevant which one is meant. Unfortunately there's the other 10% of the time where the difference is important and we must phrase things more carefully. In this case, I'm quite sure that they remove the object from the collection by removing the reference to the object to the collection.

Assorted comments on the posted code:

loopFlag could have a more informative name - what does the flag mean? Does true indicate keep looping, or stop? How about naming it keepLooping instead?

You've got a number of variables which are used only within this do loop, but for some reason are declared outside the loop: loopFlag, feeds_, generalPurposeIterator. Actually I don't know for sure that they're not used outside, but the way they're used inside the loop strongly suggests they're utterly irrelevant outside the loop. By declaring these variables in a larger scope than necessarily, you're just increasing the chance of bugs. In particular since you're concerned about garbage collection: keeping variables in scope longer than necessary just increases the likelihood that garbage collections will be unnecessarily prevented because you've accidentally left a reference somewhere you forgot about. This is especially true if the variable is declared as a class or instance variable; it is very strongly recommended that you use local variables whenever possible. A point that has been made several times in the past as I recall. If you're worried about garbage collection, please try to follow some of the basic advice people have been giving you in this regard.

Synchronizing on generalPurposeIterator serves no purpose, as no other threads should have a reference to the Iterator which you just created with the call to iterator(). (Well I suppose they might have a reference since generalPurposeIterator is needlessly declared in a wider scope than necessary, but let's assume that you eventually fix that.) However other threads may still have access to the threadJuggler, perhaps? If you need to synchronize at all, make sure the threads are synchronizing on the same shared object - in this case, probably the threadJuggler. Or perhaps some other shared instance. But in this case it's probably the data in threadJuggler that's shared and needs protecting, so that's what I'd probably sync on.

Your inner do-while construct looks unnecessarily convoluted. Let's see:

        if(generalPurposeIterator.hasNext())
        {
            do
            {
                java.lang.Integer i = 
                     (java.lang.Integer) generalPurposeIterator.next();
                ...
            }
            while(generalPurposeIterator.hasNext());
        }// end if(generalPurposeIterator.hasNext())
    }// end synchronized(generalPurposeIterator)
}

The only reason to use do-while rather than while is if you need the loop to execute at least once before checking the condition. The fact that you've explicitly inserted that same condition in an if statement before the loop indicates that you don't need do-while at all - which is usually the case for most programmers, I find. In fact do-while is considered less readable by many, simply because it's so uncommon that anyone ever needs it, people get used to never seeing it. Anyway, your code can be made simpler as a while loop:

        while (generalPurposeIterator.hasNext())
        {
            Integer i = (Integer) generalPurposeIterator.next();
            ...
        }

Actually, I see that you're still not actually needing the key i for anything other than to get the value. If you just need to loop through the values, why not loop through values() as previously suggested? This is just more obfuscation.
[ November 23, 2006: Message edited by: Jim Yingst ]

[Jim Yingst]

I believe this should be equivalent to the code you last posted:

while (!threadJuggler.isEmpty()) {
    for (Iterator it = threadJuggler.values().iterator(); it.hasNext(); ) {
        Belvedere.WordCount wc= (Belvedere.WordCount) it.next();
        if (wc.isFinished()) {
            it.remove();
            ++vocabulariesParsed;
        } else {
            Thread.yield();
        }  
    }
}

I omitted the null check because (a) I would consider it a programmer error rather than a user error, if one of the values is null (why put null in the map at all?), and (b) if a null does occur, the NullPointerException will identify the source of the problem perfectly well. If you feel that's not appropriate for your uses, well that's just a minor change. Otherwise the functionality of this code should be the same as what you showed - but (to my mind) much easier to follow.

Note that the old line "currentlyExecutingThread.sleep(0);" is dangerously misleading. Both sleep() and yield() are static methods which automatically affect the current thread. If you invoke them using an instance, that instance is ignored anyway. In the code you show, the intent is the same anyway: cause the current thread to sleep/yield. But invoking a static method using an instance is generally a bad idea, as it misleads people into thinking the instance is relevant. Tomorrow someone may try to invoke otherThread.sleep(), and be confused about why it doesn't work. I wish Java had made this illegal in the first place, but they didn't. However most good IDEs will flag this with a warning.

[Jim Yingst]

Thinking more about this, you've still got a busy loop here. Consider what happens if there's just one WordCount, but it's not done yet. This loop just keeps looping and checking, looping and checking, looping and checking, until it's done. Well, that works, but it may waste a lot of processor cycles doing nothing. In many cases a wait-notify protocol would be appropriate here. But after checking your documentation, I see that WordCount is in fact a Thread, which makes this even easier. We can use the join() method, which basically means, wait until this thread is finished:

while (!threadJuggler.isEmpty()) {
    for (Iterator it = threadJuggler.values().iterator(); it.hasNext(); ) {
        Belvedere.WordCount wc= (Belvedere.WordCount) it.next();
        wc.join(); 
        it.remove();
        ++vocabulariesParsed;
    }
}

[Nicholas Jordan]

Originally posted by Jim Yingst:
Thinking more about this, you've still got a busy loop here. Consider what happens if there's just one WordCount, but it's not done yet. This loop just keeps looping and checking, looping and checking, looping and checking, until it's done. Well, that works, but it may waste a lot of processor cycles doing nothing. In many cases a wait-notify protocol would be appropriate here. But after checking your documentation, I see that WordCount is in fact a Thread, which makes this even easier. We can use the join() method, which basically means, wait until this thread is finished:

while (!threadJuggler.isEmpty()) {
    for (Iterator it = threadJuggler.values().iterator(); it.hasNext(); ) {
        Belvedere.WordCount wc= (Belvedere.WordCount) it.next();
        wc.join(); 
        it.remove();
        ++vocabulariesParsed;
    }
}

This is totally correct ! I spent several hours constructing a detailed response, using the source returned by ultimatebb.cgi in my text editor, only to have the box hang due to power management issues when I hit the send button for the form. Luckily, I saved the work.

It may be that this should originally have been posted to Advanced, but I had no way of determining which way it would go. The original Alice.java was reposted at noon today, but the while syntax remains to be replaced in sources.

The fouled up quote tags were an attempt at indentation.

As follows, that saved effort:

Originally posted by Jim Yingst:
Thinking more about this, you've still got a busy loop here.

while (!threadJuggler.isEmpty()) {
    for (Iterator it = threadJuggler.values().iterator(); it.hasNext(); ) {
        Belvedere.WordCount wc= (Belvedere.WordCount) it.next();
        wc.join();
        it.remove();
        ++vocabulariesParsed;
    }
}

Thank you for your help thusfar, I can tell that you have given this thought.

My earler efforts to achieve indentation are noted as counter-productive; ditto the stuff about Wild Boar.

I have qute a lot to absorb here, but two thoughts jump out at me:

sleep() and yield() are static methods

which basically means, wait until this thread is finished:

We will address these two, then on to some other things.

sleep() and yield() are static methods

This is critical. Originally, I placed all the Thread management in Alice, which runs the entire application from main(), which is static normally - as it is here.

At line 139: (in main)

final Thread currentlyExecutingThread = Thread.currentThread();

I arrived at this coding because the compiler would complain about re-declaration of variable names in one method. Many times the same pattern would repeat, so I made it contractual that the variable would be moved to an outer scope - and that it would have to be managed correctly because we have to manage things correctly anyway.

This same pattern emerged as response to earlier advice in this thread about iterators, I will respond to the syntax suggested momentarily. (The advice is expected and I am ready.)

At line 317:

if(feedCount > 0)// Yet another check.

And that's when I have done enought checks and so on such that I finally decide to do something. It drops into one of the loops I have used to illustrate my question. By now, it has emerged that if I reuse a varable in a loop,

var = new Object();

, my primary concern about memory management is addressed by your response clarifiying the interchageability of the word refrence with the word variable. On each iterationg of the loop, the call to new places the reference ? iow - return from new - in var on each pass of the loop. If the immediate next line places var in the/a collection, then var is set to someting else, no references exist to the call to new except in the Map/Set/TreeMap/Hashtable/Collection

Correct ?

But how could they possibly remove an element
without removing the reference?

That's what my question was.....really and truly, no obfuscation intended.

That's what I came here to get answered.

So, to shorten things:

variableOne = new Object();
variableOne = new Object();

This is a hightly abbreviated coding of something I saw in the documentation just this morning looking for the answer to this.

IOW - The Object in the first line,.... here we go again. I spent several hours this morning recoding all of the comments in Main Class Alice to put some more effort into communicating here. In so doing, the class did in fact approach the 1,000 line mark just to phrase things in anticipation of the possibility that someone other that myself may have to look at it.

My employment of unusual metaphor is a consequence my regular work. I am wingman for a degreed engineer, we speak in highly abbreviated slang which relies on nuance, tone and inflection for communication of remote scenes. He is my Social Engineer for this project. It is practice to go dead-flat when matters of dire straits need to be commnunicated, as a signal that things are really hot. It runs on a reversal. I was trying to be effective, not cute. (no need to worry, this is expected)

Anyway, Originally, I placed all the Thread management in Alice: Now I use this design pattern all over the place. E.G. in one of the worker classes:

final Thread currentThread = Thread.getCurrentThread(); 

Then do

currentThread.setPriority()

to the end of accomplishing load balancing. Some of the stuff I have been corrected on may be addressed by telling me whether the coder you would normally advise would have to do load balancing on hundreds of threads, with no way of knowing in advance how many threads would be launched or the processor power on which the threading would be done - in the hands of an operator who cannot fail. It may be that I should have posted in Advanced or this should be moved there because it is developing into that.

Rest assured, I live in the other 10% of the time where the
difference is important and we must phrase most statements profoundly carefully.

The loopFlag variable was so named because I was fixing ten or twenty variable names in 900 lines of code in one class so I could at least get one class presentable and the loop logic where loopFlag would be used has not been worked out, so it was named to reflect that.

Which leads into the next question:

"which basically means, wait until this thread is finished:"

Reading my own docs moments ago, I notice that join() is inherited from thread.

I cannot call join() from main because of issues normally discussed in Applet / GUI / Threads.

Join blocks, to use an absurd and self-explanatory term, does it not ?

That was my original design basis for coding the busy loop as a busy loop.

No pun intended. - often, the question is the answer to it's self.

I did some adroit thread management at lines 331-409, along with posting some juicy comments by Greg Comeau at comments in 440 - 466, where

"...can be very slippery do to the complexities of the subsystems and the whole."

says it all.

keeping variables in scope longer than necessary just increases the likelihood that garbage collections will be unnecessarily prevented because you've accidentally left a reference somewhere you forgot about.

Be vewy careful.

Be vewy careful.

Humor here is for clarity, I have been corrected.

At an outer scope:

Integer someInteger;

At an inner scope:

while(condition) {
   someInteger = new Integer(1);
)

On each pass of the loop, assuming I have not left any other references anywhere else nor made in intervening assignment from someInteger to some other variable misplaced in haste to ship code, the return from new is labeled by someInteger and is only valid until the remainder of that pass ?

IOW - the repeated call to new does not leave a bunch of stuff in places created silently by the compiler behind my back that it does not tell me about. Correct ?

I was wrestling with nearly the same problem this morning reading "Efficient C++ - Performance Programming Techniques" by Dov Bulka and David Mayhew - it's not the same problem and it's not the same linguistic but bears remarkable resemblence. In that book, Messers Bulka and Mayhew who are both degreed and both critical of the compiler doing things behind your back, focus at the outset of the book on the whys and wherefores of the problem domain I recognize. This probem domain is difficult to put in words. Your wording is: "Unfortunately there's the other 10% of the time ...."

You've got a number of variables which are used only within this do loop, but for some reason are declared outside the loop: loopFlag, feeds, generalPurposeIterator. Actually I don't know for sure that they're not used outside, but the way they're used inside the loop strongly suggests they're utterly irrelevant outside the loop. By declaring these variables in a larger scope than necessarily, you're just increasing the chance of bugs. In particular since you're concerned about garbage collection: keeping variables in scope longer than necessary just increases the likelihood that garbage collections will be unnecessarily prevented because you've accidentally left a reference somewhere you forgot about. This is especially true if the variable is declared as a class or instance variable; it is very strongly recommended that you use local variables whenever possible.

I figured I would get hit on this (hit is a casual manner of speaking from field ops, the informality of it does note attenuate it's meaning, only it's reception in fora)<cite> it's fora, not forums </cite>

join() lends it's self to the design pattern that has evolved in worker classes - that would simplifiy the entire application code dramatically, and yes, I realize simplicity often facillitates reliability.

Whatever happens, no matter the cost, I cannot have this application hang, no matter what. To serve that design burden, I place what amounted to a busy loop in main() at each completion point - the type of point described by join() - so that I could harness the power of Threading to either join or die, trading brute horsepower of known engineering for a hook where I could later place monitoring.

I do not consider throwing exceptions to interrupt a thread as good design practice. Java Threads by O'Reilly is not in disagreement, be though I have about 400 lines of catches in main() in anticipation of "The other 10% of the time"

.... A point that has been made several times in the past as I recall. If you're worried about garbage collection, please try to follow some of the basic advice people have been giving you in this regard.

There are plenty of people on the open web who will eargerly engage all the drivel I can dream up. (instanceof Sidekick) advises that a 30,000 line project is clumsy and inefficient. I tried using local variables whenever possible. The compiler complained, so I started using random strings for variable names. Then the PhD's complained. (I am not trying to be funny) So my solution was to employ the naming convention advised, move *all* variables to an outer scope, then be very careful about scoping and refrence management, because I have to be careful about variable scoping and reference management anyway.

If I am going to write non-trivial code for programs to address such as described on page one, Section 3 Sunday New York Times / 12 Nov 2006:
<cite>Keeping Your Enemies Close - The Rehabilitation of a Data Company</cite;> We then have a deadly game of Cat & Rat for which only this metaphor is appropriate and in which I must be careful about variable scoping and refrence management.

This is traditionally the bailiwick of compilers: to do the caliber of work my project requires, I will have to write a compiler. For the moment, I just concentrate on what the variable is supposed to do and adopt the style of one named varable per type needed, except for primitives, and re-use it in a loop during each phase of main();

Only in the inner core of the loop do I make assignments. I have made this design decision for reasons you cite. The other stuff, such as synchronizing on something that will only be drilled-down in that call, is useless. It is left over from earlier attempts, and it's removal awaited comment from someone of your skills to the effect here discussed.

Synchronizing on generalPurposeIterator serves no purpose, as no other threads should have a reference to the Iterator which you just created with the call to iterator().

That's what I thought.

(Well I suppose they might have a reference since generalPurposeIterator is needlessly declared in a wider scope than necessary, but let's assume that you eventually fix that.)

Correct.

However other threads may still have access to the threadJuggler, perhaps?

No. - At least it should not, and anticipating your advice that I need to be really careful about this, I have made a design decision that I need to be really careful about this. (no humor intended) It is interesting to note that I could tell you had likely studied Physics, but assumed it would be Quantum and Molecular Physics, not Engineering Physics. I do not really know the difference, it is how this problem domain resembles the "existing in multiple states at once"

A good question is never answered. - (just drilled down ....)

So, to answer the question immediately quoted:

In Thread one:

/* As discussed */

In the thread launched, assuming the loop exits scope and the re-used variable is set to null, nothing in the second thread can access any variable used to launch that thread from thread main() , correct ? If not, the code will be relatively painful to live with over time.

I just had a 4" gaping facial wound cleand with no analgesic, I mean the code will be relatively painful to live with over time if I fail to get this right.

It is the coding intent that the only references to each <K,V> are the references held within the Collection allocated for that purpose in line 182;

/**
 * Hashtables are efficient under K.I.S.S test - also well known in computer science.<p>
 *
 * Generally, the default load factor (.75) offers a good tradeoff between
 * time and space costs.  Higher values decrease the space overhead but
 * increase the time cost to look up an entry (which is reflected in most
 * <tt>Hashtable</tt> operations, including <tt>get</tt> and <tt>put</tt>).
 * <p>
 * The number of entries should remain a prime number - for performance reasons -
 * so a default value of 101 gleaned from the comments in Hashtable.java is used.
 */
java.util.Hashtable threadJuggler = new java.util.Hashtable((int)101,(float) 0.75);//

It is understood by me for obvious design reasons that I have to give remarkably careful attention to management of this hashtable, in fact renaming it to get help from you guys.

The use of this variable resembles the description of hyper-threading.

Join can be used in worker threads, but that brings up the question of where the Thread.Interrupt (of Thread 2) is caught if Thread 2 was launched from Thread 1 , which went on to do other things......

Surely it is not caught in main - what I am using as a Zero-Page thread, be though it may I am not ready for AVL trees. I note glancing through Artificial Intelligence, A Modern Approach, how much of the work involves a star pattern, and can tell this will devolve into a recurrent design pattern where the number and magnitude of the vectors which splay out from a central point will vary in ways the new shop technician will be disconcerted by, but the Old Geezer or shop tech with 5+ years will recognize as a recurrent pattern in machinery.

With the original question likely to be resolved, I will be able to chat about ways to refactor - change the inputs and outputs (which may be taken as files or folders on successive loop iterations) without changing the design of a loop that seems to be following a similar pattern in every class I use it in.

The pattern is a star pattern.

A known problem before I start is trying to reuse this loop for different purpose, without being lost in a house of mirrors.

only reason to use do-while rather than while is if you need the loop to execute at least once before checking the condition. The fact that you've explicitly inserted that same condition in an if statement before the loop indicates that you don't need do-while at all - which is usually the case for most programmers, I find. In fact do-while is considered less readable by many, simply because it's so uncommon that anyone ever needs it, people get used to never seeing it.

The place this came up was trying to implement Regular Expressions as a preferred search engine. I was trying to recode MatchesLooking from the Java html documentation, and did in a few minutes or hours, adopting the logic construct seen here to get one getline() to read the first line into the buffer. Java is strongly typed. I probably did something along the line of

int didItHappen = fileVariable.getLine ();//

then check for didItHappen being geater than zero or something according to what the src.jar documentation said in the comments of something in java.io.* It's not that the initialize the buffer with a getLine() before dropping into the loop was unfixable, I just used it because I could understand it.

This construct quickly propogated throughout my code because of it's effectiveness at flowing the program from my mind onto the code.

I left it there, rather, than trying to fix it, on the basis that as soon as someone could recognize this as a recurrent pattern in greenhorn coders, an effective, but more widely used pattern, would be quickly communicated to me.

I omitted the null check because (a) I would consider it a programmer error rather than a user error, if one of the values is null (why put null in the map at all?), and (b) if a null does occur, the NullPointerException will identify the source of the problem perfectly well.

I recognized this and will be cleaning up code for you guys to look at, I just had too many other details that were worse that needed to be fixed. Carpal-Tunnel is non-trivial.

If you need to synchronize at all, make sure the threads are synchronizing on the same shared object - in this case, probably the threadJuggler. Or perhaps some other shared instance. But in this case it's probably the data in threadJuggler that's shared and needs protecting, so that's what I'd probably sync on.

It doesn't, those were beginner blunders - they will be removed. There are no other shared instances.

I believe this should be equivalent to the code you last posted

I will recode all the loops in main() using the snippet provided, but it seems to be lacking a semicolon in the for() - I got away from for loops because of scoping in C/C++, since reconsidered by the ANSI Commitee

It is the

Iterator it = threadJuggler.values().iterator();

syntax I would not have been able to figure out on my own.

In conclusion:

I most definitely have a busy loop, no question.

I managed efficiency issues by lowering Thread Priority in the monitor loop.

I can use join in the worker threads, because I can interrupt them.

I cannot use join in main, it is my zero-page thread.

I still have fixes to do, many of them, but made copious changes in the comments this morning, re-ftping Alice.ZIP at noon today.

There will be no

Tomorrow someone may try to invoke otherThread.sleep(), and be confused about why it doesn't work.

because of reasons I will P.M. you about.

[Jim Yingst]

[B][NJ]:

By now, it has emerged that if I reuse a varable in a loop,

    var = new Object();

, my primary concern about memory management is addressed by your response clarifiying the interchageability of the word refrence with the word variable. On each iterationg of the loop, the call to new places the reference ? iow - return from new - in var on each pass of the loop. If the immediate next line places var in the/a collection, then var is set to someting else, no references exist to the call to new except in the Map/Set/TreeMap/Hashtable/Collection

Correct ?[/B]

Yes. Assuming I've parsed what you're saying correctly.

[NJ]: I cannot call join() from main because of issues normally discussed in Applet / GUI / Threads.

What issues?

[NJ]: Join blocks, to use an absurd and self-explanatory term, does it not ?

Yes, unless the thread has already completed prior to the invocation of join().

[NJ]: That was my original design basis for coding the busy loop as a busy loop.

Um... what was? The fact that join() blocks? But your busy loop blocks too - it just takes up more processor cycles while doing so. So why not use join? I don't see what your reasoning is here.

[B][NJ]: At an outer scope:

    Integer someInteger;

At an inner scope:

    while(condition) {
       someInteger = new Integer(1);
    )

On each pass of the loop, assuming I have not left any other references anywhere else nor made in intervening assignment from someInteger to some other variable misplaced in haste to ship code, the return from new is labeled by someInteger and is only valid until the remainder of that pass ?[/B]

Ummm... yes. But again, there's no need to declare someInteger way out in that outer scope. It would be much safer to declare it inside the inner loop:

    while(condition) {
       Integer someInteger = new Integer(1);
       // do something with someInteger, I assume...
    )

That way it's even more clear that the value of the variable is only used within the loop.

[NJ]: There are plenty of people on the open web who will eargerly engage all the drivel I can dream up. (instanceof Sidekick) advises that a 30,000 line project is clumsy and inefficient. I tried using local variables whenever possible. The compiler complained, so I started using random strings for variable names. Then the PhD's complained. (I am not trying to be funny)

I didn't think it was supposed to be funny. I would expect that just about anyone would complain about that - assuming they don't simply shoot you first.

It sounds like the real problem here is that your individual methods are sometimes thousands of lines long. This creates many problems, one of which is the compiler complaining about redeclarations because you're re-using a variable name. In general I think declaring the variable only once in an outer scope is not the answer - unless you actually need the methods to share data contained in a variable. Instead, please, please, PLEASE try to break those thousand-line methods into many smaller methods. That way when you need a new local variable name, it will be in a scope of its own, and you won't accidentally mix up your data with some other data you worked with 500 lines in the past. Also you can give each small method a nice descriptive name for what it does - with a little practice, you can use these methods to eliminate or at least greatly reduce the need for other explanatory comments. So all these new method names I'm asking for aren't really any extra work, with a little practice, because you just take the time spent making & typing comments, and redirect it into choosing and typing method names. The resulting smaller methods are much more manageable to understand and deal with one at a time.

[NJ]: In the thread launched, assuming the loop exits scope and the re-used variable is set to null, nothing in the second thread can access any variable used to launch that thread from thread main() , correct ? If not, the code will be relatively painful to live with over time.

Honestly, it looks very painful to live with now. I don't have time to analyze carefully just what the heck you're doing in all that. I would note that one way threads could be sharing data is through shared class variables or instance variables. You've got a number of these in Alice, and they're final, but in several cases they refer to mutable objects (e.g. arrays). I can't tell offhand if the contents of these actually change, ever, but if they did, then your threads would be sharing data. OK, I think probably this is not an issue in your code, but really, I'm getting a headache just looking at that code. 1000-line methods are evil, evil, EVIL.

[NJ]: I cannot use join in main, it is my zero-page thread.

Sorry, I don't know what this means.

[Carol Enderlin]

1000-line methods are evil, evil, EVIL.

Not to mention that they can lead to 1000+ line discussions like this.

[Nicholas Jordan]

Originally posted by Jim Yingst:
[JY]:... as posted ....

Yes. Assuming I've parsed what you're saying correctly.

You did. - smiley needed right about now, not because of my code - just worried about you guy's sanity as a safety measure.

And, anticipating your response, do not worry about mine - read Programming The Universe by Seth Lloyd if you dare comprehend my mind.

[JY]: What issues?

Threading, gui responsiveness to user, interruption of Thread vis a vis if() on a boolean done .... (too many issues to list) - it is a consequence of my regular work that I must notice things only my Social Engineer would notice.

I could reduce this entire episode to shorthand slang, and he would parse all the nuances. Problem is, he is not a computer scientist - if you want, I will relate the episode to him at lunch Monday and see if he has any comment.

I can tell you his response will be a Glass-Eyed stare, like you guys are giving me, and that would be a method of communication. Were there an exception, he would lose all emoticons and speak in a studied normalacy - indicating a crisis condition exists.

[JY]: Yes, unless the thread has already completed prior to the invocation of join().

Then I cannot use it in my zero-page thread.

zero page thread: IBM's definition
thread: The smallest unit or path of execution within a process.
primary thread: The first thread created when a process is initialized

IOW - public static void main(String[] label){...}

[JY]: Um... what was? The fact that join() blocks? But your busy loop blocks too - it just takes up more processor cycles while doing so. So why not use join? I don't see what your reasoning is here.

" ...busy loop blocks too - " No it doesn't.

" ... it just takes up more processor cycles while doing so." There is no other way.

I have you guys several feet past the edge of sanity already - your Iron is clear of leather and ready. If my reasoning is not easily fathomable by now, further discussion will likely be counter-productive.

Why this is so escapes me. Solaris is a powerfully multi-threaded os, the risks involved are discussed in this comment:

/* Finalization.  Some of these primitives are grossly unsafe.          */
/* The idea is to make them both cheap, and sufficient to build         */
/* a safer layer, closer to PCedar finalization.                        */
/* The interface represents my conclusions from a long discussion       */
/* with Alan Demers, Dan Greene, Carl Hauser, Barry Hayes,              */
/* Christian Jacobi, and Russ Atkinson.  It's not perfect, and          */
/* probably nobody else agrees with it.     Hans-J. Boehm  3/13/92      */

[em]Note the words grossly unsafe.[/em] - I can tell you without looking most, if not all of the individuals either hold Doctorates in Computer Science, are remarkably talented thinkers or have extensive experience trying to get something done in the real world. More likely, some combination of these traits.

I respond to the the evil, evil, EVIL. observation with the immediate above code-comment citation.

[JY]: Ummm... yes. But again, there's no need to declare someInteger way out in that outer scope. It would be much safer to declare it inside the inner loop:

    while(condition) {
       Integer someInteger = new Integer(1);
       // do something with someInteger, I assume...
    )

That way it's even more clear that the value of the variable is only used within the loop.

Different loops have different end goals. The method by which those goals are achieved can be classified, if I have to, lending each end goal to OO - I thought I did that: We have to maintain a coherent execution path, such as described by hyper-threading .... let me put it this way: Solaris can handle 60 open file handles. (grant me that ?) This may an unusual thought to a proprietary Windows 2000 user, be though it may that various shells can " Look & Feel " like telco grade equipment, but the completion points are not uniformly distributed in time v count. As matter of fact, they will drive the newbie technician nuts.

Allright, though you guys may not think that of me, how do we manage join()'s without having the manager class block on join() ? The only way I see to do it is " waste " some processor cycles on a monitor loop, all one has to do is

java.lang.String osName = System.getProperty("os.name");
final boolean solarisWorkaround = (osName != null &&
                                   osName.indexOf("Solaris") != -1);
......
currentlyExecutingThread.setPriority(java.lang.Thread.NORM_PRIORITY);
 
(... fire off threads in a loop ...)
 
currentlyExecutingThread.setPriority(betweenThreadLaunches.intValue());
 
(.... check for done in each thread ....)
 

How do you monitor whether the operator wants to break off the session without throwing a .....

Let me guess, we check for Control-Z on System.in - correct ?

Wow, no wonder you guys cannot follow 1,000 lines of code.

Please, don't throw me out to meaninless drivel,...

[JY]: I didn't think it was supposed to be funny. I would expect that just about anyone would complain about that - assuming they don't simply shoot you first.

That's not even remotely funny. Nor is the gummed label sticker from ( censored ) I found on a school yard where I am Frank's wingman on a $30,000,000 dollar project. It has a line-art graphic of a Hollywood version of a .45 auto pointed at your nasal septum. These folks, if computer literate, would know to use Control-Z. It is their reality.

[JY]:
It sounds like the real problem here is that your individual methods are sometimes thousands of lines long. This creates many problems, one of which is the compiler complaining about redeclarations because you're re-using a variable name. In general I think declaring the variable only once in an outer scope is not the answer - unless you actually need the methods to share data contained in a variable. Instead, please, please, PLEASE try to break those thousand-line methods into many smaller methods. That way when you need a new local variable name, it will be in a scope of its own, and you won't accidentally mix up your data with some other data you worked with 500 lines in the past. Also you can give each small method a nice descriptive name for what it does - with a little practice, you can use these methods to eliminate or at least greatly reduce the need for other explanatory comments. So all these new method names I'm asking for aren't really any extra work, with a little practice, because you just take the time spent making & typing comments, and redirect it into choosing and typing method names. The resulting smaller methods are much more manageable to understand and deal with one at a time.

Okay, I will give this SERIOUS effort. Note for the record that it is incumbent on me that at no time, by any concievable failure can any code accidentally expose any data to some other reference worked with 5/50/500 lines previously or subsequently.

[JY]: Honestly, it looks very painful to live with now. I don't have time to analyze carefully just what the heck you're doing in all that. I would note that one way threads could be sharing data is through shared class variables or instance variables. You've got a number of these in Alice, and they're final, but in several cases they refer to mutable objects (e.g. arrays). I can't tell offhand if the contents of these actually change, ever, but if they did, then your threads would be sharing data. OK, I think probably this is not an issue in your code, but really, I'm getting a headache just looking at that code. 1000-line methods are evil, evil, EVIL.

It is remarkably clear to me and not painful to live with, in fact I sleep easier at night.

... don't have time to analyze carefully ... yes, I can see you are adroit enough to recognize this.

... shared class variables or instance variables ... strict compartmentalization.

... they're final, but in several cases they refer .... so final does not mean final. Expected. Guarded against in code.

final String scopingTest = "Dare not, oh newbie !"; 
String localString = new String(scopingTest);

localString now is not a reference to scopingTest, correct ?

... then your threads would be sharing data. ... Never happen, if the above list item is correct.

[JY]:
OK, I think probably this is not an issue in your code, but really, I'm getting a headache just looking at that code. 1000-line methods are evil, evil, EVIL.

So are enemies.

[JY]: Sorry, I don't know what this means.

Of all the people in the world, to have Jimmey Wiley apologize to me is unsettling. Don't worry about it though, I will look after my own sanity.
Done.

[BTW - Please re-enable html in my posts, the caliber of work here was unexpected a user-registration.]

[ This has gone to 10kb - that seems to bother you guys - I just downloaded 4mb of java sources for a project that is 10% complete, does not even turn my head - lets move on, or close the topic ]
[ November 26, 2006: Message edited by: Nicholas Jordan ]

[Henry Wong]

BTW - Please re-enable html in my posts, the caliber of work here was unexpected a user-registration.

I believe whether HTML is enabled or not, is not based on the account, but on the forums. This forum, being one of the technical forums, should have HTML disabled -- so that HTML code examples could be posted correctly.

Henry