Packaging, naming and directory structure.

[Nicholas Jordan]

My project has accumulated 11 folders, many of which are getting to the useable prototyping stage, or are already there. I need to get going on this naming and packaging pattern and will read my study guide on it while awaiting any replies or suggestions.

The ultimate root of everything, as I now understand it, will be belvedere, the one and only project that is beyond demo programs to elucidate a discussion is Alice. My directory structure now looks like:  C:\belvedere\Alice.java   in which file I have package belvedere; as the first line. I am sure this is not correct and need to be able to reference classes in other packages fluently and effectively such as: ( try not to laugh is serious  ;) package crypto; class Alligator extends SecureRandom but the compiler gives me errors on trying to reference anything that is not src.jar

Websites, links, business scenara ~ any well intentioned comment is wanted.
[ October 07, 2007: Message edited by: Nicholas Jordan ]

[Yamida Frank]

Originally posted by Nicholas Jordan:
My project has accumulated 11 folders, many of which are getting to the useable prototyping stage, or are already there. I need to get going on this naming and packaging pattern and will read my study guide on it while awaiting any replies or suggestions.

The ultimate root of everything, as I now understand it, will be belvedere, the one and only project that is beyond demo programs to elucidate a discussion is Alice. My directory structure now looks like:  C:\belvedere\Alice.java   in which file I have package belvedere; as the first line. I am sure this is not correct and need to be able to reference classes in other packages fluently and effectively such as: ( try not to laugh is serious  package crypto; class Alligator extends SecureRandom but the compiler gives me errors on trying to reference anything that is not src.jar

Websites, links, business scenara ~ any well intentioned comment is wanted.

[ October 07, 2007: Message edited by: Nicholas Jordan ]

I have tried to reply to this topic, and found no button to reply. After trying for several times, i saw this problem is moved to the beginers', i really don't agree with such desition. Since the directory hierarchy means a lot to the design, if you have a clear hierarchy, and package names easy to understand, the program will be much more robust and maintainable.
In my opinion, the first you should do is to divide your procedure into several modules, according to the roles or functions each module play, and a big module can contains a few submodules, you can design your package structure according to this analysis. also you can extract some common functionality codes to make a package, in order to make future reuse more convenient.
I plan to do a research about the jave collection framework in the following several weeks, i think this have much to do with package design and a lot of design patterns, and the most important is that Sun has made its jdk opensource, we can study the source that produced by the experts in the Sun Corpornation.
I regard this quite a good way to learn java and some core design conceptions in the designing, maybe we can have a futhure communication,haha, and my hotmail is [e-mail]

[Stan James]

Hi, this is a frustrating area the first few times through, but we can work through it pretty quickly. I guess you are compiling and running from the command line? That's a good practice at least until you figure all this stuff out.

You need a place for source code to live. Most everybody has a folder called "src". When you work with the command line Java and JavaC commands, you'll navigate here as working directory.

Under the src folder, you have a set of folders that match your package names. So in the folder src\belvedere the files will say package belvedere. In the folder src\crypto the files will say "package crypto".

To compile, you give the path to the source files. So change directory to src and enter "javac belvedere\Alice.java"

To run, you give the full package name of the class files. From src again, enter "java belvedere.Alice".

There may be more to both of those commands to deal with classpath and where to put the class files and such, but that's close. Does that answer the right question?

[Nicholas Jordan]

[Stan James:]... this is a frustrating area the first few times through, but we can work through it pretty quickly.
Yes, and I realized my study guide did not give effective treatment to the issue as it should have, thus explaining my difficulties. It is effective in many other issues so I will rely on the work here to avoid another $200 weekly book budget. I have become effective on several other basics of Java by just working through it several times until I can use the area effectively. Evenutally it becomes like a mouse, pecking at the keyboard. ( ... yuk - yuk )

[Stan James:]... In the folder src\crypto the files will say "package crypto".
My user-home has some 50+ folders, many of which splay out quickly into thousands of files and just about all of it is text. /belvedere is the root of all that I am asking about today. It is an attachment point for all code, compilations and testing that I plan to hold on to or in some way preserving the code periodically, until the project is ready for alpha. I have two machines, not connected by LAN, on which I work independently ... assuming I shold be able to do it at least twice effectively before showing anyone. Many times, people get up from the machine, thinking they have a successful build.

It ain't so.

I assume each project will live in it's own folder, from your reply, thus all files packaged as Alice will live in a directory named Alice. Because:

ai

crypto

gui

midi

randomX

server

tools

wordlists

are all currently parts of Alice ( that may or may not be re-copied into other projects ) - ( then diverging, with all the CVS issues that I may have to face at that time - not now ), they are addressed as belvedere.crypto / belvedere.randomX and so on ~ for example: randomX is a compliant class under "Minimal Standard" multiplicative congruential generator of Park and Miller .... so I will use it because it is the best I can find. ( except for BigInteger in conjunction with SecureRandom )

It occured to me this week in contemplation of this question that there are 2-3 other projects which deserve their own root in belvedere, and that is what prompted me to post the question. Let us take for a tangible, that the 50,000 + keystrokes I generated this last month could be utilized in it's own right be some team members as a Bean. ( In fact they have already said they want it, replete with Bean(boolean tooShort) concepts and full house of tools available with Beans ) I re-wrote it as a stand-alone and a Runnable with a default constructor this morning while drinking coffee.

These things pile up in a hurry at the pace I am now writing code, and the compiler sometimes leaves me thinking maybe Moosey does know best.

[Stan James: ] I guess you are compiling and running from the command line?
In general, I use TextPad - because it has most of this figured out ~ but that still means entering some paramaters in a menu-driven IDE that still functions as a CLI. Sometimes I make a batch file. Sometimes I open a command prompt. Sometimes I set classpath. Sometimes I make a softlink to java/home. I have thought about, and in small measure have tried to, make my own development environment that would not only pre-compile ( by stripping out un-needed newlines and leading eyespace in the source file ) but would as well do keyword highlighting and rapid function body / class body placement in the source file. We are not talking about downloading some pre-built such as is commonly mentioned in response to such a statement. I have had too much of that.

Winnie the Pooh does not have EMACS, ELVIS nor any of the tradionals one usually sees in undergraduate cs.

[Stan James:]... There may be more to both of those commands to deal with classpath and where to put the class files and such, but that's close. Does that answer the right question?

It is close enough to get us to phrasing the question correctly.

If compiling from cli, what should be -cp for the above described Alice if one wants to collect ALL of the class files from partial builds and master builds from a full house clean ? I want to be able to package everything in one dot jar file for later jar signing and deployment of some individual parts as Beans. Currently, I just copy compiles class files over to some arbitray location and try to do whatever from a 2-nd generation copy of whatever worked in prototyping. I released my first Alpha today to team members. It stores some 823 commonly used passwords that are too short as a stored int[] of the hashcodes of those words from the String class. My todo list today includes looking for a fast-lookup collection that can utilize the speed available from a SortedSet, which is probably the one to use because that is what I ended up with after several evolutions. Right now I am using TreeMap, but I do not think I need the Key/Value mapping - only the collection keys.

Since I am doing Extreme Programming here, I bought a copy of "Java Extreme Programming Cookbook" in anticipation of having to explain why I do not plan to use Ant. Most devleopment I am successful at starts off by rejecting some well known tool.

It's kinda like the beginners first attempt at combinatorial algorithms: Painful to watch, funny as hell.

//eof

[Stan James]

I work a couple ways ... One is to put everything in one project

project
   src
      every
      darned
      package
   bin
      every
      darned
      package

Change dir to src to compile, add "-d ..\bin" to the compile command. I'd have to try but I think you get by with no special classpath here because the compiler can find everything it needs relative to the current directory src.

The other way is to make multiple projects.

project1
   src
      com.project1.etc packages
   bin
      com.project1.etc packages
project
   src
      com.project2.etc packages
   bin
      com.project2.etc packages

This has some nice advantages but takes more effort to compile and run. A batch file that compiles everything is not too evil but ANT starts to pay off after you get over the initial learning hurdle.

Then you might want to get into making a jar for each project and putting them in a lib folder somewhere. Your classpath lists all the jars. You may already be past this, but here is something I put together to help some folks get it all straight.

Don't spend too long in build hell. Get comfortable with the mechanics of things and the options you have, and then move to an IDE like Eclipse or NetBeans. I find I'm able to compile and test day in and day out in Eclipse just fine, and use ANT only to build deployments with associated JavaDoc and other outputs.

[Nicholas Jordan]

[Stan James: ]   Don't spend too long in build hell. Get comfortable with the mechanics of things and the options you have, and then move to an IDE like Eclipse or NetBeans. I find I'm able to compile and test day in and day out in Eclipse just fine, and use ANT only to build deployments with associated JavaDoc and other outputs

I'm all too well aware of what you are telling me here, but my project demands are not consistent with the experiences I have had with IDE's ( pretty much the usual issues ) and I am insistent on seeing every line of code, every semi-colon, everything but the comments. I do use comments, and make them whenever it makes sense. I found your workup helpful, and will absorb it. I appears to me right at the moment what I need to do is decide on a design approach, then code a 'recursive descent compiler shell' that walks some directory structure I decide on and issues invocations of javac.

Based on your help, I am right now - right here deciding on design one. I can run my intended shell from .... there's a folder above what you are calling project, from which I can run the compiler. We can now issue commands with a StringBuffer.append(boolean "src"|"class" ) [ if nothing else, I can write an automated batch file writer ] and occasionally do folder.rename("belvedere" + "[" + counter.toString() +"]"), after which operation a simple "copy everything" is done from the renamed folder back into a newly created basename folder in which all work that is worth being awake when you work on it is copied, in toto, back into the build directory. ( the belvedere directory ) ~ I have used this approach for ten years and it is dramatically more effective than backup tools.

Source code and compiled class files takes up an unexpectedly small footprint on todays drives. Massive archiving can be done if you do not use archiving tools. I have tried the 'looks in subfolders' aspect of the compiler and I believe at this point is works, and works effectively. I would, to make maximum possible use of your help, have only one library directory, it would exist at the same level as the individual project folders. This is very clear to me, and is one of the goals of my post.

Thus:

DRIVE_LETTER:\belvedere
                       \Alice
                             \src
                             \class
                             \misc
                       \MailCleaner
                                   \src
                                   \class
                                   \misc
                       \Editor
                              \src
                              \class
                              \misc
                       \SearchTool
                                  \src
                                  \class
                                  \misc

The problem with this Tree Design is that now each proj source file is suddenly in the src package, which I doubt is what we started towards. What I propose is to eliminate the /src folders and keep the compiled class files in each separate class folder in each package. I think the compiler balked. (think the compiler balked) Thus, everyting worth saving for SearchTool is in SearchTool, which seems to me to be the intent and design of the whole packaging concept. I can then make SearchTool be in the belvedere.SearchTool package if I have to to keep the compiler and JVM happy. The above tree also has the limitation that within each project, there are packages which should remain under that project's domain, being a shippble jar file if some other project wants to use what has been accomplished so far.

Thus, it becomes:

DRIVE_LETTER:\belvedere
                       \Alice
                             \crypto
                             \ai
                             \gui
                             \random
                             \wordlists
                             \class
                             \misc
                       \MailCleaner
                                   \class
                                   \misc
                       \Editor
                              \class
                              \misc
                       \SearchTool
                                  \class
                                  \misc

With this design, I could move three projects out of Alice tonight that really need to get out of the Alice project, tonight. I can do all the stuff your page will open my mind to by writing custom batch file writers, then opening them from the shell. Slightly inefficient, but a drastic improvement over where it will all go if I do not do this, plus batch files help in rembering compiler options. I lost the one about how to invoke the JVM with no JIT because the 2nd development platform had to be rebuilt with low-level disk wipes to get out all the shell hooks not related to my work.

The best approach for my ide needs would be to go find the shell exec command again, and custom write the thing becuase TextPad will capture System.out if I am careful not to allow any hangs............

Hangs drive me nuts ->  

[Nicholas Jordan]

Okay, I worked on this for a few hours this afternoon and go an error message that does not really tell me what the error is:

// first line of code in CompilerShell.java is:
package jdev;
 
/*
 * Session that follows from batch file is:
 */
 
// System.out in the command line interface:
C:\jdev\class\jdev>Set path="C:\jdk1.2.1\bin"
 
// long line has a carrige return inserted in search path for posting:
C:\jdev\class\jdev>java.exe  -cp C:\jdev\class\jdev;
C:\jdev\class;C:\jdev;.\class\jdev\class;.\class\jdev;.\class CompilerShell
 
// then, showing up in the command shell is:
Exception in thread "main" java.lang.NoClassDefFoundError: CompilerShell 
(wrong name: jdev/CompilerShell)
        at java.lang.ClassLoader.defineClass0(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:403)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:101)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:248)
        at java.net.URLClassLoader.access$1(URLClassLoader.java:216)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:191)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:280)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:275)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:237)
 
C:\jdev\class\jdev>

This "Compiler Shell" resides in c:\jdev which is the mirror on my remote. The java program, Compiler Shell is supposed to handle the pacakging, naminig and directory structure issue along with recursively presenting individual directories to the java compilier, excluding directories that according to a protocol I have not yet determined.

I ran the java machine from where the compiler placed the class files. Maybe I should run it in the root of my build directory. What this thing is supposed to do is deal with these issues recursively and effectively. The platform this occured on is 1.2 - I will also want to get it running on 1.5

[Jim Yingst]

[Nick]: an error message that does not really tell me what the error is:

Exception in thread "main" java.lang.NoClassDefFoundError: CompilerShell

It can't find the class CompilerShell. If CompilerShell is actually in package jdev, then you should run it with the command

java -cp someunbearablylongandpointlesslyconvoletedclasspath jdev.CompilerShell

[Nick]: I ran the java machine from where the compiler placed the class files. Maybe I should run it in the root of my build directory.

Yes. I recommend compiling with the -d option so that the compiler puts each .class file in a directory appropriate to its package. So if ComilerShell is in package jdev, CompilerShell.class will be put in C:\foo\bar\basedir\jdev - where basedir is the directory from which you executed javac. Then run it with

C:\foo\bar\basedir>java -cp . jdev.CompilerShell

[Nicholas Jordan]

[JY: ]   java -cp someunbearablylongandpointlesslyconvoletedclasspath jdev.CompilerShell
I got this to work from the command line during this mornings builds, I had taken a copy of Java Notes - Copyleft and saw the part about the dotted filename - got it to run on the command line and tried it in the CompilerShell where I am plan to eliminate the ...longandpointless.... CompilerShell extends File, so once I figure the semantics and conventions, I can automate the recursion with: StringBuffer.setLenght(0);StringBuffer.insert(0,this.toString());, but I have to understand the base immplementation. [[too eliminate the long and pointless which was a direct consequence of trying to guess what the compiler wanted as a place to look]]

[JY: ]   Yes. I recommend compiling with the -d option so that the compiler puts each .class file in a directory appropriate to its package. So if ComilerShell is in package jdev, CompilerShell.class will be put in C:\foo\bar\basedir\jdev - where basedir is the directory from which you executed javac

What I hope to do is keep each package in a separate place (I assume this is consistent with the reason for having packages in the first place) and the directory structure is evolving rapidly (really rapidly right now) to resemble the DRIVE_LETTER:\basedir pattern shown above. What I am calling DRIVE_LETTER in fact would be a pluggable attachment point so that I can test on various jvm and hardware combinations and have an isolation level which could be a drive letter or protocol://machine\foo\bar\ or anything else that the end user of the packge may determine to be the attachment point. Similar (very similar) to user_home on multiuser systems.

To do the pace of work I am most comfortable at, I need to rapid prototype and do numerous builds on what is currently five or six development areas that need to be kept dis-entangled. EG: this morning I wrote a 100 lines of GUI code, yesterday it was 100 lines of this stuff so that I could keep everything sorted out, the day before it was a hundred lines of data model code. I know this sounds unlikely, but I have code from (quite a ways back) that is still useful as a template for java code because the c is easy to translate. I am doing what is some really forefront work because of the abilty of the JVM to capture runaway conditions such as array bounds checking and exception reporting allowing me the think in code directly in the editor.

I have found that once I can express the idea in code, I can then convey the concept to others using the OO, Patterns, UML and Refactoring ides,as well as examine the robustness of myself by applying the tests therein to what I have written.

Once the packages are largely useable individually, I would like to package everything in a jar file that is underneath the build directoy, but is excluded from the build process.

Thus, assuming jdev is the basedir:

F:\test\jdev>java -cp jdev.Alice Alice
F:\test\jdev>java -cp jdev.Alice Alice
F:\test\jdev>java -cp jdev.MailCleaner MailCleaner
F:\test\jdev>java -cp jdev.Editor Editor
F:\test\jdev>java -cp jdev.SearchTool SearchTool

could be run from a jar file at an arbitrary 'attachment point' ( as I am calling it ) to be determined by the customer's sys admin ?

To build the individual pacages I could do:

G:\develop\jdev>javac -d Alice\class *.java
G:\develop\jdev>javac -d MailCleaner\class -sourcepath MailCleaner *.java
G:\develop\jdev>javac -d Editor\class -sourcepath Editor *.java
G:\develop\jdev>javac -d SearchTool\class -sourcepath SearchTool *.java
// an such as the above pattern for five or ten projects .....

It is because of the obvious complexity of the design paradigm that I need to have fine-scale control where everything is so that I can split boderline conditions in released code by test case on individual modules. This way I can predict how the overall project might fail and construct tests such as those described in Java Extreme Programming Cookbook

I launched Compiler Shell this morning, I think I have the following code wrong, I belive I am not reading System.out from the call to: Process exec(String command)

[Jim Yingst]

[Nick]: What I hope to do is keep each package in a separate place (I assume this is consistent with the reason for having packages in the first place)

Yes, although you're splitting out different directories at a different level than the javac and java executables are inclined to (based on package structure). Combinine your own split-out directories with java's packages is going to create a somewhat more baroque system than it needs to.

[Nick]: F:\test\jdev>java -cp jdev.Alice Alice

The argument after -cp should be a list of paths to folders or jar files, separated by ; on Windows if there's more than one. There's no reason to have a . in jdev.Alice - based later lines in this post, it sounds like the class files for Alice are actually in jdev/Alice/class/jdev. The final jdev is part of the package structure, since you've said these are all in package jdev. And it's a bit confusing to have jdev be the name of multiple levels here, but so be it The classpath should include the path to the base directory that holds that package directory, which is

-cp jdev/Alice/class

(omitting the final jdev)

Meanwhile the argument after the class path should be the fully qualified name of a class. If Alice is (among other things) the name of a class, and it's in package jdev, then the fully qulified name of ht at class is jdev.Alice. So your command line would be

F:\test\jdev>java -cp jdev/Alice/class jdev.Alice

[Nicholas Jordan]

[Jim:]   Yes, although you're splitting out different directories at a different level than the javac and java executables are inclined to (based on package structure). Combinine your own split-out directories with java's packages is going to create a somewhat more baroque system than it needs to.

Wikipedia defines baroque as: The appeal of Baroque style turned consciously from the witty, intellectual qualities of 16th century Mannerist art to a visceral appeal aimed at the senses. It employed an iconography that was direct, simple, obvious, and dramatic. Subjectively, "baroque" - as a terminolory - instills in me reactionary impulses { deep impulses } which are likely not releveant: To reslove this we would have to examine somewhat more baroque system than it needs to (be). I offer today's work for your observations. Having prototyped almost 500 lines of code a day for the past 10 days (yes, effective production was lower than that) I found today that the utility package that I had written for the Inventory Tracking and Control module (independently shippable) was almost pluggable into the Alice (data-mining) module (independently shippable), and my best guess is that there will be 5 or 6 useable concepts which are expressable as independtently shippable modules that have little, but some, interdependence. These depends will be managed and edited manually, according to thoughts as each shippable evolves.

There will be small, but real differences in the editor from one program to the next. Today I managed to rapidly incorporate work from the past month into today's last-chance to get up to Alpha 0.8 on the GUI for one ( one ) module by copy-pasting JLists that I got into a BorderLayout in a JFrame with a pluggable TextArea/HTMLEditor kit in the center area.

When I go to work on say the mail filter or some as yet un-anticipated project, many of the modules that are effective for Alice and for Inventory Tracking And Control will be applicable there, but with strict 'think about it first' boundary that has to be passed through. Sometimes these boundary conditions are futile, sometimes they are productive of distortions that have to be hand preened out. The jdev in at the root above is a pluggable-name that varies from machine to machine, from mirror to main site, from Team Lead's Office to my Office, and can be changed willy-nilly up until final decisions are made as to how and where the build - deploy process is rooted.

It in fact resembles the employment of "root" in the sample server I found in JDK 1.5 .... unless I hear some really good reason to do otherwise - I will rename the dir structure to:

/root
    /Alice
         /crypto
         /ai
         /util
         /...and so on
    /itc
        /gui
        /inventory
        /util
        /... and so on 
    /mail
        /reports
        /server
        /..... 

What I am calliing root here has little, if any impact on HOW each of the projects is ( ducks-in-a-row ) to and after deployment, except for the fact that the Sun JavaDoc on packaging and naming specifies a nomeclature that leads to long package names. That, in combination with my developer guide having a big hole on the subject may be part of why am having to straighten this out while I am straigtening it out in my mind. ( suits my style better anyway )

[Jim:]   Meanwhile the argument after the class path should be the fully qualified name of a class. If Alice is (among other things) the name of a class, and it's in package jdev, then the fully qulified name of ht at class is jdev.Alice. So your command line would be

In the above dir code snippet Alice embodies a shippable, which uses Alice.crypto; Alice.ai; Alice.util; ( and others ) These are actual package names that are being built as we speak. I happened to work on the gui in Alice today, because that is where I could get the most mileage for the entire team with the resources I had today. I had largely stalled on Alice.crpyto; because of getting into the long, flat run-up that may go forever when one sees what is involved in that, for the first time.

When I get some code points - sorry, that's just the word - in Alice and itc, I will probably try to get server up in the mail project. Each of these projects has code that can be shared by slapping up a template from completed work in some other project, then doing some fast & heavy editing so that it is not mistaken for the same code in a different project with diffrerent constraints.

Consider the following possible directory / command constructions for Sample Company:

//to compile the Alice main() class for Alice
LOCAL_MACHINE \\ C:\root\sample>java   -cp sample/Alice/class sample.Alice       
LOCAL_MACHINE \\ P:\test\sample>java   -cp sample/Alice/class sample.Alice.ai    <- ai
// crypto specifically suited to Alice's ...
LOCAL_MACHINE \\ L:\temp\sample>java   -cp sample/Alice/class sample.Alice.crypto  
LOCAL_MACHINE \\ M:\first\sample>java  -cp sample/Alice/class sample.Alice.gui   <- gui for alice
// crypto specifically suited to Alice's ...
LOCAL_MACHINE \\ N:\second\sample>java -cp sample/Alice/class sample.itc        
LOCAL_MACHINE \\ O:\third\sample>java  -cp sample/Alice/class sample.itc.gui   <- gui for itc
//utilities, tailored to itc paticular needs
REMOTE_MIRROR \\ C:\fourth\sample>java -cp sample/Alice/class sample.itc.util  
REMOTE_MIRROR \\ D:\fifth\sample>java  -cp sample/Alice/class sample.Mail      <- main class for mail server
//isolates server issues from gui/userland/ai
REMOTE_MIRROR \\ E:\sixth\sample>java  -cp sample/Alice/class sample.Mail.server  
REMOTE_MIRROR \\ F:\seventh\sample>java -cp sample/Alice/class sample.Mail.filter <- site specific router 
REMOTE_MIRROR \\ G:\eighth\sample>java -cp sample/Alice/class sample.Mail.transport <- ????
//implements my experience with userland
REMOTE_MIRROR \\ H:\ninth\sample>java  -cp sample/Alice/class sample.LogonShell  
LOCAL_MACHINE \\ I:\tenth\sample>java  -cp sample/Alice/class sample.Mail.LogonShell <- sync()ing
LOCAL_MACHINE \\ J:\eleventh\sample>java -cp sample/Alice/class sample.Mail.LogonShell <- sync()ing

Try not to disbelieve me, I will be getting the planset monday for a building that houses rats valued at $22,000 apiece and we cannot make any noise while building them a new house, as well as being not replacable.

I know all about keeping everyting sync'd, that can be done once I figure out an effective schema for the packaging and directory structure as well as how it is implemeted in the package statements and names passed to the javac.exe ... Looking back at the above simplified project structure, I cannot even put the CompilerShell.java in that listing for fear of appearing facetious. It probably goes in what I am calling \sample and once it is written, can eliminate tedious manual entry of what we are discussing for hundreds of trial builds.
[ October 26, 2007: Message edited by: Nicholas Jordan ]

[Phil McCann]

Sorry for coming in at this late stage, but I'm a Java newbie with what I think is a similar problem. In other languages, e.g. Visual Basic, I have one set of directories for source and another for executables - I want to back up all the source in a lump (e.g. zip a whole chunk of directory tree), and there's no need to back up executables as I can recompile in seconds. With VB I just request compilation and get a dialog asking me where to save the executable. How can I achieve that for Java using Netbeans? If Netbeans can't do it, should I use another IDE?
[ December 16, 2007: Message edited by: Phil McCann ]

[Bert Bates]

Why do you guys torment me by having such discussions in the beginner's forum?

This is a real slippery slope kind of thread for me. The basic topic of directory structures is a really painful part of beginning programming, so from that perspective, it kinds of deserves to be here. But then you go off talking about multiple projects and i start thinking maybe we ought to be having this discussion over in the intermediate forum.

I guess that's a long way of saying that I think we've crossed the line here a bit and i'm sliding this baby over to the intermediate forum.

Bert

[Nicholas Jordan]

Thank you Bert.

If it is a few files in one directory - then it is a beginner issue, be though it may one the beginner is often not ready to accept as needful of work.

When we get to a directory tree such as the one I posted that needs cvs and versioning and cross-package interoperability, the issue becomes an intermediate. I am sitting here with five or so regex's written sucessfully and a better way of using java.util under OO design constriants ( written by a college professor ) and do not delve into the recode because of the recursive lock this puts one's thoughts in.

Suffice it to say for the moment that when I looked at ANT -> they just said flat out not to even use classpath because of parsing ambiguities. I am ready to recode several hundred lines if I can get this working at an acceptable level.
[ December 16, 2007: Message edited by: Nicholas Jordan ]

[Nicholas Jordan]

Originally posted by Phil McCann:
Sorry for coming in at this late stage, but I'm a Java newbie with what I think is a similar problem.

I do not mind at all, this is my thread and I welcome the work by you here and do not notice the time stamp on the work.

In other languages, an ASP is considered to be a European reptile. With ide's generally one can just request compilation and get on with it without asking where to save the executable.

How can I achieve that for Java using Netbeans? If Netbeans can't do it, should I use another IDE?

I am looking into that right now. I am reading Object-Oriented Design by Nicholas J. De Lillo and it gives me all the java OO I need except for this issue ( as titled in the thread )
[ December 16, 2007: Message edited by: Nicholas Jordan ]

[Stan James]

Nicholas, I'm a little lost on just where you are these days. Your post suggesting

\belvedere
    \Alice
       \src
       \class
       \misc
    \MailCleaner
       \src
       \class
       \misc
    \Editor
       \src
       \class
       \misc
    \SearchTool
       \src
       \class
       \misc

looked right on the money. But the line after it each proj source file is suddenly in the src package should not be true. In a command window change directory to any src folder and the packages start with the folder under that one for compiling or running.

So Alice/src would have a folder belvedere. Sitting in Alice/src you'd compile and run main() like ...

x:/Belvedere/Alice/src> javac -cp etc belvedere/Alligator.java
x:/Belvedere/Alice/src> java  -cp etc belvedere.Alligator         

(I left out the -d directive to put the classes in the class folder.)

You have to change directory and compile in each src directory. You can't do them all at once. You can think of packages as a way to organize files, but the higher level folders - Alice, Editor, etc - are something else, maybe build or deploy or reuse units. That is, you can build one without rebuilding the others, you can deploy one without redeploying the others, you can give one to somebody else to reuse without the others. If that's not required, maybe don't bother making separate trees.

When you compile Alice, how do you take in the Editor classes? One choice is to add Editor/class to your classpath. That's very dynamic, picks up every compiled change in Editor, but maybe a little sloppy. Another choice is to build an Editor.jar and add Editor.jar to the classpath when you compile and run Alice. It's an extra step, but helps make sure you're using the real jar that will be deployed, not some change that you might forget to build into a jar.

Note that the classpath may be different in every src folder you visit for compilation. That's part of the complexity of making separate build and deploy units, but generally well worth while. Ant and similar tools exist to document and automate this kind of thing.

One danger on this road is circular dependencies. If Alice uses classes from Editor and Editor uses classes from Alice, it's hard to build both jars from scratch. Robert Martin is the guru on the dependency management topic; his Agile Software Development book should be required reading to build anything with more than one jar. Or maybe more than one class.

Phil, are you hangin' in there? The examples with separate src and bin folders (or src and class) show how to separate source from compiled code. You can easily zip a src folder or link it to version control. Again, Ant or some automation tool might be a big help.
[ December 16, 2007: Message edited by: Stan James ]

[Nicholas Jordan]

Originally posted by Stan James:
Nicholas, I'm a little lost on just where you are these days. Your post suggesting

\belvedere
    \Alice
       \src
       \class
       \misc
    \MailCleaner
       \src
       \class
       \misc
    \Editor
       \src
       \class
       \misc
    \SearchTool
       \src
       \class
       \misc

looked right on the money. But the line after it each proj source file is suddenly in the src package should not be true. In a command window change directory to any src folder and the packages start with the folder under that one for compiling or running.

It was ( the remark ) a consequence of having \src showing up in each dir, thus requiring imports to have dot src in every source file, which as I made the remark defeated the purpose of refactoring the project into packages.

[sidekick:]   (I left out the -d directive to put the classes in the class folder.)

That is the only part that is working, the -d directive.

[sidekick:]   You have to change directory and compile in each src directory. You can't do them all at once. You can think of packages as a way to organize files, but the higher level folders - Alice, Editor, etc - are something else, maybe build or deploy or reuse units. That is, you can build one without rebuilding the others, you can deploy one without redeploying the others, you can give one to somebody else to reuse without the others. If that's not required, maybe don't bother making separate trees.

That would defeat the purpose of this post. What I called belvedere above is really and truly a build directory from which several projects are managed and cvs'd ( my idea of cvs, not some tool ) I named that package, root, dir belvedere because that naming follows the logic given for ( post title ), then belvedere came to have several projects as a natural evolution of the work. Utilities showed up first, then there was some circualar dependencies developing and I tried to implement packaging naming and directory structure so that I could import so that I could do what I intended to do. It just evolved, fell out or developed naturally. It comes from writing java source code, not everyone concepts directly in source code. What works for me is to come back an apply the higher-level tools after the project is coded and if the source does not meet the test, then look for what does not meet the test and refactor then.

[sidekick:]   When you compile Alice, how do you take in the Editor classes? One choice is to add Editor/class to your classpath. That's very dynamic, picks up every compiled change in Editor, but maybe a little sloppy.
Another choice is to build an Editor.jar and add Editor.jar to the classpath when you compile and run Alice. I an extra step, but helps make sure you're using the real jar that will be deployed, not some change that you might forget to build into a jar.

Make a dir named class that is off the root dir, named belvedere here and named LOCAL_MACHINE | REMOTE_MIRROR in the post dated #Friday, October 26, 2007 11:48 PM# - this is a sort of repository for anything that is shippable and already has several dot jar files from other workers in computer science. I would like to be able to use these files without needing a copy in each project directory. Some of these are very powerful tools by masters in the field. I am extremely experienced on keeping up with what is actually being shipped, I know it to be a profoundly deceptive task and have exceptionally strong skills in the matter.

[sidekick:]   Note that the classpath may be different in every src folder you visit for compilation. That's part of the complexity of making separate build and deploy units, but generally well worth while. Ant and similar tools exist to document and automate this kind of thing.

I looked at Ant, they had several .... the thing here is every time one goes to use one of these things there comes to be a disparate set of issues that has to be resolved and has to do with the use of the tool and I thought it better to know what was actually going on by writing every line of code myself. The reasons why are usually dismissed as Meaningless Drivel.

[sidekick:]   One danger on this road is circular dependencies. If Alice uses classes from Editor and Editor uses classes from Alice, it's hard to build both jars from scratch. Robert Martin is the guru on the dependency management topic; his Agile Software Development book should be required reading to build anything with more than one jar. Or maybe more than one class.

I have aleready figured out that I will resolve the circular dependencies by manually building as much as needful to break out of the cycle. My utilities has already evolved as the place to start. This 'manual building' was also the basis for the decision to write a CompilerShell. java and build that first. That effort failed on the classpath issue. I decided to to a java.io.File.ListFiles(); from root ( which should actually be named belvedere to follow packaging intent ) and do some sort of recursive search on if(isDirectory()){} to either dig into what I named src folder above or more efficiently just look for dot java files in each isDirectory() of a directory listing from root.

I named the root folder LOCAL_MACHINE | REMOTE_MIRROR in the post dated #Friday, October 26, 2007 11:48 PM# in an attempt to shake out a pluggable root so that I could have backups placed willy-nilly where ever I could find archiving space. It started to get overly complex for others so I culled one layer thinking I could be efficient with a pluggable tree that maybe had to be build manually, but was recoverable. To this end I revisited CompilerShell.java but made no significant progress beyond major cleanup of scratch and paste initial effort. As part of that effort I re-visited Ant and much to my relief found a flaming rebuttal of the entire classpath whatchamacallit alltogether.

[sidekick:]   Phil, are you hangin' in there? The examples with separate src and bin folders (or src and class) show how to separate source from compiled code. You can easily zip a src folder or link it to version control. Again, Ant or some automation tool might be a big help.

What happens here is that Phil's idea: and there's no need to back up executables as I can recompile in seconds. one day can arrive at a complete loss of all work due to inadvertent keystrokes from the hand moving around the keyboard. I have arrived at an efficient and relaible control for this by copying the *** ENTIRE *** whole everyting, dot class files or source or whatever, to another drive verbatim using the shell and renaming the copied folder to copy[1] and accumulating copy[1] ,copy[2], copy[3] and so on, in toto.

I am sure because of your approach that displays time on large projects reveals knowledge of a risk of total elimination of entire projects and has to be provided for and protected against by write-protected media and access controls. This is something not to be overridden by shell-available commands.

[ December 16, 2007: Message edited by: Nicholas Jordan ]
[ December 16, 2007: Message edited by: Nicholas Jordan ]

[Stan James]

I'll repeat a couple points just to highlight them, and then retreat back to my own planet.

The folder structure above does not require src in the package for every project. Instead, it requires you to navigate to the src folder in each project and compile each project separately, one at a time.

If you don't care to compile projects separately and bundle up the classes for deployment separately, skip all this pain. Put all the projects in one source folder, run javac once and generate one class folder. The package structure is enough to keep them organized.

If you want to build separately and you find ANT to have an overwhelming learning curve and setup issues for smallish projects, you're not alone. Write a batch file that does it.

cd \belvedere\Alice\src
javac ...
cd \belvedere\Alice\classes
jar Alice.jar ...
cd \belvedere\Editor\src
javac ...
cd \belvedere\Editor\classes
jar Editor.jar ...

If it bothers you that this takes a while and compiles things that haven't changed, bite the bullet and learn ANT or Maven. That's why they exist.

[Nicholas Jordan]

Well this is exactly what I was going to do, except use a Java program I wrote myself rather than a batch file. This failed largely on Process Builder and Runtime.exec(String[], String[], File), I go for sometimes 12-16 hours at a stretch writing this stuff and my mind becomes saturated . The classpath issue seems to be something which I will have to comprehend in greater detail because when I tried to do builds recently from a cli and from within Textpad, I got error messages I do not remember - they seemed to have the general nature of 'compiler cannot find class' so-and-so so I posted this so-so post in the ho-ho-hopes of {     } feeding the classpath to each javac invocation ( along with any other information needed from the command line )

At one time and on an earlier machine and javac I was able to use the @files command line option to overcome my woes I now face.

Recently, I could not get the compiler to do re-compiles even on files that I already had working builds for. I started using the packaging, naming and directory structure like I thought I was supposed to because the time had come that I needed it, and the entire build process came to an abrupt and un-anticipated halt.

[ Message edit: 12:18 PM 12/25/2007 - I tried this again this morning and ran into the same problems, acheiving a build-run on a defalut package by going to a cli and using -cp . ][everything else failed, including approaches that had worked before, I think it is the shell getting in the way]

[ December 25, 2007: Message edited by: Nicholas Jordan ]
[ December 30, 2007: Message edited by: Nicholas Jordan ]