The application - - - - - - - - - - - - - - - - - - - - - I am creating a java application which has a Plugin interface (which contains several methods).
I also have a plugin directory, which contains a bunch of JAR files. In the application I create a URLClassLoader which loads all the JAR files in this directory.
The problem - - - - - - - - - - - - - - - - - - - - - Now I want to "inspect" all the classes supported by this URLClassLoader to find all concrete classes which implement my "Plugin" interface. Currently i have a "plugin.xml" which contains some information about each of the supported plugins (including the fully qualified class name); however, I would like to make this automated and have all the information about the class inside the definition of that class - and have the application search for all classes of the designated type using a defined ClassLoader.
Can someone give me any ideas on this? Basically I need something with the following signature: or even [ May 31, 2007: Message edited by: Scott McGhee ]
Thank you for the link, I read most of it (actually i've seen this before); however, I noticed that in "Step 2" (which is really what my question is about) it does not a lot of specifics.
Here are some of my issues:
- The plugin directory can have any number of JAR files, each JAR can have any number of classes extending the Plugin interface, each plugin can have a different package declaration.
- There will be some "core" plugins located somewhere inside the main classpath (system classloader)
- I am trying to avoid having to "hard-code" the plugin classnames in a plugins.xml
- if it is at all possible I would like to be able to scan any and all class files which would be available to a particular ClassLoader (and any of the parent ClassLoaders) and see if that class implements my Plugin interface.
- I don't want to hard code the plugin class names, and i want the plugin jar files to be as light-weight as possible (i don't want some file in the META-INF, for example, to declare the class names)
- taking Ant as an example, if you create a customized Ant Task - you would need to declare the classname in a 'taskdef' - this is the kind of thing i would like to avoid.
It seems like avoiding some sort of configuration could be a problem. What happens if there's more than one class in the plugins directory that implements the requested interface? Your earlier API ideas acknowledge this possibility, which I think is a good idea. Unfortunately if you allow more than one plugin class of a given type, there doesn't seem to be any good way to automatically choose one implementation over another, is there? Requiring a user config file may seem (mildly) undesirable, but it seems much better than, say, just picking the fist implementation you find, isn't it? That could lead to unpredictable, confusing results. Perhaps in some applications you could pop up a dialog and ask the user which plugin they'd prefer to use. But in general, that seems a poor choice.
Ultimately I think you'd probably be best off just accepting some sort of config file as a necessary evil. Which doesn't seem that evil, really. Doesn't have to be xml, either. In many cases a simple properties file will work just fine.
I have a little system that loads any number of implementations of some plugins ... macros and commands and such. I gave up and listed them all in configuration. It would be cool to "discover" all these things and register them and get rid of the configuration, so I'm sympathetic to the desire to automate this.
I think we'd have to load every class we can find and see if it implements an interface. Yuck.
A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Joined: Jan 30, 2000
[Stan]: I think we'd have to load every class we can find and see if it implements an interface. Yuck.
If using reflection, yes. The code is not complex; I assume "yuck" is referring to performance. It doesn't necessarily take that long - I guess it depends how fast you need it to take. Loading the entire class does seem wasteful when we only care about finding the implemented interfaces and extended classes. If it's too slow, one could probably use BCEL or ASM to get that info from the class file without loading the entire class. I would try it with reflection first, then replace with a faster method reading bytecode if necessary.
Note that it's not enough to just check the list of implemented interfaces. you need to check each supertype as well. If A extends B which implements C which extends D, then neither A nor B will have any mention of D in their class files. If you load the class and use reflection, it's easy to learn if A implements D. But using faster bytecode methods, you need to check B and C yourself. So it requires more care and effort.
I think a problem will be that you're saying some plugins could be in the system classpath. Since a running app does not know what that is, and which files and directories it contains, you can't recurse through it trying find implementations of the plugin interface.
Or is there a way for a running app to know the classpath of the JVM that it's running in?
Ping & DNS - updated with new look and Ping home screen widget
Joined: Jan 30, 2000
System.getProperty("java.class.path") should take care of that. However it doesn't necessarily include libraries known only to the bootstrap classloader, like the main library in rt.jar and anything in extension directories. The latter can be found with System.getProperty("java.ext.dirs").
Joined: Apr 10, 2007
Originally posted by Jim Yingst: Requiring a user config file may seem (mildly) undesirable, but it seems much better than, say, just picking the fist implementation you find, isn't it? That could lead to unpredictable, confusing results. Perhaps in some applications you could pop up a dialog and ask the user which plugin they'd prefer to use. But in general, that seems a poor choice.
I was thinking that one of the methods inside the Plugin interface would have a "knickname" method (like "getKnickName()"). My application is very similar to Ant, in that the plugins are actually processors for individual XML tags (but more specific to generating web page forms). So the processor looks at the tag name, then uses the tag name as the "knickname". Then I have a factory class which, given the knickname, will return an instance of the Plugin class which has that knickname. Currently the factory uses the "plugins.xml" to find the supported plugin class names.
I want the jar files to be as portable as possible - in fact my plan is to host the JAR files in a database so that the website can have an "admin console" to upload new JAR files.
So what i want to move away from is having a central plugin configuration, and instead simply control which JAR files are used in my ClassLoader. I even want the JAR files to allow versioning, so I'll create "JAR Groups" which can then support different versions of a single JAR file.
As for plugins in the actual class path - i think I'm going to host all the 'core' plugin files in a 'core' plugin JAR file (which i'll drop in the plugins directory). In this way, the Plugin factory class can be sure that any Plugin it needs to create will be located in a JAR located in one of the JAR URLs in the "JAR Group" (and definately not in the CLASSPATH).
Possible Solution - - - - - - - - - - - - - - - - - - So I tried the following approach (which seems to be working):
1) Input to the PluginFactory's constructor will be the URLClassLoader (which contains all the JAR file URLs)
2) Create an ZipInputStream to each of the URLs in the list, search each ZipEntry to find the name - if the name endsWith '.class', then convert the name to a className (replacing '/' with '.' and removing '.class')
3) Instantiate the Class object for this className (using the URLClassLoader) - check if that Class .isAssignableFrom(Plugin.class)
4) If it is a Plugin, then create an instance of that object and invoke the 'getKnickName()' method
5) Store the knickname as a key in some hashMap, the value of which will be the Class object
6) In the main 'createPlugin()' method, take input the knickname - lookup the Class object in the hashmap, and create an instance of that Class object
Concerns with the above - - - - - - - - - - - - - - - - - - - - Step 2 needs to 'reload' the JAR files through an input stream; so physically the JAR files will be loaded twice - once to check for .class files - and once when the URLClassLoader needs to actually load the class in step 3. (this is not ideal since i would like to only process the JAR file once, if at all possible)
Also - after step 2 & 3 the URLClassLoader will have instantiated and cached every available class in the JAR files. This could be a good thing, or a bad thing - depending on how you look at it.
What do you guys think - can i do better than my solution?
I'm not sure yet what I think about my solution, since I would only accrue the cost of looking at the JAR files 2 times once (when i create the instance of my PluginFactory class for a particular JAR Group).
Or maybe should I just give in and require my Plugin Admin console to request that when you add an additional plugin JAR file at runtime that you, in addition, provide the Class names of all supported Plugins. [ June 01, 2007: Message edited by: Scott McGhee ]
Joined: Apr 10, 2007
One other thing:
Does anyone know how Eclipse creates the "Type Hierarchy" for an object?? This is kind of what I'd want - for example if I open the Type Hierarchy for the Plugin.class, it will show me all objects which implement the Plugin interface. That's pretty much exactly what i'd what, except it would be for classes within a given URLClassLoader instead of just the pre-defined classpath for the Java Project.
Joined: Jan 30, 2000
Well, it's sounding much more workable now. The nickname (note spelling) allows you to choose between multiple implementations, at least in the most common cases. Is it possible that you might have two jars with two different Plugin implementations for the same nickname? Possible ways to deal with this would be: (A) throw an error, (B) use the one with a higher version number (assuming version numbers are supported), or (C) use the one that came first in the classpath. I dislike B and C because they lead to situations where a developer thinks he's installed something, but the code is actually using a different implementation. But then, that can happen with jar files anyway, so maybe C is a perfectly sensible option.
I might use a JarFile or JarInputStream rather than ZipFile Or ZipInputStream, just because the Jar classes have a few moore specialized methods for, well, jar files. Not sure if that really matters here, just wanted you to know the Jar classes exist.
[Scott]: Step 2 needs to 'reload' the JAR files through an input stream; so physically the JAR files will be loaded twice - once to check for .class files - and once when the URLClassLoader needs to actually load the class in step 3. (this is not ideal since i would like to only process the JAR file once, if at all possible)
You don't really need to load the Jar twice. You're looping through the entries(), and for each entry corresponding to a .class file, you get the associated InputStream, corresponding to that entry only. That's pretty much how you'd normally loop through a zip or jar file. You're not revisiting any parts twice.
However, I think you will want to introduce an intermediate step which will cause you to load some parts twice. The reason is because of this:
[Scott]: Also - after step 2 & 3 the URLClassLoader will have instantiated and cached every available class in the JAR files. This could be a good thing, or a bad thing - depending on how you look at it.
My guess is it's bad. Depends how much stuff you have in all your jar files, I guess. But as long as at least some potential users of your software have some large jar files lying around in their classpaths, this is risky.
A simple alternative would be to load each new class with a new and different ClassLoader. For any class that you don't need, just ditch both the Class and ClassLoader when you're done. That should work. However it may introduce more overhead, in terms of time and memory to create new ClassLoaders. I'm not sure how significant this is.
A more elaborate alternative might be to do an initial search for classes where you switch to a new classloader every 100 classes, or every new jar file, or something similar. But in this case, you don't want to permanently retain any Class object obtained this way, because every time you do that, you also inadvertently retain the associated ClassLoader which in turn retains all the other classes loaded by that loader. So retaining one Plugin class might cause you to keep 100 other irrelevant Class objects in memory. Ugh. Instead, every time you find a Plugin class (or maybe just a Plugin class matching the desired nickname), ditch it, and reload it with a differentClassLoader - one you plan to keep around, because it contains only classes that you want to keep. Using this scheme, yes you'd end up processing some classes twice. But in return you'd use much less memory, I think.
However it's possible I'm overestimating the resources required for each new ClassLoader object. If it's small, then the simple alternative of the previous paragraph is fine: just use a new ClassLoader for each new class, and ditch the ones you don't need.
Note that this problem pretty much goes away if you use BCEL or ASM to scan files before fully loading them. That way you can determine if a class implements Plugin before you bother loading it as a Class. It's more work to get this working, but it should be both faster and less memory intensive. Maybe it's something to plan for in version 2.0.
[Scott]: Or maybe should I just give in and require my Plugin Admin console to request that when you add an additional plugin JAR file at runtime that you, in addition, provide the Class names of all supported Plugins.
You could let that be optional. If they don't supply such a list, you scan all the jar files, and error out if you find any Plugins with the same nickname. If they do supply a list, they get faster loading time, and the option of having jars with different Plugin implemenations for the same nickname. Best of both worlds, maybe?
[Scott]: Does anyone know how Eclipse creates the "Type Hierarchy" for an object??
I don't know the details offhand, but it seems like it should be possible to work something out using a combination of reflection to get supertypes, and nicknames. [ June 01, 2007: Message edited by: Jim Yingst ]
Joined: Apr 10, 2007
Thanks Jim! That was a very complete response, and it helps me a lot!
I think there are some really good ideas here, I'll probably try first to load all classes in the JAR and then dispose of that ClassLoader so that it initially gets garbage collected. If i run into any memory issues i'll consider changing it later. Thanks again!