File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Ant, Maven and Other Build Tools and the fly likes Is Maven that good? Non-Standard artifact creation. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Ant, Maven and Other Build Tools
Bookmark "Is Maven that good? Non-Standard artifact creation." Watch "Is Maven that good? Non-Standard artifact creation." New topic
Author

Is Maven that good? Non-Standard artifact creation.

Andre Treffeisen
Greenhorn

Joined: Feb 23, 2013
Posts: 5
Hello,

We work with a vendor which provides hot fixes and we need a way to integrate them in our build. We used ant before but now want to migrate to Maven.
We have a hot fix control file with an ordered list of hotfix jar files. So each line of that file (csv format) has a name of a hot fix jar and the directory in which it is located.
Each hot fix directory contains a hot fix jar which contains the classes; Additionally some of the hot fix jar files contain source files or java files.
Only some of the hot fixes in the hot fix control file are valid. This is determined by a status or a comment. If they should be skipped then they are marked with INACTIVE or as comment (# or ') at the beginning of the line. Currently it is the status name but a comment would do as well.
For example: (hot fix control file)
ACTIVE,hotfix1/hotfix1.jar
INACTIVE,hotfix2/hotfix2.jar
ACTIVE,hotfix2b/hotfix2b.jar
...
For example: (hot fix control file with comments)
hotfix1/hotfix1.jar
#hotfix2/hotfix2.jar
hotfix2b/hotfix2b.jar
...
Maven should now do the following: Unzip each valid or relevant hot fix jar and put the class files in one directory and the sources in another one (preserving the package structure).
Then it should Jar the classes folder. -> artifact 1 and Jar the sources folder. -> artifact 2. For the source files no compilation is needed (just used for debugging.)

We could now just use the ant maven plugin and call the existing ant scripts. But we want to utilise Maven as much as possible. So if we could just do it the Maven way that would be great. But some questions are really hard to tackle, does anyone have an idea?

Is there a 'standard' Maven way of doing it, avoid writing a plugin or using the ant plugin?
How can Maven consider the order of the hot fix?
How can Maven consider the status column/field or the comment in the hot fix control file?

I guess we could use the prepare-package lifecycle step to do the unzip and in the package the other part.

Would we need to do two modules to get the two jars (classes jar and sources jar) or can it both be done in one module?

Thanks for any help and input.

Anmatr.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2057
    
  22

Step 1) seperate sources from runtime artifacts
Step 2) load runtime artifacts and source artifacts into your maven repo

Done
Andre Treffeisen
Greenhorn

Joined: Feb 23, 2013
Posts: 5
Jayesh,
thank you for your reply but unfortunately I don't understand what your are refering to or what you mean. Could you please explain?
Thanks.
Anmatr.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2057
    
  22

Ok, so in maven, what happens is that you put libraries that you depend on into a maven repo. Popular examples of maven repos are Nexus and artifactory. Each artifact has a group id, artifact id and version number. Then you configure your pom.xml to be dependent on those artifacts. You put the groupid, artifact id and version number in your pom.xml. You don't check in the jars into a lib folder like you would do with ant. When you run the build, maven downloads the jar from the maven repo and puts it in compile path and classpath during unit test execution. If you ask maven to package your app (for example in a war), it will automatically include the dependent libraries in the war

Now, if you have third party libraries that change often, there are 2 ways of handling it
a) Upload the new version into maven repo and increment the version number. ALso change your build to point to new version
b) Upload the new version into maven repo as a snapshot depdnency , and use SNAPSHOT in version number field in your pom.xml. This tells maven that this library is subject to change often, and it should download the latest version whenever a new one is available

So, if you use maven the reccomended way, you don't need to do anything to manage hotfixes of 3rd party libraries. All you need to do is upload the latest version of third party libraries into your maven repo. As maven is becoming more and more popular, vendors are moving towards hosting their own maven repos that make it easier for their clients to get the latest version of the jars. You might want to check with your vendor. However, if your vendor insists on using this crazy way of releasing hotfixes, all you need to do is do the processing of their control file when you upload their releases into your maven repo. The build of your application should stay relatively simple.
Andre Treffeisen
Greenhorn

Joined: Feb 23, 2013
Posts: 5
Jayesh,
thank you for being so explicit that helps understanding.
I'd hope to tell you something better but if I may explain the way this vendor publishes hotfixes does not make you (and me) feel good, but it is the way we have to deal with. There were several attempts to change it but nothing ever happened (in 10 years).
Instead of sending out one hotfix jar, as you assume, the vendor sends out a jar for each and every fix they do (individually per client). So in our SVN we currently have 140 hot fix jars which contain a few, sometimes only one class file and (optional) source file. As we use JSE 1.6 with JWS and as there is (to my best knowledge) no guarantee in the order of the class path on the jnlp resources we have to do this tedious task of unzipping the hot fix jars, and package them together in one jar in the right order. The idea is now to use this hotfix.jar and hotfix-src.jar going forward and publish into Nexus whenever we have a new hotfix and release.

The other reason for not publishihg each of the hot fix jars to Nexus is that there can be changes in class files which were already in an earlier hot fix. If we have both of this jars on the classpath we don't know which of the two jars comes first and it might turn out that it is the old one first. So the hot fix would not work (which is what already happened, hence the repackage task). Currently I cannot see any better way of doing this, which brings us back to my original question, how to do the repackaging in Maven. In the meantime I had a further read on the assemblies and hope that we might get something with that concept.

Still my initial questions stay unanswered as far I understand.

Thank you,

Anmatr.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2057
    
  22

You shouldn't do this in maven. If your vendor is releasing individual class files then implement an utility that pulls jars from your maven repo, unpacks them, applies hotfixes and deploys them back in the maven repo. Ant is a good tool for this. However I would just do this in a scripting language that has better integration with Java.. for example groovy
Peter Johnson
author
Bartender

Joined: May 14, 2008
Posts: 5779
    
    7

I do something similar. I have various patches to certain JAR files that are available in Maven Central. In my POM I declare the official JAR in the dependencies, and the project contains only the sources I changed. The POM then compiles the sources, unpackes the JARm and then builds a new jar with different coordinates. The other projects use the "fixed" JARs coordinates, not the original one.

You could adapt that technique. Use the Depdnency plugin to unpack the original JAR and then each of the patch JARs, and then use the JAR plugin to build the new, fixed JAR. Note that you do this in a separate project. Thus you have one project per JAR file that rebuilds the JAR, applying patches and creating new patched JARs with new coordinates. Then all of you application projects use the new cooordinates in your dependencies.

Here is an example for extracting the original and one patched jar:



This should extract each of the JARs into target/classes, starting with the original and then extracting the first patched JAR on top of that.

Then when the JAR plugin runs, it'll build the new JAR file.

Some other points:
* The POM uses <packaging>jar</packaging>
* You should place the original JAR and each patched JAR separately in your repository
* The POM that does this should create a JAR with different coordinates. You can change the goupID (for example, add you project name to the original JAR's groupId) or change the version (e.g. add ".patchedXX" to the original JAR's version number where XX is patch id information)

This should be enough to give you the idea.


JBoss In Action
Andre Treffeisen
Greenhorn

Joined: Feb 23, 2013
Posts: 5
Jayesh, thanks for your reply. I didn't say that they send individual class files, the vendor sends individual jars containing classes and sometimes classes and source files. I know that ant is a good tool to do this (I already did this with ant in an earlier project) but my point was that we want to know if that is possible with Maven, hence the subject to this topic.

Peter, thank you, that looks promising and could go the right way. But I only found out that I forgot an important part in our procedure. We have to make sure that no class is available in two different jars (Java Web Start could cause issues). For example:
The core product is in product-core.jar, features are in product-feature-a.jar and product-feature-b.jar.
The first hot fix provided includes now two classes, one class is patched from product-core.jar (fix for class A.class), the other one is from product-feature-a.jar.
The second hot fix provided includes also two classes, one class is from product-core.jar (again class A.class, the other one is from product-feature-b.jar.
Now we have to unpack the first hot fix, then the second hot fix, both into the same directory "hotfix" to overwrite the first A.class with the second delivered A.class.
Then we unpack the product-core.jar and remove every class from it, which is contained in the hot fix directory. Now we repack product-core.jar and deploy to repository.
Then we unpack the product-feature-a.jar and remove every class from it, which is contained in the hot fix directory. Now we repack product-feature-a.jar and deploy to repository.
Then we unpack the product-feature-b.jar and remove every class from it, which is contained in the hot fix directory. Now we repack product-feature-b.jar and deploy to repository.
We pack now hotfix.jar which keeps all hot fixes in a separate, new jar, it will also be deployed to the repository. Optionally we have a "hotfix-src" folder which keeps the sources and pack that into hotfix-src.jar.

Looking at the dependency plugin, I think it has already some features which we could use. But, does the procedure you described rely on the sources? Unfortunately we don't have always sources to the hot fixes, so we need to work with the class files only. As I came in later into this project there are already some badly handled hot fixes there and it is quite hard to get the sources from the vendor. They try to lock down shipment of sources. So we cannot build.

All, thanks for your input so far, I know it is a tedious problem, any help highly appreciated.

Thanks,
Anmatr.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15641
    
  15

You could construct a Maven mojo that would scan the hotfix list and copy the required hotfixes into the target. The main thing I don't like about that is that it basically makes for a collection of "invisible" dependencies. Although since transitive dependencies are a Maven staple, perhaps I worry too much.

It's really annoying when people use the same filename for different versions of something, but I think Nexus can still help out there. It does allow for archiving snapshots, for example.


Customer surveys are for companies who didn't pay proper attention to begin with.
Andre Treffeisen
Greenhorn

Joined: Feb 23, 2013
Posts: 5
Hello Tim,
honestly, I have no idea what you trying to explain.
If I try to understand what you say: Simply copying the hotfixes to a target folder would not be enough. The hot fixed classes also need to come out from the original product or feature jars.

Thanks anyway,
Anmatr
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15641
    
  15

A mojo is a Maven plug-in module. You can write your own in order to extend Maven's capabilities.

In a case such as yours, that module could be written to reference a hotfix manifest and repositor(ies), then use the manifest to pull the hotfixes and insert them into the maven target. Whether you copy hotfix JARs wholesale, extract classes from hotfix jars and copy them to the target's "classes" directory, or even open JARs previously inserted into the target and update them with hotfixed classes, is all entirely up to what you program the mojo to do.
Peter Johnson
author
Bartender

Joined: May 14, 2008
Posts: 5779
    
    7

First, the dependency plugin doesn't rely on sources. I just gave my situation as an example.

So you are saying that you will get a single patch JAR file that contains patches for possibly multiple original JAR files? Yuck! A few possibilities: a) manually break up the jar file yourself; b) use the include capability of the dependency plugin to include only the desired class files; . These options are easy if there are only a few files but get really error prone if there are large number of patched files.

 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Is Maven that good? Non-Standard artifact creation.
 
Similar Threads
Running help:describe command in Maven
How to split generated classes into different jar
how do i add a jar file externally as a maven dependency in eclipse?
Why Maven?
Hot-deploying/replacing classes in WEB-INF