File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Tomcat and the fly likes Best practices for storing configuration/property files Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Murach's Java Servlets and JSP this week in the Servlets forum!
JavaRanch » Java Forums » Products » Tomcat
Bookmark "Best practices for storing configuration/property files" Watch "Best practices for storing configuration/property files" New topic
Author

Best practices for storing configuration/property files

Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

What do folks do in production environments for the assorted "Property" files that are typical in a large Java application? I'm thinking of files such as the log4j properties, properties for defining cache sizes, SMTP servers, etc.

Its easy to have the IDE put them in the WAR file with all the class files. Deploy the WAR, and you are done.

But Operations folks don't want to go into a WAR file to change the hostname of something as simple as an SMTP server. They are much more comfortable editing a simple text file using vi, emacs, etc.

So where do you put them? How do you bind them to the application?

I assume that the same solution that works for Tomcat would also work for Glassfish, JBoss, etc.
Lester Burnham
Rancher

Joined: Oct 14, 2008
Posts: 1337
In WEB-INF, generally, and have a <context-param> entry with its name in web.xml. Then a ContextListener reads the entry, and does whatever needs doing with the file's contents.

Having an editable file in WEB-INF only really works if the web app is run from a directory structure, though, not if it's run directly from a .war file, or from an exploded .war file.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

Lester Burnham wrote:Having an editable file in WEB-INF only really works if the web app is run from a directory structure, though, not if it's run directly from a .war file, or from an exploded .war file.


Since the WEB-INF is in the WAR, I don't see this has having any advantage over just putting the properties files in the it.

Exploding the WAR seems to be going in the wrong direction. the nice thing about a WAR is that you give the one file to the OPS folks and they deploy it and you are done. These config files are unlikely to change with code versions, but they are sure to change as we grow from one server to 20.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

One of my most common solutions is this:

1. Select/create an external configuration directory that will contain the config files. In Linux, that might be /etc/myapp or /opt/com/mycompany/myapp, or /var/lib/myapp, depending on circumstances. Or one of several other possible locations. /usr/local/etc used to be popular, as did /usr/local/myapp/etc. In Windows, you're more or less on your own, although SYSTEM_DRIVE:\Program Files\mycompany\myapp is popular. Or Documents and Settings, although that one's a bit "iffy" for servers.

2. Define a JNDI property that resolves to the pathname of that configuration directory. The webapp will do a JNDI lookup to get this value. The default value is in web.xml, but in Tomcat I can override it in the webapp Context.

3. If I want default property files, I put them in the application classpath and use the request getResource methods to access them.

4. In some cases, especially for volatile properties in database apps, I'll keep a properties table. I also find this useful for storing properties that are shared between multiple instances of the app (if there are any).

By using the above techniques, I can make basic deployment simple and have the option to set properties on a "management by exception" basis. Because the location of the override properties is itself overridable, I can select from development, test and production profiles without mods to the WAR, and I can run multiple app instances, each with its own set of properties.


Customer surveys are for companies who didn't pay proper attention to begin with.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

Tim Holloway wrote: Select/create an external configuration directory that will contain the config files. In Linux, that might be /etc/myapp or /opt/com/mycompany/myapp, or /var/lib/myapp, depending on circumstances.


Interesting, that is essentially the approach that I have been using, without the JNDI. Perhaps I was not off too far.

What I did was look for the file in the following order:

1) look for a special environment variable, if present, get the file path from that.
2) have Java look in the usual web/WEB-INF/classes directory

I was expectation to find out, "you dummy, this is how we do it" but so far, the solutions are isomorphic to what I'm doing.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

It's a bit "iffy" looking for OS environment variables in webapps. But the JNDI-located context variables are functionally equivalent. So yes, you have my seal of approval
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

Tim Holloway wrote:It's a bit "iffy" looking for OS environment variables in webapps.

Please tell me more. Why?

I have found that getting them set on less than a complete OS wide basis is problematic. But for a dedicated server that is running not much more than the servlet container, postfix, dns, and mysql, it seems OK to make it system wide.

I have no idea how to make it work user-specific, since the Java developers have no idea what "user" the container will be logged in as.

Tim Holloway wrote: But the JNDI-located context variables are functionally equivalent. So yes, you have my seal of approval


Based on your comment, I went off to the Sun/Oracle JNDI tutorial. I'm lost. They seem to think that "simple" mean CORBA, RMI or LDAP. Do you know a link to how to use JNDI to access what is actually just a file on a disk somewhere.

I tend to believe that if CORBA, RMI or LDAP is the solution, you are asking the wrong question.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

There was actually a certain amount of conflict just getting Sun to support environment variables at all. They're right at the edge of "write oncde/run anywhere". Windows, for examaple, has 2 environments (global and per-user), and some OS's had none. Fortunately, there was enough of a common denominator. That's part of the reason I avoid them for webapps. The other reason is that webapps and their containers are generally self-contained little worlds, and they're complex enough that injecting variable information from intangible outside sources is likely to be a source of trouble, considering that Lower Prices Everyday is a principle applied as much or more to sysadmins as it is to developers.

I think you misinterpreted the mechanism I use, though. It sounds like you think I pull the entire config in via JNDI. I don't. I may supply a few special parameter values as JNDI-resolved items, but when there's a lot of them, I use ordinary property or XML files. The JNDI is just there to get the (absolute) path location of a config file or directory as a string which I then pass to java.io.File in the usual way.

Here's a typical config:


To obtain the value for java:comp/env/config, I use basically the same javax.naming code as people use to get a DataSource, except that the returned object is a String whose value is "/etc/myapp/app.properties". That's the file I'll open to get the properties.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

I can't understand enough of JNDI to spell it.

I was assuming that the JNDI just gives the path to the normal Java Properties file.

Where does the config file go? I'm really lost on how to make JNDI do anything useful.

Thanks
pat
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

JNDI is the Java Naming and Directory Interface. It provides an abstract access to hierarchically-structured (or not) data. Although JNDI naming paths resemble filesystem paths, they're really just organized labels. So are URLs, for that matter, but that's another topic. LDAP services are a more complicated form of this function, but Tomcat just uses basic JNDI, defining structured key/value pairs. Like a hierarchy of HashMaps, if you like, although JNDI doesn't actually assume that HashMaps are in use. Whether Tomcat's actually using HashMaps or something entirely different is something we don't worry about, since JNDI provides the access mechanism.

Anway, the value for the particular key/value pairing that I established with my Environment element is simply a string that's the pathname where I find my config data. The configfile is just an ordinary file and I can put it anywhere I want to. Once I've retrieved that pathname string, I could care less that I used JDNI to obtain it. It's a string, and I use that string to do standard properties functions like so:


The JNDI code itself is simple. Like I said, take a look at one of the snippets that people have been posting lately that gets a Database Connection pool. Almost exactly the same code except for the JNDI path (java.comp/env/xxxxx). Instead of casting the results of the context.lookup method to a DataSource, cast it to a String. and that's the "stringFromJndi" value above.
Actual code may vary slightly.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

Tim, I have the conceptual ideas, but I am lacking specifics. In an upthread posting, you wrote:

Tim Holloway wrote:
Here's a typical config:



What "config" is this. I assume its some sort of JNDI config. Where does it go? I'm sure its very basic, but I'm not seeing it.

All the Sun JNDI stuff is very abstract until it gets very specific about RMI or other distributed stuff that make my eyes glaze over.

Thanks, Pat
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

The Environment element goes in as a sub-element to the Context element and all it does is map the JDNI name to a value. In this particular example, the name is "config" and the value is "/etc/myapp/app.properties", which is the pathname of my properties file. On Windows, I might have made it something like "C:/config/myapp/app.properties". The URL that Tomcat defines for this particular environment definition is "java:comp/env/config", which is what you get after prefixing the "name=" attribute with the standard JNDI prefix.

JNDI is supposed to be abstract! It supports several different directory systems, of which the basic JDNI used by Tomcat and the extended JNDI for LDAP/Active Directory are the most common. However, as I said, it's just a highfalutin' hashmap when all's said and done. You give it a name, it gives you a value. Map a string to a name, you get back the string. Map a Java DataSource to the name, you get back the DataSource.

RMI is a completely different thing. RMI is how you call remote methods on a suitable co-operative interface and has no direct relationship to JNDI. However, in order for remote users to get a handle to that interface, they register their handles with a directory server, which is generally going to be a JNDI server. But Tomcat's JNDI server doesn't publish itself publicly for external users - it's only intended for internal use by webapps, so RMI services (such as EJBs) can't be located using the Tomcat JNDI.
Pat Farrell
Rancher

Joined: Aug 11, 2007
Posts: 4646
    
    5

Tim Holloway wrote:The Environment element goes in as a sub-element to the Context element


So what is the Context element?
Does it go in some JNDI.XML ile?

While I grok that JNDI is supposed to be abstract, if I am to have any prayer of testing it, let alone implementing it, I have to know what parameters go in what file in which directory.

This reminds me of the worst parts of J2EE.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 15952
    
  19

Whoops.

http://tomcat.apache.org/tomcat-6.0-doc/deployer-howto.html#A_word_on_Contexts
 
Don't get me started about those stupid light bulbs.
 
subject: Best practices for storing configuration/property files
 
Similar Threads
Getting the current host address
Properties file or Database table ?
Updating the contents of a war file in production environment.
execute on double click
[Urgent]Accessing properties file