aspose file tools*
The moose likes Java in General and the fly likes Reading / Storing Data Without Knowing What it Is Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Reading / Storing Data Without Knowing What it Is" Watch "Reading / Storing Data Without Knowing What it Is" New topic
Author

Reading / Storing Data Without Knowing What it Is

Roger F. Gay
Ranch Hand

Joined: Feb 16, 2007
Posts: 376
I'm building a program – let's call it a generic framework - that reads configuration data from XML files and should make that data easily accessible to application programmers. The trick is that the program does not know ahead of time what's in the XML files or the structure of the files. (Other than it's XML.)

In my prototype, I used examples, and knew ahead of time what configuration data was being read. (cheating) …. I created java objects as data structures to hold the specific data from the XML files. I was then able to write a nice, pretty generic reusable processing algorithm that matches XML tag names to defined variables in the object and assigns values. Since the XML file could contain more than one instance of the same data structure, I stored as a vector of such java objects.

Now, to do the same thing without knowing the data file structure in advance, I could use reflection to build data objects as the program reads in the XML files.

I don't want to make my question confusing, but I should say that I'm at a point of wondering whether this is a good approach. So, that's kind of my real question – better ideas?

This is part of an open-source project called “High Level Logic” (HLL). The core system has configuration needs for such things as IP and port to communicate with other systems (or within HLL as a way of being loosely coupled). So, there are some things that I can handle by knowing the structure of the data files in advance.

But ultimately, I'd like to allow application programmers to define their own configuration files to extend the system in any way that they'd like, and I'm just trying to make that process as simple as possible for the application developer (and for other reasons that I won't mention just to try to keep this simple); i.e. they would not need to predefine data structures in their application programs. They could add as many application specific configuration files as they'd like. Unlike my program, the application programmers that create the XML files would know what they contain and would fetch data using a core method that takes search tree information as argument.

(I haven't completely rejected the idea of storing the XML data as Document objects and doing DOM processing to recover information - I'm just guessing that my way provides a faster way to access data when the program is running. Don't know if I want to build this in many ways and test speed though. This project has been spread out over years, and just now I can't remember whether I had other reasons to avoid this approach when I created the prototype.)


Correlation does not prove causality.
Ernest Friedman-Hill
author and iconoclast
Marshal

Joined: Jul 08, 2003
Posts: 24184
    
  34

There are so many existing libraries in the automatic XML parsing/serializing/deserializing space that it would behoove you to look closely at what exists before deciding to create something new. But in any case, I think you've described a complex problem which, as posed, really has no satisfactory solution. Consider this: if the user provides an XML file and your system generates classes dynamically, how would the user's Java code compile against those classes? Now if the XML file is changed, doesn't this force the dynamic classes to change, and therefore require the user's Java code to be recompiled?

Before setting out to write anything, or even to adopt an existing solution, you need to sit down and consider your requirements in enough detail that you can answer these questions and any others that come up.


[Jess in Action][AskingGoodQuestions]
Roger F. Gay
Ranch Hand

Joined: Feb 16, 2007
Posts: 376
Ernest Friedman-Hill wrote:There are so many existing libraries in the automatic XML parsing/serializing/deserializing space that it would behoove you to look closely at what exists before deciding to create something new. But in any case, I think you've described a complex problem which, as posed, really has no satisfactory solution. Consider this: if the user provides an XML file and your system generates classes dynamically, how would the user's Java code compile against those classes? Now if the XML file is changed, doesn't this force the dynamic classes to change, and therefore require the user's Java code to be recompiled?

Before setting out to write anything, or even to adopt an existing solution, you need to sit down and consider your requirements in enough detail that you can answer these questions and any others that come up.


I have a tendency to think of more things I should have said after posting - and edit my post - a long term personality problem. You responded so quickly, that I got caught. Above, I've added a couple of lines about just sticking with XML processing. That indeed, may be the best way to go.

In considering the changes in classes to match changes in configurations, my current thinking is that the process of dynamically creating the classes would continue always - every time the program is run - during the development process. Then application developers could turn it off once they know that the configuration is stable - or not. As long as this process only applies to initialization / configuration - it just means seconds at program start up. That's something I can live with.

Alternatively, I've been seriously considering writing a custom class loader that would check to see if the objects are available and up-to-date. I know that seems complicated, but I'm probably going to need a custom class loader anyway, for the part that I said I wasn't going to mention above - just trying to keep the question simple.

Roger F. Gay
Ranch Hand

Joined: Feb 16, 2007
Posts: 376
I should probably say I'm half new to Java. I've been doing it for a while now but I'm really an old C programmer who keeps wanting to use struct {}. I think that might have been the greatest driving force behind creating Java objects as data structure (objects).
Roger F. Gay
Ranch Hand

Joined: Feb 16, 2007
Posts: 376
On the other hand, maybe I should say something about the other reason. Now it's starting to seem more confusing not to tell; like I'm being stubborn for a secret reason.

XML files will also be used to hold plans and rule-bases. Both plans and rule-bases can contain references to other resources such as Java classes (and other things). I have to have some way to check to see if these resources are available and then to invoke them as needed (which I think will take a custom class loader).

So, partly out of a love of light-weight software, I'm sort of looking for one-size-fits-all approaches. Assuming my built in plan implementation and rule-processing is simple (otherwise I'll probably use JBoss Rules), I've been hoping that I'll be able to build directly on the XML data input processes that I use for configuration.
 
Don't get me started about those stupid light bulbs.
 
subject: Reading / Storing Data Without Knowing What it Is