Ulf Dittmer wrote:Using the "awk" language and tool, extracting the values from the CSV file is probably no big deal, but how are you going to access a DB from a shell script?
Usually via its command-line client program (SQL*PLUS, isql, pgsql, mysql, db2, etc.)
However, I recommend looking at the Pentaho Kettle ETL utility. It's open-source Java (parts of which I've improved myself), can read and write a Panamax-load of formats, allows for logic, calculations and transformations, is extensible, and commercial support is available. I'm probably going to be installing a copy for a client this morning, in fact, going in the other direction (DB2-to-CSV).
Customer surveys are for companies who didn't pay proper attention to begin with.
I do recommend using something like Kettle (Pentaho DI) for stuff like this where there's a lot of data. The overhead for starting up, running, and shutting down a database utility program on a per-row basis can be considerable. And while you can just script-and-pipe the whole file into a single invocation of the database utility, Kettle affords a more elegant (and more general) solution. Plus, it has other useful features such as the ability to run multiple transforms in parallel and works on a general basis using JDBC drivers.
Co-incidentally, a book on Pentaho DI has just come out, and it's being discussed in the JDBC Forum.