how to convert .csv files into sql server files using java

[minal silimkar]

I have .csv comma seperated file. I want to convert the file in sql server 2000 using java. How will I do it?

[Ulf Dittmer]

What do you mean by "convert into sql server file" - import it into SQL Server? Or is there some special data format that SQL Server uses for importing/exporting data?

If the former, which part is giving you trouble - reading a CSV file, or writing JDBC code to insert the data into the database?

[minal silimkar]

I want to import csv file into sql server. How will I do that?

[Ulf Dittmer]

OK. Since you posted in the JDBC forum I assume you've got the part where you read the CSV file solved. Now you need to connect to the DB, and feed it INSERT statements. What do you have so far?

[minal silimkar]

I created dsn for csv file, using Microsoft Excel Driver.
I can read data in java program using following code.
I want to import data from csv file to oracle sql or sql server.
How will I do that?

    
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.ResultSet;
 
public class CsvEx{
    public static void main( String [] args ){
        Connection con = null;
        Statement stmnt = null;
        try{
        Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
        con=DriverManager.getConnection("jdbc:odbc:csvdsn","","");
           
            stmnt = con.createStatement();
            
            String query = "select * from [sheet1$] ";
           
            ResultSet rs = stmnt.executeQuery( query );           
            
            while( rs.next() ){
                System.out.println( rs.getString(2) );
            }
        }
        catch( Exception e ){
            e.printStackTrace();
        }
        finally{
            try{
                stmnt.close();
                con.close();
            }catch( Exception e ){
                e.printStackTrace();
            }
        }
    }
}

[Ulf Dittmer]

Are you familiar with the SQL INSERT statement, and how you can use JDBC to issue INSERT statements to a database?

[Campbell Ritchie]

You don't even need to connect to the database with JDBC; you can go through the csv file line by line, and using String.split(",") get it into a String[] array.

Using something like this

fileFormatter.format("INSERT INTO my_table (name, rank, number) VALUES ('%s', '%s', %s);%n",
   splitString[3], splitString[4], splitString[5]);

you can make up an SQL script, then run the SQL script from inside the database.
You just need to know what each of the different tokens means. If any of the original tokens contains a comma, you are up the creek without a paddle.
[ August 28, 2008: Message edited by: Campbell Ritchie ]

[minal silimkar]

Originally posted by Ulf Dittmer
Are you familiar with the SQL INSERT statement, and how you can use JDBC to issue INSERT statements to a database?

I tried to insert data using sql query. But it is not inserting first record from .csv file. rest of the records are getting stored in the data base.

    
public void readData(){
Statement st;
try{
st = con.createStatement();
ResultSet rs = st.executeQuery("select * from [sheet1$] ");
            while( rs.next() ){
            insertData(rs.getInt(1),rs.getString(2));
            }
}catch(SQLException e){
e.printStackTrace();
}
}

public void insertData(int no, String name){
PreparedStatement pst;
try {
pst=con1.prepareStatement("insert into csvdata (no, name) values (?,?)");
pst.setInt(1,no);
pst.setString(2, name);
pst.executeUpdate();
System.out.println("Data successfully inserted...");
} catch (SQLException e) {
e.printStackTrace();
}
}

I know how to insert data in the table using SQL statement.
I heard about using XML it is directly possible to pass data from .csv to oracle DB. Please guide me.
[ August 28, 2008: Message edited by: minal silimkar ]

[Ulf Dittmer]

Is the first row even read from the CSV file? It's possible that the driver treats the first row as column headers, and thus ignores it.

[minal silimkar]

Originally posted by Ulf Dittmer

Is the first row even read from the CSV file? It's possible that the driver treats the first row as column headers, and thus ignores it.

I am reading the whole data from .csv file. As Ulf mentioned, is it possible that the driver treats the first row as colomn header, and thus ignores it.

Thanks Ulf and Campbell Ritchie for your guidence.

[Campbell Ritchie]

Originally posted by minal silimkar:
Thanks Ulf and Campbell Ritchie for your guidence.

You're welcome
[ August 29, 2008: Message edited by: Campbell Ritchie ]

[Nishanth Sivaraman]

This can be done in a long way, but you can get first line also.
I created an application like this once, what i done is:

please note i done in lengthy way, but i got result.

1: process file : in this step i replaced all ; or other field terminator like '\t' with ';DUMMY' or '\t DUMMY' so that i will get all tokens, i will replace this 'DUMMY' later.

public void processFile(String actFile,String genFile)throws Exception{
//try{
String lineString="";
int tkCount;
BufferedReader br = new BufferedReader(new FileReader(actFile));
System.out.println("Modification Started");
for (String line = br.readLine(); line != null; line = br.readLine()) {
// try {
lineString=lineString + line.replaceAll("\t", "\t DUMMY");
//lineString=lineString + line;
StringTokenizer tk=new StringTokenizer(lineString,"\t");
tkCount=tk.countTokens();
if(tkCount==58){
// write to file
//put line break
writeContent(genFile,lineString.replaceAll("\r\n", "<br>").replaceAll("\n", "<br>").replaceAll("\t DUMMY","\t"));
writeContent(genFile,"\r\n");
lineString="";
tk=null;
}
else{
//do nothing
System.out.println(lineString);
System.out.println("Token Count is : " +tkCount);
}
//} catch (Exception e) {
// e.printStackTrace();
//}
}
br.close();
System.out.println("Modification Completed");
//}catch(Exception e){
//e.printStackTrace();
//}

}



public static void writeContent(String fileName, String contentText)throws Exception{ // Write File Content
//try {
BufferedWriter out = new BufferedWriter(new FileWriter(fileName, true));
out.write(contentText);
out.close();
//} catch (IOException e) {
//e.printStackTrace();
//}
}

}


2: Read the file: In this step read the file to a linkedhashmap

import java.io.*;
import java.sql.*;
import java.util.*;
public class ReadFile {

StringTokenizer tokenizer;
LinkedHashMap map;
MapHandler mh;
public ReadFile()throws Exception{
mh=new MapHandler();// to handle map
}

public void readFile(String fileName)throws Exception{ //Read The file
//try{
BufferedReader in=new BufferedReader(new FileReader(fileName));
map=new LinkedHashMap();
String line=in.readLine();
System.out.println(line);


while((line=in.readLine())!=null){


System.out.println(line);
line=line.replaceAll("\t", "\t DUMMY"); //Work around to get all tokens

tokenizer=new StringTokenizer(line,"\t");

System.out.println("Beginning Map");
map.put("value1", tokenizer.nextToken());
map.put("value2", tokenizer.nextToken());

System.out.println("End of Map");

mh.processMap(map);
map.clear();

}

}
}

3: create a maphandler which takes values from map, and insert to db, here use prepared statement.


public void processMap(LinkedHashMap map)throws Exception{
System.out.println("Process map");


value1=map.get("value1").toString().replaceAll(" DUMMY", "");

value1=map.get("value1").toString().replaceAll(" DUMMY", "");

\\create prepstatemet here and insert to db

}


I am happy if this solves your problem.

Thanks,

Nishanth.S

[Ulf Dittmer]

Nishanth, the code you posted is unnecessarily hard to read, because you don't UseCodeTags. Please do that from now on.

Looking at it briefly, the CSV reading part will not work correctly for many valid CSV files. CSV parsing isn't as trivial as it looks. There's really no reason to reinvent the wheel, especially since there are a number of CSV parsing libraries available.