GeeCON Prague 2014*
The moose likes JDBC and the fly likes how to convert .csv files into sql server files using java Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


JavaRanch » Java Forums » Databases » JDBC
Bookmark "how to convert .csv files into sql server files using java" Watch "how to convert .csv files into sql server files using java" New topic
Author

how to convert .csv files into sql server files using java

minal silimkar
Ranch Hand

Joined: Nov 25, 2007
Posts: 133
I have .csv comma seperated file. I want to convert the file in sql server 2000 using java. How will I do it?


Minal Silimkar
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42061
    
  64
What do you mean by "convert into sql server file" - import it into SQL Server? Or is there some special data format that SQL Server uses for importing/exporting data?

If the former, which part is giving you trouble - reading a CSV file, or writing JDBC code to insert the data into the database?


Ping & DNS - my free Android networking tools app
minal silimkar
Ranch Hand

Joined: Nov 25, 2007
Posts: 133
I want to import csv file into sql server. How will I do that?
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42061
    
  64
OK. Since you posted in the JDBC forum I assume you've got the part where you read the CSV file solved. Now you need to connect to the DB, and feed it INSERT statements. What do you have so far?
minal silimkar
Ranch Hand

Joined: Nov 25, 2007
Posts: 133
I created dsn for csv file, using Microsoft Excel Driver.
I can read data in java program using following code.
I want to import data from csv file to oracle sql or sql server.
How will I do that?


Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42061
    
  64
Are you familiar with the SQL INSERT statement, and how you can use JDBC to issue INSERT statements to a database?
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 39102
    
  23
You don't even need to connect to the database with JDBC; you can go through the csv file line by line, and using String.split(",") get it into a String[] array.

Using something like thisyou can make up an SQL script, then run the SQL script from inside the database.
You just need to know what each of the different tokens means. If any of the original tokens contains a comma, you are up the creek without a paddle.
[ August 28, 2008: Message edited by: Campbell Ritchie ]
minal silimkar
Ranch Hand

Joined: Nov 25, 2007
Posts: 133

Originally posted by Ulf Dittmer
Are you familiar with the SQL INSERT statement, and how you can use JDBC to issue INSERT statements to a database?


I tried to insert data using sql query. But it is not inserting first record from .csv file. rest of the records are getting stored in the data base.



I know how to insert data in the table using SQL statement.
I heard about using XML it is directly possible to pass data from .csv to oracle DB. Please guide me.
[ August 28, 2008: Message edited by: minal silimkar ]
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42061
    
  64
Is the first row even read from the CSV file? It's possible that the driver treats the first row as column headers, and thus ignores it.
minal silimkar
Ranch Hand

Joined: Nov 25, 2007
Posts: 133
Originally posted by Ulf Dittmer
Is the first row even read from the CSV file? It's possible that the driver treats the first row as column headers, and thus ignores it.

I am reading the whole data from .csv file. As Ulf mentioned, is it possible that the driver treats the first row as colomn header, and thus ignores it.

Thanks Ulf and Campbell Ritchie for your guidence.
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 39102
    
  23
Originally posted by minal silimkar:
Thanks Ulf and Campbell Ritchie for your guidence.
You're welcome
[ August 29, 2008: Message edited by: Campbell Ritchie ]
Nishanth Sivaraman
Greenhorn

Joined: Aug 12, 2008
Posts: 7
This can be done in a long way, but you can get first line also.
I created an application like this once, what i done is:

please note i done in lengthy way, but i got result.

1: process file : in this step i replaced all ; or other field terminator like '\t' with ';DUMMY' or '\t DUMMY' so that i will get all tokens, i will replace this 'DUMMY' later.

public void processFile(String actFile,String genFile)throws Exception{
//try{
String lineString="";
int tkCount;
BufferedReader br = new BufferedReader(new FileReader(actFile));
System.out.println("Modification Started");
for (String line = br.readLine(); line != null; line = br.readLine()) {
// try {
lineString=lineString + line.replaceAll("\t", "\t DUMMY");
//lineString=lineString + line;
StringTokenizer tk=new StringTokenizer(lineString,"\t");
tkCount=tk.countTokens();
if(tkCount==58){
// write to file
//put line break
writeContent(genFile,lineString.replaceAll("\r\n", "<br>").replaceAll("\n", "<br>").replaceAll("\t DUMMY","\t"));
writeContent(genFile,"\r\n");
lineString="";
tk=null;
}
else{
//do nothing
System.out.println(lineString);
System.out.println("Token Count is : " +tkCount);
}
//} catch (Exception e) {
// e.printStackTrace();
//}
}
br.close();
System.out.println("Modification Completed");
//}catch(Exception e){
//e.printStackTrace();
//}

}



public static void writeContent(String fileName, String contentText)throws Exception{ // Write File Content
//try {
BufferedWriter out = new BufferedWriter(new FileWriter(fileName, true));
out.write(contentText);
out.close();
//} catch (IOException e) {
//e.printStackTrace();
//}
}

}


2: Read the file: In this step read the file to a linkedhashmap

import java.io.*;
import java.sql.*;
import java.util.*;
public class ReadFile {

StringTokenizer tokenizer;
LinkedHashMap map;
MapHandler mh;
public ReadFile()throws Exception{
mh=new MapHandler();// to handle map
}

public void readFile(String fileName)throws Exception{ //Read The file
//try{
BufferedReader in=new BufferedReader(new FileReader(fileName));
map=new LinkedHashMap();
String line=in.readLine();
System.out.println(line);


while((line=in.readLine())!=null){


System.out.println(line);
line=line.replaceAll("\t", "\t DUMMY"); //Work around to get all tokens

tokenizer=new StringTokenizer(line,"\t");

System.out.println("Beginning Map");
map.put("value1", tokenizer.nextToken());
map.put("value2", tokenizer.nextToken());

System.out.println("End of Map");

mh.processMap(map);
map.clear();

}

}
}

3: create a maphandler which takes values from map, and insert to db, here use prepared statement.


public void processMap(LinkedHashMap map)throws Exception{
System.out.println("Process map");


value1=map.get("value1").toString().replaceAll(" DUMMY", "");

value1=map.get("value1").toString().replaceAll(" DUMMY", "");

\\create prepstatemet here and insert to db

}


I am happy if this solves your problem.

Thanks,

Nishanth.S
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 42061
    
  64
Nishanth, the code you posted is unnecessarily hard to read, because you don't UseCodeTags. Please do that from now on.

Looking at it briefly, the CSV reading part will not work correctly for many valid CSV files. CSV parsing isn't as trivial as it looks. There's really no reason to reinvent the wheel, especially since there are a number of CSV parsing libraries available.
 
GeeCON Prague 2014
 
subject: how to convert .csv files into sql server files using java