Batch updation failure because of duplicate records
Joined: Apr 23, 2008
I'm trying to upload the records from a CSV file into the database. On the process I'm trying to validate those records extracted from the CSV file. Because of performance issue, I thought of using batch update which will return me an array of successfully loaded index. But even if there's a duplication of one record, the entire batch is failing and the records are not inserted into the database. Am I not supposed to use batch update for such cases? Then how do I resolve the performance and cost issue as I dont want to hit the database for each and every insertion of record.
NOTE: I'm not supposed to use LOAD DATA INFILE and use only INSERT INTO command.
Music is all around us. All you have to do is just LISTEN to it,
Joined: Jun 21, 2008
Load data into temporary table, create index(es) on this temp table (to make query faster),
then execute query to see if duplicates exist and remove duplicates,
then update the table.