Hi! I'm having a problem with a bulk load from serveral csv files to several MySQL tables. First of all, I'm running Kettle in a Windows environment, so, I can't use Kettle MySQL Bulk Loader step directly. This step actually does not work within a Windows system due to the use of mkfifo for named pipes. So, I'm using "Execute SQL script" step with variable substitution instead.
The SQL script to execute for each file to load is similar to:
set @bh_dataformat = 'mysql';
LOAD DATA infile 'file.csv' INTO TABLE table_mysql CHARACTER SET utf8 FIELDS TERMINATED BY '|' ESCAPED BY '\\' LINES TERMINATED BY '\r\n'
(col1,col2,col3,col4,@var1,@var2)
set
col5=${environment_variable},
col6 = STR_TO_DATE(@var2, '%b %d %Y %l:%i:%s:%f%p');
where ${environment_variable} is a variable already set in a previous step. I use also variable substitution within the step.
When I run just one step like this, everything works fine and data form one csv file is load within the correspondig MySQL table, but when I run a job with several transformations for loading several csv files into several MySQL tables at the sime time, there is an error like this for each file:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '${environment_variable},
col6 = STR_TO_DATE(@var2, '%b %d %Y %l:%i:%s:%f' at line 4
I can't find where the problem is, specially when the syntax for one step alone works fine.
Does anyone have any idea?
Thank you in advance.