I am using Kettle GA 5.0.1-stable. I have a transformation with several sub-transformations. Each of these transformations is set to use "Unique Connections" by checking the "Make the transformation database transactional" under the Transformation Settings dialog. I am connecting to MySQL 5.0.41 database using the jdbc connector version 5.1.28. And I am using JNDI jdbc.properties to specify my connection properties. When everything goes well, the transformation succeeds and commits to the database at the end, as expected. But once in a while (on an average 1 in 5 times), the transformation fails for the following reasons:
1) Foreign key constraint violation
2) Database deadlock during INSERT/UPDATE
After spending several days of debugging at the MySQL level (by enabling general query logging) and configuring log4jdbc-log4j2 in Kettle, I have found that whenever one of the above two issues happen, Kettle has *surely* got the *wrong* database connection. I can see that the step that fails for the above 2 mentioned issues is using a different connection instance for the same JNDI, than the rest of the steps having the same JNDI configuration. I was expecting that, by using unique connections setting, there will be one and only one connection used by the entire transformation. I don't know how Kettle got the wrong connection for a step and that too only sometimes, though all my transformation and settings are correctly using the same JNDI name for connecting to the database.
Any suggestions on where to look for resolving this issue?
1) Foreign key constraint violation
2) Database deadlock during INSERT/UPDATE
After spending several days of debugging at the MySQL level (by enabling general query logging) and configuring log4jdbc-log4j2 in Kettle, I have found that whenever one of the above two issues happen, Kettle has *surely* got the *wrong* database connection. I can see that the step that fails for the above 2 mentioned issues is using a different connection instance for the same JNDI, than the rest of the steps having the same JNDI configuration. I was expecting that, by using unique connections setting, there will be one and only one connection used by the entire transformation. I don't know how Kettle got the wrong connection for a step and that too only sometimes, though all my transformation and settings are correctly using the same JNDI name for connecting to the database.
Any suggestions on where to look for resolving this issue?