Kettle appears to be running a record count for "Dimension Lookup / Update" steps during initialization - even though the technical keys are to "Use auto increment field". For example, the "Target table" is set to "d_url" and the query "SELECT count(*) FROM d_url" is being run during step initialization. This is causing an issue because we have InnoDB dimension tables with 30M+ records and this query is causing our transformations to hang for an extended period of time. Is there a way to avoid this query or perform some type of an optimization?
↧