Update step is performing very slow.

Hi,

I am using Kettle 'pdi-ce-4.4.0-stable'.

I am reading data from mongo db, mssql server and in-memory H2 database.
After computing each minute level data, I am trying to update in-memory H2 database.
I have attached the KTR file.

For each luminaire, I need to update 24*60 ie 1440 records in in-memory H2 database.
For this I am using update step. It is taking about 9-10 minutes to complete Update step. It means 25 seconds for each update. This is too much.
In H2 db, there are 1,440,000 records in total.

I am not sure which is causing performance issue.

http://www.h2database.com/html/performance.html says 406 ms for update.

I have tried giving number of copies to 10, connection pools to 10, Auto Commit true. But still same result. I have tried giving MVCC=TRUE for H2 database. Still same result.

Worst part is I have 6000 luminaires. So it will take 10mins * 6000 = 41 Days... This is not at all acceptable.

Please help me to solve this performance issue.

-Prakash

Attached Files

• 24hr_fill_power_fact.ktr (40.9 KB)