Hi,
I have several table input steps running in parallel, which are connected to same SQL Server database and writing it to separate flat files inside the same folder.The configuration details of database and the text file are coming from variables. My problem is , If 2 flows Table input ---> TFO starts at the same time, 2 queries are being fired to two different tables of say 1000k records in each, but time for reading from the database and usage of buffer memory to write the flat file differs. In simple, bot process starts at same time , but there is a variance in Table input 1 reads from DB table1 to Table input 2 reads from DB table2 (approx 2-5 mins) and it effects in writing to flat file(time consuming).
How to make them run parallel and fast ?
I have several table input steps running in parallel, which are connected to same SQL Server database and writing it to separate flat files inside the same folder.The configuration details of database and the text file are coming from variables. My problem is , If 2 flows Table input ---> TFO starts at the same time, 2 queries are being fired to two different tables of say 1000k records in each, but time for reading from the database and usage of buffer memory to write the flat file differs. In simple, bot process starts at same time , but there is a variance in Table input 1 reads from DB table1 to Table input 2 reads from DB table2 (approx 2-5 mins) and it effects in writing to flat file(time consuming).
How to make them run parallel and fast ?