Hi all,
My question is how to append each loop result to the same output file while selecting "execute for every input row" option in a sub-job ?
Here is my scenario : some informations (filename, path, rowscount etc...) related to CSV files would be collected and passed to
Pentaho Report Designer to generate reports. for simplicity's sake, the temporary output step is plein text file and the process described as below:
[start]--[set-variable]--[get_File_Names_rows]--[get_information_for_each_file](loop) --[end]
I totally followed the tutorial here, which explains how to retrieve all files in a single "get File Names" step and loop each one to do some stuff in a sub-job.
The only difference is that I want every output row to be written in the same file NOT separated file.
Please find attached the created jobs, I just put an output text file at the last step called "generate_entities_log", but apparently it receives only the last record of the loop.
--------------------------------------------------------------------------------
The alternative I've made so far without success:
1, generate separate file from each loop and try to merge all these files (20 CSV files) together, I've checked "merge join" step, as well as posts in forum but got no idea how to make it.
2, use 'modified javascript value' step to receive all rows from each loop, and then write to output file all at once. If javascript step is put within the loop it receive only each loop record,
while outside the loop it receive only the last loop result.
Any suggestions or technical hints would be appreciated.
Thanks in advance for your help!
PS: Please modify the extension of the attached zip file to .7z, to execute the attached job, you have to defined in kettle.properties file a variable so that Kettle would know where to find the config.properties file, in which data file path are parametrized. the related tutorial is here.
Shaokun
My question is how to append each loop result to the same output file while selecting "execute for every input row" option in a sub-job ?
Here is my scenario : some informations (filename, path, rowscount etc...) related to CSV files would be collected and passed to
Pentaho Report Designer to generate reports. for simplicity's sake, the temporary output step is plein text file and the process described as below:
[start]--[set-variable]--[get_File_Names_rows]--[get_information_for_each_file](loop) --[end]
I totally followed the tutorial here, which explains how to retrieve all files in a single "get File Names" step and loop each one to do some stuff in a sub-job.
The only difference is that I want every output row to be written in the same file NOT separated file.
Please find attached the created jobs, I just put an output text file at the last step called "generate_entities_log", but apparently it receives only the last record of the loop.
--------------------------------------------------------------------------------
The alternative I've made so far without success:
1, generate separate file from each loop and try to merge all these files (20 CSV files) together, I've checked "merge join" step, as well as posts in forum but got no idea how to make it.
2, use 'modified javascript value' step to receive all rows from each loop, and then write to output file all at once. If javascript step is put within the loop it receive only each loop record,
while outside the loop it receive only the last loop result.
Any suggestions or technical hints would be appreciated.
Thanks in advance for your help!
PS: Please modify the extension of the attached zip file to .7z, to execute the attached job, you have to defined in kettle.properties file a variable so that Kettle would know where to find the config.properties file, in which data file path are parametrized. the related tutorial is here.
Shaokun
