Hi There,
So I just got Pentaho Spoon today, and the main reason I'm using it is to copy recurring folders that come in everyday. These folders are named by date, so an example would be : C:\Data\2016\07\19. Then I would want to copy that file to : D:\Data\2016\07\ . All I need is for Spoon to automatically create the folder (19) and then just copy over all the files.
So far I have the copying set up like so:
SOURCE : file:///C:/Data/2016/07
DESTINATION : file:///D:/Data/2016/07
The settings I have ticked off are: "include subfolders" ; "copy empty folders" ; "create destination folder"
With this, it is essentially having to go through all of what has already been copied and then skip it and then go to what hasn't been copied. For example, today on the 19th, I see in the logs it says that it is copying individual files to [] (for the files that I know have already been copied, and I'm guessing it means that it is copying nowhere and therefore skipping), but it also copies the files that I want and does that well. You may be asking yourself what is my issue then right? Well on the whole month, one of the "Data"s will only fill up 50gb on one drive, where as for "Data2", it'll fill up to approximately 1TB every month, and you can imagine that already on the 19th, there is still a hefty amount of data that Spoon needs to skip over.
Is there a way to have Spoon somehow skip over the files quicker? or do I have to suck it up and wait? The reason in the first place why I wanted to use this software is because everything would be automatic, but for me to manually go to the location of the data and copy it where I want is much faster, yet it is manual.
Thank you for any suggestions,
Julian
So I just got Pentaho Spoon today, and the main reason I'm using it is to copy recurring folders that come in everyday. These folders are named by date, so an example would be : C:\Data\2016\07\19. Then I would want to copy that file to : D:\Data\2016\07\ . All I need is for Spoon to automatically create the folder (19) and then just copy over all the files.
So far I have the copying set up like so:
SOURCE : file:///C:/Data/2016/07
DESTINATION : file:///D:/Data/2016/07
The settings I have ticked off are: "include subfolders" ; "copy empty folders" ; "create destination folder"
With this, it is essentially having to go through all of what has already been copied and then skip it and then go to what hasn't been copied. For example, today on the 19th, I see in the logs it says that it is copying individual files to [] (for the files that I know have already been copied, and I'm guessing it means that it is copying nowhere and therefore skipping), but it also copies the files that I want and does that well. You may be asking yourself what is my issue then right? Well on the whole month, one of the "Data"s will only fill up 50gb on one drive, where as for "Data2", it'll fill up to approximately 1TB every month, and you can imagine that already on the 19th, there is still a hefty amount of data that Spoon needs to skip over.
Is there a way to have Spoon somehow skip over the files quicker? or do I have to suck it up and wait? The reason in the first place why I wanted to use this software is because everything would be automatic, but for me to manually go to the location of the data and copy it where I want is much faster, yet it is manual.
Thank you for any suggestions,
Julian