I am debugging some issue in a transformation. When I run it in Spoon GUI, it fails because one step is getting different amount of fields from 2 input streams. This happens on both my Test and Production machine. When I run the exact same transformation from the command line with Pan, it runs fine. Also the job that ran last night had this transformation and there it also ran fine. Any idea what is wrong there?
↧
Transformation runs fine from Windows command line or scheduled job, fails from GUI
↧
Development: java vs spoon
I wondered if anyone with software developer background had some thoughts about using java (pure code) vs spoon. And also the possibility to use kettle library in java code instead of spoon.
In my point of view, spoon is some times better, specially for easier tasks and it gives quick 'big picture' understanding of what is happening. It is also fairly fast to get things done, specially connections to other resources (databases, web and more) works without problems (usually).
Spoon really is an abstraction layer above the programming language. All class structures are already finished and hidden from the user (inheritance, interfaces etc). What I miss in spoon is handling data in more detail, specially between jobs and transformations. While spoon can handle single value arguments and a single list (result rows), java can handle any number of arguments (multiple result rows, lists as arguments etc). Java also has strict types so you will hardly be in a situation where a string is confused with an int, while spoon is more of a scripting language without this safety (except between steps in a transformation). I also miss the opportunity to use basic object oriented principles, instead having to re-use code with the help of arguments.
A huge bonus in spoon is the ability to use custom java code in steps. This works well for smaller pieces of code, but the text editor in spoon for java code is nothing compared to a fully java development environment (eclipse etc). If you want a larger java code base in spoon, you could of course import it as a library, but then you would need to edit that separately (outside spoon).
Do you have any thoughts, maybe tips to better fill the gaps between spoon and java programming? Have you tried using the kettle library directly in java code?
In my point of view, spoon is some times better, specially for easier tasks and it gives quick 'big picture' understanding of what is happening. It is also fairly fast to get things done, specially connections to other resources (databases, web and more) works without problems (usually).
Spoon really is an abstraction layer above the programming language. All class structures are already finished and hidden from the user (inheritance, interfaces etc). What I miss in spoon is handling data in more detail, specially between jobs and transformations. While spoon can handle single value arguments and a single list (result rows), java can handle any number of arguments (multiple result rows, lists as arguments etc). Java also has strict types so you will hardly be in a situation where a string is confused with an int, while spoon is more of a scripting language without this safety (except between steps in a transformation). I also miss the opportunity to use basic object oriented principles, instead having to re-use code with the help of arguments.
A huge bonus in spoon is the ability to use custom java code in steps. This works well for smaller pieces of code, but the text editor in spoon for java code is nothing compared to a fully java development environment (eclipse etc). If you want a larger java code base in spoon, you could of course import it as a library, but then you would need to edit that separately (outside spoon).
Do you have any thoughts, maybe tips to better fill the gaps between spoon and java programming? Have you tried using the kettle library directly in java code?
↧
↧
Dimension Attributes
how to add dimensions attribute in Mondrian without hierarchy.
I want to filter on a specific column of dimension attribute.
Dim> User
Hierarchy
id
name
.
.
.
.
isdeleted
I want to get records of all users which are not deleted in dimension and isdeleted flag in the Hierarchy
I want to filter on a specific column of dimension attribute.
Dim> User
Hierarchy
id
name
.
.
.
.
isdeleted
I want to get records of all users which are not deleted in dimension and isdeleted flag in the Hierarchy
↧
Is there an upgrade process from 6.1 to 8x repository?
I'm in the process of migrating our Pentaho 6.1 environment to 8x. I use kettle and spoon and have the kettle repository in a database (Oracle). I could have sworn when you tried to open a repository that was of an earlier version than your current environment you'd be prompted to upgrade which in turn would issue some ddl on the backend db.
↧
IBM Websphere MQ Consumer
Hi,
We are having messages in IBM Websphere MQ messages. The format is XML.
Message Format:
<NS1:MW1_005B_3_CBO xmlns:NS1="http://sharedresources.cbo" xmlns:shd="http://sharedresources.shd" xmlns:ems="http://sharedresources.ems">
<shd:UnitInfo>
<shd:UnitId>24156</shd:UnitId>
<shd:UnitName>Gowanus5</shd:UnitName>
</NS1:MW1_005B_3_CBO>
When we are trying to consume the message using the IBM Websphere MQ consumer node getting the below error:
Error:
java.lang.ClassCastException: com.ibm.jms.JMSBytesMessage cannot be cast to javax.jms.TextMessage
But when the message format is mentioned below. Successfully consuming the messages. How can we consume message when there is namespace
<MW1_005B_3_CBO >
<UnitInfo>
<UnitId>24156</UnitId>
<UnitName>Gowanus5</UnitName>
</MW1_005B_3_CBO>
Please provide some inputs
We are having messages in IBM Websphere MQ messages. The format is XML.
Message Format:
<NS1:MW1_005B_3_CBO xmlns:NS1="http://sharedresources.cbo" xmlns:shd="http://sharedresources.shd" xmlns:ems="http://sharedresources.ems">
<shd:UnitInfo>
<shd:UnitId>24156</shd:UnitId>
<shd:UnitName>Gowanus5</shd:UnitName>
</NS1:MW1_005B_3_CBO>
When we are trying to consume the message using the IBM Websphere MQ consumer node getting the below error:
Error:
java.lang.ClassCastException: com.ibm.jms.JMSBytesMessage cannot be cast to javax.jms.TextMessage
But when the message format is mentioned below. Successfully consuming the messages. How can we consume message when there is namespace
<MW1_005B_3_CBO >
<UnitInfo>
<UnitId>24156</UnitId>
<UnitName>Gowanus5</UnitName>
</MW1_005B_3_CBO>
Please provide some inputs
↧
↧
Is there an upgrade process from data integration 3.2 to 7x repository?
I am trying to use the same database repository created for data integration 3.2 in 7.1 CE version, but unable to do so.
Can anyone please tell me if I need to do any upgrade process before using the same repository in 7.1 CE.
Can anyone please tell me if I need to do any upgrade process before using the same repository in 7.1 CE.
↧
Table output no error but also no Written
Hello all,
Sometimes I have a for me strange error/issue.
I happens with a CSV input and SQL/database table output.
I'm having this error on PDI 6 through 8.
At the preview of the CSV input I get no error and the first N rows are showing just fine.
But then at the output step PDI says x rows Read and 0 rows Written. And when all the rows are read the green "all is ok" sign appears above the step...
Is there a way how I can find out why there are no rows written?
Normally when a delimiter is wrong in a CSV, PDI gives a hard error and stops the transformation.
I cannot send the CSV for testing purposes, because it contains highly sensitive data.
Sometimes I have a for me strange error/issue.
I happens with a CSV input and SQL/database table output.
I'm having this error on PDI 6 through 8.
At the preview of the CSV input I get no error and the first N rows are showing just fine.
But then at the output step PDI says x rows Read and 0 rows Written. And when all the rows are read the green "all is ok" sign appears above the step...
Is there a way how I can find out why there are no rows written?
Normally when a delimiter is wrong in a CSV, PDI gives a hard error and stops the transformation.
I cannot send the CSV for testing purposes, because it contains highly sensitive data.
↧
Dynamically add Date time in subject and attachment name in pentaho report scheduling
Dear Team,
While scheduling pentaho reports from pentaho user console, Would like to set date time dynamically in subject and attachment names, also would like to set parameterization in receiptants.
kindly help me on this
While scheduling pentaho reports from pentaho user console, Would like to set date time dynamically in subject and attachment names, also would like to set parameterization in receiptants.
kindly help me on this
↧
How to Compare pdf file and text file in 2 different folder using PDI?
As stated in the title, I need to compare pdf and text file to ensure both file have the same name before ftp it.
I try using the file compare in job step, but it isn't suitable for my project, as i generate the text file from the db and the filename of the textfile and pdf is depend on the field in the table. The reason I'm doing this is because i want to ensure that during the ftp process, the text and pdf file will be a pair.
I try using the file compare in job step, but it isn't suitable for my project, as i generate the text file from the db and the filename of the textfile and pdf is depend on the field in the table. The reason I'm doing this is because i want to ensure that during the ftp process, the text and pdf file will be a pair.
↧
↧
Fileds to Column Names in PRD
Hello All,
I have data as follows
Now I want to provide the feature as that
User can see all the tagnames available.
This tagnems will be fetched from Table1.
He will select multuiple tags and then will select time range friom filter .Example
Data for Tag1 and Tag4 time range
After this query will be passed to my database with tag names and time range.
It should give the result shown as above from Table2.
So all the available data will be displayed to user.
Note
1. I want this to be done on run time.
2. I am getting full tag list from table1 and selected tags list & respective data,timestamp from table 2.
Sequence would be
1. Fetching all tag name from Tabl1
2. Selecting desired tag names (may be using filter)
3. Passing time range (May be using filter)
4. It will query to database
5.Getting result as shown in above table.(from table2)
The result should look like below
Kindly Help
Thanks
I have data as follows
| Tag Names |
VALUE | TIMESTAMP | ||
| Tag1 | 123.23 |
|
||
| Tag2 | 12.12 |
|
||
| Tag3 | 45.25 |
|
||
| Tag4 | 69.89 |
|
||
| Tag1 | 25.25 |
|
||
| Tag2 | 25.25 |
|
||
| Tag3 | 87.65 |
|
||
| Tag4 | 25.36 |
|
||
| Tag1 | 89.36 |
|
||
| Tag2 | 47.36 |
|
||
| Tag3 | 235.36 |
|
||
| Tag4 | 89.36 |
|
Now I want to provide the feature as that
User can see all the tagnames available.
This tagnems will be fetched from Table1.
He will select multuiple tags and then will select time range friom filter .Example
Data for Tag1 and Tag4 time range
| "2017-09-06 20:00:00" to
|
After this query will be passed to my database with tag names and time range.
It should give the result shown as above from Table2.
So all the available data will be displayed to user.
Note
1. I want this to be done on run time.
2. I am getting full tag list from table1 and selected tags list & respective data,timestamp from table 2.
Sequence would be
1. Fetching all tag name from Tabl1
2. Selecting desired tag names (may be using filter)
3. Passing time range (May be using filter)
4. It will query to database
5.Getting result as shown in above table.(from table2)
The result should look like below
| TAG1 | TAG2 | TAG3 | |
| "2017-09-06 21:30:00" | 12 | 45.24 | 12.36 |
| "2017-09-06 21:30:00" | 12.22 | 85.58 | 87.36 |
| "2017-09-06 21:30:00" | 45.22 | 25.366 | 52.25 |
Kindly Help
Thanks
↧
Pentaho user Console Mail Server setting from Command Prompt
Dear Team,
Is there any way to schedule the Pentaho reports from command prompt or setting the mail server which is presented in pentaho user console from command prompt?
Also, Is there any way to take the Pentaho BA Application report files and folders in the Linux box, I need to view my designed reports in Linux box?
Kindly advise.
Is there any way to schedule the Pentaho reports from command prompt or setting the mail server which is presented in pentaho user console from command prompt?
Also, Is there any way to take the Pentaho BA Application report files and folders in the Linux box, I need to view my designed reports in Linux box?
Kindly advise.
↧
Problemas con el repositorio
Buen día, tengo un problema con el repositorio, lo tengo alojado en una bd postgres, hoy reinicie el kettle y al momento de conectarme al repositorio los trabajos y transformaciones aparecen con fecha de hace una semana y los job que realice ayer por ejemplo no aparecen en el listado para abrir los jobs y trans guardadas, sin embargo en las tablas si esta el registro.
Cree un directorio llamado "BUC" y renombre un job de "cliente_job" a "job_clientes_principal" y al consultar en la base de datos del repositorio aparecen correctamente pero en al ejecutar el kettle no están ninguno de los cambios realizados y la fecha de actualización general es de hace 10 días
![Name: Screenshot_2.png
Views: 29
Size: 14.1 KB]()
![Name: Screenshot_3.jpg
Views: 25
Size: 22.9 KB]()
![Name: Screenshot_4.jpg
Views: 26
Size: 24.1 KB]()
![Name: Screenshot_6.jpg
Views: 26
Size: 24.3 KB]()
Cree un directorio llamado "BUC" y renombre un job de "cliente_job" a "job_clientes_principal" y al consultar en la base de datos del repositorio aparecen correctamente pero en al ejecutar el kettle no están ninguno de los cambios realizados y la fecha de actualización general es de hace 10 días
↧
Setting global variable for 2 master main reports in pentaho report designer
Dear Team,
Kindly help.
I need to set one global parameter for 2 main master reports. these 2 reports are linking each other means separate reports. rather creating 2 parameters for each reports , set one parameter and set it to 2 reports.
Kindly suggest any ideas about.
Thanks
M. Vijay Antony
Kindly help.
I need to set one global parameter for 2 main master reports. these 2 reports are linking each other means separate reports. rather creating 2 parameters for each reports , set one parameter and set it to 2 reports.
Kindly suggest any ideas about.
Thanks
M. Vijay Antony
↧
↧
Configuring Pentaho BA Application in Linux Box
Dear Team,
Kindly help.
How to configure the Pentaho BA Application in Linux, Like i need to view my Home and Public report files and folders from linux.
Kindly advise.
Kindly help.
How to configure the Pentaho BA Application in Linux, Like i need to view my Home and Public report files and folders from linux.
Kindly advise.
↧
Penatho CE 8.0 nginx config - Login and Logout redirect?
Anyone configured Pentaho 8 behind nginx?
Connecting via a load balancer https 443 (which looks after the certs) through on port 3000 to the default Pentaho 8080.
The Virtual host is set up to listen to 3000
location /pentaho/ {
proxy_pass http://<IP address>:8080/pentaho/;
proxy_http_version 1.1;
proxy_intercept_errors on;
}
So I can hit https://<external_IP>:443/penatho/
and I get a redirect to http://<internal_IP>>:3000/Login
(I can see this internally on the rproxy server as a wget on localhost:3000/pentaho get a 302 to :3000/pentaho/Login)
I can work around this....
Going direct to https://<external_IP>:443/penatho/Login
allows me to login ( the proxy_hhtp_version and proxy_interscept_errors seem to fix that)
But then I'm not able to Logout (which is another redriect)
Any ideas?
Connecting via a load balancer https 443 (which looks after the certs) through on port 3000 to the default Pentaho 8080.
The Virtual host is set up to listen to 3000
location /pentaho/ {
proxy_pass http://<IP address>:8080/pentaho/;
proxy_http_version 1.1;
proxy_intercept_errors on;
}
So I can hit https://<external_IP>:443/penatho/
and I get a redirect to http://<internal_IP>>:3000/Login
(I can see this internally on the rproxy server as a wget on localhost:3000/pentaho get a 302 to :3000/pentaho/Login)
I can work around this....
Going direct to https://<external_IP>:443/penatho/Login
allows me to login ( the proxy_hhtp_version and proxy_interscept_errors seem to fix that)
But then I'm not able to Logout (which is another redriect)
Any ideas?
↧
Replace line break from .txt
Hi!
I am trying to replace a line break (enter) that is already in a .txt.
Line one represents the wrong way (with line break in ProdName) and line two represents the way it should stay after substitution.
Remembering that this data already comer from the customer through an extraction procedure where it would be much easier to put a replace in the select, but you know how customers are. :rolleyes:
Someone can help? Thanks!
![Name: Capture.jpg
Views: 44
Size: 12.8 KB]()
I am trying to replace a line break (enter) that is already in a .txt.
Line one represents the wrong way (with line break in ProdName) and line two represents the way it should stay after substitution.
Remembering that this data already comer from the customer through an extraction procedure where it would be much easier to put a replace in the select, but you know how customers are. :rolleyes:
Someone can help? Thanks!
↧
metadata injection with sort
Array index out of exception :0 exception when i use with metadata injector step
↧
↧
Deleted public folder
Hi,
I have been playing around a bit with BI 8 and trying out some things.
You start out with a Home and Public folder.
Now I deleted the Public folder.
But there does not seem to be a way to recreate it?
As far as I can tell you can only create folder in Home and Public but not in the root.
So once you delete Public, there is no way to get it back?
I have been playing around a bit with BI 8 and trying out some things.
You start out with a Home and Public folder.
Now I deleted the Public folder.
But there does not seem to be a way to recreate it?
As far as I can tell you can only create folder in Home and Public but not in the root.
So once you delete Public, there is no way to get it back?
↧
What would it take to get a true step blocker in a transformation?
I run into this all the time. I want to read some files. Do magic with the data in them . Then when all that is done. Do something with the files/move/delete/rename. The trouble is spoon does not have any tools that work inside a transformation that can easily do this. "Blocking step" and "Block this step until steps finish" don't really block anything. It just seems silly to me, that to do such a simple thing I have to create a job and 2 transformations. I understand the parallel nature of the tool BUT there really should be a way to be able to say ..hey do all these steps until complete...then do this series of steps i.e. a blocking step. Or, failing that possibly a way to 'job' inside of a transform from one portion to another? I mean all of it, not just a row at a time blocking.
And this isn't just files either. I have many times run into a situation where i need one thing to finish before starting another and for various reasons don't want to do a separate transform/job or lose speed by single treading.
As an aside, i have had 'block step until all finish' work for what I'm taking about but that was 2 versions ago at least and i am really not sure how the heck i did get it to work, possibly single threading the whole transform ?
Attached is a simple non working transform of what I mean, not that this hasn't been brought up before:)
And this isn't just files either. I have many times run into a situation where i need one thing to finish before starting another and for various reasons don't want to do a separate transform/job or lose speed by single treading.
As an aside, i have had 'block step until all finish' work for what I'm taking about but that was 2 versions ago at least and i am really not sure how the heck i did get it to work, possibly single threading the whole transform ?
Attached is a simple non working transform of what I mean, not that this hasn't been brought up before:)
↧
Bug in pentaho data integration 8
In text file input if you try to use pipe separated files Folder it does not get parsed. If you do the same in pentaho 7 it works.
![Name: 1.jpg
Views: 40
Size: 13.3 KB]()
![Name: 2.jpg
Views: 35
Size: 17.9 KB]()
![Name: 3.jpg
Views: 36
Size: 22.5 KB]()
↧