Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live
↧

Empty report (Pentaho Reporting Output)

October 24, 2016, 2:37 am
$
0
0
Hi,
I wanted to create a report in Spoon with the "Pentaho Reporting Output" step, but the problem is that the excel is empty. I think the step cann't get the parameters from the report.prpt, but they should be allright.

I searched a solution for this problem, but things like putting the job/transformation in the report.prpt in the report designer or something like that didn't work.

I'm don't have any idea what I should do now to solve the problem, but I need this very soon to work.
I hope somebody can help me.
↧
Search

Password Encryption in BI Server

October 24, 2016, 3:05 am
$
0
0
We have been testing the application in our private environment. Before Going live we decided to perform a security audit. The auditor says the username and password from login page are being sent in plain text over the public internet and so recommends to encrypt them. The username and password can easily be seen by intercepting the request on any proxy tool. They are visible in plain text and can be intercepted by anyone inside the network.

So we are searching for a way where we can encrypt the username and password before sending to server from the client browser. Please help me solve the issue.
↧
↧

Dynamic Column Specification for MDX Query

October 25, 2016, 12:59 am
$
0
0
Dear All

I am trying to create a report that would give me the last 5 weeks worth of production data from our factories. If I create the MDX query manually it looks like this.

Code:
SELECT
NON EMPTY {[Dates.Calendar Financial - Weeks].[Fin Week No].[34]:[Dates.Calendar Financial - Weeks].[Fin Week No].[38]} ON COLUMNS,
{
[Measures].[Export],
[Measures].[Detained],
[Measures].[Discarded],
[Measures].[Private]
} ON ROWS
FROM [Production Cube]
WHERE
(
[Factories].[Factory Name].[Windhoek],
[Dates.Calendar Financial - Years].[Fin Year].[2017]
)
And the results I get look like this.

Saiku 5 week production.PNG

To run the query in Report Designer and following the sample report for SteelWheels, I came up with this query that works:

Code:
SELECT
NON EMPTY {


strToMember(Parameter("firstWeek",STRING,"[Dates.Calendar Financial - Weeks].[Fin Week No].[34]")):
strToMember(Parameter("lastWeek",STRING,"[Dates.Calendar Financial - Weeks].[Fin Week No].[38]"))} ON COLUMNS,


{
[Measures].[Export],
[Measures].[Detained],
[Measures].[Discarded],
[Measures].[Private]
} ON ROWS
FROM [Production Cube]
WHERE
(
[Factories].[Factory Name].[Windhoek],
[Dates.Calendar Financial - Years].[Fin Year].[2017]
)
In the report designer's data explorer it is then represented like this:

Report Data Explorer.png

Pulling the fields on to the report page looks like this.

Report Designer Page.jpg

When I run the report for weeks 34 to 38 all the data shows up in the report. However, if I select a different range of weeks, the data is not display. The screen grabs below shows data up to week 38 and then up to week 35.

Report Designer full data.jpg

Report Designer half data.jpg

Looking at everything I presented here, if makes sense that data will not be displayed unless it falls within weeks 34 to 38. How can I make this dynamic. I have played around with the MDX to see if I can obtain column aliases (like one can in SQL), but to no avail. Not that I am an MDX expert. I am quite new actually.

Can anyone give me some advice on how to make the dynamic so that the report can move on as the year goes on?

Thanks
Louis
↧

java.lang.OutOfMemoryError: GC overhead limit exceeded

October 25, 2016, 2:42 am
$
0
0
I am running PDI Pentaho MapReduce job on CDH5.8 (Cloudera Quickstart VM). I always get either "java.lang.OutOfMemoryError: GC overhead limit exceeded" or "java.lang.OutOfMemoryError: Java heap space" I did the following the adjust the heap size and GC overhead limit. Can someone suggest me what other thinkg I should do to overcome this problem.

1) I have increased memory in Spoon.bat and Spoon.sh (in Spoon.bat file, I have set "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms2048m" "-Xmx2048m" "-XX:MaxPermSize=2048m")
2) In hadoop-env.sh (/etc/hadoop/conf in CDH5.8), I have added "export HADOOP_CLIENT_OPTS="-XX:-UseGCOverheadLimit -Xmx4096m"3) I have added the following two properties in mapred-site.xml (/etc/hadoop/conf in CDH5.8)
 <property>
<name>mapred.child.java.opts</name>
<value>-Xmx2048m -XX:-UseGCOverheadLimit</value>
</property>
<property>
<name>mapred.job.reuse.jvm.num.tasks</name>
<value>1</value>
</property>
4) I have set - mapreduce.job.heap.memory-mb.ratio to 0.8

I am attaching the Pentaho PDI job log as below.
2016/10/25 14:40:11 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000001_0 Attempt: attempt_1477386149872_0001_m_000001_0 Event: 0
2016/10/25 14:40:11 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.inflateTable(HashMap.java:316)
at java.util.HashMap.put(HashMap.java:488)
at java.util.jar.Attributes.put(Attributes.java:148)
at java.util.jar.Attributes.putValue(Attributes.java:168)
at java.util.jar.Attributes.read(Attributes.java:421)
at java.util.jar.Manifest.read(Manifest.java:251)
at java.util.jar.Manifest.<init>(Manifest.java:69)
at java.util.jar.JarFile.getManifestFromReference(JarFile.java:181)
at java.util.jar.JarFile.getManifest(JarFile.java:167)
at sun.net.http://www.protocol.jar.URLJarFile.i...File.java:174)
at sun.net.http://www.protocol.jar.URLJarFile.g...File.java:142)
at java.util.jar.JarFile.maybeInstantiateVerifier(JarFile.java:309)
at java.util.jar.JarFile.getInputStream(JarFile.java:410)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2016/10/25 14:40:16 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2016/10/25 14:40:16 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000000_0 Attempt: attempt_1477386149872_0001_m_000000_0 Event: 1
2016/10/25 14:40:16 - Pentaho MapReduce - Exception from container-launch.
Container id: container_1477386149872_0001_01_000002
Exit code: 255
Stack trace: ExitCodeException exitCode=255:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:578)
at org.apache.hadoop.util.Shell.run(Shell.java:481)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)




Container exited with a non-zero exit code 255

2016/10/25 14:40:51 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000001_1 Attempt: attempt_1477386149872_0001_m_000001_1 Event: 2
2016/10/25 14:40:51 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149)
at java.lang.StringCoding.decode(StringCoding.java:193)
at java.lang.String.<init>(String.java:416)
at java.util.jar.Attributes.read(Attributes.java:418)
at java.util.jar.Manifest.read(Manifest.java:251)
at java.util.jar.Manifest.<init>(Manifest.java:69)
at java.util.jar.JarFile.getManifestFromReference(JarFile.java:181)
at java.util.jar.JarFile.getManifest(JarFile.java:167)
at sun.net.http://www.protocol.jar.URLJarFile.i...File.java:174)
at sun.net.http://www.protocol.jar.URLJarFile.g...File.java:142)
at java.util.jar.JarFile.maybeInstantiateVerifier(JarFile.java:309)
at java.util.jar.JarFile.getInputStream(JarFile.java:410)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
2016/10/25 14:40:56 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2016/10/25 14:40:56 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000000_1 Attempt: attempt_1477386149872_0001_m_000000_1 Event: 3
2016/10/25 14:40:56 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.substring(String.java:1913)
at sun.security.x509.X509AttributeName.<init>(X509AttributeName.java:53)
at sun.security.x509.X509CertInfo.get(X509CertInfo.java:572)
at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1784)
at sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:196)
at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:97)
at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339)
at sun.security.pkcs.PKCS7.parseSignedData(PKCS7.java:299)
at sun.security.pkcs.PKCS7.parse(PKCS7.java:158)
at sun.security.pkcs.PKCS7.parse(PKCS7.java:126)
at sun.security.pkcs.PKCS7.<init>(PKCS7.java:108)
at sun.security.util.SignatureFileVerifier.<init>(SignatureFileVerifier.java:96)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:296)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:348)
at java.util.jar.JarFile.getInputStream(JarFile.java:415)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)
↧

How to use & study Schema Workbench

October 25, 2016, 3:51 am
$
0
0
Hi,
where I study (books or internet) How to use and configure Schema Workbench?

Thanks
↧
↧

Pentaho Data Integration dynamic connection (read connection from database)

October 25, 2016, 4:57 am
$
0
0
I've posted a similar question to this a few days ago, but I could not solve the problem, even with the answers here forum.

I am very newbie in PDI and I have many questions about various concepts.

My question is:

Pentaho Data Integration: CE 6.1.0.1-196
I am newbie in Pentaho Data Integration. I need to run the same query in multiple databases. I created a table in the master database to store the connection information from other databases that need to be consulted. Below the table structure.

Code:
SQL> desc database_connection;
Name          Type          Nullable Default Comments
------------- ------------- -------- ------- --------
DATABASE_NAME VARCHAR2(32)  Y                       
JDBC_URL      VARCHAR2(512) Y                       
USERNAME      VARCHAR2(32)  Y                       
PASSWORD      VARCHAR2(32)  Y
ENABLED      VARCHAR2(1)  Y

Sample Data

Code:
DATABASE_NAME: XPTO
JDBC_URL: (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = xptosrv.xyz.com)(PORT = 1521))(LOAD_BALANCE = ON)(FAILOVER = ON)(CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = XPTO.XYZ.COM)(FAILOVER_MODE = (TYPE = SELECT)(METHOD = BASIC)(RETRIES = 180)(DELAY = 5))))
USERNAME: SYSTEM
PASSWORD: blablabla
ENABLED: Y


My .ktr files:

(set_variables.ktr)
Table Input ---> Copy rows to result
The query associated with the input table run in master database.

Code:
select database_name, jdbc_url, username, password from database_connection where enabled = 'Y'
(db_query.ktr)
Table Input ---> Table output
The query associated with the table input run o (multiple databases) and store data in table output (master database)


My .kjb files:

(run_for_each_row.kjb)
Start ---> Transformation ---> Success
Transformation filename: ${Internal.Job.Filename.Directory}/db_query.ktr
Job Properties Parameters:
DATABASE_NAME JDBC_URL PASSWORD USERNAME
(master_job.kjb)
Start ---> Transformation ---> Job for each row ---> Success
Transformation filename: ${Internal.Job.Filename.Directory}/set_variables.ktr
Job for each row filename: ${Internal.Job.Filename.Directory}/run_for_each_row.kjb
Job for each row ... Advanced tab Copy previous results to parameters -> checked Execute for every input row -> checked
Job for each row ... Parameters: DATABASE_NAME, JDBC_URL, PASSWORD, USERNAME

Execution log:

Code:
2016/10/06 10:36:15 - Spoon - Iniciando o job...
2016/10/06 10:36:15 - master_job - Início da execução do job
2016/10/06 10:36:15 - master_job - Starting entry [Transformation]
2016/10/06 10:36:15 - Transformation - Loading transformation from XML file [file:///D:/pdi/set_variables.ktr]
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - set_variables - Expedindo in?cio para transforma??o [set_variables]
2016/10/06 10:36:15 - Table input.0 - Finished reading query, closing connection.
2016/10/06 10:36:15 - Copy rows to result.0 - Finished processing (I=0, O=0, R=6, W=6, U=0, E=0)
2016/10/06 10:36:15 - Table input.0 - Finished processing (I=6, O=0, R=0, W=6, U=0, E=0)
2016/10/06 10:36:15 - master_job - Starting entry [Job for each row]
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - slave_job - Starting entry [Transformation]
2016/10/06 10:36:15 - Transformation - Loading transformation from XML file [file:///D:/pdi/db_query.ktr]
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/06 10:36:15 - db_query - Expedindo in?cio para transforma??o [db_query]
2016/10/06 10:36:15 - Table input.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : An error occurred, processing will be stopped:
2016/10/06 10:36:15 - Table input.0 - Error occurred while trying to connect to the database
2016/10/06 10:36:15 - Table input.0 -
2016/10/06 10:36:15 - Table input.0 - Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
2016/10/06 10:36:15 - Table input.0 - Erro de ES: Connect identifier was empty.
2016/10/06 10:36:15 - Table input.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Erro inicializando step [Table input]
2016/10/06 10:36:15 - Table output.0 - Connected to database [REPORT] (commit=1000)
2016/10/06 10:36:15 - db_query - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Step [Table input.0] falhou durante inicializa??o!
2016/10/06 10:36:15 - Table input.0 - Finished reading query, closing connection.
2016/10/06 10:36:15 - Transformation - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unable to prepare for execution of the transformation
2016/10/06 10:36:15 - Transformation - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleException:
2016/10/06 10:36:15 - Transformation - Falhou a inicializa??o de pelo menos um step. A Execu??o n?o pode sere iniciada!
2016/10/06 10:36:15 - Transformation -
2016/10/06 10:36:15 - Transformation -
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1142)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.trans.Trans.execute(Trans.java:612)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:1097)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.job.Job.execute(Job.java:723)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.job.Job.execute(Job.java:864)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.job.Job.execute(Job.java:608)
2016/10/06 10:36:15 - Transformation -  at org.pentaho.di.job.entries.job.JobEntryJobRunner.run(JobEntryJobRunner.java:69)
2016/10/06 10:36:15 - Transformation -  at java.lang.Thread.run(Thread.java:745)
2016/10/06 10:36:15 - slave_job - Finished job entry [Transformation] (result=[false])
2016/10/06 10:36:15 - master_job - Finished job entry [Job for each row] (result=[false])
2016/10/06 10:36:15 - master_job - Finished job entry [Transformation] (result=[false])
2016/10/06 10:36:15 - master_job - Job execution finished
2016/10/06 10:36:15 - Spoon - O Job finalizou.


Data from database_connection table is being read

Code:
2016/10/06 10:36:15 - set_variables - Expedindo in?cio para transforma??o [set_variables]
2016/10/06 10:36:15 - Table input.0 - Finished reading query, closing connection.
2016/10/06 10:36:15 - Copy rows to result.0 - Finished processing (I=0, O=0, R=6, W=6, U=0, E=0)
2016/10/06 10:36:15 - Table input.0 - Finished processing (I=6, O=0, R=0, W=6, U=0, E=0)
But I do not know what I'm doing wrong that these data are not passed as parameter.
I appreciate any help because I'm already stopped a few days ago this problem.
The examples I found here on stackoverflow and the pentaho forum did not help me much.
Project files (https://github.com/scarlosantos/pdi)
Thank you
↧

Query for GoogleMaps Overlay component

October 25, 2016, 5:29 am
$
0
0
Hello!

I already saw the samples avaliable in CDE Reference, but I don't solve my questions already.

How my query need to be to feed GoogleMaps Overlay component? It needs the countries, states, cities, lat, long?

If I able to send some countries positions to my map, how to put labels there?

I didn't find the same question in forum...

Thanks!
↧

Denormalize Table

October 25, 2016, 7:28 am
$
0
0
I am studying PDI, and am testing the denormalize, but could not understand the way pivor online columns my example

ID Code Name Desc Total Unity
500 33 1 1 Grat Wilian
500 VT 60 2 2 Wilian
500 East 4 1.90 7.60 Wilian
500 Other 4 4 4 Wilian
500 Board 5 5 5 Wilian
500 Elderly 6 5 5 Wilian
500 Freeboard 7 7 7 Wilian
500 VT HAS 8 8 8 Wilian

result:

ID Field3 Field4 Field5 Field6
500 <null> <null> <null> <null>
↧
Search

Splitting a large excel sheet into multiple Sheet

October 25, 2016, 10:59 am
$
0
0
Hi All,

Hope you are doing well.

I have a excel sheet with 100000 rows . I want it to split into ten excel sheet with 10000 rows.

How to achieve this.

Thanks in Advance.
Chinmaya
↧
↧

NullPointerException for nominal attribute on Weka Scoring

October 25, 2016, 5:53 pm
$
0
0
I've been trying to use the "Weka Scoring" step, but it keeps failing with a NullPointerException.
After some reverse engineering and testing, I found that the Nominal values for the class attribute are not being de-serialised from the model. I was able to get it working by manually adding the values and saving the model using the plugin's classes, but I need to be able to save the model straight from Weka and load it into Pentaho. Is there a specific setting I need to enable on either Weka or Spoon to make this work?

I'm using Weka 3.7.13. I was using 3.8, but some forums referenced kettle using 3.7, so I switched to that.

I'm using the "unbalanced" sample data supplied with Weka and J48 Unpruned.

Debug Log follows:
2016/10/26 11:33:29 - Spoon - Transformation opened.
2016/10/26 11:33:29 - Spoon - Launching transformation [weka_unbalanced]...
2016/10/26 11:33:29 - Spoon - Started the transformation execution.
2016/10/26 11:33:29 - weka_unbalanced - Dispatching started for transformation [weka_unbalanced]
2016/10/26 11:33:29 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/26 11:33:29 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/26 11:33:29 - CSV file input.0 - Header row skipped in file '/home/analytics/unbalanced.csv'
2016/10/26 11:33:29 - cfgbuilder - Warning: The configuration parameter [org] is not supported by the default configuration builder for scheme: sftp
2016/10/26 11:33:29 - Weka Scoring.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Unexpected error
2016/10/26 11:33:29 - Weka Scoring.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : java.lang.NullPointerException
2016/10/26 11:33:29 - Weka Scoring.0 - at weka.core.Attribute.numValues(Attribute.java:716)
2016/10/26 11:33:29 - Weka Scoring.0 - at org.pentaho.di.scoring.WekaScoringMeta.getFields(WekaScoringMeta.java:851)
2016/10/26 11:33:29 - Weka Scoring.0 - at org.pentaho.di.scoring.WekaScoring.processRow(WekaScoring.java:340)
2016/10/26 11:33:29 - Weka Scoring.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2016/10/26 11:33:29 - Weka Scoring.0 - at java.lang.Thread.run(Thread.java:745)
2016/10/26 11:33:29 - CSV file input.0 - Finished processing (I=793, O=0, R=0, W=792, U=0, E=0)
2016/10/26 11:33:29 - Weka Scoring.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
2016/10/26 11:33:29 - weka_unbalanced - Transformation detected one or more steps with errors.
2016/10/26 11:33:29 - weka_unbalanced - Transformation is killing the other steps!
2016/10/26 11:33:29 - weka_unbalanced - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Errors detected!
2016/10/26 11:33:29 - Spoon - The transformation has finished!!
2016/10/26 11:33:29 - weka_unbalanced - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Errors detected!
2016/10/26 11:33:29 - weka_unbalanced - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Errors detected!
↧

How do I convert a field's data type in User Defined Java Expression step?

October 25, 2016, 9:08 pm
$
0
0
I want to concatenate a series of strings without using Javascript.

I can do this with the User Defined Java Expression step with the following expression (and the new field value type = String):
stringField1 += stringField2 += stringField3 += "some other string not stored as a constant"

However, I also want to be able to append an incoming integer field (converted to String) to the end of this, as follows:
stringField1 += stringField2 += stringField3 += "some other string not stored as a constant" += integerField1.toString()

Now, of course, my effort above with .toString() doesn't work. But does anyone know of a way to do this conversion "on the fly" in the expression?

Otherwise, I suspect I will have to add one more "converted string" field to the stream in an earlier step. I'm just trying to avoid creating unnecessary fields in stream.

PS: I don't think I can use the Calculator step to do concats because it has an insufficient number of fields for my purpose.


Cheers,

Chris
↧

Saiku Cache Clear Functionality - Schema - Cube level

October 25, 2016, 9:12 pm
↧

Error connecting to FTP server when using GET A FILE WITH FTP job

October 26, 2016, 12:27 am
$
0
0
I'm trying to use the "Get A FILE WITH FTP" job step. After providing the server access details(FTP IP Address=xx.x.x.xx, Server port=21, User name=xxxxxx, Password=xxxx), i get the following error message: "Error Connecting: Connection Refused: Connect".
I was easily able to establish an sql connection with the same server while i was working on TRANSFORMATIONS. Now i'm working on JOBS and i'm getting above mentioned error.

Can anyone please help.
↧
↧

ARFF output plugin to working with variables to define file names

October 26, 2016, 3:56 am
$
0
0
I'm working on a project to predict supplier responses for a client that operates internationally. The output is always the same, as is the training dataset. However due to big differences per region, we need to generate unique models per region.

What I would like to do is use a generic PDI transformation to export the multiple region-datasets to ARFF files which I can proces separately using WEKA.

With other output formats I'd use a transformation parameter in the filename, however I cannot get this working with the ARFF output plugin. The plugin is always using the transformation defaults. An Excel writer in the same transformation respects the input.

Does anybody has suggestions to solve this? Or is it a known bug?

I tried the ARFF plugin from the market place, the one linked on the community doc and the 6.1 and 7 versions form this Jenkins.
Calling the transformation:
Schermafbeelding 2016-10-26 om 12.51.56.jpg

Transformation output:
Schermafbeelding 2016-10-26 om 12.51.19.jpg
↧

Duplicate rows filter and entire row goes into error table

October 26, 2016, 4:04 am
$
0
0
Hi All,

I am new in the pentaho integration tools, How will get the output like this.

Table 1 - Main input
Register No College courseid marks result_status Fail_count
001 ABC 11 35 Pass 2
001 ABC 12 20 Fail 2
001 ABC 13 30 Fail 2
002 ABC 11 50 Pass 0
002 ABC 12 60 Pass 0
002 ABC 13 60 Pass 0
101 XYZ 11 35 Pass 0
101 XYZ 12 40 Pass 0
101 XYZ 13 40 Pass 0
102 XYZ 11 50 Pass 1
102 XYZ 12 25 Fail 1
102 XYZ 13 60 Pass 1
Table 2 - Pass student ( Out put)
Register No College courseid marks result_status Fail_count
002 ABC 11 50 Pass 0
002 ABC 12 60 Pass 0
002 ABC 13 60 Pass 0
101 XYZ 11 35 Pass 0
101 XYZ 12 40 Pass 0
101 XYZ 13 40 Pass 0
Table 2 - Fail student ( Out put)
Register No College courseid marks result_status Fail_count
001 ABC 11 35 Pass 2
001 ABC 12 20 Fail 2
001 ABC 13 30 Fail 2
102 XYZ 11 50 Pass 1
102 XYZ 12 25 Fail 1
102 XYZ 13 60 Pass 1
↧
Search

Select Members in rows if the user select all in parameters

October 26, 2016, 4:33 am
$
0
0
Hi, i need to select all Members from a dimension in rows if the user select all in parameters and im trying to do that with the query below.
It doesn't seem to work.


Code:
WITH
MEMBER [Measures].[Conf. sin incidencias] AS
    Round(([Measures].[Confirmaciones totales] - [Measures].[Conf. con incidencias]),2)
SET [~ROWS] AS
    IIF({${Operacionalparameter}} = '[Direccion Operacional.Direccion].[Direcciones]',{[Direccion Operacional.Direccion].[Direccion Operacional].Members},{${Operacionalparameter}})
SELECT
NON EMPTY {[Measures].[Confirmaciones totales],
[Measures].[Rechazos],
[Measures].[Conf. con incidencias de linea],
[Measures].[Conf. con diferencias de cantidad],
[Measures].[Conf. con retraso],
[Measures].[Problemas transporte o indumentaria],
[Measures].[Problemas etiquetado],
[Measures].[Problemas temperatura],
[Measures].[Problemas lote],
[Measures].[Problemas caducidad],
[Measures].[Diferencias precio],
[Measures].[Conf. sin incidencias]}  ON COLUMNS,
NON EMPTY [~ROWS] ON ROWS
FROM [ConfirmacionesAlbaran]
Thank you.
↧

Download file using FTP in pentaho in TRANSFORMATION.

October 26, 2016, 5:05 am
$
0
0
Is there any way to retrieve a file from a web server using FTP protocol without using "Get A FILE WITH FTP" JOB STEP. I can only use TRANSFORMATION in Pentaho. Any ideas.
↧
↧

Lpad - script pdi 6.0

October 26, 2016, 6:58 am
$
0
0
I am studying part script pdi, as I am simulating an application and am trying to use the lpad function ...
I see that it is failing to understand the number of my variable fields and make the service complete with leading zeros than the maximum is 10


Script_PRodata.ktrScript_PRodata.ktr
Attached Files
↧

one Scenario

October 26, 2016, 12:56 pm
$
0
0
I have a scenario like this and how can I acheive this in PDI


input CSV file (3 columns )
=================
name,city,information
Jason,NYC,{"company1":"Microsoft","company2":"Apple","company3":"Google"}


output CSV file ( 5 columns)
==================
name,city,company1,company2,company3
Jason,NYC,Microsoft,Apple,Google
↧

Character encoding in PDI Text File Output

October 26, 2016, 2:15 pm
$
0
0
I have a JAVA code that encodes a string (which is actually a filename).


Code:
import java.net.URLEncoder;
    import java.nio.charset.StandardCharsets;
    import java.io.*;
   
    String encodedValue;
    public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException
    {
    // First, get a row from the default input hop
    //
    Object[] r = getRow();
   
    // If the row object is null, we are done processing.
    //
    if (r == null) {
            setOutputDone();
            return false;
    }
   
    // It is always safest to call createOutputRow() to ensure that your output row's Object[] is large
    // enough to handle any new fields you are creating in this step.
    //
    Object[] outputRow = createOutputRow(r, data.outputRowMeta.size());
   
    String newFileName = get(Fields.In,"to_be_encoded_path").getString(r);
   
    try{
        encodedValue = URLEncoder.encode(newFileName, "UTF-8");
    }
    catch (UnsupportedEncodingException e) {
    throw new AssertionError("UTF-8 is unknown");
    }
    // Set the value in the output field
    //
    get(Fields.Out, "encodedFileName").setValue(outputRow, encodedValue);
   
    // putRow will send the row on to the default output hop.
    //
    putRow(data.outputRowMeta, outputRow);
   
    return true;
    }
When I output this filename with its contents, using Text File Output, it gives me error because the filename can't have a '>' sign in it. It is supposed to be encoded with %3E and my code encodes it in the stream (I can preview it). But Text File Output is doing some mess here. It keeps the encoding for everything except '>' (weird). Has anyone who is using PDI, ever faced this issue?
↧
Viewing all 16689 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>