Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live

Job to send out email with xlsx file attachment fails when scheduled on linux

$
0
0
Dear all,

I have a job that extracts 200,000 records of data into an Excel file report and emails it out.


Below is my transformation flow, which is also in dropbox; https://www.dropbox.com/sh/mec6nxvsm...N6IOJj9a?dl=0:

Get environment variables-> extract data into an Excel file -> Check if File Exists -> Send mail.

When I run the job manaully, it runs fine.

When I use cron to schedule the job for Kitchen to run it, I get a Java out of memory error.

I have bumped up my memory to 4GBs but I am still facing this problem.

Are there any recommended approaches to solve this.

Thanks,

Ron

Data Validation : Best Approch

$
0
0
hi Team ,

I am new to Pentaho - trying to understand does Pentaho support inbuilt data integrity check - is there any way i don;t have to iterate each column for integrity check ? is there some generic/standard function that i can use for all integer columns or string columns etc . (trying to avoid javascript / java etc )


Quick reply on this - appreciated

Thanks

RESTful API Query Transsformation Example?

$
0
0
Does anyone have a sample transformation that you can share that is an example of making RESTful API (webservices) query and transforming the output and dumping to CSV?

Thanks in advance!

KP

help with mdxquery - java

$
0
0
Hello,

I have a problem with a mdx query in java.

I connected perfect, but when execute the query, i output this error:

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.log4j.MDC.getContext()Ljava/util/Hashtable;
at mondrian.server.Execution.copyMDC(Execution.java:106)
at mondrian.rolap.RolapConnection.execute(RolapConnection.java:624)
at mondrian.olap4j.MondrianOlap4jCellSet.execute(MondrianOlap4jCellSet.java:87)
at mondrian.olap4j.MondrianOlap4jStatement.executeOlapQueryInternal(MondrianOlap4jStatement.java:415)
at mondrian.olap4j.MondrianOlap4jStatement.executeOlapQuery(MondrianOlap4jStatement.java:347)
at hello.CreateDataBase.main(CreateDataBase.java:42)

I read that this is a problem with one class MDC but i dont know how repair this.

Thank you very much!

Tomcat 6 or Tomcat 7 ? (BI Server CE)

$
0
0
Hi

What is the best supported tomcat version 6 or 7? for a manual installation of BI Server CE 5.3.0.0-213?

Is there any performance or other issues per tomcat version?


Thanks

How to do the delete action of table records in the kettle

$
0
0
Hi all,

I want to do the insert/update/delete records in the target table. the target file has two cols key and value.
Source file col: s1, s2
Target file col: key, value
join condition: s1=key
Insert : when s1 is not exist in the target table
update: when s1 exist in the target table
delete: when key is not exist in the source table

i know the step "Insert / Update" can do the insert and update , but how to do the delete action?

Shims compilation doesn't work (you failed to publish jars on ivy-nexus.pentaho.org)

$
0
0
Dear all,

I've just downloaded the shims from here https://github.com/pentaho/pentaho-hadoop-shims and tried to build using the latest Ant.

I get the following errors:
------------------------------------

[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] module not found: org.pentaho#pentaho-aggdesigner-algorithm;5.1.3-jhyde

[ivy:resolve] ==== local: tried
[ivy:resolve] C:\Users\Ihor_Bobak\.ivy2\local\org.pentaho\pentaho-aggdesigner-algorithm\5.1.3-jhyde\ivys\ivy.xml
[ivy:resolve] -- artifact org.pentaho#pentaho-aggdesigner-algorithm;5.1.3-jhyde!pentaho-aggdesigner-algorithm.jar:
[ivy:resolve] C:\Users\Ihor_Bobak\.ivy2\local\org.pentaho\pentaho-aggdesigner-algorithm\5.1.3-jhyde\jars\pentaho-aggdesigner-algorithm-5.1.3-jhyde.jar

[ivy:resolve] ==== pentaho-ivy: tried
[ivy:resolve] http://ivy-nexus.pentaho.org/content...-jhyde.ivy.xml

[ivy:resolve] ==== pentaho-mvn: tried
[ivy:resolve] http://ivy-nexus.pentaho.org/content....1.3-jhyde.pom
[ivy:resolve] -- artifact org.pentaho#pentaho-aggdesigner-algorithm;5.1.3-jhyde!pentaho-aggdesigner-algorithm.jar:
[ivy:resolve] http://ivy-nexus.pentaho.org/content....1.3-jhyde.jar

[ivy:resolve] ==== public-maven: tried
[ivy:resolve] http://ivy-nexus.pentaho.org/content....1.3-jhyde.pom
[ivy:resolve] -- artifact org.pentaho#pentaho-aggdesigner-algorithm;5.1.3-jhyde!pentaho-aggdesigner-algorithm.jar:
[ivy:resolve] http://ivy-nexus.pentaho.org/content....1.3-jhyde.jar
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: org.pentaho#pentaho-aggdesigner-algorithm;5.1.3-jhyde: not found
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
D:\Projects\pentaho-hadoop-shims-master\build.xml:38: The following error occurred while executing this line:
D:\Projects\pentaho-hadoop-shims-master\build.xml:40: The following error occurred while executing this line:
D:\Projects\pentaho-hadoop-shims-master\build-res\subfloor.xml:773: impossible to resolve dependencies:
resolve failed - see output for details

Total time: 103 minutes 26 seconds
------------------------------------


If we manually try to get to http://ivy-nexus.pentaho.org/content/groups/omni, we don't see a folder "org.pentaho" with "pentaho-aggdesigner-algorithm" there.

Would anyone be so kind as to fix the repositories or tell here how to fix the ant build scripts to be able to fetch the pentaho-aggdesigner-algorithm and other jars?

Thank you in advance.

Kettle 5.2: Log4J plugin usage

$
0
0
Hi,

Can anyone provide any pointers on how to use the Kettle Log4j plugin? Could not find anything on the net...

Thanks,
Vinod

Files Processing in a batch - First File Issue

$
0
0
Hi,
Got a scenario, where need to process multiple files and it picks in a sorted order order. If from batch my first file is error then it doesn't do further processing [on transformations and jobs]. But if any file other than the first file [based on sorting pick] is error, then it does process every step.

Is this something multiple file handling designed that way? Is there a way NOT to stop if first file got issue?

Thanks
SHota

Excel Writer takes too long to "wrap up" a file...

$
0
0
So, I've been noticing lately there must be something wrong with Excel Writer output step. Sometimes, a simple output using it, with as low as 12 rows to output, took 3m31s to finish. Spoon's tooltip/selection tooltip (I don't know what you call that baloon with progress information that pops up like a tooltip, but the step needs to be selected for it to show) shows that after less than a second, all 12 rows have already been written, but the step won't finish.

Any idea what could be causing this? I know I started the thread blaming the step, but it could always be something I'm doing wrong...

restrict dimension values

$
0
0
hi there, I have some pretty wacky data to work with.

its essentially a fact table with a dimension table where, depending on the dimension value, the fact measure means something different (ie. hours, people etc).

I've created aggregate views on top of the fact table which restrict by dimension values so that the aggregate view is subject based and the measure means one thing (ie. hours).

but I've connected the dimension table in the Mondrian schema and that means filtering using the Saiku front end brings up all of the dimension records which makes no sense since the aggregate view restricts to a subset of those records.

is there a way in the Mondrian schema to restrict dimension usage to a subset of rows? or do I have to make a new dimension table for each aggregate view?

an info much appreciated.

The transaction in the transformation

$
0
0
Hi all,

my trans has table input, delete, insert/update steps, what is the detail transaction in this ?
The delete and insert/update have the separate transaction or just one?

if there have the different trans between delete and insert/update, but these steps process the same table, does there have a table locked issue?

Transform: Javascript replace functionality

$
0
0
Using the excel provided at: http://wdi.worldbank.org/tables
table 5.4 about stock markets

After loading the file into pentaho I've been trying to write a javascript to remove '..' from the data set using the replace function but can't seem to get it to work. Can someone help provide me with a script to remove '..' from the data set?

Tutorial for periodically updating maps (NewMapComponent)

$
0
0
Hi!

I needed to create a dashboard with a map that displays objects that sometimes move. The requirement was to automatically refresh the map and the object positions on it, just as it is possible with other components, e. g. Table.

So I created a tutorial on setting this up in German and English:
http://datascientist.at/2015/03/kart...ty-dashboards/

I hope you find this useful.

Best regards,

Balázs

Require help on pre-requisites for the execution of PRPT in UNIX

$
0
0
Hi,
I am very new to Pentaho report designer tool and just a beginner in this. So please provide your help on my very basic question.

How PRPT can be executed in Unix environment? What are the prerequisites for this?



Regards,
Lavanya

how to stop job?

$
0
0
hi,guys.
i'm using this command to run the kjb:

/usr/local/kettle/kitchen.sh /norep -file /usr/local/kettle/call_center/CallCenterJob.kjb


i use "ps -ef | grep kitchen" ,find this shell pid and kill it.
but the kjb is still running and syncing datas.
how can i find the kjb and kill it? i've already use "ps aux | grep -i "job:CallCenterJob"",but nothing find.

how to stop job?

$
0
0
hi,guys.
i use this shell to start a ktr from background:
/usr/local/kettle/kitchen.sh /norep -file /usr/local/kettle/call_center/CallCenterJob.kjb

i use the "ps -ef | grep kitchen" command and find the pid,then i kill it.

but i found that ktr is not be killed.it is still running.
how can i kill that ktr?i've already use "ps aux | grep -i "job:CallCenterJob"" to find the process.but nothing found.

Access page count in PRPT Component

$
0
0
I'd like to manual control the pages of a PRPT component.
I can use the parameter "accepted -page" to specify which page to display, but how can I get the total page count (without reading from the “pageTotalSpan” div)?

Is there a way to access “dojo.dijit.byId("pageControl").pageTotal” in the pre-Excecution function?

How to Read File and set fields dynamically

$
0
0
Hi I am newbie to Pentaho. My requirement is read the files automatically and create a output file with provided inputs header available on input files.

1.Read any bat file(Header fields are separated by $GS$). This is bat file
Ex: one$GS$two$GS$three$GS$four
1$GS$2$GS$3$GS$4
1$GS$2$GS$3$GS$4

2.Find the first row header information and convert the $GS$ into ; and provide the putput txt file
Ex: one;two;three;four
1;2;3;4
1;2;3;4

How to do that? Thanks in advance.

Kettle not connecting to postgres

$
0
0
Hello, I am configuring pentaho data-integration on our server, and I am having quite hard times connecting to postgres database.

Basic info:
data-integration 5.2.0.0-209
postgres 9.3
java version "1.7.0_75"
OpenJDK Runtime Environment (IcedTea 2.5.4) (7u75-2.5.4-1~trusty1)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
OS: Ubuntu 14.04.2 LTS


Case:
I have simple Job, it is checking if table exists in target DB. If no, it runs SQL command creating table. If yes, or after executed SQL command creating table, it is running transformation.

When I create connection in spoon, and test it, It says that it is ok, working. But after I try to run transformation, it throws error connecting to database:
Code:

2015/03/16 14:37:00 - park2report - Start of job execution
2015/03/16 14:37:00 - park2report - Starting entry [Table zones_reportdb exists]
2015/03/16 14:37:00 - Table zones_reportdb exists - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : An error occurred executing this step:
2015/03/16 14:37:00 - Table zones_reportdb exists - Error occured while trying to connect to the database
2015/03/16 14:37:00 - Table zones_reportdb exists - Required subelement 'driver'

kitchen command I am using is:
Code:

sh kitchen.sh -file=scripts/jobs/park2report.kjb
jdbc.properties file:
Code:

#SampleData/type=javax.sql.DataSource
#SampleData/driver=org.h2.Driver
#SampleData/url=jdbc:h2:file:samples/db/sampledb;IFEXISTS=TRUE
#SampleData/user=PENTAHO_USER
#SampleData/password=PASSWORD
#Quartz/type=javax.sql.DataSource
#Quartz/driver=org.hsqldb.jdbcDriver
#Quartz/url=jdbc:hsqldb:hsql://localhost/quartz
#Quartz/user=pentaho_user
#Quartz/password=password
#Hibernate/type=javax.sql.DataSource
#Hibernate/driver=org.hsqldb.jdbcDriver
#Hibernate/url=jdbc:hsqldb:hsql://localhost/hibernate
#Hibernate/user=hibuser
#Hibernate/password=password
#Shark/type=javax.sql.DataSource
#Shark/driver=org.hsqldb.jdbcDriver
#Shark/url=jdbc:hsqldb:hsql://localhost/shark
#Shark/user=sa
#Shark/password=
report-connetion/type=javax.sql.DataSource
report-connection/driver=org.postgresql.Driver
report-connetion/url=jdbc:postgresql://127.0.0.1:5432/report
report-connetion/user=******
report-connetion/password=*****
park_connection/type=javax.sql.DataSource
park-connection/driver=org.postgresql.Driver
park-connection/url=jdbc:postgresql://172.23.100.1:5432/park
park-connection/user=*****
park-connection/password=*****

When I run kitchen command alone, output is:
Code:

app_core@delta:~/flexcore/pentaho/data-integration$ sh kitchen.sh -file=scripts/jobs/park2report.kjb
/home/app_core/flexcore/pentaho/data-integration
2015/03/16 14:58:16 - Kitchen - Start of run.
2015/03/16 14:58:17 - park2report - Start of job execution
2015/03/16 14:58:17 - park2report - Starting entry [Table zones_reportdb exists]
2015/03/16 14:58:17 - Table zones_reportdb exists - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : An error occurred executing this step:
2015/03/16 14:58:17 - Table zones_reportdb exists - Error occured while trying to connect to the database
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.job.Job.run (Job.java:424)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.job.Job.execute (Job.java:532)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.job.Job.execute (Job.java:859)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.job.Job.execute (Job.java:716)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.job.entries.tableexists.JobEntryTableExists.execute (JobEntryTableExists.java:175)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.Database.connect (Database.java:361)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.Database.normalConnect (Database.java:410)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.Database.connectUsingClass (Database.java:466)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.Database.initWithNamedDataSource (Database.java:440)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource (DatabaseUtil.java:55)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.pentaho.di.core.database.util.DatabaseUtil.getDataSourceFromJndi (DatabaseUtil.java:77)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at javax.naming.InitialContext.<init> (InitialContext.java:192)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at javax.naming.InitialContext.init (InitialContext.java:242)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at javax.naming.InitialContext.getDefaultInitCtx (InitialContext.java:307)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at javax.naming.spi.NamingManager.getInitialContext (NamingManager.java:684)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.SimpleContextFactory.getInitialContext (SimpleContextFactory.java:69)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.SimpleContext.<init> (SimpleContext.java:98)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:104)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:140)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:156)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.loader.JndiLoader.load (JndiLoader.java:279)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.loader.JndiLoader.jndiPut (JndiLoader.java:321)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.jndi.AbstractContext.bind (AbstractContext.java:342)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at org.osjava.sj.jndi.AbstractContext.bind (AbstractContext.java:334)
2015/03/16 14:58:17 - Table zones_reportdb exists -  at java.util.Hashtable.put (Hashtable.java:514)
2015/03/16 14:58:17 - park2report - Starting entry [Create table zones_reportdb]
2015/03/16 14:58:17 - Create table zones_reportdb - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : An error occurred executing this job entry :
2015/03/16 14:58:17 - Create table zones_reportdb - Error occured while trying to connect to the database
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.Job.run (Job.java:424)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.Job.execute (Job.java:532)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.Job.execute (Job.java:859)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.Job.execute (Job.java:859)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.Job.execute (Job.java:716)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.job.entries.sql.JobEntrySQL.execute (JobEntrySQL.java:243)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.Database.connect (Database.java:361)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.Database.normalConnect (Database.java:410)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.Database.connectUsingClass (Database.java:466)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.Database.initWithNamedDataSource (Database.java:440)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.util.DatabaseUtil.getNamedDataSource (DatabaseUtil.java:55)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.pentaho.di.core.database.util.DatabaseUtil.getDataSourceFromJndi (DatabaseUtil.java:77)
2015/03/16 14:58:17 - Create table zones_reportdb -  at javax.naming.InitialContext.<init> (InitialContext.java:192)
2015/03/16 14:58:17 - Create table zones_reportdb -  at javax.naming.InitialContext.init (InitialContext.java:242)
2015/03/16 14:58:17 - Create table zones_reportdb -  at javax.naming.InitialContext.getDefaultInitCtx (InitialContext.java:307)
2015/03/16 14:58:17 - Create table zones_reportdb -  at javax.naming.spi.NamingManager.getInitialContext (NamingManager.java:684)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.SimpleContextFactory.getInitialContext (SimpleContextFactory.java:69)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.SimpleContext.<init> (SimpleContext.java:98)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:104)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:140)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.loader.JndiLoader.loadDirectory (JndiLoader.java:156)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.loader.JndiLoader.load (JndiLoader.java:279)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.loader.JndiLoader.jndiPut (JndiLoader.java:321)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.jndi.AbstractContext.bind (AbstractContext.java:342)
2015/03/16 14:58:17 - Create table zones_reportdb -  at org.osjava.sj.jndi.AbstractContext.bind (AbstractContext.java:334)
2015/03/16 14:58:17 - Create table zones_reportdb -  at java.util.Hashtable.put (Hashtable.java:514)
2015/03/16 14:58:17 - park2report - Finished job entry [Create table zones_reportdb] (result=[false])
2015/03/16 14:58:17 - park2report - Finished job entry [Table zones_reportdb exists] (result=[false])
2015/03/16 14:58:17 - park2report - Job execution finished
2015/03/16 14:58:17 - Kitchen - Finished!
2015/03/16 14:58:17 - Kitchen - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Finished with errors
2015/03/16 14:58:17 - Kitchen - Start=2015/03/16 14:58:16.367, Stop=2015/03/16 14:58:17.820
2015/03/16 14:58:17 - Kitchen - Processing ended after 1 seconds.
app_core@delta:~/flexcore/pentaho/data-integration$

Could please somene give me som hint, where or wat, or how should I look up for solution?

I have been trying every jdbc driver for postgres 9.3+ that I found on https://jdbc.postgresql.org/download.html
Code:

postgresql-9.3-1102-jdbc4.jar  postgresql-9.3-1103.jdbc41.jar  postgresql-9.4-1201.jdbc41.jar
postgresql-9.3-1103.jdbc3.jar  postgresql-9.3-1103.jdbc4.jar  postgresql-9.4-1201.jdbc4.jar

but none helped.

I have also tried to update java on server. It was:
Code:

java version "1.6.0_34"
OpenJDK Runtime Environment (IcedTea6 1.13.6) (6b34-1.13.6-1ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

Nowit is:
Code:

java version "1.7.0_75"
OpenJDK Runtime Environment (IcedTea 2.5.4) (7u75-2.5.4-1~trusty1)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)

Still no success.

I really dont know what else should I do.

I have quite good understanding of SQL and python, but in Java im just lost. :(
Attached Images
Viewing all 16689 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>