Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live
↧

Transactional execution with Kettle 4.4.2

January 15, 2016, 4:07 am
$
0
0
Hi Guys,

I've created a job that extracts data from several tables in an Oracle DB and loads it in another Oracle DB.
Table definition source/target is 1:1.
Job contains:
- one transformation with an 'Add constant rows' step to set table names to load data from and into and a 'Coy rows to result step'
- one job that executes for every input row and has:
- one 'Set variabes' step to set current table name
- one generic transformation to extract and load data that contains:
- one table input step with SELECT * from ${current_table} executed on source connection
- one table output step with Target table set as ${current_table} executed on destination connection

The question is if there is way to ensure that if data loading fails for one of the tables, changes for all tables will be rolled back?

From my testing, the 'Make database transactional' transformation option would helphere only if input/output steps for all (around 30) tables would be in the same transaction.

OR maybe there is another job design more adequate for this case?

Thanks in advance.
Larisa.
↧
Search

pass parameter(s) from CDE dasbhoard to report

January 15, 2016, 4:25 am
$
0
0
Hi,


So I know you're able to pass parameters throughout dashboards with a simple function and a custom parameter in the 2nd dashboard.
Now I was wondering if there is a way to do this but between a dashboard and a report.

For example I click on a specific cell or row on a dashboard table and it then opens up a report which uses the values from the table.

If there is a way to do this and someone could tell me how to achieve this that would be great!


Regards,
↧
↧

Organize a messy text file to export to DB

January 15, 2016, 5:43 am
$
0
0
Hi, I'm starting to work with PDI Kettle and I'm new in this forum. I need some help.

I have a huge problem with a text file exported from a ERP System, because the data from this file there isn't organization and there aren't separations.

Here is a sample:

Code:

NAME OF COMPANY                                                                                                        PAG:    1
ssw0216                                          DEMONSTRATIVO DE FRETES DE COLETA/ENTREGA REALIZADA                    12/01/16  07:56
UNIDADE: NAME OF UNIT                                                                          PERIODO: 01/01/16 A 08/01/16
VEICULO: XYZ1234    TIPO: TRUCK    CONTRATADO..: NAME OF UNIT
------------+---------+--------------------+--------------------+---+------+-----+------------+-----------+-----------+--------+---------
CTRC        NF        REMETENTE            DESTINATARIO        SET PESO  QTVOL    VAL MERC  VLR FRETE    % FRETE ROMANEIO CTRB
------------+---------+--------------------+--------------------+---+------+-----+------------+-----------+-----------+--------+---------
DIA 07/01/16  COLETA
BMV/020520-6    238072 NAME OF CLIENT 1 03  4.155,2    8    16.000,00    1.031,79        0,00      0-0
SUB-TOTAL  COLETA        1 COLETAS/  1 CLIENTES                    4.155,2    8    16.000,00    1.031,79

TOTAL DO DIA -    1 EVENTOS/  1 CLIENTES                            4.155,2    8    16.000,00    1.031,79
  DIARIA                                          239,73
  KM RODADOS      0 (R$  1,131/KM)                0,00
  EVENTOS/CLIENTES                                  0,00
  PESO                                              0,00
  % SOBRE FRETE                                      0,00        REMUNERACAO DE      239,73
  ICMS                                              0,00

DIA 08/01/16  COLETA
BMV/020521-4    97537 "NAME OF CLIENT 2" 016  711,9    30    16.068,22      301,42        0,00      0-0
BMV/020522-2    97540 "NAME OF CLIENT 2" 016    45,6    6      194,57      84,53        0,00      0-0
.
.
.

TOTAL DO DIA -    7 EVENTOS/  1 CLIENTES                            4.326,5    63    60.920,13    1.687,94
  DIARIA                                          239,73
  KM RODADOS      0 (R$    1,131/KM)                0,00
  EVENTOS/CLIENTES                                  0,00
  PESO                                              0,00
  % SOBRE FRETE                                      0,00          REMUNERACAO DE      532,73
  ICMS                                            131,71
I need the information in bold, and the transformation will stay like this:

Code:
VEICULO;TIPO;DIA;TIPO_OP;CTRC;NF;REMETENTE;...;KM_RODADOS;REMUNERACAO;
XYZ1234;TRUCK;07/01/16;COLETA;BMV/020520-6;238072 ;NAME OF CLIENT 1;0; 239,73;
XYZ1234;TRUCK;08/01/16;COLETA;BMV/020521-4;97537;NAME OF CLIENT 2;0; 532,73;
XYZ1234;TRUCK;08/01/16;COLETA;BMV/020522-2;97540 ;NAME OF CLIENT 2;0; 532,73;
But, first of all, i need to split the separate information someway because I just could extract the information above, missing others datas:

Code:
CTRC;NF;REMETENTE;...
BMV/020520-6;238072;NAME OF CLIENT 1;...
BMV/020521-4;97537;NAME OF CLIENT 2;...
BMV/020522-2;97540 ;NAME OF CLIENT 2;...
Can someone help me?

ps: Sorry for my english, because I'm brazilian and I can't speak english very well...
↧

Convert Single Column file into row

January 15, 2016, 5:51 am
$
0
0
I have file with one column and 10000 rows

i need to convert into rows of 200 each separated by ,

for example:

Field 1

1
2
3
4
5
6
7
....
10000

Convert to :

Field 1:
1,2,3,4....200
201,202,203,.....400
401,402,.....600


Any suggestions or help?
↧

How to use credentials file for setting database password in a secure way?

January 15, 2016, 5:52 am
$
0
0
I managed to use the a "Get Data from XML --> Set Variables" transformation sequence to store database credentials in an external XML file and set Kettle variables. These wil be used in subsequent transformations for accessing the database(s).

So far so good, but: each time the credentials are set in the step "Set Variables", the values of the variables are written to the log file:

Code:
2016/01/15 14:13:17 - Set Variables.0 - Setting environment variables...
2016/01/15 14:13:17 - Set Variables.0 - Set variable DATABASE to value [db1]
2016/01/15 14:13:17 - Set Variables.0 - Set variable PASSWORD to value [secret]
Ugh! Now this is unacceptable from a security point of view.

As an alternative, I know that I could store the DB credentials in the transformation steps directly, e.g. in "Table Output" step - thus, I would end up with an encrypted password stored in the ktr file. But this is no viable way, because if credentials changed, I would have to change multiple ktr files. This is not what I want.

Is there a way to read in the credentials from some place on a server and use it in a Kettle job without exposing them to the log?

I am using Pentaho PDI 6.0.0.
↧
↧

Error while launch spoon.bat

January 15, 2016, 8:24 am
$
0
0
Dear all,

I've got an error when i'm trying to open the spoon.bat in my windows. It would be helpful if you guys can let me know what should i do to overcome this problem?

Enclosed herewith the screenshot of the error that i've got for your references.

Thank you.
Attached Images
↧

Nested LDAP Configuration on BA Server 6.0 - Service won't start.

January 15, 2016, 10:21 am
$
0
0
I have BA server 6.0 configured and (mostly) working with LDAP authentication. The one thing I'd like to have working that isn't is group based authorization through nested LDAP groups. In trying to follow these instructions, I change the populator bean class to "org.pentaho.platform.plugin.services.security.userrole.ldap.NestedLdapAuthoritiesPopulator". However, when I make that change the BA server will not start up properly.

I will post the logs in a separate post. Can anyone help me to get nested ldap authorization working? Thanks.
↧

Report Designer and Pentaho Server DB connections

January 15, 2016, 12:50 pm
$
0
0
Looking to see if anyone can point me in the right direction here. I've recently installed Pentaho 6.0 Community Edition on a virtual machine, and have successfully set up a data connection within the Pentaho User Console to our SQL server to use in dashboard creation. The type of data connection we are using for this is JDBC (when looking at "Manage Data Sources"). From watching tutorials, when I chose a new map component or any other object requiring a datasource, I am using "sql over sqljndi" as the datasource object, and from there I just type in the name of that connection that I set up already, and everything has worked. I've never had to install any .jar files or add text to any files, or do anything other than set up the connection by entering the database name and UID/PWD.

The next thing I'd like to do is get some reports going with Pentaho Report Designer. I have Report Designer installed on my local PC. Within Report Designer, I can set up a data connection in the same manner (clicking new connection and entering DB and UID/PWD) and it appears to work just fine when running a report locally (and again, I didn't have to alter any files or install .jar files). But when I publish a report to the repository on the Pentaho server (virtual machine) and open it in a URL, it seems to show only a snap-shot of the dataset (meaning if anything changed, it still only shows the old data). I'm fairly new to javascript and JDBC so I'm sure there's something I'm missing but I don't know what. Ideally, I'd like to use the Pentaho server's connection for all dashboards AND reports, and not have to set anything up on manager's PCs. I thought that was where JNDI comes in, but I'm having a hard time finding information on how to make this work. I'm hoping someone out here can help me.

So basically I'd like to know if it is possible to have the Pentaho server's data connection that I'm already using for dashboards, also be used for reports from Report Designer, and what needs to be done to make that happen. Any suggestions? I've read a few posts and found some things on google, but a lot are old and there seems to be different instructions on what to do. Again, I'm sure me being new to the javascript world is part of the problem, but any help would be greatly appreciated.

Thanks,
Wes
↧
Search

BA Platform 6.0 (Community Edition): Sample reports

January 16, 2016, 7:21 am
$
0
0
Hello everyone,

I have just installed BA Platform 6.0 (Community Edition). I was wondering if there are some sample reports somewhere as I don't find them or any good tutorial on how to get started...

Thanks,

Lady V
↧
↧

Datasource from KTR transformation file?

January 17, 2016, 1:12 am
$
0
0
I am relatively new and try to evaluate some more details how to handle data sources in BI

One of my transformation work with a vendor specific JDBC driver. Unfortunately, the driver does not yet support quoted identifiers - the reason why I can not setup a datasource in BI with the driver.

1. I managed to create a Transformation in PDI which I would like to use as a datasource in BI. I uploaded the KTR to BI suite an can run the transformation successfully (output CSV to local filesystem...). I just don't know how to setup a datasource when selecting "Pentaho Data Services" as connection type.
2. I created a working analysis based on the static CSV file from the transformation.

Could someone explain a little bit how to use the KTR file as dynamic data source?
↧

Help on creating summary on group header

January 17, 2016, 2:04 am
$
0
0
Hi,


I'm new to pentaho. I'm trying to have a sum on group header. I created a numeric field on Group Footer and set the following properties:
- field = Cost_of_Sales
- aggregation-type = Sum
- aggregation-group = VehiclesType
The above configurations working fine. But when i try to transfer the numeric field on group header it returns zero. Anyone can give an advise on how to have a sum of a group column on group header?


Anthony
↧

CART / Oblique Trees implementation?

January 17, 2016, 6:10 pm
$
0
0
Does weka implement CART for creating oblique classification trees? I found online that it has been implemented but I couldn't find a method for it in the WEKA package. Looking for some clarification. Any help is appreciated!
↧

Variables in Pentaho Reporting

January 18, 2016, 12:57 am
$
0
0
Hi pentaho friends,

is it possible to use Variables in PRD.

For example I make a select in a subreport and want to save the result field in a varable to use it later.

Thanks
Norbert
↧
↧

Average all the value in a row

January 18, 2016, 1:42 am
$
0
0
Hello there,

I would like to ask some help regarding my situation here.I have a file inputs like this
10151377_10208293907854954_8208866969641127970_n.jpg

I would like the output to be like this below.The average is calculated for each row separately.
12573964_10208293907934956_537049479867419746_n.jpg

I had refereed to few examples but non of the examples show or explain the way I want.I had tried Group By or Calculator step,
however its totally wrong since I guess my method is not correct.
I am still new to Pentaho Data Integration.Would be grateful if there is any kind of help.

Best Regard,
Raj

Pentaho Data Integration 5.4
↧

Measure in Pentaho Schema Workbench from table other than fact table

January 18, 2016, 2:45 am
$
0
0
Can I select measure in Pentaho Schema Workbench from table other than fact table ?
↧
Search

DB connection in version 5.2 is ok, but not in version 6.0

January 18, 2016, 4:08 am
$
0
0
Hello,

The exception scenario is, I use account "test_etl" to delete table "test_mgr.project_list"

It is ok in 5.2 but not in 6.0, so I get "ORA-00942: table or view does not exist" error

I open Database connection edit window, and I click "explore"

I can find project_list table in test_mgr schema in 5.2 but not in 6.0

How could it happen?

Thanks for any comment !!!
↧

Perspectives and XUL Overlays on 6.0

January 18, 2016, 5:52 am
$
0
0
Hi all,

I'm trying to customize the user home that is displayed when a user logs into pentaho 6.
Starting from the article http://blog.oncase.com.br/pentaho-6-...#disqus_thread (that I tested exactly as it is and it worked well)

I'd like to add a new Menu in the toolbar and hide the existing one as well.
I read that this could be done using XUL Overlays.

Looking at the bluprint.xml file I notice that the candidate section is the one mapped in the class StaticPerspective
that is:

Code:
privateArrayList<XulOverlay> overlays;
this means that in the blueprint.xml file we're going to use:

Code:
<property name="overlays">
    <list>
      //here the xml code for the overlays
    </list>
</property>
Please, does anyone know which is the right code to define an XUL Overlay array in order to display a Menu on the toolbar?

Thank you so much for the support.
Max
↧
↧

Pentaho does not start after setting environmental variables

January 18, 2016, 6:10 am
$
0
0
Hello everybody, this is my first post.
I can not run Pentaho Bi Server community edition, even though the environmental variables should be properly set
Any ideas on how I could solve the issue?
Thanks
↧

How to configure multiple database connections using config files?

January 18, 2016, 7:20 am
$
0
0
We use kettle as an interface appliance for importing data into our central (OLAP) reporting database. Since there are various data sources for reporting, we have a couple of different database connections to import from.

Now I want to keep the kettle files free from database credentials so that the kettle files can run on the test server, then be deployed to the production server, using a different set of database connections.

My vision was to use an XML file holding the several connection settings (one connection per node). I read it with the "Get data from XML" step.

Code:
<?xml version="1.0" encoding="UTF-8" ?>
<connections>
    <connection>
        <host>dbhost1</host>
        <database>database1</database>
        <user>fred</user>
        <password>Encrypted 12321343423142314234234234</password>
    </connection>
    <connection>
        <host>dbhost2</host>
        <database>database2</database>
        <user>wilma</user>
        <password>Encrypted 987987987987987987987987987</password>
    </connection>
</connections>
And then follows a "Javascript Value" step which dynamically sets connection parameters like:
  • DB1_HOST = 'dbhost1'
  • DB1_DATABASE = 'database1'
  • DB1_USER = 'fred'
  • NEXTDB_HOST = 'dbhost2'
  • NEXTDB_DATABASE = 'database2'
  • NEXTDB_USER = 'wilma'
  • etc.


So the idea ist: for each "row" in the XML config file (each <connection> node), create the a set of kettle variables XX_HOST, XX_DATABASE, XX_USER and XX_PASSWORD. These variables can then be used in subsequent database access steps.

I can't figure out how to make this work. As it seems to me, the "Javascript modify value" can only populate pre-defined Variables and cannot create new ones. However, I would very much like to generate these configuration settings dynamically.

Has anybody an idea how I could accomplish this task?

Thank you very much in advance! I am using kettle 6.0.0.
↧

Pentaho Data Integration (Which Linux Flavor and Version is appropriate).

January 18, 2016, 7:30 am
$
0
0
Is there a Linux flavor (Mint, SUSE, etc) that is better adapted to run Pentaho Data Integration with less fuss?

background:
Each time that I start out with new desktop based Linux, there are a few installation and configuration requirements beyond the Oracle JRE/JDK to get Pentaho Data Integration 5.x or 6.x up and going. Usually this involves one or two specific packages "xulrunner" at version 1.9.2, Webkit at 1.0 and sometimes fighting to get a version of the swt library that makes things work.

The OS versions that I had success with after fighting with this are:
CentOS 6.5 (versions 6.x to current) (forget, but seems like we had to disable cairo to get it working)
Ubuntu 14.04 LTS. (Required swt snatched out of the eclipse install that is defaulted to the platform, xulrunner manual download, webkit 1.0)

Thanks for any pointers.
↧
Viewing all 16689 articles
Browse latest View live
Search