Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live

Conditional data connection for debugging purposes

$
0
0
Hi,

I am testing the data integration locally before running the job in "production" environment. What is best way to define conditional database connection for debugging and production?

Regards,

SK

Appending Data in Excel

$
0
0
I am using pentaho to collect and proccess data from my home automation system.

Ultimately I want to schedule it to run every 15 minutes and add the data as another row in an excel table so I can analyse it.

However I am struggling to get excel to append the data. Every time I run the transform it replaces the previous data with the current data. I have tried both the excel output and the excel writer, the configurations of which are shown below.

Is there a particular configuration required to get this to work?
Attached Images

Get namespaces of xml to create new xml

$
0
0
Hi,

i got a XML-File and want to split it into two seperate XML-Files with the same origin header-Information.
The header looks like

<EXPL xsi:schemalocation="http://www.expl.com/2015/XMLSchema expl.xsd" xmlns="http://www.expl1.com/2002/XMLSchema" xmlns:xsi="http://www.w3.org/2001/2015/XMLSchema-instance">

I tried a lot of different XPaths and got different unsatisfactory results using "Get Data from XML".
The same thing when I try to create XML with "Add xml".
Couldn't find any examples in the internet.

Any help would be appreciated
Leif

JSON Array issue in PDI 6.0

$
0
0
Hi,

I am trying to read three arrays namely comments, attachments, and messages using json input step. For this i mentioned $..items[*], $..attachments[*], and $..messages[*] in Fields tab of json input step.

I am getting below error when i am trying to run ETL:

"We found 3 values for json path [$..messages[*]], which is different that the number returned for path [$..comments[*]] (2 values). We MUST have the same number of values for all paths.".

Is it required to have the same number of elements in all the arrays? or else is there any alternative way to handle the following json?

JSON ARRAY.ktr

extn.txt
for your reference i am attaching JSON file(extn.txt) kindly save as .txt file with js in your machine and my transformation(.ktr) , kindly help me how can i achieve this issue.

Thanks
Attached Files

Databse connection issue in case of multiple jobs running togather

$
0
0
I am using PDI 6.0 Community Edition. I am facing problem related to Database Connection.
We have PDI jobs, scheduled to run during night. They transfer the data from DB to File.
The problem is, it gives below error randomly. And if we execute the job again, it works fine.
Is there there anything I can do to avoid such errors? Anyway to define connection pool size or something?

I'm getting below error:


2016/02/12 01:07:18 - TABLE_INPUT.0 - ERROR (version 6.0.0.0-353, build 1 from 2015-10-07 13.27.43 by buildguy) : An error occurred, processing will be stopped:
2016/02/12 01:07:18 - TABLE_INPUT.0 - Error occurred while trying to connect to the database
2016/02/12 01:07:18 - TABLE_INPUT.0 -
2016/02/12 01:07:18 - TABLE_INPUT.0 - Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
2016/02/12 01:07:18 - TABLE_INPUT.0 - IO Error: Connection reset

Authentication using JAAS on BI Server 6.0.1.0-386

$
0
0
I used to have Pentaho from versions 4 to 5.4 working fine using a JAAS Authentication Provider and now on 6.0.1.0-386 it's not working anymore. :(
I've double checked all the configuration and tried applying it to a clean installation, but login still fails.
On catalina I could only find errors about denied access to creating CDE folders and I'm not sure there's a relation.

[pt_47] Pentaho BI Platform server is ready.
08:31:08,443 ERROR [FileHandler] Couldn't find or create CDE . dir
org.pentaho.platform.api.repository2.unified.UnifiedRepositoryAccessDeniedException: access denied while creating folder with name "cde"

Does anyone have a clue of what might be wrong?
I'm attaching both catalina and part of the spring debug log after I try authenticating.

Thanks!
Attached Files

Numeric values are being rounded to decimals when read by using Database Join Step.

$
0
0
Hello there,

I'm using a PostgreSQL database and when I try to read a field devined as numeric(13,2), the values are rounded to decimals; e.g. if the field has the value 1.27, it is read as 1.3 by Kettle.

Where is such behaviour controlled? Is there a way to change it? Would it a Kettle misbehaviour?

The Kettle version I'm using is: Kettle - Spoon General Availability Release - 5.4.0.1-130The PostgreSQL version I'm using is 9.3

The filed is defined as below in the DDL:

...
val_lig_imp numeric(13,2)
...

Thank you.

parameterized database connection in pentaho CDE

$
0
0
I have scenario where i have one table in one database which consist all the database names,so in drop-down container i am displaying all this database name, now in another query i want to pass this database name as a parameter and want to retrieve data from the respected table.
Note : database structure is same for every client, but security reasons we have created different databases
Query one : using this i will get all the database names.
SELECT tenant from kscommon.organization;
Query two : tenant will be parameter for next query
select location from ${tenant}.organization;
but i am getting error(Error processing component)
SO can somebody guide me on this?

If statement in Database join step not working

$
0
0
I have 2 database join steps doing basically the same thing, but the second one I'm trying to use an if statement, but the first one runs just fine, the second one when it runs the if statement (it's a simple select statement with a table join) only the first field seems to get populated, then the rest of the fields end up as null, but the else statement seems to always run fine.

JavaScript return number types with precision

$
0
0
Hello, I have another question in my pdi journey.

So, I import number types from csv input, with the precision works:

img1.png

But I have to do some checks, and I use the Modified Javascript step:

image2.jpg


But, the precision is not work, I don't know why. The returned value isn't decimal, and I need to do check.

img3.png

With the check the first one, would be 0,01 and the others with 2 decimals place.
May someone help me?
Attached Images

PDI - Logging

$
0
0
Is there a way to log to file instead of to database for jobs and transformations? With stacks like ELK (ElasticSearch, Logstash and Kibana) gaining steam, it's becoming a burden to log database instead of to file where the files can be shipped to technologies like ElasticSearch and then visualized with tools like Kibana. I just can't seem to figure out how to log all that information to file instead of database.

Listener not working when using a external js file.

$
0
0
Hello,

I have a select component listening to parameter. I have a table with a function called on the clickAction. I do a fireChange and the parameter is updated. If I have the function written in the clickAction, it works, but if I use a call to an external function and I do the fireChange in that function, the listener doesn't work. Furthermore, I do another fireChange with another parameter in the same function. This parameter is listened by another table and this works both ways.

Is there a problem with select components?

Thank you.

Getting result from Query Component in Select Component

$
0
0
I've got an if/else statement in the post change property of a select component. It looks like this:
Code:

function funk() {
    var rar = TESTparamComPro.split('~');
   
    if (rar[0] == 'Com') {
        paramCompanyIDs = rar[1];
       
    } else {
        paramProfileID = rar[1];
        // I want to pass paramProfileID to a query component, and get the result back as a variable here so it'd look something like
        // queryComponentOutput = paramProfileID passed to query component
       
    }
}

I want to pass a parameter to a query component, and get that result back into this component. Is that possible?

Read many DBF files

$
0
0
Friends,

I know almost no one else uses DBF files, but my job much data is still stored in this format. So many DBF files are created. So I am in need of help! I'm needing reading several DBF files at once in the same way we can do with text files by step Text file input, without for that I need to enter several steps XBase input.


All DBF files are in the same directory and contains the same fields. The difference is in the DBF files is the period in which the records were stored in it.


So Friends, is there any way to read multiple DBF files through step XBase input?


Thank you.


Wisley Velasco - Brazil

How use weka algorithm ( java code ) in bi server and export result as chart?

$
0
0
Hi , i'm new in pentaho and bi and i want to use weka in bi server
for example use k -means algorithm ( java code ) on DWH and show result as chart
my question is how i can handle this ?
thank you for answering

Capturing Exceptions/Errors in PDI in transformation and passing forward to parent

$
0
0
Hello Every One ,

I am new to PDI , please can any one tell whether we have any procedure for capturing the runtime exceptions in PDI and passing to parent jobs for including in Email's in Parent jobs.





Thanks in advance .
Sameta

tomcat service on windows gives Error while trying to execute startup sequence

$
0
0
Pentaho 5.4 new installation on windows server 2012 with 64 bit Java 7.
I have replaced the tomcat bin directory with a 64 bit version.

1) The server starts fine with executed from a command prompt with admin privileges (start_pentaho.bat)
2) The Windows service does not start pentaho properly. Tomcat starts, but browsing to :8080 gives the error below



I thought it may be permission error, but even giving the service the Administrator credentials doesn't change the problem.
I can't find log entries which help me understand. What is different about the start sequence of the service vs the command prompt?
Because it's running as a service I don't know how to see the console but I've been looking in the log files.


One or more system listeners failed. These are set in the systemListeners.xml. org.pentaho.platform.api.engine.PentahoSystemException: PentahoSystem.ERROR_0014 - Error while trying to execute startup sequence for org.pentaho.platform.scheduler2.quartz.EmbeddedQuartzSystemListener
Please see the server console for more details on each error detected.

How to determine what data should appear in the collapse table output of the olap?

$
0
0
I am totally new to making schema - and i want to create an OLAP view for Jaspersoft report. But my understanding is too shallow so i try to study the Foodmart data and i have an intriguing question that i quite don't understand. See the screenshot below.

As you can see there are quite a few dimension + dimension usage for the Sales cube such as Store, Store Size in SQFT, Store Type, Time, Product, Promotion Media, Promotion, etc.

However despite of the few list of dimensions + dimension usage only two namely (Promotion Media + Product) is displayed in the OLAP crosstab output. My Question is how the system determine what to show on the table? why not show all the available dimension + dimension usage.

OLAP view.jpg
Attached Images

transformation becoming non responding

$
0
0
Hi Guys.

I am running my .kettle transformation in spoon.bat in which at one step i am having .52 million(529000 records). after reading .35 million(350000 records) its becoming non responding showing cpu usage almost 95 to 100%.In next step these .52 millions records are copied to 4 different location so in next step it has to write alsmost 2.1 million records. I have enclosed my .ktr file.
Properties used:
nr of rows in rowset 1000000
feedback size 6000000
memory allocation to java: "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms2048m" "-Xmx4096m" "-XX:MaxPermSize=1024m"

My system ram is 8gb.

It is a database transaction transformation.


Please help me how can i solve this issue.


Thanks in advance
Attached Files

CDE: use the value of a sql query on the definition of a bar chart not working

$
0
0
Hi.

I'm trying to add a horizontal line, to mi bar char with the value obtain with an sql query.

1. I create a sql over sqlJndi with the name thershold_software.

2. Then a Simple Parameter: Name: threshold_sw.

3. And a Query Component:
Name: threshold_QC
Result_Var: threshold_sw
Datasource: thershold_software
Post Fech:
function f2 (data){
Dashboards.fireChange('${p:threshold_sw}', data.resultset);
}

4. In my bar chart i added on extension points:
Arg : plot_add
Value:
function() {
var threshold = Dashboards.getParameterValue('${p:threshold_sw}');
alert (threshold);
return new pv.Rule()
.bottom(threshold,5)
.height(null)
.left(0)
.right(0)
.strokeStyle('red');
}


The alert show not value, like te parameter is empty.

If i add a manual value to the single parameter it shows that value.

But i want the value obtain of the query.

What i'm doing wrong??

Thanks


Viewing all 16689 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>