Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live

Pentaho Ctools and the Fine Art of BI

$
0
0
Today, Pentaho is launching a campaign specially focused at some of the work that we've been doing for a while. It's called the Fine Art of BI.

For me, this sums up the real power of BI is, and where Pentaho excels - Allowing us to forget any technical constraints and focus on business needs, knowing whatever comes out of it will be possible to implement.

A few weeks back, I shared an internal email with the Pentaho sales team. I didn't know the Marketing team was already working on this campaign! I'm not sure if I should be proud cause it's the validation of a good idea, or worried cause I'm starting to think like Marketing! :)


Here's the email - the original names have been wiped out


Hello everyone


Most of you have been involved in engagements where customers want to implement Ctools dashboards. When that happens, it's a great opportunity to take the conversation away from the technical aspects and dive deep into the business concepts.


And one of the things we always ask is for them to provide us with a mockup of what they envision their project to look like.


There are times where they already have something they can share. But other times they show some discomfort, either because they never thought about it, don't know how to do it or simply say they have the tools to do it. But it's important to insist, cause they *always* have something in mind.


I'd like to share with you an example of what was probably the best mockup we ever received from a prospect, and really shows that they don't need a lot to send us information.


Attached is a hand-made drawing we got from and the resulting mockup that Nuno's team prepared for Fred to send to them. It his our hope that by seeing a materialization of their ideas, this can become an important push towards closing the deal. And sell services along with it.


See you all next week!
And these are the files:

Before:



and after:



More...

How change the ".pentaho" and ".tonbeller" directory folder?

$
0
0
Hello.

When i start pentaho app create automatically this folders ".pentaho" , ".tonbeller" and ".kettle" in root of c:\ .
( Create the folders "c:\.pentaho" , "c:\.tonbeller" and "c:\.kettle" ). [Windows version]

I would like change this folders to "c:\pentaho\.pentaho", "c:\pentaho\.tonbeller" and "c:\pentaho\.kettle".

Using this manual, i fix my problem to set ".kettle" folder, but not work to change ".pentaho" and ".tonbeller" folders.
http://infocenter.pentaho.com/help/i..._home_dir.html

It´s possible change the "c:\.pentaho" and "c:\.tonbeller" folder?
What i need to do to fix it?

Thanks.

Logging metadata reporting and performance

$
0
0
Hi,

I'm trying to do some reporting with a metadata file with many tables. It is very slow even with 3 fields.
Is there any way to see the log of the reporting execution as it is in mondrian to see how long is the metadata conversion and the SQL execution?

I get the following error after 5 minutes or less and don't know why:


ClientAbortException: java.net.SocketException: Broken pipe
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:358)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:352)
at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:381)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:370)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:83)
at org.apache.commons.io.IOUtils.write(IOUtils.java:631)
at org.pentaho.platform.web.servlet.GenericServlet.doGet(GenericServlet.java:179)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.PentahoWebContextFilter.doFilter(PentahoWebContextFilter.java:102)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.PentahoRequestContextFilter.doFilter(PentahoRequestContextFilter.java:84)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:378)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.ExceptionTranslationFilter.doFilterHttp(ExceptionTranslationFilter.java:101)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.pentaho.platform.web.http.security.SecurityStartupFilter.doFilter(SecurityStartupFilter.java:108)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.providers.anonymous.AnonymousProcessingFilter.doFilterHttp(AnonymousProcessingFilter.java:105)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.pentaho.platform.web.http.security.RequestParameterAuthenticationFilter.doFilter(RequestParameterAuthenticationFilter.java:169)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.basicauth.BasicProcessingFilter.doFilterHttp(BasicProcessingFilter.java:174)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:278)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.logout.LogoutFilter.doFilterHttp(LogoutFilter.java:89)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.pentaho.platform.web.http.security.HttpSessionReuseDetectionFilter.doFilter(HttpSessionReuseDetectionFilter.java:134)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.wrapper.SecurityContextHolderAwareRequestFilter.doFilterHttp(SecurityContextHolderAwareRequestFilter.java:91)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:175)
at org.springframework.security.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:99)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.pentaho.ui.servlet.SystemStatusFilter.doFilter(SourceFile:72)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.pentaho.platform.web.http.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:113)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:741)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:432)
at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:347)
at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:765)
at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:574)
at org.apache.coyote.Response.doWrite(Response.java:560)
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:353)
... 69 more


Thank you very much

Problem connection database second job on server

$
0
0
Hi all,
I've a problem with a job when i call this two times in a short time.
The server where there is kettle is different with server with database but I don't think that this is a problem.

Well, when i call in shell the job the second time (the first works), I 've found this error:
2014/04/08 17:48:51 - SET VARIABLES - ERROR (version 5.0.1-stable, build 1 from 2013-11-15_16-08-58 by buildguy) :Exception: [org.pentaho.di.core.exception.KettleDatabaseException:
2014/04/08 17:48:51 -SET VARIABLES - Error occured while trying to connect to the database
2014/04/08 17:48:51 -SET VARIABLES-
2014/04/08 17:48:51 - SET VARIABLES - Invalid JNDI connection JNDI_CONNECTION : Errore di I/O: Connection reset

I don't know why first time I have no problem, it is possible that there is a problem in close connection on first call?Has anybody seen same problem?

Note: the connection with database is defined like JNDI in jbc.properties, the database is Oracle 11, driver is the last release.

Help me!

How can I print white spaces into fields when export a report in csv format

$
0
0
Hello,


I have a problem, how can I print white spaces for fields null or empty when I export a report in csv format.
I have many registers like these:


Name;LastName;Phone;Phone2;Address
Name;LastName;Phone;Phone2;Address
Name;LastName;Phone;Phone2;Address


but, if one of these field is null or empty I need fill with blank spaces to keep the format (spaces) when I open it with a text editor...


i.e. Phone2 is empty I want something like this: (Note: the underscores "_" mean blank spaces)


Name;LastName;Phone;Phone2;Address
Name;LastName;Phone;______;Address
Name;LastName;Phone;Phone2;Address


I build a function that check when a field is empty or null and replace it with blank spaces the length of field. However, when export the report in csv format I got this


Name;LastName;Phone;Phone2;Address
Name;LastName;Phone;;Address
Name;LastName;Phone;Phone2;Address


Somebody can help me please.

I use Pentaho 5.0, Pentaho Report Designer

Another Beginner Question: Add Leading Zero/Remove Trailing Zero

$
0
0
Hi Everyone,

Sorry for asking another fairly simple question, I've searched everywhere and still haven't found a solution. Here's what I am trying to accomplish in Kettle 4.4 CE (I'm going from DATABASE --> TEXT FILE), hopefully someone can help:

This is the pipe-delimited text file output I have right now:
PDT:2014-04-01/06:27:07.393 |ABCD|XX|0809123450|CrProp|amount:1.21,currency:CAD (example #1)
PDT:2014-04-01/06:27:07.393 |ABCD|XX|80912345|CrProp|amount:1.21,currency:CAD (example #2)

This is the text file output I want:
PDT:2014-04-01/06:27:07.393|ABCD|XX|080912345|CrProp|amount:1.21,currency:CAD

Which Transformation Step (or maybe other steps) do I use to Add or Remove Leading/Trailing Zero's? I just find it difficult to work with numeric values, strings are much easier. That said, hopefully I don't have to convert them string. Any help would be much appreciated!

David.

carte web services command not working properly

$
0
0
Hello,
I was able to start carte server on localhost port 8081.

Executing the transformation command http://127.0.0.1:8080/kettle/execute..._TRANSFORM.ktr works perfect and was able to monitor in carte web console.

But trying to run a JOB using http://127.0.0.1:8080/kettle/startJo...Transforms.kjb did not work and browser says "The specified job [/home/bviswana/pentaho/Jobs/RunDimTransforms.kjb] could not be found" Not sure what is wrong in the command or missing.

Referring to http://wiki.pentaho.com/display/EAI/Carte+Web+Services where it lists the services did not help and doing http://162.70.7.107:7190/kettle/exec...Transforms.kjb also error out.

Please get me some pointers as how to fire job thru URL submission to carte server. (or any other way to give to carte) We are not using DI or DB based report but all are physiical file based.

Thanks,
Balaji

Pentaho PDI 5.0.2

LINUX (LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Red Hat Enterprise Linux Server release 6.4 (Santiago) )

Java 1.7

How to add information from an Excel column into a column of a CSV step ?

$
0
0
Dear friends.

( I know my queries might very simple , therefore I hope I am not bothering anyone)

I have a table with 26 sheets, every sheet refers to a month from 2012 and 2013 and the 3 months of this year, therefore 26 sheets. This Table we receive monthly , and I need only the first column of every sheet of this table, with refers to the ID of orders which were fraud, to be added to a CSV Input step which has already a column with the same information from previous years.


I've tried to do it with the Excel Input Step, However, it didn't work and got only the sheet name . Even so, I haven 't got to select only the first column of this sheet, which refers to the “ ID-orders_fraud” as above-mentioned.

I know I could just convert the Excel file to CSV, or better, I could just convert CSV step which already exists and is feeding the ETL, to an Excel and add the first Column of IDs of the table I monthly receive doig Ctr +C and Ctr+V, and then convert this new File to CSV again, giving the same name, already with the new data from the new table.


Obviously, this is Ok for while, as I am receiving only few hundreds of cases per month, but if I start receiving 12k,20k 50k , etc new registers to add every month, I am sure that is not going to be a viable solution.

So, basically, my question is this, as you can see through my ETL :


How can I add the first column called “ pedido” (which refers to customer ID order_fraud), of every sheet from my new Excel Table, into that CSV step there (which already is made of only one column, with the same ID information, but called pedido_fraude).


Thank you in advance!

I hope the screen-shots are ok to explain the issue properly! MYfraudETL.jpgNew Table Pedidos that need to feed the table from CSV step.jpgtable that feeds the CSV step.jpg

java.lang.OutOfMemoryError: Java heap space in Excel Input Step

$
0
0
Hi,

I have this problem:

I have an Excel with 200 columns and 8000 rows.
I need to insert this information in a table.
The ETL is just an Input-Output (Excel input step and Table Output step).

I run this process in a Linux Server that have configured for Spoon 512MB and I can't change the memory.

What can I do?

Thanks for your help!!!!

Format date as yyyy-mm-dd using the Java Script Step

$
0
0
Hi all,

I have a requirement where I need to pass yesterday's date using the get system info step and assign it to a variable using Java script step and format it as yyyy-mm-dd.
Then return the formatted date of yyyy-mm-dd and use it to construct a string as

formated_date = '2014-04-07'

"/data/tmp/incomplete_payments_"+formated_date+".csv"

After constructing the string, I need to use it in the Get file from SFTP step to get the file incomplete_payments_2014-04-07.csv.

Any piece of advise on how to achieve this.

Thanks,

Ron

where is the weka log in OS X?

$
0
0
Does weka have a log file in OS X that I could see? I'm trying to connect to my mysql database, got the jdbc driver files updated, but weka isn't connecting, and there's no error message. It's just stuck on the Info box in Explorer's Open DB screen as "connecting to: jdbc:mysql://127.0.0.1:3306/analytics = true"

Call node.js in the javascript step in pentaho data integration

$
0
0
Currently I am using javascript step in pentaho data integration.
I need to use formula.js code which provides excel equivalent functions in javascript.
It internally refers a list of libraries ranging from node.js,d3.js,jstat.js and so on.
I am struggling to implement the same.
Guidance would be appreciated.
Regards, Dikesh Shah.

java.lang.StackOverflowError

$
0
0
I have a very simple transformation that uses regex to modify text in a forum database. It modifies bbcode. The transformation is a Table Input, then a Replace in String using Regex, then an Update.

The search is "\[quote="(.+?)":.+?\]((.|\s)+?)\[\/quote:.+?]". The Replace with is "(quote=$1)$2(/quote)" except with square brackets, which if I use here bbcode gets transformed.

When I run the transformation it manages to update exactly 1 record and then stops. The log is:

Unexpected Error
java.lang.StackOverflowError

then a very long list of the following repeating over and over

at java.util.regex.Pattern$GroupTail.match(Pattern.java.4227)
at java.util.regex.Pattern$BranchConn.match(Pattern.java.4078)
at java.util.regex.Pattern$CharProperty.match(Pattern.java.3345)
at java.util.regex.Pattern$Branch.match(Pattern.java.4114)
at java.util.regex.Pattern$GroupHead.match(Pattern.java.4168)
at java.util.regex.Pattern$LazyLoop.match(Pattern.java.4357)

Version 5.0.1 Stable on OS X.

PDI - Hadoop Copy Files w/Kerberos

$
0
0
Hello,

I used to be able to read/write files from/to HDFS with PDI by accessing the URI: hdfs://myserver:8020/

After securing the cluster with Kerberos it doesn't work, I believe because I need to provide some authentication.

Any ideas on how to do that?

Thanks in advance!!

Alex

Connection to Firebird DB

$
0
0
Hi,
I'm testing Pentaho server CE 5.0 and I'm trying to create a Database Connection to a Firebird db. I'm using Windows 8.1; the Pentaho server and the Firebird server run on the same machine.
I've copied the jaybird-full-2.2.4.jar inside the lib directory of Tomcat.
I've started the CE server and I've defined a connection using "Generic Database".
The settings I'm using are:
custom connection URL:
jdbc:firebirdsql:<server>[/<port>]:/<database-file>

custom driver class name:
org.firebirdsql.jdbc.FBDriver

and the user name and password.
When I test the connection I get this:


Errore di connessione al database:
org.pentaho.di.core.exception.KettleDatabaseException:
Error occured while trying to connect to the database


Error connecting to database:(using class org.firebirdsql.jdbc.FBDriver)
could not initialize class org.firebirdsql.gds.impl.GDSFactory


org.pentaho.di.core.exception.KettleDatabaseExce


I've tried the other JDBC URL as indicated in the Infocenter (JDBC Type 4, as per the jdbc driver), but it doesn't work.
Did anybody try to connect to a Firebird DB successfully?

Running sql query on a CSV file without creating a staging table

$
0
0
I have three transformations in a job.
-The first transformation reads from CSV file and populates a staging table 1
-The second transformation will run a sql script on the staging table 1 and populates another staging table2
-The third transformation creates facts and dimensions from the staging table2.

I created the staging table 1 to run the sql query on top it.

Is there a way that I can avoid creating multiple staging tables to improve performance.
Is there a way that I can run the sql query on a CSV file.

Analyzer Report prompt value after refresh

$
0
0
Hi,

I developed a dashboards which have a refresh time of 30 minnutes.
This dashboard uses a prompt which reads its data from SQL query.

In which way is possible to refresh the dashboard maintaining the last selection in the prompt?


EXAMPLE

Initial prompt value [a,b,c,d] -> Default selected value [a];

A user change the selected to [b,c]

Dashboard refresh

Prompt selection reinitialized with [a];

In which way is possible to maintain the selection after the refresh?

Thanks.

I had a mistake this a functionality alredy implemented in the prombt behaviour.

how to set columns as Measures in Pie chart?

$
0
0
Hi guys,
I need to set columns of SqlQuery as measures in Pie_Chart.


Thanks.

the query only returns 1 record :
YEAR CA Rnet
2012 20.000 80.000

Alert Message during On-click of image

$
0
0
Hi,

Can any of you help how to display an alert signle in the on-click attribute of Excel Image.

I tried javascript:alert('heeloo'); with Message Format option but on HTML output, the alert is displaying twice.

Need help in converting this Business logic into MDX

$
0
0
Hi all, I need help to write mdx for this scenario ..
Code:

table_fact:
date      type1    type2  amount    customer ... and some other dims with auto incr id
1            C        3  100          1234
2            D        2  200          1234
3            D        3  300          1234
4            C        3  100          1234
5            D        3  50            ' '
6            C        3  200            ' '
7            D        1  1000          ' '
8            D        3  100            ' '
9            D        2  500

Business rule is to get the customer id's based on type's and date with in a date period (say 2 - 8)

case 1: ratio of sum of amount of {[type1].[C], [type2].[3]} in a period to the a sum of amount of {[type1].[D], [type2].[3]
(here the condition is [type].[C] amount has to be considered after the first occurrence of [type].[D])

If the period says 2 - 8
Code:

 
                  sum{[type1].[C], [type2].[3]}    {[type1].[D], [type2].[3]        ratio
                            {date 4, 6}                        {dates 5, 8}   
customer.[1234]        300                                      150                          2.00

Even if the date range is 2- 8 type.[D] is considered only after the first occurrence of type.[C] ..
i.e dates (5, 8) are considered, 2 and 3 are not considered for sum


case 2: same as above, but the type.[C] amount is considered only after the last occurrence of type.[D]

Code:

                    sum{[type1].[C], [type2].[3]}    {[type1].[D], [type2].[3]        ratio
                                  {date 4, 6}                            {dates  8}   
customer.[1234]            300                                        100                                  3.00

i.e dates only 8 is considered for summing , 2, 3 and 5 are not considered for sum

How can I put logic in MDX to aggregate amount of type.D based on the first/last occurrence of type.C amount.

Note: I made [type1] and [type2] as degenerated dimensions, because it dont have much types (only 2 and 3 respectively)


Thanks in advance
Viewing all 16689 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>