Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live

How to upload kettle jobs and transformations in the Pentaho server?

$
0
0
Hi,

We are using Red hat Linux server, we want to run our ETL'S in Linux server without GUI.For that we want to use Pentaho server and how we can move our kettle jobs and transformations to Pentaho server and how to use Webspoon to access in our local machines.

Thank you

Detect modification in group of rows for snapshot table

$
0
0
Hello community!

In a snapshot table I need to detect when the source changed a single row among millions of rows. If a single row is deleted or a new one is inserted, I need to recreate a snapshot table.

The snapshot table is like this one:

SNAP_MIX:
mix_id INTEGER
store_id INTEGER
item_id INTEGER
version_number INTEGER
date_from TIMESTAMP
date_to TIMESTAMP

This refers to a predefined list of items that a unique store sells (a store is part of a company with many stores, but this is not relevant now).

When a new item is added or removed the snapshot must be updated, saving all previous states for future reference - managers want to know how many and what items are included and removed from the mix over time.

The question is, how can I detect when a new row is removed or a new one is created? By counting the number of rows I cannot detect when one item is removed and another one is added (with another item_id, for example). Also, I tried the Dimension Lookup step without success - because it checks for changing row by row, not in a group of rows.

I'm thinking to generate a hash value of all item_id for a given store_id, but I can't find how to do this. Is there any transformation or another way to perform this kind of change in rows?

Article 1

$
0
0
Hi,

we are getting data as special characters like this hand symbol [COLOR=#333333][FONT=&quot].

my bad luck i am not able to post the images or links in the forum

Insert script generation in order

$
0
0
Hi All,

i want to generate insert scripts in an order.
I have one parent and one child table.
There are data in these tables and want to generate insert scripts like below.


I am not able to generate scripts in the below order.I tried to loop .But i cannot loop in single rows.
Can anyone give any idea ?
How can we loop one single row and generate one row from child table.
Then return to parent for next single row, then child etc .
Another problem is i tried sql output step;But cant create "seq_p_table.nextval" without quotes.


Parent
------------


table_name: p_table
Id name
1 a
2 b
3 c


table_name :c_table
Id marks p_id
1 10 2
2 30 3
3 40 1


Id in table is a sequence.FK relations are present p_table.Id is FK to c_table.p_id
Script should generate in a file like


insert into p_table(id,name) values (seq_p_table.nextval,'a');
insert into c_table(id,marks,p_id) values(seq_c_table.nextval,10,seq_p_table.currval);


insert into p_table(id,name) values (seq_p_table.nextval,'b');
insert into c_table(id,marks,p_id) values(seq_c_table.nextval,30,seq_p_table.currval);


insert into p_table(id,name) values (seq_p_table.nextval,'c');
insert into c_table(id,marks,p_id) values(seq_c_table.nextval,10,seq_p_table.currval);


I am not getting any idea .Can anyone help to get?
I am using GA 7.1 version


Thanks.

When this bug correction take place?

Special Characters

$
0
0
Hi,

we are getting data as special characters like this hand symbol [COLOR=#333333][FONT=&quot].

my bad luck i am not able to post the images or links in the forum

Dynamic Authorization Key for Rest Client

$
0
0
I have an API that I'm trying to hit and pull back data but the problem is that the authorization token that I have to pass in/use to my Rest Client step is dynamic. I.e., I have to make a HTTP Post call to create a new token and then use that new token each time I want to do a "Get" call within my Rest Client step. I have no problem doing the HTTP post and getting the authorization key each time. Currently I am outputting that key to a json file but I can put it in a database or wherever is needed, if needed.

My question is: how do I pass this dynamic authorization key to my Rest Client step? Generally, a Rest Client step has to have a "Generate Rows" step before it, but I can't seem to pass a dynamic value to the "Generate Rows" step. If I could my problem would be solved. Any ideas?
Thanks for the help!

Regex Evaluation capture group

$
0
0
I can never get the capture groups to work.

Start with something simple:

1) Generate 1 row with 1 field, let's say TEST = "123"
2) Connect to Regex Evaluation step

Create fields to capture groups CHECKED
Field to evaluate: TEST
Regular expression: (.*)

This should capture all into one group. Define the single group as a new field, using String as type.

Result is negative, the capture is <null>. Any help?

Need PDI 4.3 EE Installer

UDJC causing hang in mapping transformation

$
0
0
I have a basic transformation that calls a mapping (sub-transformation)
The User Defined Java Class step in the mapping causes the job to hang, even though it seems like it's completed.
The UDJC step is basic, and the transformation ends ok if I remove it.

What do I need to do for this to end properly with the UDJC?

Thanks.

[I don't have permissions to upload/attach the transformations]

How to translate CDE components?

$
0
0
I have one Pentaho CDE dashbord. This dashboard has some date input component and table. I need to translate this components on Russian. How I can do this? I try to search such theme in Google, but I haven't founded anything useful. Can you help me? I am really need russian language on my dashboard!

How to translate CDE components?

$
0
0
I have one Pentaho CDE dashbord. This dashboard has some date input component and table. I need to translate this components on Russian. How I can do this? I try to search such theme in Google, but I haven't founded anything useful. Can you help me? I am really need russian language on my dashboard!

UserDefinedFunction works fine in Pentaho EE but not in Pentaho CE

$
0
0
For internal use, we installed the Pentaho CE version locally using the olapFormatter.jar library which defines a UserDefinedFunction.
The library was installed in "pentaho-server-ce-7.0.0.0-25 / pentaho-server / tomcat / webapps / pentaho / WEB-INF / lib" as it was installed on the Pentaho EE.
In Pentaho EE it works well, while in Pentaho CE when I try to load the model with the UserDefinedFunction, this is not displayed in the model form?
Both olap models loaded on both the Pentaho EE and the Pentaho CE are identical.
No errors appear from the logs.

If I load the model without the UserDefinedFunction, it is works fine!
Can you help me please?


Regards
Gianluca



Excel input Step- Spread type engine

$
0
0
Hi All,

I have 3 excel files with same format. I am doing some transformations on it. I execute it in loop.
I have selected Spread sheet Type engine as "Excel 2007 Apache POI streaming" in excel input step. It works fine for 2 excel files. But for 1 excel file it does not return any rows.
For that 1 excel file it works with "Excel 2007 Apache POI" Spread sheet Type engine. why so?
What is the diffrence between "Excel 2007 Apache POI streaming" and "Excel 2007 Apache POI" Spread sheet Type engines?
I noticed "Excel 2007 Apache POI streaming" works faster.
So How can I make Spread sheet Type engine as "Excel 2007 Apache POI streaming" work for all 3 excels?
I use penhato 7.1. Windows 7. JRE 1.8

please help
Thanks

Dynamic Input Normalization

$
0
0
Problem
I got a repetitive task to normalize data, for example, i got a csv or an excel of

id ; name ; age; address; phone
1; ane; 19; NY; 05547886

and
id ; name ; job; saving
3; ena; CivServ; $900

I use PDI(kettle) to normalize it to
id; name; info_type; data
1;ane; age; 19
1;ane; address; NY
1;ane; phone; 05547886
3; ena; job; CivServ
3; ena; saving;$900

Question
Is there a way to do it automatically, so i don't have to do a get field in Row Normaliser for each data i have? as i see we can't use variable to fill the fieldname and type

Thanks in advance

Configuration of mail server in pentaho BI

$
0
0
Hi to all, I’m having troubles with the configuration of mail server in pentaho BI. I’ve replaced the content of my config .xml file with this code and still doesn’t work.


Code:

<?xml version="1.0" encoding="utf-8"?>
<email-smtp>
<properties>
<mail.smtp.host>myHostServer</mail.smtp.host>
<mail.smtp.port>465</mail.smtp.port>
<mail.transport.protocol>SMTP</mail.transport.protocol>
<mail.smtp.starttls.enable>false</mail.smtp.starttls.enable>
<mail.smtp.auth>true</mail.smtp.auth>
<mail.smtp.ssl>true</mail.smtp.ssl>
<mail.debug>false</mail.debug>
<mail.smtp.quitwait>false</mail.smtp.quitwait>
</properties>

<mail.from.default>existingmail@existingmail.com</mail.from.default>
<mail.from.name>realmail@realmail.com</mail.from.name>
<mail.userid>USER</mail.userid>
<mail.password>PASSWORD</mail.password>
</email-smtp>

I’ve checked the user, password,hostname and mails and all of them are correct. Even if I try to configure the mail server from PUC → Administration → Mail Server it doesn’t work.

I’ve been checking other post with the same problem but there no was solution for me.

For more information here is my log too. Thanks in advance!

Code:

2018-08-23 14:37:19,873 ERROR [org.pentaho.platform.plugin.services.email.EmailService] Unable to send, email not configured correctly.
javax.mail.MessagingException: Exception reading response;
  nested exception is:
    java.net.SocketException: Connection reset
    at com.sun.mail.smtp.SMTPTransport.readServerResponse(SMTPTransport.java:2202)
    at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:1939)
    at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:654)
    at javax.mail.Service.connect(Service.java:317)
    at javax.mail.Service.connect(Service.java:176)
    at javax.mail.Service.connect(Service.java:125)
    at javax.mail.Transport.send0(Transport.java:194)
    at javax.mail.Transport.send(Transport.java:124)
    at org.pentaho.platform.plugin.services.email.EmailService.sendEmailTest(EmailService.java:187)
    at org.pentaho.platform.web.http.api.resources.EmailResource.sendEmailTest(EmailResource.java:172)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
    at org.pentaho.platform.web.servlet.JAXRSServlet.service(JAXRSServlet.java:109)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
    at org.pentaho.platform.web.servlet.JAXRSServlet.service(JAXRSServlet.java:114)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.PentahoWebContextFilter.doFilter(PentahoWebContextFilter.java:223)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.PentahoRequestContextFilter.doFilter(PentahoRequestContextFilter.java:87)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.pentaho.platform.web.http.security.RequestParameterAuthenticationFilter.doFilter(RequestParameterAuthenticationFilter.java:191)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:158)
    at org.pentaho.platform.web.http.security.PentahoBasicProcessingFilter.doFilterInternal(PentahoBasicProcessingFilter.java:125)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.pentaho.platform.web.http.filters.HttpSessionPentahoSessionIntegrationFilter.doFilter(HttpSessionPentahoSessionIntegrationFilter.java:271)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:169)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.SystemStatusFilter.doFilter(SystemStatusFilter.java:55)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:114)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.WebappRootForwardingFilter.doFilter(WebappRootForwardingFilter.java:70)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.pentaho.platform.web.http.filters.PentahoPathDecodingFilter.doFilter(PentahoPathDecodingFilter.java:34)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:94)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:502)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1132)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:684)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1539)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1495)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at com.sun.mail.util.TraceInputStream.read(TraceInputStream.java:124)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
    at com.sun.mail.util.LineInputStream.readLine(LineInputStream.java:89)
    at com.sun.mail.smtp.SMTPTransport.readServerResponse(SMTPTransport.java:2182)
    ... 98 more

Unable to populate Oracle table with Oracle Bulk Loader

$
0
0
Hi,

I am trying to populate oracle table with Oracle Bulk Loader step. The source is an SQL Server table and there is one varbinary/blob column
that contains pdf document. Whenever I check the data stream using the Preview feature of the GUI I see the correct data, but when the data
landed in data file it us somehow truncated and I can see very different data between the <startlob> and <endlob> tag.
So wrong data bulk load into the Oracle table I use oracle 12.0.1 oracle client and PDI 7.1 on windows computer

Any idea?

Jozsef

Log missing feild in excel input file

$
0
0
Hi All,
I am using excel input step, I want to log a error in new text/excel file if some columns are missing in the input excel file saying "Some columns are missing in the input file" and stop execution of transformation and respectively stop/abort main job.
How can this be achieved in pentaho 7.1 ce?


Please help
Thanks

Could Mondrian work with galaxy schema?

$
0
0
Hi All,

Could you please tell, could Pentaho Schema Workbench or Mondrian work with galaxy schema?

I mean, if there are several Fact tables connected with each other and I want to build measures from each Fact table without creating any Views.
Don't want to use any Views since it seems like a workaround rather than a right solution.
Also Pentaho Aggregation Designer cannot work with Views.

With best regards,
Bagdat

Copy results to parameters - Pentaho 8.1

$
0
0
"Copy results to parameters" in transformation job entry is not working in pentaho 8.1, but working fine with 7.1 & 8.0

My master job triggers two transformation Tr1 & Tr2.

Tr1 generate a single row, which is being copied to results.

The second transformation Tr2 has a parameter defined within it. With getVariables step, the parameter is read as stream and printed using writeToLog. In transformation properties within master job "Execute every input row" & "Copy results to parameters" and "Stream Column Name" & parameter mapping is specified.

But it is not working as in 7.1 and 8.0.


MasterJob :- start -> Tr1 -> Tr2
Tr1 :- GenerateRows (1 row stream) -> CopyRowsToResult
Tr2 :- GetVariables -> WriteToLog

Kindly advice.
Viewing all 16689 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>