Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live

Supplement for deprecated setProperty(String, Object) method of MasterReport

$
0
0
Hi,

In Pentaho Report Designer, a class with name org.pentaho.reporting.engine.classic.core.MasterReport has a deprecated method setProperty(String, object).
And in API DOC it was given, use the MasterReport’s parameters instead of using properties.

But we need to know what exactly these parameters refer to?
Will you please provide some sample code with these parameters to get the same functionality of setProperty() method. i.e adding a property to the report.

Regards,
Saritha.

Split single roq into multiple rows

$
0
0
Hi

I am having one table under having 20 column .As mention below structure:
Input:

1001 a b c d e
so on...
So i want to split one single row into multiple rows.How i can do ..Can any one help me how i can achieved .Any example that would be really help.
Output
1001 a b
1001 c d
1001 e

How to handle the attached file

$
0
0
I need urgent help on how to handle the attached text file with hierarchy.

For example, ISA, GS ST and AK1 are first statement in hierarchy. and then AK2 and IK5 is repeated.

ISA*00* *00* *ZZ*05-0413469 *ZZ*46-3251777 *150115*1457*^*00501*000000001*0*P*:~
GS*FA*05-0413469*46-3251777*20150115*145727*1*X*005010X231A1~
ST*999*0001*005010X231A1~
AK1*BE*4876*005010X220A1~
AK2*834*1001*005010X220A1~
IK5*A~

AK2*834*1014*005010X220A1~
IK5*A~
AK2*834*1015*005010X220A1~
IK5*A~
AK2*834*1016*005010X220A1~
IK5*A~
AK2*834*1017*005010X220A1~
IK3*N3*11*2100*8~
CTX*SUBSCRIBER NUMBER REF02:108168~
IK4*2*166*7~
IK3*N3*38*2100*8~
CTX*SUBSCRIBER NUMBER REF02:108168~
IK4*2*166*7~
IK3*N3*50*2100*8~
CTX*SUBSCRIBER NUMBER REF02:108168~
IK4*2*166*7~
IK5*R*5~

AK2*834*1018*005010X220A1~
IK5*A~
AK2*834*1019*005010X220A1~
IK5*A~
AK9*R*19*19*0~
SE*51*0001~
GE*1*1~
Attached Files

Insertion into Postgres stopping without throwing errors

$
0
0
Hi,

I am running the latest version of PDI CE (5.4.0.1) on Amazon Linux EC2 instance (m4.xlarge 4CPUs, 16GB RAM). I had devised a job to extract data from MYSQL server and write it to Postgres DB using Table Input -> Table Output in the transformation. The source query contains around 4.5 million records. However, my insertion transformation is hanging after inserting 2-2.5 million records. There is no error thrown and the kettle job just hangs for several minutes. After checking on the Postgres server side, there seems to be no pending requests sent. I have also tried playing around with the JVM heap size ranging from 256m to 2096m, changing rowset from 10,000 to 1000 and lowering the commit size to 1000, but the problem still persists. Anyone has any leads on what can be the root cause of this issue ?

Bad performance when requesting reports from the BA server

$
0
0
Hi, everybody.

I'm running into some performance issues with a report. The thing is report designer is a bit new to me and I'm having trouble to identify the reason for bad performance.

I've opened the report in my machine, in report designer, and the performance is acceptable (30 seconds, approximately, to generate a PDF). But when it's published in the server and I try to export it from the server, it takes 10 minutes, approximately to generate the same file. The classic-engine file in the server (version 5.3) has the default values.

During export, the memory consumption is relatively low, much below the limit set in CATALINA_OPTS, so I don't think the lack of resources is the reason for that. Also, the queries it performs aren't slow (they've been tested separately) and the result sets aren't very big (the biggest one returns 23 rows). The report itself isn't very big, it generates 9 pages only.

The report is parameterized, but it expects two parameters only. I'm not sure if that may impact that much in performance, but even if it does, I still don't understand why the report is slow to generate only in the server. In my notebook it's a lot faster to generate, yet its hardware is modest compared to the server where Pentaho BA server is running.

Any ideas? Is there any other test to be done?

Thanks in advance.

Inherited Community Version 4.4.0 - Changing Username/Passwords

$
0
0
Greetings,

I've inherited a Kettle integration [version 4.4.0] where some of the passwords have long since changed. I've read many threads trying to figure out what to do and have performed the following steps:
  1. used "encr.sh -kettle <new password>" to encrypt the password
  2. updated the appropriate XML line in the jkb file with the "Encrypted xxxxx" info from above


Yet when kitchen.sh runs, I still get an incorrect password error in the log. Is there another step that needs to be performed?

Thanks for helping the new guy

getting error while kitchen for runnig job in slave

$
0
0
Hi All,

I am getting error while running jobs in slave, below is error log....


-sh-4.1$ ./kitchen.sh -rep='Pentaho Dev DI Repo' -job='Main_Prod_Job_S3_Gmdm_Customer_Site_D' -dir='/home/~masyed/MDM/Producer' -user='admin' -password='password'
/u01/app/pentaho/data-integration
DEBUG: Using JAVA_HOME
DEBUG: _PENTAHO_JAVA_HOME=/usr/lib/jvm/jre
DEBUG: _PENTAHO_JAVA=/usr/lib/jvm/jre/bin/java
DEBUG: PENTAHO_INSTALLED_LICENSE_PATH=
2015/09/22 05:56:54 - Kitchen - Start of run.
2015/09/22 05:56:54 - RepositoriesMeta - Reading repositories XML file: /home/masyed/.kettle/repositories.xml
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/u01/app/pentaho/data-integration/launcher/../lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/u01/app/pentaho/data-integration/launcher/../lib/slf4j-log4j12-1.7.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2015/09/22 05:56:54 - General - Creating security provider
2015/09/22 05:56:54 - General - Creating repository web service
2015/09/22 05:56:54 - General - Creating repository sync web service
2015/09/22 05:56:55 - General - Logging plugin type found with ID: CheckpointLogTable
Refreshing GOE props...
2015/09/22 05:56:56 - General - Repository web service created
2015/09/22 05:56:56 - General - Creating unified repository to web service adapter
2015/09/22 05:56:56 - General - Synchronizing repository web service
Processing stopped because of an error:
com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 401: Unauthorized
The server sent HTTP status code 401: Unauthorized

ERROR: Kitchen can't continue because the job couldn't be loaded.



Any hints/comments on this will be appreciated

Implement if or else logic in PDI

$
0
0
How do I implement if-else logic using PDI 5.1 CE ?

Embed BA into Web Applications for that i need pentaho-sdk.zip where can i find it

$
0
0
Hi

I am Embed BA into Web Applications for that i need pentaho-sdk.zip as mention on "http://infocenter.pentaho.com/help/index.jsp?topic=%2Fintegrating_biserver%2Fconcept_reporting_examples.html" but on clicking this page don't have mention zip file can some help me with the mention zip file path.

And also if some has embed BA into web application please help me with some sample reference link.

Thanks in Advance.

XUL Exception Error when trying to connect to Pentaho Repository

$
0
0
Good day,

Im having some issues with my pentaho installation, when i try open my repository.

System Specs:
CentOS 6,
RedHat Linux 4
Pentaho 3.2
mysql-connector-java-5.1.36-bin
mySQL 5.1.73

Not sure what you need here?

When trying to connect to a repository, i get the error message: 1.jpg

Any advice ?

Can provide more info if required.
Attached Images

Pentaho Kettle has problems with custom fields in Hive table

$
0
0
Hi.
I'm trying to use Pentaho with custom defined Table in Hive.
Table is based on cloudera's twitter example. https://github.com/cloudera/cdh-twitter-example
For working with Table I'm custom (de)serializer.
The table looks so:
Code:

CREATE EXTERNAL TABLE tweets_one_day (
  id BIGINT,
  created_at STRING,
  source STRING,
  favorited BOOLEAN,
  retweeted_status STRUCT<
    text:STRING,
    user:STRUCT<screen_name:STRING,name:STRING>,
    retweet_count:INT>,
  entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
  text STRING,
  user STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
  in_reply_to_screen_name STRING
)

It is placed in proper directories on host with Hive and hadoop components.

If I try get information from Hive table with some thing like:
Code:

select count(*) from tweets_one_day

or

select id, user.screen_name from tweets_one_day where (user.followers_count > 10)

I get proper MapReduce job with results.
But if I try to get information about fields in this table with Design->Input->Table inpunt->Get SQL select Statmen->Get column names appears an error:
Code:

Couldn't get field info from [SELECT *
FROM tweets_one_day
]

Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception nulljava.lang.NullPointerException
    at org.apache.hadoop.hive.ql.exec.FetchOperator.needConversion(FetchOperator.java:607)
    at org.apache.hadoop.hive.ql.exec.FetchOperator.setupOutputObjectInspector(FetchOperator.java:578)
    at org.apache.hadoop.hive.ql.exec.FetchOperator.initialize(FetchOperator.java:172)
    at org.apache.hadoop.hive.ql.exec.FetchOperator.<init>(FetchOperator.java:140)
    at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:79)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:455)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112)
    at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1106)
    at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100)
    at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:171)
    at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
    at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:385)
    at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
    at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
    at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
    at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
    at com.sun.proxy.$Proxy17.executeStatementAsync(Unknown Source)
    at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271)
    at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:490)
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
    at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
    at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
    at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

I think it happens because Pentaho Kettle has no my (de)serializer.
How can I add (de)serializer in Pentaho.
Or I should do changes in hadoop cluster?

CDE + Require = Slow dashboard load

$
0
0
Hi there,

We're testing the new AMD support, in order to evaluate its usage in a future development and we're having some problems regarding the performance of the dashboards.
The JS load seems quite heavy when activating the 'require' flag (in an almost empty dashboard). 2 vs 61 JS files.

I'm no JS expert so as to evaluate the CDF code and understand / modify possible workarounds, but just wanted to know if this is something you're working on. As far as I know, this flag is optional in the 5.4 version, but will be mandatory in the forthcoming Pentaho 6.

Is there any flag we might modify in order to pack this files? Or maybe we're having some incorrect approach whereas developing the dashboard?

Thanks in advance

HTTP request binary zip data to file!

$
0
0
OK, so got really stuck on this one, hoping someone might have had a similar problem!

I am looking to download file data from eBay's API, data that takes the shape like below. I figured this is base64 encoded data so must need to decode it first, then take that binary data to create a zip file, then unzip this file to see the plain text results. I also know i need to use javascript or java to do so. Found these resources so far:

http://type-exit.org/adventures-with...lobs-to-files/
http://forums.pentaho.com/showthread...ring-to-base64

Hoping someone might be able to offer some pointers - am i on the right track, can this all be done in one script perhaps?

Here is the HTTP response:

Code:

--MIMEBoundaryurn_uuid_A04119B5D0991CCD3114430440928722218931Content-Type: application/xop+xml; charset=utf-8; type="text/xml"
Content-Transfer-Encoding: binary
Content-ID: <0.urn:uuid:A04119B5D0991CCD3114430440928722218932>


<?xml version='1.0' encoding='UTF-8'?><downloadFileResponse xmlns="http://www.ebay.com/marketplace/services"><ack>Success</ack><version>1.1.0</version><timestamp>2015-09-23T21:34:52.872Z</timestamp><fileAttachment><Size>323</Size><Data><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:urn:uuid:79DA9B4960BBAEEED81443045362881"/></Data></fileAttachment></downloadFileResponse>
--MIMEBoundaryurn_uuid_A04119B5D0991CCD3114430440928722218931
Content-Type: application/zip
Content-Transfer-Encoding: binary
Content-ID: <urn:uuid:79DA9B4960BBAEEED81443045362881>


PK  ?t6G?uL?  ?  50010323875_report.xmlUT        ??V??Vux 0  0  u?;?0D???r?CV>?@?D?,f        ???:Vr{?i????eU??>P?O??VH.?=
???gw?][?v7?p_?h?9bd%}bQ|?j z6h??a)???????????cHY?a3?|m??PK  ?t6G?uL?  ?        ??    50010323875_report.xmlUT ??Vux 0  0  PK    \  ?   
--MIMEBoundaryurn_uuid_A04119B5D0991CCD3114430440928722218931--

Any suggestions would be appreciated.

Can't connect to repository - Failed to access the wsdl

$
0
0
Hello.

We have 2 servers running separate repositories on localhost. One of these suddenly stopped connecting to the repository and we have no idea why, since we've changed nothing on our end.

Here are some snippets from pentaho.log:

2015-09-15 10:28:32,722 ERROR [org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager] Failed to persist ChangeLog (stacktrace on DEBUG log level), blockOnConnectionLoss = false
org.apache.jackrabbit.core.state.ItemStateException: PANIC: could not write to log file 13, segment 26 at offset 9592832, length 32768: Permission denied

2015-09-15 10:28:42,853 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] failed to close Connection
2015-09-15 10:28:42,853 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] Reason: Already closed.
2015-09-15 10:28:42,853 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] State/Code: null/0
2015-09-15 10:28:42,853 ERROR [org.apache.jackrabbit.core.journal.DatabaseJournal] failed to end batch

2015-09-15 10:28:43,898 ERROR [org.apache.jackrabbit.core.util.db.ConnectionHelper] Failed to execute SQL (stacktrace on DEBUG log level)
org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

2015-09-18 00:01:42,745 INFO [org.pentaho.di] 2015/09/18 00:01:42 - General - Logging plugin type found with ID: CheckpointLogTable
2015-09-18 00:02:15,135 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] failed to close Connection
2015-09-18 00:02:15,135 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] Reason: Already closed.
2015-09-18 00:02:15,135 ERROR [org.apache.jackrabbit.core.util.db.DbUtility] State/Code: null/0
2015-09-18 00:02:15,135 ERROR [org.apache.jackrabbit.core.persistence.pool.BundleDbPersistenceManager] Failed to persist ChangeLog (stacktrace on DEBUG log level), blockOnConnectionLoss = false
org.apache.jackrabbit.core.state.ItemStateException: PANIC: could not write to log file 13, segment 26 at offset 9658368, length 32768: Permission denied

And catalina log:

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\Pentaho\java\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;.

Sep 22, 2015 1:32:23 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
Sep 22, 2015 1:32:24 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/pentaho-di] registered the JDBC driver [org.h2.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Sep 22, 2015 1:32:24 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc

INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\Pentaho\java\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;.


Thanks for reading. Any pointers at all would be greatly appreciated.

Adam

MongoDB queries

$
0
0
Hi Pentaho,

I am using the Kettle api to integrate with multiple data sources.

Issues i have encountered while in development

1. Insertion to MongoDBOutput takes 2 Min of time with input as CSV file( File contain 1 million of records) while with oracle DB insertion just takes (0.4 Min).
2. Why we dont have MongoDBOutput, MongoDBDelete not implemented to SharedObject. My requirement is i have one MongoDB but different collections and i have to insert record based on the some criteria to different collections.

Set and Member in WHERE CLAUSE in MDX ?

$
0
0
Hi Team,

I am spinning my head around in MDX querying since a day and i finally understood how to pass parameters to the MDX query ? But when i am trying to slice the cube based on MEMBER and a SET in the WHERE clause i am facing No function matching error. I understood that we cant use Member and Set in where clause at the same time.

Can someone please let me know how to get the condition working (Using Member and SET filters running on the same query ? ). Any help is highly appreciated.

Thanks in advance,
Santosh

PDI 4.4 Plugin Development IMetaStore support

$
0
0
Hello Community,

Currently I am developing plugin in that <PluginName>Meta classes uses IMetaStore as method parameter. I think metastore support start with pentaho 5.0 onward.

So, If I want to use metastore support in PDI 4.4, How can i use it ?

Some of method declare in

PDI 5.X

Code:

public void loadXML( Node stepnode, List<DatabaseMeta> databases, IMetaStore metaStore)

public void getFields(RowMetaInterface row, String origin, RowMetaInterface[] info, StepMeta nextStep, VariableSpace space, Repository repository, IMetaStore metaStore) throws KettleStepException

public void readRep(Repository rep, IMetaStore metaStore, ObjectId id_step, List<DatabaseMeta> databases) throws KettleException
public void saveRep(Repository rep, IMetaStore metaStore, ObjectId id_transformation, ObjectId id_step) throws KettleException

public void check(List<CheckResultInterface> remarks, TransMeta transMeta, StepMeta stepMeta, RowMetaInterface prev, String input[], String output[], RowMetaInterface info, VariableSpace space, Repository repository, IMetaStore metaStore)

public void analyseImpact(List<DatabaseImpact> impact, TransMeta transMeta, StepMeta stepMeta, RowMetaInterface prev,
      String[] input, String[] output, RowMetaInterface info, Repository repository, IMetaStore metaStore)


PDI 4.4

Code:


public void loadXML(Node stepnode, List<DatabaseMeta> databases, Map<String, Counter> counters)


public void getFields(RowMetaInterface row, String origin, RowMetaInterface[] info, StepMeta nextStep, VariableSpace space) throws KettleStepException


public void readRep(Repository rep, ObjectId id_step, List<DatabaseMeta> databases, Map<String, Counter> counters) throws KettleException
public void saveRep(Repository rep, ObjectId id_transformation, ObjectId id_step) throws KettleException

public void check(List<CheckResultInterface> remarks, TransMeta transMeta, StepMeta stepMeta, RowMetaInterface prev, String input[], String output[], RowMetaInterface info)
public void analyseImpact(List<DatabaseImpact> impact, TransMeta transMeta, StepMeta stepMeta, RowMetaInterface prev, String[] input, String[] output, RowMetaInterface info) throws KettleStepException

Thanks & Regards
Mehul C

Left Outer Join - Not Working with Excel Input

$
0
0
Hi Folks,

I m using Kettle 4.4 community edition stable.

I have 2 excel files as input.
1. Cust_Master contains cust_id column
2. Some financial data with cust_id ​column
Now I am using Rename/Select and Sort steps for the Merge Join step to work.
Merge join takes both excel files and applies a left out join on cust_id (As per my requirement, left outer join on Cust_Master)
Everything works fine and the whole transformation is completed with errors.

Screenshot has been attached of the transformation for reference.

Hope I get respond as you all always do.
Thanks in Advance!!!
Attached Images

Pentaho Pie Chart

$
0
0
I try to add a pie chart to my report. In the report designer I've added the pie chart and configured the value and data series. My problem is that the legend shows always the value instead a readable caption.

How can I configure a legend name for a data value?

For example:
The value count is 75. There are three data points with value 25. The legend shows three times the label 25. Instead of showing the values in the legend I want to define a readable caption for the data.

Insert Row

$
0
0
All I need to do is insert one row and set some values for that row. I cannot find any way to do this easily. Should this probably just be done with a Javascript?

Something that sounds like it should be so easy is actually turning out to be confusing.
Viewing all 16689 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>