Quantcast
Channel: Pentaho Community Forums
Viewing all 16689 articles
Browse latest View live
↧

Error handling in Vertica bulk loader

April 23, 2018, 11:43 pm
$
0
0
hi,

I am currently using VERTICA BULK LOADER to load into Vertica from a file. it is working gud... but i thought of adding a additional feature of error handling to it(Error Handling).
FEATURE: Any error in the file should be captured and it should do load for the correct records.
So disabled Abort on error option from VERTICA BULK LOADER, but still the transformation is getting abort with an error in file.

pentaho version used 8.0
↧
Search

"Get file names" on HDFS

April 24, 2018, 2:23 am
$
0
0
Hi all,

I am trying to run a simple "Get File Names" step on HDFS with PDI 8.0 + JDK 8. Although my transformation is working on Win10 I am facing an error on Linux machines :

Quote:
org.apache.commons.vfs2.FileNotFolderException: Could not list the contents of "file:///home/pdi/software/pdi_8.0.0/hdfs:/mynamenode:50070/tmp" because it is not a folder.
Obviously my given Folder
Quote:
hdfs:///mynamenode:50070/tmp
is not respected correctly.

I am using a filemask ".*". When removing the filemask and adding a filename to the File/Foldername the error disappears but there is still no result.

Any ideas ?
Thank you very much
Peter
↧
↧

Modified Javascript Step - please help

April 24, 2018, 9:37 am
$
0
0
Hi

A junior member here be gentle

I want to create a step which checks the contents of a excel spreadsheet to create a address. The idea is to ignore null cells and just create address based on values given.

I cannot get the step to work

the script is


var NewAddress = "";




if(Head Office Address Line 2 = null)
{NewAddress = Head Office Address Line 1 + "," + Head Office City;}

yet when it complies it errors with the following

2018/04/24 17:30:01 - Head Office.0 - ERROR (version 7.0.0.0-25, build 1 from 2016-11-05 15.35.36 by buildguy) : org.pentaho.di.core.exception.KettleValueException:
2018/04/24 17:30:01 - Head Office.0 - Couldn't compile javascript:
2018/04/24 17:30:01 - Head Office.0 - missing ) after condition (script#6)
2018/04/24 17:30:01 - Head Office.0 -

I cannot see what the issue is as there is a ) on line 6

Can someone help

Thanks in advance
Chirag
↧

Timing issue in Execute SQL Statement?

April 24, 2018, 11:30 am
$
0
0
Hello, I am seeing an intermittent issue when running the Execute SQL Statement command (Data Integration v7.1.0.2) in a Kettle Job. In the logs, I am seeing message "Triggering heartbeat signal for <job> at every 10 seconds" repeatedly.

Thread dump details follow.

Thread-38 appears to be stuck due to waiting for a lock on DriverManager. I am not familiar enough with the code in FelixStartThread to know what it is doing. Is this an issue with the way I am using Kettle, or something else?

------

"Thread-38" #68 prio=5 os_prio=0 tid=0x0000000058956800 nid=0x1524 in Object.wait() [0x00000000633ce000]
java.lang.Thread.State: RUNNABLE
at org.mariadb.jdbc.Driver.<clinit>(Driver.java:69)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.pentaho.di.core.database.Database.connectUsingClass(Database.java:508)
- locked <0x0000000081e4bd20> (a java.lang.Class for java.sql.DriverManager)
at org.pentaho.di.core.database.Database.normalConnect(Database.java:456)
at org.pentaho.di.core.database.Database.connect(Database.java:370)
- locked <0x0000000081f2e6b8> (a org.pentaho.di.core.database.Database)
at org.pentaho.di.job.entries.sql.JobEntrySQL.execute(JobEntrySQL.java:243)
at org.pentaho.di.job.Job.execute(Job.java:724)
at org.pentaho.di.job.Job.execute(Job.java:865)
at org.pentaho.di.job.Job.execute(Job.java:865)
at org.pentaho.di.job.Job.execute(Job.java:546)
at org.pentaho.di.job.Job.run(Job.java:436)

"FelixStartLevel" #17 daemon prio=5 os_prio=0 tid=0x0000000059e1a800 nid=0x1784 in Object.wait() [0x000000005b44c000]
java.lang.Thread.State: RUNNABLE
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at java.sql.DriverManager$2.run(DriverManager.java:603)
at java.sql.DriverManager$2.run(DriverManager.java:583)
at java.security.AccessController.doPrivileged(Native Method)
at java.sql.DriverManager.loadInitialDrivers(DriverManager.java:583)
at java.sql.DriverManager.<clinit>(DriverManager.java:101)
at org.pentaho.big.data.api.jdbc.impl.ClusterInitializingDriver$$Lambda$18/100754647.registerDriver(Unknown Source)
at org.pentaho.big.data.api.jdbc.impl.ClusterInitializingDriver.<init>(ClusterInitializingDriver.java:79)
at org.pentaho.big.data.api.jdbc.impl.ClusterInitializingDriver.<init>(ClusterInitializingDriver.java:70)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.aries.blueprint.utils.ReflectionUtils.newInstance(ReflectionUtils.java:329)
at org.apache.aries.blueprint.container.BeanRecipe.newInstance(BeanRecipe.java:962)
at org.apache.aries.blueprint.container.BeanRecipe.getInstance(BeanRecipe.java:331)
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:806)
at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787)
at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)
at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:245)
at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:682)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:377)
at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:269)
- locked <0x0000000081a7c978> (a java.util.concurrent.atomic.AtomicBoolean)
at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:294)
at org.apache.aries.blueprint.container.BlueprintExtender.createContainer(BlueprintExtender.java:263)
at org.apache.aries.blueprint.container.BlueprintExtender.modifiedBundle(BlueprintExtender.java:253)
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:500)
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.customizerModified(BundleHookBundleTracker.java:433)
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$AbstractTracked.track(BundleHookBundleTracker.java:725)
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$Tracked.bundleChanged(BundleHookBundleTracker.java:463)
at org.apache.aries.util.tracker.hook.BundleHookBundleTracker$BundleEventHook.event(BundleHookBundleTracker.java:422)
at org.apache.felix.framework.util.SecureAction.invokeBundleEventHook(SecureAction.java:1127)
at org.apache.felix.framework.util.EventDispatcher.createWhitelistFromHooks(EventDispatcher.java:696)
at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:484)
at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4429)
at org.apache.felix.framework.Felix.startBundle(Felix.java:2100)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1299)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)
at java.lang.Thread.run(Thread.java:748)
↧

Yet another decimal separator post

April 26, 2018, 8:35 am
$
0
0
Easy: I want to change the decimal separator from dot "." to comma "," in a CDE table chart dashboard. However, so far I haven't been able to.

I've tried changing the locale to different spanish (es_MX, es, es_ES...) in my browser without much success. Other than that there is not much else I've done.

Is there any property in table charts that I'm missing? Maybe a combination of column types and column formats to solve it?

I don't really understand why this isn't as easy as it is in the Data Integration tool or, if it really is and I am missing it completelly.

Thanks
↧
↧

how to lock and increment a variable ot prevent data race

April 26, 2018, 9:00 am
$
0
0
I have a step that I call with 10 number of copies. With each result, I'd like to increment a variable.

Is it possible to somehow create a transformation specific lock to use when each result is incrementing the variable?
↧

Set CSV Input process one by one row

April 27, 2018, 1:57 am
$
0
0
How to set PDI process data using CSV input row one by one (next row if previous row finish processing)

iam using PDI 5.3 CE
OS windows 8 LE
↧

How to set connection option globally?

April 27, 2018, 5:41 am
$
0
0
I need to add a connection parameter in the "options" tab for Mysql db connection inside transformations. I want to define the database connection once in a transformation that sets this connection based on an external config file. I can set the option in the connection of this transformation, but it is not shared to other transformations executed later in the job. I can set connection properties like IP, user, password and port as variables. Is the same possible for the connection option?

Any other way to set connection option globally for the entire program?
↧
Search

Output CSV file size issue

April 27, 2018, 6:11 am
$
0
0
Hi all,

Total table in records:26

table input to CSV output file : Output CSV file size 7GB and same use the table input to Excel output file : Output Excel file size 20KB .
any solution how to resolve the CSV output file size.


Thanks
Manoj
↧
↧

Variables frustration

April 27, 2018, 9:04 am
$
0
0
Hi all,

I'm getting mad on PDI variables.

I just need to set a variable in a job, then after a Trasformation increment it by 100 and then run that Trasformation again for n times.

So if the flow will run 4 times I expect to have my variable set it to 300 (0, 100, 200, 300).

I'm using the Set Variable step with this syntax :

VAR = ${VAR}+100

What I get at the end is 0 + 100 + 100 + 100...

How can I achieve this ?

Any help will be greatly appreciated.


:mad::mad::mad:
↧

SingleValueQuery Expression not working in pentaho report designer 8.0

April 29, 2018, 10:13 pm
$
0
0
I am migrating reports from PRD 6.1 to PRD 8.0, and singlevaluequery expression is not working, is it a bug or do i have to do any extra configuration?
↧

Evaluate number of rows in a table step

April 30, 2018, 6:03 pm
$
0
0
I am using this step - "Evaluate number of rows in a table" in PDI 8.0 to determine whether or not data exists. I have written this SQl Script below, hoping that would help determine if rows are empty.

SELECT COUNT (*)
From TABLE
GROUP BY SSN
HAVING count(*) = 0

If I want that SELECT statement to be true then I could select "Equal to" next to Success when rows do not exist? I tried that method, assuming it would send to next step when there are no existing rows (hop set to true) but
it doesn't seem to work that way? I also thought that if this is "false", meaning rows do exist, it would send to different step (hop set to false)
I don't understand what these conditions mean - "Different from", "Smalle than", "Smaller than or equal to" ?
Can you elaborate?

Thanks,
Claudia
↧

Set Variable and Get Variable in same transformation

May 1, 2018, 4:20 am
$
0
0
I have never seen Set Variables step and Get Variables step use case in single transformation together but it is working if it's called from a parent job. My current version is PDI 7.1 and it's working on Carte server as well.

Ktr: Set Variables (parent level) > Get Set Variables.
↧
↧

Filtering lines between an header and a footer in a multirecord file

May 3, 2018, 7:40 am
$
0
0
Hi,

I need help on this topic, please.
I have a multirecord text file that has different sections. I would split every section and send it in an output file step or something similar.
The file has a similar structure:

01 .. data header 1..
02 .. data detail block1..
02 .. data detail block1..
99 .. data footer 1..
01 .. data header 2..
02 .. data detail block2..
02 .. data detail block2..
02 .. data detail block2..
02 .. data detail block2..
99 .. data footer 2..

Headers and footers have the same beginning characters (01 and 99) and all details have always the same beginning characters (02) but fields and data have different sructures

How could split the different blocks in different file output?

Thank you in advance!
Fernando
↧

Pie chart decimalvalues

May 3, 2018, 9:23 am
$
0
0
How do you display pie chart values with decimals?
↧
Search

bar chart display label vertically space between bar and label

May 3, 2018, 10:19 am
$
0
0
I want to display labels in a bar chart vertically. How do you put space between bar and label? When I put rotation as 270, the label is starting from insider of bar.
↧

Pentaho variable in SQL script?

May 3, 2018, 10:55 am
$
0
0
I'd like to use the @uid = SCOPE_IDENTITY() result of one SQL script INSERT commands in another one later down the stream.
How do I accomplish that? Can I set and use Pentaho variables inside my SQL script?
↧
↧

pdi regular expression if else scenario (or capture groups)

May 3, 2018, 2:14 pm
$
0
0
Hi folks,

I am trying to get the datetime part from a list of files

names can be like given below :

a_b_c_02022018122555.txt
a_b_c_02_2345_02022018122555_123.txt
abc02022019122555.txt
abc_pqr_02022019.txt
abc_pqr_02022019_123.txt

I need a common regex to retrieve output as follows . Basically this is equivalent to getting if(14digit_timestamp) then (get 14digit_timestamp) else (8digit_timestamp). I was trying something like (.*)([0-9]{14})[0-9]{14}|[0-9]{8}.* and cant make it work..
02022018122555
02022018122555
02022019122555
02022019
02022019
↧

Access CategorySet Data Collector

May 3, 2018, 7:00 pm
$
0
0
Hello everyone,

at the moment I stuck at trying to access a created function.

I created a new function (CategorySet Data Collector) and want to get access on it from my post-scripting which I use at a bar chart.

dataRow.get(...) doesn't work with CategorySet Data Collector.

Is there any function which allows me to do that?
↧

PDI 7.1 XML return ok - PDI 8.0 XML bad return

May 4, 2018, 12:44 am
$
0
0
Hi,

Using PDI 7.1, I send to a XML server this: http://10.133.37.12?var=1U5-TRAFO%20A.AE&begin=31032018220000&end=30042018215959&period=ALL

I get this answer: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><recordGroup><period>2591999</period><record><dateTime>31032018220000</dateTime><field><id>1U5-TRAFO A.AE</id><value>177254.911000</value></field></record></recordGroup>

Using PDI 8.0, the same file, it changes the "%20" for "+": http://10.133.37.12?var=1U5-TRAFO+A.AE&begin=31032018220000&end=30042018215959&period=ALL
I get a bad answer: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><recordGroup><period>2591999</period></recordGroup>

I'm not an expert and this is freaking me out :confused:.

These are the steps:

Thanks.
↧
Viewing all 16689 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>