Pentaho Orinet DB Connection

February 20, 2018, 4:19 am

≫ Next: BulkLoad into MSSQL, import data from \\PC\folder\file.csv

≪ Previous: Bug in pentaho data integration 8

$

0

0

Hi Team,
Is it possible to connect to Orinet DB from pentaho Community edition. I'm trying to load and then update data in Orient DB. I copied all the jar files from orient DB lib and placed in data-integration\lib folder. Can someone post connection string example. I don't see it listed in table input step Connections. Can i make use of generic database connection for this? Any input will be helpful.


Regards,
G.Sujay.

---

BulkLoad into MSSQL, import data from \\PC\folder\file.csv

February 20, 2018, 5:39 am

≫ Next: Best way to handle many parallel transformations in a job

≪ Previous: Pentaho Orinet DB Connection

$

0

0

Hello guys.

I've been trying to bulk a file into MSSQL from a sharing folder in windows that looks like: \\PC\folder\file.csv
But I couldn't be able to succeed, does this component support the bulk from a UNC path ?

Thanks in advance

Best way to handle many parallel transformations in a job

February 21, 2018, 1:16 am

≫ Next: Unable to Update the document using mongo Output step

≪ Previous: BulkLoad into MSSQL, import data from \\PC\folder\file.csv

$

0

0

I am building a Data Vault load with PDI. Since Hubs first, and then Links and Satellites can be loaded in parallel or at least in random order, I have jobs that contain a START and SUCCESS with a lot of transformations between them next to each other and not serial. The Data Vault keeps expanding, so it's becoming unreadable. What is the way to handle this efficiently? I love to make it compact.

Unable to Update the document using mongo Output step

February 21, 2018, 2:09 am

≫ Next: read files from configured path

≪ Previous: Best way to handle many parallel transformations in a job

$

0

0

Hi
I need to update the element value in array but i am unable to do ..

I am using mongo Output step that will inserting the document in my collection.Also i am maintaining my log which file is processing .Lets suppose my job will be processed test.csv file then i am inserting id,filename and add a constant status 'Kettle is running' in my log collection.Now i want to update the status filed once my kettle job is completed like 'Kettle is completed' based on id.Also i have mentioned below my mongo structure and kettle structure

MongoDB



Kettle:


Anyone suggest me where i am doing wrong

Attached Images

  

read files from configured path

February 21, 2018, 4:17 am

≫ Next: Pentaho user console scheduler multiple lines for one job

≪ Previous: Unable to Update the document using mongo Output step

$

0

0

Hello,

I have some directories, could be one or many.
each directory path is configure in one file like

Country|Path
xyz|x/y/z/job
abc|a/b/job
def|d/job

I want to read this file once, get the path and read all files which are like *.csv
If I check accept file name from previous step it will not allow the wildcard, if it allow could you tell me how ?
could you let me know how I can achieve it in the same transformation ?

Pentaho user console scheduler multiple lines for one job

February 21, 2018, 5:21 am

≫ Next: Kettle as Data Processor

≪ Previous: read files from configured path

$

0

0

Hi!

There is strange behavior of pentaho quartz scheduler.
I made simple test job named "test_job_3_min" that work 3 minutes long, upload it on server through user console and set schedule that launch it every minute.
Inside job I made standard logging every 5 sec to DB table and check if there is a copy of "test_job_3_min" launched.
So there is only one copy of this job working at every moment of a time.
I can see that one in three launches succeeded.


This is all ok and working fine, but in PUC Schedules section I can see strange lines for my job with status "Finished" along with main every minute schedule.
Some time there is one line with empty Last Run and Next Run. Some time there is one more line with Last Run specified.
Schedule for that lines is also strange - no duration specified "Every at <time>".



When I set schedule to launch job every 10 sec amount of these lines become grow and some of them have status Normal.



Can somebody explain what's this can be?
If this is normal, can I disable PUC from showing these processes?

Pentaho Server version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy

Job attached in archive: test_job_3_min.zip

UPD: left job working for the night and got 250+ lines with "test_job_3_min" in my schedule now.
This is bad, because we plan to have several such jobs scheduled to launch every 15-30 minutes.
It's will be bloody mess there... I think it's a bug and for one job must be only one line.
What we can do? Upgrade to Pentaho 8?

UPD2: lines are disappeared after Pentaho Server restart, but this is not a case

Attached Images

   

Attached Files

• test_job_3_min.zip (5.1 KB)

Kettle as Data Processor

February 21, 2018, 7:32 am

≫ Next: get filename that contains substring pentaho

≪ Previous: Pentaho user console scheduler multiple lines for one job

$

0

0

Hi All,
Can i find some details on the below please.

In relation to the upcoming law on personal data (GDPR) Kettle as Data Processor has sort of 3 key responsibilities:
- Make themselves compliant
- Educate Controllers (us, product consumers)
- Provide a frame within the technology provided to enable us to be compliant
I cant find any information on this, can anyone share some light?

Thanks

get filename that contains substring pentaho

February 21, 2018, 12:15 pm

≫ Next: Most Common & Supported No-Cost Linux Platform to run a suite of Pentaho CE Tools

≪ Previous: Kettle as Data Processor

$

0

0

I'm using the "Check if a folder is empty" step. What I need is to check only the files where the filename contains a substring with Regex.
How can I achieve this?
Thanks!

---

Most Common & Supported No-Cost Linux Platform to run a suite of Pentaho CE Tools

February 21, 2018, 1:09 pm

≫ Next: How to present time values as chart?

≪ Previous: get filename that contains substring pentaho

$

0

0

Hello, the title is pretty much the TLDR of my question.

I'm looking to match a complete system that rivals MS Win Server/MS SQL Server w/ SSIS/SSAS/SSRS/PowerBI but with Pentaho, Plug-Ins, PostgreSQL/MongoDB however cost free on software. I asked this questions recently and has been answered in detail.

What is not most common and best supported Linux distro (assuming Desktop and not EE due to some cost) is that used for the Pentaho suite? Has anyone had good luck with OpenSUSE-Leap?

And how scalable is it? If I get all this onto a box, and build out ETL that runs a decent amount of data in, and a fair sized DB, and a number of Dashboards being hosted, can it perform reasonably on the hosted dashboard side for an end user to slice and dice, drill and click about and have interactive changes within seconds, adhering to general industry response times?

Thank you,

Data_Geek

How to present time values as chart?

February 22, 2018, 2:38 pm

≫ Next: Dynamically Removing the column name

≪ Previous: Most Common & Supported No-Cost Linux Platform to run a suite of Pentaho CE Tools

$

0

0

Hi,

I don't know how to explain this issue:

I need to show the "max time" value for each category in a char, doesn't matter if it is a chart line, bar or dots.

I have the "time" values as seconds (int) but I used SEC_TO_TIME function to change it as hh:mm:ss in the select sentence.

However when I did this change the chart doesn't work anymore, it doesn't show me an error processing type, just not been display.

The cda is showing the data.

Does anyone how to present "time" value as a metric?

Thanks

Dynamically Removing the column name

February 23, 2018, 2:49 am

≫ Next: authentication issue trying to read from HDFS with Kerberos

≪ Previous: How to present time values as chart?

$

0

0

How to remove the column name dynamically if all the value are null or empty.Is there anyway to do please suggest me any idea how we can achieved through javascript step or something

authentication issue trying to read from HDFS with Kerberos

February 23, 2018, 10:51 am

≫ Next: Transformation Name: ${variable}

≪ Previous: Dynamically Removing the column name

$

0

0

I have a basic job that uses a Hadoop File Input step to read a file in HDFS and write to log the contents.
This works on a non-Kerberized cluster.

I moved the job to a Kerberized cluster and am getting "SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]" error.

I've moved the proper *-site.xml files under hdp25 as well as modified config.properties and added the info for:
authentication.kerberos.principal=user@company.com
authentication.kerberos.keytabLocation=/path/to/file.keytab
authentication.kerberos.id=hdp-kerberos
authentication.superuser.provider=hdp-kerberos

Why is it still looking for simple authentication and not reading the Kerberos settings?
Is there anything else needed for Pentaho to use Kerberos?

I even removed the shims out of the hdp25 folder, and the error message didn't change.
So the error occurs even before it reads the shims??

Transformation Name: ${variable}

February 25, 2018, 5:42 am

≫ Next: Running total in mdx

≪ Previous: authentication issue trying to read from HDFS with Kerberos

$

0

0

Hi,
I have a transformation that is being called multiple times via the "change number of copies to start".
The transformation runs some Linux commands.
The problem that I'm having is that in my logging, it just shows the generic name of the transformation and I cannot get good metrics for each command's run.
I would like to dynamically change the name of this transformation's name, so that in my logging I can isolate the different commands that are running parallel.
I've already tried to set the name to "Transformation name: ${command} but that just shows "${command}" in my logging table. Obviously not what I want.
Can any of you pros think of a way for me to achieve what I want?
Thank you!
Eric Woodall

Running total in mdx

February 25, 2018, 10:49 pm

≫ Next: Require JS

≪ Previous: Transformation Name: ${variable}

$

0

0

I want to calculate running total of my measures.I used SUM(YTD([Date].[YEAR].CurrentMember), [Measures].[ProfileCreated]) It gives a right result within same year but when a new year starts its starts with 0.I want to calculate previous year data too. How can I do?
Dimension: Date-> default->Year->Month->Day

Attached Images

  

Require JS

February 26, 2018, 2:47 am

≫ Next: Pie chart clickable

≪ Previous: Running total in mdx

$

0

0

Hi,
i have created the pie chart dashboard use to Require JS embedded dashboard parameters in pentaho CE 8.0 it working fine using yearParam parameter.
I use the click functionality on pie chart but not working.I use the two parameters like.yearParam and statusParam.

function clickone(){


require (
['dash!/public/BlabyUsingRequired/bidextractdrillreportstatuswisedashboard.wcdf'+'/generatedContent'+'&statusParam='+c+'&yearParam='+yearParam]
, function(SampleDash){
// define div id where the dashboard should be rendered
(new SampleDash("content1")).render();


});
}

---

Pie chart clickable

February 26, 2018, 4:00 am

≫ Next: Job Executor - get job name from field

≪ Previous: Require JS

$

0

0

Hi,
Pie chart clickable not working in pentaho cde 8.0 using requirejs.

Job Executor - get job name from field

February 26, 2018, 9:23 am

≫ Next: User Defined Java Class : Connect to file repository

≪ Previous: Pie chart clickable

$

0

0

In any version of PDI, is it possible to supply the job name for the Job Executor step from a field. The requirement is to run different jobs in parallel.

User Defined Java Class : Connect to file repository

February 26, 2018, 12:04 pm

≫ Next: Error Moving file after Decrypting

≪ Previous: Job Executor - get job name from field

$

0

0

Scenario : Using UDJC to execute a job which is in a file repository.
Error : Could not execute job specified in a repository since we're not connected to one

What is the piece of java code to connect to a repository.

Error Moving file after Decrypting

February 26, 2018, 1:41 pm

≫ Next: IE Compatibility mode

≪ Previous: User Defined Java Class : Connect to file repository

$

0

0

Hello,
I am running into an error moving the file after the file is decrypted using the Decrypt files with PGP step.
The file will successfully decrypt and also show up in the destination directory but will still sit in the source directory where it was picked up from. The below error message shows up:

2018/02/26 21:35:37 - Decrypt files with PGP - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : There was an error moving file [file:///C:/Pentaho/YMCATriangle/EncryptedFiles/G0487511_20180224_040438SB_000010.cdf.gpg] to [file:///C:/Users/vjp0215/Desktop/test ymca pentaho/G0487511_20180224_040438SB_000010.cdf02/26/2018 21:35:37.gpg] : [
2018/02/26 21:35:37 - Decrypt files with PGP - org.pentaho.di.core.exception.KettleException:
2018/02/26 21:35:37 - Decrypt files with PGP - Error running command: gpg: encrypted with 2048-bit RSA key, ID 5DA3B024, created 2018-01-12
2018/02/26 21:35:37 - Decrypt files with PGP - 2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP - Error running command: gpg: encrypted with 2048-bit RSA key, ID 5DA3B024, created 2018-01-12
2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP -
2018/02/26 21:35:37 - Decrypt files with PGP - ]
2018/02/26 21:35:37 - Decrypt files with PGP - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Success condition was broken. We have 1 error(s)

Can some please help me understand what could be the cause of this issue?

I am using Kleopatra GNUPG software to decrypt the files. GPG Location - C:\Program Files (x86)\GNU\GnuPG\gpg2.exe

Please assist.

Thank You.

IE Compatibility mode

February 26, 2018, 2:37 pm

≫ Next: Return Site Information via PDI

≪ Previous: Error Moving file after Decrypting

$

0

0

I need to force IE not to use the compatibility mode because otherwise dashboard is not rendering.
SEE: https://jira.pentaho.com/browse/BISERVER-12519

<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
As per various articles, these META tags need to be added before any styling is applied.

Question: where Can I modify the HEAD TAGS that Pentaho CDE automatically inserts?